1 |
=head1 Introduction |
2 |
|
3 |
PXElator is just a peace of puzzle which aims to replace system administration with nice declarative programs in perl. |
4 |
It's a experiment in replacing my work with reusable perl snippets. |
5 |
|
6 |
It tries to solve following problems: |
7 |
|
8 |
=over 2 |
9 |
|
10 |
=item * |
11 |
|
12 |
support deployment of new physical or virtual machines (ip, hostname, common configuration) |
13 |
|
14 |
=item * |
15 |
|
16 |
maintain documentation about changes on systems, good enough to be used for disaster recovery (or deployment of similar system) |
17 |
|
18 |
=item * |
19 |
|
20 |
configure systems in small chunks (virtual or containers) for better management and resource tracking using normal system administration tools (but track those changes) |
21 |
|
22 |
=item * |
23 |
|
24 |
provide overview and monitoring of network segment and services on it with alerting and trending |
25 |
|
26 |
=back |
27 |
|
28 |
=head2 Deployment of new machines |
29 |
|
30 |
What is really machine? For PXElator, it's MAC and IP address and some optional parameters (like hostname). |
31 |
It's stored on file-system, under C<conf/server.ip/machine.ip/hostname> and can be tracked using source control if needed. |
32 |
|
33 |
This is also shared state between all daemons implementing network protocols: |
34 |
|
35 |
=over 2 |
36 |
|
37 |
=item * |
38 |
|
39 |
DHCP (with PXE support) |
40 |
|
41 |
=item * |
42 |
|
43 |
TFTP (to deliver initial kernel and initrd using pxelinux) |
44 |
|
45 |
=item * |
46 |
|
47 |
HTTP (to provide alternative way to fetch files and user interface) |
48 |
|
49 |
=item * |
50 |
|
51 |
DNS (we already have data) |
52 |
|
53 |
=item * |
54 |
|
55 |
syslog |
56 |
|
57 |
=item * |
58 |
|
59 |
AMT for remote management |
60 |
|
61 |
=back |
62 |
|
63 |
Having all that protocols written in same language enables incredible flexibility in automatic configuration. |
64 |
I can issue command using installation which has only ping because I can have special DNS names which issue commands. |
65 |
|
66 |
But, to get real power, we need to aggregate that data. I'm currently using CouchDB from L<http://couchdb.apache.org/> |
67 |
to store all audit data from all services into single database. |
68 |
|
69 |
I wanted simple way to write ad-hoc queries without warring about data structure too much. At the end, |
70 |
I opted for audit role of data, and used 1 second granularity as key when storing data. |
71 |
Result of it is that 133 syslog messages from kernel right after boot you will create single document |
72 |
with 133 revisions instead of flooding your database. |
73 |
|
74 |
It would be logical to plug RRDtool L<http://oss.oetiker.ch/rrdtool/> somewhere here to provide nice graphs here, |
75 |
but that is still on TODO list. |
76 |
|
77 |
End user scenarios: |
78 |
|
79 |
=over 2 |
80 |
|
81 |
=item * |
82 |
|
83 |
Take a new machine, plug it into network, boot it from network and configure for kiosk style deployment with |
84 |
Webconverger available at L<http://webconverger.com/>. |
85 |
Kiosk should automatically turn on every morning at 7:30 and turn off at 20:30. |
86 |
|
87 |
=item * |
88 |
|
89 |
Boot virtual machine (with new ip and hostname) from backup snapshot for easy recovery or testing |
90 |
|
91 |
=item * |
92 |
|
93 |
Boot machine from network into fully configurable (writable) system for quick recovery or dedicated machine. |
94 |
This is implemented using NFS server with aufs read-write overlay on top of debootstrap base machine. |
95 |
|
96 |
=back |
97 |
|
98 |
=head2 Disaster recovery documentation for me, two years later |
99 |
|
100 |
I have been trying to write useful documentation snippets for years. |
101 |
My best effort so far is I<Sysadmin Cookbook> at L<https://sysadmin-cookbook.rot13.org/> a set of semi-structured |
102 |
shell scripts which can be executed directly on machines. |
103 |
|
104 |
This part isn't yet integrated into PXElator, but most of the recipe will become some kind of rule which you |
105 |
can enforce on some managed machine. |
106 |
|
107 |
End user scenario: |
108 |
|
109 |
=over 1 |
110 |
|
111 |
=item * |
112 |
|
113 |
Install that B<something> also on this other machine |
114 |
|
115 |
=back |
116 |
|
117 |
=head2 Configure system like you normally would but track changes |
118 |
|
119 |
This is basically requirement to track configuration changes. Currently, this feature falls out of writable snapshot |
120 |
over base system which is read-only. Overlay data is all custom configuration that I did! |
121 |
|
122 |
Tracking changes on existing machines will be implemented C<scp> to copy file on server into C<hostname/path/to/local/file> |
123 |
directory structure. This structure will be tracked using source control (probably git as opposed to subversion |
124 |
which PXElator source uses) and cron job will pull those files at some interval (daily, hourly) to create |
125 |
C<rsync+git> equivalent of BackupPC L<http://backuppc.sourceforge.net> for this setup. |
126 |
|
127 |
It's interesting to take a look how it's different from I<Puppet> and similar to I<cfengine3>: |
128 |
|
129 |
=over 2 |
130 |
|
131 |
=item * |
132 |
|
133 |
All data is kept in normal configuration files on system -- you don't need to learn new administration tools |
134 |
or somehow maintain two sources of configuration (in configuration management and on the system) |
135 |
|
136 |
=item * |
137 |
|
138 |
Introspect live system and just tries to apply corrections if needed which is similar to C<cfengine3> approach. |
139 |
|
140 |
=back |
141 |
|
142 |
End user scenario: |
143 |
|
144 |
=over 2 |
145 |
|
146 |
=item * |
147 |
|
148 |
Turn useful how-to into workable configuration without much effort |
149 |
|
150 |
=back |
151 |
|
152 |
=head2 Provide overview and monitoring |
153 |
|
154 |
This falls out from HTTP interface and from collecting of data into CouchDB. |
155 |
For now, PXElator tries to manage development environment for you, opening xterms |
156 |
(with screen inside for logging and easy scrollback) in different colors, |
157 |
and enable you to start Wireshark on active network interfaces for debugging. |
158 |
|