Cluster synchronization with csync2
Cluster synchronization with csync2 |
|
|
|
This lab experience can be directly used in any pc using the live-{CD,USB} Netkit4TIC downloadable from http://tocai.dia.uniroma3.it/.
The experience is based on four nodes:
telnet, ftp, ssh and apache.
They are organized in tree groups: (telnet,ftp), (ssh,apache) and
(telnet, ftp, ssh, apache).
For clarity the nodes are standalone servers but in our daily work
the nodes are part of two (and even more) HA-clusters in an
active-active configuration.
The csync web documentations:
Building csync2 for Debian Sarge from Debian Etch source |
|
Even if csync2 (for Sarge)
is already installed in the live-{CD,USB} we want remind
the steps necessary to build it.
Actually (April 2007) the csync2 Debian package is only present
in Etch release
so we must rebuild the package for the Sarge release.
You will need all of csync2*dsc, csync2*tar.gz and csync2*diff.gz to compile the source:
$ SITE=ftp.debian.org; \
PREFIX=debian/pool/main/c/csync2; \
wget http://$SITE/$PREFIX/csync2_1.30-1.dsc \
http://$SITE/$PREFIX/csync2_1.30.orig.tar.gz \
http://$SITE/$PREFIX/csync2_1.30-1.diff.gz
the following command extract the package into a directory called csync2-1.30:
# dpkg-source -x csync2_1.30-1.dsc
to compile the package you may cd into csync2-1.30 directory and issue the command:
# cd csync2-1.30/ # fakeroot dpkg-buildpackage -b -uc
the generated package stands in parent directory.
The package depends on librsync1 and on libsqlite0 that are
already present in Sarge.
Configure |
|
Even if csync2 is already configurated from the script we want remind
the main steps.
We want simulate two HA-clusters with one group for each cluster plus one
another group for shared operations.
In the first group (g1)
we have telnet and ftp nodes.
In the second (g2)
group we have ssh and apache nodes.
In the last group (g3) we have all nodes.
# /etc/csync2.cfg
#nossl telnet ftp ssh apache;
group g1 {
host telnet ftp; # hosts list
key /etc/csync2.key_g1; # pre-shared key
include /etc/xinetd.d;
action { # action section for apache
pattern /etc/xinetd.d;
exec "/etc/init.d/xinetd restart";
logfile "/var/log/csync2_action.log";
}
backup-directory /var/backups/csync2/g1;
backup-generations 3; # backup old files
}
group g2 {
host ssh apache; # hosts list
key /etc/csync2.key_g2; # pre-shared key
include /etc/apache;
include /etc/ssh;
action { # action section for apache
pattern /etc/apache/httpd.conf;
exec "apachectl graceful";
logfile "/var/log/csync2_action.log";
}
action { # action section bind
pattern /etc/ssh/sshd_config;
exec "/etc/init.d/ssh restart";
logfile "/var/log/csync2_action.log";
}
backup-directory /var/backups/csync2/g2;
backup-generations 3; # backup old files
}
group g3 {
host telnet ftp ssh apache; # hosts list
key /etc/csync2.key_g3; # pre-shared key
include /etc/hosts;
backup-directory /var/backups/csync2/g3;
backup-generations 3; # backup old files
}
The servers talk with each other using TCP port 30865. The servers use xinetd (or inetd) to startup:
# /etc/xinetd.d/csync2
service csync2
{
flags = REUSE
socket_type = stream
wait = no
user = root
group = root
server = /usr/sbin/csync2
server_args = -i
disable = no
# only_from = 192.168.100.1 192.168.100.2 192.168.100.3 192.168.100.4
}
To avoid csync2 startup error we submit a fake csync2.key_g2 file for g1 group and fake csync2.key_g1 file for g2 group.
To complete the csync2 configuration we must generate an SSL certificate for each node. First we setup main openssl configuration file:
# /etc/ssl/openssl.cnf [...] countryName_default = IT stateOrProvinceName_default = Italy localityName_default = Mestre-Venezia 0.organizationName_default = MIUR organizationalUnitName_default = R&D [...]
To generate the pre shared keys (for security one for each group):
# csync2 -k /etc/csync2.key_g1 # csync2 -k /etc/csync2.key_g2 # csync2 -k /etc/csync2.key_g3
and we copy it in relative (group) nodes.
Now we generate the certificate for telnet node:
telnet# openssl genrsa \
-out /etc/csync2_ssl_key.pem 1024 && \
openssl req -new \
-key /etc/csync2_ssl_key.pem \
-out /etc/csync2_ssl_cert.csr && \
openssl x509 -req -days 600 \
-in /etc/csync2_ssl_cert.csr \
-signkey /etc/csync2_ssl_key.pem \
-out /etc/csync2_ssl_cert.pem && \
chmod go= /etc/csync2_ssl_key.pem
last we generate the certificate for apache node:
apache# openssl genrsa \
-out /etc/csync2_ssl_key.pem 1024 && \
openssl req -new \
-key /etc/csync2_ssl_key.pem \
-out /etc/csync2_ssl_cert.csr && \
openssl x509 -req -days 600 \
-in /etc/csync2_ssl_cert.csr \
-signkey /etc/csync2_ssl_key.pem \
-out /etc/csync2_ssl_cert.pem && \
chmod go= /etc/csync2_ssl_key.pem
Now test it |
|
Download the
tarball,
untar it in a HOME subdirectory and
execute the following command:
realHost$ lstart
Now we are ready to test the csync2 capability. The first step is synchronizing the files:
telnet# csync2 -x ftp# csync2 -x ssh# csync2 -x apache# csync2 -x
the first time csync2 create its database and after it try to synchronize. Now we simulate a conflict in g2 group:
ssh# echo "# from ssh #" >> /etc/apache/httpd.conf apache# echo "# from apache #" >> /etc/apache/httpd.conf
csync2 can not know which version is the right one:
apache# csync2 -x While syncing file /etc/apache/httpd.conf: ERROR from peer ssh: File is also marked dirty here! Finished with 1 errors. ssh# csync2 -x While syncing file /etc/apache/httpd.conf: ERROR from peer apache: File is also marked dirty here! Finished with 1 errors.
We decide that ssh node contains the right file
(screenshot):
ssh# csync2 -f /etc/apache/httpd.conf ssh# csync2 -xv Connecting to host apache (SSL) ... Updating /etc/apache/httpd.conf on apache ... Finished with 0 errors. apache# csync2 -xv apache# find /var/backups/csync2/ /var/backups/csync2/ /var/backups/csync2/g1 /var/backups/csync2/g2 /var/backups/csync2/g2/etc_apache_httpd.conf.0 /var/backups/csync2/g3
As side effect it is performed the action to restart apache. Also if we change (from apache node) the ssh configuration files it is performed a ssh restart.
apache# # tail -n 1 /var/log/csync2_action.log
/usr/sbin/apachectl graceful: httpd gracefully restarted
Now we change /etc/hosts in a ftp
and we look at propagation in all g3 nodes:
ftp# echo "#" >> /etc/hosts ftp# csync2 -xv Marking file as dirty: /etc/hosts Connecting to host telnet (SSL) ... Updating /etc/hosts on telnet ... Connecting to host apache (SSL) ... Updating /etc/hosts on apache ... Connecting to host ssh (SSL) ... Updating /etc/hosts on ssh ... Finished with 0 errors. apache# find /var/backups/csync2/ /var/backups/csync2/ /var/backups/csync2/g1 /var/backups/csync2/g2 /var/backups/csync2/g2/etc_apache_httpd.conf.0 /var/backups/csync2/g3 /var/backups/csync2/g3/etc_hosts.0
Last we change some files from ftp node relative to g1 and g2 groups and we observer the results:
ftp# echo "# from ftp #" >> /etc/xinetd.d/telnet; \
echo "# from ftp #" >> /etc/apache/httpd.conf; \
csync2 -xv
Marking file as dirty: /etc/xinetd.d/telnet
Connecting to host telnet (SSL) ...
Updating /etc/xinetd.d/telnet on telnet ...
Finished with 0 errors.
as we expect.
|
|
Sandro Doro (email me)