Running Vservers on Debian

Running "Vservers" environment on Debian Etch

Installation
1) Install a vserver enables kernel
# apt-get update
# apt-get install linux-image-vserver-X.X.X // choose actual kernel image for your platform and
reconfigure your GRUB, or edit /boot/grub/menu.lst and set "kernel x.xx.x -vserver" to default
boot kernel.
# sync; sync; reboot
2) Install all needed userland binaries
# apt-get install util-vserver vserver-debiantools
Configuration
1) crate symlink to point to your $VROOTDIR
# mkdir /home/vservers
# ln -s /home/vservers /etc/vservers/.defaults/vdirbase
configure /etc/vserver/newvserver-vars as your need
#mcedit /etc/vserver/newvserver-vars

# Architecture: overide on non-Debian host such as Redhat otherwise dpkg
# will detect whether we are i386/powerpc/sparc/etc
#ARCH=""

# Which debian distribution (Warning. unstable and testing distributions
# change frequently so you can not expect it to work out of the box).
DIST="etch"

# Local or nearest location of a debian mirror (must include the /debian)
MIRROR="http://debian.co.il/debian"

# Default network interface for vservers:
INTERFACE="eth0"

# Package caching
#PKGCACHE=1
Creating Vservers
1) create new vserver
# newvserver -v --hostname vsrv1 --domain "example.com" --ip 192.168.1.11 \\ your domain and IPADDRESS
2) start new vserver
# vserver vsrv1 start
* Starting system log daemon...
...done.
* Starting OpenBSD Secure Shell server...
...done.
3) enter to the new vserver
# vserver vsrv1 enter
vsrv1: # apt-get update && apt-get dist-upgrade
vsrv1: # apt-get install PACKAGES (what you want)
vsrv1: # exit
4) stop the vserver
# vserver vsrv1 stop
* Stopping OpenBSD Secure Shell server... [ ok ]
* Stopping system log daemon... [ ok ]
* Sending all processes the TERM signal... [ ok ]
* Sending all processes the KILL signal... [ ok ]
* Unmounting remote and non-toplevel virtual filesystems... [ ok ]
* Shutting down LVM Volume Groups... [ ok ]
let init start the new vserver
# echo "default" > /etc/vservers/vsrv1/apps/init/mark

For extended conntrol on new vservers use following command
# vserver-info
Versions:
Kernel: 2.6.18-5-vserver-686
VS-API: 0x00020002
util-vserver: 0.30.212; Dec 9 2006, 12:26:51

Features:
CC: gcc, gcc (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)
CXX: g++, g++ (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)
CPPFLAGS: ''
CFLAGS: '-Wall -g -O2 -std=c99 -Wall -pedantic -W -funit-at-a-time'
CXXFLAGS: '-g -O2 -ansi -Wall -pedantic -W -fmessage-length=0 -funit-at-a-time'
build/host: i486-pc-linux-gnu/i486-pc-linux-gnu
Use dietlibc: yes
Build C++ programs: yes
Build C99 programs: yes
Available APIs: compat,v11,fscompat,v13,net,v21,oldproc,olduts
ext2fs Source: e2fsprogs
syscall(2) invocation: alternative
vserver(2) syscall#: 273/glibc

Paths:
prefix: /usr
sysconf-Directory: /etc
cfg-Directory: /etc/vservers
initrd-Directory: $(sysconfdir)/init.d
pkgstate-Directory: /var/run/vservers
vserver-Rootdir: /var/lib/vservers

# vserver-stat \\get statistics about vservers
CTX PROC VSZ RSS userTIME sysTIME UPTIME NAME
0 58 109.1M 17.6M 19d16h01 2d08h04 112d03h38 root server
49160 379 2G 650.6M 4d18h38 11h01m48 48d01h05 devel
49161 121 3.1G 324M 1h35m03 6m03s50 48d01h05 sys
49162 245 3.5G 412M 2d23m32 9h04m23 48d01h05 vsrv1

# vps -ef vsrv1: \\like ps but only for vservers
root 8102 0 MAIN 8100 0 12:40 pts/1 00:00:00 -bash
root 8210 49159 vsrv1 5542 0 12:49 ? 00:00:00 sshd: root@pts/2
root 8212 49159 vsrv1 8210 0 12:49 pts/2 00:00:00 -bash
root 8271 1 ALL_PROC 8102 0 12:57 pts/1 00:00:00 vps -ef
root 8272 1 ALL_PROC 8271 0 12:57 pts/1 00:00:00 ps -ef

Useful vserver binaries
vapt-get: use apt-get in given or all vservers
# vserver vsrv1 enter
vsrv1: # htop
bash: htop: command not found
vsrv1: # exit
# vapt-get vsrv1 -- install htop
Reading package lists... Done
Building dependency tree... Done
The following NEW packages will be installed:
htop
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 40.2kB of archives.
After unpacking 164kB of additional disk space will be used.
Get:1 http://debian.co.il/debian etch htop 0.5.2-1 [40.2kB]
Fetched 40.2kB in 0s (710kB/s)

Preconfiguring packages ...
Selecting previously deselected package htop.
(Reading database ... 9417 files and directories currently installed.)
Unpacking htop (from .../archives/htop_0.5.2-1_i386.deb) ...
Setting up htop (0.5.2-1) ...
# vserver vsrv1 enter
vsrv1: # htop

Very important !!
If after login via ssh to virtual server You have been redirected
to mother server - please login to mother server
and change in /etc/ssh/sshd_config variable "ListenAddress 0.0.0.0"
to your IP .
Regards.

Corporate IM gateway

Corporate Instant Messaging system.
Task: installation of corporate Instant Messaging gateway with possibility of logging of all conversations.
Including adjustment of transport for others IM networks (ICQ, MSN, AIM ) As well as authentication of users in LDAP.
The choice between LCS (Live Communication Server M$) and XMPP/Jabber has been made instantly ;) .
It is necessary to choose only which Jabber server to use.
From the considered variants:
chime (Java) http://www.codecobra.com/chime/
DJabberd (Perl) http://danga.com/djabberd/
ejabberd (Erlang) Home page and Community Site
jabberd14 (C) http://jabberd.org/
jabberd2 (C) http://jabberd2.xiaoka.com/
Openfire (Wildfire Server) (Java) http://www.igniterealtime.org/projects/openfire/
OpenIM (Java) http://www.open-im.net/
pretzel (Python) http://code.google.com/p/pretzel/
psyced (LPC) http://www.psyced.org/
Tigase (Java) http://www.tigase.org/
WPJabber (C) http://spik.wp.pl/jabber.html
xmppd.py (Python) http://xmpppy.sourceforge.net/

For comparison the following document has been taken: http://www.jabber.org/admin/jsc/

From the advantages listed above - ejabberd option has been choosen mostly for the following reasons :
1.Protocol-standards compatibility (XMPP Core, XMPP IM),
2.fully distributable
3.database can be replicated to many nodes .
4.The default database, Mnesia, is suitable for small, as well as big deployments.
5.code can be updated while ejabberd is running (feature of Erlang)
6.loading and unloading of modules while ejabberd is running is possible (feature of Erlang)
7.Modular design
Installation
I have install ejabberd on FreeBSD 6.2 , I think that there won`t be large differences in Linux operating system.
Installation performed by ports using “portinstall”.
Before starting installation - it is necessary to download following software:
diablo-jdk
tzupdater (JDK US DST Timezone Update Tool)
Let's execute following actions:
cd /usr/ports/distfiles/
lynx 'http://www.FreeBSDFoundation.org/cgi-bin/download?download=diablo-caffe-freebsd6-i386-1.5.0_07-b01.tar.bz2'
(Read and follow the license agreement)
Download via your web browser http://java.sun.com/javase/downloads/index.jsp
and copy tzupdater-x_x_x_x_x.zip to /usr/ports/distfiles/
installing ejabberd:
cd /usr/ports/
portinstall -p ejabberd-1.1.4
Don't enable support ODBC .
If all these components successfully compiled and installed - the following line should appear in /etc/rc.conf

ejabberd_enable="YES"

DNS Records

eJabberd server and clients, are able to use DNS SRV records for hostname resolution. DNS SRV records allow for delegation of services — by port — to other hosts.
I described setting for BIND , if You use other DNS server - please check following documentation: http://jabberd.jabberstudio.org/2/docs/section05.html#5_7
There are 3 SRV records that could be created for a eJabberd server installation:
_jabber._tcp.your_domain.com. 86400 IN SRV 5 0 5269 host.your_domain.com.
_xmpp-server._tcp.your_domain.com. 86400 IN SRV 5 0 5269 host.your_domain.com.
_xmpp-client._tcp.your_domain.com. 86400 IN SRV 5 0 5222 host.your_domain.com.

Replace “you_domain” with Your Domain Name and host with hostname, and don't forget to put “.” after the domain name.

Let's configure eJabber.
cp /usr/local/etc/ejabberd/ejabberd.cfg.example /usr/local/etc/ejabberd/ejabberd.cfg
cp /usr/local/etc/ejabberd/ejabberd.defaults.example /usr/local/etc/ejabberd/ejabberd.defaults
and edit fail:
vi /usr/local/etc/ejabberd/ejabberd.cfg
{acl, admin, {user, "alex"}}.
Let's add here our user who will have the Administrator permission.
Change “localhost” to Your Server name.
% Host name:
{hosts, ["host.your_domain.com"]}.
Configure support SSL:
vi /usr/local/etc/ejabberd/ejabberd.cfg
% Listened ports:
{listen,
[{5222, ejabberd_c2s, [{access, c2s},
{max_stanza_size, 65536},
starttls, {certfile, "/usr/local/etc/ejabberd/server.pem"},
{shaper, c2s_shaper}]},
{5223, ejabberd_c2s, [{access, c2s},
{max_stanza_size, 65536},
tls, {certfile, "/usr/local/etc/ejabberd/server.pem"},
{shaper, c2s_shaper}]},
% Use STARTTLS+Dialback for S2S connections
{s2s_use_starttls, true}.
{s2s_certfile, "/usr/local/etc/ejabberd/server.pem"}. Full path to file sertificate !!!
Now We will create the certificate :
cd /usr/local/etc/ejabberd/
openssl req -new -x509 -nodes -newkey rsa:1024 -days 3650 -keyout privkey.pem -out server.pem -subj
"/C=XX/ST=XX/L=XX/O=XX/OU=XX/CN=host.your_domain.com/emailAddress="postmaster@your_domain.com
cat privkey.pem >> server.pem
rm privkey.pem

...and start our server by: /usr/local/etc/rc.d/ejabberd start
(Try to telnet localhost 5222 or 5223 for SSL in order to check this.)

Now you have working and reliable jabber server.

LDAP

What about authentication abilities ? We need to authenticate users against LDAP server and to use LDAP Directory as vCard storage.
Edit the file - ejabberd.cfg

 vi /usr/local/etc/ejabberd/ejabberd.cfg
% Authentication method.  
%{auth_method, internal}.         <---!!! Comment this line !!!!

% For LDAP authentication use these lines instead of above one:
{auth_method, ldap}.
{ldap_servers, ["ldap.your_domain.com"]}.    % List of LDAP servers
{ldap_uidattr, "uid"}.            % LDAP attribute that holds user ID
{ldap_base, "ou=People,dc=your_domain,dc=com"}. % Search base of LDAP directory
{ldap_rootdn, "cn=jabber,ou=DSA,dc=your_domain,dc=com"}. % LDAP manager
{ldap_password, "password"}. % Password to LDAP manager

Now We want to use user`s LDAP-info as their vCard. In order to implement this -  find and edit the following
line in section 

% Used modules:

{mod_vcard_ldap, [{host, "ldapyour_domain.com"}]},

and then just restart eJabber /usr/local/etc/rc.d/ejabberd restart

Logging Messages

I usеd for this purpose "Bandersnatch" software. http://funkypenguin.co.za/
Originally , this soft works perfectly , but have a little issue with different encoding.
I need to log different languages such a France, Russian, Hebrew, Spain in my company.
So, I made a little correction in the Perl script and Data Base structure for "Bandersnatch".
If someone will be interested - please ask me by e-mail, I will send you by the reply
attached files.

...unpack "Bandersnatch" archive to special directory and make executable bandersnatch Perl script

chmod +x bandersnatch
Cofigure ejabberd:

Edit ejabberd.cfg and add this line to the 'modules' section:

{mod_service_log, [{loggers, ["bandlog.your_domain.com"]}]},

Add the following lines to 'listen' section to give eJabberd ability to listen for Bandersnatch connect.
% listen for Bandersnatch connections  
{5526, ejabberd_service, [{ip, {127, 0, 0, 1}}, {access, all},
                {hosts, ["bandlog.your_domain.com"],
                [{password, "password"}]}]}

Create DB for Log System 

mysql -u root -pPassword bandlog < bandersnatch.sql (I recommend use my file bandersnatch.sql)

Configure Bandersnatch:
edit config.xml
<server>
<connectiontype>tcpip</connectiontype>
<hostname>localhost</hostname>
<port>5526</port>
<secret>password</secret>
</server>
<component>
<name>bandersnatch@bandlog.your_domain.com</name>
</component>
<mysql>
<server>localhost</server>
<dbname>bandlog</dbname>
<username>bandlog</username>
<password>bandlog</password>
</mysql>

Do not forget to add to DNS a new A record ( like this example ):

bandlog A 192.168.1.1 ( jabber server's IP)

Now restart Your eJabberd server:
/usr/local/etc/rc.d/ejabberd restart
And run in "screen" bandersnatch:
./path/to/bandersnatch/bandrsnatch config.xml
if You see output like this:
Bandersnatch: Connected to Jabber server (localhost) ...
Bandersnatch: Connected to MySQL database (bandlog@localhost) ...
You have successfully installed the system !


Transports
I srongly recomend read befor:
http://wiki.blathersource.org/wiki/index.php/PyICQt
http://wiki.blathersource.org/wiki/index.php/PyAIMt
http://delx.cjb.net/pymsnt/docs/user.html
but if you have reached this point without a problem, with "transport" installation You should not have any problems.
I describe it together (ICQ, MSN, AIM) . At the moment of a writing of this article in OS FreeBSD port jabber-yahoo-2.3.2_2 has been marked IGNORE (broken)
please check for status of Yahoo transport on site http://yahoo-transport-2.jabberstudio.org/

So, let's install all needed software:
portinstall -p jabber-pyicq-transport-0.8a
portinstall -p jabber-pymsn-transport-0.11.2_2,1
portinstall -p jabber-pyaim-transport-0.8a

Add following "A" records to DNS
icq.your_domain.com
msn.your_domain.com
aim.your_domain.com

Edit following rc scripts in /usr/local/etc/rc.d:
jabber-pyaim-transport
: ${jabber_pyaim_enable="YES"}
: ${jabber_pyaim_dir="/usr/local/lib/jabber/pyaim"}
: ${jabber_pyaim_piddir="/var/spool/ejabberd/pid"}
: ${jabber_pyaim_user="ejabberd"}

jabber-pyicq-transport
: ${jabber_pyicq_enable="YES"}
: ${jabber_pyicq_dir="/usr/local/lib/jabber/pyicq"}
: ${jabber_pyicq_piddir="/var/spool/ejabberd/pid"}
: ${jabber_pyicq_user="ejabberd"}

jabber-pymsn-transport
: ${jabber_pymsn_enable="YES"}
: ${jabber_pymsn_dir="/usr/local/lib/jabber/pymsn"}
: ${jabber_pymsn_piddir="/var/spool/ejabberd/pid"}
: ${jabber_pymsn_user="ejabberd"}

Do not forgot make symlink and change owner:
chown -R /usr/local/lib/jabber/
chown -R /var/spool/ejabberd/
ln -s /var/spool/ejabberd /var/spool/jabber

Now let's edit config of eJabberd server:
vi /usr/local/etc/ejabberd/ejabberd.cfg

in section
% Listened ports:
add
% listen for PyICQt connections
{5347, ejabberd_service, [{access, all},{host, "icq.your_domain.com",
[{password, "preved"}]}]},

% listen for PyMSNt connections
{5348, ejabberd_service, [{host, "msn.your_domain.com",
[{password, "password"}]}]},

% listen for PyAIMt connections
{5349, ejabberd_service, [{host, "aim.your_domain.com",
[{password, "password"}]}]}

Edit following config of transports:

jabber-pyicq.xml
<jid>icq.your_domain.com</jid>
<spooldir>/var/spool/ejabberd</spooldir>
<pid>/var/spool/ejabberd/pid/PyICQt.pid</pid>
<mainServer>127.0.0.1</mainServer>
<port>5347</port>
<secret>preved</secret>
jabber-pymsn.xml
<jid>msn.your_domain.com</jid>
<spooldir>/var/spool/ejabberd</spooldir>
<pid>/var/spool/ejabberd/pid/PyMSNt.pid</pid>
<mainServer>127.0.0.1</mainServer>
<port>5348</port>
<secret>password</secret>
<getAllAvatars/>
jabber-pyaim.xml
<jid>aim.your_domain.com</jid>
<spooldir>/var/spool/ejabberd</spooldir>
<pid>/var/spool/ejabberd/pid/PyAIMt.pid</pid>
<mainServer>127.0.0.1</mainServer>
<port>5349</port>
<secret>password</secret>


Last stap - add to /etc/rc.conf following lines:
jabber_pyicq_enable="YES"
jabber_pymsn_enable="YES"
jabber_pyaim_enable="YES"

Now restart Your eJabberd server:
/usr/local/etc/rc.d/ejabberd restart

and run all transports:
/usr/local/etc/rc.d/jabber-pyaim-transport start
/usr/local/etc/rc.d/jabber-pyicq-transport start
/usr/local/etc/rc.d/jabber-pymsn-transport start



P.S.
about a Jabber Client software You can find more here http://en.wikipedia.org/wiki/List_of_Jabber_client_software
I recommend "PSI" and "Gajem".

PostgreSQL checkpoints

PostgreSQL checkpoints.

In documentation ( http://www.postgresql.org/docs/ ) PostgreSQL checkpoints are described as follows:
checkpoint_segments
Maximum distance between automatic WAL checkpoints, in log file segments (each segment is normally 16 megabytes). The default is three segments. This parameter can only be set in the postgresql.conf file or on the server command line.
checkpoint_timeout
Maximum time between automatic WAL checkpoints, in seconds. The default is five minutes (5min). This parameter can only be set in the postgresql.conf file or on the server command line.
checkpoint_warning
Write a message to the server log if checkpoints caused by the filling of checkpoint segment files happen closer together than this many seconds (which suggests that checkpoint_segments ought to be raised). The default is 30 seconds (30s). Zero disables the warning. This parameter can only be set in the postgresql.conf file or on the server command line.
I shall describe it is more expanded:

When the transaction log no longer has available space, the installation of a so-called control checkpoint takes place. This serves as an order for the system to remove all the unprocessed material to a disk, so the log can become available for use once again. Additionally, the control checkpoint may be installed not upon occasion, but after a certain period of time, typically standing at 5 min. When there is massive logging to the database, the transaction log can become overloaded too rapidly. This, in turn, will lead to a substantial deceleration of the aforementioned material removal to a disk.

Establishing the installation frequency of control checkpoints:

The installation of control checkpoints must take place every couple of minutes. If the installation takes place too frequently (for instance, every minute), the productivity of the system would notably decrease. To establish the current frequency, at which the system installs control checkpoints, you can use the log's analysis of timestamps. First, however, make sure that you've allowed access to the log. Check follow options in the file postgresql.conf:

log_timestamp = true

After that changes the file of a configuration will be automatically re-read, and you can observe in a logfile of server PostgreSQL following records:
2007-02-11 21:17:32 LOG: recycled transaction log file 0000000000000000
2007-02-11 21:17:33 LOG: recycled transaction log file 0000000000000001
2007-02-11 21:17:33 LOG: recycled transaction log file 0000000000000002
Estimate the period between two control points. It will allow you to define their frequency.
In the example resulted above you can see, that control points were made each 40 seconds that is too frequent operation ,
and that slows down productivity of system as a whole. By the way, be not surprised, having found out records with the same
time mark (see above). Frequently the same control point enters the name in the log several times.

Reduction of frequency of installation of Control points

Reduction of frequency of Control points leads to increase in number of files of anticipatory caching, created in data/pg_xlog. Each file has the size of 16 mbyte, that as a result can affect considerably the general free space on a disk. Installation by default means minimization of number of such files. For reduction of frequency of control points you should change following parameter:

checkpoint_segments = 3
Its initial value is equal 3.
Gradually increase this value until the interval between control
points not begins to be equaled to several minutes.
The next record in the logfile , which you can observe, can be like this:

LOG: XLogWrite: new log file created - consider increasing WAL_FILES

It means, that the parameter wal_files requiers increase in a file
postgresql.conf.