Using GIT to Manage Config Files

One thing I like about GIT, among others, is that it creates a repository locally. This makes simple things easy, still not disallowing complex things.

A good use of GIT is to manage the configuration files in a GNU/Linux system. This article describes the way I do it.

In this example, I will use the Squid proxy/cache server configuration as an example, but the principles apply to any configuration.

Ok, so here are the steps.

Zeroth step is to install GIT. ;-) On Debian/Ubuntu systems, install the "git-core" package.

First create a git repository in /etc/. On Ubuntu systems, you may have to use "sudo", or become root with "sudo -s" before running the following commands.

# cd /etc
# git init

This will create a GIT repository in /etc/.git/. This step needs to be done only once. Any recent version of GIT should accept the "init" command, otherwise run "git init-db".

Now add and commit a file, so GIT will create the "master" branch. We add the "hostname" file in this example.

# git add hostname
# git commit -m "Initial commit." hostname

If you don't use the "-m" option, GIT will start an editor to enter a comment.

I prefer to use three branches.

• master: This is the branch that holds the working configuration of the running system.
• stock: This branch contains the "default" configuration files as installed.
• play: This branch is used for experimenting.

Now that the "master" branch is already created, let's create the other two.

# git branch stock
# git branch play

You can always use "git branch" command to see available branches with active branch highlighted.

In this example, we install Squid, and manage its configuration file /etc/squid/squid.conf using GIT.

Let us switch to the "stock" branch, install Squid, add the config file, commit it, and switch back to the "master" branch. Switching to a branch is called "checking out".

# git checkout stock
# apt-get install squid
# git add squid/squid.conf
# git commit -m "Adding stock config file." squid/squid.conf
# git checkout master

It is always a good idea to switch back quickly to the "master" branch, as we need to keep the "stock" branch clean without any of our changes going there accidentally.

Now you will notice that the "squid/squid.conf" file has gone missing, because we added it in the "stock" branch, not in "master".

To get the config file to the "master" branch, we need to merge it with the "stock" branch.

# git merge stock

I do all my experimenting, however small, in the "play" branch. Before starting an experiment, I switch to the "play" branch and merge it with "master", so both "master" and "play" will be identical at the beginning.

# git checkout play
# git merge master

Now the experimenting begins. I would change the config file, start/restart Squid, and 1001 other things.

During experimenting, it is very useful to view the changes I have done by using the "git diff" command.

Even if the experiments are not successful, I would still commit any partial work to continue later. There is nothing to worry as we are in the "play" branch. Use the "git branch" command to verify this.

# git commit -m "Partially finished experiment." squid/squid.conf

To get the system back to its previous state, we can always switch to the "master" branch.

# git checkout master

Make sure to commit the changes in the "play" branch before checking out the "master" branch. Otherwise, those changes will be lost.

Once the experimenting is successful in the "play" branch, we can merge them to the master branch.

# git checkout master
# git merge play

When backing up the system, make sure you include /etc/.git/ to save the history of your configurations.

Use "git log" to view the history of commits.

Happy gitting!

Read more...

A Rare Video of SWRD

In this recent blog post, Sam reminds how S W R D Bandaranaike changed national language in 1956, something most people in the present generation [wrongly] call a "Sinhala-only" policy.

English was the Language in Sri Lanka until Bandaranayaka changed it in to Sinhala “with reasonable use of Tamil” (the part everyone forget to mention when they talk about this), and later on, Tamil also considered as national language.

This rare video on youtube is an interview with SWRD, where he states the "reasonable use of Tamil" part. He also admires Tamil as a "rich language with literature and so on".

Read more...

Congratulations Shilpa Sayura

Sri Lanka's Shilpa Sayura project has won the Stockholm Challenge 2008.

The Jury had studies 400 projects from around the globe and selected 145 finalists in six categories. Shilpa Sayura won the first place in the Education category.

Congratulations Niranjan and the team for this great achievement!

I once had the opportunity to see a Shilpa Sayura event at Kandiyapitawewa village. Pictures from that event are here.

Read more...

Mr Donald, Please Correct the Alphabet First!

I have already replied to Mr Donald Gaminitillake's mudslinging campaign against Sinhala Unicode, which he wields through akuru.org web site and by hijacking discussions on various blogs and forums.

Mr Donald's motives are quite clear. He claims that every Sinhala character shape needs an individual "code point", and has applied for a patent for this "invention". With Sinhala Unicode becoming mainstream, avenues for making money with his pending patent are going thin.

So he is doing what any desperate human being (or animal for that matter) would do; try everything to remove the "opponent".

One of the examples Mr Donald always uses is the absence of character "du" in the Sinhala Unicode codepage.

Of course he conveniently forgets to mention that "da" and "papilla" are in fact available. Well, it requires a bit of brains to put them together. ;-)

Mr Donald, there are lots of missing characters in the Sinhala Hodiya (alphabet), including your infamous "du", let alone "yansaya" and "rakaransaya". If you love the Sinhala language so much as you claim, please start a campaign to "fix" Hodiya!

I have previously pointed out this similarity between Hodiya and Sinhala Unicode, and why "du" + "papilla" is as good as "du". This blog post discusses technicalities in detail including the matter of "yansaya" and "rakaransaya".

Unfortunately for Mr Donald, his "opponent", namely Sinhala Unicode, is growing stronger day by day. Implementations are maturing, more standards compliant fonts are beginning to appear, and as I wrote earlier, more web sites and blogs are now Unicode compliant (e.g.: Sinhala Bloggers, Sinhala Wikipedia, Sinhala Blogs and of course our own Sinhala GNU/Linux).

Read more...

External Projectors and GNU/Linux

Some GNU/Linux computers seem to have problems connecting to projectors. While my earlier ThinkPad R51 always obeyed Fn+F7 combination to turn on output to an external projector, recently acquired R52 did not.

After some research I found that XRandR has good support for output hotplugging. Although graphical tools are available to do this, I found the following commands useful.

% xrandr --auto
% xrandr --output --auto

The first one usually works, and it has to be issued as the same user running X, and not root.

In rare cases when parts of the screen is cropped due to the projector having a smaller resolution, use the -s option to reset the resolution:

% xrandr -s 1024x768

Running xrandr without options would show what is going on with the screen modes.

Read more...

Why Native Language is Important for Web

After reading this blog post about blogging Sinhala, I felt like writing my thoughts about the topic.

There are very few "yes" or "no" answers in life, so I don't think it is correct to rule "blogging in Sinhala is a good idea" or vice versa. Most answers can begin with an "it depends", and I think it is true here, too.

In certain circumstances, using in English on the Web is a good idea. When addressing a global audience, or selling to a product on the Web to the global market, not using English will definitely not serve the purpose.

A key argument for using Sinhala is about addressing certain audiences who are not fluent in other languages.

I think there is a more important reason. Certain things can only be done in Sinhala, and this argument holds for any other language.

A blog post is not always a piece of information to be transmitted to a maximum audience. Sometimes it is a work of art. Works of art are diverse, and this diversity is not only limited to language.

Sinhala is not only a communication medium. It also has a very rich literature: poetry, writings and what not. Being a living language, new Sinhala literature is made every day. And if Web is the medium for such literature, obviously, Sinhala has to be the language.

Check out this blog post for example. (You may need to enable Unicode support). It is a collection of Sinhala poetry from an online "hitiwana kavi maduwa", where people used poetry to communicate. I am sure there are lots of readers who appreciate such work. I can hardly imagine how such a blog post can be in English.

So I think the answer to most questions of life applies here as well: it depends. ;-)

Read more...

LaTeX and Sinhala Unicode

When we met at Excel World on last 17th, Bud, Srimal and myself started talking about using Sinhala Unicode in TeX / LaTeX.

It didn't occur to me that Chamath, who also created one of the first Sinhala FOSS keyboard drivers, has already created a preprocessor for LaTeX called sintex which reads Sinhala files in Unicode/UTF-8. In fact, not only had I replied to his announcement, but also sent a patch to Debianize it! Life is too complex, and I am too human to keep track of all these.

But that forgetfulness turned out to be a lucky incident, as our pursuit lead to something more useful!

So we started creating a preprocessor for Vasantha Saparamadu's Sinhala TeX package which uses Samanala transliteration scheme.

However, Bud pointed out that the generated PDF files will have ASCII characters instead of Unicode, making it a problem for search engines that index them, and convert them for "HTML view" pages.

After some research, we found XeTeX, a Unicode enabled version of LaTeX.

XeTeX uses ICU for text layout, and ICU versions after 3.6 supports Sinhala out of the box. However, latest stable version 0.996 of XeTeX uses statically linked ICU 3.4. I managed to patch the "tetex-xetex" package that comes with Debian and make it recognize Sinhala. The patches were also submitted to Debian.

XeTeX font changes are always manual, which made the source look ugly. After a bit of research, I found zhspacing package, which among other things automatically sets fonts for Chinese characters. But it is a complicated package, but I managed to get an idea of how it uses character class feature in the latest XeTeX version 0.997.

Downloading the latest version of XeTeX from SVN repository and building for Debian was not difficult, except I had to edit debian/control files to replace tetex-base and tetex-bin dependency to their texlive counterparts. I had to first get xdvipdfmx. Here is a rough sketch of the work.

% mkdir xdvipdfmx
% cd xdvipdfmx
% svn co http://scripts.sil.org/svn-view/xdvipdfmx/TRUNK
% cd TRUNK
% chmod +x debian/rules
# dpkg-buildpackage -b
# cd ..
# dpkg --purge dvipdfmx
# dpkg -i xdvipdfmx...deb
% cd ..

% mkdir xetex
% cd xetex
% svn co http://scripts.sil.org/svn-view/xetex/TRUNK
% cd TRUNK
% vi debian/control
% chmod +x debian/rules
# dpkg-buildpackage -b
# cd ..
# dpkg --purge texlive-xetex
# dpkg -i xetex...deb

As the XeTeX web site had warned, the Debian build files provided by vanilla XeTeX were not up to date. After installing I had to create a /etc/texmf/fmt.d/10local.cnf with the following two lines:

xetex   xetex  -             *xetex.ini
xelatex xetex  language.dat  *xelatex.ini

and then run the following commands:

# update-fmutil
# fmutil-sys --enablefmt xetex
# fmutil-sys --enablefmt xelatex

to make "xelatex" command to work properly.

After getting latest version of XeTeX working, the last remaining step was to create a small style file, which I called "sinhala.sty", to make automatic font switching for Sinhala.

% sinhala.sty version 20080420
% Typesetting mixed Sinhala documents in XeTeX
%
% Copyright (C) 2008 by Anuradha Ratnaweera
%
\ifx\XeTeXrevision\@undefined
  \errmessage{XeTeX is required to use sinhala}
\fi
\ifx\XeTeXinterchartokenstate\@undefined
  \errmessage{XeTeX 0.997 or above required to use sinhala}
\fi
\ProvidesPackage{sinhala}[2008/04/20]
\RequirePackage{fontspec}
\newfontinstance{\sifont}[Script=Sinhala]{LKLUG}
\newcommand\latinfont{\fontfamily{lmr}\selectfont}
\XeTeXinterchartokenstate = 1
\newcount\cnt\cnt="0D80
\loop
  \XeTeXcharclass\cnt=10 \ifnum\cnt<"0DFF \advance\cnt1
\repeat
\XeTeXcharclass "200C = 10
\XeTeXcharclass "200D = 10
\XeTeXinterchartoks 0 10 = {\sifont}
\XeTeXinterchartoks 255 10 = {\sifont}
\XeTeXinterchartoks 10 0 = {\latinfont}
\XeTeXinterchartoks 10 255 = {\latinfont}

So, all you need is XeTeX 0.997 and sinhala.sty to write LaTeX files using Sinhala Unicode.

Read more...

Goodbye xorg.conf!

After reading this article on xrandr, I wanted to see how total autoconfiguration works on X Windows.

As a start, I tried removing xorg.conf file completely and restart X. The sky didn't fall down! In fact, I didn't notice any change. Everything from USB hotplug to OpenGL continued to work as before.

The only tweak needed was to the old font system. Unlike fontconfig, the old X font system seems to depend on the "font path" set in xorg.conf. This was a problem for using my custom SUN22x12 font in xterm. After adding the following lines to ~/.fvwm/preferences/Startup, this problem was gone, too.

AddToFunc InitFunction
+ I Exec exec /usr/bin/xset +fp /usr/local/share/fonts

Yes, I use fvwm-crystal, a "polished" version of FVWM. Old school, so what?

Read more...

Simplifying Digital Camera Access on GNU/Linux

Digital camera access is simple enough on GNU/Linux, but with a couple of tweaks here and there, it can be made even simpler.

Summary: I keep my photos in directories named "yyyy-mm-dd" by the date taken. When I plug in the camera, photos are automatically downloaded and sent to correct directories. If you like to know how I did it, please read on!

Accessing digital cameras has always been simple on GNU/Linux. A large number of digital cameras are supported out of the box. When using the shell, arguably gphoto is the most convenient. Running gphoto with the "P" option autodetects the camera and downloads all the photos in it.

% gphoto2 -P

I keep all my photos in a "photos" directory with subdirectories in the "yyyy-dd-mm" format indicating the date taken.

First step of simplification is to automatically put each image into the correct location. Digital cameras put a lot of Exif information into each image, so extracting the date taken is quite straightforward. I use a simple tool called exif to do this.

Both gphoto2 and exif are available on Debian.

# apt-get install gphoto2 exif

After some trial and error, I figured that the images taken with my Canon PowerShot S3 IS have a tag 0x132 indicating the date each photo was taken.

% exif -t 0x132 IMG_0416.JPG 
EXIF entry 'Date and Time' (0x132, 'Date and Time')...
Tag: 0x132 ('DateTime')
  Format: 2 ('Ascii')
  Components: 20
  Size: 20
  Value: 2007:10:22 06:05:49

What we want is in the "Value:" line. After filtering that line with grep, and using sed a couple of times, we can get the date in yyyy-dd-mm format.

% exif -t 0x132 IMG_0416.JPG | \
    grep 'Value: ' | \                           # Filter the line with "Value:"
    sed 's/.*Value: \(....:..:..\) .*/\1/' | \   # Get the yyyy:mm:dd part of the value line
    sed 's/:/-/g'                                # convert ":" to "-"
2007-10-22

If you want to understand exactly what each step is doing, try the above pipeline by adding one filter at a time.

Then I put together a small script to move each image in the current directory to ~/pictures/yyyy-mm-dd/ subdirectories where I want them.

gphoto2 -P

for i in *.JPG
do
    date=$(exif -t 0x132 $i | \
        grep 'Value: ' | \
        sed 's/.*Value: \(....:..:..\) .*/\1/' | \
        sed 's/:/-/g')
    dir="/home/anuradha/pictures/test/$date"
    mkdir -p "$dir"
    mv -f "$i" "$dir"
done

Notice that I use a test directory. I saved this in ~/bin/, and made it executable.

Now comes the fun part. After connecting the camera to the computer, I used "lsusb" to find out its vendor ID and product ID are 04a9:311a. The following udev rule in /etc/udev/rules.d/010_local.rules invokes the above script whenever this camera is plugged in.

ACTION=="add", BUS=="usb", \
    SYSFS{idVendor}=="04a9", SYSFS{idProduct}=="311a", \
    RUN+="/home/anuradha/bin/pictures.sh"

Well, matters are a little more complicated. Udev seems to invoke the script multiple times. So I added two extra "features" to stop that.

• Adding a lock file to prevent multiple simultaneous running of the script.
• Use a "timestamp" file at the end of the script, and not run again "too soon" (60 seconds turned out to be ok).

These made sure that the script is run only once when the camera is plugged in.

I used the number of seconds since the Unix Epoch given by the stat and date commands. If the timestamp file was created less than 60 seconds ago, the script aborts.

So here is the complete script:

#!/bin/bash

set -e

user=anuradha
group=users
pictures="/home/$user/Pictures"
download="$pictures/.download"
logdir="$pictures/log"
log=$(date +"$logdir/%Y-%m-%d");
cooldown=60

lock=/tmp/.pictures.download
lasttime=$lock.time

# Avoid multiple simultaneous runs
ln -s $lock $lock || exit 0

# Abort if we had run less than $cooldown seconds ago
if [ -f "$lasttime" ]
then
    t1=$(stat -c '%Z' $lasttime)
    t2=$(date +'%s');
    dt=$((t2 - t1))
    if [ $dt -lt $cooldown ]
    then
        rm -f $lock
        exit 0
    fi
fi

# Take it slowly ;-)
sleep 3

mkdir -p $download
mkdir -p $logdir
rm -f $download/*

# Get the photos, all of them
cd $download
gphoto2 -P

for i in *.JPG
do
    date=$(exif -t 0x132 $i | \
        grep 'Value: ' | \
        sed 's/.*Value: \(....:..:..\) .*/\1/' | \
        sed 's/:/-/g')
    dir="$pictures/$date"
    if [ ! -f "$dir/$i" ]
    then
        [ -d $dir ] || mkdir -p $dir
        chown $user:$group $i
        chmod 644 $i
        chown $user:$group $dir
        mv -f $i $dir
        echo "$date/$i" >> $log
    fi
done

cd
rmdir $download

# Add a timestamp
touch $lasttime

rm -f $lock
exit 0

Read more...

Network Traffic Accounting

After totally automating my Mobitel 3G connection, the next natural step was to setup some kind of a traffic accounting system. I wanted to avoid tools that monitor individual packets, because that was an unnecessary overhead. vnStat turned out to be a perfect match.

Here are the steps in setting up vnstat on Debian. Good news is that vnstat in Debian comes with proper crontab entries and network up/down hooks already in place.

• First step, obviously is to install vnstat:

  # apt-get install vnstat
  

• Create a new configuration:

  # vnstat --showconfig > /etc/vnstat.conf
  

• Edit /etc/vnstat.conf and set the default interface to "ppp0".
• Create an empty database for ppp0:

  # vnstat -u -i ppp0
  

Now vnstat starts counting network traffic. The default crontab seems to run "vnstat -u" every 5 minutes.

Then I installed this simple web based frontend called vnStat PHP frontend. Installation is just a matter of unpacking:

# cd /usr/local/src/
# wget http://www.sqweek.com/sqweek/files/vnstat_php_frontend-1.3.tar.gz
# cd /var/www
# tar -xzvf /usr/local/src/vnstat_php_frontend-1.3.tar.gz
# mv vnstat_php_frontend-1.3 vnstat

Then I had to edit /var/www/vnstat/config.php and set the following values.

$iface_list          = array('ppp0');
$iface_title['ppp0'] = 'Mobitel 3G';
$vnstat_bin          = '/usr/bin/vnstat';

Pointing a browser to http://localhost/vmstat/ showed that everything is working fine.

I also have the following .htaccess file in vnstat directory to avoid access from remote hosts:

Order deny,allow
Deny from all
Allow from 127.0.0.0/255.0.0.0 ::1/128

Read more...