SIGQUIT

FSCONS 2011 and GNU get-together

aleksander — Fri, 04 Nov 2011 13:08:30 +0000

I’ll be attending FSCONS 2011 in Göteborg (Sweden) next week (11/11/11), which also holds a GNU get-together, with lots of usual suspects.

Carlos Garnacho, another Lanedian, will be giving a talk about Theming GTK+ applications with CSS, which you shouldn’t miss.

So, if you are interested in developing custom GTK+, LibreOffice, NetworkManager, ModemManager or Tracker features; or any other services provided by Lanedo GmbH, just email me and we’ll be able to talk in Göteborg.

See you there!

Filed under: GNU Planet, Lanedo Planet, Meetings, Planets Tagged: fscons, ghm, gnu

Your custom kernel in MeeGo

aleksander — Fri, 02 Sep 2011 09:20:58 +0000

In my last post I covered the steps needed to create custom MeeGo images and play with them.

One of the things you may also want to do is to try new or modified kernels in your MeeGo images. You can do this either installing RPMs or directly compiling the kernel yourself. The following steps will cover this last case.

1. Where to compile the kernel

No, do not try to compile the kernel directly in a raw image launched within KVM/QEMU.

The easiest way to compile your own kernel is to prepare a loop image and chroot into it (See section “6. Create a loop image and chroot into it” in Your custom MeeGo builds).

Note: In order to completely compile the kernel, you will need quite a lot of empty space in the chroot. In order to easily get this, just use a big enough root partition size (given in MBytes) in the Kickstart file used to create the loop image.

2. Get into the chroot

Assuming meego-core-ia32.img is a loop image created, just:
$> sudo mic-chroot meego-core-ia32.img

3. Ensure only one kernel is available

The MeeGo Image Creation tools will expect to have exactly only one kernel available. Therefore, if you just created the loop image, you will need to remove the default kernel RPM installed.
# rpm -e kernel

4. Install the MeeGo kernel source and build dependencies

You will need to enable the source repositories in zypper first:
First, list the available repositories in zypper:
# zypper lr -u

Note that these repositories were initially configured in the Kickstart file used to create the loop image.

Then, enable the source repositories if not already done.
# zypper mr -e 1.2.0-non-oss-source # zypper mr -e 1.2.0-oss-source # zypper mr -e 1.2.0-updates-non-oss-source # zypper mr -e 1.2.0-updates-oss-source

Once you have these enabled, you can install the kernel source, and all the build dependencies will also get installed:
# zypper si kernel

Install additional required packages which were not listed in the build dependencies (not sure why not listed).
# zypper in gcc

5. Get your new kernel, configure, compile and install

First, download the tarball and decompress it:
# cd /root # wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.0.4.tar.bz2 # tar -jxvf linux-3.0.4.tar.bz2 -C /usr/src

Prepare the kernel configuration file, based on the one from the MeeGo kernel packaging. In this case we’re using a newer kernel version, so you will be prompted for the values of the new configuration parameters. You can just try use the default ones if you don’t know what they mean.
# cd /usr/src/linux-3.0.4 # cat /root/rpmbuild/SOURCES/config-generic /root/rpmbuild/SOURCES/config-x86 > .config # make oldconfig

Note: Of course, you can fully skip using the MeeGo kernel configuration file and fully configure it yourself with, for example, make menuconfig.

Then, compile the new kernel and the kernel modules, and install them.
# make -j8 # make -j8 modules # make modules_install # make install

Note: In addition to the specific kernel configuration, MeeGo packaging also provides several kernel patches. In this case, we are not including them.

6. Make the kernel bootable

In order to make the kernel bootable in the image, you will need to create an initial ramdisk under /boot. But you shouldn’t use mkinitrd for that, use the provided mkliveinitrd instead:
# /usr/libexec/mkliveinitrd -f /boot/initrd.img-3.0.4 3.0.4

Once done, just exit the chroot:
# exit

7. Create a LiveUSB image to test the new kernel

The steps to convert a loop image to a LiveUSB image and write it into an external USB disk are explained in my previous post about Your custom MeeGo builds, so I won’t explain much more about it.

Just a warning: always make sure that you have ONLY ONE kernel available under /boot. If you happen to have more than one, for example when make install-ing twice (old ones get renamed with a .old extension), mic-image-convertor won’t like it and will exit with an error message like this one:
Error: Unable to copy valid kernels or initrds, please check the repo

Filed under: Lanedo Planet, MeeGo, Operating Systems, Planets Tagged: kernel, MeeGo

Your custom MeeGo builds

aleksander — Thu, 01 Sep 2011 16:27:04 +0000

NOTE! For the long and detailed explanation on how to create custom MeeGo images, just jump to the MeeGo wiki: http://wiki.meego.com/Image_Creation

These steps shown here are the minimum set required in an Ubuntu 11.04 host to be able to create MeeGo images easily. They do not try to be extensive, or very explanatory, these are just my personal notes on how to do it

1. Install MeeGo Image Creator

Include the MeeGo tools and SDK repositories to your APT configuration, by adding the following lines to your /etc/apt/sources.list:
deb http://repo.meego.com/MeeGo/tools/repos/ubuntu/10.10/ / deb http://repo.meego.com/MeeGo/sdk/host/repos/ubuntu/10.10/ /

Add the MeeGo repository key:
$> gpg --keyserver subkeys.pgp.net --recv 0BC7BEC479FC1F8A $> gpg --export --armor 0BC7BEC479FC1F8A | sudo apt-key add -

Update the APT cache:
$> sudo apt-get update

Install the mic2 package:
$> sudo apt-get install mic2

Note: ‘mic2′ is also in natty/universe, just not the latest version.

2. Prepare a MIC bootstrap

In order to be able to create images from within your Ubuntu 11.04, you will need to use a MIC bootstrap. If you’re using one of the supported platforms (Ubuntu 10.10 for example), you can possibly skip using it. The generated bootstrap will be a chroot-able location which contains MIC and all its exact dependencies.
$> sudo mic-create-bootstrap \ -n trunk \ -k /opt/meego/repo-cache \ -r http://repo.meego.com/MeeGo/builds/trunk/latest/repos/oss/ia32/packages \ -o /opt/meego/ia32-bootstrap
This previous command will create a trunk named bootstrap in /opt/meego/ia32-bootstrap using /opt/meego/repo-cache as path to store cached packages.

3. Prepare a kickstart file

Kickstart files are the ones defining the contents of the image to be built. Not only what packages will be installed, but also which default users will be created, the default root password, the default locale settings, the default keyboard layout…

Preparing a new kickstart file is best done by taking an officially used one, and modifying it as you like (a.k.a. the autotools way):
http://repo.meego.com/MeeGo/releases/1.2.0/builddata/image-configs/

The kickstart files I’ve been using are based on the upstream official ones, but with updated repo URLs and bigger root partition sizes. You can find them here:

http://www.lanedo.com/~aleksander/meego/meego-core-ia32-1.2.0.ks (Core)
http://www.lanedo.com/~aleksander/meego/meego-netbook-ia32-1.2.0.ks (Netbook)

Note: If you want to know more about the format of kickstart files, what they do and such, just check the Fedora wiki.

4. Create a raw image and launch it under KVM

The MeeGo Image Creator can build ‘raw’ images which can be then launched under QEMU/KVM. The following command will build one of such ‘raw’ images, using the previously created bootstrap and one of the kickstart files we prepared. Note that the command also defines a cache path, where the RPMs downloaded from the MeeGo online repos will be stored (and therefore available for new images created afterwards).
$> sudo mic-image-creator \ --bootstrap=/opt/meego/ia32-bootstrap \ --format=raw \ --config=meego-core-ia32-1.2.0.ks \ --cache=/opt/meego/repo-cache

You can then use mic-vm-launcher as follows to launch the image under QEMU/KVM:
$> sudo mic-vm-launcher meego-core-ia32-raw/meego-core-ia32-sda.raw

Or call kvm directly and tweak the parameters used to launch the image:
$> sudo kvm \ -s \ -m 1024 \ -hda meego-core-ia32-raw/meego-core-ia32-sda.raw \ -vga std

Note: Ensure that virtualization is enabled in your BIOS, or KVM won’t be able to work properly. You can use ‘$> sudo kvm-ok‘ to check whether it is enabled or not.

5. Create a LiveUSB image and write it to a USB dongle

You will probably want to test your MeeGo build in the real hardware, and for that using LiveUSB images is one of the best options. The MeeGo Image Creator also allows building LiveCD and LiveUSB images directly from kickstart files; again, just changing the ‘format‘ argument:
$> sudo mic-image-creator \ --bootstrap=/opt/meego/ia32-bootstrap \ --format=liveusb \ --config=meego-core-ia32-1.2.0.ks \ --cache=/opt/meego/repo-cache

Once you get the image, you can use ‘mic-image-writer‘ to directly copy it to a connected USB disk. This tool will scan all available external disks, and prompt a menu to select the one where you want to write the image to. The ‘-c‘ switch (--console) just tells the tool to avoid launching any GUI, and present the menu directly in the console. During this step, the USB disk is also prepared with sysconfig/isolinux to boot the newly written image.
$> sudo mic-image-writer -c meego-core-ia32.usbimg

Now you’re ready to boot your hardware with the LiveUSB image.

6. Create a loop image and chroot into it

With the MeeGo Image Creator, you can also build loop images, just changing the ‘format‘ parameter in the call to mic-image-creator:
$> sudo mic-image-creator \ --bootstrap=/opt/meego/ia32-bootstrap \ --format=loop \ --config=meego-core-ia32-1.2.0.ks \ --cache=/opt/meego/repo-cache

Once you have the loop image built, you can chroot into it:
$> sudo mic-chroot meego-core-ia32.img

While in the chroot, you can remove or update existing packages, and even add new ones. When you are done, just type $> exit to leave the chroot.

Note: If you want to use zypper inside the chroot, ensure that you included ‘@MeeGo Base Development’ group of packages in your kickstart file.

7. Convert loop images to LiveUSB images

Once you have your loop image modified as you wanted, you can then convert it to a LiveUSB image so that it can be written to an external USB disk, as in the previous step. Converting between image types is done with the ‘mic-image-convertor‘ tool:
$> sudo mic-image-convertor \ --source-image=meego-core-ia32.img \ --target-format=liveusb

Additional notes

Be careful with mic-chroot.
The Meego Image Creation set of tools make a hard use of disk space under /var/tmp. If you (like me) usually don’t have much space left in your root partition, you’d better symlink that path to a partition without disk space worries.
I’ve found several times that when I try to build new images, the tools tell me that the packages downloaded in the cache are corrupted, and need to be re-downloaded. Just make sure you have a good connection
Again, visit http://wiki.meego.com/Image_Creation if you want to have further details on these tools.

Unaccenting words, or at least trying to…

aleksander — Thu, 21 Jul 2011 14:57:50 +0000

So today I found that Tracker in MeeGo packaging was still depending on libunac, while it shouldn’t. And that has reminded me that I had a blog post still unfinished about why and how we removed the libunac dependency in Tracker… so here it goes

One of the features supported in Tracker is doing FTS searches for words without considering ‘accents’. Of course, we’re not talking about accent as in the specific pronunciation of words relative to a location or nation. Our ‘unaccenting’ mechanism, as we call it, refers to the process of removing combining diacritical marks from characters, so that users can look for words with or without these marks. Therefore, this ‘unaccenting’ applies not only to diacritics in Latin alphabets, but also to other alphabets like Arabic, Greek, Hebrew or Korean which also have special combining diacritical marks.

In the previous 0.8 stable series of Tracker, the unaccenting mechanism was completely done by the ‘unac’ library. We were not really convinced that unac was a good option in our case, as it involved extra conversions from UTF-8 to UTF-16 and back, and measurements showed that it was one of the most time consuming steps during FTS parsing. In order to improve the situation, and as we already did ourselves some Unicode normalization work before passing the work to unac, we ended up writing our own unaccenting mechanism in Tracker for 0.10.

The method is applied to all our three Unicode-support backends (GNU libunistring, ICU and GLib), and roughly involves just two steps:

Apply a compatibility decomposition to the word (NFKD normalization).
Remove all combining diacritical marks, this is, all Unicode points within the following ranges:
- Basic range: [U+0300,U+036F]
- Supplement: [U+1DC0,U+1DFF]
- For Symbols: [U+20D0,U+20FF]
- Half marks: [U+FE20,U+FE2F]

Instead of NFKD, NFD decomposition can also be used in the method, but as the main purpose of the unaccenting is a Full Text Search in Tracker, compatibility of Unicode points is also a desired feature which we could get in the same step.

Looking at an example may explain it easier. Consider the French word “école”, which has a diacritic on top of the first ‘e’ character. This accented ‘e’ character can be encoded in UTF-8 in either a composed or decomposed way:

(NFC, composed) [0xC3 0xA9] 0×63 0x6F 0x6C 0×65
(NFD, decomposed) 0×65 [0xCC 0x81] 0×63 0x6F 0x6C 0×65

The UTF-8 encoding of the composed way (NFC or NFKC) will (probably) always need less bytes than the decomposed (NFD or NFKD) counterpart. This is because the accented ‘e’ character will be represented in composed way as a single Unicode point: ‘é’ U+00E9 (UTF-8: [0xC3 0xA9]). In the decomposed way, the same accented ‘e’ character is represented as a base character ‘e’ U+0065 (UTF-8: 0×65) plus a combining mark ‘ ́ ‘ U+0301 (UTF-8: [0xCC 0x81]).

For either of the previous two representations of ‘école’, the removal of combining diacritical marks is as we have already described:

First, get the word NFKD-normalized (or NFD if point compatibility is not needed):
- (NFC) [0xC3 0xA9] 0×63 0x6F 0x6C 0×65 —>
  (NFKD) 0×65 [0xCC 0x81] 0×63 0x6F 0x6C 0×65
- (NFD) 0×65 [0xCC 0x81] 0×63 0x6F 0x6C 0×65 —>
  (NFKD) 0×65 [0xCC 0x81] 0×63 0x6F 0x6C 0×65
Once we have the word decomposed, we can now just walk each unicode point found in the string, and remove those which end up falling into one of the ranges applicable to diacritics. In this case, only the accent on top of the ‘e’ character is found and removed: U+0301 (UTF-8: [0xCC 0x81]):
- (NFKD) 0×65 [0xCC 0x81] 0×63 0x6F 0x6C 0×65 —>
  (unaccented) 0×65 0×63 0x6F 0x6C 0×65

This new method not only worked perfectly in all the test cases we could think of, it was even much faster than using the unac library (up to 73% faster in the best case, and same speed in more complex cases).

Filed under: Development, GNU Planet, Lanedo Planet, Planets Tagged: diacritics, normalization, tracker, unaccenting, unicode

ModemManager, now with Iridium satellite network support

aleksander — Thu, 14 Jul 2011 11:01:47 +0000

ModemManager and the Iridium satellite network

I recently sent a new ‘iridium’ plugin for review upstream, this time for Iridium modems. The plugin was developed using a Iridium 9522B Satellite Transceiver modem connected through RS232, properly handled by ModemManager’s plugin system thanks to the extended RS232 support available in git master. The ‘iridium’ plugin handles these modems as any other GSM modem, even if it has nothing to do with GSM technologies.

Iridium is a constellation of 66 active (plus spares) LEO satellites orbiting at an altitude of 781 km, which gives phone and network coverage to every point in Earth. It was initially thought to be a constellation of 77 satellites, therefore named ‘Iridium’ after the chemical element with atomic number 77. The name didn’t change to ‘Dysprosium‘ when it was redesigned to maintain only 66 active satellites, no wonder why.

Even if the Iridium modems expose a GSM-modem like AT command set, several special things needed to be considered. For example, IP address setup via PPP needed more than the 20s hardcoded in NetworkManager, due to the extreme latency of the satellite network. Therefore, NM was also updated to allow ModemManager plugins to specify a specific ‘IpTimeout’ value.

See my email to the NM mailing list for further information on how to use ModemManager with Iridium support.

Ammonit Measurement GmbH sponsoring some hardware for ModemManager development

In Lanedo we have worked with Ammonit Measurement GmbH to help them with the improvement of ModemManager to handle Wavecom, Cinterion and Iridium modems. The guys at Ammonit were kind enough to sponsor some modems, so that I can spend my free time in developing and improving ModemManager, as well as in testing the modems before stable releases (Dan will probably be happy for that):

Sierra Wireless Fastrack Xtend FXT009 (GPRS modem, USB, handled by the ‘wavecom’ plugin)
Cinterion TC63i (GPRS modem, RS232, handled by the ‘cinterion’ plugin)

So, thanks Ammonit!

Filed under: Development, GNU Planet, Lanedo Planet, Planets Tagged: gnu/linux, iridium, ModemManager, network config, NetworkManager

GNU PDF, MeeGo conference in San Francisco, new Cinterion plugin for ModemManager…

aleksander — Thu, 02 Jun 2011 14:09:55 +0000

So, May has ended up being a very very busy month.

GNU PDF
After more than 2 months and tons of hours spent in trying not to break the thing (and fixing it when I did break it), I finally finished refactoring the API of the Streams module in GNU PDF in order to use the new error reporting methods. Jose is still reviewing the branch with the changes, so meanwhile I started refactoring the API of the Filesystem module. The Filesystem module is supposed to be just a small layer of abstraction on top of different filesystem backends, but not even the disk backend is fully implemented yet, with some basic pieces still missing. So as part of the API rework, I’ll try to include all missing bits in both the disk and http filesystem backends. Luckily, we got quite a lot of new people coming around the mailing list and providing patches and useful comments for these tasks, so it should be just a matter of time.

ModemManager
Another month hacking ModemManager, working on a new Cinterion plugin. After several changes in the generic code base of the daemon, plugins can now also implement specific features for RS232 modems. The new Cinterion plugin is not available in git master yet, so if you want to give it a try with Cinterion/Siemens USB or RS232 modems, get the following git repo: git://gitorious.org/lanedo/modemmanager.git and checkout the ‘plugin-cinterion’ branch from there.

MeeGo Conference 2011
Lanedo GmbH was one of the sponsors of the MeeGo Conference 2011 in San Francisco, so I had the opportunity to visit the US(of)A again. And I was not the only Spanish guy around: Luis de Bethencourt (Collabora), Iago del Toral (Igalia) and Quim Gil (Nokia) where also in the conference. With Nokia focusing on “that Harmattan thing” as some call it, Intel seems to be the one pushing MeeGo forward (even if not all decisions taken are shared by everyone). It is nice to see that the world didn’t end neither on February 11th (Elopocalypse) nor on May 21st (The Judgment Day, which now seems postponed until October)

Oh, I also turned 30 this past month… I keep on trying to forget it.

Filed under: Development, GNU Planet, Lanedo Planet, Meetings, Planets Tagged: gnu pdf, MeeGo, ModemManager

ModemManager is fun

aleksander — Thu, 05 May 2011 15:56:39 +0000

These last weeks I’ve been out of my usual Tracker work, and have been hacking on NetworkManager/ModemManager so that they can work properly with Sierra Wireless Airlink Fastrack Xtend modems.

Ended up writing a new wavecom plugin for this purpose (Wavecom was bought by Sierra Wireless in 2009), which Dan already merged in the master branch of ModemManager.

The changes in the new wavecom plugin with respect to the generic GSM plugin are:

AT+CPIN? replies are not ended with OK. A custom regex match was added to support CPIN? replies without OK (also had to hack the generic MM code to handle plugin-provided regular expressions).
Disabled default power-up comand, AT+CFUN=1. This command does a full software reset in the device, including the USB stack, so the connection is closed. No additional command seems to be needed, as the modem wakes up by default in CFUN=1 (full functionality) mode. Anyway, an additional check was added during initialization so that this mode is ensured.
Enabled RTS/CTS instead of XOFF/XON flow control.
Implemented access technology retrieval using AT+WGPRS=9,2 to query network capabilities
Ensure the modem uses the highest possible mobile class. These modems can be configured to use different mobile classes. For each kind of modem, the best mobile class is the highest one available, in the following order:
- Class A (3G only mode)
- Class B (PS or CS, GPRS/EDGE or GSM)
- Class CG (PS only, GPRS/EDGE)
- Class CC (CS only, GSM)
Implemented setting and getting allowed mode, for both 2G and 3G based modems (using AT+WWSM command in the 3G case).
Implemented setting and getting frequency bands for 2G modems (with AT+WMBS) and 3G modems (with AT+WUBS).

Still, the plugin could be improved even more. For example, there is no secondary port handling in the current plugin implementation, which means that signal strength value will not be updated while the modem is in connected state. This kind of modems support setting up virtual ports by multiplexing the serial channel (CMUX), but ModemManager doesn’t handle this yet.

If anyone has one of this Wavecom/SierraWireless modems at home, please test it with latest ModemManager and report any errors found.

Filed under: Development, GNU Planet, Lanedo Planet, Planets Tagged: freedesktop, gnome, ModemManager, NetworkManager, sierra-wireless, wavecom

Why ––as–needed doesn’t work as expected for your libraries on your autotools project

aleksander — Wed, 16 Feb 2011 15:10:21 +0000

Introduction to ––as–needed
Quoting from the Gentoo Linux documentation for ––as–needed [1]:

The –as-needed flag is passed to the GNU linker (GNU ld). The flag tells the linker to link in the produced binary only the libraries containing symbols actually used by the binary itself. This binary can be either a final executable or another library.

In a brief, this means that if you pass unneeded libraries to link to when linking your program, the linker will skip those dependencies when that flag is used.

Example, without ––as–needed
Imagine an example where you want to have a “libb.so” shared library, which needs glib-2.0. You’re lazy, and you don’t want to check whether that library will need gthread-2.0, gio-2.0 or gmodule-2.0, so you just compile and link assuming they are used:
$> gcc `pkg-config --cflags glib-2.0 gthread-2.0 gio-2.0 gmodule-2.0` -c src/libb/b.c -fPIC -DPIC -o b.o $> gcc -shared b.o `pkg-config --libs glib-2.0 gthread-2.0 gio-2.0 gmodule-2.0` -o libb.so

Where “b.c” just contains:
#include "glib.h" int test_b (int number) { g_debug ("Hello world from libb: %d", number); }

If you now show all unused direct dependencies of libb.so, with ldd, you get all unneeded ones:
$> ldd -u -r libb.so Unused direct dependencies: /usr/lib/libgio-2.0.so.0 /usr/lib/libgobject-2.0.so.0 /usr/lib/libgthread-2.0.so.0 /usr/lib/libgmodule-2.0.so.0 /lib/librt.so.1 /lib/libpthread.so.0

Example, with ––as–needed
Lets compile and link now with the ––as–needed linker flag (note that you need to tell gcc to pass the option to the linker, so you’ll need to use “–Wl,––as–needed“):
$> gcc `pkg-config --cflags glib-2.0 gthread-2.0 gio-2.0 gmodule-2.0` -c src/libb/b.c -fPIC -DPIC -o b.o $> gcc -shared -Wl,--as-needed b.o `pkg-config --libs glib-2.0 gthread-2.0 gio-2.0 gmodule-2.0` -o libb.so

And check the list of unused dependencies:
$> ldd -u -r libb.so Unused direct dependencies: /lib/libpthread.so.0

So it really seems that the ––as–needed linker option worked here and cleaned up all unneeded dependencies, nice!

But wait! Be careful with this statement in the GNU Linker manpage [2]:

This option affects ELF DT_NEEDED tags for dynamic libraries mentioned on the command line after the ––as–needed option.

You should thus, consider ––as–needed not as a global option to the GNU Linker, but as a specific option for specific dependencies. You could for example apply the option only to some of the libraries you’re linking to:
$> gcc -shared b.o -pthread -Wl,--export-dynamic -lgio-2.0 -lgobject-2.0 -Wl,--as-needed -lgthread-2.0 -lgmodule-2.0 -lrt -lglib-2.0 -o libb.so $> ldd -u -r libb.so Unused direct dependencies: /usr/lib/libgio-2.0.so.0 /usr/lib/libgobject-2.0.so.0 /lib/libpthread.so.0

Example, with autotools
You probably don’t compile and link your libraries using plain gcc, and instead use some GNU automagic (autoconf, automake, libtool…). If that is the case, and following the previous example, you would probably use PKG_CHECK_MODULES in your configure.ac:
[configure.ac] ... PKG_CHECK_MODULES(GLIB, [glib-2.0 gthread-2.0 gmodule-2.0 gio-2.0])

And you’ll probably create your shared library using libtool, so in your Makefile.am you will have:
[Makefile.am] lib_LTLIBRARIES = lib/libb.la lib_libb_la_SOURCES = src/libb/b.c lib_libb_la_CFLAGS = $(GLIB_CFLAGS) lib_libb_la_LIBADD = $(GLIB_LIBS)

If you compile this autotools project without any additional option, you will end up having “ldd -u -r” report the same unused dependencies as before (gio, gobject, gthread…)

Example, with autotools and ––as–needed
Once you have your autotools project ready, you can try to enable the ––as–needed linker flag passing it in LDFLAGS during configure:
$> ./configure LDFLAGS="-Wl,--as-needed"

This should pass LDFLAGS during the linking operation while running `make’ … and it does pass it, but in a way where it doesn’t have any effect… libtool will call gcc like this:
libtool: link: gcc -shared .libs/lib_libb_la-b.o /usr/lib/libgio-2.0.so /usr/lib/libgobject-2.0.so /usr/lib/libgmodule-2.0.so /usr/lib/libgthread-2.0.so -lrt /usr/lib/libglib-2.0.so -pthread -Wl,--as-needed -pthread -Wl,--export-dynamic -pthread -Wl,-soname -Wl,libb.so.0 -o lib/.libs/libb.so.0.0.0

As you can see, ––as–needed is given after all libraries to link to, which is like doing nothing. If you check for unused dependencies in the generated shared library, you will get:
$> ldd -u -r lib/.libs/libb.so Unused direct dependencies: /usr/lib/libgio-2.0.so.0 /usr/lib/libgobject-2.0.so.0 /usr/lib/libgmodule-2.0.so.0 /usr/lib/libgthread-2.0.so.0 /lib/librt.so.1 /lib/libpthread.so.0

Now, what is the correct way then to enable ––as–needed in this project? Said previously that this option shouldn’t be considered global, and thus passing it in LDFLAGS when running configure doesn’t seem to be a good choice.

This issue is an already known bug in GNU Libtool [4], and while some projects patch their ltmain.sh [5], there are already efforts to push a patch upstream [6].

What about ––as–needed when linking programs?
This is a whole different story. Assume we also have a “a.c” program which uses “libb.so”, and which also has all previous glib, gio and friends as dependencies (but doesn’t use them):
[Makefile.am] bin_PROGRAMS = bin/a bin_a_SOURCES = src/a/a.c bin_a_CPPFLAGS = -I$(top_srcdir)/src/libb bin_a_CFLAGS = $(GLIB_CFLAGS) bin_a_LDADD = lib/libb.la $(GLIB_LIBS)

We do again try to pass the linker option in LDFLAGS during configure:
$> ./configure LDFLAGS="-Wl,--as-needed"

We check how libtool passes the option when linking:
libtool: link: gcc -pthread -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -g -O2 -Wl,--as-needed -o bin/.libs/a bin_a-a.o -pthread -Wl,--export-dynamic lib/.libs/libb.so /usr/lib/libgio-2.0.so /usr/lib/libgobject-2.0.so /usr/lib/libgmodule-2.0.so /usr/lib/libgthread-2.0.so -lrt /usr/lib/libglib-2.0.so -pthread

In this case, the option is passed before all linking dependencies, so ––as–needed should take effect. We check with ldd the unused dependencies in the generated binary program:
$> ldd -u -r bin/.libs/a Unused direct dependencies: /lib/libpthread.so.0

So, even if we couldn’t tell the linker to remove all unused dependencies from the generated shared libraries, the removal takes place in case of generated programs.

You could arguably say that you don’t need to remove unneeded dependencies on shared libraries that you compile, as the programs linking with your library may really need them. In this example, if a program links to our “libb.so” and the program needs gio libraries, if we didn’t remove the unneeded dependencies in “libb.so”, the program would satisfy the gio dependency directly from “libb.so”. But that is definitely not the way to go, if the program needs gio because it uses some of its symbols, it should explicitly link to it, not assume that “libb.so” will force the dependency.

References:
[1] Gentoo Linux documentation for ––as–needed
[2] ld(1) GNU Linker
[3] Full autotools-based example
[4] GNU libtool bug report for –as-needed failing with libraries
[5] Meego libcontentaction patch and Meego Tracker patch
[6] Debian bug #347650

Filed under: Development, Lanedo Planet Tagged: gcc, gnu, gnu ld, gnu libtool, gnu linker

PVOL in the AstronomíA magazine

aleksander — Mon, 15 Nov 2010 14:38:25 +0000

I started writing the PVOL (Planetary Virtual Observatory and Laboratory) system during the first NVO Summer School which took place in Aspen (Colorado, USA) in the summer of 2004. The system is up and running since November 2004 in the Ajax Cluster of the Planetary Sciences Group, in the University of the Basque Country, and available at www.pvol.ehu.es.

We currently have more than 200 registered contributors (50 of them allowed to directly publish new images), more than 9200 images of Jupiter (first one from year 2000) and more than 1800 images of Saturn. Last image received was just from yesterday:

Good news is that not only amateur observers are publishing lots of new images of Giant Planets every day (mainly Jupiter and Saturn), but also that two of the guys in the Planetary Sciences Group also wrote an article about the PVOL system and how it helps on the research of the atmospheres of these planets. The publication is not available online, it seems, so if you want to take a look at it you’ll need to buy the magazine (Spanish only):

“Pequeños telescopios al estudio de los planetas gigantes“, por Jon Legarreta y Ricardo Hueso, AstronomíA, November 2010.

Filed under: Projects, PVOL Tagged: Astronomy, PVOL

m4 macro to check if SQLite was compiled thread-safe

aleksander — Tue, 05 Oct 2010 18:38:35 +0000

If your application is using SQLite, you may be assuming it is always compiled thread-safe, but this is not true. It will only be compiled thread-safe (with proper mutexes enabled) if the SQLITE_THREADSAFE preprocessor macro is set to 1 (serialized) or 2 (multi-threaded) (1/serialized is the default if not explicitly stated). If SQLITE_THREADSAFE is set to 0 during compilation of SQLite, it will not be thread-safe. Also note that if compiled thread-safe (1 or 2), the thread-safety mode (serialized or multi-threaded) can later be changed in runtime. See SQLite and Multiple Threads for more information.

SQLite provides the sqlite3_threadsafe() method to check whether the library was compiled thread-safe, but if your application really needs a thread-safe SQLite, you don’t want to check it during runtime. Thus, I wrote a small m4 macro that you can use in your configure.ac to check it when running ./configure

Just add this file in your m4 directory:
http://es.gnu.org/~aleksander/sqlite/sqlite-threadsafe.m4

And then, in your configure.ac:
AX_SQLITE_THREADSAFE if test "x$ax_cv_sqlite_threadsafe" != "xyes"; then AC_MSG_ERROR([sqlite3 is not compiled thread-safe]) fi

Filed under: Development, Lanedo Planet Tagged: autoconf, m4, sqlite, thread-safe

Trisquel Taranis is here, including a MINI edition!

aleksander — Wed, 29 Sep 2010 16:46:56 +0000

After a great job done during several months by Rubén and the rest of the Trisquel team, the long-waited Trisquel 4.0 Taranis” was published during the Software Freedom Day.

This new release of Trisquel is based on Ubuntu 10.04 “Lucid”, and provides a fully-free distribution of GNU/Linux, running the 100% free Linux-libre 2.6.32 kernel. This release comes in two different primary versions, the “Standard Edition” and the “Mini Edition”.

The “Standard Edition” is meant for standard desktop computers, and includes:

GNOME 2.30
OpenOffice.org 3.2
Gimp 2.6
ABrowser 3.6.9 (Firefox-based web browser)

The “Mini Edition”, focused on computers with CPU/RAM limitations like Netbooks, includes:

LXDE 0.5
Midori 0.2

I’ve been using Trisquel Taranis Standard Edition since some months ago, while it was not even in beta stage; but also wanted to check the look and feel and performance of the Mini version, so I decided to try it in a VM.

The boot menu when installing Trisquel Taranis is really appealing:

Remember that you can try Trisquel GNU/Linux without installing it!

This is the initial look and feel of Trisquel Taranis Mini after installing it and logging in the user account. Note to Windows users, what’s the difference between a fresh Windows installation and a GNU/Linux installation? Just check the image…

Answer: The “welcome” popup in GNU/Linux tells you about new software updates, it doesn’t tell you “Your computer is at risk!!!” because you didn’t install the antivirus or whatever. GNU/Linux doesn’t need an antivirus, and a fresh install is just SECURE, not like a Windows one.

The lightweight Mini edition will use only ~90MB of RAM when booted, which makes it perfect for netbooks and laptops without much memory:

So, if you want to give it a try, Download Trisquel Taranis! And if you like it, you can also Donate to the project, or buy some nice Trisquel Merchandise…

Filed under: Operating Systems Tagged: gnu/linux, taranis, trisquel, ubuntu

The Octopus Library 0.0.1 released

aleksander — Fri, 10 Sep 2010 10:48:45 +0000

This is the first version of the Octopus Library, a C library providing advanced file system crawling and monitoring capabilities on top of GLib’s GIO. This library, released under the LGPLv3+, is a full rewrite of some of the features provided by Gnome Tracker‘s libtracker-miner library.

Online documentation is available in:
http://es.gnu.org/~aleksander/octopus/doc

The sources can be downloaded from:
http://es.gnu.org/~aleksander/octopus/releases/octopus-0.0.1.tar.gz

The mdsum is:
cca4e5152e9586abc57d330bad7bee09 octopus-0.0.1.tar.gz

This release is also GPG signed. You can download the signature from:
http://es.gnu.org/~aleksander/octopus/releases/octopus-0.0.1.tar.gz.sig

Some of the features provided by this new library are:
* Recursive crawling on a set of configured directories. For example, it may be configured to recursively crawl /home and /media.
* Directories inside the tree to recurse can be set to be ignored. For example, it may be configured to recursively crawl /home/user but not /home/user/.private.
* Directories may be configured to be crawled, but not recursively. For example, /home may be configured to be non-recursively crawled.
* Any possible combination of crawled, recursively-crawled and ignored is supported, as long as the same path is not added with different options.
* For every file or directory found during the crawling operation, a new event will be notified to configured signal listeners.
* During the filesystem crawling, monitors may be enabled in directories.
* Event merging; so that multiple events of the same type on the same file are all combined in a single event. Event merging will also act on different event types, trying to combine them in the best possible way. For example, if a file is CREATED, UPDATED and then REMOVED no event will be notified.
* Real-Time event notification. Both merged or raw events can be set to be notified in real-time.
* On-Demand iteration of events. If enabled, the monitor will store all received events in a proper order, so that afterwards they can be iterated. You could for example, configure a timeout every 30 minutes to iterate all raw or merged events which happened since the last iteration.
* Proper order of events. When notifying or iterating events the order may not be exactly the same as the one in the original events, but it is assured that if followed all events in the order given by the Octopus Monitor, the result is exactly the same as if followed in the original order.

In the roadmap for next versions:
* Asynchronous API will be available in release 0.0.2.
* A ‘database’ of already crawled directories will be optional, storing mtimes of each found file, so that when the program starts and re-crawls a common directory structure, it will check if the files in the tree changed since the last time crawled. This will be available in release 0.0.3.

Please send bug reports and suggestions to aleksander (at) gnu.org or aleksander (at) lanedo.com

Filed under: Development, Lanedo Planet, Octopus, Projects Tagged: lgplv3, octopus

Ubuntu packaging updates: dbus 1.3.1, sqlite 3.7.1 and gnu psychosynth 0.1.5

aleksander — Tue, 24 Aug 2010 14:25:16 +0000

SQLite and DBus

I’ve been lately packaging the latest releases of several programs for Ubuntu Lucid (or Trisquel Taranis), mainly due to our bleeding-edge requirements in Tracker. You can find them in my personal PPAs in Launchpad:

SQLite 3.7.1 (enabling Write-Ahead logging):
https://launchpad.net/~aleksander-m/+archive/sqlite3
dbus 1.3.1 (enabling file descriptor passing):
https://launchpad.net/~aleksander-m/+archive/dbus-development

The real packaging work was done by the Debian guys, of course (kudos to them!); I just prepared the necessary bits to make them get compiled as Ubuntu packages.

GNU Psychosynth

In addition to this, during the last GNU Hackers Meeting in The Hague, I also changed the Ubuntu packaging of GNU Psychosynth, the most entertaining music synthesizer I’ve ever played with. The latest packages are now available in the GNU Psychosynth Team’s PPA, and right now only Ubuntu versions >= Lucid will be supported (as they no longer require Andrew Fenn’s Ogre Development PPA).

More information in: http://www.psychosynth.com/index.php/Ubuntu_Repository

Oh, and Juan Pedro was able to do another demonstration of GNU Psychosynth during the meeting, which got recorded, so that you can understand what this great program does… watch it and enjoy!
http://audio-video.gnu.org/video/ghm2010/GNU_Psycosynth.ogv

Filed under: Lanedo Planet, Packaging Tagged: dbus, ghm, gnu psychosynth, sqlite, trisquel, ubuntu

My pictures from Aspen here and there

aleksander — Fri, 16 Jul 2010 15:35:58 +0000

Sometime ago the people organizing the Boostcon contacted me to see if they could publish a picture I took in Aspen during my participation in the NVOSS. Even if I’m a pure ANSI C guy, I accepted

This last week, the german online magazine ScienceGarden contacted me for exactly the same reason, but they chose different pictures:

So, if any other person wants to publish more images of Aspen, feel free, and a reference is always appreciated!

Filed under: Personal

Lanedo, Tracker, Devhelp, GHM, GUADEC…

aleksander — Fri, 16 Jul 2010 09:06:49 +0000

So, starting on April 1st (not a fools’ day joke!), I’m now employed by Lanedo GmbH, and full-time working on the Gnome Tracker project, mainly involved in FTS and the filesystem miner.

It was great to work with everyone in Azetti Networks. Keep Jardin and Waterfall alive!

I also hacked a new feature for Gnome Devhelp, the possibility of enabling or disabling specific books, which was integrated into git master last week, and will make it for the 3.0 release.

Oh, and I’ll be attending both the GHM and GUADEC in The Hague!

Filed under: Development, Lanedo Planet, Personal Tagged: devhelp, ghm, guadec, lanedo, tracker

Gaztea Irratia & Rhythmbox

aleksander — Fri, 12 Mar 2010 11:25:38 +0000

First thing when I go back to the Basque Country to visit my family is to tune Gaztea Irratia in my car’s radio.

Now, I can listen to it from anywhere using Gnome’s Rhythmbox and the Shoutcast live stream. Just add a new radio station in Rhythmbox with this URL:

http://yp.shoutcast.com/sbin/tunein-station.pls?id=77647

And enjoy!

Filed under: Programs Tagged: gaztea irratia, gnome, internet radio, rhythmbox

Understanding Valgrind memory leak reports

aleksander — Thu, 04 Feb 2010 00:42:44 +0000

I tried to write a tutorial focusing on the type of memory leaks as detected internally by Valgrind and the generated output leak reports.

The PDF is available in the following URL:
http://es.gnu.org/~aleksander/valgrind/valgrind-memcheck.pdf

And the simple C tester to generate each type of memory leak (cases 1 to 9 in the Valgrind Manual) is available here:
http://es.gnu.org/~aleksander/valgrind/valgrind-memcheck.c

Comments, fixes, whatever… highly appreciated. You could even send me a diff file to the original LaTeX source, available in:
http://es.gnu.org/~aleksander/valgrind/valgrind-memcheck.tex

The document is licensed under the GFDL 1.3, and the tester is published in the public domain.

Filed under: Development, Writting Tagged: c, gnu/linux, valgrind

The Pirate Banana

aleksander — Tue, 02 Feb 2010 19:20:00 +0000

This is probably the weirdest drawing I’ve ever made… My brother-in-law asked me if I could draw a “Pirate Banana” to be used in the Carnival party soon in February… yeah, don’t ask me why he asked for that.

But here is what it came up after a couple of hours… (click image to see it bigger).

The Pirate Banana

Licensed under the Creative Commons Attribution-Share Alike 3.0 License

Filed under: Drawing, Personal Tagged: Drawing, pirate banana

See you at FOSDEM!

aleksander — Mon, 18 Jan 2010 19:35:56 +0000

This year, I will be attending FOSDEM 2010 in Brussels (Belgium).

See you there!

Posted in Meetings

Knight’s Tour brute force algorithm example

aleksander — Tue, 12 Jan 2010 23:50:22 +0000

(From Wikipedia) The Knight’s Tour is a mathematical problem involving a knight on a chessboard. The knight is placed on the empty board and, moving according to the rules of chess, must visit each square exactly once. A knight’s tour is called a closed tour if the knight ends on a square attacking the square from which it began (so that it may tour the board again immediately with the same path). Otherwise the tour is open.

Just developed my brute-force algorithm implementation, using GLib and based on a simple recursive function.

Download and try it here!

As an example of result, this is the open Knight’s Tour found by the algorithm in a 5×5 board, starting from position [0,0]:
[0,0][1,2][2,4][4,3][3,1][1,0][2,2][0,3][1,1][3,0][4,2][3,4][1,3][0,1][2,0][4,1][3,3][1,4][0,2][2,1][4,0][3,2][4,4][2,3][0,4]

And this one, the first Knight’s Tour found by the algorithm in a 8×8 board, starting from position [0,0]:
[0,0][1,2][2,4][3,6][5,7][7,6][6,4][7,2][6,0][4,1][5,3][6,5][7,7][5,6][7,5][6,3][7,1][5,0][6,2][7,4][5,5][6,7][4,6][5,4][6,6][4,5][3,3][5,2][7,3][6,1][4,0][2,1][4,2][3,0][1,1][0,3][2,2][0,1][2,0][3,2][4,4][2,3][0,4][1,6][3,7][2,5][1,7][0,5][1,3][3,4][1,5][0,7][2,6][4,7][3,5][2,7][0,6][1,4][0,2][1,0][3,1][4,3][5,1][7,0]

Both examples above show open Knight’s Tours.

Posted in Development Tagged: algorithms, glib

Simple LaTeX template for Books

aleksander — Tue, 12 Jan 2010 00:51:12 +0000

I prepared a simple LaTeX template for Books and/or eBooks (“book” document class), showing the usage of several common packages (“geometry”, “inputenc”, “setspace”, “fancyhdr”, “tocloft”}.

You can download the source LaTeX template here!

And get a PDF using GNU Texinfo..
$> texi2pdf book_template.tex

Resulting PDF can be seen here!

Posted in Development, Writting Tagged: eBook, LaTeX, texinfo

Fools rush in where angels fear to tread

aleksander — Tue, 05 Jan 2010 21:59:00 +0000

I really had to write an email (quoted below) to the author of this blog post regarding the old GNU/Linux vs Linux battle…

Just wanted to tell you that you are sooooo wrong in most of the things
you say… and that it is very very funny how you say you can remove all
GNU apps from several distros.

Did you remove GNU coreutils? Then how can you for example do a simple
“ls” in the command line?

Oh, and do you know what the “G” means in GNOME?
http://en.wikipedia.org/wiki/GNOME

Glib? GTK+? All GNU packages.

And you’re part of an Ubuntu LoCo team —> What about Bazaar, widely
used in Ubuntu, as it is written by Canonical people… Also a GNU
package!!
https://lists.ubuntu.com/archives/bazaar-announce/2008-February/000135.html

Tell me now what you get if you remove all GNU packages from Ubuntu (if
theoretically possible). Including GLib, GTK+ (and of course all their
dependencies); including coreutils, bash, gdb, gcc… including the
glibc (GNU C library)!

Now what you will probably get?

Probably *nothing*. Even the Linux kernel depends on the glibc.

Now, next time, I would suggest you just read a little bit more and
understand what GNU means, before writing such wrong post.

Check which is the whole list of GNU packages here:
http://directory.fsf.org/GNU

Posted in Operating Systems Tagged: gnu, gnu/linux, linux

g_source_unref() and g_source_destroy() are your friends

aleksander — Tue, 05 Jan 2010 21:05:31 +0000

After almost 2 years developing GLib-based applications, I understood the proper way of using GSource objects. Yes, quite a shame, but better now than never.

Now, after re-reading carefully the GLib Reference Manual, I see that it is quite clearly explained the difference between g_source_destroy() and g_source_unref(). So the typical suggestion is still the best one: RTFM!!

In our applications, we usually need to attach timeout operations to the context of an specific thread, not to the main thread context. Thus, we cannot use g_timeout_add() or the pretty new g_timeout_add_seconds().

For example, you could create a GThreadPool, and in the function to be executed in each thread, you could create a specific context for the thread, plus a main loop in the context:

    /* Create a GLib Main Context */
    context = g_main_context_new();

    /* Create a Main Loop in the context*/
    main_loop = g_main_loop_new(context,
                                FALSE);

Once you have a new context and main loop, you can just create a new GSource, and attach it to the main loop. As soon as you create the GSource, its reference count is 1, and as soon as you attach it to the main loop, its reference count will be 2.

    /* Create new timeout source to be called
     * every 5 seconds.
     * Reference count of source is 1 once created */
    source = g_timeout_source_new(TIMEOUT_MSECS);

    /* Set callback to be called in each timeout */
    g_source_set_callback(source,
                          (GSourceFunc)__timeout_func,
                          main_loop,
                          NULL);

    /* Attach the GSource in the GMainContext.
     * Reference count of source is 2 after this call */
    g_source_attach(source,
                    context);

Of, course, you will be now running the main loop:

    /* Run the main loop, until it is stopped */
    g_main_loop_run(main_loop);

The key now is how to destroy the GSource properly. When calling g_source_destroy(), you are doing 2 things: first, telling the main loop to forget about the GSource; and second, decrementing the reference count of the GSource. Then, you still need to call g_source_unref() to fully decrement the reference counter so that the GSource is disposed.

    /* We did an attach() with the GSource, so we need to
     * destroy() it */
    g_source_destroy(source);

    /* We need to unref() the GSource to end the last reference
     * we got */
    g_source_unref(source);

To end the example, once the main loop is stopped, you will also need to properly dispose the GMainContext and GMainLoop objects:

    /* The main loop should be destroyed before the context */
    g_main_loop_unref(main_loop);

    /* Finally, destroy the context */
    g_main_context_unref(context);

This is just one way of keeping the GSource references properly managed:

You get one reference when you create the GSource
You get a new reference when you attach it in the context
You release one reference when you destroy it from the context
You release last reference when you unref the final one

Of course, you could also choose to avoid storing the “extra” reference, and leave alive only the one inside the main context:

You get one reference when you create the GSource
You get a new reference when you attach it in the context
You release one reference when you unref the the GSource —> Now, the only reference is inside the GMainContext
You release last reference when you destroy it from the context

You can check this simple example in the following program I prepared, released into public domain:
http://es.gnu.org/~aleksander/glib/test-gsource.c

Hope it helps someone out there to fully understand the difference between g_source_destroy() and g_source_unref().

As last comment… I would really rename the following functions in the library, so that no one else is confused with the “unref” and “destroy” terms:

Rename g_source_attach() to g_source_attach_to_context()
Rename g_source_destroy() to g_source_destroy_from_context()

The main reason is that these two operations actually act on the GMainContext, while the name of the functions do not suggest that fact.

Posted in Development Tagged: glib, gsource

Stack corruption: improper use of FD_SET

aleksander — Wed, 23 Dec 2009 22:53:24 +0000

So here is a very simple way of corrupting your stack when using select() to poll a given file descriptor. In El Jardin library, we used to use select() to see if incoming data was available for reading in a given socket. Using select() allows to have a maximum wait time so that the execution in the thread is not blocked until the data arrives. The code was just something like this (assuming “fd” an integer specifying the file descriptor number in the process):

fd_set rset;
struct timeval tv;

/* Initialize the array of flags, specifying the
 * FD we want to monitor */
FD_ZERO(&rset);
FD_SET(fd, &rset);

/* Set max wait time to 1 second */
tv.tv_sec = 1;
tv_tv_usec = 0;

/* Run select! */
if(select(fd+1, &rset, NULL, NULL, &tv) < 0)
{
    /* Check errno */
}

You may see this kind of code in lots of examples of usage for select(). But please, read the constraints carefully also! Some things you need to understand:

fd_set is an array of bits, of FD_SETSIZE elements.
FD_ZERO is a macro clearing (setting to ’0′ all bits in the fd_set array).
FD_SET is a macro setting to ’1′ the bit for the specific file descriptor you want select() to check.

And the most important thing once you understood the previous ones:

FD_SETSIZE is usually defined to 1024 in GNU/Linux systems

This clearly means that the maximum file descriptor number to be used in select() must be 1024.

The GNU C Library documentation actually explains it perfectly:

The value of this macro is the maximum number of file descriptors
that a fd_set object can hold information about. On systems with
a fixed maximum number, FD_SETSIZE is at least that number. On
some systems, including GNU, there is no absolute limit on the
number of descriptors open, but this macro still has a constant
value which controls the number of bits in an fd_set; if you get
a file descriptor with a value as high as FD_SETSIZE, you cannot
put that descriptor into an fd_set.

Now, if you actually do what the GNU C Library documentation tells you not to do (using a FD with value higher than 1024 in this case), what you get 100% sure is a stack corruption. Why?

In the above example, the fd_set array is in the stack
FD_SET macro apparently doesn’t know about the FD_SETSIZE value, so even if you pass a FD greater than 1024, it will actually set to “1″ the corresponding bit in the fd_set array of bits, which actually is OUTSIDE the array. Thus, corrupting the stack.

Did it happen this to you?

In our case, we first noticed using the GNU Debugger that some pointers in the stack magically changed their value, and only in a single bit (new address was previous address plus a multiple of 2). Also, that bit was different in different core files analyzed (depending on the FD number being used).
In some other cases, the stack was so corrupted that GDB was not even able of showing a proper backtrace

But anyway, this problem was not only because of an improper use of select(). We also discovered a FD leak (socket open, never closed) which was making the FD number go up and up until being greater than 1024 after some hours. So best suggestion in this case: use Valgrind and track FDs:

# valgrind --track-fds=yes ./my-program

In El Jardin, we solved it (LP#497570) avoiding the use of select(), and using poll() instead, which doesn’t have this hard-coded limit of 1024. Other options are using epoll(), or defining your own FD_SETSIZE value after having included the system headers.

Posted in Development Tagged: gdb, jardin, poll, select, stack corruption, valgrind

gNewSense in my home server

aleksander — Thu, 04 Jun 2009 19:23:09 +0000

Decided to drop Fedora10 in my home Asus PX24 server, and installed gNewSense, one of the full-free distributions promoted by the Free Software Foundation and the GNU project.

Go download it!
http://www.gnewsense.org/

In order to have all components working perfectly, Wireless networking among them, I needed to download latest linux-libre kernel. This kernel doesn’t ship any proprietary type of driver, not even those trying to seem free software but hiding the real source code into unintelligible hex characters, which are the ones GPL-ed.

Free your server!!

Posted in Operating Systems Tagged: fedora, gnewsense, gnu, gnu/linux

Jazztel vs Ya.Com

aleksander — Tue, 07 Apr 2009 21:56:28 +0000

I guess this is only interesting for spanish people…

————–

Por fin, Ya.Com no me da servicio de internet. Y digo por fin porque llevaba desde comienzos de Enero con un servicio de ADSL pésimo. Un resumen (casi) corto:

* Comienzos de Enero: Mi ADSL de repente se comporta de forma extraña. El enlace cae constantemente, intenta sincronizar a velocidades altas (tenía contrato hasta 20 Mbps), y vuelve a caer, con lo que intenta sincronizar a menor velocidad. Y así, hasta que el enlace cae a 300/400 Kbps (a veces hasta 1Mbps como máximo).

* Como mi contrato es de 20Mbps, el mínimo por ley que te deben garantizar es el 10%, esto es, 2Mbps.

* Llamo al servicio de Ya.Com. Después de 1 hora de charla, el técnico llega a la conclusión de que el problema es del router, que hay que comprar uno nuevo.

* Como el router originalmente era el que me mandó Ya.Com, intento que Ya.Com me mande uno nuevo, ya que ese era defectuoso, según el técnico. El técnico me asegura que no será ningún problema que me den un router nuevo.

* Tras 4 días esperando a que alguien se decida a llamarme, llamo de nuevo. No hay router nuevo. Sólo me lo dan con un contrato de permanencia de 12 meses.

* Llamo al teléfono de bajas de Ya.Com y digo que me quiero dar de baja. Oh, sorpresa, me ofrecen 6 meses a mitad de precio si me quedo, y que me pasarán con un técnico de nivel 2. Consejo: si alguna vez os dicen eso, daros por engañados. Después de minutos de espera con una música horrible, el técnico te descuelga, dice cuatro palabras y la comunicación se corta. Esto es más que común. En tres meses de llamadas me ha pasado día sí y día también.

* Consiguen convencerme con lo de la oferta a mitad de precio y el técnico de nivel 2. Se ve que si no les pasan un técnico de nivel 2 al momento a la gente de bajas, te dicen que te llaman ellos y que te pasan al técnico. Mira, esto sí es verdad. La gente del departamento de bajas te llama, y te dice que te pasa con el técnico de nivel 2. Y hasta ahi. Pasan minutos de música y cuando el técnico de nivel 2 te coge, dice 4 palabras y te cuelga. Oh, sorpresa.

* Como el servicio ténico era imposible, decidí comprar un router nuevo. El primer técnico que me atendió estaba bastante seguro de que era el problema… así que me compré uno nuevo (más de 100 euros), lo enchufo, lo configuro y… voilá! Pasa lo mismo… intentos de sincronización de ADSL hasta que se queda estable en menos de 1Mbps…

* Más llamadas al servicio técnico. Que mirarán en la central y me llaman. Que harán una prueba conjunta con Telefónica (la que me daba la línea) y me llaman…

* Más llamadas al servicio técnico, me dicen que no ven ningún problema en la central, y que confirman que la sincronización de ADSL se reintenta hasta que queda estable por debajo de 1Mbps. Si configuro el router para no utilizar ADSL2+, sino sólo ADSL2 es más estable, pero la velocidad no supera 1Mbps.

* Cambio cables de telefonía. Desde enero el router estaba directamente empalmado al PTR de Telefónica, para asegurarnos que no era la instalación interna.

* Finalmente me llama un técnico muy agradable. Mira oye, que va a ser problema del PTR de Telefónica, que llame a Telefónica para que me lo cambien y que no me cobrarán nada. Que en Ya.Com han hecho ya todo todo lo que estaba en sus manos, y que ellos no pueden hacer más, que es el PTR fijo. Lo que nos faltaba, vamos. Se ve que les dan un curso de forwardear marrones a los técnicos de Ya.Com.

* Telefónica viene y me planta un PTR nuevo. Y, oh sorpresa, sigue pasando lo mismo…

* Después de alguna semana más, me llama una técnica y me planta lo siguiente: Que la solución para mi problema es que cambie mi contrato de 20Mbps a 10Mbps, y entonces se soluciona todo. Parece razonable? No. Recordemos que por ley te deben garantizar el 10% de lo contratado. Con mi contrato de 20Mbps estaban fastidiados porque el 10% eran 2Mbps, y desde Enero mi conexión no superaba 1Mbps. No tiene sentido que por un simple cambio de contrato, los problemas en la línea se solucionen. Yo le dije que si esa era la solución, primero me diesen 10Mbps, que yo los vea, y luego me cambiaba de contrato. Insistía que eso no podía ser, que primero era el cambio de contrato (algo absurdo). O sea, que me cambio a 10Mbps y se me solucionan los problemas? Mejor dicho, se les solucionan a ellos! Si paso a 10Mbps de contrato, por ley me tienen que garantizar 1Mbps, cosa que sí tenía! Tócate los pies!!! Si me llego a cambiar de contrato, ellos ya se lavan las manos con mi problema.

* Relleno el formulario para darme de baja en la web de Ya.Com, espero 24h, y llamo al departamento de bajas para confirmarlo (después de 3 meses estos pasos ya los tenía claros).

* Hoy, más de 3 meses después de empezar con los problemas, y tras decenas de llamadas, he visto que la luz de sincronización en el router estaba fija, y la de datos apagada. El router de Jazztel lo tienen todavía en Seur, así que como ya tengo uno en casa, he llamado a Jazztel para que me den los datos de conexión (PPPoE en este caso necesita usuario y password, no como en Ya.Com).

* Después de 5 minutos con el servicio técnico de Jazztel, ya tengo internet. Y no sólo eso… la conexión de Ya.Com, en su mejor momento me sincronizaba a 13Mbps… la conexión de Jazztel me sincroniza a 20Mbps! Estupendo vamos.

En resumen: Si dudas entre Ya.Com y Jazztel… mi experiencia es que Jazztel mejor… (por ahora! cruzo dedos…)

Posted in Personal Tagged: ISP spain, Jazztel, Spanish, Ya.Com

The Core Pattern (core_pattern), or how to specify filename and path for core dumps

aleksander — Fri, 13 Mar 2009 20:41:51 +0000

1. Introduction to Core Dumps

In most GNU/Linux systems (all of those I personally have used, at least), core dump files generated after an uncaught signal in a process (as a SIGSEGV or SIGQUIT), are generated in the base directory where the program was executed, and named as “core” or “core.PID”.

For example:
$> cd /home/user $> ulimit -c unlimited $> kill -s SIGSEGV $$

This will trigger a segmentation fault in your current shell (you probably guessed it after seeing that the shell session where you executed it was closed), and generate a core file in:
/home/user/core

Now… is it possible to change where that file is generated by default instead of the current directory? And is it possible to change the name of that generated file? The answer is YES! to both. Let’s see how we can get this.

2. The Core Pattern in Kernel

Since some years ago, the kernel configuration includes a file named “core_pattern”:
/proc/sys/kernel/core_pattern

In my system, that file contains just this single word:
core

As expected, this pattern shows how the core file will be generated. Two things can be understood from the previous line: The filename of the core dump file generated will be “core”; and second, the current directory will be used to store it (as the path specified is completely relative to the current directory).

Now, if we change the contents of that file… (as root, of course)
$> mkdir -p /tmp/cores $> chmod a+rwx /tmp/cores $> echo "/tmp/cores/core.%e.%p.%h.%t" > /proc/sys/kernel/core_pattern

And we run the same as before:
$> cd /home/user $> ulimit -c unlimited $> $> kill -s SIGSEGV $$

We get… voilá!
/tmp/cores/core.bash.8539.drehbahn-mbp.1236975953

Not only the program name (“bash“) or the PID (“8539“), but also the hostname (“drehbahn-mbp“) and the unix time (“1236975953“) are appended in the name of the core file!! And of course, it is stored in the absolute path we specified (“/tmp/cores/“).

You can use the following pattern elements in the core_pattern file:
%p: pid %: '%' is dropped %%: output one '%' %u: uid %g: gid %s: signal number %t: UNIX time of dump %h: hostname %e: executable filename %: both are dropped

Isn’t is great?! Imagine that you have a cluster of machines and you want to use a NFS directory to store all core files from all the nodes. You will be able to detect which node generated the core file (with the hostname), which program generated it (with the program name), and also when did it happen (with the unix time).

3. Configure it forever

The changes done before are only applicable until the next reboot. In order to make the change in all future reboots, you will need to add the following in “/etc/sysctl.conf“:
# Own core file pattern... kernel.core_pattern=/tmp/cores/core.%e.%p.%h.%t

sysctl.conf is the file controlling every configuration under /proc/sys

Posted in Development Tagged: core, coreutils, core_pattern, gnu/linux, kernel

Fedora 10 in Asus Mini Nova Lite PX24 (2): Wireless Networking

aleksander — Wed, 11 Mar 2009 20:32:10 +0000

My friend Tomasz gave me yesterday the configuration to enable Wireless Networking at system-level (skipping NetworkManager) for Ubuntu. It’s just modifying /etc/network/interfaces to setup the Wireless device directly there:
auto lo iface lo inet loopback iface wlan0 inet dhcp wireless-key s:your_f****_password_here wireless-essid your_f****_essid_here

I wanted the same in my home Fedora 10 server, but as you already may know, Fedora 10 is based on Red Hat, so forget about the debian-like network configuration, and welcome to the system-config-* scripts!

You can edit your system-level network configuration, executing, as root:
$> system-config-network
(or clicking on System->Administration->Network)

In the Wireless Settings tab you should select “Managed” mode, and I also specified the correct SSID of my home wireless network. Be really careful if adding the WEP key in HEX, as you must prepend “0x” to the whole key or it won’t work.

I thought that this would be enough, but nope. NetworkManager is launched by default at system startup, not the /etc/init.d/network script, which is disabled by default. You just need to disable NetworkManager, and enable the standard “network” script in order to have the system-level wireless configuration, executing, as root:
$> service NetworkManager stop $> chkconfig --level 35 NetworkManager off $> service network start $> chkconfig --level 35 network on

I really like chkconfig, but not sure if the “service” command is a needed replacement of executing the script from /etc/init.d directly.

Posted in Operating Systems Tagged: chkconfig, fedora, network config, NetworkManager, service, system-config-*, wireless

Fedora 10 in Asus Mini Nova Lite PX24 (1)

aleksander — Wed, 11 Mar 2009 20:10:31 +0000

I recently bought a small Asus Mini Nova Lite PX24 and of course, installed GNU/Linux on it. I chose Fedora 10 because at that time, it was the only distro I tested with the required driver for the Wireless interface:
Atheros Communications Inc. AR242x 802.11abg Wireless PCI Express Adapter

Other distros I have tried in the Asus PX24 are:
* Ubuntu 8.04 Hardy Heron [Doesn't work]
* OpenSUSE-11.1-GNOME [Doesn't work]
* Debian 5.0 Lenny [Doesn't work]
* Ubuntu 9.04 Jaunty Jackalope Alpha-5 [Works]

I was a Kubuntu fan until release 8.10, which completely destroyed the usability and stability of the distribution when shipping a non-stable version of KDE4. Switched then back to Gnome in Ubuntu 8.10, far more stable than the KDE counterpart. So now that I know that next stable version of Ubuntu also makes the Wireless networking work out-of-the-box, I thought of changing the PX24 back to Ubuntu.

But I won’t… I want to try yum, RH’s system-config-whatever scripts, SELinux… all those things that are not available in Ubuntu.

Posted in Operating Systems Tagged: fedora, gnu/linux, kubuntu, suse, ubuntu, wireless

Save my time & bandwidth: Think before replying in HTML

aleksander — Mon, 09 Mar 2009 22:18:33 +0000

I get in tons of emails the following text appended to the sender’s signature (or similar):
Save a tree: Think before you print this email

Usually, that text is included along with some big JPEG image included within an HTML-formatted colourful email, which of course, takes its time to render in my email desktop application or even worse, shown as attached image in GMail interface.

Now I am thinking in adding this other text after my signature in the emails:
Save my time & bandwidth: Think before you reply to this email in HTML

Posted in Personal Tagged: email, html

Nokia Phonebook exported via Bluetooth to my car

aleksander — Sun, 01 Mar 2009 22:05:10 +0000

I bought a new car last week, and I wanted to connect my Nokia N78 with the Bluetooth call manager in the car to be able to get the Phonebook (a.k.a. Contact list) of my phone accessible from my car’s screen and controls. After googleing a lot for the solution, I found the “Nokia 810 Contact Download Application for Nokia S60 3rd edition devices” available in Nokia’s support website.

This program is supposed to work in the following Nokia phones: Nokia E50, E60, E61, E61i, E62, E66, E70, E71, E90 Communicator, N71, N73, N77, N75, N76, N80, N91, N91 8GB, N92, N93, N95, 3250, 5500 Sport, 5700 XpressMusic, and 6290.

I downloaded it and installed, and voilà, now I have my phone’s address book (only that in the SIM card, anyway), accessible from the controls in my car! So even if the N78 is not in the list provided in Nokia’s website, I can confirm it works nice.

Download the SYS file
Download the manual in PDF

Posted in Programs Tagged: bluetooth, car, nokia, phonebook, S60

ARTE

aleksander — Sat, 28 Feb 2009 14:09:30 +0000

Not so long ago, I asked my first employer if they were still using the Unit Testing framework I developed for them while I was still in University. It was simple, with lots of hardcoded stuff and not very professional, but it worked perfectly for the needs in that moment. They told me they were not really using it, so I asked them to make it Free Software and they agreed.

A couple of weeks ago, ARTE was released licensed under the GPLv3+ license, fully copyrighted by Panda Security S.L..

I now started to improve ARTE, with better documentation and based on GLib library. ARTE is hosted in SourceForge, and you can download the first stable release here.

Posted in Projects Tagged: arte, gplv3

Having fun with an A380

aleksander — Wed, 25 Feb 2009 19:06:55 +0000

So imagine that you are a pilot of small aeroplanes. Really small ones, like the ones for two people with propellers. You have piloted that aeroplane tons of times.

One day, you find yourself in a holiday trip flying in a big A380, and both main pilots get ill and can’t pilot the aircraft. You are an experienced pilot of small aeroplanes, so you decide to pilot the huge A380 yourself.

Now, what do you do?

Ask for help to the people in the airport to guide you with the aeroplane details and how to pilot it
RTFM, even if you know it may take some time to learn it
Fuck everything! You are a pilot! You have done it tons of times! This big A380 shouldn’t be too different, right?

Oh, alright, you chose last option, hum? Then, as it is normal, your experience with the small aeroplanes has nothing to do with the A380. Even if at the end the result is the same (an aeroplane flies), they are different in nature. Engines are different, power is different, size is different, everything is different. So the plane starts to go down, and you continue trying different things… you press this button, then that one, oh shit… it’s going down… then you push that lever, nothing, oh maybe that red button there…
Shit it doesn’t work! Do I have a manual? Oh yes, there it is… ok… where is the chapter “how to start the propellers”? Humm.. where is it? Humm… Can’t find it!!! How is this possible! There is no chapter where it explains how to start the propellers!!?

Nice, so you realized it doesn’t work the way you thought it should (A380 doesn’t have propellers, among others). Again, what do you do?

Ask for help to the people in the airport to guide you with the aeroplane details and how to pilot it
RTFM carefully, not only looking for the chapter explaining how to start the propellers, as you already noticed that maybe the A380 doesn’t have propellers
Blame every engineer who built the aircraft
Blame not only every engineer who built it, but also the one who wrote the manual
Blame every engineer who built it, the one who wrote the manual AND also say that your small aeroplane is also much much better than this A380 you don’t understand

How will this end? Of course, not well.

What’s the moral in this? If you don’t know how to do something, even if you think you know it, better ask someone or read the manual carefully.

Now, why this post? Because of this one: Having fun with bzr

And yes I know that Git is not a small aeroplane and neither GNU Bazaar a huge A380. But no one can say that Git is easier to understand than Bazaar. You just need to understand that even if both are Distributed VCS, their way to work is different, their interface is different… in a brief, they are completely different VCS.

P.S.: The command you were looking for so desperately was “branch”, neither “get” nor “waldo” nor any other thing. A380 doesn’t have propellers! Don’t look for how to start them in the manual!

P.P.S.: Also, the URL of the repository you were using was probably not the good one: http://www.mega-nerd.com/Bzr/libsndfile-pub/ doesn’t exist (404 Not Found). A good choice would have been to go to an upper directory and look for the existing URLs… http://www.mega-nerd.com/Bzr

Posted in Development Tagged: Git, Gnu Bazaar, VCS

Debian 5.0 “lenny” is now stable

aleksander — Sun, 22 Feb 2009 13:09:04 +0000

Some of you may be used to the every 6-months release of a new stable version of Ubuntu GNU/Linux [1]. That’s quite a hard strategy to follow, as it seems that the important thing is to have the release out, even if not as perfect as desired. I see this approach as coming from the world of proprietary software companies, where the important thing is to reach the release date no matter what, so that the commercial department doesn’t need to re-print the leaflets or re-design the web page to change the release date of the product. This can lead into very very wrong decisions and situations: for example, shipping KDE4 in Kubuntu 8.10 even if the release of KDE4 shipped was lacking of lots of things (and then we have to read things like these [2] to justify the decision!!).

Debian GNU/Linux [3] uses a complete different approach. There are always 3 main version of Debian available at any time: stable, testing and unstable [4]. The stable version contains what the developers consider is a real stable version. The testing version contains usually latest releases of the upstream packages, and this version will be the one converting itself into stable in the next step. The unstable release contains all the testing-pending packages submitted by the Debian developers. In this approach, you always know that stability is assured in every stable release of Debian.

Lenny, from Toy Story

This is the main reason why a new stable version of Debian, like the Debian 5.0 codenamed lenny [5] is always a special event: it happens only when it must happen, and never before.

References:

Posted in Operating Systems Tagged: debian, gnu/linux, kde4, kubuntu, ubuntu

Avoid G_TYPE_INSTANCE_GET_PRIVATE() in GObjects

aleksander — Fri, 13 Feb 2009 12:33:49 +0000

When developing GObjects in Glib/GObject framework, it is usual to have a structure defined in the source file with all the private members of the object. Doing this, the internals of the object are not published in its API, so that no other module outside can modify them without using the defined GObject methods.

There are several ways to achieve this, and one of them is the recommended one in GObject’s tutorial [1], which uses the G_TYPE_INSTANCE_GET_PRIVATE [2] macro to get the glib-instantiated private structure. This private structure, allocated for each instance of the object, is specified in the Class initialization function with g_type_class_add_private(), and allocated every time a new GObject instance is created.

Usually, you will have a macro for your specific GObject which will call G_TYPE_INSTANCE_GET_PRIVATE:

#define MY_GOBJECT_GET_PRIVATE(o) \
    (G_TYPE_INSTANCE_GET_PRIVATE((o), MY_GOBJECT_TYPE, MyGObjectPrivate))

This seems a good way of doing it, and quite simple to use. Glib will take care of allocating and deallocating that private structure for us… so why avoid it?

Well… if you ever tried to run Valgrind’s callgrind tool to measure how much time your program spends in a given function, you will see the reason. Other people already did that and published results on the web, so go and check them [3] or just try it with your application:

$> valgrind --tool=callgrind /usr/local/bin/your_program


Now, an easy way of getting the same result, with a little bit more of work but achieving the best performance, is just using an opaque pointer in the GObject’s public structure and define, allocate and deallocate it yourself in the source file (so also can be treated as private, as the users of the API don’t know the internals of the structure).
These are the basic changes you need to do to avoid calling G_TYPE_INSTANCE_GET_PRIVATE():
**** In the HEADER of the GObject, when the struct defining the Object is specified, add a “gpointer priv” variable.
struct _MyGObject
{
    /** Parent object */
    GObject parent;
    /** Private data pointer */
    gpointer priv;
}


**** Then, in the SOURCE of the GObject, modify the GET_PRIVATE macro so that instead of doing the standard GObject lookup for the correct type, we just get the opaque “priv” pointer defined in the header, and we cast it to the correct type.
#define MY_GOBJECT_GET_PRIVATE(o) \
    ((MyGObjectPrivate *)((MY_GOBJECT(o))->priv))


**** As we won’t use the automatic allocation of the Private data structure, we need to allocate it ourselves in the _init() function.
static void
my_gobject_init(MyGObject *self)
{
    /* Allocate Private data structure */
    (MY_GOBJECT(self))->priv = \
        (MyGObjectPrivate *) g_malloc0(sizeof(MyGObjectPrivate));
    /* If correctly allocated, initialize parameters */
    if((MY_GOBJECT(self))->priv != NULL)
    {
        MyGObjectPrivate *priv = MY_GOBJECT_GET_PRIVATE(self);
        /* Initialize private data, if any */
    }
}


**** And finally, the last change is just de-allocating the structure when no more needed in the _dispose() function.
static void
my_gobject_dispose(GObject *object)
{
    MyGObject *self = (MyGObject *)object;
    MyGObjectPrivate *priv = MY_GOBJECT_GET_PRIVATE(self);
    /* Check if not NULL! To avoid calling dispose multiple times */
    if(priv != NULL)
    {
        /* Deallocate contents of the private data, if any */
        /* Deallocate private data structure */
        g_free(priv);
        /* And finally set the opaque pointer back to NULL, so that
         *  we don't deallocate it twice. */
        (MY_GOBJECT(self))->priv = NULL;
    }
}


References:

[1] http://library.gnome.org/devel/gobject/stable/howto-gobject.html#howto-gobject-header
[2] http://library.gnome.org/devel/gobject/stable/gobject-Type-Information.html#G-TYPE-INSTANCE-GET-PRIVATE–CAPS
[3] http://bugzilla.gnome.org/show_bug.cgi?id=322174


Posted in Development Tagged: glib, gnome, gobject, valgrind

Happy 1234567890s!

aleksander — Sun, 08 Feb 2009 20:12:07 +0000

On February 14th 2009 (Valentine’s day), 00:31:30 CET, the Unix time (seconds elapsed since 00:00 UTC January 1st 2009) will arrive to the “1234567890′ value.

So happy 1234567890s!

You can use the `date’ command from GNU coreutils to know when will it happen in your specific timezone:

$> date -d @1234567890

Posted in Operating Systems Tagged: coreutils, date, gnu, unix

Please, forget it as soon as possible

aleksander — Mon, 01 Dec 2008 19:36:57 +0000

New development version of Glib just released.

GLib 2.19.1 is now available for download at:

ftp://ftp.gtk.org/pub/glib/2.19/

http://download.gnome.org/sources/glib/2.19/

glib-2.19.1.tar.bz2 md5sum: 9309139a515408d9c99558d051c18302
glib-2.19.1.tar.gz md5sum: eab90f5965f3d004b8787989be5cca85

This is the second development release leading up to GLib 2.20.

It’s great they include the changelog with fixed bugs:

* Bugs fixed:
562538 GObject interface tutorial shouldn’t finalise with
“Please forget everything”

This one is great (among others, including multiple mem-leaks fixed).

Until this last release, the GObject tutorial had the following comment at the end of the “Non-instantiable classed types: interfaces” section:

“Now that you have read this section, you can forget about it. Please, forget it as soon as possible. “

What I am not sure about is: did they remove it because now interfaces are great? or just to skip publishing the idea that even GObject developers think that interfaces suck? Who knows…

Previous documentation:
http://library.gnome.org/devel/gobject/2.18/gtype-non-instantiable-classed.html
Latest documentation:
http://library.gnome.org/devel/gobject/unstable/gtype-non-instantiable-classed.html

Posted in Development Tagged: glib, gnome, gobject

What’s next? Buy a new PC with Windows already broken

aleksander — Tue, 25 Nov 2008 20:23:44 +0000

Broken Windows in new PC

I was looking for a new PC to be used as a server at home (running GNU/Linux, of course), when I discovered this nice Packard Bell laptop with pre-installed and pre-broken Windows.

This is really nice. Why wait to break the OS yourself when you can already buy it broken? And it’s only 699€! I want three of this!

Posted in Operating Systems Tagged: Microsoft Windows

Valgrind for Mac OS X – first impressions

aleksander — Tue, 30 Sep 2008 20:31:00 +0000

Not so long ago, Apple developer Greg Parker, published a patch for a specific revision of Valgrind so that it can work in Mac OS X. You can check the details of how to install it in his webpage, it doesn’t take more than 5 minutes to do it:

http://www.sealiesoftware.com/valgrind

I made a simple test to see how well or bad it works, with definitely lost and still reachable memory leaks. The test goes as follows:
#include #include /* Global string, still-reachable */ static char *global_str; int main(int argc, char **argv) { /* Function-local string, * definitely-lost */ char *str = NULL; /* These are 10+1 bytes leaked */ str = strdup("1234567890"); /* These are 15+1 bytes leaked */ global_str = strdup("123456789012345"); return 0; }

We compile it…
$> gcc -o test test.c

Now, let’s see if this valgrind port catches our memory leaks.
$> valgrind --show-reachable=yes --leak-check=full --leak-resolution=high ./test

Well, valgrind runs and we start checking its output:
==13340== LEAK SUMMARY: ==13340== definitely lost: 11 bytes in 1 blocks. ==13340== possibly lost: 0 bytes in 0 blocks. ==13340== still reachable: 316 bytes in 8 blocks. ==13340== suppressed: 0 bytes in 0 blocks.

We were only expecting 11 bytes definitely lost (local-function pointer) and 16 bytes still reachable (global pointer), but we have lots more still reachable. Let’s see the detailed results:
==13340== ERROR ==13340== 11 bytes in 1 blocks are definitely lost in loss record 1 of 4 ==13340== at 0x22E53: malloc+99 (in /usr/local/lib/valgrind/x86-darwin/vgpreload_memcheck.so) ==13340== by 0x21A3D1: strdup+33 (in /usr/lib/libSystem.B.dylib) ==13340== by 0x1FBB: main+33 (in ./test) ==13340== ==13340== ERROR ==13340== 16 bytes in 1 blocks are still reachable in loss record 2 of 4 ==13340== at 0x22E53: malloc+99 (in /usr/local/lib/valgrind/x86-darwin/vgpreload_memcheck.so) ==13340== by 0x21A3D1: strdup+33 (in /usr/lib/libSystem.B.dylib) ==13340== by 0x1FCC: main+50 (in ./test)

Ok, here are our memory leaks. Let’s see what is the other stuff.
==13340== ERROR ==13340== 60 bytes in 1 blocks are still reachable in loss record 3 of 4 ==13340== at 0x22E53: malloc+99 (in /usr/local/lib/valgrind/x86-darwin/vgpreload_memcheck.so) ==13340== by 0x1F7E89: get_or_create_key_element+157 (in /usr/lib/libSystem.B.dylib) ==13340== by 0x1F7DBB: _keymgr_get_and_lock_processwide_ptr_2+21 (in /usr/lib/libSystem.B.dylib) (.....) ==13340== ERROR ==13340== 240 bytes in 6 blocks are still reachable in loss record 4 of 4 ==13340== at 0x24DD9: calloc+105 (in /usr/local/lib/valgrind/x86-darwin/vgpreload_memcheck.so) ==13340== by 0x1F79D9: dwarf2_unwind_dyld_add_image_hook+40 (in /usr/lib/libSystem.B.dylib) ==13340== by 0x8FE03D61: dyld::registerAddCallback(void (*)(mach_header const*, long))+145 (in /usr/lib/dyld) (.....)

As expected (Greg already warned about it) we are getting some leak reports of system libraries, as being still reachable. The “still reachable” means that when the program ended some allocations were done previously using pointers that are ‘global’ to the program, so still reachable when program ended. In other words, valgrind found pointers to allocated memory in the context when the program ended, and the only context available in that time is the global one. All the other contexts (different functions called, even main()) dissappear when the program ends, and pointers declared in those functions which are used to allocate memory are the “definitely lost” ones.

Well, I would say that there is still work to be done, but I don’t see having these system library allocation leak reports a real problem. Once you have linked your programs against Glib, you will get used to see this kind of error reports, but multiplied by 1000. Glib does lots of memory allocations in g_type_init() that are not deallocated with a g_type_deinit() (there is no such function), so they are deallocated by the OS when the program ends. To deal with the ones from Glib, it’s usual to skip using
leak-resolution=high and use Valgrind suppression files.

Probably these reported errors are not comparable to the ones from Glib, but the way to deal with them should be really similar. So, good job Greg! and let’s see if this port can be improved until it’s completely included in Valgrind sources.

Posted in Development, Operating Systems Tagged: glib, mac os x, valgrind

It’s all about Freedom

aleksander — Mon, 29 Sep 2008 21:19:27 +0000

I am quite a GNU/Linux fan. I work developing apps for GNU systems, I collaborate with the development of some GNU projects, I mainly use free software applications… but last year I bought a Mac Book Pro. My first impression of Mac OS X was great. It’s really a great operating system, it has a nice terminal, great support for GNU tools (gcc, gdb, autotools…), and multiple free software applications available. But, there’s always a but… the world of free software is full of nice apps, and not all those apps (or an equivalent replacement) are available for Mac OS X.

I already have a dual boot in my Mac Book Pro, with Kubuntu GNU/Linux available in a 15GB partition. When I installed it, it was basically because most of the programs I was using for development were not available in Mac OS X (valgrind, electric fence, meld, those are essential!). Then, I realized that I was going back again to the years when I had a dual boot with Windows XP and GNU/Linux. In those years, I needed Windows XP to play games, and for nothing else. So here I am again in th same situation. But… wait… the question is… why do I need Mac OS X now?

This is the list of programs I normally use which are available in both Mac OS X and GNU/Linux:

GNU emacs
GNU compiler collection (gcc)
GNU debugger (gdb)
Mozilla Firefox
Mozilla Thunderbird
VLC
GNU image manipulation program (gimp)
CVS, Subversion & GNU Bazaar
OpenVPN
glib

I also use other programs in Mac OS X which have a perfect replacement in GNU/Linux:

iTunes -> Amarok, which is even better, free software, and doesn’t support DRM [1]
NetNewsWire (free as in free beer)-> Akregator. An RSS reader, free as in freedom.
Colloquy -> Konversation. An IRC client.
Adium -> Kopete. An instant messenger.
Disk Utility -> K3b. Probably one of the bests CD burning programs.

This is the list of programs I use and which are only available in GNU/Linux:

Meld
Valgrind (there seems to be a non-complete port of Valgrind for Mac OS X. Didn’t check it) [2]
Electric fence
Yakuake

This is the list of programs I use and which are only available in Mac OS X:

This change back to GNU/Linux could be understood as just based on my needs at a given moment, which in part is true, but I also want to use a Free Software operating system, and Free Software programs. At the end, it’s all about Freedom [3].

References

Posted in Operating Systems Tagged: gnu, gnu/linux, mac os x

Single dot emacs file and per-computer configuration

aleksander — Sun, 28 Sep 2008 13:59:50 +0000

1. Introduction

Once you have found the perfect emacs configuration stuff for you, you don’t usually want to change it. But it can happen that you want to use the same dot emacs configuration in different computers or even different systems.

I use emacs at home (Mac OS X and Ubuntu) and also at work (Kubuntu). It happens that when I write ChangeLog entries (C-x 4 a) I usually want to have different email addresses in the entries (my work email in the computers I use at work, and my GNU email in the computers I use at home). In the same way, I have a specific font type configuration for my GNU systems, but I keep the default one in my Mac O X emacs.

So basically you just need to detect different systems and computers, and apply specific configurations.

2. Get system type and system name

To detect different systems, you can use the ‘system-type‘ variable, and to detect different computers (with different hostnames, of course), you can use the ‘system-name‘ variable.

The following defuns show how you can know wich system type and name you have in each machine:

;; Get current system's name (defun insert-system-name() (interactive) "Get current system's name" (insert (format "%s" system-name)) )

;; Get current system type (defun insert-system-type() (interactive) "Get current system type" (insert (format "%s" system-type)) )

Those previous commands are interactive, so you can just type ‘M-x insert-system-name‘ or ‘M-x insert-system-type‘ to test them.

3. Configuration depending on the system type

In order to include system-type based configurations, you can define functions to check if the current system is the one you are looking for. Check these two functions to check if system is Mac OS X (darwin) or GNU (gnu/linux).

;; Check if system is Darwin/Mac OS X (defun system-type-is-darwin () (interactive) "Return true if system is darwin-based (Mac OS X)" (string-equal system-type "darwin") )

;; Check if system is GNU/Linux (defun system-type-is-gnu () (interactive) "Return true if system is GNU/Linux-based" (string-equal system-type "gnu/linux") )

Once you have these functions, you can put the system-dependent configuration:

;; disable closing emacs in Mac OS X (if (system-type-is-darwin) (global-unset-key "\C-z") )

Note that ‘darwin‘ is the default system name for Mac OS X systems, and ‘gnu/linux‘ is the default system name for GNU/Linux systems.

3. Configuration depending on the system name

Now, you also want to put machine-dependent configuration, so you can define functions like this to detect in which machine you are currently located:

;; Check if the system is my Kubuntu GNU/Linux at work (defun system-is-my-workpc () (interactive) "Return true if the system we are running on is my PC at work" (string-equal system-name "dev002.workplace.com") )

And then you just need to use the previous function in the specific configurations (note that I’m using the hostname of my PC at work. Use your own hostname for you functions, dude).

;; For my machine in my worplace, setup my work email address (if (system-is-my-workpc) (setq user-mail-address "my.name@workplace.com") )

Posted in GNU Emacs Tagged: dot emacs, emacs, gnu

Debugging programs with a external symbol file

aleksander — Mon, 15 Sep 2008 16:42:33 +0000

As explained in one of my previous posts, objcopy utility can be used to store the debugging information generated by gcc compilation in an external file. This is very useful when you want to debug a program using a core file generated by the stripped binary.

Imagine that you’re a developer in a company which distributes program binaries which are stripped before being delivered to the end users (you probably don’t need too much imagination for this). Of course, that evil company doesn’t distribute the source code of the program.

One of your users experiences a problem with your program. You tell him to run:
ulimit -c unlimited
This will tell the system that the size allowed for the core files generated is unlimited. These core files are generated when some unhandled signals are received in the program, like SIGSEGV when a segmentation fault occurs, and include a complete dump of the program’s memory, program stack and such when the crash occured.

If you don’t get a core file after a segmentation fault and you already executed the ulimit command, it may be due to different reasons, like these:

Kernel limitation (there’s a specific maximum size for the core files)
Available disk space (you don’t have enough disk space to store your core file).
…

If you got a complete valid core file after a program crash, you may want to debug it with the GNU Debugger gdb. Of course, you don’t want the user of the program to know the internals of the code (remember that the company is evil and doesn’t distribute the source code), so in order to debug the core file generated by the user’s execution, you need your symbol file for that specific compilation of the program.

gdb ./myprogram corefile

Once you have started the GNU debugger with the previous command, you are able to investigate what happened in the program abort. If you don’t have debugging symbols in the binary, you will get a backtrace equivalent to this one:
Core was generated by `./myprogram'. Program terminated with signal 11, Segmentation fault. [New process 8986] #0 0x080483dd in myfunction () (gdb) bt #0 0x080483dd in myfunction () #1 0x080483d1 in myfunction () #2 0x080483d1 in myfunction () #3 0x080483d1 in myfunction () #4 0x080483d1 in myfunction () #5 0x080483d1 in myfunction () #6 0x080483d1 in myfunction () #7 0x080483d1 in myfunction () #8 0x080483d1 in myfunction () #9 0x080483d1 in myfunction () #10 0x080483d1 in myfunction () #11 0x0804840e in main ()

Not really useful isn’t it? That’s the maximum information your user will know about the problem.

But you are the developer and when you compiled the program, and before you redistributed it, you stripped the binary and stored the debugging information in an external symbol file. So you just need to tell gdb where that file is:
(gdb) symbol-file myprogram.debug Reading symbols from /home/drehbahn/myprogram.debug...done.

And magic happens. Now you can get a pretty backtrace of the program stack:
(gdb) bt #0 0x080483dd in myfunction (value=10) at myprogram.c:26 #1 0x080483d1 in myfunction (value=9) at myprogram.c:20 #2 0x080483d1 in myfunction (value=8) at myprogram.c:20 #3 0x080483d1 in myfunction (value=7) at myprogram.c:20 #4 0x080483d1 in myfunction (value=6) at myprogram.c:20 #5 0x080483d1 in myfunction (value=5) at myprogram.c:20 #6 0x080483d1 in myfunction (value=4) at myprogram.c:20 #7 0x080483d1 in myfunction (value=3) at myprogram.c:20 #8 0x080483d1 in myfunction (value=2) at myprogram.c:20 #9 0x080483d1 in myfunction (value=1) at myprogram.c:20 #10 0x080483d1 in myfunction (value=0) at myprogram.c:20 #11 0x0804840e in main () at myprogram.c:33

And you can step into the first frame to see the real details of the problem which caused the segfault:
(gdb) fr 0 #0 0x080483dd in myfunction (value=10) at myprogram.c:26 26 char character = *ptr; /* oops */ (gdb) list 21 } 22 else 23 { 24 /* Create nice segfault... */ 25 char *ptr = NULL; 26 char character = *ptr; /* oops */ 27 printf("Did I really arrive here?\n"); 28 } 29 } 30

So, yes, it’s easy to debug a program with a separate symbol file created with objcopy.

And even better, you don’t need to be an evil company which doesn’t publish the source code of their apps to use this approach. Stripping binaries before shipping the programs is quite common in the world of Free Software, as size of the binaries really matters. The difference is that in this case you can always download not only the original source code that was used to generate the binary, but also the specific symbol files for direct debugging with the GNU Debugger.

[References]

[See also]

Make your programs lose weight: Stripping binaries

Spanish MBP key modifiers for GNU emacs (dot emacs)

aleksander — Sun, 14 Sep 2008 14:47:43 +0000

GNU Emacs (native version for Mac OS X) needs extra key bindings when used in a spanish keyboard Mac Book Pro. The following lines show how the main needed key bindings can be set up.

;; Key modifiers for Mac OS X Emacs.app in spanish MBP keyboard (global-set-key (kbd "M-1") "|") (global-set-key (kbd "M-2") "@") (global-set-key (kbd "M-3") "#") (global-set-key (kbd "M-º") "\\") (global-set-key (kbd "M-ç") "}") (global-set-key (kbd "M-+") "]")

Insert date string (dot emacs)

aleksander — Sun, 14 Sep 2008 14:33:31 +0000

Need to add a common date string in your source code?

Add it to your .emacs!

;; Insert date string (defun insert-date-string () "Insert a nicely formated date string." (interactive) (insert (format-time-string "%a %b %d %H:%M:%S %Y")))

Make your programs lose weight: stripping binaries

aleksander — Sun, 14 Sep 2008 12:16:35 +0000

`objcopy‘ is one of the utilities included in the GNU Binutils package. This tool allows copying binary files while transforming them in the process.

One of the typical purposes of modifying binary files after they have been compiled is to separate the debugging symbols and sections and the real executable. If you compiled you binary with debugging information (option ‘-g’ in gcc), the size of the binary will grow a lot due to the extra symbols, sections and human-readable stuff stored within the binary. That information is really useful when you are debugging your app, anyway, so you don’t really want to completely get rid of it.

So, we can use objcopy to move all the debugging information from the binary to another file, so that:

Binary is smaller
Binary does not include any ‘human-readable’ information of the program
Debugging information can be kept by the developers, so that if any ‘core’ file is received for that specific binary, debugging with GDB is possible.

[Step 1] Copy the binary to another file, keeping only the debugging information. This will create the symbol file for the binary, and will be kept by the developers of the program for debugging purposes:
objcopy --only-keep-debug $BINARY_PATH $DEBUG_INFO_FILE

[Step 2] Once you have created the symbol file, you can now remove all the debugging information from the binary itself:
objcopy --strip-debug $BINARY_PATH

[Step 3] After you have created both files, an extra optional option is to add a debug-link to the binary, to specify which is the exact file containing the debug information for that binary:
objcopy --add-gnu-debuglink=$DEBUG_INFO_FILE $BINARY_PATH
Please note that this third step is optional, as we can use specific GDB commands when debugging a core file to specify in which file the debug information is located.

[References]