Category Archives: FreeDesktop Planet

QMI/Gobi management in the kernel: qmi_wwan or GobiNet?

videotape

Introduction

Gobi chipsets are mobile broadband modems developed by Qualcomm, and they are nowadays used by lots of different manufacturers, including Sierra Wireless, ZTE, Huawei and of course Qualcomm themselves.

These devices will usually expose several interfaces in the USB layer, and each interface will then be published to userspace as different ‘ports’ (not the correct name, but I guess easier to understand). Some of the interfaces wil give access to serial ports (e.g. ttys) in the modem, which will let users execute standard connection procedures using the AT protocol and a PPP session. The main problem with using a PPP session over a serial port is that it makes it very difficult, if not totally impossible, to handle datarates above 3G, like LTE. So, in addition to these serial ports, Gobi modems also provide access to a control port (speaking the QMI protocol) and a network interface (think of it as a standard ethernet interface). The connection procedure then can be executed purely through QMI (e.g. providing APN, authentication…) and then userspace can use a much more convenient network interface for the real data communication.

For a long time, the only way to use such QMI+net pair in the Linux kernel was to use the out-of-tree GobiNet drivers provided by Qualcomm or by other manufacturers, along with user-space tools also developed by them (some of them free/open, some of them proprietary). Luckily, a couple of years ago a new qmi_wwan driver was developed by Bjørn Mork and merged into the upstream kernel. This new driver provided access to both the QMI port and the network interface, but was much simpler than the original GobiNet one. The scope was reduced so much, that most of the work that the GobiNet driver was doing in kernel-space, now it had to be done by userspace applications. There are now at least 3 different user-space implementations allowing to use QMI devices through the qmi_wwan port: ofono, uqmi and of course, libqmi.

The question, though, still remains. What should I use? The upstream qmi_wwan kernel driver and user-space utilities like libqmi? Or rather, the out-of-tree GobiNet driver and user-space utilities provided by manufacturers? I’m probably totally biased, but I’ll try to compare the two approaches by pointing out their main differences.

Note: you may want to read the ‘Introduction to libqmi‘ post I wrote a while ago first.

in-tree vs out-of-tree

The qmi_wwan driver is maintained within the upstream Linux kernel (in-tree). This, alone, is a huge advantage compared to GobiNet. Kernel updates may modify the internal interfaces they expose for the different drivers, and being within the same sources as all the other ones, the qmi_wwan driver will also get those updates without further effort. Whenever you install a kernel, you know you’ll have the qmi_wwan driver applicable to that same kernel version ready, so its use is very straightforward. The qmi_wwan driver also contains support for Gobi-based devices from all vendors, so regardless of whether you have a Sierra Wireless modem or a Huawei one (just to name a few), the driver will be able to make your device work as expected in the kernel.

GobiNet is a whole different story. There is not just one GobiNet: each manufacturer keeps its own. If you’re using a Sierra Wireless device you’ll likely want to use the GobiNet driver maintained by them, so that for example, the specific VID/PID pairs are already included in the driver; or going a bit deeper, so that the driver knows which is supposed to be the QMI/WWAN interface number that should be used (different vendors have different USB interface layouts). In addition to the problem of requiring to look for the GobiNet driver most suitable for your device, having the drivers maintained out-of-tree means that they need to provide a single set of sources for a very long set of kernel versions. The sources, therefore, are full of #ifdefs enabling/disabling different code paths depending on the kernel version targeted, so maintaining it gets to be much more complicated than if they just had it in-tree.

Note: Interestingly, we’ve already seen fixes that were first implemented in qmi_wwan ‘ported’ to GobiNet variants.

Complexity

The qmi_wwan driver is simple; it will just get a USB interface and split it into a QMI-capable /dev/cdc-wdm port (through the cdc-wdm driver) and a wwan network interface. As the kernel only provides basic transport to and from the device, it is left to user-space the need to manage the QMI protocol completely, including service client allocations/releases as well as the whole internal CTL service. Note, though, that this is not a problem; user-space tools like libqmi will do this work nicely.

The GobiNet driver is instead very complex. The driver also exposes a control interface (e.g. /dev/qcqmi) and a network interface, but all the work that is done through the internal CTL service is done at kernel-level. So all client allocations/releases for the different services are actually performed internally, not exposed to user-space. Users will just be able to request client allocations via ioctl() calls, and client releases will be automatically managed within the kernel. In general, it is never advisable to have such a complex driver. As complexity of a driver increases, so does the likelyhood of having errors, and crashes in a driver could affect the whole kernel. Quoting Bjørn, the smaller the device driver is, the more robust the system is.

Note: Some Android devices also support QMI-capable chipsets through GobiNet (everything hidden in the kernel and the RIL). In this case, though, you may see that shared memory can also be used to talk to the QMI device, instead of a /dev/qcqmi port.

Device initialization

One of the first tasks that is done while communicating with the Gobi device is to set it up (e.g. decide which link-layer protocol to use in the network interface) and make sure that the modem is ready to talk QMI. In the case of the GobiNet driver, this is all done in kernel-space; while in the case of qmi_wwan everything can be managed in user-space. The libqmi library allows several actions to be performed during device initialization, including the setting of the link-layer protocol to use. There are, for example, models from Sierra Wireless (like the new MC7305) which expose by default one QMI+network interface (#8) configured to use 802.3 (ethernet headers) and another QMI+network interface (#10) configured to use raw IP (no ethernet headers). With libqmi, we can switch the second one to use 802.3, which is what qmi_wwan expects, thus allowing us to use both QMI+net pairs at the same time.

Multiple processes talking QMI

One of the problems of qmi_wwan is that only one process is capable of using the control port at a given time. The GobiNet driver, instead, allows multiple processes to concurrently access the device, as each process would get assigned different QMI clients with different client IDs directly from the kernel, hence, not interfering with each other. In order to handle this issue, libqmi (since version 1.8) was extended to implement a ‘qmi-proxy’ process which would be the only one accessing the QMI port, but which would allow different process to communicate with the device concurrently (by sharing and synchronizing the CTL service among the connected peers).

User-space libraries

The GobiNet driver is designed to be used along with Qualcomm’s C++ GobiAPI library in user-space. On top of this library, other manufacturers (like Sierra Wireless) provide additional libraries to use specific features of their devices. This GobiAPI library will handle itself all the ioctl() calls required to e.g. allocate new clients, and will also provide a high level API to access the different QMI services and operations in the device.

In the case of the qmi_wwan driver, as already said, there are several implementations which will let you talk QMI with the device. libqmi, which I maintain, is one of them. libqmi provides a GLib-based C library, and therefore it exposes objects and interfaces which provide access to the most used QMI services in any kind of device. The CTL service, the internal one which was managed in the kernel by GobiNet, will be managed internally by libqmi and therefore mostly hidden to the users of the library.

Note: It is not (yet) possible to mix GobiAPI with qmi_wwan and e.g. libqmi with GobiNet. Therefore, it is not (yet) possible to use libqmi or qmicli in e.g. an Android device with a QMI-capable chipset.

User-space command line tools

I am no really aware of any general purpose command line tool developed to be used with the GobiNet driver (well, firmware loader applications, but those are not general purpose). The lack of command line tools may be likely due to the fact that, as QMI clients are released automatically by the GobiNet kernel, it is not easy (if at all possible) to leave a QMI client allocated and re-use it over and over by a command line tool which executes an action and exits.

With qmi_wwan, though, as clients are not automatically released, command line tools are much easier to handle. The libqmi project includes a qmicli tool which is able to execute independent QMI requests in each run of the program, even re-using the same QMI client in each of the runs if needed. This is especially important when launching a connection, as the WDS client which executes the “Start Network” command must be kept registered as long as the connection is open, or otherwise the connection will get dropped.

New firmware loading

The process of loading new firmware into a QMI-based device is not straightforward. It involves several interactions at QMI-level, plus a QDL based download of the firware to the device (kind of what gobi_loader does for Gobi 2K). Sadly, there is not yet a way to perform this operation when using qmi_wwan and its user-space tools. If you’re in the need of updating the firmware of the device, the only choice left is to use the GobiNet driver plus the vendor-provided programs.

Support

One of the advantages of the GobiNet driver is that every manufacturer will (should) give direct support for their devices if that kernel driver is used. Actually, there are vendors which will only give support for the hardware if their driver is the one in use. I’m therefore assuming that GobiNet may be a good choice for companies if they want to rely in the vendor-provided support, but likely not for standard users which just happen to have a device of this kind in their systems.

But, even if it is not the official support, you can anyway still get in touch with the libqmi mailing list if you’re experiencing issues with your QMI device; or contact companies or individuals (e.g. me!) which provide commercial support for the qmi_wwan driver and libqmi/qmicli integration needs.

GNOME Asia 2014

I spent last weekend in Beijing attending GNOME Asia 2014; yeah, long trip from Europe just for 3 days, but it was totally worth it. The worst part of it was of course fighting jet lag when I arrived, and fighting it again 3 days later when I came back to Spain :) The conference was really well organized [1], so kudos to all the local team!

After a quick sleep on Friday morning, I attended the development and documentation training sessions that Kat, André and Dave gave. They were quite interesting, especially since I’m not involved in the real user documentation that GNOME provides. I have to say that these guys do an amazing job, not only teaching during conferences, but also through the whole year.

There are, from my point of view, two main ways of learning new things:

  • The ‘engineer’ way: Learning things as you need them, what you would do when you start writing an application and looking for examples of how to do what you want to do (autotools, anyone?). It is a very ‘engineer’ way, as you pick black boxes that you’ll use to build something bigger, while not fully understanding what the black box does inside.
  • The ‘scientific’ way: When you learn something in order to fully understand it and be able to teach others. This approach takes a lot longer, as you need to make sure that everything you learn is accurate and you end up questioning the things that are not clear enough. Learning stuff to teach others is actually what you do in University; you’re learning things that will afterwards need to be explained in an exam to someone who knows more about the subject than you do.

Sure, both ways have their ups and downs, but if you want to write software you need to be able to switch between those two mindsets constantly. You’ll use the engineer way when reading API docs, looking for the bits and pieces that you need to build your stuff. You’ll use the ‘scientific’ way when you need to start learning a new technology, or when you need more detail on how to do things. While the API docs are taken care of by the library developers, it is the documentation team the one making sure that user guides, tutorials, and other developer resources are kept up to date, which are definitely some of the toughest and most important tasks done to help newcomers and other developers. So go on, go learn GNOME technologies and teach others, join the documentation team! ;)

GNOME Asia is not a usual conference. If you have attended a Desktop Summit, GUADEC or FOSDEM before, all those conferences are built by developers and for developers. The focus of those conferences is usually not (explicitly) to attract newcomers, but instead to be a show of the latest and shiniest things happening in the projects. Of course we also had part of that in Beijing, say Lennart’s talk about the status of systemd or Allan’s talk about application bundles. Those both were very good talks, but likely too specific for most of the audience. Instead, I chose to talk about something more basic, focused on attracting newcomers wanting to write applications, and so I gave an Introduction to D-Bus talk, including some examples. It is the same talk I gave last year in GUADEC-ES, but in English this time (my Mandarin is not good enough).

I would like to thank the GNOME Foundation for sponsoring the flight to Beijing, and of course to all the local team who, as I already said, did an amazing job.

sponsored-badge-simple

[1] …except for the tea-less tea-breaks ;)

Profiles in Devhelp

I’m at the GNOME Development Experience Hackfest in Berlin, and one of the things that I wanted to target during these days was to keep on looking at how we can enable different profiles in Devhelp.

devhelp-profiles

As you probably know, Devhelp will show you the documentation of libraries installed in your system (usually only if you have the -devel or -docs package of the library installed). While this is already enough for most users, there is also the case where a developer wants to target a different version (older or newer) of the library than the one installed in the system.

A typical case for this is developing applications using GNOME’s jhbuild infrastructure, targeted either to a given GNOME release or to git master of the involved modules. In this case, if you want to use new methods of let’s say GTK+, you usually end up needing to fire up a web browser and looking for the latest GTK+ documentation either in developer.gnome.org or in your jhbuild’s ${prefix}/share/gtk-doc/html directory.

In order to avoid this, I’m prototyping some ideas to let the users switch between different profiles, e.g.:

  • The ‘local’ profile, which is equivalent to what Devhelp currently shows.
  • A user-defined ‘jhbuild’ profile, which could point to the install prefix of the jhbuild setup.
  • Other user-defined profiles, which could point to other prefixes where the user has installed the newer (or older) libraries and their documentation.
  • Profiles for each new GNOME release, e.g. 3.12, which could get downloaded from developer.gnome.org as a tarball containing all documentation for a given release.

The most challenging case is probably the last one, given that it would require some extra work in the website in order to make sure the documentation tarball is generated and published in every new release, plus of course client-side management of these downloaded profiles in Devhelp.

For now this is just a basic set of ideas, the final result may or may not be similar; we’re of course open to suggestions!

sponsored-badge-simple

Follow

Get every new post delivered to your Inbox.

Join 57 other followers