Winter is over, Spring is here, and along with the heat, rains and insects, ModemManager arrives with support for modems using the new MBIM protocol, available through the new cdc-mbim kernel driver in Linux >= 3.8 and a preliminary version of libmbim.
The current implementation provides:
- PIN unlock/change/enable/disable capabilities
- 3GPP modem support (GSM, HSPA, LTE…)
- Basic connectivity support
It doesn’t provide yet most of the side-features available in other modems (e.g. SMS messaging, location…), but it is a good start. If you don’t want to use MBIM just yet, check Bjørn’s list of issues and alternatives.
Where do I get it?
The ‘libmbim’ library can be found in the following Lanedo-maintained gitorious repository:
We also released a tarball in libqmi’s release place:
If you’re part of the freedesktop.org sitewranglers group, please check the still open libmbim project request.
The support is already in ModemManager git master, and will be enabled if libmbim is found during configure. You can also explicitly request to use MBIM support with the new –with-mbim configure switch.
Special thanks go to Lanedo for sponsoring the initial libmbim development, Bjørn Mork for his suggestions and tests with multiple MBIM-capable modems, and Huawei for sending me a E367 with MBIM firmware.
So I wanted to have ModemManager in OpenWRT to handle my mobile broadband connections.
OpenWRT has its own built-in support to handle mobile broadband modems of different types, mainly for AT-controlled modems, but also for QMI-controlled modems through libqmi and qmicli (now available in upstream OpenWRT). But hell, why not package ModemManager for that work, and provide a unified single DBus API to control any kind of mobile broadband modem in the same way?
ModemManager itself isn’t a very complex daemon, so it doesn’t have many dependencies, just: glib/gio, dbus, udev/gudev and libqmi. In the world of OpenWRT, though, none of these are blessed dependencies, as they have a lightweight replacement for everything: libubox (instead of glib), ubus (instead of dbus) and hotplug2 (instead of udev). These replacements target minimal embedded systems, and are perfectly integrated into OpenWRT and provide the building blocks of other core software like the netifd network interface daemon.
Still, having ModemManager available as a OpenWRT package seems like a good idea if you’re not very constrained by the hardware you’re going to use. If you are building a system which should support multiple broadband modems from different vendors, or just building a base system for one specific modem model but leaving the door open to change the model in the future, ModemManager seems a good choice.
systemd’s udev in OpenWRT
Since April 2012, udev sources are now integrated within the systemd source code. And that means that udev packages in OpenWRT would need to get updated in order to get built from that new repository. systemd provides some hints on what would be the best way of creating a minimal systemd build which can be used to gather just the udev-specific libraries and binaries. The easiest way involves compiling the whole systemd package with all optional features disabled, and that would get us not only systemd (which we won’t need), but also udevd and libudev. Easy, yes, but then we have some issues in the environment that we would need to handle…
In OpenWRT, the udev package is available in the trunk repository, where only the core OpenWRT packages are given. But in order to get the new udev compiled out of systemd we end up needing additional dependencies (dbus, libcap) which are only available in the packages repository. Moreover, if you also want to compile the glib-based gudev library, glib is also needed, which is also only available in the packages repository. And you cannot require a package from trunk to depend on a package from packages, so… ended up creating a new systemd-udev package within the packages repository.
As I was targeting the uClibc toolchain, I also ended up requiring some additional patches both in uClibc (e.g. to enable utmpx) and systemd to get the thing properly compiled. The patches applied in the systemd sources are just to handle missing features in uClibc (execvpe, secure_getenv…). Whenever uClibc includes those, we’ll be able to remove those patches.
ModemManager in OpenWRT
I prepared an easy setup of ModemManager packaging in OpenWRT, where all possible plugins get compiled. It wouldn’t have been difficult to allow selecting via configuration which plugins to get compiled, but didn’t want to spend time on that.
I sent the patches to the OpenWRT mailing list already, but if you want to give it a quick try, you can just clone the Lanedo OpenWRT repositories (trunk and packages) from gitorious.org:
$> git clone git://gitorious.org/lanedo/openwrt.git
$> git clone git://gitorious.org/lanedo/openwrt-packages.git
You’ll want both ‘modemmanager-support‘ branches in those git repositories.
ModemManager will be available under the ‘Network’ section in the OpenWRT build configuration; but only after explicitly selecting the following kernel modules (all under Kernel Modules/USB support):
Once all above are enabled, ModemManager can be selected, and it will itself pull all its other required dependencies (systemd-udev, glib, dbus, libqmi…).
No LuCI? No connection manager?
Well, not yet . If you want to use ModemManager you still need to have your own connection manager software to manage the connectivity. ModemManager will get your mobile broadband modem connected, but you still need to either call pppd or setup the wwan interface yourself (depending on the data port type). This is really not a big deal; and you can just setup a shell script using the provided mmcli command line interface to talk to ModemManager.
Linux kernel 3.8 comes with a new ‘cdc-mbim‘ driver… let’s see what all this is about…
What is MBIM?
This new USB networking subclass defines two separate new features:
- A new MBIM USB device model, providing multiple IP connections over a single USB interface, and without the need of 802.3 frames (as was the case with ECM and NCM)
- A new MBIM control protocol to talk to modem devices
MBIM message types
The protocol defines different message types with different formats.
Some of them are used to establish the channel of communication with the modem:
- Open Message (Host->Modem): Initialization request.
- Open Done Message (Host<-Modem): Initialization response.
- Close Message (Host->Modem): Close request.
- Close Done Message (Host<-Modem): Close response.
Some of the messages are used to report errors in the protocol; which may be sent either from the host or from the modem:
- Host Error Message (Host->Modem): Host-reported error.
- Modem Error Message (Host<-Modem): Modem-reported error.
And finally, some messages provide access to the different CIDs (Command IDs) defined in each Service.
- Command Message (Host->Modem): Request of a given command.
- Command Done Message (Host<-Modem): Response to a given command.
- Indication Message (Host<-Modem): Unsolicited messages sent by the modem.
These last three messages support protocol-defined fragmentation. If the sender of the message finds out that the message is longer than the maximum control transfer size defined when the communication channel was opened (with an Open Message), it will be able to split it into sorted chunks and send them one by one over the wire. Each of these fragments specify the total amount of fragments expected, as well as the order in the sequence. The receiver of the fragments will therefore need to wait for all fragments to arrive – which should arrive in order – before processing the message.
The protocol defines a basic set of different Services:
- Basic Connect: Which provides the support for basic IP connectivity
- SMS: SMS messaging
- USSD: Unstructured Supplementary Service Data
- Phonebook: Handling contacts and such
- STK: SIM toolkit
- Device Service Stream
Only the Basic Connect one is mandatory in every MBIM device; others are optional. Vendors or even Network Operators can also extend the functionality of the device with other services; for example to support other protocols embedded within MBIM (e.g. QMI or AT within MBIM); or just to provide specific new features.
For each service, MBIM defines a set of Commands (CIDs); and each command can then be divided into 3 actions:
- “Set“: An user-requested action to change the modem state or configuration
- “Query“: An user-requested action to query the modem state or configuration
- “Notification“: An unsolicited report of the modem state or configuration
Not every action is supported by every command. For example, the “Device Caps” command in the “Basic Connect” service only supports the “Query” action, while the “Radio State” command of the same service supports all “Set“, “Query” and “Notification“.
But how does this match with the message types defined before?
- Host-created “Set” and “Query” requests are sent using “Command Messages“.
- Modem-created “Set” and “Query” responses are sent using “Command Done Messages“.
- Modem-created “Notifications” are sent using “Indication Messages“.
MBIM basic types
The protocol not only defines which action is supported in each command; it also defines the contents of the request, response and indication messages. The contents of each message are composed of collections of basic types defined by the protocol:
- Unsigned 32bit integers: Even for the most simple values (e.g. booleans), little-endian 32 bit unsigned integers are used.
- Strings: UTF-16LE encoded strings are always used.
- UUIDs: The protocol defines a special 16-byte-long UUID type, e.g. to define Service IDs
- Arrays: Collection of N values of a given basic type.
- Structs: Sequence of other basic types, given in a specific order.
As you can see, there is not much effort (none, actually) into making the protocol efficient in terms of overhead introduced; more than half of the bytes in each message will very likely end up being NUL bytes. Also, it is assured that each field within a message is aligned in a 32bit boundary, which makes it easier to read independent fields directly from the binary stream.
How does this compare to QMI?
There are quite some differences between this protocol and QMI. To name a few:
- Less basic types in MBIM, specially regarding signed/unsigned integers. QMI defines many more integer types, of different sizes and with different sign.
- Basic strings are given in ASCII in QMI
- Struct types are re-used in MBIM, as opposed to QMI, where each TLV would define the struct contents as it needed.
- There is no ‘client allocation’ needed in MBIM, i.e. the process opening the MBIM port doesn’t need to allocate clients for the different services.
- Fragmentation is built-in in the MBIM protocol; QMI required special TLVs and logic to handle messages that may be longer than the standard message size (e.g. when transferring the PRL list).
- Binary representation of the message is completely different. In QMI you would have a list of TLVs, one after the other. In MBIM, the contents of the message are split into two different sections: one which contains the fixed-size values and one for the variable-size values. For example, a string in MBIM is defined by two values in the fixed-size region (offset and size), while the real string data is within the variable-size section.
The ‘libmbim‘ library is an attempt to write a protocol support library, as previously done with ‘libqmi‘.
The current codebase, GLib/GObject/GIO based, is pretty similar to what libqmi provided, with a ‘MbimDevice‘ GObject to handle the communication through the
/dev/cdc-wdm port, as well as ‘MbimMessage‘ types to handle the creation of commands. Given that there is no need for client allocations in MBIM, there is no ‘MbimClient‘ object needed. Also, given that the contents of each message are pre-defined, it wasn’t considered the need of input and output ‘bundles’ to handle collections of TLVs, as done in libqmi.
Fragmentation of incoming/outgoing messages is handled internally by the ‘MbimDevice‘, which is a nice thing to have, as it simplifies the usage of the API. The user just creates a message, as long as it needs it to be, and the ‘MbimDevice‘ will take care of splitting it into fragments. When receiving a message, the user will also receive already the full message, once all fragments have been combined internally.
The messages and the services are defined in a JSON dictionary, and as with libqmi, all the message handling code is auto-generated from there. The capabilities of the mbim-codegen are still not in pair with those in qmi-codegen yet, though. The generator doesn’t support e.g. “Set” requests with parameters, or “Indication” messages. As previously said, this currently is a proof-of-concept, someday we’ll support all those properly.
The project comes with a command line utility (mbimcli), which allows (will allow) running “Get” or “Set” commands directly from the shell. There aren’t many supported commands yet, so I cannot show many more examples than this one:
$ sudo mbimcli \
-d /dev/cdc-wdm1 \
[/dev/cdc-wdm1] Subscriber ready status retrieved:
Ready state: 'device-locked'
Subscriber ID: 'unknown'
SIM ICCID: '984310311520086950F1'
Ready info: 'unknown'
Telephone numbers: ''
--verbose will show you the raw binary message contents!
Where do I get it?
The ‘libmbim’ library and the ‘mbimcli’ utility can be found in the following Lanedo-maintained gitorious repository:
There is already a request to host this project in the freedesktop.org infrastructure, so it may end up getting moved there at some point.