Blog Archives

Changing modes and capabilities in ModemManager

I don’t really recall when my first interaction with ModemManager was, I just remember saying “wait, this modem just works?”. But I do remember one day when I spent a couple of hours trying to understand why my modem wouldn’t switch to 2G-only mode even if I explicitly selected it in the network-manager-applet. Truth be told, I didn’t dig much in the issue that day; I barely knew what ModemManager was, or how it would interface with NetworkManager, or how to really debug it. And here I am possibly 4 years after that day, trying to get that same issue fixed.


ModemManager has always allowed to specify which ‘network type‘ to use; or, rather than network type, ‘allowed and preferred‘ modes as we name them. It basically is a way to tell your modem that you want to use one technology preferred over another (e.g. allow 2G and 3G, but prefer 3G), or even tell the modem to use only one technology type (e.g. 3G only). Even modern phones allow you to turn off 3G support and only use 2G, in order to save battery. The main problem with ModemManager’s way of handling this issue was that there was a predefined set of combinations to select (as exposed by the applet), and that not all combinations are supported by all modems. Even worse, ModemManager may not know how to use them, or the modem itself may not support mode switching at all (which was actually what was happening with my modem 4 years ago). Therefore, the UI would just try to show all the options and hope for the best when launching the mobile broadband connection. And there it comes the next issue; allowing to select mode preferences during the connection setup just makes the modem restart the whole radio stack, and the connection attempt may end up timing out, as the whole network registration process needs to be done from scratch before connecting…

Allowed and Preferred modes

In the new ModemManager interfaces, each Modem object will expose a “SupportedModes” property listing all the mode combinations (allowed + preferred) the modem actually supports. Graphical user interfaces will therefore be able to provide mode switching options listing only those combinations that will work. If a modem doesn’t support mode switching, no such list should be provided, and the user will not get confused. At any time, the Modem object will also expose a “CurrentModes” property, showing which is the currently selected combination of allowed and preferred modes. And the “SetCurrentModes()” method will allow to switch current modes, accepting as input only combinations which are given in “SupportedModes” (or the special allowed=ANY and preferred=NONE).

Also, changing current modes directly when calling Simple.Connect() will no longer be possible. This means that NetworkManager will never request allowed mode switching during a connection attempt (and hence no radio stack reloading in the modem causing timeouts). The logical place to put allowed mode switching is therefore a system configuration application like the GNOME Control Center or similar, which should allow mode switching at any time, not only just during a connection attempt. A good side effect of this change is that the NetworkManager connection settings now contain only connection-related configuration, which in the case of 3GPP devices can be linked to the SIM in use, leaving out all modem-specific configuration.


There was a time when modems were either 3GPP (GSM/GPRS/UMTS/HSPA…) or 3GPP2 (CDMA/EV-DO…). Nowadays, modems with multiple capabilities are pretty common, specially since LTE is around (LTE, even if 3GPP, is also 3GPP2’s blessed 4G technology, instead of the superhero named one which is forgotten by everyone already). ModemManager will now allow to change capabilities in addition to allowed and preferred modes; so a user with a modem which can work both in 3GPP and 3GPP2 networks will be able to switch from one to the other directly from the user interface. Of course, if the modem supports this (currently only QMI-based modems).

The new “SupportedCapabilities” property will expose all capability combinations supported by the modem, while “CurrentCapabilities” will expose which are the current ones being used at any given time. For example, a modem with “gsm-umts”, “cdma-evdo” and “lte” capabilities may support configuring only “cdma-evdo”, or “gsm-umts”+”lte”. Changing current capabilities is now possible through the “SetCurrentCapabilities()” method, which has a logic very similar to that of the “SetCurrentModes()” method. If a modem supports multiple capability combinations as exposed by “SupportedCapabilities”, this method will allow changing between them. The main difference with mode changing is that we will force a device power-cycle when this change is done, so the modem will disappear and reappear again with the new capabilities.

Capabilities and allowed/preferred modes have a lot in common, so much that there is a single interface in QMI based modems to change them. Therefore, when a modem allows changing capabilities, the list of allowed/preferred mode combinations may (and very likely will) be different depending on the current capabilities in the modem. For example, LTE-enabled QMI-powered modems will not be able to switch allowed/preferred modes when they have “lte” among the current capabilities, but they will be able if the capabilities are changed to only “gsm-umts”. This is not a big deal, as mode preference (e.g. 3G preferred) is not applicable when the modem does LTE (there is no way of saying allow 2G, 3G and 4G but prefer 3G).


ModemManager also allows to specify which frequency bands to use in the modem, but unlike with modes and capabilities, the “SupportedBands” property is not a list of all possible band combinations supported. Instead, it’s just a bitmask with all supported bands, without specifying whether an actual combination is going to work in “SetCurrentBands()” or not. Listing combinations instead of just the bitmask would be truly too much… But anyway, changing frequency bands is not a feature that a normal user should play with, so just don’t do it. I actually bricked a Pantech UML290 myself playing with this…


All these updates, plus some other ones, are available in the ‘aleksander/api-breaks‘ branch in the ModemManager git repository, which should hit git master very soon, likely this week. These ones should be the last API breaks done before releasing the new ModemManager, and will be kept stable after that.

Integrating the new ModemManager in GNOME3

(C) allenran917@ Flickr – CC by-nc-nd

It was something like September 2011, shortly after the last Desktop Summit in Berlin, when I started the work improving ModemManager with the new DBus API, GDBus-based DBus support, port-type-agnostic implementations, dynamic DBus interfaces, built-in org.freedesktop.DBus.ObjectManager interface support, built-in org.freedesktop.DBus.Properties.PropertiesChanged signal support, the new libmm-glib client library, the new mmcli command line tool… Took me around half a year to port most of the core stuff, and some more months to port all the plugins we had. And then, suddenly, git master wasn’t that unfinished thing any more, and we were even implementing lots of new features like GPS support in the Location interface, improved SMS Messaging capabilities, ability to switch Firmware on supported modems, and last but definitely not least, support for new QMI-enabled modems through the just officially released libqmi

And when I thought it was all done already, I woke up from my dream and realized that not even me was really using the new ModemManager as it wasn’t integrated in the desktop… not even in NetworkManager. So there I was again, with a whole new set of things to fix…

NetworkManager integration

There is already a set of patches available to include integration of the new ModemManager interface in NetworkManager; pending for review in the NM mailing list. This is the second iteration already, after having some of the patches from the first iteration already merged in git master.

This integration is probably the most complex one in the list, as it really has to deal with quite different DBus interfaces, but the overall look of it seems to be quite good from my point of view. With the patches on, NetworkManager exposes a new –with-modem-manager-1 configure switch which defaults to ‘auto’ (compile the MM1 support if libmm-glib found). Note that both the old and new ModemManager would be supported in this case, to easier fallback to the old version if the user experiences problems with the new ModemManager.

All in all, the required changes to NetworkManager are quite well defined, implemented mainly as a new ‘NMModemBroadband’ object and some additional bits to monitor added/removed modems through the org.freedesktop.DBus.ObjectManager interface.

gnome-shell integration

A proper integration in my desktop of choice (gnome-shell based GNOME 3) required to integrate the new interface in the Network indicator of the shell. As in every good day, I first had to deal with several issues I found, but I ended up submitting a patch that properly talks to the new ModemManager1 interface when needed. As with NetworkManager, this patch would support both the old and the new interfaces at the same time.

Not that this was a big change, anyway. The Network indicator only uses the ModemManager interface to grab Operator Name, MCCMNC and/or SID, and signal strength. Something that could even be included directly in the NMDevice exposed by NetworkManager actually…

Right now the only big issue pending here is to properly unlock the modem when it gets enabled, something that I still need to check how to do it or where to do it.

Truth be told, hacking the shell has ended up being quite nice, even with my total lack of knowledge in JavaScript. My only suggestion here would be: get a new enough distribution with a recent enough GNOME3 (e.g. the unreleased Fedora 18) and jhbuild buildone from there. I think I’ve never seen a clean jhbuild from scratch succeed… maybe it’s just bad luck.

And then came the applet…

network-manager-applet integration

Why the hell would Bluetooth DUN not work with my implementation, I was thinking, until I found that even if GNOME3 doesn’t rely on nm-applet, it still uses some of its functionality through libnm-gtk/libnma, like the code to decide which kind of modem we’re dealing with and launch the appropriate mobile connection wizard. Ended up also submitting patches to include the missing functionality in network-manager-applet.

For those not using GNOME3 and gnome-shell; note that I didn’t implement the support for the new ModemManager1 interface in the nm-applet itself; I just did the bluetooth-guess-my-modem-type bits required in libnma. If I feel in the mood I may even try to implement the proper support in the applet, but I wouldn’t mind some help with this… patches welcome! Same goes for other desktops relying on NetworkManager (KDE, xfce…); I wouldn’t mind to update those myself as well, but I truly don’t have that much free time.

gnome-control-center integration

Ok, so gnome-shell integration is more or less ready now; we should be finished, right? Well, not just yet. gnome-control-center also talks to ModemManager, in this case to get Operator Name and Equipment Identifier, which btw were not getting properly loaded and updated. Once that fixed, I finally got a new patch to have the control center talk to the new interface.

Fedora 18 packages

I bet you won’t go one by one to all the patches I linked before and apply them in your custom compiled NetworkManager, gnome-shell, network-manager-applet and gnome-control-center… but for all those brave Fedora 18 users, you can try with the 64-bit packages that I built for me, all available here:

If you want to rebuild these packages yourself, you’ll find the source tarballs here and the packaging repositories here. The packaging is really awful – I suck at it – so you’ll probably need to install the RPMs with –force 🙂 Note that the git repo for packaging has several git submodules, one for each item packaged, so remember to “git submodule init” and “git submodule update“.

We done yet?

Are these all the patches needed to have the best ModemManager experience in GNOME3? No! The list of pending tasks for this purpose grows a bit every day…

SMS goodies in ModemManager

(C) josullivan.59 @ Flickr – CC by-nc-sa

Lots of new SMS-related features have landed in ModemManager this week, and as I have as much memory as a fish used to have, I need to write down all the commands I’ve used to test all this; hence this post.

Note that all the stuff I talk about here applies to both AT and QMI based modems.


Multiple storages

The current stable 0.5/0.6 releases will only use the core/plugin defined ‘default’ storages for reading and sending short messages. This means that if you happen to have stored SMS in another non-default storage ModemManager won’t show them to you.

In the new 0.7 codebase, ModemManager reads the messages from all the available storages. It is able to read not only DELIVER-type PDUs from received SMS, but also SUBMIT-type PDUs from user-created SMS which got stored. The “Sms.Store()” method in the interface now allows (optionally) specifying in which storage the SMS should be kept.

The following commands allow to create a new SMS and store it in the “sm” (SIM) storage:

$> sudo mmcli -m 0 \
              --messaging-create-sms="text='Hello world',number='+1234567890'

Successfully created new SMS:
    /org/freedesktop/ModemManager1/SMS/21 (unknown)
$> sudo mmcli -s 21 --store-in-storage="sm"
successfully stored the SMS
$> sudo mmcli -s 21
SMS '/org/freedesktop/ModemManager1/SMS/21'
  Content    |              number: '+1234567890'
             |                text: 'Hello world'
  Properties |            PDU type: 'submit'
             |               state: 'stored'
             |                smsc: 'unknown'
             |            validity: '0'
             |               class: '0'
             |             storage: 'sm'
             |     delivery report: 'not requested'
             |   message reference: '0'

Note that you can create, store and send SMS objects regardless of what storage you’re using for each, ModemManager will take care of properly synchronizing the access to each of them.

The Messaging interface also defines new properties to list all the supported storages and to specify which is the default storage being used for received messages (and for stored ones when no specific one given in “Sms.Store()“):

$> sudo mmcli -m 0 --messaging-status
  Messaging | supported storages: 'sm, me'
            |    default storage: 'me'


Text multipart SMS

We support receiving text multipart messages in ModemManager since some stable releases ago; but support for sending multipart messages was still missing. Not any more! Roberto Majadas of OpenShine started this work some months ago with a patch for MM 0.6, which I took over and ported to git master and the new 0.7 codebase.

Of course, we support both the default GSM 7-bit alphabet, as well as UCS-2:

  • GSM 7 encoded multipart messages: If the message is less or equal than 160 septets a singlepart SMS will be used; otherwise the input text will be split into chunks of 153 septets and sent as a multipart message.
  • UCS-2 encoded multipart messages: The GSM 7 alphabet and its extensions are not enough to cover all scripts available out there. Luckily, we can send PDUs with UCS-2 encoded text (UCS-2 covers all the basic plane of Unicode, and is equivalent to UTF-16 in that range). When using this encoding, if the input text is less or equal than 70 UCS-2 characters (140 bytes) a singlepart SMS will be used; otherwise the input text will be split into chunks of 67 UCS-2 characters (134 bytes) and sent as a multipart message. Note anyway that this doesn’t mean that we only allow 70-byte long input strings to fit into a singlepart message; as the conversion from the input UTF-8 text to UCS-2 isn’t always increasing the required number of bytes. As an example, we can now send 210 bytes of UTF-8 encoded chinese text into a single SMS with 140-bytes of UCS-2.

Note that from the API user’s point of view, the SMS objects are not considered multipart or singlepart; that is taken as internal information only required for the PDU-based implementation.

Multipart messages can be created by writing a long enough ‘text‘ value passed in the ‘--messaging-create-sms‘ command, as shown before.


Raw data multipart SMS

In addition to text short messages we now also properly support raw binary data singlepart and multipart messages (reported as 8bit encoded). This means that both the sender and receiver can agree in any text encoding to be used between them, or even send in one or more SMS whatever raw binary stream you have.

The ‘mmcli’ command line utility was improved with several commands to test this feature. For example, you can now send using just SMS messages whatever file you have in your system, just doing:

$> sudo mmcli -m 0 \
       --messaging-create-sms="number='+1234567890'" \

Successfully created new SMS:
    /org/freedesktop/ModemManager1/SMS/22 (unknown)
$> sudo mmcli -s 22 --send
successfully sent the SMS

When the receiver gets all the parts of the message, she can now recover the sent file with another ‘mmcli’ command in her ModemManager setup:

$> sudo mmcli -m 0 \

Found 1 SMS messages:
    /org/freedesktop/ModemManager1/SMS/0 (received)
$> sudo mmcli -s 0 \

But beware! The file is sent in chunks of 134 bytes, and there is a maximum of 255 parts in a given multipart SMS, so this leaves you with a maximum size of around 33 Kbytes 🙂 And not only that, if you’re paying e.g. 0.10€ for each SMS sent, you would be paying more than 25€ for those 33 Kbytes. Yes, needless to say, this is therefore the most useless use case ever. But still, works as a proof of concept.


Delivery reports

Last and least, you can now also request delivery reports when sending an SMS, by passing the ‘delivery-report-request=yes‘ key/value pair in the ‘--messaging-create-sms‘ command.

$> sudo mmcli -m 0 \

Successfully created new SMS:
  /org/freedesktop/ModemManager1/SMS/0 (unknown)
$> sudo mmcli -s 0 --send
successfully sent the SMS
$> sudo mmcli -m 0 --messaging-list-sms
Found 21 SMS messages:
  /org/freedesktop/ModemManager1/SMS/0 (sent)
  /org/freedesktop/ModemManager1/SMS/1 (received)
$> sudo mmcli -s 1
SMS '/org/freedesktop/ModemManager1/SMS/1'
  Content    |              number: '+1234567890'
  Properties |            PDU type: 'status-report'
             |               state: 'received'
             |                smsc: '+9876543210'
             |            validity: '0'
             |               class: '0'
             |             storage: 'unknown'
             |   message reference: '137'
             |           timestamp: '120914120619+02'
             |      delivery state: 'temporary-error-sc-specific-reason' (0x30)
             | discharge timestamp: '120914120619+02'

You can then match the delivery report with the message sent by using the ‘message reference‘ value, which should be the same in both. This also applies to multipart messages, where the ‘message reference’ of our SMS object is the one reported when sending the last part, and therefore we only request the delivery report in the last part sent.