Category Archives: Lanedo Planet

Sharing a QMI port between processes

A bit of background…

I’ve blogged earlier about how the QMI protocol defines different services and allows to have multiple clients allocated for each service. Allowing multiple clients (i.e. QMI transactions with different client ids) was originally implemented in the protocol to allow multiple user-space applications to talk to the Gobi device, as the original Qualcomm kernel drivers allowed this. In these drivers, the management of client allocations and releases (through the implicit CTL service) was done by the kernel itself, and thus, multiple user-space applications could request new clients at the same time, leaving the work of managing the actual QMI communication with the device to the kernel.

But that is no longer the case with the ‘qmi_wwan‘ driver available in the upstream Linux kernel. In this case, the implicit CTL service is also managed in user-space, and there should be a single process involved in talking to the QMI port. The side effect of this behavior was that the whole client allocation logic didn’t make much sense any longer, as the process owning the port would be the only one being able to create clients.

Luckily, we designed the libqmi interface in a way that the CTL service was completely hidden to the user of the library. A user would just create a ‘QmiDevice‘ object, and afterwards allocate new ‘QmiClient‘ objects using the device object. All the internal CTL service communication between the user-space process and the actual modem device would be run internally by objects we created.

So, with the target of allowing multiple user-space processes to use the same QMI port in mind, a new ‘qmi-proxy‘ daemon was created in libqmi. Whenever a ‘QmiDevice‘ is opened with the new ‘QMI_DEVICE_OPEN_FLAGS_PROXY‘ flag, we will try to launch the ‘qmi-proxy’, which will be the only process talking to the actual QMI port. Every other process using the ‘QMI_DEVICE_OPEN_FLAGS_PROXY‘ flag, including the one which launched the proxy, will now send the QMI requests to the proxy, and receive the QMI responses and indications from the proxy.

Internals

In order to implement the proxy-based communication in libqmi’s ‘QmiDevice‘, we didn’t have to do many changes:

  • We extended our implementation of the CTL service with a new ‘Internal Proxy Open’ command, which allows the application to tell the proxy which QMI port wants to use. This command uses the 0xFF00 command id, far away from the current set of command ids currently available for the CTL service, so not likely to collide with any new update of the service ever.
  • All the communication from and to the proxy are plain QMI commands, including requests and responses in the CTL service. The only critical thing to do was to make sure that the ‘QmiDevice‘-side CTL transaction ids (i.e. unique CTL transaction in a given process) were translated to the ‘qmi-proxy’-side CTL transaction ids (i.e. unique CTL transaction with the actual device).
  • The CTL service is the only one requiring the transaction id translation, as for every other service clients need to be explicitly allocated and released. In other words, different applications will allocate different clients with different client ids, and once they all have its own client id, they can just use the transaction id that they prefer.
  • The ‘qmi-proxy’ only analyzes each transferred QMI message in very specific situations. In addition to the CTL transaction id translation mentioned above, the proxy will also take care of forwarding QMI indications (unsolicited messages) to the process owning the client id stated in the indication itself, or to all processes with a client for the service if the ‘broadcast’ client id is used. As an example, a process could request to get notified about signal quality updates via unsolicited “NAS signal info” messages, and the qmi-proxy will make sure that those indications are only sent to the specific process which requested them.
  • In order not to use resources when not needed, the ‘qmi-proxy’ is automatically started when first requested, and will be automatically stopped 30s after the last process disappears or releases the last client. We are using a unix socket with an abstract name, so there are no worries of race conditions where proxies are launched at the same time, and no need to make sure that a socket file is never left in the filesystem when the ‘qmi-proxy’ crashes unexpectedly.
  • To simplify the logic of the ‘qmi-proxy’, it will not take care of following which QMI ports are available in the system. As with the case of not using the proxy, each application needs to know which ports are available and can be requested to be open. If a port disappears and the proxy still receives requests to that port, it will just return a QMI error to the application which sent the request.

All the previous logic, as said earlier, is managed internally by libqmi and hidden to the user of the library. The key point is to modify your application to use the ‘QMI_DEVICE_OPEN_FLAGS_PROXY‘ flag.

ModemManager and qmicli

We also improved the ‘qmicli’ command line interface to allow a new ‘--device-open-proxy‘ argument (or the shorter ‘-p‘), which will make use of the new flag to request the ‘qmi-proxy’ to be used. And same for ModemManager; if the version of libqmi is >= 1.7 (currently git master), the new flag will be used by default. We can finally use both ‘qmicli’ and ModemManager at the same time, as ModemManager won’t be the only process owning the port.

Wait…

But why would anyone want to let other applications use the QMI port? Wouldn’t that collide with the logic of ModemManager?

Well, yes and no. It really depends on which kind of other applications you want to have running alongside ModemManager. If you want to run yet another connection manager, taking care of connecting and disconnecting the QMI port and such, then yes, it will likely collide with ModemManager’s own logic. But that should never be the case. The QMI protocol provides access to tons of features which are not handled by ModemManager, and that is where other applications will benefit from the proxy. Think of custom applications which want to get a deeper view of the status of the radio network using the NAS service, or a GPS application which wants more than what ModemManager provides, or a SIM browser application reading SIM files with the UIM service, or… well, you name it.

When will this be available?

The ‘qmi-proxy’ support is currently available in libqmi git master and is scheduled to be released in version 1.8 some day in the future. We just released 1.6 last week, so it may still take some time.

TL;DR

You can now write applications to use a QMI port with libqmi, even if ModemManager is running.

ModemManager 1.0.0 is out!

Dan's images with lots of modems are boring, so here's one with my tomatoes.

Dan’s images with lots of modems are boring, so here’s one with my tomatoes.

It took a bit more than 2 years since we started designing and implementing the new ModemManager1 interface, but here we are, finally releasing a new stable version of ModemManager with all the changes we’ve been doing in the past years; including QMI and MBIM modem support among other things.

Not much more to say here, just read the release announcement, download it, and have fun (or report bugs).

And of course, thanks to everyone involved in the new release!

Changing modes and capabilities in ModemManager

I don’t really recall when my first interaction with ModemManager was, I just remember saying “wait, this modem just works?”. But I do remember one day when I spent a couple of hours trying to understand why my modem wouldn’t switch to 2G-only mode even if I explicitly selected it in the network-manager-applet. Truth be told, I didn’t dig much in the issue that day; I barely knew what ModemManager was, or how it would interface with NetworkManager, or how to really debug it. And here I am possibly 4 years after that day, trying to get that same issue fixed.

allowed-and-preferred-modes

ModemManager has always allowed to specify which ‘network type‘ to use; or, rather than network type, ‘allowed and preferred‘ modes as we name them. It basically is a way to tell your modem that you want to use one technology preferred over another (e.g. allow 2G and 3G, but prefer 3G), or even tell the modem to use only one technology type (e.g. 3G only). Even modern phones allow you to turn off 3G support and only use 2G, in order to save battery. The main problem with ModemManager’s way of handling this issue was that there was a predefined set of combinations to select (as exposed by the applet), and that not all combinations are supported by all modems. Even worse, ModemManager may not know how to use them, or the modem itself may not support mode switching at all (which was actually what was happening with my modem 4 years ago). Therefore, the UI would just try to show all the options and hope for the best when launching the mobile broadband connection. And there it comes the next issue; allowing to select mode preferences during the connection setup just makes the modem restart the whole radio stack, and the connection attempt may end up timing out, as the whole network registration process needs to be done from scratch before connecting…

Allowed and Preferred modes

In the new ModemManager interfaces, each Modem object will expose a “SupportedModes” property listing all the mode combinations (allowed + preferred) the modem actually supports. Graphical user interfaces will therefore be able to provide mode switching options listing only those combinations that will work. If a modem doesn’t support mode switching, no such list should be provided, and the user will not get confused. At any time, the Modem object will also expose a “CurrentModes” property, showing which is the currently selected combination of allowed and preferred modes. And the “SetCurrentModes()” method will allow to switch current modes, accepting as input only combinations which are given in “SupportedModes” (or the special allowed=ANY and preferred=NONE).

Also, changing current modes directly when calling Simple.Connect() will no longer be possible. This means that NetworkManager will never request allowed mode switching during a connection attempt (and hence no radio stack reloading in the modem causing timeouts). The logical place to put allowed mode switching is therefore a system configuration application like the GNOME Control Center or similar, which should allow mode switching at any time, not only just during a connection attempt. A good side effect of this change is that the NetworkManager connection settings now contain only connection-related configuration, which in the case of 3GPP devices can be linked to the SIM in use, leaving out all modem-specific configuration.

Capabilities

There was a time when modems were either 3GPP (GSM/GPRS/UMTS/HSPA…) or 3GPP2 (CDMA/EV-DO…). Nowadays, modems with multiple capabilities are pretty common, specially since LTE is around (LTE, even if 3GPP, is also 3GPP2′s blessed 4G technology, instead of the superhero named one which is forgotten by everyone already). ModemManager will now allow to change capabilities in addition to allowed and preferred modes; so a user with a modem which can work both in 3GPP and 3GPP2 networks will be able to switch from one to the other directly from the user interface. Of course, if the modem supports this (currently only QMI-based modems).

The new “SupportedCapabilities” property will expose all capability combinations supported by the modem, while “CurrentCapabilities” will expose which are the current ones being used at any given time. For example, a modem with “gsm-umts”, “cdma-evdo” and “lte” capabilities may support configuring only “cdma-evdo”, or “gsm-umts”+”lte”. Changing current capabilities is now possible through the “SetCurrentCapabilities()” method, which has a logic very similar to that of the “SetCurrentModes()” method. If a modem supports multiple capability combinations as exposed by “SupportedCapabilities”, this method will allow changing between them. The main difference with mode changing is that we will force a device power-cycle when this change is done, so the modem will disappear and reappear again with the new capabilities.

Capabilities and allowed/preferred modes have a lot in common, so much that there is a single interface in QMI based modems to change them. Therefore, when a modem allows changing capabilities, the list of allowed/preferred mode combinations may (and very likely will) be different depending on the current capabilities in the modem. For example, LTE-enabled QMI-powered modems will not be able to switch allowed/preferred modes when they have “lte” among the current capabilities, but they will be able if the capabilities are changed to only “gsm-umts”. This is not a big deal, as mode preference (e.g. 3G preferred) is not applicable when the modem does LTE (there is no way of saying allow 2G, 3G and 4G but prefer 3G).

Bands

ModemManager also allows to specify which frequency bands to use in the modem, but unlike with modes and capabilities, the “SupportedBands” property is not a list of all possible band combinations supported. Instead, it’s just a bitmask with all supported bands, without specifying whether an actual combination is going to work in “SetCurrentBands()” or not. Listing combinations instead of just the bitmask would be truly too much… But anyway, changing frequency bands is not a feature that a normal user should play with, so just don’t do it. I actually bricked a Pantech UML290 myself playing with this…

aleksander/api-breaks

All these updates, plus some other ones, are available in the ‘aleksander/api-breaks‘ branch in the ModemManager git repository, which should hit git master very soon, likely this week. These ones should be the last API breaks done before releasing the new ModemManager, and will be kept stable after that.

Follow

Get every new post delivered to your Inbox.

Join 36 other followers