Managing a Linux IoT Device Fleet: Solutions and Strategies

Managing 10–1000 Linux-based IoT devices (e.g. Raspberry Pis) poses unique challenges. You need to keep systems updated (OS packages and Docker containers), maintain configuration as code, and support devices with intermittent connectivity. All of this should be done cost-effectively (preferably with open-source, self-hosted tools to avoid vendor lock-in). In this report, we explore both fully managed platforms and custom open-source approaches for IoT fleet management. We compare popular tools like Balena, Mender, Ansible (Pull mode), NixOS, and others, and discuss how to architect a solution that meets the following requirements:

OS Updates: Apply system updates (e.g. Debian/Ubuntu apt upgrade).
Container Updates: Deploy updates to Docker containers running on devices.
Config Management: Manage system configuration as code (infrastructure-as-code).
Intermittent Connectivity: Handle devices on cellular/4G or otherwise offline for long periods.
Pull-Based Updates: Devices should pull updates periodically (no constant connectivity or open inbound ports required).
Remote Access: Provide remote shell or SSH access for troubleshooting.
Preference for Open-Source: Solutions that can be self-hosted and avoid high recurring costs or proprietary lock-in.
Extensibility: Acceptable to use partial components and scripts to build a custom solution (user can write code and front-ends).

We will first give an overview of the types of solutions available (from all-in-one fleet management platforms to configuration management and OS-level approaches), then compare specific tools and their trade-offs. Finally, we recommend some architectures and combinations of tools suitable for managing a fleet of IoT Linux devices under these constraints.

Fully Managed vs. Self-Hosted Approaches

When selecting a fleet management solution, a fundamental decision is "build vs. buy." Fully managed IoT device management services (often proprietary) can provide a one-stop solution for deploying and monitoring devices, but they typically come with recurring costs and potential vendor lock-in. In contrast, self-hosted open-source tools offer more control and no license fees, but may require more integration work.

Fully Managed Platforms (Proprietary): Examples include AWS IoT Device Management, Azure IoT Hub/Edge, Google IoT Core, and SaaS platforms like balenaCloud, JFrog Connect (Upswift), Particle, etc. These services are convenient and often "tick all the boxes" for device provisioning, OTA updates, and remote access. For instance, AWS IoT Device Management lets you register devices, organize fleets, and push software/firmware updates with fine-grained control. 5 Best Linux IoT Device Management Platforms for Seamless Monitoring and Control However, such services charge per device or per update action, which can become expensive at scale, and they tie your solution to a specific cloud vendor's ecosystem. If avoiding high recurring costs and lock-in is a priority, you may lean toward self-hosted or open-source options.

Open-Source & Self-Hosted Solutions: There are several mature open-source projects for IoT fleet management. These can often be hosted on your own servers or cloud instances, avoiding vendor lock. Notable examples include openBalena (the open-source version of Balena's platform), Mender, Deviceplane, and others we'll explore below. Open-source solutions give you flexibility to customize or integrate with your own tooling, and the only cost is the infrastructure to run them. The downside is the learning curve and maintenance overhead – you become responsible for running the management server and possibly stitching multiple tools together (for example, using one tool for base OS updates and another for application containers).

In practice, many teams choose a hybrid approach: using open-source components as building blocks to create a custom management stack that fits their needs. For example, one could use an OTA update manager for system updates, plus a configuration management tool for fine-grained settings, and perhaps a VPN or remote shell service for access. The following sections discuss the key categories of tools and some specific options in each category.

Open-Source IoT Fleet Management Platforms

Several platforms aim to provide end-to-end management of Linux IoT devices. These typically include an agent or special OS on the device and a central server (cloud or self-hosted) that coordinates deployments, much like a "miniature cloud" for your devices. Below we discuss Balena, Mender, and Deviceplane – all of which are open-source or have open-source core components – as well as notes on similar solutions.

Balena (openBalena / balenaCloud)

Balena platform architecture, using balenaOS (a minimal Linux with Docker) on devices, and a cloud or self-hosted API server. Applications are built as containers and pushed to a registry, and devices proactively fetch updates via the Balena API. Choosing the Right IoT Fleet Management System | ICS

Balena is a container-centric IoT deployment platform. Balena provides a complete toolchain for building, deploying, and managing fleets of Linux devices. The platform includes a custom lightweight OS (balenaOS) for devices, which is based on Yocto Linux and optimized to run Docker containers via the BalenaEngine container runtime. The typical workflow is: you develop your application as one or more Docker containers, and use Balena's build system to create an image. This image (which includes balenaOS and your containers) gets pushed to a registry and then rolled out to devices in the fleet. Each device runs a Balena Supervisor service that communicates with the cloud, checks for updates, and downloads new container images when available.

Key Features: Balena covers almost all aspects of device management – provisioning new devices, orchestrating application updates, monitoring device status, and providing remote access (through an integrated VPN/SSH service). In fact, Balena offers "full-cycle device management: build, deploy, provision, control, decommission etc." The first 10 devices on balenaCloud (the hosted service) are free, which allows easy evaluation. Balena supports many hardware types (Raspberry Pi, Jetson, Beaglebone, etc.) and any programming language that can run in a Linux container. 5 Best Linux IoT Device Management Platforms for Seamless Monitoring and Control

OS Updates: With Balena, the base OS on devices is balenaOS, which Balena manages for you. Updates to the OS itself (security patches, kernel updates) are handled via Balena's infrastructure (balenaOS uses an atomic update mechanism similar to OSTree under the hood). As a user, you typically don't apt upgrade the device; instead, you rely on Balena to push OS updates when needed. This means apt-based updates are not done on the host – any OS packages you need must be part of your container or included in a custom balenaOS build if absolutely required. The benefit is a controlled, uniform base image across all devices.

Application (Container) Updates: This is Balena's strength – application code runs in Docker containers that can be updated independently of the host OS. When you release a new version of your app (as a new container image), the Balena cloud will notify online devices to fetch the update. Devices with intermittent connectivity will apply the update when they come back online (the Balena Supervisor continuously polls or maintains a connection to get updates). Container updates are non-disruptive and safe because they download fully before swapping, and you can define multiple release channels or use phased rollouts for safety.

Configuration Management: Because devices are meant to be identical (each running the containers you define), configuration changes are typically handled by updating container environments or using Balena's API to set device-specific environment variables. There isn't a traditional config management tool (like Ansible) inside Balena, since the paradigm is to treat everything as part of the container image (infrastructure-as-code at the container level). This works well if your use-case fits into the container model. However, it imposes some development constraints – you must containerize your app and use balenaOS. It's not possible to easily modify the host OS or install extra packages on the host; you are somewhat locked into the provided OS image and features. Choosing the Right IoT Fleet Management System | ICS

Connectivity Model: Balena devices initiate a secure connection to the cloud (through a VPN or API gateway). This means no inbound ports need to be opened on the device side; updates and commands are delivered when the device is online. If a device is offline, it will simply miss updates until it reconnects, at which point it syncs with the latest release. In effect, it's a pull-based mechanism (device checks in to get new container images), though implemented via a persistent connection for instant updates.

Remote Access: Balena provides built-in remote access. Via the balenaCloud dashboard (or CLI), you can open a web terminal into a container or even the host OS, using the VPN tunnel. This makes remote debugging straightforward. (In open-source openBalena, which is the self-hosted edition, similar functionality is available, but you'll need to manage the VPN/SSH setup yourself to some extent.)

Open vs. Proprietary: The core of Balena is open source. openBalena is the self-hosted server stack (APIs, database, etc.) that you can deploy under an open license. BalenaOS is also open source. However, Balena's cloud service and some features (like the web UI, device metrics, etc.) are proprietary or only provided as part of balenaCloud. OpenBalena is headless (no web interface out-of-the-box) and was, as of 2022, still in beta. It gives you the back-end infrastructure to register devices and push updates, but you may need to build your own UI or workflows on top of it. Importantly, using Balena inherently means using balenaOS on your devices. This is a form of vendor lock-in – the device provisioning and update flow is specific to Balena's ecosystem, and it's "impossible to add new features or make any amendments to existing flows" on the BalenaOS or supervisor. The upside is a tightly integrated system that works reliably for fast-to-market solutions; the downside is less flexibility to, say, run a vanilla Raspbian OS or use alternative update methods. Choosing the Right IoT Fleet Management System | ICS

In summary, Balena (especially balenaCloud) is a powerful option if you are willing to containerize all applications and use their provided OS. It "ticks all the boxes" for device management simplicity, but you trade off some openness. For a self-hosted deployment with no recurring fees, openBalena can be used to avoid lock-in fears – just note the features or polish might not equal the hosted version.

Mender (Open Source OTA Update Manager)

Mender's client-server architecture for OTA updates. Devices run a Mender client that polls the server for update artifacts (e.g. a root filesystem image or package). The server (which can be self-hosted) manages deployments, authenticates devices, and stores update artifacts (in S3, Minio, etc.). Mender supports robust features like A/B partitioning, atomic install, delta updates, and phased rollouts. Choosing the Right IoT Fleet Management System | ICS

Mender is an open-source, end-to-end over-the-air (OTA) software update manager for embedded Linux devices. Unlike Balena, which is a full fleet management with container focus, Mender's core purpose is to deliver software updates (especially firmware or OS updates) in a robust, secure way. It can be thought of as an "OTA infrastructure": you have a Mender server (with management UI and APIs) and a Mender client running on devices.

Key Features: Mender was designed with reliability in mind for IoT/embedded scenarios. It typically uses a dual A/B partition scheme on the device: when you deploy a new system update, it writes the new image to the inactive partition, then swaps boot to that partition on next reboot – if the update fails or the device doesn't boot properly, it can roll back to the previous partition. This gives atomic, fail-safe updates. Mender supports code signing, compression, and even delta updates (binary diffs) to minimize bandwidth. It also has features for phased rollout (gradually update a percentage of devices, to reduce risk).

OS Updates: Mender excels at full system updates. Originally, Mender would deploy entire disk images (or root filesystem images) to devices. This ensures all packages and software on the device are updated in one coordinated step with the ability to rollback. The downside is that an image might be large; Mender mitigates this with delta updates, which send only differences from the current version. Mender's approach is great for critical firmware that must not fail mid-update (e.g. devices in remote locations). If your use case involves occasionally pushing a new OS build to devices (with updated kernel or base libraries), Mender is a strong choice.

Application/Container Updates: In addition to full images, Mender introduced Update Modules which allow more flexible update types. For example, there are modules to deploy a single file, run a script, or even update a Docker container. This means you can use Mender to update just an application on the device without reflashing the whole OS. For instance, a Docker Update Module could fetch a new container image and restart the container. While not as instantaneous as Balena's method, it allows Mender to handle application updates in a pinch. Generally, though, if you heavily use containers, you might combine Mender with a container registry approach rather than using Mender for every app update.

Configuration Management: Mender is not a full config management system, but it provides some capabilities to distribute configuration data. It has a feature called "State Scripts" and now "Configuration" where you can send JSON config to the device. A clever approach is to use Mender in conjunction with a tool like Ansible – for example, use Mender to deploy a small config artifact that tells a device which Ansible playbook to run (we discuss this hybrid approach later). In essence, Mender ensures a reliable delivery of a payload, and that payload could be anything (including instructions to run configuration scripts). The Mender server provides an inventory of devices and their attributes, and you can group devices to send certain updates or configurations to specific sets (e.g. by device type or customer). This covers infrastructure-as-code partially: you define what artifact or config goes to which group in the Mender UI or via its REST API.

Connectivity Model: Mender clients operate in pull mode. Each device periodically polls the server (over HTTPS) to ask "Is there an update for me?". If yes, it downloads the artifact and then the deployment proceeds. If a device is offline, the server just waits until the next time the device checks in. This design inherently supports intermittent connectivity – devices update on their own schedule when they have connectivity. The server does not need to reach the device directly (no open inbound ports on the device), which is ideal for NAT'ed or cellular connections. This aligns perfectly with the requirement for pull-based updates and offline tolerance. For instance, the device "authenticates and pulls for new applications and OS versions" as shown in Mender's architecture. Choosing the Right IoT Fleet Management System | ICS

Remote Access: In its early versions, Mender focused purely on updates, but newer versions (and particularly the commercial tiers) have add-ons for remote troubleshooting, including a remote shell (terminal) and file transfer. There is a component called mender-connect which sets up a secure websocket to allow a remote terminal session from the server UI. Mender Connect | Mender documentation Mender's open-source edition includes these add-ons under permissive licenses (they were open-sourced in 2020-2021 timeframe). So, with some setup, you can get a remote shell to your devices through the Mender server UI, similar to how Balena offers web terminals. Surprisingly Easy IoT Device Management - edi This is a big plus, as it means you might not need a separate VPN or SSH solution – you can troubleshoot devices via Mender's interface (especially useful if devices are behind cellular NAT). Remote logging and file transfer are also possible through these add-ons (Enterprise versions also offer remote port forwarding and desktop access).

Open-Source vs. Hosted: The Mender server is open-source (Apache 2.0) and built in a modular, microservices way. You can self-host it (they provide Docker Compose or Kubernetes deployment for the server). There is also a hosted Mender service (with free trial and paid plans for more devices or features). The open-source edition has all core update features, while the paid tiers add convenience features, professional support, and maybe advanced add-ons. Importantly, you have the flexibility to switch between hosted and self-managed: Mender can be run in-house for environments that don't allow public cloud (e.g. military or hospital networks). This flexibility is a highlight: "the combination of a hosted service and self-managed server gives full flexibility to users". You won't be locked in because you can always fall back to running your own Mender server if needed. Choosing the Right IoT Fleet Management System | ICS

Trade-offs: Compared to Balena, Mender is less all-in-one. It doesn't provide a built-in application build pipeline or a specialized OS – you can use Mender with your existing Linux distribution (Debian, Yocto, etc.) by installing the Mender client. This makes it more flexible (no need to adopt a custom OS), but you may need to integrate Mender into your OS build or install its client packages. Mender's focus on robustness (A/B updates) is great for system reliability, but deploying full images for small changes can be heavy – hence why combining it with a config management tool or using update modules is common. The ICS IoT blog noted that Mender is "extendable" and open, but "lacks the completeness and maturity of Balena" in terms of providing a full development ecosystem. In practice, many use Mender alongside other tools (for example, use Mender for periodic OS updates, but manage app containers via a separate mechanism). We will see an example of such a hybrid approach in a later section.

Combining Mender (OTA) with Ansible (Config) – Example

It's worth highlighting a hybrid approach that has been demonstrated in the community: using Mender together with Ansible and Git (GitOps style). In this pattern, Mender handles delivering a small artifact that basically contains a pointer to a desired Ansible playbook/role, and then on the device a hook script runs Ansible locally to apply that role. This was shown by a developer who turned a Raspberry Pi into a kiosk using Mender+Ansible. Surprisingly Easy IoT Device Management - edi The flow was:

Base Image: A minimal Debian image with Mender client, Ansible, and Git pre-installed on the device. (Image built with tools like edi – a Debian image builder).
Mender Artifact (Config): The user creates a Mender "update" that isn't a full OS, but just a JSON configuration containing parameters like playbook_url and playbook_version (Git commit or branch). Surprisingly Easy IoT Device Management - edi
Device pulls config via Mender: When applied, Mender delivers this JSON to the device (wrapped in an artifact). A state script on the device triggers upon receiving it. Surprisingly Easy IoT Device Management - edi
Git checkout: The script reads the JSON (using jq) to get the Git URL and version, then performs a git clone or git pull of the Ansible playbook repository at that version. Surprisingly Easy IoT Device Management - edi
Local Ansible apply: It then runs ansible-playbook on the pulled playbook to configure the device (install packages, set up services, etc.). Surprisingly Easy IoT Device Management - edi
Reboot if needed: If the playbook_needs_reboot flag was set, it reboots the device to finalize any changes (for example, if the kernel was updated). Surprisingly Easy IoT Device Management - edi

This approach effectively uses Mender as a reliable delivery mechanism (steps 1-2, which is atomic and can rollback if the config application fails) and Ansible as the flexible configuration tool to actually make changes. The developer noted the setup is "lightweight, scalable, and reuses powerful tools."

The only overhead was the base image size (Ansible and Git added ~100MB, which was acceptable for them).

For more details on this approach, see Surprisingly Easy IoT Device Management - edi.

Such a combination gives a lot of confidence: Mender ensures that if an Ansible run somehow breaks networking or the system, the device will rollback to the previous state (since the Ansible run can be considered part of the update transaction). Meanwhile, Ansible gives the ease of using apt, managing files, etc., without having to rebuild full disk images for every small change.

Takeaway

Combining tools can yield a solution where each handles what it's best at (Mender for robustness, Ansible for flexibility). For example: - Use Mender for base OS updates every few months, but use Deviceplane or Ansible for frequent app updates. - Use Balena for container deploys but have an Ansible container inside it doing some extra config tasks – though that gets complex.

Immutable & Declarative OS Approaches (NixOS, Ubuntu Core, etc.)

Another strategy is to use an operating system that inherently supports atomic, remote updates and is configured declaratively. NixOS is a prime example of this, and it was specifically mentioned as a consideration.

NixOS on IoT Devices

NixOS is a Linux distribution designed with declarative configuration and reproducible builds. Every aspect...