ROX System Architecture
This document outlines the system architecture commonly used in our projects. It reflects multiple iterations and trade-offs, resulting in a solution that has proven to be scalable and maintainable across various commercial applications.
We provide an overview of the system structure, including links to relevant software modules and technologies, focusing on how the system functions, without delving into the rationales behind its design.
Our architecture is tailored for mobile autonomous robots, emphasizing scalability and maintainability. It easily scales by adding or removing modules at the Application Layer without impacting other layers. The Sensors & Actuators Layer can expand flexibly through CAN bus and TCP networks, while USB or serial connections are supported but offer limited scalability.
System overview
Note
To keep this document concise, some technical terms are used without extensive explanation. This should not pose a challenge for seasoned developers. For novice users, this document serves as a good starting point for further research. Online resources such as search engines and AI assistants can provide additional information as needed.
System Layers Overview
The architecture is structured into several distinct layers, each with a specific role in the robot's operation:
Layer | Description |
---|---|
Sensors & Actuators Layer (SAL) | Consists of sensors and actuators that enable the robot to interact with the physical world. Sensors gather information, while actuators perform actions. |
Hardware Interface Layer (HIL) | Acts as a bridge between the sensors/actuators and the embedded control system. Typically connects to an industrial PC running Linux for further processing. |
Application Layer (AL) | Responsible for executing the core logic of the robot. Manages tasks such as navigation, obstacle detection, and decision-making, adaptable to the robot's mission. |
Middleware | Supports inter-process communication and facilitates connections to external systems. Ensures smooth data exchange between different components. |
Network Interface Layer | Connects the system to the digital world, providing channels for both internal communication and external connections. |
Cloud Layer | Offers remote functionality, including diagnostics and fleet management, and provides desktop tools for system analysis and monitoring. |
Web User Interface (Web UI) | Enables human-machine interaction through devices like tablets or phones, offering a user-friendly way to control, monitor, and interact with the robot in real time. |
This layered architecture provides a clear separation of responsibilities, making it highly modular and adaptable. Modules in the Application Layer can be adjusted without requiring changes to lower-level hardware interfaces or the upper layers, enabling efficient development and ease of cusomization.
System Layers
Sensors & Actuators
This layer includes the physical peripherals such as sensors and actuators.
- We use standard industrial 24V automation components, like capacitive sensors, inductive sensors, and relay switches. These components are connected to ICU1, which act as Remote IO hubs or handle low-level control loops when realtime behavior is needed.
- We use BLDC motors with high-performance servo controllers. These provide accurate position & velocity control while enabling precise control of acceleration and torque. The motors can be used for linear (pushers, lifts etc.) or angular actuation like driving wheels.
- localisation systems like RTK-GPS.
- various types of Human-Machine interaction devices like buttons, switches, gamepads etc.
Hardware Interface Layer
The Hardware Interface Layer connects physical hardware to the software components of the system.
-
CAN Bus: This is the preferred method of interfacing. It is a robust, proven technology widely used in automotive and agricultural technology applications. The interface requires only two wires and supports both centralized and distributed hardware deployments. The speed provided by the CAN 2.0 standard is sufficient for most sensors and actuators, including servo control.
-
Ethernet: Ethernet provides a high-speed interface suitable for devices such as cameras, or for data-intensive operations and web UI interactions.
-
USB/Serial: These interfaces are less suited for industrial applications due to their limited robustness, but are sometimes necessary if a device does not support CAN or Ethernet.
Application Layer
The Application Layer handles the main system logic. The architecture is set up as follows:
-
Subsystems: The system is divided into various subsystems, each with a specific responsibility and interface. For example, the HAL Subsystem is responsible for providing a uniform and abstract machine interface. It receives high-level commands like velocity and curvature, translating them into individual servo setpoints. It also reads data from GPS and other sensors to perform localization and reports the machine's position.
-
Docker Containers: Each subsystem runs in a dedicated Docker container, ensuring isolation and modularity for efficient maintenance and scalability.
-
Nodes: Each subsystem contains one or more
Nodes
, each running its own tasks. This setup allows for efficient, non-blocking concurrent operations, improving system responsiveness and performance.
Subsystem modules
We have developed several driver modules that can be quickly deployed as a subsystem
. These are available as open-source modules:
- RTK-GPS units
- Septentrio for clear-sky applications
- Fixposition - for challenging conditions, using visual odometry and wheel odometry to supplement RTK-GPS.
- Odrive-Can - high-level driver for ODrive 3.6 BLDC controller.
- Heartbeat - send system health status to your own cloud server.
- Gamepad - interface to standard gamepads and joysticks like Logitech.
- Action Trees provides a framework for implementing machine behavior with hierachial task decomposition. It breaks down complex tasks into simpler, independent steps that can be run sequentially or in parallel.
Middleware
The Middleware layer is what "ties it all together." There are many options for inter-process communication, but we have chosen a balance between proven technology, simplicity, and performance.
The middleware is similar to the ROS pub/sub system, but we use MQTT instead of DDS to simplify configuration and system interfacing. This trade-off means the system cannot efficiently pass large chunks of binary data, such as images. However, in this architecture, this is not an issue as nodes handle large data streams internally and only post aggregated results to the MQTT topics.
Technologies Used:
- MQTT: Used for communication between subsystems and for streaming data. Messages are passed as
JSON
, with receiving nodes handling data validation usingpydantic
where necessary. - FastAPI: Provides a
REST
interface for the machine, allowing for easy external interactions. - WS Bridge: Our custom module that translates between MQTT and WebSocket protocols, facilitating streaming data for web-based user interfaces.
Network Interface
This layer connects the machine to the outside world, enabling both remote and local communication. We typically use:
- LTE Connection: Provides reliable connectivity through an SSH tunnel to our cloud infrastructure.
- Wired Ethernet / WiFi: Commonly used during research and development.
- Local WiFi Access Point: Allows easy connection to local web-based user interfaces for convenient on-site interaction.
Cloud Layer
The cloud layer facilitates centralized control and diagnostics, offering various services:
- Time Series Database: Stores diagnostic data over time, aiding in long-term analysis.
- Data Analysis Dashboards: Provides visual insights into system performance and diagnostics.
- Fleet Management Tools: Includes features like remote updates and monitoring for multiple units.
- Central MQTT Server: Handles heartbeat signals and diagnostic data from all connected units, enabling real-time monitoring.
Technologies Used:
- InfluxDB: For storing time-series data.
- Grafana: To visualize diagnostic and performance data.
- Ansible: Used for fleet management and remote configuration.
Tip
We offer consulting-based support and development services.
Need advice or help integrating or customizing our modules?
Feel free to reach out at info@roxautomation.com. Your first 30-minute consultation is free!
-
ICU
stands for Integrated Control Unit. It is similar to ECUs (Electronic Control Units) used in automotive applications, with ICU being more flexible. An easily programmable microcontroller and industrial IO make an ICU in essence a 'mini-PLC' ↩