A futuristic control room with illuminated computer screens and electronic devices.

HILFE

Hardware-In-The-Loop tests for everyone

Simplify complex hardware tests

The development of meaningful hardware tests and corresponding test environments for embedded boards repeatedly presents our customers with considerable challenges. Some companies seem to avoid this hurdle by completely dispensing with hardware tests. This observation gave rise to our original idea of offering hardware test setups as a service for project customers - with the aim of speeding up their development processes and improving the quality of results. This service would add a valuable component to our portfolio as a "Smart Devices" division.

Labgrid

While researching existing projects, we identified the open source project "Labgrid" as the most promising Python-based tool in this area and decided to test it in practice. However, when analyzing the functionalities, we realized that Labgrid does not offer any network abstractions. This is crucial to isolate DUTs (Devices Under Test) across different network types such as wireless, internet or LoRa. This allows external interference to be avoided and the test environment to be deterministic through defined test networks. Nevertheless, Labgrid offers other valuable functions that we need for our project: Remote access to configurable DUT resources via test hosts is possible, as is coordinated resource management. A central coordinator handles both resource registration and the automatic distribution of DUT resources to various test scenarios. At the same time, it allows developers direct remote access to the hardware, enabling live tests to be carried out on real systems.

One of our first implementation projects involved using Labgrid to perform tests on a Yocto-based "Blinky" application. The aim was to check whether the LED frequency was as expected. This test can be carried out in various ways: directly via cabling, optically or using a logic analyzer.

In our first example, we use Raspberry Pi's as test hosts. These are connected directly to a Raspberry Pi Compute Module - our Device Under Test (DUT) - to measure the flashing LED. The DUT runs an application created with Yocto that not only controls the LED, but also provides us with a console via a serial interface.

Two relays controlled by the test host are used to completely power down the DUT and to activate the flashable state before flashing a new Yocto image. Unlike a conventional Raspberry Pi, which relies on an SD card for flashing - the physical removal of which makes automated updating difficult - the Compute Module offers the advantage that images can be flashed directly to its integrated eMMC. Thanks to the USB flash option, this process is easy to automate and allows for seamless and efficient firmware updates.

From local tests to CI/CD

The first approach proved to be uncomplicated and easy to handle. However, visual inspection and the integration of a logic analyzer can also be easily implemented with Labgrid. This is thanks to the system's high level of extensibility, supported of course by the large number of useful Python libraries that are available for such applications.

The end-to-end integration from an Azure DevOps pipeline through to testing the DUT was also implemented quickly and efficiently. The test runs also work reliably remotely: Test hosts, which are managed via Ansible, host both Azure Agents and Labgrid Exporters - those instances that provide resources and communicate with the central coordinator.

As in the example above, the test hosts are connected to their corresponding DUTs via suitable interfaces. In our specific case, this is done via GPIO pins, which are connected to the flashing LED and the jumper for setting the boot mode, as well as via a webcam, a USB-to-UART adapter for console access to the DUT and a USB-to-MicroUSB for flashing it. A test run can now be started via the pipeline, which downloads the latest image for the DUT (depending on the use case, this can also be rebuilt each time - for example on a build host provided especially for this purpose), carries out the flash process, executes all defined tests on and with the DUT and uploads the result back to Azure DevOps.

At the same time, developers can use a webcam to monitor how the DUT is currently behaving by observing status LEDs or displays, for example. The ability to integrate any number of test hosts into the system means that Labgrid automatically creates a flexible and scalable test infrastructure.

The outlook

After we had already created this solid foundation, a larger, further goal emerged in parallel: the design of a test rack that would enable the addition of new DUTs - i.e. our customers' devices - and the associated test hosts with minimal effort. This is to be operated in a separate test network that is separate from MaibornWolff's in-house lab WIFI. This is the only way we can securely offer different customers external access options via VPN access. This would enable SMD to host one or more useful test setups for customers within a few hours or days - with access options for our own developers as well as for our teams and the test pipelines. This allows us to speed up development processes for our customers and improve the quality of results at the same time.