Assistance system as an everyday aid for visually impaired

At the Chair of Factory Automation and Production Systems (FAPS) at Friedrich-Alexander University Erlangen-Nuremberg (FAU), the project "Learning Assistance System for Orientation and Mobility of Visually Impaired People in Everyday Life" (LOMOBI) started this year. Over the next two years, FAU researchers, led by medical engineer Sina Martin, will develop a sophisticated system designed to guide people across different landscapes and through unfamiliar scenarios. The journey will culminate in a final two-kilometer test run, taking participants from their starting point in a chair to a local bakery.

ESI 2.0 successor

LOMOBI is the continuation of ESI 2.0, which FAPS conducted together with the Interdisciplinary Center for Embedded Systems (IZ ESI) from 2013 to 2014. ESI 2.0's goal was to enable blind or visually impaired athletes to run self-determined on forest and gravel trails with the help of an assistance system. For this purpose, a backpack was developed that has a built-in navigation system. Feedback signals such as vibrations or sound sequences emitted via bone-sound headphones warn joggers of obstacles, which are transmitted in real time using 3D sensor technology.

LOMOBI: Flexible, GPS-independent, predictive

The FAU researchers hope to expand the scope of the old assistance system: Firstly, LOMOBI is envisioned to operate without GPS coverage, so that users can also move around inside buildings. Another goal is to enable the user to walk safelyacross a wide array of surfaces and intersections . Although not the primary focus of the current project, but still very valuable for users: The system should also learn typical human walking behavior so that it can plan a path around oncoming people.

Under Sina Martin’s supervision, Matthias Kalenberg is concentrating primarily on navigation, while his colleague Helmut Engelhardt is responsible for image processing.

Continuous learning

LOMOBI should continuously improve and learn: It is important to the researchers that even unknown situations are no obstacle for the assistance system. By the end of the project in 2025, they would therefore like to set up a web platform that both LOMOBI users and sighted people can access to share images of environments. The individual systems integrate this data to learn new ways of doing things. In addition, sighted people can use the platform to correct the data of environments that have proven to be problematic for visually impaired people.

User at the center of development

While the ESI 2.0 project has already yielded valuable findings, the current research work focuses on the user's experience. To ensure that LOMOBI really meets the needs of blind and visually impaired people, Matthias Kalenberg and Helmut Engelhardt collaborated closely with the "Zentrum für selbstbestimmtes Leben Behinderter e.V." (ZSL Erlangen) to exchange ideas with members there within a workshop. The focus was directed towards ethical, legal and social dimensions: Is there a potential risk of stigmatization? How should the system’s design be conceptualized? How will data security and the general safety of those affected be dealt with?

LOMOBI in test

Later on, there will be extensive studies with LOMOBI in use. The final test is a walk from the FAPS building to a local bakery about two kilometers away, guided by the LOMOBI backpack. The way there includes basically everything that LOMOBI is supposed to handle: Sections not covered by GPS, changing surfaces such as carpet, tile, asphalt or gravel, transitions such as steps and floor ledges, and oncoming passenger traffic.

The LOMOBI project is funded by the German Federal Ministry of Education and Research (BMBF) to the tune of around 465,000 euros.