Methodology and System Design
The proposed smart touch sensing system integrates the optical interferometric sensor of ams OSRAM with the Xylo SNN processor from SynSense, a low-power edge processor designed for efficient, low-dimensional signal processing. This hardware platform combines dedicated signal pre-processing with a SNN inference engine, enabling compact and energy-efficient on-device intelligence.
At the front end, the processor performs feature extraction directly from the raw electrical output of the sensor. In particular, the incoming signal is decomposed into 16 frequency channels spanning a range from approximately 50 Hz to 17 KHz. This stage not only filters noise and structures the signal, but also converts it into spiking representations, which serve as input features for the subsequent neuromorphic processing stages.
The overall processing pipeline of the system is as follows: the sensor relies on self-mixing interferometry, where a near-infrared reflects off a target (the touch interface) and re-enters the laser cavity, producing measurable interferometric disturbances. Then the generated signal is fed into the front-end module of the Xylo processor for filtering and feature generation. Next, the extracted spiking features are passed to the SNN inference engine, which has been pre-trained for the specific task of gesture recognition. Finally, the output activity of the last network layer is decoded to produce the predicted gesture or interaction class.
This tightly integrated pipeline enables real-time, low-latency interpretation of tactile signals while maintaining a compact (< 100KB) memory footprint and energy-efficient implementation (~500 μW ), making it well suited for edge-based intelligent sensing applications.
Challenges and Lessons Learned
Transitioning from acoustic sensing to interferometric signals introduces several non-trivial challenges, particularly when implemented on a Xylo development kit originally designed for audio processing. Unlike audio-based applications, no publicly available datasets exist for gesture recognition using VCSEL-based self-mixing interferometry, requiring entirely custom data collection as well as careful definition of gesture classes and labeling strategies. Beyond signal compatibility, the interferometric modality itself brings sensitivity and stability challenges. The captured signals are highly dependent on precise mechanical alignment, surface properties, and small variations in displacement, making them more fragile compared to conventional acoustic signals. This sensitivity is compounded by the system’s susceptibility to environmental acoustic noise: the mechanical lever used for sensing responds not only to intentional touch interactions but also to ambient vibrations, leading to unintended signal contamination and reduced signal-to-noise ratio. In addition, variability across users and interaction styles can further complicate generalization, requiring robust normalization and potentially more diverse training data. Addressing these challenges requires tight co-design across sensing hardware, mechanical structure, and signal processing, as well as iterative calibration and validation to ensure reliable and repeatable performance under real-world conditions.
Conclusion
In summary, the VCSEL-based touch sensor paired with the Xylo SNN processor delivers a compact and energy-efficient tactile solution. By converting touch-induced signals into spiking features across 16 frequency channels, the system leverages the SNN’s lightweight architecture to perform real-time inference with minimal power. This design not only reduces the footprint compared to conventional touch systems but also highlights the potential of combining optical sensing with neuromorphic processing for highly responsive, low-power edge applications.
Blog signed by: SYNS team