Savor: Vision
Role: Solo Developer
Description: I created this computer vision module to work alongside Savor, to automate pantry inventory tracking by training and deploying YOLOv11 instance segmentation and object tracking models onto edge hardware. The CV module is trained using Python training frameworks to be deployed to low latency C++ production environments on the Raspberry Pi 5.
-
Edge AI Optimisation:
Developed a pipeline to train custom segmentation models and compile them
using the Hailo Dataflow Compiler (DFC). This process converts standard PyTorch models
into highly optimised
HEFbinaries capable of running on the Hailo-8 NPU (26 TOPS) . - C++ Inference Engine: I Engineered a lightweight C++ runtime application using HailoRT. This replaces Python bindings to minimise overhead, ensuring maximum throughput and reducing CPU load on the host device.
-
Custom Post-Processing:
Implemented complex post-processing logic in C++ to handle the Hailo NPU's raw tensor output.
This includes custom decoding for segmentation masks, matrix multiplication for mask assembly,
and efficient Non-Maximum Suppression (
NMS). -
Hardware Integration:
Designed the system to utilise the PCIe bandwidth of the Raspberry Pi 5 effectively, managing
data flow between the host memory (
DDR) and the NPU's internalSRAMto bypass hardware limitations in specific neural network layers.
This is a project currently in active development so make sure to check the devlogs to stay updated!