AR navigation perception SDK
A perception SDK for AR navigation on phones and car head units: a multi-stream parallel framework with automatic profiling. 12 features delivered in four months with CPU usage under 70%, shipped on the JMC CX743 production vehicle.
12perception features in four months
< 70%CPU usage on the head unit
1production vehicle launch (JMC CX743)
On this page
The problem
AR navigation overlays guidance on the live camera view, so perception has to run in real time on hardware that is also running navigation, maps and the car's own software. A phone or head-unit CPU has little headroom. A feature that is accurate but late is useless, and an SDK that starves the rest of the system will not ship.
Design
Camera frames
Parallel perception streamsroad, parking slots, …
Results to navigation
AR guidance
- Multi-stream parallel framework. Perception tasks run as parallel streams, so a slow task does not hold up the others and new tasks can be added without rewriting the pipeline.
- Automatic profiling built in. Every stream reports its timing and CPU cost continuously, so performance regressions show up during development, not in the car.
- On-device models deployed and tuned for the target hardware.
Results
Visual positioning SDK
My work also included the SDK for the visual positioning system used alongside AR navigation.

Lessons
- Budget the CPU first, then the features. Real-time on shared hardware is a scheduling problem as much as a model problem.
- Make performance visible all the time. Built-in profiling caught regressions long before integration.