QCraft 轻舟智航
Oct 2022 – presentModel Quantization & Deployment Lead, Perception ML-Infra
Joined as Machine Learning Infrastructure Engineer; promoted to L5 in 2024; team lead since 2025.
Float-consistent quantization: four generations of method
- Problem: Production tolerates almost no regression (an AEB target detected one frame late is a failure), yet QAT trained on the task loss alone shifts the precision/recall operating point, so what ships is effectively a retrained model.
- Scale-search PTQ (2023, patent filed): searches quantization parameters in stages against cosine similarity to the float outputs: calibration method × parameter grid → layer-wise backward/forward search → layer-wise quantization. On a traffic-light detector, default calibration gave mAP 0.747 and layer-wise quantization restored it to 0.803 (float 0.800).
- LSQ-based AutoQuant (2024) for CNN + Transformer hybrids (vector map, temporal tracking): learns step sizes, constrains weight drift with AdaRound-style rounding, and supervises with float intermediate features, with cross-layer equalization (CLE) and LayerNorm–Linear fusion applied first.
- Distillation QAT → large-scale QAD (2025–2026): float model as teacher, PTQ model as the start, task loss plus feature/output distillation. On the end-to-end planner, single-variable experiments (25 key metrics, 388 shared scalars) showed that scene-boundary state reset, an independent teacher history and late-stage weight anchoring are all required.
- Result: Quantization loss on five production perception models fell from about 2 points to about 0.5; the quantized end-to-end planner went to on-vehicle testing; model owners have started running auto-QAT releases themselves.
Quantization error accumulation in a prediction-planning model (QuestNet v3)
- Problem: After quantization the model produced abnormal trajectories; PTQ and standard QAT both failed and a production delivery was blocked.
- Per-layer and cumulative error analysis showed the error compounding through the deep layers; sensitive and robust layers are treated differently. Experiments ruled out “freeze some layers and fine-tune” in favor of two-stage quantization with full QAT.
- Drawing on the chip's operator characteristics, moved the whole pipeline to INT16 for numeric headroom against the accumulation.
- Result: Trajectory anomalies eliminated; merged to mainline and release-validated. “Mid-fusion models need full QAT” is now team practice.
Fixed-point numerical correctness: adaptive LUTs and layered alignment
- Adaptive lookup tables: on the NPU, rsqrt, reciprocal, exp and sigmoid run as fixed-point piecewise LUTs. The algorithm measures error on the chip's actual fixed-point output codes (not a float approximation), keeps the original 6-table boundaries, recursively bisects the worst segment, and blocks export if the budget cannot meet the bound. Across 600 LUT modules in four production graphs: 0.217 LSB mean, 2 LSB max (uniform 126-table: 11,381 LSB max), with 85% fewer tables.
- Layered alignment: Float → QAT → in-house IR → LibTorch → HBIR → HBM → on-board post-processing, each stage compared only with the previous one on identical inputs, so a mismatch has exactly one possible source. Pinned down an rsqrt LUT split point in a norm layer, an upsample align_corners mismatch and pose-precision bugs.
- Release gates: results graded by evidence level (source / build / simulation / board / vehicle), default-off feature flags, and a regression set of past quantization failures re-run before every release; caught issues pre-release on three automaker programs.
Toolchain and platform: from one-click conversion to a quantization-deployment platform
- QNN Web, a one-click multi-chip conversion service (gRPC front/back end, configs generated from the ONNX graph, TensorRT timing cache; later a job database, queueing and MCP endpoints for AI agents): 4,273 production conversions, build time from 30–60 min to about 5, about 80% less manual effort.
- Split multi-chip quantization into a generic layer, an AI auto-adaptation layer and a platform layer; per-model adaptation went from about 2 weeks to 3–5 days. Led the team onto Qualcomm (model splitting and latency tuning; inference latency cut by more than 50%), TensorRT QDQ INT8 and VLM quantization on Horizon J6P.
- Since Nov 2026, leading the target architecture of a platform-style quantization-deployment framework (design draft, roadmap to Q2 2027): fixed-point simulation as the single source of truth, seven versioned contracts (artifact identity, IR metadata, IO contract, golden IO, precision profile, target capability, thresholds) across the training, conversion and on-vehicle codebases, blocking gates at every hop, and a sensitivity × latency-cost loop that plans mixed precision.
Latency modeling, model architecture and training efficiency
- Structural latency model: predicts on-board latency from module structure; the occupancy model was predicted at 19.38 ms and measured at 19.25 ms (0.7% error). It ruled out shipping “wider range + 10 Hz” before any code was written and confirmed that MapFormer query merging saves 2.1–3.3 ms.
- OneModel phase 2 (in progress): one shared image backbone and neck for five perception tasks and 9 cameras, estimated to save 5–9 ms per frame. Backbone sized by PCA redundancy analysis of the five tasks' checkpoint features; pretraining weights synthesized by channel alignment plus a closed-form linear solve; quantization-friendly ReLU6 with surrogate gradients; receptive-field center offset fixed.
- Migrated BEV occupancy training to PyTorch 2.x: 1.56 s → 0.71 s per iteration (−54%; V100, batch 16, AMP), 9% less GPU memory, scaling benchmarked from 1 to 112 GPUs.
- End-to-end driving model architecture (design proposal, not yet trained): a camera-first, token-based design: 9 cameras tokenized through the shared encoder, a streaming transformer with pose-aware memory, a multi-mode plan head parameterized by curvature and acceleration knots, with a distill-then-INT8 path designed in from the start.
Team leadership and program delivery
- Lead a 6-engineer quantization and deployment team (dotted-line from 2025, direct reports from 2026): an owner and a backup for each of about 19 production models, quarterly goals with P0/P1 priorities, work assigned by urgency × difficulty. In Q2 2026, with four production programs in parallel on a two-week release cadence, no release slipped because of quantization.
- Technical lead since June 2026 for the AXERA chip program (validated on the M97 700-TOPS dev board; target chip M95, 350 TOPS): staged bring-up plan, model adaptation and quantization, road-test issue triage. The urban NOA v4 stack runs on test vehicles, with no frame drops on key models.
- AI-assisted automation: a Feishu bot plus coding-agent skills adapt new model features for quantization (60 person-day project). Maintain a 33-page knowledge base, run monthly quantization sessions and coach engineers to independent model ownership.