Modern cities rely on camera networks to monitor traffic, detect congestion, and respond to incidents. But most traditional systems that track vehicles across locations do so using license plate recognition which means collecting and processing personal data tied directly to vehicle owners. What if you could achieve the same goal without ever reading a single plate?
The problem: re-identifying vehicles without identifiers
Imagine a vehicle passing through an intersection monitored by Camera A, then appearing again ten minutes later under Camera B. A human observer would struggle to confidently re-identify it. An automated system relying on license plates could do it trivially at the cost of linking every observation to a registered owner. For privacy-conscious smart city deployments, that trade-off is increasingly unacceptable.
The alternative is appearance-based re-identification (ReID), training a neural network to extract a compact feature vector, essentially a visual fingerprint from a cropped vehicle image. Two images of the same vehicle should produce similar vectors, even when captured from different angles, in different lighting, or by different cameras. No plate, no personal data, no direct link to an owner.
A modular pipeline on the edge
Rather than processing video in the cloud, the system runs entirely on NVIDIA Jetson Orin edge devices deployed roadside, this design choice matters for both latency and privacy, as data stays local and no raw video leaves the device.
The pipeline is split into three independent modules: object detection, feature extraction, and database operations. Detection uses a YOLOv8 model combined with ByteTrack for multi-frame vehicle tracking. Once a vehicle is detected and tracked, a cropped image is passed to the feature extraction module, which generates a 256-dimensional embedding. That vector is then compared against previously stored embeddings in an OpenSearch vector database using cosine similarity. If the vehicle has been seen before, at this camera or another, it can be matched.
The modules communicate over MQTT, each running in its own Docker container. This separation was a deliberate engineering choice: different teams developed each module independently, and containerization resolved the dependency conflicts that inevitably arise on JetPack-based Jetson platforms.
Optimizing for the edge
Running deep learning inference on a resource-constrained device requires more than just porting a model. The feature extraction model was first converted from PyTorch to ONNX, then compiled to a TensorRT engine. The results were meaningful: average inference time per image dropped from roughly 40 ms (PyTorch) to 35 ms (ONNX) and finally to around 15 ms with TensorRT, a near 3x speedup. GPU memory usage fell from approximately 2 GB to just 0.2 GB. For a device that must simultaneously run detection, feature extraction, and database queries, that headroom matters.
A zone-based filtering mechanism further reduces unnecessary computation: feature vectors are only extracted when a tracked vehicle enters a new region of the camera’s field of view, avoiding redundant processing when a vehicle sits stationary or circles within a single zone. During one test run on the AICity dataset, 5771 detection messages were published, but only 473 were forwarded for full feature extraction after filtering. That approximately 12:1 reduction is exactly the kind of load management needed on constrained edge AI hardware.
Fisheye cameras: an open question
One emerging direction is the use of 360-degree fisheye cameras, which can cover wider areas with fewer units, an attractive alternative for intersections and parking environments. The distortion introduced by fisheye optics is significant, but initial experiments were cautiously encouraging, re-identification Rank-1 accuracy on synthetically distorted images remained close to the baseline (73.24% vs. 72.90%), and average cosine similarity in the embedding space showed only a minor shift. These are preliminary results on simulated distortion, and we are careful not to overstate them, but they suggest the feature extractor is reasonably robust to moderate geometric deformation.
Privacy by architecture
Perhaps the most important aspect of this work is what the system does not do. It does not read license plates. It does not upload video to a cloud server. All inference runs locally on the edge device, and the vector embeddings stored in the database carry no direct personal identifier. The architecture is designed so that even the meta-edge server which aggregates data from multiple roadside units never receives raw images.
This is a concrete example of privacy-by-design in a computer vision context delivering the same traffic monitoring capability, without the surveillance footprint.
What comes next
The current implementation is a proof of concept running on a single Jetson device. The next phase expands this to a distributed architecture with multiple roadside units feeding a shared meta-edge server via Kafka, with TLS-secured MQTT and role-based access controls throughout. The system also shares hardware with an air quality forecasting use case with ML model retraining scheduled overnight, when the re-identification pipeline is inactive (as this solution in current form can only operate at daytime), to avoid resource contention.
The research is ongoing, but the direction is clear: robust vehicle analytics, built from the ground up with privacy in mind.