New High-Speed Method Can Detect, Classify Objects Without Images - How?

Scientists have developed a new high-speed method that can detect, classify and locate multiple objects without the need for images or complex scene reconstruction. But how is this possible?

The innovative approach is ideal for spotting hazards while driving and is expected to solve existing problems of heavy communication load, high computing overhead, and many more.

SPOD Technique

The team, led by Liheng Bian from the Beijing Institute of Technology, developed a technique called image-free single-pixel object detection (SPOD).

The SPOD technique uses a single-pixel detector that enables multi-object detection directly from a small number of 2D measurements.

Using a small yet optimized structured light pattern, the team swiftly scanned the entire scene to gather 2D measurements, which were then processed by a transformer-based encoder, a type of deep learning model that extracted high-dimensional features from the scene.

These features were then sent to a multi-scale attention network-based decoder, which simultaneously provided information on the class, location, and size of all targets in the scene.

"Compared to the full-size pattern used by other single-pixel detection methods, the small, optimized pattern produces better image-free sensing performance," group member Lintao Peng said in a press release statement.

"Also, the multi-scale attention network in the SPOD decoder reinforces the network's attention to the target area in the scene. This allows more efficient extraction of scene features, enabling state-of-the-art object detection performance."

The SPOD method can perform all three visual perception tasks of classification, single object recognition, and tracking simultaneously while achieving an object detection accuracy slightly above 80%.

"For autonomous driving, SPOD could be used with lidar to help improve scene reconstruction speed and object detection accuracy. We believe that it has a high enough detection rate and accuracy for autonomous driving while also reducing the transmission bandwidth and computing resource requirements needed for object detection," said Bian.

Proof-of-Concept

The researchers created a proof-of-concept setup to demonstrate SPOD, where they used images randomly chosen from the Pascal Voc 2012 test dataset and printed them on film to use as target scenes. With a sampling rate of 5%, SPOD completed spatial light modulation and image-free object detection per scene in an average time of just 0.016 seconds.

However, SPOD's object detection capability is still limited as it can only detect the 80 categories available in the existing object detection dataset used to train the model, as stated by Peng.

But the pre-trained model can be fine-tuned to accomplish image-free multi-object detection of new target classes, such as pedestrian, vehicle, or boat detection, to fulfill specific requirements.

The researchers aim to expand the image-free perception technology to various detectors and computational acquisition systems to attain reconstruction-free sensing technology.

The findings of the work were published in the journal Optics Letters.