Deep learning applications

Vulnerable Road User detection in FMCW Radar

A "fully convolutional" neural network model processes a tensor of arbitrary size to produce output with a corresponding size using inference and learning. Although computationally complex, a direct, full resolution, deep network has been proven to be very efficient for tasks such as semantical segmentation and labeling. Inspired by standard computer vision techniques that use multi-resolution image representation, the recent trend in generative CNN design revolves around the multi-scale approaches such as the U-net.
In the proposed method, we focus on detecting moving pedestrians and cyclists from time series of Radar cubes. The novelty of our approach lies in the combined detection and classification design, where the network outputs a spatial probability of occupancy grid for the VRU object class. A combination of classical signal processing and AI is used to pre-process a 4-D Radar tensor free of static content, which is an input in a 3-D U-Net CNN. The CNN can then specialize to localize and separate the unique micro-Doppler signatures of moving people from other moving objets in the scene.

Pre-processing steps - Left: input range-azimuth-Doppler cube, center: power normalization, ego-velocity estimation and static content removal, right: tensor of temporally concatenatenated cubes.

In order to effectively learn the model coefficients, we employ a training strategy using weak supervision from other, more accurate sensors. During training, the scene is captured by both radar as well as camera and lidar. We run object detection on the camera frames, which we then match to objects in the lidar point cloud. The matched objects consist of image coordinates, ground plane position and detection confidence.

A traffic scene captured by camera, radar and lidar. Objects detected in the camera frame (green) are matched to lidar objects to generate the weak supervision training data (auto-labels).

Experiments show that such information can be used to effectively supervise the radar CNN training process. Our model trained using automatically labeled data is able to significantly outperform classical radar object detectors by as much as 16% AP.

For more details please refer to the conference paper: "Weakly supervised deep learning method for vulnerable road user detection in FMCW radar", The 23rd IEEE International Conference on Intelligent Transportation Systems September 20 – 23, 2020 Link

DEMO

Results obtained on a content-independent test set. Top: VRU occupancy using classical signal processing, bottom: VRU occupancy using the proposed method.