Deep learning applications
Vulnerable Road User detection in FMCW Radar
A "fully convolutional" neural network model processes a tensor of arbitrary size to produce output with a corresponding size using inference and learning. Although computationally complex, a direct, full resolution, deep network has been proven to be very efficient for tasks such as semantical segmentation and labeling.
Inspired by standard computer vision techniques that use multi-resolution image representation, the recent trend in generative CNN design revolves around the multi-scale approaches such as the U-net.
In the proposed method, we focus on detecting moving pedestrians and cyclists from time series of Radar cubes. The novelty of our approach lies in the combined detection and classification design, where the network outputs a spatial probability of occupancy grid for the VRU object class. A combination of classical signal processing and AI is used to pre-process a 4-D Radar tensor free of static content, which is an input in a 3-D U-Net CNN. The CNN can then specialize to localize and separate the unique micro-Doppler signatures of moving people from other moving objets in the scene.
Pre-processing steps - Left: input range-azimuth-Doppler cube, center: power normalization, ego-velocity estimation and static content removal, right: tensor of temporally concatenatenated cubes.
A traffic scene captured by camera, radar and lidar. Objects detected in the camera frame (green) are matched to lidar objects to generate the weak supervision training data (auto-labels).
For more details please refer to the conference paper: "Weakly supervised deep learning method for vulnerable road user detection in FMCW radar", The 23rd IEEE International Conference on Intelligent Transportation Systems September 20 – 23, 2020 Link
Results obtained on a content-independent test set. Top: VRU occupancy using classical signal processing, bottom: VRU occupancy using the proposed method.