Object Detection in the Formula Student Driverless

Formula Student and Formula Student Driverless

The Formula Student is an international engineering and design competition where student teams from all over the world have the task to build a one-seated racecar with either an electric motor or a combustion engine. The teams compete against in international competitions in different disciplines and classes. There is one class for electric vehicles, one class for combustion vehicles and a class for driverless vehicles, where there is no separation between the engine type. Within the different disciplines in a competition, teams can gain a maximum of 1000 points - the team with the highest points wins the whole competition, though there are winners in each discipline. The disciplines are divided into static and dynamic events: in the static events, the teams have to present and defend their car, design, costs and businessplan and get judged by a jury from the (automotive) industry. In the dynamic events, the racecars have to drive different tracks to show the ability for acceleration, performance, reliability and driveability. The tracks are always marked with cones where each misplaced or dropped cone gives penalty seconds. For safety reasons, the cars are driving against time and there are no more than 5 cars on the track at the same time.

In Formula Student Driverless, the cars have to show their combined performance of their driving strengths and the autonomous system in three disciplines: acceleration, skidpad and trackdrive. For acceleration, the cars have to drive a 75 m straight, where the car with fastest time for this distance is the winner. In skidpad, the vehicles must drive two circles: first, they have to go clockwise two times, and then counterclockwise twice. In trackdrive, a handling course with a combination of straights and curves has to be driven for 10 laps. The cars have to stop by themselves afterwards.

For perceiving the track, the borders are marked with yellow cones on the right and blue cones on the left and sometimes there is a painted line on the ground. The starting and stop zones are marked with bigger orange cones. The challenge now is to detect and track these cones to stay within the track limits and find the way through the course.

Developing a concept

The autonomous system needs to perceive the track so that it can drive through safely without hitting one or more cones. As the only consistent marking of the track limits are the different colored cones, a vision and sensor system has to be used.

With the help of Basler, we selected the Basler Dart daA1600-60uc for the first development steps for the vision system. Most FSD teams use more than one camera, but our first goal is to gather data and develop a well thought concept to participate in FSD within the next 2 years.

After selecting the camera, the first step was to find a good place and angle for the camera to be mounted and design the mounting. After some tests on the vehicle, it was decided that the camera does not need any upwards or downwards facing angle and that it would be best to mount it just beneath the TSAL (Tractive System Active Light) on the mainhoop. The mounting then was developed within the CAD model of our racecar and was then 3D-printed in cooperation with BAHSYS.

In order to perceive the track, the racecar has to detect and track the cones on the track limits. This is a typical task for object detection with a neural network: a neural net gets trained on many pictures of how the cones are looking in different situations and can then detect the coordinates of those cones in an image. Combined with other sensors like LIDAR or Radar, the distance to the cones can be measured and the racecar can perceive the track limits.

Challenges faced

Due to a constant issue with the planetary gears in the 2018 racecar, the car did not drive many meters this year. That lead to the problem that the camera could not be tested many times on the car but rather it had to be used “by feet” while walking through the track. With the limitation of not having the exact height and angle as on the racecar, we gathered a lot of video footage from test tracks with different weather conditions.

Future tasks and outlook

Right now, the BRS Motorsport team is still focusing on the Formula Student Electric class for 2019 and plans to participate in the FSD class in 2020 with a second vehicle. The FSD vehicle will be an older racecar that will be equipped with the autonomous system as well as the necessary hardware like pedal actuators, the special emergency braking and shutoff system, the required sensors and the vision system. To reach this goal, the electric vehicle from 2019 will serve as a test system for the vision system and the software. With the gathered data from the 2019 season, the neural net can be trained and the control system can be implemented. This is especially important as the Formula Student Germany decided to skip the driverless class from 2021 on and make the acceleration discipline driverless, so that every vehicle needs to have a driverless system installed, if they want to get points for all disciplines.