Making Your Self-Driving Robot

We will build an autonomous robot. Captured by robot's camera, video stream will be analyzed by a neural network. The network will be running on the onboard Raspberry Pi that will steer the robot. Before you start with the project, I want you to answer two questions. First, how everything will be attached mechanically? Second, what will be the energy source? While your autonomous robot can work from a cardboard box, having a mechanically sound chassis will greatly improve the result on the AI training stage. And the question of the energy source is essential for any autonomous robot. Ready to answer? Then let's go!

Parts And Tools

For this project I used:

  • Raspberry Pi 3B, 1 GB RAM (priceless 😉)
  • Power bank (for mobile phone)
  • USB to micro-USB cable
  • SD memory card class 10, at least 16 GB, $11.99
  • Chassis + 2 DC motors $13.99
  • Raspberry Pi camera $17.21
  • L298N dual H-bridge controller $6.89
  • 9 V battery (local store)
  • Jumper wires
  • Something to attach the components to chassis (I used rubber bands and adhesive putty)
  • Enclosure for the electronics

Tools:

  • Soldering iron $20.07
  • A laptop with WiFi and SSH client (for robot training)
  • Multimeter (I didn't have one at hand, but it would have been useful)
  • Small screw driver
  • Wire strippers (optional)
  • Breadboard (optional)

Note that the prices may vary and they are here for indicative budget only. You can also order similar parts on AliExpress or other sites. The bottom line is that you can easily build a self-driving robot under $100.

I decided to power the Raspberry Pi board and motors separately so that the board will be reliably powered, even when the motors battery exhausts itself. The power bank for that purpose (originally, for a mobile phone) and a USB cable I already had at home, so they didn't count towards the budget.

I recommend at least 16 GB SD memory card so that you can install the Raspberry Pi OS and have some storage space for training data acquisition (images from the camera). Regarding camera itself, usually it is recommended to use a wide-angle camera. However, feel free to use any camera you have (as I did).

The H-bridge circuit is useful to allow DC motors to run forwards or backwards. Unlike some tutorials, mine H-bridge required between 7 V and 35 V driving voltage. To see if the circuit is working, check if the red LED is on. With 6 V driving voltage, it was still off. Therefore, I used a 9 V battery to power the motors. Finally, using a laptop with Linux will make your life easier. But feel free to use whatever you have.

Part I: Let The Hack Begin

Assemble The Robot

1. Assemble the chassis.

Attach the wheels, the ball caster, and the motors. Don't mind the battery holder, I didn't use it in the end.

2. Solder wires to the motors.

3. Connect motors to the H-bridge controller.

Here you will need a small screw driver.

4. Connect the motors controller to the Raspberry Pi.

Here I use a breadboard in-between, but feel free to connect the jumper wires directly. You can choose any free (non-specialized) GPIOs pins. Please refer to the board's pinout.

Raspberry Pi pinout. Source: [pinout.xyz](https://pinout.xyz/).

Raspberry Pi pinout. Source: pinout.xyz.

For instance, I have chosen pins 15, 11, 13, and 31. Note those are Raspberry Pi pins, not GPIO numbers. For example, the above pins correspond to GPIO 22, GPIO 17, GPIO 27, and GPIO 6. Remember those magic numbers.

Please do not confuse pins and GPIO numbers. The gpiozero Python module refers to GPIO numbers, whereas the software that we will use later requires Raspberry Pi pins.

To determine the order in which you connect the jumper wires to the controller you may want to check the controller's data sheet. Or determine the pins experimentally, testing the digital inputs in isolation. There are four wires: two controlling the left wheel (IN1 and IN2) and two for the right wheel control (IN3 and IN4). For example, when both IN1 and IN3 digital signals are high, the robot will move forward.

Note that you will have to connect the ground (GND) of the controller board and the ground of Raspberry. Otherwise, your controller may not be able to interpret the digital signal it receives from the Raspberry.

5. Power up the motors controller with a 9 V battery.

I didn't have a battery connector at hand, so I used a chewing gum to attach the wires to the battery.

6. Power up the Raspberry Pi.

Finally, power up the Raspberry Pi with a power bank using a small USB cable.

You may want to assemble the modules neatly so that the robot is actually autonomous. Make sure that the hardware is fixed well and will not move during the ride.

Sanity Check

Let us test what robot we have built so far! First, we connect to the Raspberry Pi. This is done via SSH. Setup Raspberry Pi WiFi connection and connect to your board. Now, we can play with the robot interactively. Being connected to the Raspberry Pi via SSH, type python to open up the Python interpreter. Then define your robot object

from gpiozero import Robot
robot = Robot(left = (22, 17), right = (27, 6))

Note the GPIO numbers we have discussed before. Those are your connections to the bridge controller. To make the robot do something

robot.left(0.4)

Here 0.4 is the speed with which to rotate the left wheel.

To stop the robot, issue

robot.stop()

Type exit() (or press Ctrl-D) to exit the Python's interpreter's shell. In your favourite text editor you can create a sequence of actions to pre-program your robot.

from gpiozero import Robot
from time import sleep
robot = Robot(left = (22, 17), right = (27, 6))

print("Left")
robot.left(0.4)
sleep(1)
robot.stop()

print("Right")
robot.right(0.4)
sleep(1)
robot.stop()

print("Forward")
robot.forward(0.4)
sleep(3)
robot.stop()

Here we ask the robot to rotate left for a second, then rotate right for a second, and finally, move forward for three seconds. Save it to a file on your Raspberry Pi, e.g. robo.py, and execute

python robo.py

If we have missed something, here is an official Raspberry Pi tutorial to assist with assembling your robot. Before you continue, now is a good time to attach the camera.

Part II: Remote Control

Now that the robot is based on the Raspberry Pi, we can do amazing things with it. For example, we can benefit from the DonkeyCar, an open source platform to build a small scale self driving car. You will have to follow the instructions to install the necessary software on your Raspberry Pi.

As soon as you finish with installation, create your Donkeycar App. Connect to your Pi with SSH and type

donkey createcar --path ~/mycar

Then check the newly created project in ~/mycar. Thankfully, Donkey supports our two-wheel robot almost out of the box. The remaining bits are to edit config.py. Set

DRIVE_TRAIN_TYPE = "DC_TWO_WHEEL"

This will select the appropriate motor's driver. Finally, set the pins as discussed before:

HBRIDGE_PIN_LEFT_FWD = 15
HBRIDGE_PIN_LEFT_BWD = 11
HBRIDGE_PIN_RIGHT_FWD = 13
HBRIDGE_PIN_RIGHT_BWD = 31

Now, test if everything was successful.

cd ~/mycar
python manage.py drive

In under a minute, the Donkey server should be up and running. As a result, you will be able to access a control web interface using http://<your car's IP>:8887. This will allow you to remotely control the car and capture the video stream with its camera. Like in this video.

Part III: Making It Self-Driving

Being able to control your car remotely is useful to collect the training data for the neural network. But first we may want to set up a driving track like this.

Now, drive several laps on your track. The captured image data are automatically stored in ~/mycar/data. Having collected a sufficient amount of data we pass to the driving model (neural network) training. Here is how the world looks like from the car's viewpoint.

Green vector illustrates human-driver commands and the blue one is the model after training.

You want to revisit ~/mycar/config.py to set the neural network parameters, e.g. LEARNING_RATE and BATCH_SIZE. The key training command is

python manage.py train --type linear --model models/mypilot.h5

You can experiment with any of the driving models: linear, categorical, rnn, imu, behavior, etc. More housekeeping details can be found in this notebook adapted for TensorFlow 2. The original version can be found here.

Et voilĂ , your self-driving car is ready!

Now, run the auto-pilot

screen
cd ~/mycar
python manage.py drive --model models/mypilot.h5

You can learn more about neural networks from my tutorials.

What Could Be Done Better

  1. The mechanical part could be better. After I have mounted the acrylic chasssis, I have realized that the motors do not hold firmly. Moreover, there was a friction between one of the wheels and chassis. Therefore, I had to programmatically adjust the power to balance the wheels.

  2. Moreover, I believe that having a streering wheel motor would be much more efficient, instead of differential drive, as they call this construction in the robotics community. To test in the future.

  3. Likewise, it is important to keep your camera in the same position during training and during self-piloting. Also make sure that the battery is always charged.

  4. It would be nice to have a 9 V battery connector, instead of a piece of putty. As simple as that.

  5. Quality of the training data. The better is your driving strategy, the better training data you will produce, the better the car will learn its turns.

  6. Transfer learning. You can pretrain your driving model in a virtual environment such as Donkey Gym. See also this video.


A robot from 70s I have seen at Musée des Arts et Métiers in Paris. The robot inspired me for this project.

A robot from 70s I have seen at Musée des Arts et Métiers in Paris. The robot inspired me for this project.

I would like to thank the folks from the Integnano group who gave me a Raspberry Pi as a birthday gift. You are amazing!


As always, if you have any questions, remarks, or spotted any typos please send me a message.

Learn More

Related