We will build an autonomous robot. Captured by robot's camera, video stream will be analyzed by a neural network. The network will be running on the onboard Raspberry Pi that will steer the robot. Before you start with the project, I want you to answer two questions. First, how everything will be attached mechanically? Second, what will be the energy source? While your autonomous robot can work from a cardboard box, having a mechanically sound chassis will greatly improve the result on the AI training stage. And the question of the energy source is essential for any autonomous robot. Ready to answer? Then let's go!
Parts And Tools
For this project I used:
- Raspberry Pi 3B, 1 GB RAM (priceless 😉)
- Power bank (for mobile phone)
- USB to micro-USB cable
- SD memory card class 10, at least 16 GB, $11.99
- Chassis + 2 DC motors $13.99
- Raspberry Pi camera $17.21
- L298N dual H-bridge controller $6.89
- 9 V battery (local store)
- Jumper wires
- Something to attach the components to chassis (I used rubber bands and adhesive putty)
- Enclosure for the electronics
- Soldering iron $20.07
- A laptop with WiFi and SSH client (for robot training)
- Multimeter (I didn't have one at hand, but it would have been useful)
- Small screw driver
- Wire strippers (optional)
- Breadboard (optional)
Note that the prices may vary and they are here for indicative budget only. You can also order similar parts on AliExpress or other sites. The bottom line is that you can easily build a self-driving robot under $100.
I decided to power the Raspberry Pi board and motors separately so that the board will be reliably powered, even when the motors battery exhausts itself. The power bank for that purpose (originally, for a mobile phone) and a USB cable I already had at home, so they didn't count towards the budget.
I recommend at least 16 GB SD memory card so that you can install the Raspberry Pi OS and have some storage space for training data acquisition (images from the camera). Regarding camera itself, usually it is recommended to use a wide-angle camera. However, feel free to use any camera you have (as I did).
The H-bridge circuit is useful to allow DC motors to run forwards or backwards. Unlike some tutorials, mine H-bridge required between 7 V and 35 V driving voltage. To see if the circuit is working, check if the red LED is on. With 6 V driving voltage, it was still off. Therefore, I used a 9 V battery to power the motors. Finally, using a laptop with Linux will make your life easier. But feel free to use whatever you have.
Part I: Let The Hack Begin
Assemble The Robot
1. Assemble the chassis.
Attach the wheels, the ball caster, and the motors. Don't mind the battery holder, I didn't use it in the end.
2. Solder wires to the motors.
3. Connect motors to the H-bridge controller.
Here you will need a small screw driver.
4. Connect the motors controller to the Raspberry Pi.
Here I use a breadboard in-between, but feel free to connect the jumper wires directly. You can choose any free (non-specialized) GPIOs pins. Please refer to the board's pinout.
For instance, I have chosen pins
31. Note those
are Raspberry Pi pins, not GPIO numbers. For example, the above pins
correspond to GPIO 22, GPIO 17, GPIO 27, and GPIO 6. Remember those magic
Please do not confuse pins and GPIO numbers. The
gpiozeroPython module refers to GPIO numbers, whereas the software that we will use later requires Raspberry Pi pins.
To determine the order in which you connect the jumper wires to the controller
you may want to check the controller's data sheet. Or determine the pins
experimentally, testing the digital inputs in isolation. There are four
wires: two controlling the left wheel (
two for the right wheel control (
For example, when both
IN3 digital signals are high,
the robot will move forward.
Note that you will have to connect the ground (GND) of the controller board and the ground of Raspberry. Otherwise, your controller may not be able to interpret the digital signal it receives from the Raspberry.
5. Power up the motors controller with a 9 V battery.
I didn't have a battery connector at hand, so I used a chewing gum to attach the wires to the battery.
6. Power up the Raspberry Pi.
Finally, power up the Raspberry Pi with a power bank using a small USB cable.
You may want to assemble the modules neatly so that the robot is actually autonomous. Make sure that the hardware is fixed well and will not move during the ride.
Let us test what robot we have built so far! First, we connect to the
Raspberry Pi. This is done via SSH.
Setup Raspberry Pi WiFi connection
and connect to your board.
Now, we can play with the robot interactively.
Being connected to the Raspberry Pi via SSH, type
python to open up the Python interpreter.
Then define your robot object
from gpiozero import Robot robot = Robot(left = (22, 17), right = (27, 6))
Note the GPIO numbers we have discussed before. Those are your connections to the bridge controller. To make the robot do something
0.4 is the speed with which to rotate the left wheel.
To stop the robot, issue
exit() (or press Ctrl-D) to exit the Python's interpreter's shell. In
your favourite text editor you can create a sequence of actions to pre-program
from gpiozero import Robot from time import sleep robot = Robot(left = (22, 17), right = (27, 6)) print("Left") robot.left(0.4) sleep(1) robot.stop() print("Right") robot.right(0.4) sleep(1) robot.stop() print("Forward") robot.forward(0.4) sleep(3) robot.stop()
Here we ask the robot to rotate left for a second,
then rotate right for a second, and finally, move
forward for three seconds.
Save it to a file on your Raspberry Pi, e.g.
Part II: Remote Control
Now that the robot is based on the Raspberry Pi, we can do amazing things with it. For example, we can benefit from the DonkeyCar, an open source platform to build a small scale self driving car. You will have to follow the instructions to install the necessary software on your Raspberry Pi.
As soon as you finish with installation, create your Donkeycar App. Connect to your Pi with SSH and type
donkey createcar --path ~/mycar
Then check the newly created project in
~/mycar. Thankfully, Donkey
supports our two-wheel robot almost out of the box. The remaining bits are to
DRIVE_TRAIN_TYPE = "DC_TWO_WHEEL"
This will select the appropriate motor's driver. Finally, set the pins as discussed before:
HBRIDGE_PIN_LEFT_FWD = 15 HBRIDGE_PIN_LEFT_BWD = 11 HBRIDGE_PIN_RIGHT_FWD = 13 HBRIDGE_PIN_RIGHT_BWD = 31
Now, test if everything was successful.
cd ~/mycar python manage.py drive
In under a minute, the Donkey server should be up
and running. As a result, you will be able
to access a control web interface
http://<your car's IP>:8887.
This will allow you to remotely control the car
and capture the video stream with its camera.
Like in this video.
Part III: Making It Self-Driving
Being able to control your car remotely is useful to collect the training data for the neural network. But first we may want to set up a driving track like this.
Now, drive several laps on your track. The captured image data are automatically
Having collected a sufficient amount of data
we pass to the driving model (neural network) training.
Here is how the world looks like from the car's viewpoint.
Green vector illustrates human-driver commands and the blue one is the model after training.
You want to revisit
~/mycar/config.py to set the neural network parameters,
The key training command is
python manage.py train --type linear --model models/mypilot.h5
You can experiment with any of the driving models:
More housekeeping details can be found in
this notebook adapted for TensorFlow 2.
The original version can be found
Et voilà, your self-driving car is ready!
Now, run the auto-pilot
screen cd ~/mycar python manage.py drive --model models/mypilot.h5
You can learn more about neural networks from my tutorials.
What Could Be Done Better
The mechanical part could be better. After I have mounted the acrylic chasssis, I have realized that the motors do not hold firmly. Moreover, there was a friction between one of the wheels and chassis. Therefore, I had to programmatically adjust the power to balance the wheels.
Moreover, I believe that having a streering wheel motor would be much more efficient, instead of differential drive, as they call this construction in the robotics community. To test in the future.
Likewise, it is important to keep your camera in the same position during training and during self-piloting. Also make sure that the battery is always charged.
It would be nice to have a 9 V battery connector, instead of a piece of putty. As simple as that.
Quality of the training data. The better is your driving strategy, the better training data you will produce, the better the car will learn its turns.
I would like to thank the folks from the Integnano group who gave me a Raspberry Pi as a birthday gift. You are amazing!
As always, if you have any questions, remarks, or spotted any typos please send me a message.