Advanced

The advanced guide to Raspberry Pi Robotics


Learn the fundamentals of robotics - Part 2

The Robot revolution is coming. Robots are no longer just machines of science fiction, from self-driving cars, to flying drones, robots are on the march. Over the next few years robots are going to be seen all over the place and will be increasingly used in agriculture, manufacture, medicine, education, as well as in our own homes. The amazing thing is that now almost anyone can become a roboticist and if you have a Raspberry Pi you’re already half way there.



Content to come soon!


Speech Synthesis

There are a number of open-source methods and software solutions for giving your system text-to-speech (TTS) capabilities.

Read more

Speech Recognition

One very promising project I've discovered for robotics is called Jasper. This project claims that you can Control anything with your voice using this open source software! Citing their webpage: Jasper is an open source platform for developing always-on, voice-controlled applications.

Read more

Machine Vision

With the Raspberry Pi's HD camera module you have the power to capture visual data in a faster manner than using a usb webcam. Recently the excellent open source computer vision library OpenCV was implemented on the Raspberry Pi. This gives great processing capabilities for analysing visual data and opens up a world of possibilities and useful applications.

Read more

Start-up Scripts

This documents explains how to automatically start scripts on booting of the RPi using bashrc.

Read more

As you can see it's really simple to get started on robotics, all you need is a few basic materials, enthusiasm about the topic and some spare time. Nowadays everyone can became a roboticist.

Raspberry Pi Speech Synthesis

There are a number of open-source methods and software solutions for giving your system text-to-speech (TTS) capabilities. The best solution we found was using eSpeak as it had a nice range of voices and was not too processor intensive (good for the Raspberry Pi!) eSpeak is known as, a formant synthesis TTS engine which computes the pronunciation and intonation rather than using cut up bits of recorded voice like other systems use.



eSpeak is lightweight and portable, it is very well supported and has a stable release in the Raspian and Ubuntu repositories (no proprietary lock-ins or service charges). It provides a simple and straightforward command line interface that can be easily integrated into Python as well as other languages. It even allows us to record straight to a WAV sound file with a simple option in the command linel . Best of all it has a Stephen Hawking-like style that gave it a fitting dose of panache and smugness... it's also great fun finding out what it mispronounces!


There are a number of open-source methods and software solutions for giving your system text-to-speech (TTS) capabilities. Thee best solution we found was using eSpeak as it had a nice range of voices and was not too processor intensive (good for the Raspberry Pi!) eSpeak is known as, a formant synthesis TTS engine which computes the pronunciation and intonation rather than using cut up bits of recorded voice like other systems use. This makes it lightweight and portable. It is, very well supported and has a stable release in the Raspian and Ubuntu repositories (no proprietary lock-ins or service charges). It provides a simple and straightforward command line interface that can be easily integrated into Python as well as other languages. It even allows us to record straight to a WAV sound file with a simple option in the command linel . Best of all it has a Stephen Hawking-like style that gave it a fitting dose of panache and smugness... it's also great fun finding out what it mispronounces!


For getting started the best thing to do is to just try installing espeak and see if it works "out of the box". Don't forget to have an amplified speaker or headphones plugged in to your Raspberry Pi and switched on . For help in setting up audio or for debugging any audio problems you have please see this great post on Raspberry Pi Spy


To install eSpeak type on your command line:

$> sudo apt-get install espeak

This will install espeak and espeak-data packages. Then try issuing a command straight to espeak:

$> espeak "Hello, I'm sure we can be friends."

Small Tip: Be careful with punctuation - using an exclamation mark before the end quote (!") gets the bash shell all excited about special functions so if you want to leave on an exclamation mark, put a space between it and the end quote (! "), and of course watchout for quote marks within the string.


The first time eSpeak runs it will probably have a short delay before speaking which seems to disappear on subsequent executions. You will also probably get quite a long list of warnings about "ALSA lib", "server sockets" and the "jack server" - these seem pretty harmless and not to be worried about. The important thing is that it speaks to you! More details of TTS and working with eSpeak can be see here.

eSpeak has a very simple command line structure:

$> espeak [options] [string to say]

Running just ( $> espeak ) will allow you to type in things to say without having to run espeak each time - the lines are spoken when you press enter. To quit use Ctrl-C to break out of the program.

The options allow you to change the language, variation, pitch, speed, volume, stresses and many other parameters of the speech and even allow you to write out a WAV file (in which case it is not said out loud). For a list of all the options run the standard '-h' option ( $> espeak -h ) or look at the manual page ( $> man espeak ). For instance:

$> espeak -ven -s220 -p25 "Oh, you're playing with my voice..."

Sets the language (-v) to English (en), the speed (-s) to 220 words per minute (range of 80 to 450, default 175) and the pitch (-p) to 25 (range of 0 to 99, default 50).

Now that we can give a Raspberry Pi robot the power of speech what kind of things do you think it could be used for? May be it could let you know whenever you have a new email and read it out loud and I'm sure you can thinking of several other things. As a simple code example lets consider our PiBot's speech being triggered every time it bumps into something. Here we've added a hardware sensor that gets triggered every time it bumps into anything, from our PiBot python library we've exposed this event as a PiBot.isBumped function.

from espeak import espeak
import PiBot
import random def bumpReply(): if PiBot.isBumped(): # Check if PiBot has been bumped responses = ['Ouch, that hurt!', 'Watch where you are going, you almost hurt me', 'Ouch, be careful'] speak = random.choice(responses) # Get a random response to speak back to the user espeak.synth(speak) # Speak back to the user!

Raspberry Pi Speech Recognition

One very promising project I've discovered for robotics is called Jasper. This project claims that you can Control anything with your voice using this open source software! Citing their webpage: ‘Jasper is an open source platform for developing always-on, voice-controlled applications.’.


For years ‘voice control’ has been an aspirational technology for the world's most advanced (and expensive) robots and its remarkable that this now free software that makes this achievable with an inexpensive Raspberry Pi robot. As an example lets see how Jasper can be used in a robotic context. In this case we'll use Jasper to voice control the PiBots ‘dance’ command. This is to really just to illustrate the potential of Jasper in robotics. Equally you could use Jasper to voice command many other things that may be useful to you.


To get Jasper runnning in your projects, you'll need to head over to the Jasper Project and follow the install instructions there. We've loaded on of their fresh SD images for simplicity, and configured Jasper like the tutorial asks us to do, but with one small change; We've commented out the Crontab job, that is already in Crontab -e so we can boot Jasper ourself on when we want to.

#@reboot /home/pi/jasper/boot/boot.sh;

After you've configured Jasper, generated a user profile and restarted your Pi, we're ready to get Jasper running for the first time! Navigate to the boot file by typing this:

cd /jasper/boot/

Then we'll want to boot Jasper by running boot.sh:

./boot.sh

Jasper will load up, and you'll be able to see the progress of him loading him self. Once he's finished he'll ask how he can help you. Jasper works by identifying specific spoken words (trigger words) that can then activate an action (e.g. execute a python function). The spoken words that you want to use as triggers and given to Jasper through a string list. Each python script that you add for Jasper needs to contain a String List called WORDS and two functions, one named isValid() and handle(). The isValid() function relates to words being recognised, and the handle() function relates to actions that occur.

The WORDS string array holds the words that you want to extract from the speech. As an example lets choose the single word “dance”. We'll declare it like this:

WORDS = [“dance”]

We'll also want to set the priority this script has over other scripts, the higher the number the more important and further in front of other scripts with similar words it will be, we'll set ours at 1 for now. So a script with priority 0 that has “dance” in it's words array, will be behind our script for use.

PRIORITY = 1

The isValid() function checks the audio input (or more so the transcripted text input from Jaspers audio recognition engine) from the user, to tell Jasper this is the script you want. It's important that you included all the words. This will check the input from the user and return true if this script is related to the input text.

def isValid(text):
return bool(re.search(r'\bdance\b', text, re.IGNORECASE))

The handle() function will basically perform an action in relation to the input. Here is where Jasper will respond to his input. You'll need to pass input, mic and profile as variables which give you more options with Jasper. In this example I've just got Jasper to acknowledge that he's going to dance and then call a function from the PiBot script we have to get our PiBot dancing!

def isValid(text):
return bool(re.search(r'\bdance\b', text, re.IGNORECASE))

def handle(input, mic, profile):
mic.say(“Yeah, sure, watch these moves.”) # Tell the user to watch my dance moves
PiBot.dance # Make PiBot Dance


Full Script:

__author__ = 'alexgray'
import PiBot
import re
WORDS = ["dance"]
PRIORITY = 1 def is_valid(text): """ Returns True if the input is related to "dance". Arguments: text -- user-input, typically transcribed speech """ return bool(re.search(r'\bdance\b', text, re.IGNORECASE)) def handle(text, mic, profile): """ Makes PiBot dance Arguments: text -- user-input, typically transcribed speech mic -- used to interact with the user (for both input and output) profile -- contains information related to the user (e.g., phone number) """ line = "Watch these moves!" mic.say(line) # Tell the user to watch my dance moves PiBot.dance # Make PiBot Dance

Machine Vision

With the Raspberry Pi's HD camera module you have the power to capture visual data in a faster manner than using a usb webcam. Recently the excellent open source computer vision library OpenCV was implemented on the Raspberry Pi. This gives great processing capabilities for analysing visual data and opens up a world of possibilities and useful applications.



Machine vision is a powerful tool that allows for the automation of a lot of tasks that would normally require a human. This guide will go through how to setup your raspberry pi to take advantage of this tool.


Resources:

  • • Raspberry Pi
  • • Raspbian
  • • Ubuntu
  • • Raspberry Pi Camera or USB Webcam

Setup

As always, it is useful to make sure everything is up to date before beginning:

sudo apt-get update

There are two main ways to do machine vision with raspberry pi’s: you can run it locally, or you can run it remotely.


Locally

The advantage to running it locally on your raspberry pi is that it allows your system to be self-contained. The disadvantage is that machine vision can be a very resource intensive tool and could prevent you from doing much else on your raspberry pi. You will also be limited in the machine vision tools you can employ.


First you will need to install python opencv. Enter:

sudo apt-get install python-opencv

Now, at the top of any python code you write in which you wish to use opencv you will need to put:

Import cv2

It can also be useful to include:

import numpy as np

Face recognition using OpenCV

Facial recognition allows for a variety of interesting tasks to be performed by the raspberry pi. From advanced security systems to interesting human-robot interaction it opens the door to more advanced functionality.



Face recognition, blob tracking, motion detection and gesture mapping are all possible using OpenCV on the Raspberry Pi. It is of course very exciting to implement these things on a robot. For details on how to setup opencv on the raspberry pi please see the machine vision basics tab.


The main way to do facial and object recognition relies on trained classifiers for the facial feature/object you are trying to detect. There are two main ways to go about doing this: You can create your own (link “create your own” to the title below), or you can use a prebuilt one (link “prebuilt one” to the title below). The prebuilt ones are a good place to start and work fairly well so we will start with them.

Prebuilt resources:


The most commonly used prebuilt classifier is the haarcascade. To use it, define a variable like so:

face_cascade = cv2.CascadeClassifier('location_of_resource/haarcascade_frontalface_alt.xml')

where “location_of_resource” is the prebuilt classifiers location relative to the code that is calling it. To detect faces, convert the image being used to gray scale and then run:

faces = face_cascade.detectMultiScale(im_gray, 1.3, 5)

faces will contain two sets of coordinates for each face detected, the upper left corner and bottom right corner of the area detected as a face. A rectangle can be drawn around each face using: for (x,y,w,h) in faces:

cv2.rectangle(im_flip,(x,y),(x+w,y+h),(255,0,0),2)

and the center point of each face can be marked and recorded by expanding the for loop to:

faces_center=[]
for (x,y,w,h) in faces:
cv2.rectangle(im_flip,(x,y),(x+w,y+h),(255,0,0),2)
#find center point
faces_center.append(x+(w/2))
faces_center.append(y+(w/2))
cv2.circle(im_flip,(faces_center[0],faces_center[1]),2,(255,0,0),2)

You can now find each faces distance from the center of the image by first finding the center point of the image:

im_width, im_height = im_gray.shape[:2]

and then comparing the center point of the faces to the center point of the image. The below shows if statements that can inform the user of how to move to adjust the face to be in the center of the screen:

if faces_center: #check if faces_center has anything in it
#follow face

if im_width/2 > faces_center[0]:
print "go left"

if im_width/2 < faces_center[0]:
print "go right"

if im_height/2 > faces_center[0]:
print "go up"

if im_height/2 < faces_center[0]:
print "go down"

Each print statement could be replaced with a motor command to allow a robot to follow a face. If you want to see the output, don’t forget to include:

cv2.imshow( "Image", im_flip )

For playing around with the various classifiers, it can be useful to create a quick way of swapping between them. You can accomplish this with an array of if statements tied to key strokes:

def cascade_choice(choice = 1):
if choice == 1:
face_cascade = cv2.CascadeClassifier('cascade_resources/haarcascade_frontalface_alt.xml')
if choice == 2:
face_cascade = cv2.CascadeClassifier('cascade_resources/haarcascade_eye.xml')
if choice == 3:
face_cascade = cv2.CascadeClassifier('cascade_resources/haarcascade_smile.xml')
if choice == 4:
face_cascade = cv2.CascadeClassifier('cascade_resources/lbpcascade_frontalface.xml')
if choice == 5:
face_cascade = cv2.CascadeClassifier('cascade_resources/lbpcascade_profileface.xml')
if choice == 6:
face_cascade = cv2.CascadeClassifier('cascade_resources/lbpcascade_silverware.xml')
if choice == 7:
face_cascade = cv2.CascadeClassifier('cascade_resources/hogcascade_pedestrians.xml')
if choice == 8:
face_cascade = cv2.CascadeClassifier('cascade_resources/inria_caltech-17.01.2013.xml')
if choice == 8:
face_cascade = cv2.CascadeClassifier('cascade_resources/soft-cascade-17.12.2012.xml')
else:
print "invalid cascade selection"

Create your own If you want a more customized (but less generalized) object/facial recognition system you can build your own. To do this some helpful resources can be obtained from Adafruit. In order to be able to recognise a face we need a number of pictures of that person. We can do that with the Python capture-positives.py script. As this accesses the Raspberry Pi camera it needs to run as root:

sudo python capture-positives.py

Multiple images of the same face should be taken from different angles. We use these images to train the facial recognition model. This will take some minutes to finish. Enter:

python train.py

After this part we will have to adjust the config.py script in order to configure our servo motor's movement. The various options are detailed in the above link and will vary depending on your application. The final Python script that we need initialises the Raspberry Pi camera and performs the facial recognition. This version of the code is adapted from the box.py script from Tony Dicola's OpenCV Facial Recognition project on GitHub GitHub. This script detects a single face and is the code we need to run our Raspberry Pi in face recognition mode.


__author__ = 'alexgray adapted from Tony Dicola'
##Full soruce: https://github.com/tdicola/pi-facerec-box

import cv2
import config
import face

def init_camera():
camera = config.get_camera()
return camera

def face_detect(camera):
## Get image from camera
image = camera.read()
## Convert image to grayscale
image = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
## Get coordinates of single face in captured image
## Coordinates will mean a face was detected
result = face.detect_single(image)
## If no face, return False, otherwise return True, we found a face
if result is None:
return False
else:
return True

def main():
## Init our camera
camera = init_camera()
while True:
# If the camera see's a face, see if we know that face
if face_detect(camera):
print("Hi there, nice to meet you!")

Starting python scripts on startup

This documents explains how to automatically start scripts on booting of the RPi using bashrc.



First things first, we need to know where our Python script is, so we can call it, my script is kept in this directory: /home/pi/tiddly_bot/my_code.py


Copy your file path and then type the code below to open .bashrc file.

sudo nano .bashrc

Once you've done this the .bashrc file will open in a nano editor. Here you need to go to the bottom of the file to add your script to start up. Right at the bottom add this:

sudo python /home/pi/tiddly_bot/my_code.py

The only bit you will need to type is "sudo python" because we've already copied the file path, so we can just paste that in. Each time you start an instance of the command line, this script will run, so when we SSH into the pi we'll have our TiddlyBot script run that will let us use the TiddlyBot.

Similarly, we can add this script to Crontab instead of using .bashrc. You can get Crontab to do more elaborate recursive jobs, but for now we just want to run a single script at start-up. We'll need to open up Crontab by typing this into the command line:

crontab -e

We want our script to run on start-up so we'll add this line to the end of this file:

@reboot /home/pi/tiddly_bot/my_code.py &

This means at a reboot of the system, run the script in this location once.

If you are needing help before the articles are completed please see the useful links for some great resources on the web for learning more about various topics.

© 2014 The Pi Club - Raspberry Pi is a trademark of the Raspberry Pi foundation.