In this article by Lentin Joseph, the author of the book ROS Robotic Projects, we learn how one of the capability in most of the service and social robots is face detection and tracking. The robot can identify faces and it can move its head according to the human face move around it. There are numerous implementation of face detection and tracking system in web. Most of the trackers are having a pan and tilt mechanism and a camera is mounted on the top of the servos. In this article, we are going to see a simple tracker which is having only pan mechanism. We are going to use a USB webcam which is mounted on AX-12 Dynamixel servo.

(For more resources related to this topic, see here.)

You can see following topics on this article:

• Overview of the project
• Hardware and software prerequisites

Overview of the project

The aim of the project is to build a simple face tracker which can track face only in the horizontal axis of camera. The tracker is having a webcam, Dynamixel servo called AX-12 and a supporting bracket to mount camera on the servo. The servo tracker will follow the face until it align to the center of the image which is getting from webcam. Once it reaches the center, it will stop and wait for the face movement. The face detection is done using OpenCV and ROS interface, and controlling the servo is done using Dynamixel motor driver in ROS.

We are going to create two ROS packages for this complete tracking system, one is for face detection and finding centroid of face and next is for sending commands to servo to track the face using the centroid values.

Ok!! Let's start discussing the hardware and software prerequisites of this project.

Hardware and software prerequisites

Following table of hardware components which can be used for building this project. You can also see a rough price of each component and purchase link of the same.

List of hardware components:

The URLs and price can vary. If the links are not available, you can do a google search may do the job. The shipping charges and tax are excluded from the price.

If you are thinking that, the total cost is not affordable for you, then there are cheap alternatives to do this project too. The main heart of this project is Dynamixel servo. We may can replace this servo with RC servos which only cost around $10 and using an Arduino board cost around $20 can be used to control the servo too, so you may can think about porting the face tracker project work using Arduino and RC servo

Ok, let's look on to the software prerequisites of the project. The prerequisites include ROS framework, OS version and ROS packages:

The above table will gives you an idea about which all are the software we are going to be used for this project. We may need both Windows and Ubuntu for doing this project. It will be great if you have dual operating system on your computer

Let's see how to install these software first

Installing dependent ROS packages

We have already installed and configured Ubuntu 16.04 and ROS Kinetic on it. Here are the dependent packages we need to install for this project.

Installing usb_cam ROS package

Let's see the use of usb_cam package in ROS first. The usb_cam package is ROS driver for Video4Linux (V4L) USB camera. The V4L is a collection of devices drivers in Linux for real time video capture from webcams. The usb_cam ROS package work using the V4L devices and publish the video stream from devices as ROS image messages. We can subscribe it and do our own processing using it. The official ROS page of this package is given in the above table. You may can check this page for different settings and configuration this package offers.

Creating ROS workspace for dependencies

Before starting installing usb_cam package, let's create a ROS workspace for keeping the dependencies of the entire projects mentioned in the book. We may can create another workspace for keeping the project code.

Create a ROS workspace called ros_project_dependencies_ws in home folder. Clone the usb_cam package into the src folder:

$ git clone https://github.com/bosch-ros-pkg/usb_cam.git

Build the workspace using catkin_make

After building the package, install v4l-util Ubuntu package. It is a collection of command line V4L utilities which is using by usb_cam package:

$ sudo apt-get install v4l-utils

Configuring webcam on Ubuntu 16.04

After installing these two, we can connect the webcam to PC to check it properly detected in our PC.

Take a terminal and execute dmesg command to check the kernel logs. If your camera is detected in Linux, it may give logs like this{

$ dmesg

Kernels logs of webcam device

You can use any webcam which has driver support in Linux. In this project, iBall Face2Face (http://www.iball.co.in/Product/Face2Face-C8-0--Rev-3-0-/90) webcam is used for tracking. You can also go for a popular webcam which is mentioned as a hardware prerequisite. You can opt that for better performance and tracking.

If our webcam has support in Ubuntu, we may can open the video device using a tool called cheese. Cheese is simply a webcam viewer.

Enter the command cheese in the terminal, if it is not available you can install it using following command:

$ sudo apt-get install cheese

If the driver and device are proper, you may get the video stream from webcam like this:

Webcam video streaming using cheese

Congratulation!!, your webcam is working well in Ubuntu, but are we done with everything? No. The next thing is to test the ROS usb_cam package. We have to make sure that is working well in ROS!!

Interfacing Webcam to ROS

Let's test the webcam using usb_cam package. The following command is used to launch the usb_cam nodes to display images from webcam and publishing ROS image topics at the same time:

$ roslaunch usb_cam usb_cam-test.launch

If everything works fine, you will get the image stream and logs in the terminal as shown below:

Working of usb_cam package in ROS

The image is displayed using image_view package in ROS, which is subscribing the topic called /usb_cam/image_raw

Here are the topics, that usb_cam node is publishing:

Figure 4: The topics publishing by usb_cam node

We have just done with interfacing a webcam in ROS. So what's next? We have to interface AX-12 Dynamixel servo to ROS. Before proceeding to interfacing, we have to do something to configure this servo.

Next we are going to see how to configure a Dynamixel servo AX-12A.

Configuring a Dynamixel servo using RoboPlus

The configuring of Dynamixel servo can be done using a software called RoboPlus providing by ROBOTIS INC (http://en.robotis.com/index/), the manufacturer of Dynamixel servos.

For configuring Dynamixel, you have to switch your operating system to Windows. The tool RoboPlus will work on Windows. In this project, we are going to configure the servo in Windows 7.

Here is the link to download RoboPlus:

http://www.robotis.com/download/software/RoboPlusWeb%28v1.1.3.0%29.exe.

If the link is not working, you can just search in google to get the RoboPlus 1.1.3 version. After installing the software, you will get the following window, navigate to Expert tab in the software for getting the application for configuring Dynamixel:

Dynamixel Manager in RoboPlus

Before taking the Dynamixel Wizard and do configuring, we have to connect the Dynamixel and properly powered. Following image of AX-12A servo that we are using for this project and its pin connection.

AX-12A Dynamixel and its connection diagram

Unlike other RC servos, AX-12 is an intelligent actuator which is having a microcontroller which can monitoring every parameters of servo and customize all the servo parameters. It is having a geared drive and the output of the servo is connected to servo horn. We may can connect any links on this servo horn. There are two connection ports behind each servo. Each port is having pins such as VCC, GND and Data. The ports of Dynamixel are daisy chained so that we can connect another servo from one servo. Here is the connection diagram of Dynamixel with PC.

AX-12A Dynamixel and its connection diagram

The main hardware component which interfacing Dynamixel to PC is called USB to Dynamixel. This is a USB to serial adapter which can convert USB to RS232, RS 484 and TTL. In AX-12 motors, the data communication is using TTL. From the Figure AX 12A Dynamixel and its connection diagram, we can seen that there are three pins in each port. The data pin is used to send and receive from AX-12 and power pins are used to power the servo. The input voltage range of AX-12A Dynamixel is from 9V to 12V. The second port in each Dynamixel can be used for daisy chaining. We can connect up to 254 servos using this chaining

Official links of AX-12A servo and USB to Dynamixel

AX-12A: http://www.trossenrobotics.com/dynamixel-ax-12-robot-actuator.aspx

USB to Dynamixel: http://www.trossenrobotics.com/robotis-bioloid-usb2dynamixel.aspx

For working with Dynamixel, we should know some more things. Let's have a look on some of the important specification of AX-12A servo. The specifications are taken from the servo manual.

Figure 8: AX-12A Specification

The Dynamixel servos can communicate to PC to a maximum speed of 1 Mbps. It can also give feedback of various parameters such as its position, temperature and current load. Unlike RC servos, this can rotate up to 300 degrees and communication is mainly using digital packets.

Powering and connecting Dynamixel to PC

Now we are going to connect Dynamixel to PC. Given below a standard way of connecting Dynamixel to PC:

Connecting Dynamixel to PC

The three pin cable can be first connected to any of the port of AX-12 and other side have to connect to the way to connect 6 port power hub. From the 6-port power hub, connect another cable to the USB to Dynamixel. We have to select the switch of USB to Dynamixel to TTL mode. The power can be either be connected through a 12V adapter or through battery. The 12V adapter is having 2.1X5.5 female barrel jack, so you should check the specification of male adapter plug while purchasing.

Setting USB to Dynamixel driver on PC

As we have already discussed the USB to Dynamixel adapter is a USB to serial convertor, which is having an FTDI chip (http://www.ftdichip.com/) on it. We have to install a proper FTDI driver on the PC for detecting the device. The driver may need for Windows but not for Linux, because FTDI drivers are built in the Linux kernel. If you install the RoboPlus software, the driver may be already installed along with it. If it is not, you can manually install from the RoboPlus installation folder.

Plug the USB to Dynamixel to the Windows PC, and check the device manager. (Right click on My Computer | Properties | Device Manager). If the device is properly detected, you can see like following figure:

Figure 10: COM Port of USB to Dynamixel

If you are getting a COM port for USB to Dynamixel, then you can start the Dynamixel Manager from RoboPlus. You can connect to the serial port number from the list and click the Search button to scan for Dynamixel as shown in following figure.

Select the COM port from the list and connecting to the port is marked as 1. After connecting to the COM port, select the default baud rate as 1 Mbps and click the Start searching button:

COM Port of USB to Dynamixel

If you are getting a list of servo in the left side panel, it means that your PC have detected a Dynamixel servo. If the servo is not detecting, you can do following steps to debug:

Make sure that supply is proper and connections are proper using a multi meter. Make sure that servo LED on the back is blinking when power on. If it is not coming, it can be a problem with servo or power supply.

Upgrade the firmware of servo using Dynamixel Manager from the option marked as 6. The wizard is shown in the following figure. During wizard, you may need power off the supply and ON it again for detecting the servo.

After detecting the servo, you have to select the servo model and install the new firmware. This may help to detect the servo in the Dynamixel manager if the existing servo firmware is out dated.

Dynamixel recovery wizard

If the servos are listing on the Dynamixel manager, click on a servo and you can see its complete configuration. We have to modify some values inside the configurations for our current face tracker project. Here are the parameters:

• ID : Set the ID as 1
• Baud rate: 1
• Moving Speed: 100
• Goal Position: 512

The modified servo settings are shown in the following figure:

Modified Dynamixel firmware settings

After doing these settings, you can check the servo is working good or not by changing its Goal position.

Yes!! Now you are done with Dynamixel configuration, Congratulation!! What's next? We want to interface Dynamixel to ROS.

Summary

This article was about building a face tracker using webcam and Dynamixel motor. The software we have used was ROS and OpenCV. Initially you can see how to configure the webcam and Dynamixel motor and after configuring, we were trying to build two package for tracking. One package does the face detection and second package is a controller which can send position command to Dynamixel to track the face. We have discussed the use of all files inside the packages and did a final run to show the complete working of the system.

Resources for Article:

Further resources on this subject:

• Using ROS with UAVs [article]
• Hardware Overview [article]
• Arduino Development [article]