What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Processing 2: Creative Coding Hotshot

Chapter 1. Romeo and Juliet

Robots and performing arts share a long history. In fact, the word "Robot" was first coined in 1920 for a play by the Czech author Karel Čapek named "Rossum's Universal Robots". The play featured six robots, but since nobody was able to build a talking Robot at that time, humans had to play them. Times have changed a lot and we don't need humans to disguise themselves as robots anymore. For this project, we will do it the other way round and make some robots who play the humans. Unfortunately, "Rossum's Universal Robots" would require nine humans and six robots, so I chose a scene that's simpler to perform. We are going to build a pair of robots who play the humans in the famous balcony scene from Romeo and Juliet.

Making Processing talk

Since Processing has no speaking capabilities out of the box, our first task is adding an external library using the new Processing Library Manager. We will use the ttslib package, which is a wrapper library around the FreeTTS library.

We will also create a short, speaking Processing sketch to check the installation.

Engage Thrusters

Processing can be extended by contributed libraries. Most of these additional libraries can be installed by navigating to Sketch | Import Library… | Add Library..., as shown in the following screenshot:
In the Library Manager dialog, enter ttslib in the search field to filter the list of libraries.
Click on the ttslib entry and then on the Install button, as shown in the following screenshot, to download and install the library:
To use the new library, we need to import it to our sketch. We do this by clicking on the Sketch menu and choosing Import Library... and then ttslib.
We will now add the setup() and draw() methods to our sketch. We will leave the draw() method empty for now and instantiate a TTS object in the setup() method. Your sketch should look like the following code snippet:
```
import guru.ttslib.*;

TTS tts;
void setup() {
  tts = new TTS();
}

void draw() {
}
```
Tip
Downloading the example code
You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
Now we will add a mousePressed() method to our sketch, which will get called if someone clicks on our sketch window. In this method, we are calling the speak() method of the TTS object we created in the setup() method.
```
void mousePressed() {
  tts.speak("Hello, I am a Computer");
}
```
Click on the Run button to start the Processing sketch. A little gray window should appear.
Turn on your speakers or put on your headphones, and click on the gray window. If nothing went wrong, a friendly male computer voice named kevin16 should greet you now.

Objective Complete - Mini Debriefing

In steps 1 to 3, we installed an additional library to Processing. The ttslib is a wrapper library around the FreeTTS text-to-speech engine.

Then we created a simple Processing sketch that imports the installed library and creates an instance of the TTS class. The TTS objects match the speakers we need in our sketches. In this case, we created only one speaker and added a mousePressed() method that calls the speak() method of our tts object.

Reading Shakespeare

In this part of the project, we are going to create a Drama thread and teach Processing how to read a Shakespeare script. This thread runs in the background and is controlling the performance. We focus on reading and executing the play in this task, and add the speakers in the next one.

Prepare for Lift Off

Our sketch needs to know which line of the script is read by which robot. So we need to convert the Shakespeare script into a more machine-readable format. For every line of text, we need to know which speaker should read the line. So we take the script and add the letter J and a separation character that is used nowhere else in the script, in front of every line our Juliet-Robot should speak, and we add R and the separation letter for every line our Romeo-Robot should speak. After all these steps, our text file looks something like the following:

R# Lady, by yonder blessed moon I vow,
R# That tips with silver all these fruit-tree tops --

J# O, swear not by the moon, the inconstant moon,
J# That monthly changes in her circled orb,
J# Lest that thy love prove likewise variable.

R# What shall I swear by?

J# Do not swear at all.
J# Or if thou wilt, swear by thy gracious self,
J# Which is the god of my idolatry,
J# And I'll believe thee.

I have already converted the script of the play into this format, and it can be downloaded from the book's support page at http://www.packtpub.com/support.

Engage Thrusters

Let's write our parser:

Let's start a new sketch by navigating to File | New.
Add a setup() and a draw() method.
Now add the prepared script to the Processing sketch by navigating to Sketch | Add File and selecting the file you just downloaded.

Add the following line to your setup() method:

void setup() {
  String[] rawLines = loadStrings( "romeo_and_juliet.txt" );
}

If you renamed your text file, change the filename accordingly.
Create a new tab by clicking on the little arrow icon on the right and choosing New Tab.
Name the class Line. This class will hold our text lines and the speaker.

Add the following code to the tab we just created:

public class Line {
  String speaker;
  String text;

  public Line( String speaker, String text ) {
    this.speaker = speaker;
    this.text = text;
  }
}

Switch back to our main tab and add the following highlighted lines of code to the setup() method:

void setup() {
  String[] rawLines = loadStrings( "romeo_and_juliet.txt" );

  ArrayList lines = new ArrayList();
  for ( int i=0; i<rawLines.length; i++) {
    if (!"".equals(rawLines[i])) {
      String[] tmp = rawLines[i].split("#");
      lines.add( new Line( tmp[0], tmp[1].trim() ));
    }
  }

}

We have read our text lines and parsed them into the lines array list, but we still need a class that does something with our text lines. So create another tab by clicking on the arrow icon and choosing New Tab from the menu; name it Drama.

Our Drama class will be a thread that runs in the background and tells each of the speaker objects to read one line of text. Add the following lines of code to your Drama class:

public class Drama extends Thread {
  int current;
  ArrayList lines;
  boolean running;

  public Drama( ArrayList lines ) {
    this.lines = lines;
    current = 0;
    running = false;
  }

  public int getCurrent() {
    return current;
  }

  public Line getLine( int num ) {
    if ( num >=0 && num < lines.size()) {
      return (Line)lines.get( num );
    } else {
      return null;
    }
  }

  public boolean isRunning() {
    return running;
  }
}

Now we add a run() method that gets executed in the background if we start our thread. Since we have no speaker objects yet, we will print the lines on the console and include a little pause after each line.

  public void run() {
    running = true;

    for ( int i =0; i < lines.size(); i++) {
      current = i;
      Line l = (Line)lines.get(i);
      System.out.println( l.text );
      delay( 1 ); 
    }
    running = false;
  }

Switch back to the main sketch tab and add the highlighted code to the setup() method to create a drama thread object, and then feed it the parsed text-lines.

Drama drama;

void setup() {

  String[] rawLines = loadStrings( "romeo_and_juliet.txt" );
  ArrayList lines = new ArrayList();
  for ( int i=0; i<rawLines.length; i++) {
    if (!"".equals(rawLines[i])) {
      String[] tmp = rawLines[i].split("#");
      lines.add( new Line( tmp[0], tmp[1].trim() ));
    }
  }

  drama = new Drama( lines );
}

So far our sketch parses the text lines and creates a Drama thread object. What we need next is a method to start it. So add a mousePressed() method to start the drama thread.
```
void mousePressed() {
  if ( !drama.isRunning()) {
    drama.start();
  }
}
```

Now add a little bit of text to the draw() method to tell the user what to do. Add the following code to the draw() method:

void draw() {
  background(255);
  textAlign(CENTER);
  fill(0);

  text( "Click here for Drama", width/2, height/2 );
}

Currently, our sketch window is way too small to contain the text, and we also want to use a bigger font. To change the window size, we simply add the following line to the setup() method:

void setup() {

  size( 800, 400 );
  
  String[] rawLines = loadStrings( "romeo_and_juliet.txt" );
  ArrayList lines = new ArrayList();
  for ( int i=0; i<rawLines.length; i++) {
    if (!"".equals(rawLines[i])) {
      String[] tmp = rawLines[i].split("#");
      lines.add( new Line( tmp[0], tmp[1].trim() ));
    }
  }

  drama = new Drama( lines );
}

To change the used font, we need to tell Processing which font to use. The easiest way to find out the names of the fonts that are currently installed on the computer is to create a new sketch, type the following line, and run the sketch:
```
    println(PFont.list());
```

Copy one of the font names you like and add the following line to the Romeo and Juliet sketch:

  void setup() {

    size( 800, 400 );
    textFont( createFont( "Georgia", 24 ));

...

Replace the font name in the code lines with one of the fonts on your computer.

Objective Complete - Mini Debriefing

In this section, we wrote the code that parses a text file and generates a list of Line objects. These objects are then used by a Drama thread that runs in the background as soon as anyone clicks on the sketch window. Currently, the Drama thread prints out the text line on the console.

In steps 6 to 8, we created the Line class. This class is a very simple, so-called Plain Old Java Object (POJO) that holds our text lines, but it doesn't add any functionality.

The code that is controlling the performance of our play was created in steps 10 to 12. We created a thread that is able to run in the background, since in the next step we want to be able to use the draw() method and some TTS objects simultaneously.

The code block in step 12 defines a Boolean variable named running, which we used in the mousePressed() method to check if the sketch is already running or should be started.

Classified Intel

In step 17, we used the list() method of the PFont class to get a list of installed fonts. This is a very common pattern in Processing. You would use the same approach to get a list of installed midi-interfaces, web-cams, serial-ports, and so on.

Adding more actors

In this task, we will combine the things we did in the previous two tasks and add some TTS objects to our Drama thread. We will need two robot actors for this scene speaking with different voices, and since we want to build robots containing a speaker each, we need one of our TTS objects to speak on the left speaker and the other one on the right.

Unfortunately, FreeTTS only comes with one male voice, so we will have to increase the pitch of the voice for our Juliet-Robot.

Engage Thrusters

First, we open the sketch from the previous task and start by creating two TTS objects, one for each of our robot actors. Both use the default voice named kevin16, but we change the pitch for our Juliet-Robot.

void setup() {
  size( 800, 400 );
  textFont( createFont( "Georgia", 24 ));

  String[] rawLines = loadStrings( "romeo_and_juliet.txt" );
  ArrayList lines = new ArrayList();
  for ( int i=0; i<rawLines.length; i++) {
    if (!"".equals(rawLines[i])) {
      String[] tmp = rawLines[i].split("#");
      lines.add( new Line( tmp[0], tmp[1].trim() ));
    }
  }

  TTS romeo = new TTS();
  TTS juliet = new TTS();
  juliet.setPitchShift( 2.4 );
  drama = new Drama( lines, romeo, juliet );
}

Switch to the Drama thread and add two variables to our actors.

public class Drama extends Thread {
  TTS romeo;
  TTS juliet;

  int current;
  ArrayList lines;
  boolean running;
  …

We also need to extend the constructor of our Drama class to enable us to add the actors.

  public Drama( ArrayList lines, TTS romeo, TTS juliet ) {
    this.lines = lines;
    this.romeo = romeo;
    this.juliet = juliet;
    current = 0;
    running = false;
  }

In the run() method, we take the current line and choose the actor object depending on the actor variable we added to each line. We also don't need the delay() and the println() methods anymore.

  public void run() {
    running = true;

    for ( int i =0; i < lines.size(); i++) {
      current = i;
      Line l = (Line)lines.get(i);
      if ( "J".equals( l.speaker )) {
        juliet.speak( l.text );
      }
      if ( "R".equals( l.speaker )) {
        romeo.speak( l.text );
      }
    }
    running = false;
  }

Since each of our robots gets his own speaker, we don't want to hear the text on both speakers. We want one robot to use the right one and the other robot, the left one. Fortunately, ttslib provides a speakLeft() and a speakRight() method, which do exactly what we need. So change the two speak() methods to look like the following:
```
  ...
  if ( "J".equals( l.speaker )) {
      juliet.speakLeft( l.text );
  }
  if ( "R".equals( l.speaker )) {
    romeo.speakRight( l.text );
  }
  ...
```
Currently, our draw() method is somewhat boring and also somewhat misleading if the drama thread is already running. So we will change it to display five lines of the currently read script. Add an if statement to the draw() method that checks the state of the running variable we added to our Drama thread earlier.
```
void draw() {
  background(255);
  textAlign(CENTER);
  fill(0);

  if ( !drama.isRunning() ) {
    text( "Click here for Drama", width/2, height/2 );
  } else {
  }
}
```

Now we add a for loop that displays the previous two lines, the line that's currently being read, and the next two lines of text in our sketch window. We will now change the text alignment so that it matches the speaker of our robot actors. The text will be aligned to the right if our robot uses speakRight(), and it will be aligned to the left if our robot uses speakLeft().

void draw() {
  background(255);
  textAlign(CENTER);
  fill(0);

  if ( !drama.isRunning() ) {
    text( "Click here for Drama", width/2, height/2 );
  }
  else {
    int current = drama.getCurrent();
    for ( int i = -2; i < 3; i ++) {
      Line l = drama.getLine(i + current);
      if ( l != null) {
        if ( "J".equals( l.speaker )) {
          textAlign( LEFT );
          text( l.text, 10, height/2 + i * 30 );
        }
        else {
          textAlign( RIGHT );
          text( l.text, width - 10, height/2 + i * 30 );
        }
      }
    }
  }
}

To show the current line more prominently, we will change the color of the text. We set the color of the current line to black and all other lines to a lighter gray by adding a fill() statement to the for loop.

    for ( int i = -2; i < 3; i ++) {
      fill( abs(i) * 100 );
      Line l = drama.getLine(i + current);
      if ( l != null) {
        if ( "J".equals( l.speaker )) {
          textAlign( LEFT );
          text( l.text, 10, height/2 + i * 30 );
        }
        else {
          textAlign( RIGHT );
          text( l.text, width - 10, height/2 + i * 30 );
        }
      }
    }

Now run your code and click on the sketch window to start the drama thread.

Objective Complete - Mini Debriefing

In this task of our current mission, we added two TTS objects and changed the voice parameters to make them sound different in step 1. Then we extended our Drama thread and added TTS objects for the voices of our robot actors. In steps 4 and 5, we modified the run method to use the voices we just created instead of just printing the text lines.

In steps 6 to 9, we made changes to the draw() method and made it display five lines of text. The line that's currently spoken is black, and the two lines before and after it fade to a light gray.

The fill() method is used to change not only the fill color of an object, but also the text color. Because the index of our for loop runs from -2 to 2, we can simply take the absolute value and multiply it with 100 to get the gray level. The following is a screenshot of the running sketch:

Classified Intel

FreeTTS and ttslib also allow you to use a binary TTS engine named MBROLA. Unfortunately, it's only distributed in binary form, and at the time of writing, it only works on Linux. So if you are using Linux and want to give it a try, you can make the following changes to our Romeo and Juliet sketch:

Open http://tcts.fpms.ac.be/synthesis/mbrola.html in your browser and click on Download. Download the MBROLA binary for your platform.
Download the female_us1 voice from the MBROLA site.
Create a folder for the MBROLA binary and unzip the two packages you just downloaded. Make sure that the path to the MBROLA binary contains no blanks, since FreeTTS can't deal with it.
Rename the MBROLA binary to mbrola.

Now go back to your Romeo and Juliet sketch and add the following highlighted line to your setup() method:

void setup() {
  System.setProperty("mbrola.base", "/path/to/mbrola/");

  size( 800, 400 );
  textFont( createFont( "Georgia", 24 ));

  String[] rawLines = loadStrings( "romeo_and_juliet.txt" );
  ArrayList lines = new ArrayList();
  for ( int i=0; i<rawLines.length; i++) {
    if (!"".equals(rawLines[i])) {
      String[] tmp = rawLines[i].split("#");
      lines.add( new Line( tmp[0], tmp[1].trim() ));
    }
  }

  drama = new Drama( lines );
  TTS romeo = new TTS();
  TTS juliet = new TTS();
  juliet.setPitchShift( 2.4 );
  drama = new Drama( lines, romeo, juliet );
}

Make the path of the system property point to the folder where your mbrola binary and the us1 voice are located.
Now you can change the Juliet TTS object to the following:
```
    TTS juliet = new TTS( "mbrola_us1" );
```
You will also need to change the pitch of the voice as mbrola_us1 is already a female voice and we don't need to simulate it anymore.

MBROLA is a text-to-speech engine developed at the Faculte Polytechnique de Mons in Belgium. The author requires every publication that mentions their work to mention their book, An Introduction To Text-To-Speech Synthesis, Thierry Dutoit, Kluwer Academic Publishers, Dordrecht, Hardbound, ISBN 1-4020-0369-2, April 1997, 312 pp.

Building robots

Now we are ready for the final task of our Romeo and Juliet project. We will take some cardboard boxes, Styrofoam spheres, and a pair of cheap speakers or headphones and turn them into our robot-actors.

The robots described in this section are just my version. If you like, you can build them to be completely different and as complex or as simple as you want.

Prepare for Lift Off

To build our robots, we need some materials and tools, which should not be too hard to find. I used the following for my robot-actors:

Two cardboard boxes for the bodies
Two Styrofoam spheres for the heads
A pair of cheap speakers or headphones
Googly eyes
Acrylic paint
Marker pens
Needles
A hot glue gun

The following picture shows the main materials I used for my robots:

Engage Thrusters

Let's build some robots:

Disassemble your speakers and try to get rid of the body. We only need the speakers, but make sure you don't remove or destroy the cables.
Cut a hole into your cardboard boxes where the speakers should go. Make the holes a bit smaller than the speakers, as shown in the following picture, because we need to glue them to the box later:
Paint your cardboard boxes. I made one white and the other green.
While the boxes are drying, paint some hair on the heads like in the following picture. I made Romeo-Robot's hair black and Juliet-Robot's hair brown.
After the paint has dried, fix the speakers to the cardboard box using some hot glue, as shown in the following picture. Make sure you don't get any glue on the membrane of your speakers.
Now use some markers to draw a face for the robots. You can see the faces of my robots in the following picture:
Use a needle to attach the heads to the bodies of the robots.
Now connect your robot-actors to your sound card and place Juliet on a balcony (for example an empty shoe-box) and make them act.

Objective Complete - Mini Debriefing

In this task, we completed our robot-actors by building bodies for them. We used some cardboard boxes, painted them, and added a cheap pair of speakers by gluing them into the boxes. Each robot got a head made of a painted Styrofoam sphere.

As I already said in the introduction to this task, there is no right or wrong way to build your robots. Build them as small or as big as you like. Add some hair, make them a nose, sew a dress for Juliet, draw Romeo a mustache, and so on.

Key benefits

Teach your computer to create physical objects, visualize data, and program a custom hardware controller

Create projects that can be run on a variety of platforms, ranging from desktop computers to Android smartphones

Each chapter presents a complete project and guides you through the implementation using easy-to-follow, step-by-step instructions

Description

Processing makes it convenient for developers, artists, and designers to create their own projects easily and efficiently. Processing offers you a platform for expressing your ideas and engaging audiences in new ways. This book teaches you everything you need to know to explore new frontiers in animation and interactivity with the help of Processing."Processing 2: Creative Coding Hotshot' will present you with nine exciting projects that will take you beyond the basics and show you how you can make your programs see, hear, and even feel! With these projects, you will also learn how to build your own hardware controllers and integrate devices such as a Kinect senor board in your Processing sketches.Processing is an exciting programming environment for programmers and visual artists alike that makes it easier to create interactive programs.Through nine complete projects, "Processing 2: Creative Coding Hotshot' will help you explore the exciting possibilities that this open source language provides. The topics we will cover range from creating robot - actors performing Shakespeare's "Romeo and Juliet", to generating objects for 3D printing, and you will learn how to run your processing sketches nearly anywhere from a desktop computer to a browser or a mobile device.

Who is this book for?

This book targets Processing developers ,visual artists, creative professionals, and students who want to move to the next level of learning Processing for gaining inspiration, work, or just for fun. The book assumes a basic understanding of programming. However, this book is also recommended to non-artistic readers, looking to expand their graphics and develop their creativity.

What you will learn

Make your computer see, hear, and feel with Processing

Have fun with entertaining projects while learning new tricks with Processing

Use Processing to teach a pair of cardboard robots to enact famous plays

Write code you can run on a variety of devices ranging from your desktop computer to your Android smartphone

Learn to build a custom hardware controller and control it using Processing

Use Processing to create motion sensor games you can play using your Kinect

Design objects you can print using a 3D printer with Processing

What do you get with Print?

Instant access to your digital eBook copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Download this book in EPUB and PDF formats

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Frequently bought together

€41.99

€37.99

Processing 2: Creative Programming Cookbook

€37.99

Total € 117.97

Filter reviews by

All

Amazon verified reviews

thomas Aug 29, 2013

i am an experienced programmer and wasn't looking for a book explaining the basic principles of the processing language, i just wanted to use processing for making my ideas visible quickly, drawing from the big pool of libraries processing offers. this book exceeded my expectations by far. i built speaking robots in chapter one, finally got to do something with the kinect i had lying around for years but never got to play with, ventured into audio analysis and even learnt how to use shaders!the projects are built up in a couple of small steps that should take no longer than 15 to 30 minutes to accomplish, and always reward you with something that's cool, that works, and that you know beforehand will play a key role in the project you're trying to accomplish. the small steps invite you to begin exploring all their aspects, try out variations and test what's possible. the suggestions for further extensions after every project are funny and challenging :)this book shows you how processing can help you concentrate on the creative work, and not on a programming language or interface barrier. i loved it!

Amazon Verified review

Hashiba Aug 19, 2013

Processing 2: Creative Coding Hotshot is the perfect book for Processing users that want to tackle more complex projects or for developers with some Java experience that want to dive into the new Processing 2.0 release.The structure of the projects allows you to follow along or "hack as you go". Each chapter introduces the underlying principles of each library and how to use them, but what is more important is how the descriptions puts you in the right "mindset". I suggest you to type the code of each chapter and afterwards expand the examples with your own classes.I specially enjoyed how the author introduced GLSL filters, taking advantage of the OpenGL integration, and building on top of the previous examples.No more excuses, grab this book and start coding like a boss.

Amazon Customer Apr 30, 2015

Processing is a great free coding language supported by donations. It is simpler to use than Java but compiles into a java program. It has simple 3D and animation capabilities.

B. Pauzenberger Jul 11, 2013

Neun Projekte, die verschiedenste Technologien kombinieren und in Processing-Anwendungen bündeln, die echt Spaß machen. Auch kleine Hardware-Basteleien kommen nicht zu kurz.Fazit: Kaufempfehlung!

H. Lamers Jul 29, 2013

Processing 2: Creative Coding Hotshot is about connecting Processing with hardware. The book is divided into 9 projects which all have their own chapter. To go roughly through them Processing 2: Creative Coding Hotshot shows you how to let Processing talk. How to connect Processing with Kinect hardware or specialized controllers. Processing 2: Creative Coding Hotshot explains also how to create a game which allows you to cover all internet capable platforms. There is a chapter that shows you how to create a geographic information system that takes a web server log-file and shows where the requests of the readers came from. And the last chapter explains how to create and export a 3D object with Processing that can be printed by a 3d printer.What I really liked about Processing 2: Creative Coding Hotshot was the suggestions listed at the end of each chapter. Most of the time when I study a book I just go to the next chapter but these suggestions made it easier for me to continue with the work or make a different approach to the same subject in order to try to reach another level. To summarize Processing 2: Creative Coding Hotshot. If you're not into Processing connected to hardware this book is a good choice to get into it.

Processing 2: Creative Coding Hotshot: Learn Processing with exciting and engaging projects to make your computer talk, see, hear, express emotions, and even design physical objects

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs