Adding more actors

In this task, we will combine the things we did in the previous two tasks and add some TTS objects to our Drama thread. We will need two robot actors for this scene speaking with different voices, and since we want to build robots containing a speaker each, we need one of our TTS objects to speak on the left speaker and the other one on the right.

Unfortunately, FreeTTS only comes with one male voice, so we will have to increase the pitch of the voice for our Juliet-Robot.

Engage Thrusters

1. First, we open the sketch from the previous task and start by creating two TTS objects, one for each of our robot actors. Both use the default voice named kevin16, but we change the pitch for our Juliet-Robot.

   void setup() {
     size( 800, 400 );
     textFont( createFont( "Georgia", 24 ));
   
     String[] rawLines = loadStrings( "romeo_and_juliet.txt" );
     ArrayList lines = new ArrayList();
     for ( int i=0; i<rawLines.length; i++) {
       if (!"".equals(rawLines[i])) {
         String[] tmp = rawLines[i].split("#");
         lines.add( new Line( tmp[0], tmp[1].trim() ));
       }
     }
   
     TTS romeo = new TTS();
     TTS juliet = new TTS();
     juliet.setPitchShift( 2.4 );
     drama = new Drama( lines, romeo, juliet );
   }

2. Switch to the Drama thread and add two variables to our actors.

   public class Drama extends Thread {
     TTS romeo;
     TTS juliet;
   
     int current;
     ArrayList lines;
     boolean running;
     …

3. We also need to extend the constructor of our Drama class to enable us to add the actors.

     public Drama( ArrayList lines, TTS romeo, TTS juliet ) {
       this.lines = lines;
       this.romeo = romeo;
       this.juliet = juliet;
       current = 0;
       running = false;
     }

4. In the run() method, we take the current line and choose the actor object depending on the actor variable we added to each line. We also don't need the delay() and the println() methods anymore.

     public void run() {
       running = true;
   
       for ( int i =0; i < lines.size(); i++) {
         current = i;
         Line l = (Line)lines.get(i);
         if ( "J".equals( l.speaker )) {
           juliet.speak( l.text );
         }
         if ( "R".equals( l.speaker )) {
           romeo.speak( l.text );
         }
       }
       running = false;
     }

5. Since each of our robots gets his own speaker, we don't want to hear the text on both speakers. We want one robot to use the right one and the other robot, the left one. Fortunately, ttslib provides a speakLeft() and a speakRight() method, which do exactly what we need. So change the two speak() methods to look like the following:

     ...
     if ( "J".equals( l.speaker )) {
         juliet.speakLeft( l.text );
     }
     if ( "R".equals( l.speaker )) {
       romeo.speakRight( l.text );
     }
     ...

6. Currently, our draw() method is somewhat boring and also somewhat misleading if the drama thread is already running. So we will change it to display five lines of the currently read script. Add an if statement to the draw() method that checks the state of the running variable we added to our Drama thread earlier.

   void draw() {
     background(255);
     textAlign(CENTER);
     fill(0);
   
     if ( !drama.isRunning() ) {
       text( "Click here for Drama", width/2, height/2 );
     } else {
     }
   }

7. Now we add a for loop that displays the previous two lines, the line that's currently being read, and the next two lines of text in our sketch window. We will now change the text alignment so that it matches the speaker of our robot actors. The text will be aligned to the right if our robot uses speakRight(), and it will be aligned to the left if our robot uses speakLeft().

   void draw() {
     background(255);
     textAlign(CENTER);
     fill(0);
   
     if ( !drama.isRunning() ) {
       text( "Click here for Drama", width/2, height/2 );
     }
     else {
       int current = drama.getCurrent();
       for ( int i = -2; i < 3; i ++) {
         Line l = drama.getLine(i + current);
         if ( l != null) {
           if ( "J".equals( l.speaker )) {
             textAlign( LEFT );
             text( l.text, 10, height/2 + i * 30 );
           }
           else {
             textAlign( RIGHT );
             text( l.text, width - 10, height/2 + i * 30 );
           }
         }
       }
     }
   }

8. To show the current line more prominently, we will change the color of the text. We set the color of the current line to black and all other lines to a lighter gray by adding a fill() statement to the for loop.

       for ( int i = -2; i < 3; i ++) {
         fill( abs(i) * 100 );
         Line l = drama.getLine(i + current);
         if ( l != null) {
           if ( "J".equals( l.speaker )) {
             textAlign( LEFT );
             text( l.text, 10, height/2 + i * 30 );
           }
           else {
             textAlign( RIGHT );
             text( l.text, width - 10, height/2 + i * 30 );
           }
         }
       }

9. Now run your code and click on the sketch window to start the drama thread.

Objective Complete - Mini Debriefing

In this task of our current mission, we added two TTS objects and changed the voice parameters to make them sound different in step 1. Then we extended our Drama thread and added TTS objects for the voices of our robot actors. In steps 4 and 5, we modified the run method to use the voices we just created instead of just printing the text lines.

In steps 6 to 9, we made changes to the draw() method and made it display five lines of text. The line that's currently spoken is black, and the two lines before and after it fade to a light gray.

The fill() method is used to change not only the fill color of an object, but also the text color. Because the index of our for loop runs from -2 to 2, we can simply take the absolute value and multiply it with 100 to get the gray level. The following is a screenshot of the running sketch:

Classified Intel

FreeTTS and ttslib also allow you to use a binary TTS engine named MBROLA. Unfortunately, it's only distributed in binary form, and at the time of writing, it only works on Linux. So if you are using Linux and want to give it a try, you can make the following changes to our Romeo and Juliet sketch:

1. Open http://tcts.fpms.ac.be/synthesis/mbrola.html in your browser and click on Download. Download the MBROLA binary for your platform.

2. Download the female_us1 voice from the MBROLA site.

3. Create a folder for the MBROLA binary and unzip the two packages you just downloaded. Make sure that the path to the MBROLA binary contains no blanks, since FreeTTS can't deal with it.

4. Rename the MBROLA binary to mbrola.

5. Now go back to your Romeo and Juliet sketch and add the following highlighted line to your setup() method:

   void setup() {
     System.setProperty("mbrola.base", "/path/to/mbrola/");
   
     size( 800, 400 );
     textFont( createFont( "Georgia", 24 ));
   
     String[] rawLines = loadStrings( "romeo_and_juliet.txt" );
     ArrayList lines = new ArrayList();
     for ( int i=0; i<rawLines.length; i++) {
       if (!"".equals(rawLines[i])) {
         String[] tmp = rawLines[i].split("#");
         lines.add( new Line( tmp[0], tmp[1].trim() ));
       }
     }
   
     drama = new Drama( lines );
     TTS romeo = new TTS();
     TTS juliet = new TTS();
     juliet.setPitchShift( 2.4 );
     drama = new Drama( lines, romeo, juliet );
   }

6. Make the path of the system property point to the folder where your mbrola binary and the us1 voice are located.

7. Now you can change the Juliet TTS object to the following:

       TTS juliet = new TTS( "mbrola_us1" );

8. You will also need to change the pitch of the voice as mbrola_us1 is already a female voice and we don't need to simulate it anymore.

MBROLA is a text-to-speech engine developed at the Faculte Polytechnique de Mons in Belgium. The author requires every publication that mentions their work to mention their book, An Introduction To Text-To-Speech Synthesis, Thierry Dutoit, Kluwer Academic Publishers, Dordrecht, Hardbound, ISBN 1-4020-0369-2, April 1997, 312 pp.