Running the topology
We have a few more small items to address before we run the topology:
- Copy the
logconfig.ini
file from the second example in Chapter 3, Introducing Petrel, to this topology's directory. - Create a file called
setup.sh
. Petrel will package this script with the topology and run it at startup. This script installs the third-party Python libraries used by the topology. The file looks like this:pip install -U pip pip install nltk==3.0.1 oauthlib==0.7.2 tweepy==3.2.0
- Create a file called
manifest.txt
with these two lines:logconfig.ini setup.sh
- Before running the topology, let's review the list of files that we've created. Make sure you have created these files correctly:
topology.yaml
twitterstream.py
splitsentence.py
rollingcount.py
intermediaterankings.py
totalrankings.py
manifest.txt
setup.sh
- Run the topology with this command:
petrel submit --config topology.yaml --logdir `pwd`
Once the topology starts running, open another terminal in the topology
directory. Enter the...