Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Python for Secret Agents
Python for Secret Agents

Python for Secret Agents: Analyze, encrypt, and uncover intelligence data using Python, the essential tool for all aspiring secret agents

eBook
£7.99 £17.99
Paperback
£22.99
Subscription
Free Trial
Renews at £16.99p/m

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Table of content icon View table of contents Preview book icon Preview Book

Python for Secret Agents

Chapter 2. Acquiring Intelligence Data

We're going to acquire intelligence data from a variety of sources. We might interview people. We might steal files from a secret underground base. We might search the World Wide Web (WWW), and this is what we'll focus on in this chapter. Using our own cameras or recording devices is the subject of the next chapter.

Important espionage targets include natural resources, popular opinion, and strategic economic strengths. This kind of background information is useful in a number of ways. A great deal of the world's data is already on the Web, and the rest will get there eventually. Any modern search for intelligence starts with the Web.

We can use Python libraries such as http.client and urllib to get data from remote servers and transfer files to other servers. Once we've found remote files of interest, we're going to need a number of Python libraries to parse and extract data from these libraries.

In Chapter 1, Our...

Accessing data from the Internet

The WWW and Internet are based on a series of agreements called Request for Comments (RFC). The RFCs define the standards and protocols to interconnect different networks, that is, the rules for internetworking. The WWW is defined by a subset of these RFCs that specifies the protocols, behaviors of hosts and agents (servers and clients), and file formats, among other details.

In a way, the Internet is a controlled chaos. Most software developers agree to follow the RFCs. Some don't. If their idea is really good, it can catch on, even though it doesn't precisely follow the standards. We often see this in the way some browsers don't work with some websites. This can cause confusion and questions. We'll often have to perform both espionage and plain old debugging to figure out what's available on a given website.

Python provides a variety of modules that implement the software defined in the Internet RFCs. We'll look at some of...

Using a REST API in Python

A great deal of intelligence data is available through REST APIs. Much of the data is available in simple JSON, CSV, or XML documents. In order to make sense of this data, we need to be able to parse these various kinds of serialization formats. We'll focus on JSON because it's widely used. Sadly, it's not universal.

A REST protocol is essentially HTTP. It will leverage POST, GET, PUT, and DELETE requests to implement the essential four stages in the life of persistent data: Create, Retrieve, Update, and Delete (CRUD) rules.

We'll look at currency conversion as a simple web API. This can both help us bribe our information sources as well as provide important information on the overall state of a nation's economy. We can measure national economies against each other as well as measure them against non-national crypto currencies such as bitcoins.

We'll get exchange and currency information from http://www.coinbase.com. There are a lot...

Organizing collections of data

We introduced some data collections earlier in the chapter. It's time to come clean on what these collections are and how we can use them effectively. As we observed in Chapter 1, Our Espionage Toolkit, Python offers a tower of different types of numbers. The commonly used numbers are built in; the more specialized numbers are imported from the standard library.

In a similar way, Python has a number of built-in collections. There is also a very large number of additional collection types available in the standard library. We'll look at the built-in lists, tuples, dictionaries, and sets. These cover the essential bases to work with groups of data items.

Using a Python list

The Python list class can be summarized as a mutable sequence. Mutability means that we can add, change, and remove items (the list can be changed). Sequence means that the items are accessed based on their positions within the list.

The syntax is pleasantly simple; we put the data...

Solving problems – currency conversion rates

The problem we have is that our informants are always asking for odd or unusual currencies. This isn't really all that surprising; we're dealing with spies and criminals on the run. They always seem to need obscure foreign currencies for their own nefarious projects.

We can get a big pile of international exchange rates using a piece of code like the following:

    query_exchange_rates= "http://www.coinbase.com/api/v1/currencies/exchange_rates/"

    with urllib.request.urlopen( query_exchange_rates ) as document:
        pprint.pprint( document.info().items() )
        exchange_rates= json.loads( document.read().decode("utf-8") )

The query string is a simple URL. When we make the request, we get back a long string of bytes. We decode this to make a proper string and use json.loads() to build a Python object.

The problem is that we get a giant dictionary object that's not really all that useful. It looks...

Summary

In this chapter, we saw the basics of using Python to access data available on the WWW. We used the HTTP protocol and the FTP protocol to transfer files around. We can use HTTPS and FTPS to assure that our data is kept secret.

We looked at using RESTful web services to gather data from information sources that have a defined API. RESTful web services are available for many kinds of data. They allow us to gather and analyze data from a variety of sources without a lot of tedious, error-prone pointing and clicking.

We also saw how to work with the various Python collections: lists, tuples, dictionaries, and sets. This gives us ways to capture and manage larger collections of information.

We looked at the JSON notation for Python objects. This is a handy way to transmit objects through the WWW. It's also handy for saving objects locally on our personal computer.

In the next chapter, we'll look at how we can work with image files. These are a bit more complex than JSON files,...

Left arrow icon Right arrow icon

Description

If you are a Python beginner who is looking to learn the language through interesting projects, this book is for you. A basic knowledge of programming and statistics is beneficial to get the most out of the book.

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Aug 26, 2014
Length: 216 pages
Edition : 1st
Language : English
ISBN-13 : 9781783980437
Category :
Languages :
Concepts :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Billing Address

Product Details

Publication date : Aug 26, 2014
Length: 216 pages
Edition : 1st
Language : English
ISBN-13 : 9781783980437
Category :
Languages :
Concepts :

Packt Subscriptions

See our plans and pricing
Modal Close icon
£16.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
£169.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just £5 each
Feature tick icon Exclusive print discounts
£234.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just £5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total £ 81.97
Mastering Object-oriented Python
£37.99
Python for Secret Agents
£22.99
Python for Secret Agents - Volume II
£20.99
Total £ 81.97 Stars icon
Banner background image

Table of Contents

6 Chapters
1. Our Espionage Toolkit Chevron down icon Chevron up icon
2. Acquiring Intelligence Data Chevron down icon Chevron up icon
3. Encoding Secret Messages with Steganography Chevron down icon Chevron up icon
4. Drops, Hideouts, Meetups, and Lairs Chevron down icon Chevron up icon
5. A Spymaster's More Sensitive Analyses Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
(9 Ratings)
5 star 44.4%
4 star 33.3%
3 star 11.1%
2 star 0%
1 star 11.1%
Filter icon Filter
Top Reviews

Filter reviews by




cracker Mar 01, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
A fun take on learning Python of tying all the lessons to the needs of spy. This gives a real good idea of what programming can do and an introduction to common programming problems. Many beginner books tend to focus heavily on syntax without a theme to tie it all together.
Amazon Verified review Amazon
A. Zubarev Oct 10, 2014
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This book is about learning Python, but in such a way you will be hardly able to put the book aside. So do not be fooled by the title. Yes, you will learn about modules, classes and even unit tests!And even though Python for Secret Agents is seemingly aimed at beginner programmers it should be pure fun for the rest of us. In my view this book constitutes serious work covering such important and frequently used techniques as accessing remote data, getting files over FTP, RESTAPIs, JSON(including serialization), ZIP archives, Geospatial calculations or simply teaching such common techniques as file path processing, numerical computations, lists and dictionaries or Unit Testing. And clever statistical data processing, too. NumPy, SciPy libraries are covered which is a great plus.I was able to extract a ton of useful approaches to dealing with web data – BeatifulSoup is among the ones. The book became more and more exiting as I progressed through it until that all unfolded into one aha moment and finally exploded in a-la Cirque De Soleil kind of finale - last chapter: “A Spymaster’s More SensitiveAnalyses”.A great, great rare read I did not experience for a long time. Thank you Steven F. Lott! I am looking forward to reading more books from you.For the sake of this review and by means to downgrade my mark if I had to mention a few deficiencies that would be the lack of sample output (I just not always had the time for running each code example), nor does it provide examples on how to work with shape data (geolocations), but the Shapes is old school.So all in all it is worth your buck.Five out of five!
Amazon Verified review Amazon
izartxo Aug 18, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Divertido a la hora de aprender conceptos diferentes sobre python. Se lee muy rápido y para un usuario medio se comprende perfectamente
Amazon Verified review Amazon
Zippy Oct 23, 2014
Full star icon Full star icon Full star icon Full star icon Full star icon 5
With a title like “Python for Secret Agents,” one might think that F. Steven Lott’s latest instructive effort in the Python world would be geared toward the kind of Alan Turing-like eccentric geniuses who decoded secret German messages at Bletchley Park during World War II.In reality, the secret agent theme is a clever conceit to both help teach the Python programming language and explain how to work with a number of useful add-on libraries, such as PILLOW and Beautiful Soup. (Don’t get me wrong, there ARE some actual secret agent-like code examples, such as embedding a secret message in an image and a simple, inefficient password cracker to open an encrypted zip file – not exactly Edward Snowden-level material that will keep the NSA biting its collective fingernails in the middle of the night, but interesting and informative nonetheless.)Lott has written this book in such a way that the fundamental ideas in one chapter are leveraged and assembled into a subsequent chapter, and then those and other elements get combined into larger projects in still later chapters. Thus, it is recommended that you read the chapters in sequence, from beginning to end.Lott covers a surprising amount of material, from installing Python to using Internet protocols and data-interchange formats (FTP, JSON, etc.) and Representational State Transfer (REST), to dealing with Bitcoin and its brethren cryptocurrencies, parsing and file compression/decompression, to steganography (the general science of hiding information) and converting geographic coordinates into addresses and vice versa.(Whereas the goal of cryptography is to make data unreadable by a third party, steganography simply hides the data from a third party, such as embedding a message in an image or other innocuous file.)There’s even a chapter on “A Spymaster’s More Sensitive Analyses,” which elucidates data analysis (comparisons, correlations, statistics) in conjunction with an explanation of generator expressions and classes of objects.Finally, the concept of modules, how to build them and how to construct a test and self-testing module is presented.Steven Lott has produced an amusing and enjoyable Python tour for a type of programmer unfamiliar with Python who is poised just between total “clueless” beginner and intermediate in terms of general programming knowledge and experience.
Amazon Verified review Amazon
Daniel Lee Oct 02, 2014
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
If you're looking for a fun, quick read that will show you some neat tricks for mashing up and manipulating data in Python, this is the right book for you. Python for Secret Agents has a wide variety of fun tricks. It's got nice code examples that help you get to know many Pythonic best practices, and although it doesn't descend below the language's surface, it does give you glimpses of several builtin or freely available packages for solving problems across the board.The book is clearly intended for beginners, and it's written so that it won't lose you if you read it. However, in keeping with the Pythonic philosophy, it tends to implement things only if they're needed or if implementing them allows the author to show something useful to his audience. This has the positive effect that readers above the intermediate level can enjoy the book because they're introduced to libraries that they might not have worked with before.That's not because the author works with exotic packages, but rather because he covers such a broad range of topics. I especially appreciated how the author always connected disparate sections that could have been written on their own. As it is, stringing them together brought them a lot closer to real-life projects.For example, the author shows some simple text parsing tricks, followed by file compression and decompression. He then strings the whole thing together in a dictionary-based, very basic brute force password cracker that can decrypt a poorly encrypted zip file. Along the way, the author gently pounds the best practice into your head of always working with context managers when you have resources that need to be closed again.This knowledge of context managers is brought into the next chapter, where you use them while downloading data from the Internet. The reader is then shown how to encode REST queries and parse XML in order to pull information straight out of different sources in the Internet. A short excursion into Python collection types helps the user to be ready to serialize and deserialize the data they find.This knowledge is then applied again in a nice demonstration of some simple steganography, followed by a more complex mash-up of the content from the previous chapters, where the user serializes and deserializes geographic coordinates and uses them to look up objects with a REST API. The author shows an example of how to efficiently process the acquired objects using generators, connecting disparate data sources into a meaningful application tailored to the project's needs.The book was a lot of fun to read, and although it is not challenging, it provides a broad focus without sacrificing clarity or clearness. Beginners that start here will have the luck of being indoctrinated with generator expressions, context managers, simple unit tests, docstrings and exception handling, and they'll learn the simplest beginnings of object orientation too.I can thoroughly recommend the book to any beginner wanting to look into Python, and it's worth thumbing through the table of contents if you're past that level just to see if there are any examples that might be of interest to you. I'll probably never pick it up again, but it was a fun read with some nice tricks I'll be sure to remember.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.