Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Python for Secret Agents
Python for Secret Agents

Python for Secret Agents: Analyze, encrypt, and uncover intelligence data using Python, the essential tool for all aspiring secret agents

Arrow left icon
Profile Icon Steven F. Lott
Arrow right icon
€18.99 per month
Full star icon Full star icon Full star icon Full star icon Empty star icon 4 (9 Ratings)
Paperback Aug 2014 216 pages 1st Edition
eBook
€8.99 €17.99
Paperback
€22.99
Subscription
Free Trial
Renews at €18.99p/m
Arrow left icon
Profile Icon Steven F. Lott
Arrow right icon
€18.99 per month
Full star icon Full star icon Full star icon Full star icon Empty star icon 4 (9 Ratings)
Paperback Aug 2014 216 pages 1st Edition
eBook
€8.99 €17.99
Paperback
€22.99
Subscription
Free Trial
Renews at €18.99p/m
eBook
€8.99 €17.99
Paperback
€22.99
Subscription
Free Trial
Renews at €18.99p/m

What do you get with a Packt Subscription?

Free for first 7 days. $19.99 p/m after that. Cancel any time!
Product feature icon Unlimited ad-free access to the largest independent learning library in tech. Access this title and thousands more!
Product feature icon 50+ new titles added per month, including many first-to-market concepts and exclusive early access to books as they are being written.
Product feature icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Product feature icon Thousands of reference materials covering every tech concept you need to stay up to date.
Subscribe now
View plans & pricing
Table of content icon View table of contents Preview book icon Preview Book

Python for Secret Agents

Chapter 2. Acquiring Intelligence Data

We're going to acquire intelligence data from a variety of sources. We might interview people. We might steal files from a secret underground base. We might search the World Wide Web (WWW), and this is what we'll focus on in this chapter. Using our own cameras or recording devices is the subject of the next chapter.

Important espionage targets include natural resources, popular opinion, and strategic economic strengths. This kind of background information is useful in a number of ways. A great deal of the world's data is already on the Web, and the rest will get there eventually. Any modern search for intelligence starts with the Web.

We can use Python libraries such as http.client and urllib to get data from remote servers and transfer files to other servers. Once we've found remote files of interest, we're going to need a number of Python libraries to parse and extract data from these libraries.

In Chapter 1, Our...

Accessing data from the Internet

The WWW and Internet are based on a series of agreements called Request for Comments (RFC). The RFCs define the standards and protocols to interconnect different networks, that is, the rules for internetworking. The WWW is defined by a subset of these RFCs that specifies the protocols, behaviors of hosts and agents (servers and clients), and file formats, among other details.

In a way, the Internet is a controlled chaos. Most software developers agree to follow the RFCs. Some don't. If their idea is really good, it can catch on, even though it doesn't precisely follow the standards. We often see this in the way some browsers don't work with some websites. This can cause confusion and questions. We'll often have to perform both espionage and plain old debugging to figure out what's available on a given website.

Python provides a variety of modules that implement the software defined in the Internet RFCs. We'll look at some of...

Using a REST API in Python

A great deal of intelligence data is available through REST APIs. Much of the data is available in simple JSON, CSV, or XML documents. In order to make sense of this data, we need to be able to parse these various kinds of serialization formats. We'll focus on JSON because it's widely used. Sadly, it's not universal.

A REST protocol is essentially HTTP. It will leverage POST, GET, PUT, and DELETE requests to implement the essential four stages in the life of persistent data: Create, Retrieve, Update, and Delete (CRUD) rules.

We'll look at currency conversion as a simple web API. This can both help us bribe our information sources as well as provide important information on the overall state of a nation's economy. We can measure national economies against each other as well as measure them against non-national crypto currencies such as bitcoins.

We'll get exchange and currency information from http://www.coinbase.com. There are a lot...

Organizing collections of data

We introduced some data collections earlier in the chapter. It's time to come clean on what these collections are and how we can use them effectively. As we observed in Chapter 1, Our Espionage Toolkit, Python offers a tower of different types of numbers. The commonly used numbers are built in; the more specialized numbers are imported from the standard library.

In a similar way, Python has a number of built-in collections. There is also a very large number of additional collection types available in the standard library. We'll look at the built-in lists, tuples, dictionaries, and sets. These cover the essential bases to work with groups of data items.

Using a Python list

The Python list class can be summarized as a mutable sequence. Mutability means that we can add, change, and remove items (the list can be changed). Sequence means that the items are accessed based on their positions within the list.

The syntax is pleasantly simple; we put the data...

Solving problems – currency conversion rates

The problem we have is that our informants are always asking for odd or unusual currencies. This isn't really all that surprising; we're dealing with spies and criminals on the run. They always seem to need obscure foreign currencies for their own nefarious projects.

We can get a big pile of international exchange rates using a piece of code like the following:

    query_exchange_rates= "http://www.coinbase.com/api/v1/currencies/exchange_rates/"

    with urllib.request.urlopen( query_exchange_rates ) as document:
        pprint.pprint( document.info().items() )
        exchange_rates= json.loads( document.read().decode("utf-8") )

The query string is a simple URL. When we make the request, we get back a long string of bytes. We decode this to make a proper string and use json.loads() to build a Python object.

The problem is that we get a giant dictionary object that's not really all that useful. It looks...

Summary

In this chapter, we saw the basics of using Python to access data available on the WWW. We used the HTTP protocol and the FTP protocol to transfer files around. We can use HTTPS and FTPS to assure that our data is kept secret.

We looked at using RESTful web services to gather data from information sources that have a defined API. RESTful web services are available for many kinds of data. They allow us to gather and analyze data from a variety of sources without a lot of tedious, error-prone pointing and clicking.

We also saw how to work with the various Python collections: lists, tuples, dictionaries, and sets. This gives us ways to capture and manage larger collections of information.

We looked at the JSON notation for Python objects. This is a handy way to transmit objects through the WWW. It's also handy for saving objects locally on our personal computer.

In the next chapter, we'll look at how we can work with image files. These are a bit more complex than JSON files,...

Left arrow icon Right arrow icon

Description

If you are a Python beginner who is looking to learn the language through interesting projects, this book is for you. A basic knowledge of programming and statistics is beneficial to get the most out of the book.

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Aug 26, 2014
Length: 216 pages
Edition : 1st
Language : English
ISBN-13 : 9781783980420
Category :
Languages :
Concepts :

What do you get with a Packt Subscription?

Free for first 7 days. $19.99 p/m after that. Cancel any time!
Product feature icon Unlimited ad-free access to the largest independent learning library in tech. Access this title and thousands more!
Product feature icon 50+ new titles added per month, including many first-to-market concepts and exclusive early access to books as they are being written.
Product feature icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Product feature icon Thousands of reference materials covering every tech concept you need to stay up to date.
Subscribe now
View plans & pricing

Product Details

Publication date : Aug 26, 2014
Length: 216 pages
Edition : 1st
Language : English
ISBN-13 : 9781783980420
Category :
Languages :
Concepts :

Packt Subscriptions

See our plans and pricing
Modal Close icon
€18.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
€189.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts
€264.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total 81.97
Mastering Object-oriented Python
€37.99
Python for Secret Agents
€22.99
Python for Secret Agents - Volume II
€20.99
Total 81.97 Stars icon
Banner background image

Table of Contents

6 Chapters
1. Our Espionage Toolkit Chevron down icon Chevron up icon
2. Acquiring Intelligence Data Chevron down icon Chevron up icon
3. Encoding Secret Messages with Steganography Chevron down icon Chevron up icon
4. Drops, Hideouts, Meetups, and Lairs Chevron down icon Chevron up icon
5. A Spymaster's More Sensitive Analyses Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
(9 Ratings)
5 star 44.4%
4 star 33.3%
3 star 11.1%
2 star 0%
1 star 11.1%
Filter icon Filter
Top Reviews

Filter reviews by




cracker Mar 01, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
A fun take on learning Python of tying all the lessons to the needs of spy. This gives a real good idea of what programming can do and an introduction to common programming problems. Many beginner books tend to focus heavily on syntax without a theme to tie it all together.
Amazon Verified review Amazon
A. Zubarev Oct 10, 2014
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This book is about learning Python, but in such a way you will be hardly able to put the book aside. So do not be fooled by the title. Yes, you will learn about modules, classes and even unit tests!And even though Python for Secret Agents is seemingly aimed at beginner programmers it should be pure fun for the rest of us. In my view this book constitutes serious work covering such important and frequently used techniques as accessing remote data, getting files over FTP, RESTAPIs, JSON(including serialization), ZIP archives, Geospatial calculations or simply teaching such common techniques as file path processing, numerical computations, lists and dictionaries or Unit Testing. And clever statistical data processing, too. NumPy, SciPy libraries are covered which is a great plus.I was able to extract a ton of useful approaches to dealing with web data – BeatifulSoup is among the ones. The book became more and more exiting as I progressed through it until that all unfolded into one aha moment and finally exploded in a-la Cirque De Soleil kind of finale - last chapter: “A Spymaster’s More SensitiveAnalyses”.A great, great rare read I did not experience for a long time. Thank you Steven F. Lott! I am looking forward to reading more books from you.For the sake of this review and by means to downgrade my mark if I had to mention a few deficiencies that would be the lack of sample output (I just not always had the time for running each code example), nor does it provide examples on how to work with shape data (geolocations), but the Shapes is old school.So all in all it is worth your buck.Five out of five!
Amazon Verified review Amazon
izartxo Aug 18, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Divertido a la hora de aprender conceptos diferentes sobre python. Se lee muy rápido y para un usuario medio se comprende perfectamente
Amazon Verified review Amazon
Zippy Oct 23, 2014
Full star icon Full star icon Full star icon Full star icon Full star icon 5
With a title like “Python for Secret Agents,” one might think that F. Steven Lott’s latest instructive effort in the Python world would be geared toward the kind of Alan Turing-like eccentric geniuses who decoded secret German messages at Bletchley Park during World War II.In reality, the secret agent theme is a clever conceit to both help teach the Python programming language and explain how to work with a number of useful add-on libraries, such as PILLOW and Beautiful Soup. (Don’t get me wrong, there ARE some actual secret agent-like code examples, such as embedding a secret message in an image and a simple, inefficient password cracker to open an encrypted zip file – not exactly Edward Snowden-level material that will keep the NSA biting its collective fingernails in the middle of the night, but interesting and informative nonetheless.)Lott has written this book in such a way that the fundamental ideas in one chapter are leveraged and assembled into a subsequent chapter, and then those and other elements get combined into larger projects in still later chapters. Thus, it is recommended that you read the chapters in sequence, from beginning to end.Lott covers a surprising amount of material, from installing Python to using Internet protocols and data-interchange formats (FTP, JSON, etc.) and Representational State Transfer (REST), to dealing with Bitcoin and its brethren cryptocurrencies, parsing and file compression/decompression, to steganography (the general science of hiding information) and converting geographic coordinates into addresses and vice versa.(Whereas the goal of cryptography is to make data unreadable by a third party, steganography simply hides the data from a third party, such as embedding a message in an image or other innocuous file.)There’s even a chapter on “A Spymaster’s More Sensitive Analyses,” which elucidates data analysis (comparisons, correlations, statistics) in conjunction with an explanation of generator expressions and classes of objects.Finally, the concept of modules, how to build them and how to construct a test and self-testing module is presented.Steven Lott has produced an amusing and enjoyable Python tour for a type of programmer unfamiliar with Python who is poised just between total “clueless” beginner and intermediate in terms of general programming knowledge and experience.
Amazon Verified review Amazon
Daniel Lee Oct 02, 2014
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
If you're looking for a fun, quick read that will show you some neat tricks for mashing up and manipulating data in Python, this is the right book for you. Python for Secret Agents has a wide variety of fun tricks. It's got nice code examples that help you get to know many Pythonic best practices, and although it doesn't descend below the language's surface, it does give you glimpses of several builtin or freely available packages for solving problems across the board.The book is clearly intended for beginners, and it's written so that it won't lose you if you read it. However, in keeping with the Pythonic philosophy, it tends to implement things only if they're needed or if implementing them allows the author to show something useful to his audience. This has the positive effect that readers above the intermediate level can enjoy the book because they're introduced to libraries that they might not have worked with before.That's not because the author works with exotic packages, but rather because he covers such a broad range of topics. I especially appreciated how the author always connected disparate sections that could have been written on their own. As it is, stringing them together brought them a lot closer to real-life projects.For example, the author shows some simple text parsing tricks, followed by file compression and decompression. He then strings the whole thing together in a dictionary-based, very basic brute force password cracker that can decrypt a poorly encrypted zip file. Along the way, the author gently pounds the best practice into your head of always working with context managers when you have resources that need to be closed again.This knowledge of context managers is brought into the next chapter, where you use them while downloading data from the Internet. The reader is then shown how to encode REST queries and parse XML in order to pull information straight out of different sources in the Internet. A short excursion into Python collection types helps the user to be ready to serialize and deserialize the data they find.This knowledge is then applied again in a nice demonstration of some simple steganography, followed by a more complex mash-up of the content from the previous chapters, where the user serializes and deserializes geographic coordinates and uses them to look up objects with a REST API. The author shows an example of how to efficiently process the acquired objects using generators, connecting disparate data sources into a meaningful application tailored to the project's needs.The book was a lot of fun to read, and although it is not challenging, it provides a broad focus without sacrificing clarity or clearness. Beginners that start here will have the luck of being indoctrinated with generator expressions, context managers, simple unit tests, docstrings and exception handling, and they'll learn the simplest beginnings of object orientation too.I can thoroughly recommend the book to any beginner wanting to look into Python, and it's worth thumbing through the table of contents if you're past that level just to see if there are any examples that might be of interest to you. I'll probably never pick it up again, but it was a fun read with some nice tricks I'll be sure to remember.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is included in a Packt subscription? Chevron down icon Chevron up icon

A subscription provides you with full access to view all Packt and licnesed content online, this includes exclusive access to Early Access titles. Depending on the tier chosen you can also earn credits and discounts to use for owning content

How can I cancel my subscription? Chevron down icon Chevron up icon

To cancel your subscription with us simply go to the account page - found in the top right of the page or at https://subscription.packtpub.com/my-account/subscription - From here you will see the ‘cancel subscription’ button in the grey box with your subscription information in.

What are credits? Chevron down icon Chevron up icon

Credits can be earned from reading 40 section of any title within the payment cycle - a month starting from the day of subscription payment. You also earn a Credit every month if you subscribe to our annual or 18 month plans. Credits can be used to buy books DRM free, the same way that you would pay for a book. Your credits can be found in the subscription homepage - subscription.packtpub.com - clicking on ‘the my’ library dropdown and selecting ‘credits’.

What happens if an Early Access Course is cancelled? Chevron down icon Chevron up icon

Projects are rarely cancelled, but sometimes it's unavoidable. If an Early Access course is cancelled or excessively delayed, you can exchange your purchase for another course. For further details, please contact us here.

Where can I send feedback about an Early Access title? Chevron down icon Chevron up icon

If you have any feedback about the product you're reading, or Early Access in general, then please fill out a contact form here and we'll make sure the feedback gets to the right team. 

Can I download the code files for Early Access titles? Chevron down icon Chevron up icon

We try to ensure that all books in Early Access have code available to use, download, and fork on GitHub. This helps us be more agile in the development of the book, and helps keep the often changing code base of new versions and new technologies as up to date as possible. Unfortunately, however, there will be rare cases when it is not possible for us to have downloadable code samples available until publication.

When we publish the book, the code files will also be available to download from the Packt website.

How accurate is the publication date? Chevron down icon Chevron up icon

The publication date is as accurate as we can be at any point in the project. Unfortunately, delays can happen. Often those delays are out of our control, such as changes to the technology code base or delays in the tech release. We do our best to give you an accurate estimate of the publication date at any given time, and as more chapters are delivered, the more accurate the delivery date will become.

How will I know when new chapters are ready? Chevron down icon Chevron up icon

We'll let you know every time there has been an update to a course that you've bought in Early Access. You'll get an email to let you know there has been a new chapter, or a change to a previous chapter. The new chapters are automatically added to your account, so you can also check back there any time you're ready and download or read them online.

I am a Packt subscriber, do I get Early Access? Chevron down icon Chevron up icon

Yes, all Early Access content is fully available through your subscription. You will need to have a paid for or active trial subscription in order to access all titles.

How is Early Access delivered? Chevron down icon Chevron up icon

Early Access is currently only available as a PDF or through our online reader. As we make changes or add new chapters, the files in your Packt account will be updated so you can download them again or view them online immediately.

How do I buy Early Access content? Chevron down icon Chevron up icon

Early Access is a way of us getting our content to you quicker, but the method of buying the Early Access course is still the same. Just find the course you want to buy, go through the check-out steps, and you’ll get a confirmation email from us with information and a link to the relevant Early Access courses.

What is Early Access? Chevron down icon Chevron up icon

Keeping up to date with the latest technology is difficult; new versions, new frameworks, new techniques. This feature gives you a head-start to our content, as it's being created. With Early Access you'll receive each chapter as it's written, and get regular updates throughout the product's development, as well as the final course as soon as it's ready.We created Early Access as a means of giving you the information you need, as soon as it's available. As we go through the process of developing a course, 99% of it can be ready but we can't publish until that last 1% falls in to place. Early Access helps to unlock the potential of our content early, to help you start your learning when you need it most. You not only get access to every chapter as it's delivered, edited, and updated, but you'll also get the finalized, DRM-free product to download in any format you want when it's published. As a member of Packt, you'll also be eligible for our exclusive offers, including a free course every day, and discounts on new and popular titles.