Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Python for Secret Agents
Python for Secret Agents

Python for Secret Agents: Analyze, encrypt, and uncover intelligence data using Python, the essential tool for all aspiring secret agents

eBook
€8.99 €17.99
Paperback
€22.99
Subscription
Free Trial
Renews at €18.99p/m

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Table of content icon View table of contents Preview book icon Preview Book

Python for Secret Agents

Chapter 2. Acquiring Intelligence Data

We're going to acquire intelligence data from a variety of sources. We might interview people. We might steal files from a secret underground base. We might search the World Wide Web (WWW), and this is what we'll focus on in this chapter. Using our own cameras or recording devices is the subject of the next chapter.

Important espionage targets include natural resources, popular opinion, and strategic economic strengths. This kind of background information is useful in a number of ways. A great deal of the world's data is already on the Web, and the rest will get there eventually. Any modern search for intelligence starts with the Web.

We can use Python libraries such as http.client and urllib to get data from remote servers and transfer files to other servers. Once we've found remote files of interest, we're going to need a number of Python libraries to parse and extract data from these libraries.

In Chapter 1, Our...

Accessing data from the Internet

The WWW and Internet are based on a series of agreements called Request for Comments (RFC). The RFCs define the standards and protocols to interconnect different networks, that is, the rules for internetworking. The WWW is defined by a subset of these RFCs that specifies the protocols, behaviors of hosts and agents (servers and clients), and file formats, among other details.

In a way, the Internet is a controlled chaos. Most software developers agree to follow the RFCs. Some don't. If their idea is really good, it can catch on, even though it doesn't precisely follow the standards. We often see this in the way some browsers don't work with some websites. This can cause confusion and questions. We'll often have to perform both espionage and plain old debugging to figure out what's available on a given website.

Python provides a variety of modules that implement the software defined in the Internet RFCs. We'll look at some of...

Using a REST API in Python

A great deal of intelligence data is available through REST APIs. Much of the data is available in simple JSON, CSV, or XML documents. In order to make sense of this data, we need to be able to parse these various kinds of serialization formats. We'll focus on JSON because it's widely used. Sadly, it's not universal.

A REST protocol is essentially HTTP. It will leverage POST, GET, PUT, and DELETE requests to implement the essential four stages in the life of persistent data: Create, Retrieve, Update, and Delete (CRUD) rules.

We'll look at currency conversion as a simple web API. This can both help us bribe our information sources as well as provide important information on the overall state of a nation's economy. We can measure national economies against each other as well as measure them against non-national crypto currencies such as bitcoins.

We'll get exchange and currency information from http://www.coinbase.com. There are a lot...

Organizing collections of data

We introduced some data collections earlier in the chapter. It's time to come clean on what these collections are and how we can use them effectively. As we observed in Chapter 1, Our Espionage Toolkit, Python offers a tower of different types of numbers. The commonly used numbers are built in; the more specialized numbers are imported from the standard library.

In a similar way, Python has a number of built-in collections. There is also a very large number of additional collection types available in the standard library. We'll look at the built-in lists, tuples, dictionaries, and sets. These cover the essential bases to work with groups of data items.

Using a Python list

The Python list class can be summarized as a mutable sequence. Mutability means that we can add, change, and remove items (the list can be changed). Sequence means that the items are accessed based on their positions within the list.

The syntax is pleasantly simple; we put the data...

Solving problems – currency conversion rates

The problem we have is that our informants are always asking for odd or unusual currencies. This isn't really all that surprising; we're dealing with spies and criminals on the run. They always seem to need obscure foreign currencies for their own nefarious projects.

We can get a big pile of international exchange rates using a piece of code like the following:

    query_exchange_rates= "http://www.coinbase.com/api/v1/currencies/exchange_rates/"

    with urllib.request.urlopen( query_exchange_rates ) as document:
        pprint.pprint( document.info().items() )
        exchange_rates= json.loads( document.read().decode("utf-8") )

The query string is a simple URL. When we make the request, we get back a long string of bytes. We decode this to make a proper string and use json.loads() to build a Python object.

The problem is that we get a giant dictionary object that's not really all that useful. It looks...

Summary

In this chapter, we saw the basics of using Python to access data available on the WWW. We used the HTTP protocol and the FTP protocol to transfer files around. We can use HTTPS and FTPS to assure that our data is kept secret.

We looked at using RESTful web services to gather data from information sources that have a defined API. RESTful web services are available for many kinds of data. They allow us to gather and analyze data from a variety of sources without a lot of tedious, error-prone pointing and clicking.

We also saw how to work with the various Python collections: lists, tuples, dictionaries, and sets. This gives us ways to capture and manage larger collections of information.

We looked at the JSON notation for Python objects. This is a handy way to transmit objects through the WWW. It's also handy for saving objects locally on our personal computer.

In the next chapter, we'll look at how we can work with image files. These are a bit more complex than JSON files,...

Left arrow icon Right arrow icon

Description

If you are a Python beginner who is looking to learn the language through interesting projects, this book is for you. A basic knowledge of programming and statistics is beneficial to get the most out of the book.
Estimated delivery fee Deliver to Bulgaria

Premium delivery 7 - 10 business days

€25.95
(Includes tracking information)

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Aug 26, 2014
Length: 216 pages
Edition : 1st
Language : English
ISBN-13 : 9781783980420
Category :
Languages :
Concepts :

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Estimated delivery fee Deliver to Bulgaria

Premium delivery 7 - 10 business days

€25.95
(Includes tracking information)

Product Details

Publication date : Aug 26, 2014
Length: 216 pages
Edition : 1st
Language : English
ISBN-13 : 9781783980420
Category :
Languages :
Concepts :

Packt Subscriptions

See our plans and pricing
Modal Close icon
€18.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
€189.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts
€264.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total 81.97
Mastering Object-oriented Python
€37.99
Python for Secret Agents
€22.99
Python for Secret Agents - Volume II
€20.99
Total 81.97 Stars icon
Banner background image

Table of Contents

6 Chapters
1. Our Espionage Toolkit Chevron down icon Chevron up icon
2. Acquiring Intelligence Data Chevron down icon Chevron up icon
3. Encoding Secret Messages with Steganography Chevron down icon Chevron up icon
4. Drops, Hideouts, Meetups, and Lairs Chevron down icon Chevron up icon
5. A Spymaster's More Sensitive Analyses Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
(9 Ratings)
5 star 44.4%
4 star 33.3%
3 star 11.1%
2 star 0%
1 star 11.1%
Filter icon Filter
Top Reviews

Filter reviews by




cracker Mar 01, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
A fun take on learning Python of tying all the lessons to the needs of spy. This gives a real good idea of what programming can do and an introduction to common programming problems. Many beginner books tend to focus heavily on syntax without a theme to tie it all together.
Amazon Verified review Amazon
A. Zubarev Oct 10, 2014
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This book is about learning Python, but in such a way you will be hardly able to put the book aside. So do not be fooled by the title. Yes, you will learn about modules, classes and even unit tests!And even though Python for Secret Agents is seemingly aimed at beginner programmers it should be pure fun for the rest of us. In my view this book constitutes serious work covering such important and frequently used techniques as accessing remote data, getting files over FTP, RESTAPIs, JSON(including serialization), ZIP archives, Geospatial calculations or simply teaching such common techniques as file path processing, numerical computations, lists and dictionaries or Unit Testing. And clever statistical data processing, too. NumPy, SciPy libraries are covered which is a great plus.I was able to extract a ton of useful approaches to dealing with web data – BeatifulSoup is among the ones. The book became more and more exiting as I progressed through it until that all unfolded into one aha moment and finally exploded in a-la Cirque De Soleil kind of finale - last chapter: “A Spymaster’s More SensitiveAnalyses”.A great, great rare read I did not experience for a long time. Thank you Steven F. Lott! I am looking forward to reading more books from you.For the sake of this review and by means to downgrade my mark if I had to mention a few deficiencies that would be the lack of sample output (I just not always had the time for running each code example), nor does it provide examples on how to work with shape data (geolocations), but the Shapes is old school.So all in all it is worth your buck.Five out of five!
Amazon Verified review Amazon
izartxo Aug 18, 2015
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Divertido a la hora de aprender conceptos diferentes sobre python. Se lee muy rápido y para un usuario medio se comprende perfectamente
Amazon Verified review Amazon
Zippy Oct 23, 2014
Full star icon Full star icon Full star icon Full star icon Full star icon 5
With a title like “Python for Secret Agents,” one might think that F. Steven Lott’s latest instructive effort in the Python world would be geared toward the kind of Alan Turing-like eccentric geniuses who decoded secret German messages at Bletchley Park during World War II.In reality, the secret agent theme is a clever conceit to both help teach the Python programming language and explain how to work with a number of useful add-on libraries, such as PILLOW and Beautiful Soup. (Don’t get me wrong, there ARE some actual secret agent-like code examples, such as embedding a secret message in an image and a simple, inefficient password cracker to open an encrypted zip file – not exactly Edward Snowden-level material that will keep the NSA biting its collective fingernails in the middle of the night, but interesting and informative nonetheless.)Lott has written this book in such a way that the fundamental ideas in one chapter are leveraged and assembled into a subsequent chapter, and then those and other elements get combined into larger projects in still later chapters. Thus, it is recommended that you read the chapters in sequence, from beginning to end.Lott covers a surprising amount of material, from installing Python to using Internet protocols and data-interchange formats (FTP, JSON, etc.) and Representational State Transfer (REST), to dealing with Bitcoin and its brethren cryptocurrencies, parsing and file compression/decompression, to steganography (the general science of hiding information) and converting geographic coordinates into addresses and vice versa.(Whereas the goal of cryptography is to make data unreadable by a third party, steganography simply hides the data from a third party, such as embedding a message in an image or other innocuous file.)There’s even a chapter on “A Spymaster’s More Sensitive Analyses,” which elucidates data analysis (comparisons, correlations, statistics) in conjunction with an explanation of generator expressions and classes of objects.Finally, the concept of modules, how to build them and how to construct a test and self-testing module is presented.Steven Lott has produced an amusing and enjoyable Python tour for a type of programmer unfamiliar with Python who is poised just between total “clueless” beginner and intermediate in terms of general programming knowledge and experience.
Amazon Verified review Amazon
Daniel Lee Oct 02, 2014
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
If you're looking for a fun, quick read that will show you some neat tricks for mashing up and manipulating data in Python, this is the right book for you. Python for Secret Agents has a wide variety of fun tricks. It's got nice code examples that help you get to know many Pythonic best practices, and although it doesn't descend below the language's surface, it does give you glimpses of several builtin or freely available packages for solving problems across the board.The book is clearly intended for beginners, and it's written so that it won't lose you if you read it. However, in keeping with the Pythonic philosophy, it tends to implement things only if they're needed or if implementing them allows the author to show something useful to his audience. This has the positive effect that readers above the intermediate level can enjoy the book because they're introduced to libraries that they might not have worked with before.That's not because the author works with exotic packages, but rather because he covers such a broad range of topics. I especially appreciated how the author always connected disparate sections that could have been written on their own. As it is, stringing them together brought them a lot closer to real-life projects.For example, the author shows some simple text parsing tricks, followed by file compression and decompression. He then strings the whole thing together in a dictionary-based, very basic brute force password cracker that can decrypt a poorly encrypted zip file. Along the way, the author gently pounds the best practice into your head of always working with context managers when you have resources that need to be closed again.This knowledge of context managers is brought into the next chapter, where you use them while downloading data from the Internet. The reader is then shown how to encode REST queries and parse XML in order to pull information straight out of different sources in the Internet. A short excursion into Python collection types helps the user to be ready to serialize and deserialize the data they find.This knowledge is then applied again in a nice demonstration of some simple steganography, followed by a more complex mash-up of the content from the previous chapters, where the user serializes and deserializes geographic coordinates and uses them to look up objects with a REST API. The author shows an example of how to efficiently process the acquired objects using generators, connecting disparate data sources into a meaningful application tailored to the project's needs.The book was a lot of fun to read, and although it is not challenging, it provides a broad focus without sacrificing clarity or clearness. Beginners that start here will have the luck of being indoctrinated with generator expressions, context managers, simple unit tests, docstrings and exception handling, and they'll learn the simplest beginnings of object orientation too.I can thoroughly recommend the book to any beginner wanting to look into Python, and it's worth thumbing through the table of contents if you're past that level just to see if there are any examples that might be of interest to you. I'll probably never pick it up again, but it was a fun read with some nice tricks I'll be sure to remember.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is the delivery time and cost of print book? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
What is custom duty/charge? Chevron down icon Chevron up icon

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order? Chevron down icon Chevron up icon

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges? Chevron down icon Chevron up icon

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

  • If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
  • Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.
How can I cancel my order? Chevron down icon Chevron up icon

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact customercare@packt.com with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at customercare@packt.com using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy? Chevron down icon Chevron up icon

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on customercare@packt.com with the order number and issue details as explained below:

  1. If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on customercare@packt.com within one hour of placing the order and we will replace/refund you the item cost.
  2. Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on customercare@packt.com who will be able to resolve this issue for you.
  3. You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
  4. Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
  5. If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
  6. Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on customercare@packt.com within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged? Chevron down icon Chevron up icon

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use? Chevron down icon Chevron up icon

You can pay with the following card types:

  1. Visa Debit
  2. Visa Credit
  3. MasterCard
  4. PayPal
What is the delivery time and cost of print books? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela