Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Python for Secret Agents - Volume II

You're reading from   Python for Secret Agents - Volume II Gather, analyze, and decode data to reveal hidden facts using Python, the perfect tool for all aspiring secret agents

Arrow left icon
Product type Paperback
Published in Dec 2015
Publisher
ISBN-13 9781785283406
Length 180 pages
Edition 2nd Edition
Languages
Arrow right icon
Author (1):
Arrow left icon
Steven F. Lott Steven F. Lott
Author Profile Icon Steven F. Lott
Steven F. Lott
Arrow right icon
View More author details
Toc

Reading and understanding the raw data

Files come in a variety of formats. Even a file that appears to be simple text is often a UTF-8 encoding of Unicode characters. When we're processing data to extract intelligence, we need to look at three tiers of representation:

  • Physical Format: We might have a text file encoded in UTF-8, or we might have a GZIP file, which is a compressed version of the text file. Across these different physical formats, we can find a common structure. In the case of log files, the common structure is a line of text which represents a single event.
  • Logical Layout: After we've extracted data from the physical form, we often find that the order of the fields is slightly different or some optional fields are missing. The trick of using named groups in a regular expression gives us a way to handle variations in the logical layouts by using different regular expressions depending on the details of the layout.
  • Conceptual Content: This is the data we were looking...
You have been reading a chapter from
Python for Secret Agents - Volume II - Second Edition
Published in: Dec 2015
Publisher:
ISBN-13: 9781785283406
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime
Banner background image