We were introduced to the basic concepts of different fundamental data structures in the previous chapter. We learned about lists, sets, dictionaries, tuples, and strings. However, what we have covered so far were only basic operations on those data structures. They have much more to offer once you learn how to utilize them effectively. In this chapter, we will venture further into the land of data structures. We will learn about advanced operations and manipulations and use fundamental data structures to represent more complex and higher-level data structures; this is often handy while wrangling data in real life. These higher-level topics will include stacks, queues, interiors, and file operations.

In this chapter, we will also learn how to open a file using built-in Python methods and about the many different file operations, such as reading and writing data, and safely closing files once we are done. We will also take a look at some of the problems to avoid while...

We will start this chapter by discussing advanced data structures. Initially, we will be revisiting lists. Then, we will construct a stack and a queue, explore multiple-element membership checking to check whether the data is accurate, and throw a bit of functional programming in for good measure. Don't worry if all of this sounds intimidating. We will take things step by step, and you will feel confident about handling advanced data structures once you have finished this chapter.

Before we jump into constructing data structures, we'll look at a few methods to manipulate them.

Iterator

Iterators in Python are very useful when dealing with data as they allow you to parse the data one unit at a time. Iterators are stateful, which means it will be helpful to keep track of the previous state. An iterator is an object that implements the next method—meaning an iterator can iterate over collections such as lists, tuples, dictionaries,...

In the previous topic, we investigated a few advanced data structures and also learned neat and useful functional programming methods to manipulate them without side effects. In this topic, we will learn about a few OS-level functions in Python, such as working with files, but these could also include working with printers, and even the internet. We will concentrate mainly on file-related functions and learn how to open a file, read the data line by line or all at once, and finally, how to cleanly close the file we opened. The closing operation of a file should be done cautiously, which is ignored most of the time by developers. When handling file operations, we often run into very strange and hard-to-track-down bugs because a process opened a file and did not close it properly. We will apply a few of the techniques we have learned about to a file that we will read to practice our data wrangling skills further.

Exercise 2.08: File Operations

In...

This chapter covered manipulation techniques of advanced data structures such as stacks and queues. We then focused on different methods of functional programming, including iterators, and combined lists and functions together. Later, we looked at OS-level functions and the management of environment variables. We examined how, using Python, we can open, close, and even write to local files in a variety of ways. Knowing how to deal with files in a clean way is a critical skill in a data wrangler's repertoire. Toward the end, we tested our newly learned skills by creating our own CSV parser.

In the next chapter, we will be dealing with the three most important libraries, namely NumPy, pandas, and matplotlib.