Chapter 7: Pachyderm Operations
In Chapter 6, Creating Your First Pipeline, we created our first pipeline, as well as learning how to create Pachyderm repositories, put data into a repository, create and run a pipeline, and view the results of the pipeline. We now know how to create a standard Pachyderm pipeline specification and include our scripts in it so that they can run against data in our input repository.
In this chapter, we will review all the different ways to put data inside of Pachyderm and export it to outside systems. We will learn how to update the code that runs inside of your pipeline and what the process of updating the pipeline specification is. We will learn how to build a Docker container and test it locally before uploading it to a registry.
We will also look into the most common troubleshooting steps that you should perform when a pipeline fails.
This chapter will cover the following topics:
- Reviewing the standard Pachyderm workflow
- Executing...
