Designing Databricks: Day One

“Design is not just what it looks like and feels like. Design is how it works.”

- Steve Jobs

This chapter will introduce concepts and topics that engineers, data scientists, and people in similar roles should know to set themselves up for success in the Databricks Data Intelligence Platform. When setting up your data and AI platform, in our case, Databricks, there are always best practices to follow. We share those in this chapter to give you a better understanding of the setup options and their impacts; these can be strategic decisions that impact the entire data product workflow, as well as simply matters of preference. We start by explaining Databrick’s general architecture and key terminology, then cover the most important decisions to be made during platform setup, and conclude with code examples and configurations to download the data for our example projects. We also introduce a variety of platform features and components...