Tips and tricks with Databricks notebooks
Since we will go through notebooks, it makes sense to mention a few tricks you can do with Databricks notebooks.
Magic
Databricks notebooks can use magic, which involves mixing in some type of non-Python component using the % syntax.
Markdown
Markdown is an advantageous way to format text, much like HTML, but it’s much simpler to write and learn. To invoke a Markdown cell, simply type %md at the start of your cell.
Other languages
When working with notebooks, it can be handy to run a command in a language other than what the notebook is set up to run. The language magic works by typing %[language] at the start of the notebook cell. For example, you can invoke %sql, %r, %scala, and %python using the language magically. Keep in mind that it is impossible to pass variables between languages, and the context of the language magic is limited to the cell itself.
Terminal
To gain terminal access to the driver node, use...
