9. Azure Big Data solutions
In the previous chapter, you learned about the various security strategies that can be implemented on Azure. With a secure application, we manage vast amounts of data. Big data has been gaining significant traction over the last few years. Specialized tools, software, and storage are required to handle it. Interestingly, these tools, platforms, and storage options were not available as services a few years back. However, with new cloud technology, Azure provides numerous tools, platforms, and resources to create big data solutions easily. This chapter will detail the complete architecture for ingesting, cleaning, filtering, and visualizing data in a meaningful way.
The following topics will be covered in this chapter:
- Big data overview
- Data integration
- Extract-Transform-Load (ETL)
- Data Factory
- Data Lake Storage
- Tools ecosystems such as Spark, Databricks, and Hadoop
- Databricks
