Summary
In this chapter, we dove headfirst into Photon Engine. We discussed the standard Apache Spark execution model and what has made Apache Spark so fast. Then, we discussed the prevalent query engine design models and why the vectorization model was chosen to replace the code generation design of Apache Spark. We learned about the core concept of vectorization and how it enables Photon to be as fast as it is. Finally, we discussed what Photon can and cannot do now and what its known feature roadmap is.
Before we end this chapter, I will provide you with one final reminder – the aim of this chapter is only to give you a conceptual idea of how Photon works and why is it so fast. All the concepts have been simplified for better understanding. To deep dive into the nuances, follow the content in the section Further Reading.
With that, we have a complete understanding of the Databricks SQL toolset and its storage and computation technologies. In the next chapter, we will...
