Apache Tez

Apache Tez is part of the Stinger initiative led by Hortonworks to make the Hive enterprise ready and suitable for interactive SQL queries. The Tez design is based on research done by Microsoft on parallel and distributed computing.

Tez entered the Apache Incubator in February 2013 and graduated to a top-level project in July 2014.

Tez is basically an embeddable and extensible framework to build high-performance batch and interactive data-processing applications that need to integrate easily with YARN.

Confusion often arises when Tez is thought of as an engine. Tez is not a general-purpose engine, but more of a framework for tools to express their purpose-built needs. Tez, for example, enables Hive, Pig, and others to build their own purpose-built engines and embed them in those technologies to express their purpose-built needs. Projects such as Hive, Pig, and Cascading now have significant improvements in response times when they use Tez instead of MapReduce.

Tez generalizes the...