Hive query optimizers

After type checking and semantic analysis of the query, a number of rule-based transformations are applied to optimize the query. We will discuss some of these optimizations here. Custom optimizations can be written by implementing the org.apache.hadoop.hive.ql.optimizer.Transform interface. This interface has one method that takes in a ParseContext object and returns another after the transformation. The ParseContext object has the current operator tree, among other information.

The following are the few optimizations that are already available with Hive 0.13.0:

• ColumnPruner: This operator tree is walked to determine the minimal number of columns in the base table that are required to fulfill the query. Any additional columns in the base table are pruned away by inserting a SELECT statement when reading the base tables. This reduces the amount of data read, processed, and written.

• GlobalLimitOptimizer: When a LIMIT operator is used in a query, this particular optimizer...