We present DBToaster, a novel query compilation framework for producing high performance compiled query executors that incrementally and continuously answer standing aggregate queries using in-memory views. DBToaster's motivating applications include data stream processing, online data warehouses, and financial applications.
DBToaster generates query engines that perform main-memory processing of standing queries (views), fed by high-volume data streams, by recursively compiling view maintenance (VM) queries into simple C++ functions for evaluating database updates (deltas).
While today's VM algorithms consider the impact of single deltas on view queries to produce maintenance queries, we recursively consider deltas of maintenance queries and compile to thoroughly transform queries into code. Recursive compilation successively elides certain scans and joins, and eliminates significant query plan interpreter overheads.
We also briefly present ongoing work on the Spread system, a distributed main-memory data warehouse built using DBToaster's recursive compilation algorithm, where rather than producing C++ code, DBToaster produces view maintenance messages that are processed by Spread nodes running in a cluster environment.
