Candidate architectures

We have two viable choices when looking to extract and process business events as they occur. One of them is the new Microsoft StreamInsight engine and the other is the BizTalk integration bus.

Candidate architecture #1‑StreamInsight

SQL Server 2008 R2 includes StreamInsight as a new product for low latency processing of high-volume data streams. Watson Media is looking for a way to parse high volumes of data, perform data analysis in seconds, and join traditionally disconnected data sources. StreamInsight offers an efficient, high performing way to latch onto stream producers and handle a rich set of queries across streams.

Solution design aspects

One of the key requirements of this solution is the ability to quickly and reliably process large volumes of data. We are dealing with very small bits of information (log events, click events, and so on) that need to be rapidly consumed. StreamInsight is built to provide efficient, in-memory data stream processing against...