Navigating through data

The problem with data collection starts at the very beginning: every data provider offers its own data, quite often in its own format. Some data providers offer only compressed data or snapshots (see later in this chapter) while others broadcast tick-by-tick data; very few data providers also offer order book (depth of market or DOM; see Chapter 3, FX Market Overview from a Developer’s Standpoint) data.

So, first and above all, you should decide on the data granularity: whether your trading algorithm requires tick-by-tick or compressed data, and if you need DOM data or not. At this stage, you may feel lost with these questions, but don’t worry—you will get a very clear understanding of your data needs when you’ve accomplished further chapters of this book.

Let’s consider all three cases and see how we can actually handle ticks, snapshots, and DOM market data.

Tick data and snapshots

I would like to reiterate that...