One Billion Days of Options Data + Huge API Update

It has been a while since our last post. We have opted to do more sporadic, higher-quality updates.

One Billion Days of Options Data

Our 4 nodes are still hammering away at processing the 10+ years of historical data we took delivery of. We have reached 1 billion days / entries of data recorded - a huge milestone. We expect to have the full 10 years recorded by late September / Early October.

Adding more capacity

Yesterday our data center operations team conducted an installation of a midplane drive cage on NJ1. This will allow us to increase our storage capacity for that server by 80 Terabytes!

Huge API Update V3.0

Make sure you update your API and terminal! The latest versions of the API and Terminal can be found here The changelog is below.

What’s Happening Next Week?

We will be pushing out an API and terminal update that contains some quality of life improvements such as the option to request open, high, low, close (OHLC) data on any interval you want! Our next Python API update will also eliminate the hassle using a terminal and will run it for you.

- Bailey Danseglio CEO @ Theta Data

Free Historical Options Data + New API Features

Free Options Data

Access historical End of Day (EOD) data with our python API by using the EOD request type. This data is completely free - all you need is a Theta Data account. Below is a code snippet for making a EOD request with our python API. We will work to add more free datatypes in the future, so stay tuned!

Terminal & API Version 0.2.0 is Here

We are constantly improving our service. Our API engineers implemented two customer suggestions this week. One of which was to add a method for killing the terminal directly from the API and the other is a configuration option to toggle polling vs blocking input. A polling operation keeps trying to read data, which uses a lot of system resources because it keeps the processor at a higher clock rate / busy. This can reduce latency and improve performance for downstream processing of the data. A blocking message uses less system resources and sleeps until a message is available. Blocking adds a small amount of latency. It is now used by default. As always, the latest version of the terminal can be found on the API page.

What’s happening next week?

We’re still working to add Greeks-related data types. These data types will help customers measure different types of risk on option contracts. We will be adding free historical equities data for all options API subscribers. The amount of historical data each customer has access to will be the same as their options data permissions.

A Cool Fact

Our quest for processing 10 years of historical options data will end when we record our last historical day, June 1st, 2012.

Thanks for tuning in, check back in a week for more!

- Bailey Danseglio CEO @ Theta Data

Towards 27 Million Years of Data

This is our first blog post. We will post weekly updates every Friday. Without further ado, let's jump into what's happening at Theta Data.

Processing 10 years of options data

Uncompressed, this data is a whopping 4 petabytes. Yes, that's 4 million gigabytes. Our data center operations (DC OPs) team built 4 nodes to process the data. These nodes process a combined 16 million quotes / trades per second. We are averaging around 2 months of historical data added to our database every day.

Node specs: AMD 5950x (16-core), 64GB of RAM, 500GB NVME SSD.

Free Data

We are working diligently to add free End of day (EOD) data. EOD data contains the open, high, low, close, trade prices along with the total amount of trades (count) and the total contracts traded (volume).

What's happening next week?

We will add support for all the request types listed here. But wait, there's more... We will also have 2 years of data recorded by next Friday.

A Cool Fact

By the time we're finished recording the 10 years of data, the database will have over 10 billion entries. Each entry is a day of data for a certain contract, that equates to 27+ million years of options data.

Thanks for tuning in, check back in a week for more!

- Bailey Danseglio CEO @ Theta Data