How to backfill years of minute bars

The short way

If you want US equity minute bars going back to 2016 and you are willing to paginate for them, Alpaca is the shortest route with a free key:

from alpaca.data.historical import StockHistoricalDataClient
from alpaca.data.requests import StockBarsRequest
from alpaca.data.timeframe import TimeFrame
from datetime import datetime

client = StockHistoricalDataClient("YOUR_API_KEY", "YOUR_SECRET_KEY")

request = StockBarsRequest(
    symbol_or_symbols=["SPY"],
    timeframe=TimeFrame.Minute,
    start=datetime(2016, 1, 1),
    end=datetime(2016, 2, 1),
)

bars = client.get_stock_bars(request).df    # multi-index DataFrame

Underneath it is one REST endpoint, and the three parameters that decide what you get back are easier to see there than through the client:

GET https://data.alpaca.markets/v2/stocks/bars
    ?symbols=SPY
    &timeframe=1Min
    &start=2016-01-01
    &end=2016-02-01
    &adjustment=all        # raw | split | dividend | spin-off | all
    &feed=sip              # sip | iex | boats | otc
    &limit=10000
    &page_token=<from the previous response>

limit caps at 10,000 and the documentation is explicit that it counts data points across every symbol rather than per symbol, so a multi-symbol request is not ten times the page. Anything longer comes back with a next_page_token you hand straight back as page_token. That loop is the whole job, and it is also the whole problem: a regular US session is 390 minutes, so one ticker-year is about 98,000 bars and ten pages, and a thousand tickers over ten years is a script that runs for days.

adjustment and feed are the two that quietly change the numbers rather than the volume of them. Both are discussed below.

What the options are

Paginate an API. Alpaca as above: stock history to 2016 on both plans, no flat files and no Parquet, so a backfill is JSON pages and nothing else. The free plan is the IEX feed alone and the paid one is $99 a month for the consolidated tape; a brokerage account and its onboarding are mandatory either way.

Flat files on a subscription. Massive — the vendor that traded as Polygon.io until October 2025 — serves minute aggregates over REST and the same data as flat files over an S3-compatible interface, which is the difference between ten thousand requests and one aws s3 sync. Read the history window rather than the headline price: Basic is free and holds two years, Starter at $29 a month holds five, Developer at $79 ten, and the 20-plus-year archive starts at Advanced, $199. The $29 and $79 tiers are 15-minute delayed, which does not matter for a 2019 backfill and matters a great deal if the same key is also feeding something live.

Buy the archive once. FirstRate Data sells zipped CSV you keep: 1-minute, 5-minute, 30-minute, hourly and daily bars, stocks and ETFs from January 2000, at $599.94 for 16,304 stock tickers or $839.94 for the combined bundle. Kibot is the same shape a little further back — minute and daily bars from 1998, gzipped CSV, $990 for all US stocks and ETFs. Both ship unadjusted and adjusted series side by side, which is the feature that matters most and is invisible on a pricing page. Neither is sliceable: a FirstRate bundle is organised by ticker range and always carries the full history, so there is no way to ask for one symbol or one year out of it.

Meter it by the byte. Databento has no subscription floor on historical data and bills the uncompressed gigabyte, which makes it the only sensible purchase when the requirement is one month of forty symbols rather than an archive. Bars are aggregated from the same exchange feed the live product reads, and the historical and live clients return identical record types.

Lease the researched version. AlgoSeek leases US equity data from 2007 — delisted names included — with minute bars carrying up to 90 fields rather than plain OHLCV: VWAP, trade count, buy and sell pressure, spread statistics. It comes with a security master that follows a ticker through mergers and delistings, and nightly-recalculated adjustment factors. It is $1,500 a month for a single dataset on a twelve-month minimum, and when the term ends you delete the files.

The split in that list is one decision, not five. An API is cheap to start and expensive to finish; a bought archive is the reverse. Everything else follows from which end of that you are on.

Where this breaks

The exchange licence is not in the vendor's price. US consolidated equity prices are administered under plans with their own published charge list, and on the schedule as published the numbers are the exchange's rather than any vendor's: $45 a month per display device for a professional subscriber using one or two of them, $1.00 a month per non-professional subscriber, $2,000 a month for non-display use of Network A last-sale information and the same again for quotation information, and $1,000 a month for redistribution on each network. None of that is avoidable by choosing a cheaper reseller, because none of it is the reseller's money.

For a backfill the good news is that most of it does not apply: AlgoSeek states that its historical data carries no exchange fees in any asset class, and the one-time vendors sell an archive rather than a feed. What survives is redistribution, and it survives inside the vendor's own licence rather than the exchange's. FirstRate Data permits derived charts and analysis and forbids handing users raw rows, and is explicit that dropping the volume column does not help. Kibot grants internal use on two computers and says the restriction outlives cancellation. Alpaca answers it in one line: you cannot redistribute Alpaca API data. If anything downstream of your database is a chart other people can see, that is the clause to read first. Why real-time stock data is so expensive has the mechanics of the live side.

Adjusted bars are a view, not a fact. Alpaca's adjustment parameter takes raw, split, dividend, spin-off and all, and the values combine. That flexibility is the warning: a series pulled with adjustment=all is computed against the corporate-action history as it stands today. Store it, wait for a 4-for-1 split, and every row you already wrote is now wrong by a factor of four relative to a fresh pull — not because anybody made a mistake, but because the question "what did this trade at" was answered in a currency that keeps being redenominated. The fix is to store unadjusted bars plus the split and dividend table and adjust on read, which is why FirstRate Data and Kibot both ship three parallel series and why reconciling against a broker statement needs the unadjusted one. Why adjusted close differs is the long version.

The tickers that stopped trading are the ones that matter. A universe assembled from what trades today has already removed every company that failed, and a backtest over it is measuring a group selected for having survived. The vendors differ sharply here and say so. FirstRate Data carries roughly 7,000 delisted names among 16,304, suffixed -DELISTED so a reused symbol does not collide, while stating plainly that the universe is incomplete and thins with age. Kibot answers the same question with "partially" — delisted names are included by how liquid they were at delisting — and warns that one symbol file can hold two unrelated companies separated by a gap. AlgoSeek includes delisted securities from 2007 and maintains a security master to keep them distinct. If the question is index membership rather than mere existence, Norgate Data does that job at daily resolution and historical index constituents explains why it is a separate purchase.

Sessions are a boundary you have to choose, not one you inherit. NYSE's own calendar puts the core session at 9:30 a.m. to 4:00 p.m. ET, order entry in the pre-opening session from 6:30 a.m., and a late session of 4:00 p.m. to 8:00 p.m. on NYSE American, NYSE Arca, NYSE National and NYSE Texas rather than on the Tape A market itself. Vendors draw their own lines inside that. Kibot ships pre-market from 8:00 AM and after-hours to 6:30 PM ET, and says filtering is looser outside the regular session so spikes get through. FirstRate Data timestamps the start of each bar in US Eastern — except crypto, which is UTC — and omits zero-volume bars entirely, so a gap means no prints rather than missing data. Alpaca puts overnight activity behind a separate boats feed value. Three consequences follow: your 09:30 bar may or may not include the opening auction, your bar count per day is not constant, and two vendors' "daily volume" will differ by whatever each decided extended hours meant. Pick one definition, write it down, and filter at load time rather than at query time.

The files are larger than the estimate. FirstRate Data's stock bundle is 28 GB zipped and the combined bundle 40 GB; Databento's own documentation tells you to move from streaming to batch download past 5 GB. A billion rows is not a CSV directory and it is not a Postgres table either. Storing tick data without regretting it covers the shapes that work, and ArcticDB, QuestDB and DuckDB are the cards behind it. Decide the storage layer before the download, because re-ingesting 40 GB because you partitioned by symbol instead of by date is a weekend.

If you outgrow this

When you need somewhere that is not the United States, everything above stops. Tick Data is the card that goes furthest: roughly thirty equity markets on five continents, sold by the symbol-year under a perpetual licence, with London from January 2000, Euronext from April 2002 and Tokyo, Hong Kong, Sydney and São Paulo from 2008. The minimum order is $1,000 for a new client.

When a bar stops being enough, the next thing is the tape itself — trades and quotes, from which you can build any bar you like and answer questions a bar cannot, such as where inside the spread a print landed. Databento and AlgoSeek both sell it, and both will tell you that US equities have no order book to buy: the consolidated tape is top-of-book, so depth exists in futures and nowhere else.

When the backfill becomes a pipeline, the job stops being a download and starts being storage, scheduling and reconciliation. That is tick data storage rather than this page.

If daily bars would have done, they usually would have, and they are three orders of magnitude smaller — how to download historical prices in Python is the cheaper version of this question.

The rest of the field is in market data APIs.