lucyd
Open app

How to backtest a prediction market strategy on real order book data

· Lucyd

Most prediction market strategies look great on a chart of last prices and fall apart in live trading. The reason is execution: the price you saw was not the price you could get, your order sat behind others in the queue, and by the time it reached the book the market had moved. A useful backtest has to replay the order book, not the price line.

lucyd-backtest is a free, open-source toolkit that does exactly that for Lucyd's Bitcoin Up or Down markets. It downloads the recorded L2 order book, every match of the order book and the Pyth BTC/USD prices, and runs your strategy over them with latency, slippage and queue position. It works from Python (pip install lucyd-backtest) and from Rust, and the code is on GitHub under the MIT license.

What gets tested

Every market asks one question: will BTC be higher at the end of its period than at the start? The price at the start is the strike, and the market settles on the Pyth BTC/USD price at the end (how settlement works). Two tokens, Up and Down, trade between 0.0025 and 0.9975 USDC; the winner pays 1 USDC. Rounds run back to back in six lengths: 5 minutes, 15 minutes, 30 minutes, 1 hour, 4 hours and 1 day.

The same question is traded on Polymarket and Kalshi, so a strategy idea tested here carries over, with one big difference: Lucyd has no trading fees, while on Polymarket's 15-minute crypto markets the taker fee can be larger than the edge of a short-term strategy (fee comparison).

Install

pip install lucyd-backtest

Python 3.9 or newer on Linux, macOS or Windows. The engine is written in Rust and ships compiled inside the wheel, so there is nothing to build. The only dependencies are pandas and pyarrow.

Step 1. Load the data

One call picks the markets by series and date range, downloads what they need and caches it. A finished market is never downloaded twice.

import lucyd_backtest as lb
data = lb.load(period="15m", start="2026-10-26", end="2026-10-28")
data.markets          # one row per market: start, end, strike, final price, winner, volume
data.fills()          # every match of the order book: price, size, side, settlement
data.prices()         # Pyth BTC/USD ticks
data.book_updates()   # recorded book snapshots and deltas
data.events()         # all of it as one time-ordered stream

Everything is a pandas DataFrame, so you can explore it in Jupyter before writing a strategy. You can also rebuild the full book of any market at any moment:

book = data.book_at(data.markets.address[0], "2026-10-26T00:05:00Z")
book.best_bid("up"), book.asks("down")

The same files are public on the data page as Parquet if you prefer DuckDB or your own tools. The order book history goes back to early October 2026, fills to about October 20 and Pyth prices to October 27. Gaps show up in data.warnings, never as errors.

Step 2. Write a strategy

A strategy is a class with handlers for the events it cares about. This one waits until 30 seconds before the end of each round and buys the side Bitcoin is on, if it is not too expensive:

class LateSide(lb.Strategy):
    timer_ms = 1000
    def __init__(self, seconds_before=30, max_price=0.95):
        self.seconds_before, self.max_price, self.done = seconds_before, max_price, set()
    def on_timer(self, ctx):
        for m in ctx.open_markets():
            if m.address in self.done or m.time_to_end_ms > self.seconds_before * 1000:
                continue
            btc = ctx.market_price(m.address)
            if m.strike is None or btc is None:
                continue
            side = "up" if btc.price > m.strike else "down"
            ask = ctx.best_ask(m.address, side)
            if ask and ask.price <= self.max_price:
                ctx.market_order(m.address, side, "buy", 20, limit=self.max_price)
                self.done.add(m.address)

The context ctx shows the state as of now and nothing later, so look-ahead bias cannot sneak in. Other handlers: on_market_open, on_book (the book changed, optionally throttled), on_fill (someone else traded), on_price (Pyth tick), on_order (your order was accepted, filled, cancelled or rejected), on_resolution and more. Orders are ctx.limit(...) for a resting GTC order and ctx.market_order(...) for fill-and-kill with an optional worst price.

Step 3. Run it

report = lb.backtest(data, LateSide(), initial_usdc=1000, submit_latency_ms=100)
report                          # in Jupyter: the full interactive report
report.summary["pnl"], report.validation
report.save_html("report.html")

The same data, parameters and seed give the same report, bit for bit, so you can compare two versions of a strategy fairly.

How orders are filled

This is where a backtest earns its keep. The engine follows the rules of the real order book:

  • Latency. An order reaches the book 100 ms after the decision by default, a cancel takes effect 100 ms later, and the order can fill in between. Add random jitter to see how sensitive the strategy is.
  • Slippage. A taking order walks the recorded book from the best price, and what it took stays taken until the recording shows new orders at that level.
  • Queue position. A resting order joins the back of the queue at its price. Recorded fills eat the queue in front of it before it fills; if the book trades through its price, it fills fully. Two queue models are available: pessimistic (cancellations at your price were behind you) and proportional.
  • One ladder per market. A buy of Up at p meets a buy of Down at 1 − p, as on the real exchange, so liquidity is never counted twice.
  • Settlement. Winning tokens pay 1 USDC at resolution, and a voided round pays 0.5 per token. Optional settlement failures and fees let you stress-test the result.

The limits, stated plainly: other traders do not react to your orders, and queue position is an estimate. The execution notes describe the model in full.

Read the report

The last 30% of the date range is held out and reported separately as the validation sample. Tune the strategy on the first part, judge it on the second; a strategy that only works on the data it was tuned on is overfit.

The report includes P&L and return, maximum drawdown and how long it lasted, Sharpe and Sortino ratios, win rate and profit factor by market, the longest losing streak, exposure, average taker slippage and the fill rate of limit orders. A metric the sample cannot define comes back empty rather than as a misleading infinity, and too few fills trigger a warning.

The HTML version is one self-contained file that opens offline and reads well on a phone: the equity curve with the validation split, drawdown, P&L per market, and the price path of each market with your own buys and sells on it.

Rust and the command line

The same engine is a Rust crate with a command-line tool. Scaffold a project with a working example strategy and run it:

cargo install --git https://github.com/lucyd-trade/lucyd-backtest
lucyd-backtest new my-strategy
cd my-strategy
cargo run --release

The CLI also lists markets, downloads data, exports the event stream to Parquet or CSV, and runs the built-in example strategies, such as a two-sided market making ladder, with latency, jitter and queue settings as flags.

Let an AI agent write the strategy

The repository includes an AGENTS.md that tells Claude Code, Cursor, Codex and other coding agents how to scaffold a project, which data exists for which dates and how to read the report. Ask something like: "create a strategy that buys the side BTC is on 30 seconds before the end of 5-minute markets, backtest it on yesterday and show the validation metrics". When the strategy holds up, the Lucyd MCP server lets the same agent trade it on live markets.

Before you trust the numbers

Lucyd runs on the Ethereum Sepolia testnet with test USDC, and most of its liquidity comes from the platform's own market maker. The books and fills are real records of the exchange, but not of a market full of independent traders. Use the toolkit to build and test strategies and tooling; do not read the P&L as a forecast of real-money returns. To check a strategy live without risk, run it as a trading bot on testnet.

FAQ

Is lucyd-backtest free? Yes. The code is MIT-licensed on GitHub, the package is on PyPI, and the historical data is free to download without a key.

Can I backtest Polymarket Bitcoin Up or Down strategies with it? The data is Lucyd's, but the markets ask the same question on the same kind of order book, so the toolkit is a practical place to test an Up or Down idea with real execution. Keep in mind that Polymarket charges taker fees on these markets and Lucyd does not.

Do I need Rust? No. pip install lucyd-backtest ships the compiled engine; Rust is only needed to use the crate directly.

What data does it use? The L2 order book of every round, every match of the order book with its settlement status, on-chain trades and Pyth BTC/USD prices. Details are on the data page.

How realistic are the fills? Orders pay latency, walk the recorded book and wait in the queue behind recorded orders. Other traders do not react to yours, so treat results on thin books with care.