Repository navigation
Options & markets analytics tool - #2
Merged
Merged
Conversation
Prices come from the vendor; everything analytical is derived here. The provider dataclasses have no fields for vendor greeks or impliedVolatility, so that boundary is enforced by the type definitions rather than by care. - black_scholes: BSM pricing plus the five analytic greeks, with a continuous dividend yield. Degenerate branches (T<=0, sigma<=0) collapse to discounted forward intrinsic instead of dividing by zero. No scipy — the normal CDF is math.erf, and chains are small enough that scalar Python is milliseconds. Verified against reference values and against finite differences of price(). - implied_vol: Newton-Raphson on vega with a bisection fallback for the wings, where vega underflows and the Newton step explodes. Price tolerance scales with the quote, and bisection also stops on bracket width — an absolute tolerance let sub-penny wing contracts satisfy the price test while sigma was still wrong in the fourth decimal. Round-trip error is now under 4e-9 across sigma 1e-4..5 and T 1e-4..2y. Every failure mode returns None with a named status and a human-readable explanation; none returns a number. - providers: MarketDataProvider interface, a yfinance implementation with backoff around Yahoo's throttling, and a FixtureProvider replaying a real 1,122-contract capture. The fixture is what makes the suite runnable with no network and keeps the demo working when the unofficial endpoint rate-limits, which it did during development. - chain: one tidy typed frame, nullable dtypes so "no reported volume" stays null rather than becoming a real zero. Unsolvable strikes are kept and labelled, not dropped. - cli: `python -m options_tool chain AAPL --expiry 2027-01-15`, with --provider fixture for the offline path. Live SPY 0-DTE output has delta running -0.04 to -0.96, gamma and vega both peaking at the ATM strike, and a correctly shaped put smile. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The IV-history problem is the reason this project is interesting: free providers don't sell historical implied volatility, so the tool accumulates its own from day one forward. - models: tickers / snapshots / contracts. Idempotency is a database guarantee — unique constraints on (ticker_id, snapshot_date) and (snapshot_id, expiry, strike, right) together make (ticker, date, expiry, strike, right) unique by construction, so a buggy writer still cannot duplicate a day. Each snapshot stores the risk-free rate and dividend yield in force at capture, without which a stored IV can't be interpreted later. - atm_iv: reduces a day's chains to one number — 30-day constant-maturity ATM IV, interpolated to the money within each expiry and then across expiries in total variance, since variance is what's additive in time. Returns None when 30 days can't be bracketed rather than extrapolating; an invented early value would become the window's minimum for a year. - snapshot: ON CONFLICT DO UPDATE upserts, chunked under SQLite's 999-parameter cap. One failing expiry or ticker doesn't cost the rest of the run. - iv_rank: the formula is trivial; refusing to compute it is the point. Below min_history_days the result carries rank=None plus "insufficient history (3/20 days)". A flat window returns undefined rather than a conventional 50. Percentile is reported alongside rank because one outlier sets the range rank is measured against but moves percentile by a single observation. - session: SQLite needs PRAGMA foreign_keys=ON for the schema's CASCADE to do anything, and WAL so the snapshot job doesn't block API reads. - alembic with render_as_batch=True, since SQLite can't ALTER a column in place. Initial migration applies and reverses cleanly. Verified end to end on the fixture: two runs the same day held stored contract rows at 1122 -> 1122, reported as "refreshed" not "captured", with zero duplicate (ticker, date, expiry, strike, right) groups in SQL. ivrank on a one-day database prints "insufficient history (1/20 days)". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nine endpoints, every response a Pydantic model, /docs browsable. - payoff: both curves — intrinsic at expiry, and every leg repriced through black_scholes.price at the remaining time for today. Reusing the M1 pricer is why a spread's greeks and its P/L curve cannot disagree. The extremes are derived analytically from the payoff's terminal slopes and its kink points, not read off the plotting grid. A first pass got this wrong in both directions: it called a long put's profit unlimited and a covered call's loss unlimited. Only the upside can be unbounded — the underlying can rise without limit but cannot fall below zero — so the downside extreme is exactly the value at S=0. Verified against hand-computed figures for long/ short calls and puts, vertical spreads, iron condors, collars and covered calls. Breakevens are solved segment-by-segment on the piecewise-linear payoff, so one beyond the outermost strike is still found. A leg with no implied volatility yields no today-curve and a reason, rather than a curve silently drawn at an assumed vol — on a chart there is nothing to distinguish the two. - errors: every failure is a structured body with a machine-readable code. The distinction that matters is 503 (Yahoo throttling, retryable) vs 404 (no such symbol). ExpiryNotFoundError was split out after an unlisted expiry came back as 502 implying the vendor was down; it is now a 404 carrying the available expiries so a client can correct itself in one round trip. The catch-all logs the traceback and returns internal_error — no bare 500 with a stack trace. - summarise_chain now returns a ChainStats dataclass rather than dict[str, object], which removed seven type: ignore casts at the route layer. - routes use Annotated dependencies, the current FastAPI idiom, so the dependency call is not evaluated in an argument default. Idempotency holds across interfaces: a snapshot taken by the CLI is recognised and refreshed by POST /snapshot rather than duplicated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The suite does not merely avoid the network, it is prevented from reaching it: pytest-socket runs with --disable-socket, and tests/test_offline.py asserts the block is really in place so the guarantee cannot rot when someone edits config. AF_UNIX stays permitted because asyncio builds its event-loop self-pipe from socketpair(); outbound AF_INET, which is the part that matters, is blocked. Coverage: 96% on analytics/, 94% overall. CI gates at 90% and 85% rather than quoting a number in the README. Writing the tests found three real defects: - **Snapshot date was the UTC date.** A daily job run after 20:00 Eastern filed under the following day, so a 15:00 run and a 20:30 retry on the same trading afternoon produced two rows — defeating idempotency in exactly the scenario a cron timer creates. Snapshots are now keyed on the US market date. - **IV solver tolerance scaled to the wrong quantity.** It scaled to the total premium, but only the extrinsic part carries information about volatility. A deep ITM put worth $59.36 with four hundredths of a millionth in time value satisfied the price tolerance while sigma was wrong in the fourth decimal. Scaling to extrinsic value, plus a Newton termination on step size rather than residual alone, took that case from 4.2e-4 error to 2.8e-9. - **Payoff extremes** (fixed in M3, now regression-covered by golden data). Three test assumptions were wrong and got corrected rather than the code: gamma checked by a second difference of price is dominated by float64 cancellation at h=1e-5, so it is now checked as the first derivative of delta; delta is monotone in strike only at constant volatility, so strictness is asserted on synthetic constant-vol data and only trend on a real smile; and an ATM option's time value decays as sqrt(T), so near-expiry convergence is asserted against that analytic bound instead of a guessed constant. tests/fixtures/payoff_golden.json is read by both this suite and (from M5) the vitest suite, so the Python engine and the TypeScript preview cannot diverge without a test going red. CI: ruff, ruff format, mypy, pytest on 3.11 and 3.12, a migration up/down/up check, a CLI smoke test that greps for "refreshed" and "insufficient history", and a job that fails if a .env is ever tracked. The web job gates on web/package.json existing so it switches itself on when M5 lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI caught this on the first run: pinning python_version = "3.11" made mypy reject numpy's own stubs under 3.12 with "Type statement is only supported in Python 3.12 and greater" — the stubs installed for 3.12 use PEP 695 syntax that a 3.11 target cannot parse. Following the running interpreter checks each matrix entry against the stubs actually installed for it. Python 3.11 was already fully green: lint, format, types, 459 tests, the migration up/down/up check and the offline CLI smoke test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three pages — watchlist, chain, spread builder — verified end to end against a
running API with real data, not just a passing build.
The IV-rank card is the visible half of the project's central decision. When
there is not enough stored history it renders "insufficient history (1/20 days)"
with a progress bar toward the window and today's ATM IV alongside, at the same
visual weight as a real reading. A blank would look like a loading bug and a
zero would read as "volatility is at its lows"; neither is what the data says.
Nullable analytics stay nullable the whole way to the screen. lib/format.ts
renders every missing value as an em dash and nothing coalesces null to zero, so
no component can print a zero that looks like a measurement. Chain rows whose IV
did not solve are kept, greyed, and labelled with the reason in place of the
number.
lib/payoff.ts duplicates the Python payoff maths so the builder redraws
instantly while a leg is edited. Duplication is exactly how two implementations
drift, so both are pinned to the same golden fixture: vitest reads
api/tests/fixtures/payoff_golden.json and reproduces all eleven structures to
nine decimals, including the long put (finite profit) and the ratio spread (two
breakevens, unlimited loss on a debit).
Errors are typed end to end. The client distinguishes a throttled vendor
(retryable, with a "this usually clears within a minute" note) from an unknown
ticker (not retryable, so no retry button), and an unreachable API says how to
start it rather than surfacing "Failed to fetch".
Two fixes found by screenshotting the running app rather than trusting the
build: Recharts animates lines on mount, so the first paint depended on wall
time and rendered an empty chart — now isAnimationActive={false}, which is also
the deterministic choice. And the presets carried no per-leg volatility, so a
new user's first view was the today-curve's unavailable state rather than the
working one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Multi-stage builds for both services: wheels compiled in a stage that carries a toolchain so the runtime image does not ship one, Next.js standalone output so the web image copies the modules actually reached rather than the whole tree, non-root users, and health checks that test whether the service can actually serve rather than merely whether the process is alive. This machine has no Docker, so these would otherwise ship written and unbuilt. CI now builds both images and smoke-tests them: the API container must answer /health with database_reachable true, and the snapshot command must print "captured" on its first run and "refreshed" on its second inside the container, with the stored row count unchanged. That turns the deploy story from "the syntax looks right" into "this compiles and runs" — while still stopping short of claiming a deployment. The IV history is the only irreplaceable state, so it lives in a named volume rather than the container filesystem, and deploy/README.md documents a backup one-liner alongside the upgrade order (migrate, then start new code). The timer fires weekdays at 21:15 UTC, which is after the US equity close in both EST and EDT — capturing mid-session would make the stored volatility depend on what time the job happened to run. Persistent=true is set deliberately: a day missed to a reboot is a permanent gap, because historical implied volatility cannot be bought from a free source or backfilled. The README and deploy/README.md both state plainly that nothing has been deployed to a host yet, and no resume bullet claims otherwise. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- screener: elevated or depressed IV rank, plus volume and open interest against their own trailing median. Median rather than mean because option volume is spiky — one expiry-week spike would drag a mean upward for weeks and suppress every later reading. Below ten stored days it declines to judge activity at all, on the same reasoning as the IV-rank guard: "2x the median" over six days describes the sample, not the market. It reports how many symbols it screened alongside the hits. An empty result would otherwise be ambiguous between "nothing was flagged" and "nothing could be checked", and the symbols it could not check are named with the reason. - earnings: an optional provider method defaulting to None, so a source without forward earnings dates does not break the chain view. The yfinance implementation parses the three shapes Yahoo's calendar returns and swallows its failures — losing a decorative flag is acceptable, losing the chain because the flag failed is not. The API returns 200 with known:false rather than a 404, since "we do not know" is a different answer from "none is scheduled". - export: daily and per-contract CSV. Missing values are written empty rather than zero — a blank cell reads as "not available" in a spreadsheet, a 0 reads as a measurement. The daily export carries the risk-free rate and dividend yield per row so a stored IV stays interpretable outside this tool. 509 tests, 97% coverage on analytics (screener 100%). README gains a "what I would do differently" section naming Black-Scholes vs American exercise, the zero dividend assumption, and SQLite's ceiling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tejas12972
marked this pull request as ready for review
September 13, 2026 01:33
…ling Restructures the deployment rather than bolting auth onto the existing shape. The API now has no public address at all — it is reachable only over Fly's private network, from the Next.js server, which proxies to it. Three problems disappear instead of being managed: - one auth check covers everything, because the browser can only reach the API through /api/* on the web app, behind the same middleware as every other route - CORS stops applying, so there is no allow-list to keep in sync with a domain - the API address becomes runtime config; it used to be compiled into the client bundle, so an image only worked against the host it was built for Auth is HTTP Basic in Next.js middleware, with a constant-time comparison and both sides always compared so a wrong username and a wrong password take the same path. /healthz is exempt because Fly's checker cannot present credentials, and weakening the middleware to let it through would be the worse trade. 19 tests cover the ugly cases: malformed base64, missing scheme, a password containing a colon, and near-miss credentials. The daily snapshot moves into the API process. A Fly volume attaches to exactly one machine, and the job writes to the same SQLite file the API reads, so a separate scheduled machine could not open it. It catches up on start — if the machine was down through the scheduled time and today has no snapshot, it runs immediately — because a missed day is a permanent hole that no free source can backfill. A failing run is logged and the loop survives it; tomorrow still needs to run. Two real bugs fixed: - **Startup crashed on any database the CLI had touched.** init_db() builds the schema with create_all and leaves no alembic_version row, so `upgrade head` tried to create the tables again and died with "table tickers already exists". Anyone who ran `options-tool snapshot` before first starting the API could not start it at all. The schema is now adopted by stamping. Covered by test_startup.py, which reproduces the original failure. - **No .dockerignore**, so every build uploaded 751MB of context including api/.venv and web/node_modules — and a host-built .venv of macOS binaries could shadow the Linux one inside the image. Also: /live split from /health, because a readiness probe that touches the database will restart a container for a storage problem that restarting cannot fix. Request ids on every response and log line, since one chain request fans out into several vendor calls and retries. JSON logging for deployments. Migrations applied at startup before serving. /health reports whether the scheduler is running, so a deployment that silently stopped accumulating history is visible rather than discovered weeks later when IV rank never arrives. CI gains an auth test against the real web image: /healthz open, everything else 401, correct credentials 200. 547 Python tests, 48 TypeScript. Verified end to end locally — auth blocks, the proxy serves, and the UI renders identically through the new same-origin data path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One command after `fly auth login`: creates both apps, the volume and the login secret, deploys API before web (the web config resolves the API by internal DNS, so the reverse order fails its health check), and seeds the watchlist — without which the daily job has nothing to snapshot. Re-runnable after a partial failure. It skips anything that already exists rather than erroring, and will not silently rotate a password that is already set, since Fly secrets are write-only and an unannounced rotation would lock you out with no way to read the new value back. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Without an explicit as_of, snapshot_ticker fell back to datetime.now() while taking prices from the provider. For a live vendor the two are the same instant; for a replayed fixture they are not, so the capture was aged past its quotes and filed under the wrong day. Once the fixture's front expiry (2026-09-14) went behind the calendar, the 30-day point could no longer be bracketed, atm_iv_30d was correctly null, and eight tests began failing on the date alone. Each chain is now priced at its own as_of, matching what the /chain route and the CLI already do, and the snapshot is filed under the first chain's market date. An explicit as_of still wins. Live behaviour is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds out
SPEC.md, turning this repo from a LICENSE-and-README stub into aworking options analytics tool. Milestones 1–7 are complete and CI is green.
The two decisions that are the point
Greeks are computed here, not read from the vendor. Yahoo ships
deltaandimpliedVolatilitycolumns; this tool ignores both. Prices come from theprovider, and Black-Scholes-Merton pricing, all five analytic greeks, and the
implied-volatility solve (Newton-Raphson with a bisection fallback) happen
locally.
OptionQuotehas no field a vendor greek could live in, so theboundary is enforced by the type definitions rather than by discipline.
IV history is built, not faked. IV rank needs historical implied volatility,
which free APIs do not sell. Rather than substitute realised volatility, a daily
snapshot job captures the full chain into SQLite and history accumulates from day
one. Each day reduces to a 30-day constant-maturity ATM vol, interpolated across
expiries in total variance because variance is what is additive in time. Until
20 stored days exist, IV rank returns
nulland the UI renders"insufficient history (3/20 days)" rather than a zero that would read as
"volatility is at its lows".
Milestones
Verified, not asserted
against finite differences of this project's own pricer
the wings where Newton hands off to bisection
Nonewith a named reason, never a numberthe database rejects a duplicate even with the upsert bypassed
pytest-socket;tests/test_offline.pyasserts the block is actually in place
fixture shared with the vitest suite so Python and TypeScript cannot diverge
and both the fixture and live-data CLI paths work
Three real bugs this found
under the next day — two runs on one trading afternoon would have created two
rows and double-weighted that day in the rank window.
value carries volatility information. A deep ITM put worth $59 with a sliver
of time value passed the price check while its sigma was wrong in the fourth
decimal. Scaling to extrinsic took it from 4.2e-4 error to 2.8e-9.
loss unlimited. Only the upside can be unbounded — the underlying can rise
without limit but cannot fall below zero.
What this does not claim
Nothing here is deployed. The Docker images build and are smoke-tested in CI
(the API container must answer
/health, and the snapshot must reportrefreshedon its second in-container run with an unchanged row count), but nohost is running them. The README and
deploy/README.mdboth say so.No claim of profitability, returns, or backtested edge appears anywhere. Not
investment advice; the data source is an unofficial Yahoo scraper and the README
says that too.
🤖 Generated with Claude Code