IBKR August Improvement & Development Handover

The highest-value improvement is not adding isolated API commands. It is turning the existing CLI and reports into a single, source-aware research data service:

CLI / nightly / weekly
        |
        v
Unix-socket daemon queue
        |
        v
One IBKR connection + one _ib_lock
        |
        v
Normalized research collectors
        |
        +--> quote / batch quote / snapshot
        +--> portfolio --enrich
        +--> nightly report
        +--> weekly report
        +--> AI advisor

The development team should implement the following four decisions first:

1. Daemon-only IBKR access. Nightly and weekly reports must not open additional direct IBKR connections while the daemon is active. All IBKR requests go through the daemon queue and _ib_lock.
2. Source-aware rich data contract. Every result must expose source, timestamp, age, actual market-data type, per-field availability, and request errors. Delayed, stale, missing, and not-entitled data must never be represented as live or as zero.
3. Research baseline. Add rich generic-tick quotes, held-option liquidity/risk data, and cached daily historical bars. These three features have the best value for research, nightly reports, weekly reports, and AI recommendations.
4. One-call agent bundle (new P0 driver). Hermes currently needs 3–4 CLI calls and roughly 2–3 minutes to gather analysis data. Collapse that into one ibkr bundle call served by parallel IBKR + Futu collection and background caches, with a hard latency budget: warm ≤ 3s, cold ≤ 10s, worst case bounded 60s. Futu is the primary speed lever for universe-wide market data; IBKR remains the authority for account and execution state.

The existing IBKR Improvement Plan July 2026.md already defines much of the target rich snapshot. This document turns that direction into a complete API assessment, report strategy, implementation sequence, test plan, and handover checklist. It supersedes no existing code or plan until explicitly approved.

2.1 Primary ib_async sources

• ib_async API reference, release 2.1.0
• ib_async client source
• ib_async objects source
• ib_async IB implementation

The repository already depends on and uses ib_async throughout src/kit_ibkr/core/connection.py, src/kit_ibkr/daemon.py, src/kit_ibkr/core/orders.py, src/kit_ibkr/core/flexquery.py, and the CLI/report paths. This plan adds collectors and contracts around more of the existing ib_async API; it does not propose adopting ib_async from scratch. The research session inspected this repository's .venv, which reports ib_async version 2.1.0. That is evidence about the local environment only, not a claim about the development/deployment environment or the latest release. The repository currently declares ib_async>=1.0.0 in pyproject.toml; before implementation, inspect the target environment's version and signatures, then pin only after compatibility testing.

2.2 Primary IBKR sources

• Historical data limitations and pacing
• Historical bars
• Available tick types
• Market-data requests
• Market-data type: live/frozen/delayed
• Market-data subscriptions
• Option computations
• Options and chain discovery
• Fundamental data and WSH
• WSH corporate-event filters
• News
• Market scanners
• PnL subscriptions
• Executions and commissions
• Market depth

2.3 Repository evidence

• Daemon queue, locking, dispatch, reconnect: src/kit_ibkr/daemon.py
• IBKR wrapper: src/kit_ibkr/core/connection.py
• Current quote/batch quote/details CLI: src/kit_ibkr/cli/main.py
• Nightly data collection: src/kit_ibkr/report/data.py
• Nightly analysis: src/kit_ibkr/report/analysis.py
• Weekly aggregation and historical prices: src/kit_ibkr/report/weekly.py
• Upcoming earnings/economic data: src/kit_ibkr/report/upcoming.py
• Futu nightly enrichment: src/kit_ibkr/report/futu_enrich.py
• Current Futu capability inventory: docs/futu-api-inventory.md
• Existing rich-market-data direction: IBKR Improvement Plan July 2026.md
• Daemon architecture: archive/Daemon-v2-Design.md
• Nightly report contract: docs/Nightly-Report.md
• Weekly report contract: docs/Weekly-Report.md

3.1 Existing daemon architecture is the correct authority boundary

DaemonV2 currently owns:

• one Unix-socket server;
• one command queue;
• one IBKR client;
• _ib_lock for serializing IBKR operations;
• per-command timeout handling;
• reconnect handling;
• command dispatch for portfolio, quote, batch quote, chain, details, orders, executions, and roll checks.

This is the correct place for new research API calls. ib_async is event-driven and the repository already treats the gateway connection as a single shared resource.

3.2 Current quote output is too small for research

_cmd_quote_locked() and _cmd_batch_quote_locked() currently return primarily:

{
  "price": 399.02,
  "close": 402.29,
  "bid": 398.98,
  "ask": 399.06
}

Option quotes add basic model/bid/ask Greek fields, but the output does not preserve:

• Ticker.marketDataType;
• range fields;
• average volume and real-time volume;
• raw fundamental ratios;
• dividends;
• shortable shares;
• option volume/open interest;
• request and receipt timestamps;
• field-level availability or IBKR error details.

3.3 Reports still open additional gateway connections

src/kit_ibkr/report/data.py uses REPORT_CLIENT_ID = 88 and opens a direct connection. src/kit_ibkr/report/weekly.py uses WEEKLY_CLIENT_ID = 87 for historical prices. The code comments already warn that a daemon plus a second direct connection can cause connection resets or timeouts.

This is the highest-priority architectural gap. Adding more API calls to those direct paths would increase the failure surface.

3.4 Weekly history is underused and has ambiguous fallback provenance

The weekly gateway path currently:

• opens a direct client;
• requests one-day TRADES bars one symbol at a time;
• keeps first close, last close, change percentage, and sparkline;
• falls back to Yahoo for missing data.

It does not currently calculate or expose:

• OHLC and volume history;
• 5D/1M/3M/6M/1Y returns;
• realized volatility;
• ATR;
• SMA20/SMA50/SMA200;
• maximum drawdown;
• relative performance versus SPY/QQQ;
• exact historical request parameters and age.

The current code can also label an overall result as gateway when Yahoo filled individual missing symbols. The final contract must carry provenance per symbol.

3.5 Existing Futu integration is useful but has a strict role

The repository already uses Futu for read-only supplemental data:

• earnings calendar;
• economic calendar;
• Fed watch;
• short interest;
• capital flow;
• market snapshots, bars, option chains, and strategy analysis.

Per AGENTS.md and docs/futu-api-inventory.md, Futu must not become the source of truth for:

• IBKR positions;
• account cash;
• fills/executions;
• P&L;
• orders;
• BAG execution.

Every Futu result must remain explicitly labelled source=futu. This August extension now treats Futu as a first-class supplemental data plane, not merely an emergency fallback. Futu may be the preferred source for research fields that IBKR does not expose cheaply (capital flow, analyst consensus, valuation detail, financial breakdowns, macro calendar, and research content) and may run independently to reduce report wall-clock time. It must remain separately source-labelled and must never overwrite IBKR/Flex account, portfolio, fills, orders, P&L, BAG, or execution truth.

1. IBKR first. If the gateway and daemon are healthy, use IBKR for current price, range, fundamentals, history, options, contract metadata, and portfolio analysis.
2. One daemon connection. No nightly or weekly direct IBKR connection while the daemon is available. Do not create parallel direct connections for research calls.
3. Read-only scope. This plan must not add automatic order placement, order modification, cancellation, rolling, rebalancing, or execution strategy logic.
4. Backward compatibility. Existing top-level quote keys (price, close, bid, ask) remain available. Rich fields are additive or nested.
5. No silent fallback. A missing IBKR field returns null plus an availability state. External fallback is opt-in or explicitly documented in the result with source, timestamp, and reason.
6. No disguised delay. Expose actual Ticker.marketDataType: live, frozen, delayed, or delayed-frozen. Do not call delayed data live.
7. No fabricated values. Missing, not-entitled, unsupported, timeout, and error states are not converted into zero, empty success, or an inferred value.
8. Preserve raw provider data. Keep raw FundamentalRatios, raw XML, raw WSH JSON, and raw headline identifiers alongside normalized fields.
9. Bound every request. Every async request has an outer deadline. Every subscription has try/finally cleanup.
10. Keep trading math authoritative. PMCC, vertical-spread, max-profit, max-loss, DTE, and defined-risk calculations remain in core/ and are not reimplemented in Futu or report templates.
11. Keep account truth authoritative. IBKR and Flex remain the only account/position/fill/P&L/order authorities. Futu is never account truth.
12. Do not treat indicators as predictions. Historical-derived indicators are context for the advisor, not forecasts.

5.1 Contract identity and market metadata

reqContractDetailsAsync() returns no result for an unknown contract and can return multiple results for ambiguity. The normalized contract identity must distinguish unknown_contract and ambiguous_contract from a network timeout.

5.2 Rich stock and ETF market data

reqMktData(contract, genericTickList, snapshot=False, regulatorySnapshot=False, mktDataOptions=[]) returns a Ticker that fills progressively. The following generic ticks are the primary research set:

The standard Ticker also exposes bid/ask/last/open/high/low/close/volume/VWAP, timestamps, marketDataType, and contract identity.

The collector must use reqMktData() for generic ticks, wait for a bounded period, copy the populated fields into a plain normalized object, and call cancelMktData(contract) in finally. reqTickersAsync() alone does not request the generic tick list.

5.3 Held-option liquidity and risk

For selected held option contracts, request:

Option Ticker objects retain bidGreeks, askGreeks, lastGreeks, and modelGreeks. Each OptionComputation documents:

impliedVol, delta, optPrice, pvDividend, gamma, vega, theta, undPrice

The report should preserve side-specific Greeks and identify whether the displayed value is model-, bid-, ask-, last-, or locally-computed. The current modelGreeks or bidGreeks or askGreeks selection is convenient but loses provenance.

The research layer may derive:

• bid/ask midpoint;
• spread in dollars and percentage;
• put/call volume ratio;
• put/call open-interest ratio;
• IV minus HV;
• distance to strike and breakeven;
• DTE and percent of max profit.

It must not claim dealer positioning, opening/closing flow, aggressor side, or institutional intent because these fields are not supplied by the documented API.

5.4 Historical bars and schedules

reqHistoricalDataAsync() returns BarDataList containing BarData fields:

date, open, high, low, close, volume, average, barCount

Supported research-relevant whatToShow values include:

TRADES
MIDPOINT
BID
ASK
BID_ASK
ADJUSTED_LAST
HISTORICAL_VOLATILITY
OPTION_IMPLIED_VOLATILITY

Recommended scheduled requests:

• nightly: compact daily context for held underlyings;
• weekly: one-year daily bars for held underlyings plus SPY/QQQ;
• on-demand: requested history horizon, with explicit cost/latency;
• intraday minute bars: only on explicit snapshot/research request.

The repository must use reqHistoricalScheduleAsync() or contract session metadata instead of assuming every asset follows a hardcoded NYSE calendar.

5.5 Fundamental data and analyst estimates

reqFundamentalDataAsync(contract, reportType, fundamentalDataOptions=[]) returns a raw XML string. Documented report types are:

ReportsFinSummary
ReportsOwnership
ReportSnapshot
ReportsFinStatements
RESC
CalendarReport

RESC is the analyst-estimates path. CalendarReport is a company calendar, not a macroeconomic calendar.

Implementation requirements:

• retain raw XML;
• parse and store report date/latest available date;
• preserve units and period labels;
• cache by contract/report type;
• classify entitlement and parse failures explicitly;
• normalize only ratio aliases observed in real payloads and protected by tests;
• never treat unavailable ratios or IBKR sentinel values as zero;
• do not present stale cached XML as live market data.

5.6 WSH corporate events

The documented path is:

metadata = await ib.getWshMetaDataAsync()
events = await ib.getWshEventDataAsync(WshEventData(...))

The official IBKR documentation describes earnings dates, dividend dates, option expirations, splits, spinoffs, and conferences. WSH requires the Wall Street Horizon Enchilada Pro subscription. Metadata must be requested before event data, and the official documentation warns that TWS does not support multiple concurrent metadata/event requests.

Use WSH as an optional IBKR corporate-event source. Keep Futu for economic calendar and Fed watch. Do not silently replace the existing Futu event source until entitlement, payload shape, and date semantics are verified in this account.

5.7 News

Documented methods:

reqNewsProvidersAsync()
reqHistoricalNewsAsync(conId, providerCodes, startDateTime, endDateTime, totalResults)
reqNewsArticleAsync(providerCode, articleId)

Historical news is limited to 300 headlines per request by the documented API. IBKR news requires API-specific provider subscriptions. The normalized result should retain provider code, article ID, headline, timestamp, and optional article body status. News is a weekly/on-demand enrichment, not a nightly baseline.

Before exposing a public news command, verify the installed wrapper's aggregation semantics for reqHistoricalNewsAsync() because the installed signature returns HistoricalNews | None, not a statically typed list.

5.8 Scanners

reqScannerDataAsync() takes ScannerSubscription and returns a one-shot scan result. Scanner fields include:

numberOfRows
instrument
locationCode
scanCode
abovePrice / belowPrice
aboveVolume
marketCapAbove / marketCapBelow
averageOptionVolumeAbove
stockTypeFilter

Official IBKR limits include a maximum of 50 results per scan code and 10 active API scans. Scanner output contains contract information, not complete bid/ask/last/volume data. A scanner command must therefore use a bounded top-N follow-up rich-quote step.

Scanners are for on-demand research. Do not run a full-universe scan in every nightly report.

5.9 Account, PnL, executions, and commissions

Existing account and portfolio state already uses IBKR account summary and portfolio objects. Additive improvements:

• reqExecutionsAsync(execFilter) / Fill for recent execution and commission details;
• commission amount, currency, realized P&L, exchange, order reference, and liquidity fields in weekly attribution;
• reqPnL(account, modelCode) and reqPnLSingle(account, modelCode, conId) for current live subscriptions.

In this repository's inspected .venv (ib_async 2.1.0), reqPnLAsync and reqPnLSingleAsync do not exist. This is not a portability claim about other installed versions. The development team must verify the target version separately and use only confirmed signatures; where reqPnL/reqPnLSingle are available, account for their synchronous subscription-start and event/cancel lifecycle rather than assuming undocumented async variants.

PnL subscriptions provide current/intraday data; they do not reconstruct historical per-position attribution. Historical attribution requires persisted nightly snapshots reconciled against Flex equity and cash flow.

5.10 Deferred microstructure and pricing APIs

The following APIs are documented but not suitable for the default nightly/weekly baseline:

• reqMktDepth() / reqMktDepthExchangesAsync();
• reqTickByTickData();
• reqRealTimeBars();
• reqHistoricalTicksAsync();
• calculateImpliedVolatilityAsync();
• calculateOptionPriceAsync();
• reqHistogramDataAsync().

They remain useful for explicit on-demand execution or scenario research. They require strict cleanup, entitlement handling, bounded request counts, and clear data semantics.

P0-1: Make the daemon the only IBKR data plane

Value: Critical Effort: Medium Risk: Medium Cadence: Every CLI/report IBKR request

Change report/data.py and report/weekly.py so reports request IBKR data through the daemon rather than creating client IDs 87 and 88. If the daemon is unavailable, reports should use an explicitly labelled Flex/Futu/Yahoo fallback path or fail the requested IBKR section; they must not silently open another competing gateway connection.

Acceptance:

• no report path opens a second IBKR connection when the daemon is healthy;
• all research commands are serialized under the daemon's _ib_lock;
• daemon remains responsive after a timeout or canceled subscription;
• existing account/order commands continue to work;
• direct connection code is removed or restricted to an explicit standalone diagnostic mode.

P0-2: Introduce a source-aware research envelope

Value: Critical Effort: Small/Medium Risk: Medium Cadence: Every quote, snapshot, report section, and fallback

Create MarketSnapshot and shared provenance/availability models. The proposed availability vocabulary is:

available
missing
not_entitled
unsupported
timeout
error
stale

Every result should include:

source
source_detail
requested_at
received_at
as_of
age_ms
market_data_type
availability
errors
request

Use null for absent values. Keep source and age at both result and field/section level where mixed data is possible.

P0-3: Add generic-tick rich quote collection

Value: Critical Effort: Medium Risk: Medium/High Cadence: Default quote and nightly held-underlying snapshot

Implement stock/ETF generic tick collection for 165,221,233,236,258,293,294,295,411,456. Preserve all existing basic keys. Add range, volume quality, raw ratios, dividends, market type, and contract identity.

Do not request every optional field for every symbol if the account is not entitled. The collector should record an error/availability state and continue returning basic price data.

P0-4: Add held-option liquidity and risk enrichment

Value: High Effort: Medium Risk: Medium/High Cadence: Nightly held legs, near-strike legs, and explicit option research

Use option generic ticks 100,101,104,105,106 only for held legs or a bounded user-selected strike set. Add bid/ask/mid/spread, IV/HV, side-specific/model Greeks, option volume/OI, underlying price, and risk-quality flags.

Do not request a full chain's every strike during nightly generation.

P0-5: Replace weekly price-only history with cached IBKR historical context

Value: High Effort: Medium Risk: Medium Cadence: Weekly baseline; nightly compact context; on-demand full snapshot

Create a historical bar cache keyed by qualified contract and exact request parameters. Add derived metrics:

• period return;
• realized volatility;
• ATR;
• SMA20/SMA50/SMA200 when enough bars exist;
• maximum drawdown;
• average volume and relative volume;
• benchmark-relative performance.

All derived metrics must carry the source bars, date window, and formula label.

P0-6: Fix report fallback provenance

Value: High Effort: Small/Medium Risk: Medium Cadence: Nightly/weekly

The current report may mix gateway and Yahoo data under an overall source label. Change output to per-symbol/per-section provenance. If Yahoo remains available as emergency fallback, label:

{
  "source": "yahoo",
  "fallback_reason": "ibkr_history_unavailable",
  "as_of": "..."
}

Do not use external market data as IBKR account, position, execution, or P&L truth.

P1-1: Add contract metadata and session-aware calendar data

Use reqContractDetailsAsync() and reqHistoricalScheduleAsync() to improve details, history windows, timezone handling, early-close handling, and US/HK correctness.

P1-2: Add fundamental XML and analyst-estimate research

Add an on-demand snapshot --fundamentals section and a weekly cached refresh for selected holdings. Use generic 258 as a cheap baseline and XML reports as optional deep research.

P1-3: Add WSH corporate events as an optional IBKR source

Probe metadata, cache it, serialize event requests, and return NOT_ENTITLED when unavailable. Use for corporate events only. Keep Futu for economic calendar/Fed watch.

P1-4: Add execution and commission attribution

Extend ibkr executions and weekly advisor input with commission, currency, realized P&L, exchange, order reference, and liquidity fields. Continue using Flex for historical trade and cash-flow completeness.

P1-5: Add bounded on-demand scanner research

Expose ibkr screen or equivalent. Return scanner rows and optionally enrich only the top N with rich quote data. Never run full-universe scan as a report prerequisite.

P1-6: Carry Flex freshness and completeness into reports

Preserve from_cache, last_fetch, cache age, Flex code 1019 warnings, and statement-generation lag in the nightly/weekly data contract. A stale Flex report must be visible to the AI advisor and HTML/Discord output.

P2-1: Add optional news digest

Add provider probing, historical headline collection, deduplication by provider/article ID, and optional article retrieval. Use weekly/on-demand only.

P2-2: Persist daily PnLSingle snapshots for attribution

Use reqPnLSingle() with explicit cancellation to capture current position PnL. Persist snapshots and reconcile against Flex NLV/cash-flow. Do not claim retrospective attribution from a single current snapshot.

P2-3: Add IBKR scenario/model validation

Expose calculateImpliedVolatilityAsync() and calculateOptionPriceAsync() as on-demand model tools. Keep raw IBKR model result separate from local Black-Scholes approximations and Futu strategy-analysis output.

P2-4: Add optional microstructure research

Consider market depth, tick-by-tick, real-time bars, and historical ticks only for explicit user requests. They are not report-baseline data.

7.1 Common provenance envelope

{
  "source": "ibkr",
  "source_detail": "IBKR Gateway / paper / US.SMART",
  "requested_at": "2026-08-16T22:00:00Z",
  "received_at": "2026-08-16T22:00:02Z",
  "as_of": "2026-08-16T21:59:58Z",
  "age_ms": 2000,
  "market_data_type": "delayed",
  "availability": "available",
  "errors": [],
  "request": {
    "contract_id": 265598,
    "use_rth": true
  }
}

Allowed market_data_type values:

live
frozen
delayed
delayed_frozen
unknown

Allowed availability values:

available
missing
not_entitled
unsupported
timeout
error
stale

7.2 Market snapshot contract

{
  "symbol": "AAPL",
  "identity": {
    "conid": 265598,
    "sec_type": "STK",
    "exchange": "SMART",
    "primary_exchange": "NASDAQ",
    "currency": "USD",
    "multiplier": "1",
    "local_symbol": "AAPL",
    "trading_class": "NMS"
  },
  "market": {
    "bid": null,
    "ask": null,
    "last": null,
    "mark": null,
    "close": null,
    "open": null,
    "high": null,
    "low": null,
    "volume": null,
    "vwap": null,
    "bid_size": null,
    "ask_size": null
  },
  "range": {
    "low_13w": null,
    "high_13w": null,
    "low_26w": null,
    "high_26w": null,
    "low_52w": null,
    "high_52w": null,
    "average_volume": null,
    "position_in_52w_pct": null,
    "distance_to_52w_high_pct": null
  },
  "quality": {
    "spread_dollars": null,
    "spread_percent": null,
    "relative_volume": null,
    "trade_count": null,
    "trade_rate": null,
    "volume_rate": null,
    "shortable_shares": null,
    "rt_hist_volatility": null
  },
  "fundamentals": {
    "raw_ratios": null,
    "normalized": {},
    "report_date": null
  },
  "dividends": {
    "past_12_months": null,
    "next_12_months": null,
    "next_date": null,
    "next_amount": null
  },
  "provenance": {},
  "availability": {}
}

7.3 Option snapshot contract

Additive option fields:

bid, ask, last, close, mid
spread_dollars, spread_percent
implied_volatility, historical_volatility
bid_greeks, ask_greeks, last_greeks, model_greeks
call_volume, put_volume
call_open_interest, put_open_interest
average_option_volume
underlying_price
DTE, intrinsic_value, time_value, breakeven_distance
risk_flags

Each Greek record must preserve its source and side. Locally computed Greeks must be labelled approximate and must not overwrite IBKR model data.

7.4 Historical contract

{
  "source": "ibkr_historical",
  "contract_id": 265598,
  "start_date": "2025-08-16",
  "end_date": "2026-08-16",
  "duration": "1 Y",
  "bar_size": "1 day",
  "what_to_show": "TRADES",
  "use_rth": true,
  "timezone": "America/New_York",
  "bars": [],
  "derived": {
    "return_1m_pct": null,
    "return_3m_pct": null,
    "return_6m_pct": null,
    "return_1y_pct": null,
    "realized_volatility_pct": null,
    "atr": null,
    "sma20": null,
    "sma50": null,
    "sma200": null,
    "max_drawdown_pct": null
  },
  "availability": "available",
  "errors": []
}

Derived fields must state the formula/window in code or metadata. Insufficient history returns null and missing, not a fabricated value.

8.1 Existing commands: additive enrichment

ibkr quote SYMBOL
ibkr batch quote SYMBOL [SYMBOL ...]
ibkr portfolio

Keep current top-level fields and add nested rich sections. Human display may remain compact; JSON output must retain the full agent-facing contract.

8.2 New agent-oriented snapshot

ibkr snapshot SYMBOL
ibkr snapshot SYMBOL --history 1y
ibkr snapshot SYMBOL --fundamentals
ibkr snapshot SYMBOL --events
ibkr snapshot SYMBOL --options --expiry YYYYMMDD
ibkr snapshot SYMBOL --history 1y --fundamentals --events

A snapshot is one serialized daemon job. Optional sections fail independently: a missing WSH entitlement must not discard a valid price/history section.

8.3 Enriched portfolio

ibkr portfolio --enrich

Default portfolio remains fast. --enrich adds held-underlying snapshots, held-option liquidity, market-data type, source/freshness, and quality flags. No watchlist symbols may be mixed into holdings.

8.4 Capability diagnostics

ibkr data capabilities SYMBOL

This should show which generic ticks, fundamental reports, WSH/news providers, history windows, and market-data types are available or unavailable. It must not silently call Yahoo.

8.5 On-demand screen and news

P1/P2 commands may be exposed after the core data plane is stable:

ibkr screen --scan-code HOT_BY_VOLUME
ibkr news SYMBOL --days 7

Both commands must return source, entitlement, request, and per-symbol failure information.

8.6 Nightly report additions

Nightly should consume, for current holdings and held option legs:

• 52-week range position;
• mark/bid/ask/VWAP;
• average and real-time volume where available;
• IV/HV and option volume/OI;
• next dividend date/amount;
• actual market-data type and age;
• missing/entitlement/wide-spread flags;
• Flex cache age and completeness;
• optional WSH corporate events when entitled.

The AI advisor must receive source and quality labels so it does not treat approximate Greeks, delayed quotes, Yahoo fallback, or stale Flex data as live facts.

8.7 Weekly report additions

Weekly should consume:

• cached IBKR daily bars;
• 1M/3M/6M/1Y returns;
• realized volatility, ATR, moving averages, drawdown;
• per-symbol source and fallback reason;
• benchmark-relative performance;
• valuation/analyst-estimate snapshots;
• WSH corporate events if entitled;
• Futu economic calendar and Fed watch, explicitly labelled Futu;
• execution commission and realized P&L attribution;
• data completeness, cache age, and missing session warnings.

All tasks below are read-only research/data tasks. Each task must use the repository's existing TDD conventions, skip unrelated formatting/refactoring, and run focused tests before the next task. No task may add order execution behavior.

DP-00 — Freeze dependency and source policy

Priority: P0 Dependencies: None Suggested effort: Small

Files:

• pyproject.toml
• AGENTS.md if command/source policy changes
• this document

Work:

1. Record and test the supported ib_async version; do not leave the project dependent on an unbounded >=1.0.0 API surface.
2. Confirm the source hierarchy:

• IBKR: current market, contracts, options, account and portfolio;
• Flex: historical account/trade/equity/cash-flow fallback;
• Futu: explicitly labelled supplemental market/macro data;
• Yahoo: explicit emergency fallback only, never silent.

1. Confirm read-only scope and no auto-order behavior.

Acceptance:

• Version is reproducible in CI/development environments.
• Source hierarchy is documented and agreed by the development team.
• No feature task can silently change account/P&L authority.

DP-01 — Move report IBKR work behind the daemon

Priority: P0 Dependencies: DP-00 Suggested effort: Medium

Files:

• src/kit_ibkr/daemon.py
• src/kit_ibkr/core/connection.py
• src/kit_ibkr/report/data.py
• src/kit_ibkr/report/weekly.py
• scripts/nightly_report.py
• scripts/weekly_report.py
• tests/unit/test_daemon_client.py
• tests/unit/test_fallback.py
• tests/unit/test_weekly.py

Work:

1. Add daemon request types for report-safe research operations.
2. Replace report direct client IDs 87/88 for IBKR research calls.
3. Keep report-level Flex fallback if the daemon is unavailable.
4. Make direct connection use an explicit standalone diagnostic path only, if still needed.
5. Preserve report timeout and reconnect semantics.

Acceptance:

• Running the daemon and nightly/weekly concurrently does not create a second gateway session.
• A timed-out research request cancels cleanly and does not leave a stale subscription.
• Existing report fallback tests continue to pass.

DP-02 — Create the normalized research schema

Priority: P0 Dependencies: DP-00 Suggested effort: Small/Medium

Files:

• Create src/kit_ibkr/models/market_data.py
• Modify src/kit_ibkr/models/__init__.py
• Create tests/unit/test_market_data.py

Work:

1. Define common provenance and availability structures.
2. Define MarketSnapshot, OptionSnapshot, and historical result structures using repository conventions.
3. Retain raw provider payloads.
4. Serialize None fields intentionally.
5. Preserve old quote fields at the top level through adapter serializers.

Acceptance:

• Schema always includes identity, market, quality, availability, and provenance.
• Valid availability states are enforced.
• source, market_data_type, requested/received timestamps, and age are present.
• Missing values remain distinguishable from zero.
• Existing quote/batch tests can accept additive fields.

DP-03 — Build public IBClient research wrappers

Priority: P0 Dependencies: DP-01, DP-02 Suggested effort: Medium

Files:

• src/kit_ibkr/core/connection.py
• tests/unit/test_connection.py

Work:

Add focused wrappers for:

• generic market-data collection;
• historical bars;
• historical schedules;
• contract details and market rules;
• fundamentals;
• WSH metadata/events;
• news providers/headlines/articles;
• executions/commission details;
• bounded scanner data;
• explicit subscription cleanup.

Do not expose private client._ib calls from report code after migration.

Acceptance:

• Every wrapper raises a clear not-connected error.
• Every async wrapper is externally bounded by its caller.
• Subscription wrappers expose the object needed for cancellation.
• Sync PnL subscription methods are documented accurately; no fake async method names are introduced.

DP-04 — Implement generic-tick collector and rich quote

Priority: P0 Dependencies: DP-02, DP-03 Suggested effort: Medium

Files:

• Create src/kit_ibkr/core/market_data.py
• Modify src/kit_ibkr/daemon.py
• Modify src/kit_ibkr/cli/main.py
• Modify tests/unit/test_batch_quote.py
• Modify tests/unit/test_market_data.py

Work:

1. Qualify one contract per requested symbol.
2. Request baseline generic ticks with reqMktData().
3. Wait for required baseline data with a bounded timeout.
4. Copy fields into plain data structures.
5. Always cancel the ticker in finally.
6. Map marketDataType 1/2/3/4 to live/frozen/delayed/delayed-frozen.
7. Capture errorEvent details without discarding valid fields.
8. Preserve raw ratios and dividends.

Acceptance:

• ibkr quote SYMBOL retains existing price/close/bid/ask keys.
• Rich quote exposes range, volume quality, ratios/dividends where available.
• Delayed data is labelled delayed.
• Generic tick failure returns field-level availability, not a failed whole quote.
• Invalid symbols remain isolated in batch results.
• Market-data subscriptions are canceled on success, timeout, and exception.

DP-05 — Enrich held options and portfolio strategies

Priority: P0 Dependencies: DP-04 Suggested effort: Medium/High

Files:

• src/kit_ibkr/daemon.py
• src/kit_ibkr/core/portfolio.py
• src/kit_ibkr/models/position.py
• src/kit_ibkr/models/combo.py
• src/kit_ibkr/report/data.py
• src/kit_ibkr/report/analysis.py
• tests/unit/test_portfolio_raw.py
• tests/unit/test_greeks.py
• tests/unit/test_market_value.py
• tests/unit/test_market_data.py

Work:

1. Request option generic ticks only for held legs and bounded selected strikes.
2. Preserve model/bid/ask/last Greek provenance.
3. Add volume/OI/HV/IV and spread quality.
4. Add underlying mark and market-data type.
5. Add flags for missing bid/ask, wide spread, delayed data, unavailable Greeks, and unavailable option entitlement.
6. Keep combo-engine max-profit/max-loss/PMCC math unchanged.

Acceptance:

• Nightly option legs expose executable bid/ask/mid context.
• Local approximate Greeks cannot silently overwrite IBKR model Greeks.
• Defined-risk and PMCC calculations remain numerically unchanged.
• Full-chain requests are not introduced into nightly.

DP-06 — Build historical-bar cache and derived metrics

Priority: P0 Dependencies: DP-01, DP-03 Suggested effort: Medium

Files:

• src/kit_ibkr/core/market_data.py
• Create or extend a research cache module under src/kit_ibkr/core/
• src/kit_ibkr/daemon.py
• src/kit_ibkr/report/weekly.py
• tests/unit/test_weekly.py
• Create tests/unit/test_historical_market_data.py

Work:

1. Cache by conId/contract, duration, bar size, whatToShow, RTH flag, timezone, and date window.
2. Use one-day RTH bars for weekly baseline.
3. Add return, volatility, ATR, SMA, drawdown, volume, and benchmark calculations.
4. Use reqHistoricalScheduleAsync() or contract timezone/session data for date filtering.
5. Implement bounded serial requests and exact-request deduplication.
6. Carry source, request parameters, bar timestamps, and completeness.
7. Migrate weekly report away from direct private _ib.reqHistoricalDataAsync().

Acceptance:

• Repeated identical requests use cache and do not trigger duplicate gateway traffic.
• Incomplete history is visible and does not fabricate metrics.
• Historical request pacing is respected.
• Weekly data identifies each symbol's actual source.
• Derived metrics have deterministic focused tests.

DP-07 — Integrate provenance into nightly/weekly/AI/HTML

Priority: P0 Dependencies: DP-02, DP-04, DP-05, DP-06 Suggested effort: Medium

Files:

• src/kit_ibkr/report/data.py
• src/kit_ibkr/report/analysis.py
• src/kit_ibkr/report/weekly.py
• src/kit_ibkr/report/advisor.py
• src/kit_ibkr/report/nightly_html.py
• src/kit_ibkr/report/weekly_html.py
• src/kit_ibkr/report/discord.py
• tests/unit/test_analysis.py
• tests/unit/test_html_v3.py
• tests/unit/test_advisor.py
• tests/unit/test_weekly.py

Work:

1. Add data-quality banners for delayed/stale/not-entitled/incomplete fields.
2. Add research sections to AI prompts with source and age.
3. Keep existing HTML section contracts and make new sections conditional.
4. Fix mixed gateway/Yahoo provenance per symbol.
5. Surface Flex cache age, from_cache, 1019 warning, and settlement completeness.
6. Keep report generation successful when optional enrichment fails.

Acceptance:

• AI prompt cannot mistake approximate, delayed, stale, or fallback data for live data.
• HTML/Discord reports show source and completeness warnings.
• Optional research failure does not break core portfolio report.
• Existing fixture data remains backwards compatible.

DP-08 — Fundamentals and analyst estimates

Priority: P1 Dependencies: DP-03, DP-04 Suggested effort: Medium/High

Files:

• src/kit_ibkr/core/connection.py
• src/kit_ibkr/core/market_data.py
• src/kit_ibkr/daemon.py
• src/kit_ibkr/cli/main.py
• Create fundamental parser/cache module under src/kit_ibkr/core/
• tests/unit/test_fundamentals.py

Work:

1. Add snapshot --fundamentals.
2. Support ReportSnapshot, ReportsFinSummary, and RESC first.
3. Store raw XML and report metadata.
4. Add entitlement/parse/stale states.
5. Normalize only aliases proven by fixtures from actual account payloads.
6. Refresh weekly or on demand, not for every nightly symbol by default.

Acceptance:

• Raw XML is retained.
• Report date/latest data date is visible.
• Entitlement errors return not_entitled.
• Unknown ratio tags do not break parsing.
• No stale report is presented as live.

DP-09 — WSH corporate-event integration

Priority: P1 Dependencies: DP-03, DP-07 Suggested effort: Medium/High

Files:

• src/kit_ibkr/core/connection.py
• src/kit_ibkr/daemon.py
• src/kit_ibkr/cli/main.py
• src/kit_ibkr/report/upcoming.py
• src/kit_ibkr/report/weekly.py
• src/kit_ibkr/report/nightly_html.py
• src/kit_ibkr/report/weekly_html.py
• tests/unit/test_wsh_events.py
• tests/unit/test_upcoming.py

Work:

1. Request and cache WSH metadata once per gateway session/day.
2. Serialize event requests; never issue concurrent WSH metadata/event requests.
3. Add snapshot --events or ibkr events.
4. Normalize earnings/dividends/expirations/splits/spinoffs/conferences while preserving raw JSON.
5. Return not_entitled when WSH subscription is absent.
6. Keep Futu economic calendar and Fed watch unchanged and source-labelled.

Acceptance:

• No WSH entitlement does not fail weekly report.
• Duplicate concurrent WSH requests are impossible within the daemon.
• Event dates carry source and timezone/as-of data.
• Existing Futu macro data remains available.

DP-10 — Execution and commission attribution

Priority: P1 Dependencies: DP-01, DP-07 Suggested effort: Small/Medium

Files:

• src/kit_ibkr/core/connection.py
• src/kit_ibkr/daemon.py
• src/kit_ibkr/cli/main.py
• src/kit_ibkr/report/weekly.py
• src/kit_ibkr/report/advisor.py
• tests/unit/test_orders.py
• Create or extend tests/unit/test_executions.py

Work:

1. Extend executions output with commission, currency, realized P&L, exchange, order reference, order ID, and liquidity fields when present.
2. Keep Flex as the longer historical trade authority.
3. Add weekly grouping by symbol/strategy where reliable.
4. Distinguish gross P&L, commission, and net realized P&L.

Acceptance:

• Existing executions users retain current fields.
• Commission is never treated as realized P&L.
• Weekly attribution clearly states API time coverage and Flex coverage.

DP-11 — Bounded on-demand scanner

Priority: P1 Dependencies: DP-03, DP-04 Suggested effort: Medium

Files:

• src/kit_ibkr/core/connection.py
• src/kit_ibkr/daemon.py
• src/kit_ibkr/cli/main.py
• tests/unit/test_scanner.py

Work:

1. Add a safe scanner request schema.
2. Expose scan code, market/location, row limit, price/volume/market-cap filters.
3. Return scanner contracts with rank and metadata.
4. Optionally enrich a bounded top-N with rich quote.
5. Enforce 50-row and active-scan limits.
6. Keep scanner out of nightly/weekly baseline.

Acceptance:

• Invalid scan parameters return validation errors.
• Scanner subscription is canceled on all paths.
• Top-N enrichment has a hard limit.
• Per-symbol quote failure does not discard scanner rows.

DP-12 — Optional news digest

Priority: P2 Dependencies: DP-03, DP-07 Suggested effort: Medium

Files:

• src/kit_ibkr/core/connection.py
• src/kit_ibkr/daemon.py
• src/kit_ibkr/cli/main.py
• src/kit_ibkr/report/weekly.py
• src/kit_ibkr/report/advisor.py
• tests/unit/test_news.py

Work:

1. Probe subscribed providers first.
2. Fetch bounded historical headlines for selected held symbols.
3. Deduplicate by provider code and article ID.
4. Retrieve article bodies only when explicitly requested.
5. Return provider entitlement and article-body availability.

Acceptance:

• No news subscription produces an explicit unavailable state.
• A headline request never requires article-body success.
• Weekly digest remains bounded and source-labelled.

DP-13 — Persisted PnLSingle snapshots

Priority: P2 Dependencies: DP-05, DP-07 Suggested effort: Medium/High

Files:

• src/kit_ibkr/core/connection.py
• src/kit_ibkr/daemon.py
• Create research snapshot persistence module
• src/kit_ibkr/report/weekly.py
• tests/unit/test_pnl_snapshots.py

Work:

1. Start reqPnLSingle() only for selected held contracts.
2. Capture current values and daily PnL.
3. Cancel each subscription explicitly.
4. Persist timestamped snapshots.
5. Reconcile aggregate values with Flex NLV and cash flow.
6. Flag reconciliation differences rather than silently adjusting them.

Acceptance:

• No PnL subscription survives a report job.
• Historical attribution is based on persisted snapshots, not retrospective inference.
• Reconciliation output is visible to reports and AI.

DP-14 — Scenario and microstructure tools

Priority: P2 Dependencies: DP-04, DP-05 Suggested effort: Small/Medium per tool

Work:

Add only when an actual user workflow requires them:

• calculateImpliedVolatilityAsync() and calculateOptionPriceAsync() for on-demand model validation;
• reqMktDepth() for selected execution-quality checks;
• reqTickByTickData() / historical ticks for explicit research;
• reqRealTimeBars() for selected intraday studies.

Acceptance:

• Every subscription has strict cancellation.
• Results are not added to nightly baseline.
• Raw inputs and model assumptions are visible.
• No scenario result mutates portfolio marks or order behavior.

DP-15 — Rollout, smoke test, and operational handover

Priority: P0 after implementation Dependencies: DP-01 through the selected release scope Suggested effort: Medium

Work:

1. Run focused unit tests for every changed contract.
2. Run a paper-gateway read-only smoke test.
3. Run nightly with --no-ai --no-discord.
4. Run weekly with --no-ai --no-discord --no-gateway and gateway mode.
5. Verify daemon health and queue responsiveness during snapshots.
6. Verify report behavior when gateway, Futu, WSH, news, and Flex are independently unavailable.
7. Update docs/Nightly-Report.md, docs/Weekly-Report.md, AGENTS.md, and the relevant operator runbook after behavior is finalized.

Acceptance:

• Read-only smoke commands complete without order-side effects.
• Every unavailable data path is labelled.
• Daemon reconnects after a canceled/failed research request.
• Nightly and weekly HTML/Discord output clearly distinguishes IBKR, Flex, Futu, and explicit external fallback.
• Rollback to the previous report contract remains possible through additive fields.

10.1 Async/event-loop policy

The ib_async documentation's core rule is that user code must not block the event loop for too long. Development rules:

• use *Async methods where documented;
• use asyncio.wait_for() at the daemon command boundary;
• never use time.sleep() in async collectors;
• do not issue new requests recursively inside an event handler;
• move expensive local calculations outside the locked IBKR request section when safe;
• use IB.sleep() only where an ib_async event-loop wait is specifically needed.

10.2 Subscription cleanup

Always cancel in finally:

cancelMktData
cancelHistoricalData
cancelMktDepth
cancelTickByTickData
cancelRealTimeBars
cancelScannerSubscription
cancelWshMetaData
cancelWshEventData
cancelPnL
cancelPnLSingle

A timeout is not complete until the corresponding request/subscription has been canceled or the connection has been safely reset.

10.3 Error classification

The repository currently captures errorEvent, while ib_async defaults RaiseRequestErrors=False. New collectors must avoid interpreting an empty response as success. Map errors to:

CONTRACT_NOT_FOUND
AMBIGUOUS_CONTRACT
NOT_ENTITLED
UNSUPPORTED
TIMEOUT
PACING
GATEWAY_DISCONNECTED
PARSE_ERROR
STALE_CACHE
UNKNOWN_ERROR

Include IBKR request ID/error code/message where available.

10.4 Request budgets

ib_async.Client has automatic transport throttling with default MaxRequests=45 per RequestsInterval=1 second. This is not the same as IBKR historical-data pacing.

The development team must additionally respect the official historical limitations:

• maximum 50 simultaneous historical requests;
• no identical small-bar request within 15 seconds;
• no six or more same contract/exchange/tick requests in two seconds;
• no more than 60 historical requests in ten minutes;
• BID_ASK counts twice;
• request only a few thousand bars per request;
• cancel requests that take minutes instead of retrying rapidly.

Use exact-request caches and serialized/small queues. Never use broad asyncio.gather() over an unbounded symbol or strike list.

10.5 Time and timezone policy

• formatDate=2 is required when UTC-aware intraday timestamps are desired.
• No timezone in an IBKR historical request uses the TWS login timezone.
• Use contract/session metadata for US/HK/futures rather than assuming UTC or NYSE.
• Store timezone, useRTH, and the exact requested window in every history result.
• Date completeness warnings must distinguish holiday/early close from missing data.

These ideas are not part of the P0 baseline:

1. Macro/economic calendar through ib_async. No documented ib_async macro-calendar endpoint was found. Keep Futu economic calendar and Fed watch as source-labelled supplemental data.
2. Duplicate Futu capabilities. Do not rebuild Futu earnings, economic, Fed, short-interest, capital-flow, option-chain, bars, or strategy-analysis services through IBKR unless there is a documented data-quality reason.
3. Full-universe nightly scanner. Scanner output requires follow-up quotes and has documented row/active-scan limits.
4. Full option chain every night. Chain metadata does not provide every strike's quote/IV/OI; selected strikes only.
5. Silent Yahoo primary/fallback market data. If retained operationally, source and fallback reason are mandatory. Never use it as IBKR account/P&L truth.
6. Market depth/tick-by-tick report baseline. Valuable for execution research, not required for nightly/weekly portfolio analysis.
7. Permanent streaming subscriptions by default. Snapshot evaluation is safer; persistent watchlists require separate state, reconnect, deduplication, and alert-spam controls.
8. Historical attribution reconstructed from current PnL. Persist daily snapshots instead.
9. Dealer-flow claims from option volume/OI. The documented fields do not provide aggressor side, opening/closing state, or dealer position.
10. Trading/order enhancements in this project phase. Existing order system remains unchanged.
11. Cosmetic HTML/3D work before data quality. Improve data contract, freshness, and attribution first.

12.1 Unit tests

Required behavior coverage:

• tests/unit/test_market_data.py
• schema and availability states;
• delayed/frozen/live mapping;
• raw generic ticks retained;
• null versus zero;
• cancellation on timeout/error;
• range and quality derivations.
• tests/unit/test_historical_market_data.py
• deterministic bars;
• date/timezone filtering;
• returns, ATR, realized volatility, moving averages, drawdown;
• insufficient history;
• cache hit and exact request key;
• incomplete windows.
• tests/unit/test_fundamentals.py
• XML parsing;
• unknown tags;
• report date and stale state;
• entitlement failure.
• tests/unit/test_wsh_events.py
• metadata-before-event order;
• one in-flight request;
• raw JSON retention;
• unavailable entitlement.
• tests/unit/test_news.py
• provider discovery;
• headline/article separation;
• deduplication;
• max result validation.
• tests/unit/test_scanner.py
• filter validation;
• 50-row limit;
• top-N enrichment isolation;
• cancellation.
• tests/unit/test_pnl_snapshots.py
• cancellation;
• persistence envelope;
• reconciliation warning.
• Existing suites to preserve:
• test_batch_quote.py;
• test_connection.py;
• test_portfolio_raw.py;
• test_analysis.py;
• test_weekly.py;
• test_fallback.py;
• test_upcoming.py;
• test_advisor.py;
• test_html_v3.py.

12.2 Read-only paper smoke tests

After the focused unit tests pass and the paper daemon is healthy:

.venv/bin/ibkr health
.venv/bin/ibkr quote AAPL
.venv/bin/ibkr batch quote SPY QQQ NVDA
.venv/bin/ibkr snapshot AAPL --history 1y
.venv/bin/ibkr portfolio --enrich
.venv/bin/python scripts/nightly_report.py --no-ai --no-discord --no-pagedrop
.venv/bin/python scripts/weekly_report.py --no-ai --no-discord

If optional permissions are absent, the expected result is an explicit unavailable state, not a command failure or fabricated field.

12.3 Failure scenarios

The team must verify independently:

• daemon unavailable;
• gateway reconnecting;
• invalid contract;
• ambiguous contract;
• no live market-data subscription;
• delayed data only;
• generic tick not entitled;
• WSH not entitled;
• news provider absent;
• fundamental XML unavailable;
• historical request timeout;
• historical cache stale;
• Futu OpenD unavailable;
• Flex statement generation code 1019;
• mixed per-symbol fallback;
• canceled research request followed by a normal quote.

Stage 1 — Schema-only release

• Add models and serializers.
• No report behavior changes beyond additive metadata.
• Run unit tests and fixture tests.

Stage 2 — Paper daemon rich quote

• Enable generic ticks for a small test symbol set.
• Verify live/delayed/frozen labels.
• Verify subscriptions cancel.
• Verify daemon stays responsive.

Stage 3 — Paper nightly canary

• Enable held-underlying and held-option enrichment.
• Generate HTML locally without AI/Discord.
• Inspect source and age for every section.
• Compare portfolio/P&L values against the previous report.

Stage 4 — Paper weekly canary

• Enable cached daily bars and derived metrics.
• Compare weekly NLV/Flex values against the previous report.
• Confirm market history never changes account/P&L truth.
• Confirm per-symbol fallback labels.

Stage 5 — Optional entitlements

Enable fundamentals, WSH, and news one at a time. Each capability must be independently disableable and must return explicit unavailable states.

Stage 6 — Live read-only verification

Run the same smoke tests with IBKR_MODE=live, with no order commands. Verify the daemon, audit logs, report output, cache, and reconnect behavior. No live trading change is part of this plan.

Stage 7 — Documentation and operations handover

Update:

• AGENTS.md;
• docs/Nightly-Report.md;
• docs/Weekly-Report.md;
• the CLI/user guide;
• the operator runbook;
• this document's implementation status and release notes.

Architecture

• All IBKR requests route through the daemon.
• No report path creates client IDs 87/88 while daemon is active.
• _ib_lock protects every gateway operation.
• Long calculations happen outside the gateway lock where safe.
• Research timeout causes cleanup and does not poison the daemon.

Data contract

• Existing quote fields remain compatible.
• Rich fields include source, as-of, age, market-data type, and availability.
• Raw ratios/XML/WSH/headline identifiers are retained.
• Missing is not zero.
• Delayed is not live.
• Per-symbol mixed sources are visible.

Nightly/weekly

• Nightly has held-underlying and held-option research context.
• Weekly has cached daily historical context and deterministic derived metrics.
• Flex cache age and completeness are visible.
• Futu macro data remains source-labelled.
• Optional research failure does not break core reports.
• AI prompts include data-quality context.

Operations

• Historical pacing policy is implemented.
• Exact request cache is implemented for scheduled bars.
• All subscriptions cancel in finally.
• WSH requests are serialized.
• News/fundamental/WSH entitlement errors are actionable.
• ib_async version is pinned/tested.
• Paper read-only smoke tests pass.
• Live read-only smoke tests pass.

Scope control

• No automatic order behavior was added.
• No Futu account/P&L/order authority was introduced.
• No full-chain/full-universe nightly request was added.
• No silent Yahoo fallback was introduced.
• No unsupported dealer-flow or sentiment claim was added.

If the team needs a narrowly scoped first sprint, implement only:

1. DP-00 dependency/source policy;
2. DP-01 daemon-only report routing;
3. DP-02 provenance/availability schema;
4. DP-03 public IBClient wrappers;
5. DP-04 rich quote collector;
6. focused tests and paper smoke tests.

Do not start WSH, news, scanner, market depth, or full fundamentals until the daemon data plane, cancellation, provenance, and tests are stable.

The first sprint is successful when an agent can run:

.venv/bin/ibkr snapshot AAPL

and reliably distinguish:

• live versus delayed data;
• available versus not-entitled fields;
• current quote versus historical context;
• IBKR data versus explicit fallback data;
• real values versus missing values.

That is the foundation required before nightly and weekly AI analysis can safely consume the expanded IBKR API surface.

16.1 Decision

Futu OpenD should become a first-class, read-only companion service beside IBKR Gateway. It is not a replacement for the IBKR daemon and it must not be routed through /root/.kit-ibkr/daemon.sock or the IBKR _ib_lock.

IBKR daemon / Gateway
  account, positions, cash, fills, orders, BAGs, P&L
  current execution-related quote and option marks when healthy

Futu OpenD / OpenQuoteContext
  fast batch market snapshots
  historical K-line cache
  option chain / strategy quote / strategy analytics
  capital flow, short interest, research, valuation, corporate actions
  earnings/economic/Fed data, news and sentiment content

The word first-class means:

• it has an explicit adapter, configuration, cache, timeout, quota, tests, and report contract;
• it is selected deliberately by data domain and freshness, not by an opaque fallback;
• it can run concurrently with the IBKR portfolio request because it is a separate read-only local data plane;
• every returned field remains labelled source=futu or source=futu_content;
• it never becomes the authority for IBKR account or execution state.

16.2 Source arbitration rules

When both providers return a current quote, store both values and a comparison record. Do not auto-select a provider merely because it is numerically closer. For execution decisions, IBKR remains primary when healthy. For research-only fields absent from IBKR, Futu is the explicit primary supplemental source.

16.3 Official Futu evidence

Primary sources reviewed:

• Futu Agent Hub
• Official Futu API Skill
• Futu API reference
• Futu API limits
• Futu field mapping
• Futu API introduction
• Futu permissions and quotas
• Market snapshot
• Historical K-lines
• Subscriptions
• Option strategy analysis

The official Agent Hub exposes two distinct classes of capability:

1. Futu API Skill: OpenD + SDK, covering market snapshots, K-lines, order book, ticks, options, research, screeners, and calendars.
2. Content Skills: public HTTP content services for news/announcements/research search, individual-stock interpretation, and community sentiment. These do not require OpenD, but they are content data rather than executable market data and must use a separate source=futu_content label.

16.4 Futu API capability matrix

The current QUOTE_ALLOWLIST deliberately exposes only a subset. Do not broadly expose every inventory endpoint. Each new method needs an explicit read-only wrapper, quota/permission note, response fixture, and source envelope.

16.5 What Futu can make faster

There is no measured Futu-vs-IBKR benchmark in the repository yet, so the following are engineering hypotheses to validate, not guaranteed speed claims.

Strongest likely speed win: batch snapshots

Futu get_market_snapshot(code_list) is one request for a list and the official endpoint supports up to 400 codes per request. This is a better fit than opening one request per underlying. It can provide a fast research snapshot for all held underlyings while the IBKR daemon performs the authoritative portfolio/account request.

Implement:

1. derive held underlying symbols from IBKR data;
2. canonicalize to Futu codes;
3. deduplicate and chunk at <=400;
4. call get_market_snapshot once per chunk;
5. preserve Futu update_time and retrieval time;
6. compare with IBKR quote without overwriting it.

Strongest likely history win: cached request_history_kline

Futu historical K-lines return OHLCV plus pe_ratio, turnover_rate, turnover, change_rate, and last_close. The endpoint supports up to 1,000 rows per page with page_req_key. It is a useful cache/fallback for weekly underlyings and fields that IBKR does not return in the same response.

Do not use the current bars() implementation for historical weekly data by default: it subscribes, fetches get_cur_kline(), and immediately unsubscribes. The official subscription docs state that a subscription is a lease and should remain for at least one minute before cancellation; closing a context does not necessarily release quota immediately. Use request_history_kline() for historical bars and reserve subscriptions for explicit realtime work.

Strongest research-value win: data IBKR does not currently provide

Futu can add genuinely new fields rather than merely duplicate quotes:

• capital flow and size-bucket distribution;
• short interest and days to cover;
• analyst consensus and rating changes;
• valuation history/distribution;
• financial statements and revenue breakdown;
• insider, institution, and shareholder changes;
• buybacks, dividends, splits, and corporate actions;
• macro economic calendar and Fed watch;
• public news/announcement/research search;
• community sentiment as explicitly labelled context.

These should be treated as first-class research enrichments, not as fallback quotes.

What Futu must not do for speed

• Do not open one OpenQuoteContext per symbol.
• Do not use unbounded asyncio.to_thread() calls for every symbol.
• Do not parallelize the same Futu context until thread-safety and measured limits are established.
• Do not create persistent subscriptions nightly merely to avoid a snapshot call.
• Do not claim a faster provider is more correct for an executable IBKR order.

16.6 Futu provenance contract

The existing envelope() has source, source_detail, as_of, and delayed_or_realtime. Extend it without breaking the current fields:

{
  "source": "futu",
  "source_detail": "Futu OpenD / US Stocks LV3",
  "endpoint": "get_market_snapshot",
  "requested_at": "2026-08-16T22:00:00Z",
  "received_at": "2026-08-16T22:00:00.240Z",
  "as_of": "2026-08-16T21:59:59.900Z",
  "age_ms": 340,
  "delayed_or_realtime": "realtime",
  "market": "US",
  "canonical_symbol": "AAPL",
  "provider_code": "US.AAPL",
  "availability": "available",
  "quota": {"checked": true, "remaining": null},
  "errors": [],
  "data": {}
}

For content skills use:

{
  "source": "futu_content",
  "endpoint": "news_search",
  "retrieved_at": "...",
  "language": "zh-HK",
  "result_limit": 50,
  "data": []
}

futu_enrich.py currently extracts ['data'] and thereby discards the outer source/as-of envelope. Preserve the envelope at section and symbol level. A Futu error must also carry source metadata, endpoint, provider code, and error classification.

16.7 Quota, entitlement, and lifecycle policy

Official Futu documentation establishes these constraints:

• get_market_snapshot accepts up to 400 codes per call and is rate-limited; the official page states up to 60 snapshot calls per 30 seconds.
• request_history_kline returns up to 1,000 rows per page and uses page_req_key for pagination.
• Each subscription consumes one unit per code/data type.
• Unsubscription is not immediate: official docs require at least one minute before cancellation and quota release can be deferred across connections.
• Market rights depend on market and entitlement level; the Futu API rights are not identical to App rights.
• OpenD is a local gateway; every OpenQuoteContext must be closed.

There is a documentation discrepancy to resolve before production quota logic:

• official permissions page states historical K-line quota deduplication over the most recent 7 days;
• the current Agent Hub API_LIMITS.md says 30 days;
• local docs/futu-api-inventory.md records a measured quota tier but not a definitive deduplication window;
• older docs/Futu-Enhancement.md records a different quota tier.

Until runtime verification is complete, use the conservative policy:

1. call get_history_kl_quota(get_detail=True) before each history batch;
2. cache exact history requests locally;
3. assume repeated requests may consume quota after the shorter documented window;
4. record quota response and SDK/OpenD version in logs;
5. stop gracefully when remaining quota is insufficient;
6. never infer quota from a hardcoded number.

16.8 Futu performance architecture

The current Futu adapter creates throwaway contexts in several paths. Refactor to a scoped context:

one CLI invocation       -> one FutuQuoteClient/context
one nightly enrichment   -> one scoped context per endpoint group
one weekly job           -> one scoped context with cache/quota checks
long-lived watchlist     -> explicit separate subscription service

Recommended first implementation is one context with serialized calls and batch endpoints. Add a bounded context pool only after measurement proves it helps and OpenD's connection/subscription behavior is safe.

All Futu calls need:

• preflight is_open();
• an outer wall-clock timeout;
• typed errors (FUTU_UNAVAILABLE, FUTU_NOT_ENTITLED, FUTU_QUOTA_EXHAUSTED, FUTU_RATE_LIMIT, FUTU_PARSE_ERROR);
• requested_at, received_at, provider update_time where available;
• context close in finally;
• no event-loop blocking from SDK calls;
• bounded concurrency for per-symbol endpoints;
• request deduplication within a report run.

16.9 Nightly and weekly integration

Nightly

After the IBKR daemon returns the authoritative portfolio, run Futu enrichment independently:

1. one batched Futu snapshot for all held underlyings;
2. one bounded batch/cache path for option or strategy data required by held legs;
3. bounded short-interest and capital-flow calls only for symbols not already cached;
4. attach Futu data under analysis.futu_research without changing summary, positions, pnl, or cash;
5. render source/as-of/age and per-symbol errors in HTML, Discord, and advisor prompt.

Do not create one Futu context and two requests per symbol as the default. The current futu_enrich.py implementation uses unbounded asyncio.to_thread() per symbol; replace it with batch snapshot plus a bounded endpoint worker.

Weekly

Use Futu for:

• macro calendar and Fed watch;
• earnings calendar with estimate/actual/IV/option-volume fields;
• valuation and analyst consensus refresh;
• corporate actions;
• capital-flow/short-interest trend;
• cached historical K-line supplement or explicit IBKR history fallback;
• news/announcement digest when enabled.

IBKR/Flex still supplies account NLV, cash flow, trades, fills, and portfolio truth. Futu data must never enter the weekly NLV or P&L calculation directly.

16.10 Explicit Futu CLI surface

Keep Futu separate from the IBKR daemon and expose explicit commands only:

ibkr futu health
ibkr futu quote AAPL MSFT
ibkr futu history AAPL --interval 1d --count 365
ibkr futu quota
ibkr futu market-state AAPL
ibkr futu orderbook AAPL --levels 10
ibkr futu ticker AAPL --count 100
ibkr futu short AAPL
ibkr futu flow AAPL
ibkr futu fedwatch
ibkr futu earnings AAPL
ibkr futu research AAPL

Every success and error remains source=futu; no command sends data through the IBKR Unix socket. Do not expose trade/account methods through this read-only adapter.

FP-00 — Resolve Futu version, quota, and configuration matrix

Priority: P0 Dependencies: None Suggested effort: Small

Files:

• pyproject.toml
• .env.example
• src/kit_ibkr/futu/client.py
• src/kit_ibkr/config.py
• docs/Futu-Enhancement.md
• docs/futu-api-inventory.md

Work:

1. Record the target OpenD and futu-api versions.
2. Resolve the 7-day versus 30-day historical quota documentation discrepancy with a controlled runtime check.
3. Add configurable FUTU_OPEND_HOST, FUTU_OPEND_PORT, and Futu timeout settings while retaining safe localhost defaults.
4. Decide whether futu-api is a pinned mandatory dependency or a pinned optional extra; lazy imports and fake-client tests must continue to work without OpenD.
5. Keep credentials and OpenD config outside the repository.

Acceptance:

• Target version matrix is documented.
• Runtime quota policy is based on observed get_history_kl_quota() output, not hardcoded assumptions.
• OpenD unavailable is a fast typed failure.
• No credentials, password hashes, or login config enter source control.

FP-01 — Upgrade source/provenance envelope

Priority: P0 Dependencies: FP-00 Suggested effort: Small/Medium

Files:

• src/kit_ibkr/futu/models.py
• src/kit_ibkr/futu/service.py
• src/kit_ibkr/report/futu_enrich.py
• src/kit_ibkr/report/upcoming.py
• src/kit_ibkr/report/weekly.py
• tests/unit/test_futu.py
• Create tests/unit/test_futu_report.py

Work:

1. Extend envelope() with endpoint, requested/received times, age, availability, errors, request parameters, provider code, and quota metadata.
2. Preserve source=futu on each symbol and each report section.
3. Preserve Futu envelopes instead of extracting only ['data'].
4. Add source=futu_content for public content skills.
5. Keep legacy data shape compatible for current callers.

Acceptance:

• Futu success and error responses carry source and endpoint.
• Nightly Futu flow rows show source/as-of/error state.
• Upcoming events preserve Futu source after legacy normalization.
• No Futu field can be mistaken for IBKR account/P&L data.

FP-02 — Reuse one context and add batch snapshot/cache

Priority: P0 Dependencies: FP-01 Suggested effort: Medium

Files:

• src/kit_ibkr/futu/client.py
• src/kit_ibkr/futu/service.py
• src/kit_ibkr/report/futu_enrich.py
• src/kit_ibkr/report/upcoming.py
• tests/unit/test_futu.py
• tests/unit/test_futu_report.py

Work:

1. Scope one OpenQuoteContext per CLI/report endpoint group instead of one context per symbol.
2. Batch get_market_snapshot() calls and chunk canonical codes at 400.
3. Deduplicate symbols across quote, flow, earnings, and report sections.
4. Add an in-run cache keyed by canonical provider code and endpoint parameters.
5. Add an outer timeout around SDK calls.
6. Keep bounded concurrency for endpoints that require one-symbol calls; do not use unbounded asyncio.to_thread().

Acceptance:

• A nightly snapshot of N symbols uses one batched snapshot call per <=400 symbols.
• Context creation/close count is bounded and observable.
• Repeated same-run requests hit cache.
• One symbol error does not discard valid symbols.
• Event loop remains responsive during SDK calls.

FP-03 — Add historical K-line cache and quota guard

Priority: P0 Dependencies: FP-00, FP-02 Suggested effort: Medium

Files:

• src/kit_ibkr/futu/client.py
• src/kit_ibkr/futu/models.py
• src/kit_ibkr/futu/service.py
• Create src/kit_ibkr/futu/cache.py
• src/kit_ibkr/report/weekly.py
• tests/unit/test_futu.py
• Create tests/unit/test_futu_history.py

Work:

1. Add read-only wrappers for request_history_kline() and get_history_kl_quota().
2. Check remaining quota before the first page.
3. Fetch page_req_key pages until complete or bounded maximum.
4. Cache exact code/date/ktype/autype/session/extended-time requests.
5. Record whether bars are raw, QFQ, HFQ, or another adjustment mode.
6. Use Futu history as a weekly supplement/fallback, not a silent replacement for IBKR account metrics.

Acceptance:

• History pagination is complete and deterministic.
• Quota exhaustion returns a typed warning and does not break the report.
• Adjusted versus unadjusted bars are explicit.
• Cache hits avoid duplicate quota-consuming requests.
• Weekly per-symbol history source is visible.

FP-04 — Add Futu speed/freshness diagnostics

Priority: P0 Dependencies: FP-02, FP-03 Suggested effort: Small/Medium

Files:

• src/kit_ibkr/futu/service.py
• src/kit_ibkr/cli/main.py
• src/kit_ibkr/report/futu_enrich.py
• tests/unit/test_futu.py

Work:

1. Measure context-open, request, normalization, and close durations.
2. Record provider update_time separately from local receive time.
3. Add ibkr futu quota and extend ibkr futu health with OpenD/API state.
4. Add per-endpoint error and rate-limit classification.
5. Store p50/p95 metrics in audit logs without logging credentials or raw sensitive config.

Acceptance:

• Speed claims are backed by measured data.
• A report can distinguish provider staleness from local latency.
• Quota and OpenD state are visible before expensive jobs.

FP-05 — Add option and strategy research enrichments

Priority: P1 Dependencies: FP-01, FP-02 Suggested effort: Medium

Files:

• src/kit_ibkr/futu/client.py
• src/kit_ibkr/futu/models.py
• src/kit_ibkr/futu/service.py
• src/kit_ibkr/cli/main.py
• src/kit_ibkr/report/futu_enrich.py
• tests/unit/test_futu.py
• Create tests/unit/test_futu_options.py

Work:

1. Add deliberate allowlist entries for get_option_quote, get_option_strategy, get_option_strategy_spread, option volatility, exercise probability, and market statistics.
2. Keep selected expiries/strikes bounded.
3. Use Futu get_option_strategy_analysis() for independent combination bid/ask and probability context.
4. Preserve IBKR core combo math as authoritative.
5. Add cross-provider comparison fields rather than replacing IBKR leg prices.

Acceptance:

• Held strategy rows can show IBKR and Futu quote/analysis side by side.
• Combination bid/ask is not manually calculated from unrelated leg snapshots.
• Full-chain nightly collection is prohibited by configuration and tests.

FP-06 — Add research fundamentals and corporate-action adapter

Priority: P1 Dependencies: FP-01, FP-02 Suggested effort: Medium/High

Files:

• src/kit_ibkr/futu/client.py
• src/kit_ibkr/futu/service.py
• src/kit_ibkr/futu/models.py
• src/kit_ibkr/report/weekly.py
• src/kit_ibkr/report/advisor.py
• tests/unit/test_futu_research.py

Work:

Add individually tested wrappers for:

• financial statements;
• revenue breakdown;
• analyst consensus and rating summary;
• valuation detail;
• Morningstar report metadata;
• dividends, buybacks, splits;
• insider, institution, and shareholder changes;
• operational efficiency and company profile.

Normalize only fields observed in fixtures. Preserve raw provider rows, report dates, units, and source. Do not mix Futu research values into IBKR P&L or account calculations.

Acceptance:

• Weekly advisor receives valuation/research context with source/as-of.
• Unknown or missing fields remain null.
• Corporate actions can trigger report warnings without changing positions.
• Permission/enum failures are isolated per endpoint.

FP-07 — Harden event, flow, and calendar enrichment

Priority: P1 Dependencies: FP-01, FP-02, FP-04 Suggested effort: Medium

Files:

• src/kit_ibkr/futu/service.py
• src/kit_ibkr/report/futu_enrich.py
• src/kit_ibkr/report/upcoming.py
• src/kit_ibkr/report/weekly.py
• src/kit_ibkr/report/nightly_html.py
• src/kit_ibkr/report/weekly_html.py
• tests/unit/test_futu_report.py
• tests/unit/test_upcoming.py
• tests/unit/test_weekly.py

Work:

1. Batch what can be batched.
2. Bound per-symbol short-interest and capital-flow work.
3. Preserve latest-valid-time from flow responses.
4. Page through economic-calendar results instead of dropping next_page/has_more.
5. Add Futu source labels to upcoming events and HTML/Discord sections.
6. Include capital distribution, daily short volume, and flow trend only when the endpoint is entitled and cached.

Acceptance:

• Nightly flow enrichment has bounded wall-clock time.
• Partial Futu outage produces per-symbol warnings.
• Weekly calendar is complete within a configured page/event bound.
• Futu data never changes account summary or P&L.

FP-08 — Add explicit Futu CLI wrappers

Priority: P1 Dependencies: FP-01 through FP-07 as applicable Suggested effort: Medium

Files:

• src/kit_ibkr/cli/main.py
• src/kit_ibkr/futu/service.py
• tests/unit/test_futu_cli.py

Work:

Add only the commands with a stable adapter contract:

futu history
futu quota
futu market-state
futu orderbook
futu ticker
futu short
futu flow
futu fedwatch
futu earnings
futu research

Keep _run_futu() outside the IBKR daemon path. Every success/error output must carry source=futu.

Acceptance:

• CLI validation is deterministic.
• Futu commands do not send Unix-socket daemon requests.
• Unsupported market prefixes are rejected or explicitly mapped; they do not silently default to US.
• No trade/account context is reachable from the read-only command group.

FP-09 — Add public content skills as an optional research digest

Priority: P2 Dependencies: FP-01, FP-04 Suggested effort: Medium

Work:

1. Add a separate HTTP client for Futu Agent Hub content endpoints, not through OpenD.
2. Support news/announcement/research search with a hard result limit.
3. Preserve URL, title, provider, language, publication time, retrieval time, and query.
4. Keep individual-stock interpretation and sentiment as optional AI context, never as an unqualified trading signal.
5. Add network timeout, retry budget, and content-source availability states.

Acceptance:

• Content data is labelled futu_content, never futu market data or IBKR data.
• Search results are bounded and deduplicated.
• Content outage does not break nightly/weekly report generation.

FP-10 — Cross-provider benchmark and rollout gate

Priority: P0 after FP-02/03 Dependencies: FP-02, FP-03, FP-04 Suggested effort: Medium

Work:

Measure, rather than assume, the following in paper/read-only mode:

• context open and close latency;
• Futu snapshot p50/p95 for 1, 10, 50, 100, and 400 codes;
• IBKR batch quote p50/p95 for the same symbol sets;
• history first-page and full-pagination latency;
• cache hit latency versus gateway/OpenD request latency;
• option chain and selected-strike quote latency;
• nightly flow enrichment wall-clock time for 1, 5, 10, and 20 symbols;
• error, timeout, quota, and partial-result rates;
• provider timestamp age and local receive age;
• OpenD context count and subscription quota before/after jobs.

Store benchmark results with SDK/OpenD versions and account entitlement state. Do not state that Futu is faster until a measured workload shows it.

Acceptance:

• A source-selection decision is backed by p50/p95 and data-quality evidence.
• Batch snapshot path is measurably cheaper than per-symbol calls.
• History cache reduces repeat calls and quota usage.
• No benchmark uses a trade context or alters account state.

Snapshot benchmark

Run the same canonical US/HK symbol set through:

Futu get_market_snapshot: 1 / 10 / 50 / 100 / 400 codes
IBKR batch quote:          1 / 10 / 50 codes

Record p50, p95, timeout rate, provider update_time age, local receive age, and per-symbol missing fields. A 400-code Futu request is a throughput test, not a recommendation to use 400-code nightly reports.

Historical benchmark

For 5, 10, and 20 underlyings:

IBKR reqHistoricalDataAsync, 1Y, 1 day, RTH
Futu request_history_kline, 1Y, K_DAY, explicit autype

Compare OHLCV completeness, adjusted/unadjusted semantics, bar timestamps, request time, quota usage, and derived return/volatility consistency. Do not merge bars from two providers into one series without a source boundary.

Nightly enrichment benchmark

Measure the current implementation against the proposed implementation:

Current: one Futu context per symbol, short_interest + capital_flow
Target:  one scoped context, batched snapshot, bounded flow worker, cache

The target is expected to reduce connection overhead and duplicate requests, but the benefit must be measured. A faster call with stale or differently timestamped data is not an improvement.

Option benchmark

For selected held legs and one selected expiry:

• IBKR held-leg quote and Greeks;
• Futu option snapshot / strategy analysis;
• combination bid/ask availability;
• IV/HV/volume/OI completeness;
• source/as-of age;
• no full-chain expansion.

The output is a cross-provider research comparison, not an automatic execution-price choice.

Data-plane boundary

• Futu remains an independent OpenD data plane.
• Futu never routes through the IBKR Unix socket or _ib_lock.
• Futu remains read-only and uses no Open*TradeContext.
• IBKR/Flex remain account, position, cash, fill, order, P&L, and BAG authorities.

Speed and lifecycle

• One scoped Futu context replaces per-symbol context creation.
• Snapshot calls batch and chunk at the documented maximum.
• Historical calls check quota before fetching.
• Exact history requests are cached.
• Subscription calls have explicit lease/cancellation handling.
• Every SDK call has an outer timeout.
• Per-symbol work has bounded concurrency.
• p50/p95 speed claims are measured, not inferred.

Provenance

• Every Futu response contains source, endpoint, provider code, as-of, receive time, age, and availability.
• futu_enrich.py does not discard the envelope.
• Futu content uses source=futu_content.
• Mixed IBKR/Futu/Yahoo data is visible per symbol and section.
• Delayed, stale, missing, quota-exhausted, and not-entitled states are distinct.

Research/report

• Nightly uses one batched Futu snapshot for held underlyings.
• Nightly flow enrichment is bounded and cached.
• Weekly uses Futu macro/event/research fields additively.
• Weekly account NLV/P&L remains IBKR/Flex-derived.
• Futu option strategy analysis is labelled informational and does not change core combo math.
• Public news/sentiment is optional and source-labelled.

Documentation consistency

• Quota discrepancy between official authority and Agent Hub API_LIMITS.md is resolved or conservatively handled.
• docs/Futu-Enhancement.md no longer contradicts implemented nightly/weekly integrations.
• docs/futu-api-inventory.md distinguishes tested, adapter-exposed, and inventory-only APIs.
• Target OpenD/SDK versions and entitlement state are recorded without credentials.

The combined IBKR + Futu delivery order is:

1. IBKR DP-00–DP-04: version policy, daemon-only report path, common schema, wrappers, rich quote.
2. Futu FP-00–FP-04: version/quota policy, source envelope, scoped context, batch snapshot, history cache, speed diagnostics.
3. IBKR DP-05–DP-07: held-option enrichment, historical cache, report/advisor provenance.
4. Futu FP-05–FP-07: option analytics, research fundamentals, flow/calendar hardening.
5. IBKR DP-08–DP-11: fundamentals, WSH, executions, scanner.
6. Futu FP-08–FP-10: explicit CLI, content digest, benchmark gate.
7. Only after the above: news, persistent PnL snapshots, depth/tick-by-tick, and other P2 tools.

The first practical speed milestone is not “replace IBKR with Futu.” It is:

one IBKR daemon portfolio request
  + one Futu batched snapshot request
  + cached historical/research data
  + no duplicate contexts
  + explicit source and age

This should reduce avoidable report work while preserving IBKR correctness and making Futu's genuinely unique research data available to the AI advisor.

21.1 Baseline at planning time

• 32 work packages defined (16 IBKR DP-*, 11 Futu FP-*, 5 agent-bundle DB-*);
• 381 unit tests passing on tests/unit (2026-08-16 baseline);
• repository already integrates ib_async 2.1.0 in the local environment and Futu OpenD 10.9.6918 with futu-api==10.10.7008;
• paper and live gateways exist but are not required for most unit work.

21.2 Effort model

The plan's effort labels convert to working days as follows:

Per-task totals (midpoint): IBKR ≈ 51 engineer-days, Futu ≈ 30 engineer-days. Integration, entitlement probes, benchmark runs, and report regressions typically add 15–25% overhead.

21.3 Critical paths

IBKR P0 chain

DP-00 (0.5d)
  → DP-01 daemon routing (3d)
  → DP-02 schema (1.5d)
  → DP-03 wrappers (3d)
  → DP-04 rich quote (3d)
  → DP-05 held options (5.5d)
  → DP-07 report provenance (3d)

Critical path ≈ 19 working days ≈ 4 weeks.

Futu P0 chain

FP-00 (0.5d)
  → FP-01 envelope (1.5d)
  → FP-02 scoped context + batch snapshot (3d)
  → FP-03 history cache + quota guard (3d)
  → FP-04 diagnostics (1.5d)
  → FP-10 benchmark gate (3d)

Critical path ≈ 13 working days ≈ 2.5 weeks and runs in parallel with the IBKR chain because Futu is an independent local data plane.

21.4 Weekly sprint plan (one developer + agent assistance)

21.5 Total estimates

The schedule is dominated by the IBKR P0 chain, not by Futu. The Futu chain can be built concurrently.

21.6 Timeline risks

1. DP-01 daemon routing regression risk. Report paths currently use direct client IDs 87/88. Migration must preserve gateway→Flex fallback and all 381 existing tests.
2. Entitlement unknowns. WSH, IBKR fundamentals, IBKR news, and some Futu research endpoints may be NOT_ENTITLED in the current accounts. Probe first; tasks then complete as explicit unavailable states, not as feature failures.
3. Quota window discrepancy. Futu history deduplication window (7 vs 30 days) must be resolved at runtime; cache design depends on it.
4. Gateway/OpenD availability. Paper smoke tests require both gateways; live verification requires an explicit approval window.
5. Version drift. Pin ib_async and futu-api at DP-00/FP-00 before feature work begins.

21.7 First actions

Start immediately with:

1. DP-00 — pin/test ib_async, lock the source hierarchy.
2. FP-00 — pin/test futu-api + OpenD, resolve the quota-window discrepancy with one controlled runtime check.
3. DP-02 + FP-01 — land the shared provenance schema and Futu envelope contract in parallel.
4. Then DP-01 (daemon routing) and FP-02 (scoped Futu context).

No feature work starts before the version matrix and provenance contract are locked.

22.1 Current pain

Hermes must make 3–4 calls and wait roughly 2–3 minutes to gather:

1. account summary + portfolio (one call);
2. per-symbol quotes or batch quotes (one or more calls);
3. option/strategy or historical context (additional calls);
4. research/event context (Futu calls or none).

Root causes:

• several CLI round-trips instead of one composite request;
• per-symbol IBKR quote qualification where a batch endpoint would suffice;
• historical data fetched synchronously on demand instead of served from cache;
• report direct connections and subscription patterns that make worst cases slow;
• research fields scattered across commands instead of one payload.

22.2 Target contract

ibkr bundle
ibkr bundle --with-history --with-options --with-research --with-events
ibkr bundle --fresh   # bypass caches where safe; bounded and explicit

One Unix-socket call returns one JSON payload:

{
  "account": {},
  "portfolio": {
    "summary": {},
    "combos": [],
    "singles": []
  },
  "universe": {
    "quotes": {
      "AAPL": {
        "source": "futu",
        "last": 0,
        "bid": 0,
        "ask": 0,
        "52w_high": 0,
        "52w_low": 0,
        "pe_ttm": null,
        "pb": null,
        "volume": 0,
        "pre_market": null,
        "after_hours": null,
        "as_of": "2026-08-16T22:00:00Z",
        "age_ms": 300
      }
    },
    "history": {
      "AAPL": {
        "source": "futu_history_cache",
        "return_1w_pct": null,
        "return_1m_pct": null,
        "realized_vol_pct": null,
        "sma20": null,
        "sma50": null,
        "sma200": null,
        "max_drawdown_pct": null,
        "bars": [],
        "cached_at": "2026-08-16T21:55:00Z"
      }
    }
  },
  "options": {
    "held_legs": [],
    "strategy_analysis": []
  },
  "research": {
    "short_interest": {},
    "capital_flow": {},
    "valuation": {},
    "analyst_consensus": {}
  },
  "events": {
    "earnings": [],
    "economic": [],
    "fed_watch": {}
  },
  "provenance": {
    "requested_at": "2026-08-16T22:00:00Z",
    "completed_at": "2026-08-16T22:00:02Z",
    "wall_ms": 2000,
    "sources": {
      "account": "ibkr",
      "quotes": "futu",
      "history": "futu_history_cache",
      "options": "ibkr",
      "research": "futu",
      "events": "futu"
    }
  },
  "warnings": []
}

22.3 Latency budget

Key mechanism: the bundle gathers the IBKR daemon payload and the Futu payload in parallel because they are independent local planes:

CLI bundle
  ├─► IBKR daemon socket  (account, portfolio, held options)  t_ibkr
  └─► Futu OpenD          (batch quotes, history cache, research, events)  t_futu
        wall_time ≈ max(t_ibkr, t_futu) + merge

Never place the IBKR daemon call and the Futu call in one serial chain.

22.4 Quantity of data added for the agent

The bundle adds, per held underlying, data the current flow does not return:

• 52-week high/low and position in range;
• PE TTM / PB / market cap / dividend context;
• current volume, average volume, relative volume;
• pre-market / after-hours / overnight price and change;
• 1W/1M/3M/6M/1Y returns, realized volatility, SMA20/50/200, max drawdown;
• held-leg IV/HV/volume/OI/spread and Futu combination bid/ask, max P/L, probability;
• short interest, days-to-cover, capital-flow buckets;
• valuation and analyst consensus;
• upcoming earnings with estimate/actual/IV, economic calendar, Fed watch.

All fields carry source/as-of/age. The AI advisor receives the same payload format as the nightly/weekly report enrichment so prompt-building code is shared.

22.5 New P0 tasks

DB-00 — Agent bundle contract and dispatcher

Priority: P0 Dependencies: DP-00, DP-02, FP-00, FP-01 Suggested effort: Small/Medium

Files:

• src/kit_ibkr/cli/main.py
• src/kit_ibkr/daemon.py
• Create src/kit_ibkr/core/bundle.py
• Create tests/unit/test_bundle.py

Work:

1. Define the AgentBundle schema with additive sections and provenance.
2. Implement ibkr bundle as one CLI entry that fans out to the IBKR daemon and Futu OpenD concurrently.
3. Merge results with per-section source and age.
4. Return partial sections with warnings instead of failing the whole bundle.
5. Share prompt-building between bundle and nightly/weekly advisor.

Acceptance:

• One CLI call returns account + portfolio + universe + optional sections.
• Warm call is ≤ 3s in the measured smoke test.
• Section failure never discards account/portfolio data.
• Bundle output feeds the existing advisor prompt builder unchanged in shape.

DB-01 — Futu batch universe quotes with 60s cache

Priority: P0 Dependencies: FP-01, FP-02 Suggested effort: Medium

Work:

1. Batch all held underlyings into ≤400-code get_market_snapshot() chunks.
2. Normalize last/bid/ask/volume/52-week/PE/PB/dividend/pre/after/overnight fields.
3. Cache per universe snapshot for 60 seconds inside the daemon-independent Futu layer.
4. Preserve provider update_time and local receive time.

Acceptance:

• One universe snapshot = one Futu call per ≤400 codes.
• Second bundle call within 60s uses cache (0 network).
• 20-symbol universe is a single call, measured, not inferred.

DB-02 — Precomputed history cache for held underlyings

Priority: P0 Dependencies: DP-06, FP-03 Suggested effort: Medium

Work:

1. Nightly/weekly jobs precompute daily bars + derived metrics for held underlyings and SPY/QQQ.
2. Bundle serves history from cache; missing entries are filled in background, not synchronously.
3. Serve stale cache with age labels instead of blocking the agent.

Acceptance:

• Bundle history section is a cache read on the happy path.
• Freshness and completeness labels are always present.
• Agent never waits for per-symbol history in the bundle hot path.

DB-03 — Research and event precompute pipeline

Priority: P0 Dependencies: FP-01, FP-02, FP-04, FP-07 Suggested effort: Medium

Work:

1. Move short interest, capital flow, valuation, analyst consensus, earnings, economic, and Fed data into a precomputed research cache refreshed by scheduled jobs.
2. Bundle reads research/events from cache with age and source.
3. Expose --fresh to bypass within quota/timeout limits.

Acceptance:

• Bundle research section is cache-backed.
• Per-symbol fan-out never runs on the agent hot path.
• Quota checks happen in the background job, not in the bundle call.

DB-04 — Hermes latency benchmark and regression gate

Priority: P0 Dependencies: DB-00 through DB-03 Suggested effort: Small/Medium

Work:

1. Record the pre-change baseline: call count and wall time for the current 3–4 call flow.
2. Measure bundle warm/cold/fresh and degraded modes.
3. Fail CI if warm bundle exceeds 3s or if a regression adds a serial round-trip.

Acceptance:

• Pre-change baseline is recorded with p50/p95.
• Post-change p50/p95 published in the handover.
• A measured regression blocks the merge.

22.6 Priority changes

The following move to or are accelerated by P0:

• Hermes bundle (DB-00…DB-04) — new P0;
• Futu batch snapshot speed (FP-02/DB-01) — P0;
• Futu history cache (FP-03/DB-02) — P0;
• research/event precompute (FP-07/DB-03) — P0;
• provenance for agent output (DP-02/DP-07) — P0;
• daemon-only routing (DP-01) — P0 as a prerequisite for bundle safety.

The client-facing halves of DP-04, DP-06, and DP-07 are delivered through the bundle tasks; the underlying collectors remain as written.

23.1 Sprint order

23.2 Revised totals

The speed milestone is now the first deliverable, not the last.

23.3 Updated first actions

1. DP-00 + FP-00 — pin versions, resolve Futu quota-window discrepancy.
2. DP-02 + FP-01 — provenance schema and Futu envelope.
3. DB-00 — bundle contract and concurrent IBKR+Futu dispatcher.
4. DB-01 — Futu batch universe snapshot with cache.
5. DB-04 — record the 3–4 call / 2–3 minute baseline before touching the hot path, then measure after each week.

Speed and data quantity are P0 acceptance criteria for every subsequent task.