What these tools are
The app is built on two main building blocks:
- lightweight-charts-python — A Python wrapper for TradingView’s Lightweight Charts JavaScript library. It provides candlestick charts, real-time bar updates, line indicators, toolbox (trendlines, horizontals), timeframe controls, and works in desktop GUI contexts. Ideal for backtesting UIs and live dashboards without a browser.
- Rithmic API — Rithmic’s protocol for market data and order routing (futures, CME, etc.). The app uses it in two ways: (1) fetching historical and live time bars via a subprocess that runs Rithmic’s sample time-bar script, and (2) sending bracket orders (entry + SL + TP) to Rithmic Paper or live when a strategy signals a trade.
Project overview
Historical OHLCV is stored in SQLite; the same bar data feeds a backtest engine that computes pivots, session high/low (NY, London, Asia), and strategy logic. The chart is built with lightweight_charts.Chart(toolbox=True): timeframes, indicators (e.g. RSI subchart), deltas, and a “go live” mode that pulls new bars from Rithmic and calls chart.update(). Strategy results are shown in tables (by direction, run-up, P&L); trades can be exported to Excel. Optional integration posts bracket orders to Rithmic via a separate subprocess when the backtest or live logic triggers an entry.
Charts & live data
- Single main chart with toolbox, logarithmic price scale, and configurable precision.
- Topbar: mode (Live vs Hist), timezone, indicators, timeframe switcher, deltas/pivots toggles.
- Live mode: bars requested from Rithmic (SampleTimeBar subprocess); new bar pushed with
chart.update(series). - Hotkeys for adding bars and exiting; click subscription for placing levels or markers.
Backtest & execution infra
- Backtest runs over SQLite OHLCV; structure (unbroken levels, pivots) and sessions computed per timeframe.
- Strategy class tracks opened/closed trades, recalc on new extrema; export to Excel.
- Optional: when a trade is taken, bracket order (entry, SL, TP) sent to Rithmic via subprocess (sample bracket-order script).
- Multi-timeframe params (1m, 5m, 15m, 1h, 4h, 1d, renko) drive both backtest and chart.
Beyond the app — the research & execution stack
The charting app sits on top of a larger engine that drives history and live trading through the same code path.
Market-structure engine
- An "unbroken level" engine tracks swing pivots per direction and timeframe, maintains the set of levels price hasn't broken, and emits structure/retest events.
- A streaming bar aggregator builds arbitrary timeframes (including renko) tick-by-tick from one source stream — the same code path drives backtests and live.
- Sessions are segmented with real timezones (NY / London / Asia) and chained next-session scheduling.
- Trade accounting supports partial exits (per-leg SL/TP lists), running max-runup / max-drawdown recomputed on every extrema tick, and per-trade feature export.
Leakage-aware ML trade filtering
- XGBoost classifiers trained on "did this trade profit", with splits grouped by futures contract and a whole held-out contract as out-of-time data.
- The probability cutoff is chosen on validation only, then applied to test; calibration curves are part of every run.
- Feature manifests are serialized next to each model so live inference can never silently reorder columns.
- Before a feature is allowed to exist: mutual information, a depth-1 stump AUC with bootstrap 95% CI, and a label-permutation p-value.
Dual-broker live execution
- Bracket orders (entry + SL + TP as OCO) submitted through Sierra Chart's SCBridge, or through Rithmic's R|Protocol directly — protobuf over TLS WebSocket (login, market data, order routing, order-update subscription).
- The live consumer slices only genuinely-new bars from each overlapping realtime frame with an O(log n) watermark, idempotent under duplicate pushes.
- Backtest entries are code-generated into both a Pine Script v6 indicator and a C++ study, so simulated fills can be visually reconciled on TradingView and Sierra charts.
Native C++ orderflow studies
- ACSIL DLL studies read Sierra Chart's binary intraday file directly with shared buffered I/O — a fix for a documented production deadlock where holding the per-record API lock starved the live tick writer.
- A bar is only finalized once the file provably contains the next bar's open, so a lagging disk flush can never truncate it.
- Volume-anomaly detection scores bars against context-adaptive baselines rather than a single global average, with graceful fallback during warmup — the specific conditioning stays private.
- Every bar is replayed tick-by-tick and the first signal print is latched permanently — a historical recalculation reproduces exactly what a live watcher saw.
Representative code excerpts
Chart setup with toolbox and live update, fetching bar data from Rithmic, binary-format parsing in the C++ studies, and idempotent live-stream consumption.
Chart with toolbox and live update
chart_var.chart = Chart(toolbox=True, width=1000, inner_width=0.75, inner_height=1)
chart_var.chart.legend(True)
chart_var.chart.topbar.switcher('mode', (symbol+' Live', symbol+' Hist'), ...)
chart_var.chart.topbar.menu('timezone', ('UTC', 'NY', 'JST', 'LON'), ...)
chart_var.chart.topbar.menu('indicators', ['None', ...], func=on_indicator_selection)
chart_var.chart.topbar.menu('timeframe', timeframes, func=on_timeframe_selection)
chart_var.chart.price_scale(mode='logarithmic')
chart_var.go_live() # sets last bars and subscribes to live OHLC updates
# In go_live(): ser = pd.Series({'time': ..., 'open': ..., 'high': ..., 'low': ..., 'close': ...})
# chart.update(ser)
The chart is created with the toolbox for drawing; topbar controls switch between live and historical data, timezone, indicators (e.g. RSI in a subchart), and timeframe. go_live() sets the initial range and then updates the chart from the live OHLC feed (e.g. from Rithmic).
Fetching bars from Rithmic
def get_rithmic_data(data_max):
last_bar_time = int(data_max)
current_time = int(time.time()) - 60
while last_bar_time < (current_time - current_time % 60):
reader_process = subprocess.Popen(
[python_path, '-u', sampleTimeBar,
"wss://rprotocol.rithmic.com:443",
"Rithmic Paper Trading", "...", "CME", ticker, str(int(last_bar_time))],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, ...
)
# ... read stdout for bar lines; append to all_bar_data
last_bar_time = all_bar_data[-1][0] / 1000
return all_bar_data
Historical gaps or live bars are requested by calling Rithmic’s time-bar sample script in a subprocess; the script connects to Rithmic, requests bars from last_bar_time, and streams them to stdout. The main app parses the output and returns a list of bars for backtest or chart update.
Binary .scid parsing, size-checked at compile time
#pragma pack(push, 1)
struct ScidRecord
{
int64_t DateTimeUS;
float Open, High, Low, Close;
uint32_t NumTrades;
uint32_t TotalVolume;
uint32_t BidVolume;
uint32_t AskVolume;
};
#pragma pack(pop)
static_assert(sizeof(ScidRecord) == 40, "scid record must be 40 bytes");
The C++ studies parse Sierra Chart's intraday file format themselves — with the record layout pinned by a
static_assert so a platform header change can never silently corrupt the parse.
Idempotent live-stream consumption
response = response_queue.get()
df = response.as_df()
idx = df.index
if last_ts is None:
out = df.iloc[0:0]
else:
# O(log n) binary search: position strictly after last_ts
pos = idx.searchsorted(last_ts, side="right")
out = df.iloc[pos:]
for ts, _is_closed, open_, high, low, last, net_volume in out.itertuples(index=True, name=None):
process_bar([ts.value // 1_000_000, open_, high, low, last, net_volume], start_renko, renko=1)
last_ts = idx[-1] # advance even on empty slicesEach realtime push overlaps the previous one; a binary-search watermark slices out only genuinely-new bars and advances even on empty slices, so duplicate pushes are harmless by construction.