Large Data
Large data is handled through downsampling (Line / Area / Baseline) and autobinning (Histogram / Candlestick / Bar). tvl.line(...) downsamples, tvl.histogram(...) autobins, and both run in the Deephaven engine and ship only post-aggregation buckets to the browser.
What does “large” mean here?
- Small (no downsampling needed): Roughly the chart’s pixel width × 1 row per pixel. For an 1800-pixel-wide chart, that’s a few thousand rows. Below this, the renderer can draw every point.
- Medium: Tens of thousands to ~100k rows. Downsampling activates, but the cost is dominated by network ship time. Interactive performance is good.
- Large: 100k to several million rows. Server-side aggregation reduces to ~viewport-width buckets, which is the actual data the renderer needs.
- Huge: 10M+ rows. Aggregations are still fast (Deephaven’s engine is column-oriented and parallel), but at this size query-engine cost matters as much as render cost.
What are the costs?
| Cost | Where it lives | Driven by |
|---|---|---|
| Aggregation (server-side) | Deephaven engine | Row count × bin reduction complexity |
| Serialization | Server → client wire | Bin count (not row count after aggregation) |
| Client-side rendering | Browser, lightweight-charts | Bin count, viewport pixel width, animation frame rate |
| Re-aggregation on pan/zoom | Engine + wire | Bin count budget, range listener thresholds |
The big win of the server-side paths is that serialization and rendering cost scale with bin count, not row count. A 10M-row table downsampled to a 2000-pixel-wide chart serializes ~2000 buckets. The same size as a 2000-row table without downsampling.
Downsampling vs. autobinning: when to use which
| Series type | Path | Why |
|---|---|---|
| Line, Area, Baseline | Min/max downsampling | Min/max-per-pixel preserves spikes; cheaper than full aggregation |
| Candlestick, Bar | Autobinning (OHLC reduction) | OHLC requires first/max/min/last per bin |
| Histogram | Autobinning (sum/count/avg/last) | Bars need a true reduction; min/max would lose meaning |
You don’t usually have to pick. tvl.line() uses downsampling, tvl.histogram() uses autobinning, etc. The choice is determined by the chart type.
Examples
Plot a 1M-row line with one call
tvl.data.large_prices() is a 1M-row intraday price fixture. The chart needs no special configuration. Downsampling activates automatically.
Drag-zoom into a small window: the chart refetches a finer aggregation. Click “reset zoom” (double-click an axis if tvl.scale(axis_double_click_reset=True) was passed to handle_scale): the coarse view returns.
Autobin a histogram over a large table
For a histogram on a large table, switch to tvl.histogram() and let autobinning pick the bin width. Use bin_count= to control density without committing to a specific duration.
The engine aggregates 1M rows down to ~200 average-per-bin rows; the chart renders 200 bars.
Force a specific bin width
When you want a known cadence (e.g. “daily” or “hourly”) regardless of viewport width, set bin_width explicitly. See autobin for the supported ISO-8601 grammar.
Daily bins on a 1M-row, 10-year intraday series give ~3650 bars. Well above the viewport budget but still cheap to ship and render.
OHLC autobinning on raw ticks
For an OHLC chart over a tick-level table, autobinning reduces each bin to first/max/min/last. The four reductions happen in a single agg_by call in the engine.
OHLC reduction is fixed; the only choice is the cadence.
Multiple downsampled series on one chart
Each series downsamples independently. Overlaying two 1M-row series costs roughly twice as much wire bandwidth, not 1M × 1M. Bucket budgets are per-series.
Picking parameters
A few rules of thumb:
- Pick
bin_countto match what you actually want to show. A bar chart with 200 visible bars is readable; 2000 is mush. Setbin_countclose to your target density. - Use
bin_widthfor predictable cadences. When the chart is meant to show “daily bars,” setbin_width="P1D"explicitly so the cadence doesn’t drift with zoom. - Leave
auto_bin=Noneon small inputs. The default tri-state (None= auto-detect by size) means small tables skip aggregation, large tables get it. Explicitly settingTrueon small tables wastes a round-trip.
API Reference
The per-type pages carry the full signature for each chart type that participates in large-data paths:
tvl.line. Client-side downsampling.tvl.histogram,tvl.candlestick,tvl.bar. Server-side autobin.
The chart-level options that govern downsampling live on tvl.chart.