Release notes for Deephaven Core version 42

Deephaven Community Core version 42.0 is here, with new features, improvements, and bug fixes. This release focuses on two things: faster data on the wire and richer APIs for every client language. Let's see what's new!

New features

Formulas in rollups

Rollup tables now support formula aggregations, letting you compute derived values directly within a rollup rather than precomputing them in a separate step.

Consider a table of sales broken down by region and category:

You can now express the revenue calculation as a formula aggregation inside the rollup, and the engine re-aggregates it correctly at every level of the hierarchy:

Expand hierarchical tables to a specific depth

A new API lets you expand a hierarchical table to a specific depth in a single call, making it easier to programmatically control how much of a tree or rollup is revealed. This is useful when building dashboards or export pipelines that need to show a consistent view of a hierarchy — for example, always expanding to the region level of a rollup without manually toggling each node.

Server-side APIs

General

• Force read utility — a new ForceReadUtility forces all table data to be read, which is useful for warming caches and validating data availability.
• Structured filters are now supported in the Core Python client, and the wrapping of structured filters has been completed.
• The update graph now exposes methods to get the start time of the current cycle.
• The clock filter can now execute in parallel.
• Formula views now create their execution contexts proactively.
• Match filter construction is faster, thanks to an improved getBits implementation in the match filter factory.
• The QueryCompiler now emits trace-level logging.
• The sparse array column source recycler capacity is now configurable.
• A new minimumUngroupBase configuration property controls the minimum base used by ungroup.
• A getColumnSource accessor has been added to ColumnDefinition.
• New array-type ColumnDefinition constructors have been added.
• Plugins can now supply auth transformations.

Python

• A Python wrapper for Configuration has been added.
• The Python instant_val function now accepts an allow_string parameter.

Barrage

• Run-end encoding — the Barrage protocol now supports run-end encoding to compress repeated values on the wire. Real-world time-series data often has long runs of the same value — the same symbol, the same status code, the same price — and encoding these as a single value with a count rather than repeating them reduces bandwidth and speeds up streaming delivery.
• Growing subscriptions by default — Barrage subscriptions now grow incrementally by default.

Client APIs

JavaScript

• Column restriction API — a new JS API lets you restrict which columns a client can access.
• TreeTable expanded state — the JS API can now save and restore the expanded state of a tree table.
• The JS API now uses grpc-java internally.

Java

• The Java client now supports LocalTime, LocalDate, and Duration.

C#

• The C# library is now built for both net8.0 and net10.0.
• The C# client now supports ColumnSource.

C++

• The C++ client adds middleware for proper authentication handling when its Flight client is used by third-party consumers.

Data I/O

• Parquet codec mapping — Parquet reading now supports codec mapping.
• Parquet ENUM as String — the Parquet ENUM logical type is now read as a String.
• Iceberg now permits switching partition ordering.

UI updates

The web UI has been updated to version 1.22.1, bringing various improvements and fixes.

Improvements

• Faster sorting — sorting now skips unnecessary single-value columns, reducing work for tables with constant columns.
• Symbol table memory — singleton storage has been added to symbol state managers (SSMs), and SSMs are skipped for singleton values at the base and for rollup children.
• Predicate pushdown for sorted regions — predicate pushdown now optimizes sorted regions and allows column-region optimizations during filtering.
• Column binary search — range and match filters now use binary-search optimizations on sorted column regions.
• Lower memory for single-listener tables — tables with only one listener now use less memory.

Breaking changes

• Input table restrictions format — input table restrictions are now encoded with protobuf instead of JSON.
• PartitionedTable proxy join-key sanity check — the PartitionedTable proxy now defaults its join-key sanity check to False. Queries that previously failed because join keys span two constituents no longer fail. To restore the prior behavior, explicitly set sanity checking to True when creating the proxy.

Bug fixes

Core engine

• Fixed a remote table column bitset mismatch that could produce incorrect results.
• Fixed a race condition and duplicated work in update_by caching.
• Fixed an update_by bug where the input-column-to-slot map used an incorrect default value.
• merge no longer produces inconsistent results without a systemic thread.
• Fixed an array index out-of-bounds error in union redirection when estimating pushdown on merged tables.
• Fixed a deferred view that produced incorrect results when it referenced index or column variables.
• A deferred view table no longer coalesces its underlying table earlier than necessary.
• Symbol table joins using contiguous or hash redirection no longer use the wrong null value.
• Rollup re-aggregation now carries the source row key through SortedFirst and SortedLast operations.
• Corrected the guard logic for exponential moving (EM) aggregation options.
• Table constructors now correctly validate column names in more cases.
• Corrected shift handling for the tree table filter.
• Fixed a null pointer exception and a deadlock in the Barrage subscription request handler.
• Fixed a null pointer exception when a pushdown match filter had the wrong types.
• where_in no longer throws a ClassCastException with a single-column partial data index.
• Fixed an issue where dictionaries would skip the last region if it had only one value.
• Anonymous classes now work in the query language.
• ObjectVector get and iterator are now permitted in user expressions.
• The server no longer includes the Jetty version in HTTP responses by default.
• Fixed a capacity edge-case when encoding binary arrays.
• Fixed a rowset bug and corrected chunk closing in UnionChunkWriter.
• Reindexing filters now support barriers.
• Improved reordering of stateful partition filters.
• AggSpecFormula is now validated through the column expression validator.
• Static source tables now ensure table location liveness.
• Shifted-column operations now handle concurrent instantiation correctly.
• Corrected reporting of the WorkerHeapSize column.

Python

• pytz and zoneinfo are now soft dependencies.
• numba has been removed from the Python API's required install dependencies.
• TypeAliasType is now handled correctly in Python function parsing.
• Handled a breaking change in string data type inference introduced in Pandas 3.0.0.
• open_table now keeps its traceback when an error occurs.
• Guarded against new NumPy errors raised for invalid type hints.

Groovy

• Improved Groovy, QueryLibrary, and query-language-parser classloader handling.
• Fixed a ConcurrentModificationException from the Groovy script session change observer.

User interface

• Fixed non-string one-click key comparison.
• Plots now correctly handle an absent axis calendar.

Client APIs

• Fixed JS API regressions that appeared after the gRPC refactor.
• The JS API now closes the session on disconnect.
• The JS API can now upload the same data types it can download.
• Fixed Kafka column index incrementing for complex specs.
• The C# client now uses a custom truststore if specified, and MaxReceiveMessageSize has been increased to 100 MB.
• Fixed Base64Encode in the C++ client.

Dependencies

• Updated Netty to 4.1.135.Final, Jetty to 12.1.9, and Confluent to 8.2.1.
• Updated Parquet to 1.17.1 and JPY to 2.0.0.

Platform

• Deephaven now tests against Java 25.

Reach out

Try the new formula rollups or structured filter APIs and let us know what you build. Our Slack community is the best place to share what you're working on, ask questions, and get help from the team.