Release notes for Deephaven Core version 41

Deephaven Community Core version 41.0 is now available. Let's see what's new!

Release highlights

• Stateless filters and selectables by default — filters and selectables are now stateless by default, improving performance and enabling better parallelization.
• Improved NULL and NaN handling in sorts and filters — sort operations now consistently place NULL first and NaN last. Breaking change: arithmetic filters now use IEEE floating-point semantics for NaN (i.e., two NaN values are neither equal nor participate in comparisons).
• Mathematical aggregations — mathematical aggregation operations now properly ignore NULL values and poison results with NaN when appropriate.
• Enhanced C# client — the C# client now supports custom SSL certificates, override authority, and a new ping entry point.
• Structured input table errors — input table updates now return structured errors for better error handling.

New features

Stateless filters and selectables

Filters and selectables are now stateless by default. This change improves performance by enabling better parallelization and reduces memory overhead. Stateless operations don't maintain internal state between invocations, making them safer for concurrent execution.

IncrementalReleaseFilter for add-only tables

The IncrementalReleaseFilter now supports add-only tables, allowing you to incrementally release rows from add-only source tables. This is useful for controlled data release scenarios.

TailInitializationFilter with row counts

A new TailInitializationFilter variant allows filtering based on row counts rather than timestamps. This provides more flexibility when initializing tables with a specific number of trailing rows.

Server-side APIs

General

• Structured input table errors — input table updates now return structured errors, making it easier to identify and handle specific update failures.
• A ColumnExpressionValidator has been added to TicketResolverOptions and PluginOptions, enabling service-loaded plugins to validate column expressions.
• Server-side timing is now included in JsCommandResult for performance analysis.
• Configuration now accepts a PropertyInputStreamLoader constructor parameter for custom property loading.

Python

• New ConcurrencyControl and Selectable wrappers have been added to the Python Server API.

Client APIs

C#

• Custom SSL certificates — the C# client now supports custom certificates in SSL authentication.
• Override authority — the client honors ClientOptions.OverrideAuthority and tolerates "Revocation Status Unknown" when using override authority.
• Ping entry point — a new ping entry point has been added for connection health checks.

Data I/O

• Parquet dictionary support for predicate pushdown — predicate pushdown now supports Parquet dictionary encoding, improving filter performance on dictionary-encoded columns.
• Avro String[] types — Avro String[] types are now supported for data ingestion.

UI updates

The web UI has been updated to version 1.10.0, bringing various improvements and fixes.

Improvements

General

• Ubuntu 24.04 base images — server base images have been updated to Ubuntu LTS 24.04.
• Improved NULL/NaN handling in sorts — sort operations now consistently place NULL values first and NaN values last.
• Mathematical aggregations — mathematical aggregation operations now properly ignore NULL values and poison results with NaN when appropriate.
• Better schema information — improved schema information for complex types.
• Update graph logging — log messages now include the update graph name for easier debugging.
• Column definitions are now properly retained when columns are retained.
• Column renames now properly mask collisions.

Filters and barriers

• Table-level data indexing is now supported for all filters.
• Implicit barriers are now added to serial selectables.

Python

• Applied linting and type-checking on Core Python APIs.
• The _to_sequence function is now public to maintain compatibility with Deephaven Enterprise Python APIs.

Breaking changes

• NaN handling in filters — arithmetic filters now use IEEE floating-point semantics for NaN (i.e., two NaN values are neither equal nor participate in comparisons).
• Stateless filters and selectables by default — filters and selectables are now stateless by default. Code that relied on stateful behavior may need to be updated.
• Versioning scheme — Deephaven has moved to a vX.Y versioning scheme (e.g., v41.0 instead of v0.41.0).
• Build requirements — Java 11 and Java 17 are no longer required to build Deephaven. The project now uses the default Java toolchain.

Bug fixes

Core engine

• where_in and where_not_in (DynamicWhereFilter) now handle NaN and +/-0 properly.
• Fixed NaN/NULL comparison behavior in filters and the query language.
• Python formula parallelism is now prevented in non-free-threaded Python to avoid concurrency issues.
• Corrected sort(float) and sort(double) to fix chunk sorting issues.
• Fixed selectDistinct on SourceTable throwing an exception while notifying.
• Fixed partition column filter handling in PartitionAwareSourceTable.
• Allow prevFlusher for distinct operators to run when not using exposeInternal.
• Fixed a ClassCastException when subscribing to BarrageTable with a ZonedDateTime.
• Fixed predicate pushdown for renamed columns in merged tables.
• Ensure DataIndex pushdown is not short-circuited.
• Fixed an issue where JsTreeTable copy was breaking advanced filters on tree tables.
• Corrected the value attribute usage for Pandas Timedelta to avoid type-checking errors.
• Fixed input tables that couldn't be edited after other columns were joined on.
• Improved URI parsing.

Client APIs

• The C# client now uses custom truststore if specified, even without override authority.
• JavaScript client now properly throws errors when worker connection has been disconnected.

Dependencies

• Migrated the lz4 dependency to a new package.
• Updated Jetty to 12.1.5 and gRPC to 1.76.2.
• Updated deephaven-csv to 0.18.0 for a shutdown fix.
• Improved JULI log level translation.
• Various other dependency updates including Iceberg 1.10.1, Jackson 2.20.1, and Groovy 3.0.25.

Reach out

Our Slack community continues to grow! Join us there for updates and help with your queries.