Release notes for Deephaven 0.27

Deephaven Community Core Release 0.27 brings exciting new features, quality-of-life improvements, and bug fixes. Of note are our new real-time data ingestion tool, improved Parquet support, and multi-joins.

Full release notes for 0.27.0 and 0.27.1 can be found on GitHub.

Table Publisher

Deephaven's Table Publisher brings a new way to write real-time data to Deephaven tables. Data is programmatically added to a blink table, which only retains rows from only the current update cycle. Blink tables can easily be converted to other table types to suit any use case. Anyone who uses DynamicTableWriter should get excited, since this is more efficient, scalable, and compatible with asyncio workflows.

Watch our developer demo to see this in action.

Multi-join

A multi-join allows users to join 3 or more tables together in an efficient and simple manner. The join happens using a single hash table regardless of the number of tables given. It's currently available in the Groovy server-side API, and will be available in the Python server-side API in the next release.

See a demonstration of multi-join on our YouTube channel.

Parquet improvements

Changes to Deephaven's Parquet library include:

• Users can specify a schema when reading hierarchical Parquet tables, rather than having to let the engine infer it from directory structure.
• Better support for read-modify-write workflows.
• Better support for reading and writing columns of arrays of data.
• Bug fixes for reading and writing nulls.

Bidirectional plugins

This release improves existing plugin support for custom object types. In addition to providing a snapshot of an object's contents, an ObjectType plugin can continue sending updates and receiving requests from clients without re-fetching the object.

The first demo in the video below shows an example of using bidirectional plugins to create a real-time chat application with a JavaScript front end:

Popular Deephaven plugins for pandas, matplotlib, plotly, and Plotly Express have been updated: simply run pip to get the latest version. User documentation on how to create streaming plugins is coming soon.

Support for streaming messages required small breaking changes in existing Deephaven plugins, summarized below.

Breaking changes

Updating existing Java `ObjectType`` plugins

Existing Java plugins not being rewritten to support bidirectional streaming should make the following changes:

• Classes extending from io.deephaven.plugin.type.ObjectTypeBase should change to io.deephaven.plugin.type.ObjectTypeBase.FetchOnly.
• Classes extending from io.deephaven.plugin.type.ObjectTypeClassBase should change to io.deephaven.plugin.type.ObjectTypeClassBase.FetchOnly.
• Imports for io.deephaven.plugin.type.ObjectType.Exporter should be changed to io.deephaven.plugin.type.Exporter.
• Update any deprecated calls to Exporter.reference(...) to the non-deprecated, one argument overload. This means the return type will be Reference instead of Optional<Reference>.

Updating existing Python ObjectType plugins

• Change imports and references from deephaven.plugin.object to deephaven.plugin.object_type.
• Change imports and references from deephaven.plugin.Registration.Callback to deephaven.plugin.Callback.
• Classes extending from deephaven.plugin.object_type.ObjectType should change to deephaven.plugin.object_Type.FetchOnlyObjectType.
• Update any calls to deephaven.plugin.object_type.Exporter.reference() to only take one argument, the object being exported.

Python time library

lower_bin and upper_bin have been removed from the Python time library. The methods were occasionally used instead of the preferred built-in query language methods. Users who want to bin times in tables should always use the built-in query language methods lowerBin and upperBin, as they're much faster.

Reach out

We look forward to hearing from you in Slack or GitHub discussions!