| Open or closed? | Closed, commercial software | ✅ | | |
| Open, source-available software | | ✅ | Open for collaboration! In future Enterprise versions, you'll be able to support multiple deephaven-core engine versions. |
| Data in / data out | Read Live Deephaven Format Tables from Data Import Server | ✅ | ✅ | |
| Read Parquet or Deephaven Format Historical Tables | ✅ | ✅ | |
| Parquet improvements | | ✅ | Faster writing, better support for read-modify-write workflows, reading and writing columns of arrays, LZV4_raw compression, better errors, and specifying a schema when reading hierarchical tables. |
| Partitions & hierarchical groups, reading & writing | ✅ | ✅ | |
| Non-clustered single column indexes | ✅ | ✅ | Single-column indices on static tables. |
| Clustered-indexes (groups) | ✅ | ✅ | Single-column clustered indices on static tables. |
| Live data indexes | | | Live data indexes that can be added to any table at runtime. Arriving in 0.34 Community and Grizzly Enterprise |
| Multi-column indexes | | | Some operations take advantage of multi-column indexes. Arriving in 0.34 Community and Grizzly Enterprise |
| Read Multi-level Partitioned Hive Tables | | ✅ | |
| Access Parquet Files on S3 Natively (w/o a FUSE driver) | | ✅ | From Core+ 0.33.0 |
| Merge System Tables | ✅ | | Can be read from Either Core+ or Legacy. |
| Write user tables | ✅ | ✅ | From Core+ version 0.31.0 |
| Read user tables | ✅ | ✅ | |
| Persistent Input Tables | ✅ | | Grizzly (2Q24) will have improved Input Tables in Core+. |
| Import-driven lastBy | ✅ | ✅ | From Core+ version 0.31.0 |
| Table loggers | ✅ | ✅ | Core+ supports only primitive types and Strings |
| Generated Row-by-Row Loggers | ✅ | | |
| JDBC (in Java) | ✅ | ✅ | |
| ODBC / ADBC / Connectorx / FlightSQL reading (Python) | | ✅ | |
| Kafka ingester | ✅ | ✅ | From Core+ version 0.27.1 |
| Adhoc Kafka connections | | ✅ | Ingest to a memory table in the query |
| Kafka lastBy Tables | | ✅ | |
| Kafka producer | | ✅ | |
| JSON Kafka Payloads | ✅ | ✅ | More enhancements on the way. |
| Avro Kafka Payloads | ✅ | ✅ | |
| POJO Kafka Payloads | ✅ | | |
| Solace Ingester | ✅ | | |
| SBE producer | ✅ | | A custom protocol built on top of Simple Binary Encoding. No authentication is provided. For Core+ workers, Barrage is a superior alternative. |
| SBE consumer | ✅ | | A custom protocol built on top of Simple Binary Encoding. No authentication is provided. For Core+ workers, Barrage is a superior alternative. |
| Read Feather | | ✅ | Via io.deephaven.extensions.arrow.ArrowWrapperTools#readFeather. |
| Arrow wrapper tools for future innovation & interoperability | | ✅ | For static data, Arrow opens an interesting ecosystem for interoperability with other engines and databases (ibis, substrait, ADBC). |
| Data mesh (connecting workers to workers) | Barrage producer | | ✅ | |
| Barrage consumer | ✅ | ✅ | |
| Preemptive Tables | ✅ | | Preemptive Tables are a Java-serialization based protocol, which does not support clients in other languages. Barrage is a gRPC alternative, which is used by Deephaven Core workers. |
| Cross Cluster Table Subscriptions | | ✅ | Since 1.20231218.109 -- URI subscriptions. |
| Query / Code / engine capabilities & approach | Faster, more natural parallelization | | ✅ | |
| Open benchmarking library for performance testing | | ✅ | Open GitHub project for real-time and batch processing, with comparisons to other engines -- highly customizable. |
| Open Addressed Join/Aggregation Hashing | | ✅ | Improved Performance vs. Legacy. |
| Groovy / Java interface | ✅ | ✅ | |
| Legacy, Python syntax | ✅ | | |
| Pythonic API | | ✅ | Core+ workers have a Python Table API that does not rely on automatic translation of Java method overloads. |
| Python UDF Improvements | | ✅ | |
| Python Type Hints | | ✅ | Supports type hints for functions return arrays, lists, or other sequences of data or that have optional returns. |
| Vectorization of Python functions | | ✅ | Tries to automatically vectorize Python functions to take advantage of chunking and minimize the number of Python-Java boundary crossings. |
| Function generated tables | | ✅ | Create ticking tables via a Python function, triggered by one or more source tables, or by a refresh interval. |
| Python future.annotations | | ✅ | Support for future.annotations, which are helpful for delayed evaluations. |
| Python formula decomposition | | | Release 0.35 in Community, expected as a Grizzly patch in Enterprise. |
| Improved NumPy, pandas integrations | | ✅ | |
| Numba integration for improved Python performance | | ✅ | The open-source just-in-time (JIT) compiler for Python, used to translate portions of Python code into optimized machine code using LLVM. |
| "Deephaven learn" library for updating integration with ML libraries | | ✅ | Gather-compute-scatter. |
| Declarative Query Syntax Tree (QST) API | | ✅ | |
| SQL Alpha | | ✅ | Use SQL to query Deephaven tables (select, etc.) |
| Deephaven time library for dbDateTime, etc. | ✅ | ✅ | |
| Java time library for Instant, Duration, etc. | | ✅ | Support for 3 patterns: Buillt-in Java functions, Java object methods, and interoperable Python datetimes. |
| Calendar, businessTime support, etc. | ✅ | ✅ | Core+ rewritten to use new time libraries. |
| Ring tables | | ✅ | |
| Blink tables | | ✅ | |
| updateBy/update_by for cumulative, window, rolling aggregations | | ✅ | |
| Rollup WAvg, WSum | | ✅ | From 0.32.1 |
| Expand All in Hierarchical Tables | | ✅ | |
| RangeJoin / range_join | | ✅ | From Core+ version 0.24 |
| TableMaps | ✅ | | Partitioned tables in Core+ provide the functionality of TableMaps. |
| Partitioned Tables | | ✅ | Powerful for parallelization and parameterized queries. |
| Vectorized Column Iterators | | ✅ | |
| Downsampler | ✅ | | |
| Replay Query | ✅ | | You can replay within Core+ scripts, but there is no replay query yet. |
| ZipJoin | ✅ | | |
| Basic multi-core map-reduce builder | ✅ | | |
| Clients | Pure Python client | | ✅ | The Legacy client relies on jpy. The Core+ client uses gRPC and Pure Python. For optional ticking support, the C++ client is used as a backend. |
| Jupyter kernel | ✅ | | The Legacy Jupyter integration requires a "bootstrap" kernel, which then connects to a Deephaven worker. All Python evaluations must occur on the server. |
| Jupyter Client Integration | | ✅ | The Core+ Python client can be run in any Jupyter kernel. The Deephaven grid widget is used to display tables. |
| Java Client | ✅ | ✅ | |
| C++ Client | | ✅ | |
| Go Client (snapshots, not streaming) | | ✅ | |
| R Client | ✅ | ✅ | The Legacy client relies on jpy. The Core+ R client uses the C++ client internally. |
| C# OpenAPI Library | ✅ | | |
| C# Client | | | Core+ target, arriving March 2024. |
| Excel Client | ✅ | | Arriving to Core+ April 2024. |
| UI/UX and workflow | WebUI / Web-IDE / PQ workflow | ✅ | ✅ | |
| Swing console | ✅ | | |
| Import Web Notebooks from Groovy and Python | | ✅ | |
| Plots: Deephaven plotting library supported (Groovy, Java) | ✅ | ✅ | |
| Plots: TICKING Plotly Express (top ~30 plots in real time) | | ✅ | The "Deephaven Express" integration also allows for full control of the figure and its styling. |
| Plots: STATIC Plotly, matplotlib, seaborn, ggplot, etc. for snapshots | | ✅ | |
| Embeddable iframes | | ✅ | |
| Deephaven ipy widget library | | ✅ | |
| Bidirectional plugins | | ✅ | Used for delivering front-end JS code or back-end libraries. |
| Builder-driven UI Parameterized PQs | ✅ | Use Deephaven.ui instead. | |
| deephaven.ui: Programmatic parameterized PQs | | ✅ | |
| deephaven.ui: Rich callback framework | | ✅ | React Spectrum is integrated. |
| deephaven.ui: Programmatical layouts | | ✅ | |
| deephaven.ui: Theming | | ✅ | Set colors. Soon fonts, etc. |
| Builder-driven Pivot Widget | ✅ | | |
| deephaven.ui: Pivot widget plugin | | ✅ | |
| Column grouping | ✅ | ✅ | |
| Column statistics viewable in WebUI | ✅ | ✅ | |
| Column statistic unique value counts | | ✅ | |
| System | PyDocs and Javadocs, autocomplete in popular IDEs | | ✅ | |
| Queries Survive Controller Restart | | ✅ | Vermilion+, Core+ only |
| Multiple WorkerKinds | | ✅ | Permits distinct Python Virtual Environments, Java Libraries, Test Versions of Worker. |
| Replica PQs | | | Grizzly. A PQ type that can have N identical workers + M hot backups, with support for consumers' round-robin. |