Converting Legacy scripts to Core+ Scripts Cheat Sheet

See the User Guide in the Core docs and Core+ docs for comprehensive documentation on the new APIs.

Data access methods

Empty .where() clauses

In a Legacy worker, a .where() method applied to an uncoalesced table (e.g., the return from db.t before filtering by a partitioning column) would coalesce the table. Alternatively, the table could explicitly be realized with the .coalsece() method. In Core+, you must use the .coalesce() method instead of .where().

Snapshots

In a Legacy worker, trigger.snapshot(source, ...) snapshots table source according to table trigger. In Core+, the method is called snapshotWhen and the positions of the tables are inverted in the call: the trigger is the argument and the source is the table on which the operation is applied: source.snapshotWhen(trigger, ...).

In Legacy, the second argument to the snapshot call is a boolean specifying if an initial snapshot should be made. In Core+, the equivalent is to pass SnapshotWhenOptions.Flag.INITIAL in the optional list of flags.

Time handling

The Deephaven Core+ time library is very different from the Legacy Library, having been thoroughly modernized as described in these blog posts:

• Time library rewrite: part 1
• Time library rewrite: part 2

In particular, Legacy queries that used currentDateNy() must use today(), business calendars are entirely re-written, and the syntax of periods as input to time_table have changed.

To get the current date, you must use the today() function, which takes an optional time zone. You can use a full time zone name like America/New_York or the shortcut ET for Eastern time. The equivalent function to currentDateNy() is either today(timeZone(`America/New_York`)) or today('ET'), as in the following example:

quotes = db.liveTable("Market", "EqQuote").where("Date=today('ET')")

quotes = db.live_table("Market", "EqQuote").where("Date=today('ET')")

When the time zone is omitted, the today() function uses the value of the user.timezone property.

The common Legacy idiom lastBusinessDateNy() to get yesterday's data has been replaced by functions on the BusinessCalendar class. First, you must retrieve the calendar with the calendar() function, which on a default Core+ installation returns the USNYSE business calendar. To verify your default business calendar, you can run the following command:

println(io.deephaven.time.calendar.Calendars.calendar().name())

import deephaven.calendar

print(deephaven.calendar.calendar().name())

You can specify an alternative business calendar name as an argument to the calendar() function. For filtering Date partitions, you may either use the minusBusinessDays method and pass today() as a String argument or use the pastBusinessDate method and convert the resulting java.time.LocalDate to a String.

As a shortcut, you can elide calendar() for the default calendar.

The following examples all retrieve yesterday's data:

quotes = db.historicalTable("Market", "EqQuote").where("Date=minusBusinessDays(today(), 1)")
quotes2 = db.historicalTable("Market", "EqQuote").where("Date=pastBusinessDate(1).toString()")
quotes3 = db.historicalTable("Market", "EqQuote").where("Date=calendar().minusBusinessDays(today(), 1)")
quotes4 = db.historicalTable("Market", "EqQuote").where("Date=calendar().pastBusinessDate(1).toString()")
quotes5 = db.historicalTable("Market", "EqQuote").where("Date=calendar(`USNYSE`).minusBusinessDays(today(), 1)")
quotes6 = db.historicalTable("Market", "EqQuote").where("Date=calendar(`USNYSE`).pastBusinessDate(1).toString()")
quotes7 = db.historical_table("Market", "EqQuote").where("Date=formatDate(calendar(`USNYSE`).pastBusinessDate(1))")

quotes = db.historical_table("Market", "EqQuote").where(
    "Date=minusBusinessDays(today(), 1)"
)
quotes2 = db.historical_table("Market", "EqQuote").where(
    "Date=pastBusinessDate(1).toString()"
)
quotes3 = db.historical_table("Market", "EqQuote").where(
    "Date=calendar().minusBusinessDays(today(), 1)"
)
quotes4 = db.historical_table("Market", "EqQuote").where(
    "Date=calendar().pastBusinessDate(1).toString()"
)
quotes5 = db.historical_table("Market", "EqQuote").where(
    "Date=calendar(`USNYSE`).minusBusinessDays(today(), 1)"
)
quotes6 = db.historical_table("Market", "EqQuote").where(
    "Date=calendar(`USNYSE`).pastBusinessDate(1).toString()"
)
quotes7 = db.historical_table("Market", "EqQuote").where(
    "Date=formatDate(calendar(`USNYSE`).pastBusinessDate(1))"
)

For time tables, you must use the new Period syntax. For example:

fiveSeconds = timeTable("PT5s")
oneHour = timeTable("PT1h")

from deephaven import time_table

five_seconds = time_table("PT5s")
one_hour = time_table("PT1h")

Python

Python-only summary of changes

• Method names are now snake_case. For example, lastBy becomes last_by.

• Aggregation names (previously ComboAggregateFactory) have changed, as detailed below. For example, AggLast becomes agg.last.

• In instances of more than one parameter, arguments may take array lists instead of multiple strings; e.g., t.update(["Update Statement 1", "Update Statement 2", ...]). These are marked in the "Python method renames" table below. For comparison:

  • In a Legacy script, AggTypes will be comma-separated: source.by(AggCombo(AggType("Col1"), AggType("Col2 = NewCol1")), "Key1" , "Key2").

  • In Core, this becomes an agg_list within arrays:

    agg_list=[
       agg.first(["Col1 = NewCol1"]),
       agg.last(["Col2 = NewCol2"]),
    ]
    
    source.agg_by(agg_list, by=["GroupingColumns..."])
    

Core+ Python import equivalents

Python method renames

| Type | Legacy | Core+ | Syntax | | ------------------ | ----------------------------- | --------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | Table constructors | db.timeTable | time_table | time_table("PT1S") Note: requires from deephaven import time_table | | | ttools.emptyTable | empty_table | empty_table(N) Note: requires from deephaven import empty_table | | Aggregation | AggArray | agg.group | .agg_by([agg.group(cols=["Y"])], by=["X"]) | | | AggCombo(<original params>) | agg_list | agg_list = [<original params>] | | | AggCount | agg.count_ | .agg_by([agg.count_(col="Number")] | | | AggFirst | agg.first | .agg_by([agg.first(cols=["Y"])], by=["X"]) | | | AggLast | agg.last | agg_by([agg.last(cols=["Y"])], by=["X"]) | | | AggMax | agg.max_ | .agg_by([agg.max_(cols=["Y"])], by=["X"]) | | | AggMed | agg.median | .agg_by([agg.median(cols=["Number"])], by=["X"]) | | | AggMin | agg.min_ | .agg_by([agg.min_(cols=["Y"])], by=["X"]) | | | AggPct | agg.pct | .agg_by([agg.pct(percentile=0.68, cols=["PctNumber = Number"])], by=["X"]) | | | AggStd | agg.std | .agg_by([agg.std(cols=["Number"])], by=["X"]) | | | AggSum | agg.sum_ | .agg_by([agg.sum_(cols=["Number"])], by=["X"]) | | | AggVar | agg.var | .agg_by([agg.var(cols=["Number"])], by=["X"]) | | | avgBy | avg_by | .avg_by() , .avg_by(by=["X"]) | | | countBy | count_by | .count_by("Count") | | | lastBy | last_by | .last_by | | | firstBy | first_by | .first_by | | | headBy | head_by | .head_by(2, by=["X"] | | | maxBy | max_by | .max_by() | | | medianBy | median_by | .median_by() | | | minBy | min_by | .min_by() | | | stdBy | std_by | .std_by() | | | sumBy | sum_by | .sum_by() | | | tailBy | tail_by | .tail_by(2, by=["X"]) | | | varBy | var_by | .var_by() | | | by | group_by | .group_by() | | Filter | headPct | head_pct | .head_pct(pct=0.40) | | | tailPct | tail_pct | .tail_pct(pct=0.40) | | | where | where | .where(filters=["Number > 3"]) | | | whereIn | where_in | .where_in(filter_table=filter, cols=["X = Values"]) | | | whereNotIn | where_not_in | .where_not_in(filter_table=filter, cols=["X = Values"]) | | Sort | sortDescending | sort_descending | .sort_descending(order_by=["X"]) | | Selection | dropColumns | drop_columns | .drop_columns(cols=["B", "D"]) | | | moveColumns | move_columns | .move_columns(idx=1, cols=["C"]) | | | moveUpColumns | move_columns_up | .move_columns_up(cols=["B"]) | | | renameColumns | rename_columns | .rename_columns(cols=["Fruit = A", "Type = C"]) | | | select | select | .select(formulas=["B"]) | | | updateView | update_view | .update_view(formulas=["X = B", "Y = sqrt(C)"]) | | | update | update | .update(formulas=["A", "X = B", "Y = sqrt(C)"]) | | | view | view | .view(formulas=["B"]) | | Join | exactJoin | exact_join | left.exact_join(table=right, on=["X"]) | | | leftJoin | left_outer_join | left_outer_join(l_table=left, r_table=right, on=["I"]) | | | naturalJoin | natural_join | left.natural_join(table=right, on=["DeptID"]) | | | merge | merge | merge([source1, source2]) | | Format | formatColumnWhere | format_column_where | .format_column_where(col="X", cond="X > 2", formula="RED") | | | formatColumns | format_columns | .format_columns(["A = B > 2 ? BLUE : NO_FORMATTING", "C = Decimal(`0.00%`)"]) | | | formatRowWhere | format_row_where | .format_row_where(cond="X > 2 && X < 4", formula="RED") | | Metadata | getMeta() | meta_table | .meta_table | | | size() | size | size | | Plotting | catHistPlot | plot_cat_hist | .plot_cat_hist(series_name="Keys count", t=source, category="Keys") | | | catPlot | plot_cat | plot_cat(series_name="Cagetories Plot", t=source, category="Categories", y="Values") | | | chartTitle | chart_title | chart_title(title="Name") | | | figureTitle | figure_title | figure_title(title="Name") | | | histPlot | plot_xy_hist | plot_xy_hist(series_name="Histogram Values", t=source, x="Values", nbins=N) | | | lineStyle | LineEndStyle | | | | newChart | new_chart | new_chart(index=int, row=int, col=int) | | | ohlcPlot | plot_ohlc | | .plot_ohlc(series_name="Name", t=source, x="TimestampBin", open="Open", high="High", low="Low", close="Close") | | | piePlot | plot_pie | .plot_pie(series_name="Name",t=source,category="category",y="y-values") | | | twinX | x_twin | x_twin(name="string") | | Time | convertDateTime | to_j_instant | to_j_instant("2022-01-01T00:00:00 ET") Note: requires from deephaven.time import to_j_instant |

Miscellaneous