Class TableTools

java.lang.Object
io.deephaven.engine.util.TableTools

public class TableTools extends Object
Tools for working with tables. This includes methods to examine tables, combine them, convert them to and from CSV files, and create and manipulate columns.
  • Field Details

  • Constructor Details

    • TableTools

      public TableTools()
  • Method Details

    • show

      public static void show(Table source, String... columns)
      Prints the first few rows of a table to standard output.
      Parameters:
      source - a Deephaven table object
      columns - varargs of column names to display
    • showWithRowSet

      public static void showWithRowSet(Table source, String... columns)
      Prints the first few rows of a table to standard output, and also prints the details of the row keys and row positions that provided the values.
      Parameters:
      source - a Deephaven table object
      columns - varargs of column names to display
    • showCommaDelimited

      public static void showCommaDelimited(Table source, String... columns)
      Prints the first few rows of a table to standard output, with commas between values.
      Parameters:
      source - a Deephaven table object
      columns - varargs of column names to display
    • show

      public static void show(Table source, ZoneId timeZone, String... columns)
      Prints the first few rows of a table to standard output.
      Parameters:
      source - a Deephaven table object
      timeZone - a time zone constant relative to which date time data should be adjusted
      columns - varargs of column names to display
    • show

      public static void show(Table source, long maxRowCount, String... columns)
      Prints the first few rows of a table to standard output.
      Parameters:
      source - a Deephaven table object
      maxRowCount - the number of rows to return
      columns - varargs of column names to display
    • showWithRowSet

      public static void showWithRowSet(Table source, long maxRowCount, String... columns)
      Prints the first few rows of a table to standard output, and also prints the details of the row keys and row positions that provided the values.
      Parameters:
      source - a Deephaven table object
      maxRowCount - the number of rows to return
      columns - varargs of column names to display
    • showCommaDelimited

      public static void showCommaDelimited(Table source, long maxRowCount, String... columns)
      Prints the first few rows of a table to standard output, with commas between values.
      Parameters:
      source - a Deephaven table object
      maxRowCount - the number of rows to return
      columns - varargs of column names to display
    • show

      public static void show(Table source, long maxRowCount, ZoneId timeZone, String... columns)
      Prints the first few rows of a table to standard output.
      Parameters:
      source - a Deephaven table object
      maxRowCount - the number of rows to return
      timeZone - a time zone constant relative to which date time data should be adjusted
      columns - varargs of column names to display
    • show

      public static void show(Table source, long maxRowCount, ZoneId timeZone, PrintStream out, String... columns)
      Prints the first few rows of a table to standard output.
      Parameters:
      source - a Deephaven table object
      maxRowCount - the number of rows to return
      timeZone - a time zone constant relative to which date time data should be adjusted
      out - a PrintStream destination to which to print the data
      columns - varargs of column names to display
    • showWithRowSet

      public static void showWithRowSet(Table source, long maxRowCount, ZoneId timeZone, PrintStream out, String... columns)
      Prints the first few rows of a table to standard output, and also prints the details of the row keys and row positions that provided the values.
      Parameters:
      source - a Deephaven table object
      maxRowCount - the number of rows to return
      timeZone - a time zone constant relative to which date time data should be adjusted
      out - a PrintStream destination to which to print the data
      columns - varargs of column names to display
    • showWithRowSet

      public static void showWithRowSet(Table source, long firstRow, long lastRow, PrintStream out, String... columns)
      Prints the first few rows of a table to standard output, and also prints the details of the row keys and row positions that provided the values.
      Parameters:
      source - a Deephaven table object
      firstRow - the firstRow to display
      lastRow - the lastRow (exclusive) to display
      out - a PrintStream destination to which to print the data
      columns - varargs of column names to display
    • show

      public static void show(Table source, long maxRowCount, ZoneId timeZone, String delimiter, PrintStream out, boolean showRowSet, String... columns)
      Prints the first few rows of a table to standard output.
      Parameters:
      source - a Deephaven table object
      maxRowCount - the number of rows to return
      timeZone - a time zone constant relative to which date time data should be adjusted
      delimiter - a String value to use between printed values
      out - a PrintStream destination to which to print the data
      showRowSet - a boolean indicating whether to also print rowSet details
      columns - varargs of column names to display
    • showWithRowSet

      public static void showWithRowSet(Table source, long firstRow, long lastRow, String... columns)
      Prints the first few rows of a table to standard output, and also prints the details of the row keys and row positions that provided the values.
      Parameters:
      source - a Deephaven table object
      firstRow - the firstRow to display
      lastRow - the lastRow (exclusive) to display
      columns - varargs of column names to display
    • string

      public static String string(Table t, String... columns)
      Returns the first few rows of a table as a pipe-delimited string.
      Parameters:
      t - a Deephaven table object
      columns - varargs of columns to include in the result
      Returns:
      a String
    • string

      public static String string(Table t, int size, String... columns)
      Returns the first few rows of a table as a pipe-delimited string.
      Parameters:
      t - a Deephaven table object
      size - the number of rows to return
      columns - varargs of columns to include in the result
      Returns:
      a String
    • string

      public static String string(Table t, ZoneId timeZone, String... columns)
      Returns the first few rows of a table as a pipe-delimited string.
      Parameters:
      t - a Deephaven table object
      timeZone - a time zone constant relative to which date time data should be adjusted
      columns - varargs of columns to include in the result
      Returns:
      a String
    • string

      public static String string(@NotNull @NotNull Table table, int size, ZoneId timeZone, @NotNull @NotNull String... columns)
      Returns the first few rows of a table as a pipe-delimited string.
      Parameters:
      table - a Deephaven table object
      size - the number of rows to return
      timeZone - a time zone constant relative to which date time data should be adjusted
      columns - varargs of columns to include in the result
      Returns:
      a String
    • html

      public static String html(Table source)
      Returns a printout of a table formatted as HTML. Limit use to small tables to avoid running out of memory.
      Parameters:
      source - a Deephaven table object
      Returns:
      a String of the table printout formatted as HTML
    • diff

      public static String diff(Table actualResult, Table expectedResult, long maxDiffLines)
      Computes the difference of two tables for use in verification.
      Parameters:
      actualResult - first Deephaven table object to compare
      expectedResult - second Deephaven table object to compare
      maxDiffLines - stop comparing after this many differences are found
      Returns:
      String report of the detected differences
    • diff

      public static String diff(Table actualResult, Table expectedResult, long maxDiffLines, EnumSet<TableDiff.DiffItems> itemsToSkip)
      Computes the difference of two tables for use in verification.
      Parameters:
      actualResult - first Deephaven table object to compare
      expectedResult - second Deephaven table object to compare
      maxDiffLines - stop comparing after this many differences are found
      itemsToSkip - EnumSet of checks not to perform, such as checking column order, or exact match of double values
      Returns:
      String report of the detected differences
    • diffPair

      public static Pair<String,Long> diffPair(Table actualResult, Table expectedResult, long maxDiffLines, EnumSet<TableDiff.DiffItems> itemsToSkip)
      Computes the difference of two tables for use in verification.
      Parameters:
      actualResult - first Deephaven table object to compare
      expectedResult - second Deephaven table object to compare
      maxDiffLines - stop comparing after this many differences are found
      itemsToSkip - EnumSet of checks not to perform, such as checking column order, or exact match of double values
      Returns:
      a pair of String report of the detected differences, and the first different row (0 if there are no different data values)
    • nullToNullString

      public static String nullToNullString(Object obj)
    • colSource

      public static <T> ColumnSource<T> colSource(Class<T> clazz, Collection<T> values)
      Creates an in-memory column of the specified type for a collection of values.
      Type Parameters:
      T - the type to use for the new column
      Parameters:
      clazz - the class to use for the new column
      values - a collection of values to populate the new column
      Returns:
      a Deephaven ColumnSource object
    • objColSource

      public static <T> ColumnSource<T> objColSource(T... values)
      Creates an in-memory column of the specified type for a collection of values
      Type Parameters:
      T - the type to use for the new column
      Parameters:
      values - a collection of values to populate the new column
      Returns:
      a Deephaven ColumnSource object
    • colSource

      public static ColumnSource<Long> colSource(long... values)
      Creates an in-memory column of type long for a collection of values.
      Parameters:
      values - a collection of values to populate the new column
      Returns:
      a Deephaven ColumnSource object
    • colSource

      public static ColumnSource<Integer> colSource(int... values)
      Creates an in-memory column of type int for a collection of values.
      Parameters:
      values - a collection of values to populate the new column
      Returns:
      a Deephaven ColumnSource object
    • colSource

      public static ColumnSource<Short> colSource(short... values)
      Creates an in-memory column of type short for a collection of values.
      Parameters:
      values - a collection of values to populate the new column
      Returns:
      a Deephaven ColumnSource object
    • colSource

      public static ColumnSource<Byte> colSource(byte... values)
      Creates an in-memory column of type byte for a collection of values.
      Parameters:
      values - a collection of values to populate the new column
      Returns:
      a Deephaven ColumnSource object
    • colSource

      public static ColumnSource<Character> colSource(char... values)
      Creates an in-memory column of type char for a collection of values.
      Parameters:
      values - a collection of values to populate the new column
      Returns:
      a Deephaven ColumnSource object
    • colSource

      public static ColumnSource<Double> colSource(double... values)
      Creates an in-memory column of type double for a collection of values.
      Parameters:
      values - a collection of values to populate the new column
      Returns:
      a Deephaven ColumnSource object
    • colSource

      public static ColumnSource<Float> colSource(float... values)
      Creates an in-memory column of type float for a collection of values.
      Parameters:
      values - a collection of values to populate the new column
      Returns:
      a Deephaven ColumnSource object
    • col

      public static <T> ColumnHolder<T> col(String name, T... data)
      Returns a ColumnHolder that can be used when creating in-memory tables.
      Type Parameters:
      T - the type of the column
      Parameters:
      name - name of the column
      data - a list of values for the column
      Returns:
      a Deephaven ColumnHolder object
    • stringCol

      public static ColumnHolder<String> stringCol(String name, String... data)
      Returns a ColumnHolder of type String that can be used when creating in-memory tables.
      Parameters:
      name - name of the column
      data - a list of values for the column
      Returns:
      a Deephaven ColumnHolder object
    • instantCol

      public static ColumnHolder<Instant> instantCol(String name, Instant... data)
      Returns a ColumnHolder of type Instant that can be used when creating in-memory tables.
      Parameters:
      name - name of the column
      data - a list of values for the column
      Returns:
      a Deephaven ColumnHolder object
    • booleanCol

      public static ColumnHolder<Boolean> booleanCol(String name, Boolean... data)
      Returns a ColumnHolder of type Boolean that can be used when creating in-memory tables.
      Parameters:
      name - name of the column
      data - a list of values for the column
      Returns:
      a Deephaven ColumnHolder object
    • longCol

      public static ColumnHolder<Long> longCol(String name, long... data)
      Returns a ColumnHolder of type long that can be used when creating in-memory tables.
      Parameters:
      name - name of the column
      data - a list of values for the column
      Returns:
      a Deephaven ColumnHolder object
    • intCol

      public static ColumnHolder<Integer> intCol(String name, int... data)
      Returns a ColumnHolder of type int that can be used when creating in-memory tables.
      Parameters:
      name - name of the column
      data - a list of values for the column
      Returns:
      a Deephaven ColumnHolder object
    • shortCol

      public static ColumnHolder<Short> shortCol(String name, short... data)
      Returns a ColumnHolder of type short that can be used when creating in-memory tables.
      Parameters:
      name - name of the column
      data - a list of values for the column
      Returns:
      a Deephaven ColumnHolder object
    • byteCol

      public static ColumnHolder<Byte> byteCol(String name, byte... data)
      Returns a ColumnHolder of type byte that can be used when creating in-memory tables.
      Parameters:
      name - name of the column
      data - a list of values for the column
      Returns:
      a Deephaven ColumnHolder object
    • charCol

      public static ColumnHolder<Character> charCol(String name, char... data)
      Returns a ColumnHolder of type char that can be used when creating in-memory tables.
      Parameters:
      name - name of the column
      data - a list of values for the column
      Returns:
      a Deephaven ColumnHolder object
    • doubleCol

      public static ColumnHolder<Double> doubleCol(String name, double... data)
      Returns a ColumnHolder of type double that can be used when creating in-memory tables.
      Parameters:
      name - name of the column
      data - a list of values for the column
      Returns:
      a Deephaven ColumnHolder object
    • floatCol

      public static ColumnHolder<Float> floatCol(String name, float... data)
      Returns a ColumnHolder of type float that can be used when creating in-memory tables.
      Parameters:
      name - name of the column
      data - a list of values for the column
      Returns:
      a Deephaven ColumnHolder object
    • emptyTable

      public static Table emptyTable(long size)
      Returns a new, empty Deephaven Table.
      Parameters:
      size - the number of rows to allocate space for
      Returns:
      a Deephaven Table with no columns.
    • newTable

      public static Table newTable(long size, List<String> names, List<ColumnSource<?>> columnSources)
      Creates a new Table.
      Parameters:
      size - the number of rows to allocate
      names - a List of column names
      columnSources - a List of the ColumnSource(s)
      Returns:
      a Deephaven Table
    • newTable

      public static Table newTable(long size, Map<String,ColumnSource<?>> columns)
      Creates a new Table.
      Parameters:
      size - the number of rows to allocate
      columns - a Map of column names and ColumnSources
      Returns:
      a Deephaven Table
    • newTable

      public static Table newTable(TableDefinition definition)
      Creates a new Table.
      Parameters:
      definition - the TableDefinition (column names and properties) to use for the new table
      Returns:
      an empty Deephaven Table
    • newTable

      public static Table newTable(ColumnHolder<?>... columnHolders)
      Creates a new Table.
      Parameters:
      columnHolders - a list of ColumnHolders from which to create the table
      Returns:
      a Deephaven Table
    • newTable

      public static Table newTable(TableDefinition definition, ColumnHolder<?>... columnHolders)
    • metaTable

      public static Table metaTable(TableDefinition definition)
      Creates a metadata Table representing the columns in definition. Will include the following columns:
      Name
      ColumnDefinition.getName()
      DataType
      From ColumnDefinition.getDataType(), result of Class.getCanonicalName() if non-null, else Class.getName()
      ColumnType
      ColumnDefinition#getColumnType()
      IsPartitioning
      ColumnDefinition.isPartitioning()
      Parameters:
      definition - the definition
      Returns:
      the metadata Table
    • timeTable

      public static Table timeTable(String period)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      period - time interval between new row additions.
      Returns:
      time table
    • timeTable

      public static Table timeTable(String period, ReplayerInterface replayer)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      period - time interval between new row additions
      replayer - data replayer
      Returns:
      time table
    • timeTable

      public static Table timeTable(Instant startTime, String period)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      startTime - start time for adding new rows
      period - time interval between new row additions
      Returns:
      time table
    • timeTable

      public static Table timeTable(Instant startTime, String period, ReplayerInterface replayer)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      startTime - start time for adding new rows
      period - time interval between new row additions
      replayer - data replayer
      Returns:
      time table
    • timeTable

      public static Table timeTable(String startTime, String period)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      startTime - start time for adding new rows
      period - time interval between new row additions
      Returns:
      time table
    • timeTable

      public static Table timeTable(String startTime, String period, ReplayerInterface replayer)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      startTime - start time for adding new rows
      period - time interval between new row additions
      replayer - data replayer
      Returns:
      time table
    • timeTable

      public static Table timeTable(long periodNanos)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      periodNanos - time interval between new row additions in nanoseconds.
      Returns:
      time table
    • timeTable

      public static Table timeTable(long periodNanos, ReplayerInterface replayer)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      periodNanos - time interval between new row additions in nanoseconds.
      replayer - data replayer
      Returns:
      time table
    • timeTable

      public static Table timeTable(Instant startTime, long periodNanos)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      startTime - start time for adding new rows
      periodNanos - time interval between new row additions in nanoseconds.
      Returns:
      time table
    • timeTable

      public static Table timeTable(Instant startTime, long periodNanos, ReplayerInterface replayer)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      startTime - start time for adding new rows
      periodNanos - time interval between new row additions in nanoseconds.
      replayer - data replayer
      Returns:
      time table
    • timeTable

      public static Table timeTable(String startTime, long periodNanos)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      startTime - start time for adding new rows
      periodNanos - time interval between new row additions in nanoseconds.
      Returns:
      time table
    • timeTable

      public static Table timeTable(String startTime, long periodNanos, ReplayerInterface replayer)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      startTime - start time for adding new rows
      periodNanos - time interval between new row additions in nanoseconds.
      replayer - data replayer
      Returns:
      time table
    • timeTable

      public static Table timeTable(Clock clock, Instant startTime, long periodNanos)
      Creates a table that adds a new row on a regular interval.
      Parameters:
      clock - the clock
      startTime - start time for adding new rows
      periodNanos - time interval between new row additions in nanoseconds.
      Returns:
      time table
    • timeTableBuilder

      public static TimeTable.Builder timeTableBuilder()
      Creates a new time table builder.
      Returns:
      a time table builder
    • merge

      public static Table merge(List<Table> theList)
      Concatenates multiple Deephaven Tables into a single Table.

      The resultant table will have rows from the same table together, in the order they are specified as inputs.

      When ticking tables grow, they may run out of the 'pre-allocated' space for newly added rows. When more key- space is needed, tables in higher key-space are shifted to yet higher key-space to make room for new rows. Shifts are handled efficiently, but some downstream operations generate a linear O(n) amount of work per shifted row. When possible, one should favor ordering the constituent tables first by static/non-ticking sources followed by tables that are expected to grow at slower rates, and finally by tables that grow without bound.

      Parameters:
      theList - a List of Tables to be concatenated
      Returns:
      a Deephaven table object
    • merge

      public static Table merge(Collection<Table> tables)
      Concatenates multiple Deephaven Tables into a single Table.

      The resultant table will have rows from the same table together, in the order they are specified as inputs.

      When ticking tables grow, they may run out of the 'pre-allocated' space for newly added rows. When more key- space is needed, tables in higher key-space are shifted to yet higher key-space to make room for new rows. Shifts are handled efficiently, but some downstream operations generate a linear O(n) amount of work per shifted row. When possible, one should favor ordering the constituent tables first by static/non-ticking sources followed by tables that are expected to grow at slower rates, and finally by tables that grow without bound.

      Parameters:
      tables - a Collection of Tables to be concatenated
      Returns:
      a Deephaven table object
    • merge

      public static Table merge(Table... tables)
      Concatenates multiple Deephaven Tables into a single Table.

      The resultant table will have rows from the same table together, in the order they are specified as inputs.

      When ticking tables grow, they may run out of the 'pre-allocated' space for newly added rows. When more key- space is needed, tables in higher key-space are shifted to yet higher key-space to make room for new rows. Shifts are handled efficiently, but some downstream operations generate a linear O(n) amount of work per shifted row. When possible, one should favor ordering the constituent tables first by static/non-ticking sources followed by tables that are expected to grow at slower rates, and finally by tables that grow without bound.

      Parameters:
      tables - a list of Tables to be concatenated
      Returns:
      a Deephaven table object
    • mergeSorted

      public static Table mergeSorted(String keyColumn, Table... tables)
      Concatenates multiple sorted Deephaven Tables into a single Table sorted by the specified key column.

      The input tables must each individually be sorted by keyColumn, otherwise results are undefined.

      Parameters:
      tables - sorted Tables to be concatenated
      keyColumn - the column to use when sorting the concatenated results
      Returns:
      a Deephaven table object
    • mergeSorted

      public static Table mergeSorted(String keyColumn, Collection<Table> tables)
      Concatenates multiple sorted Deephaven Tables into a single Table sorted by the specified key column.

      The input tables must each individually be sorted by keyColumn, otherwise results are undefined.

      Parameters:
      tables - a Collection of sorted Tables to be concatenated
      keyColumn - the column to use when sorting the concatenated results
      Returns:
      a Deephaven table object
    • roundDecimalColumns

      @ScriptApi public static Table roundDecimalColumns(Table table)
      Produce a new table with all the columns of this table, in the same order, but with double and float columns rounded to longs.
      Returns:
      The new Table, with all double and float columns rounded to longs.
    • roundDecimalColumnsExcept

      @ScriptApi public static Table roundDecimalColumnsExcept(Table table, String... columnsNotToRound)
      Produce a new table with all the columns of this table, in the same order, but with all double and float columns rounded to longs, except for the specified columnsNotToRound.
      Parameters:
      columnsNotToRound - The names of the double and float columns not to round to longs
      Returns:
      The new Table, with columns modified as explained above
    • roundDecimalColumns

      @ScriptApi public static Table roundDecimalColumns(Table table, String... columns)
      Produce a new table with all the columns of this table, in the same order, but with double and float columns rounded to longs.
      Parameters:
      columns - The names of the double and float columns to round.
      Returns:
      The new Table, with the specified columns rounded to longs.
      Throws:
      IllegalArgumentException - If columns is null, or if one of the specified columns is neither a double column nor a float column.
    • computeFingerprint

      public static byte[] computeFingerprint(Table source) throws IOException

      Compute the SHA256 hash of the input table.

      The hash is computed using every value in each row, using toString for unrecognized objects. The hash also includes the input table definition column names and types.

      Parameters:
      source - The table to fingerprint
      Returns:
      The SHA256 hash of the table data and TableDefinition
      Throws:
      IOException - If an error occurs during the hashing.
    • base64Fingerprint

      public static String base64Fingerprint(Table source) throws IOException

      Compute the SHA256 hash of the input table and return it in base64 string format.

      Parameters:
      source - The table to fingerprint
      Returns:
      The SHA256 hash of the table data and TableDefinition
      Throws:
      IOException - If an error occurs during the hashing.
    • nullTypeAsString

      public static String nullTypeAsString(Class<?> dataType)
    • typeFromName

      public static Class<?> typeFromName(String dataTypeStr)