Skip to main content

Deephaven's modernized time library

· 15 min read
DALL·E prompt: digital art of a clean aseptic futuristic modern library full of clocks
Chip Kent
Rewriting date-time handling brings major benefits
note

Since publishing this blog, we have further improved our time library. See Deephaven's modernized time library -- Part 2 for more details.

Deephaven Core v0.25 introduces major changes to modernize and standardize its date-time handling. The query libraries were originally written more than a decade ago. Over that time, Java's approach to date-time data has changed, so a rewrite was in order. v0.25 brings Deephaven up to speed with current standards.

This rewrite is a breaking change. Your data is safe but many of your queries may need to be updated. A stable API is a high priority for Deephaven and our customers; we did not take this decision lightly. Instead of a series of small changes, we decided that one large change would result in less disruption for users. We appreciate the time and effort involved and thank you for going on this journey with us.

If you are having difficulty porting your code, please contact Deephaven on Slack.

Overview

  • Standard Java date-time types (Instant, LocalDate, LocalTime, ZoneId, ZonedDateTime, Period, Duration) replace custom Deephaven date-time types (DateTime, TimeZone, Period).
  • Functions have been thoughtfully and consistently renamed.
  • Missing functions have been added.
  • Rarely used functions have been removed.
  • Inconsistent function behavior has been resolved.
  • Functions are fully documented (Javadoc and Pydoc).
  • Unit test coverage is now >90%.
  • References to the Joda time library have been removed.
  • ISO standard formats replace Deephaven-specific formats.
  • More date-time types and formats can be parsed.
  • User-configurable time zone aliases replace hard-coded time zone aliases (e.g., NY).
  • The Java ZoneId default time zone replaces two default time zones.

Porting

We have renamed functions and deleted a few that are rarely used.

  • See our Pydoc to reference the new Python API.
  • See our Javadoc to reference the new Java API.

To aid in porting code, the following tables indicate approximate relationships between old and new function names.

Java API changes

TypeNew Method / ConstantOld Method / Constant
ArithmeticdiffDaysdayDiff, diffDay
ArithmeticdiffMicros-
ArithmeticdiffMillis-
ArithmeticdiffMinutes-
ArithmeticdiffNanosdiffNanos, diff
ArithmeticdiffSeconds-
ArithmeticdiffYears365diffYear, yearDiff
ArithmeticdiffYearsAvg-
Arithmeticminusminus
Arithmeticplusplus
Arithmetic-cappedTimeOffset
BinninglowerBinlowerBin
BinningupperBinupperBin
ChronologyatMidnightdateAtMidnight, millisToDateAtMidnight
ChronologydayOfMonthdayOfMonth
ChronologydayOfWeekdayOfWeek
ChronologydayOfYeardayOfYear
ChronologyhourOfDayhourOfDay
ChronologymicrosOfMillimicrosOfMilli
ChronologymicrosOfSecond-
ChronologymillisOfDaymillisOfDay
ChronologymillisOfSecondmillisOfSecond
ChronologyminuteOfDayminuteOfDay
ChronologyminuteOfHourminuteOfHour
ChronologymonthOfYearmonthOfYear
ChronologynanosOfDaynanosOfDay
ChronologynanosOfMilli-
ChronologynanosOfSecondnanosOfSecond
ChronologysecondOfDaysecondOfDay
ChronologysecondOfMinutesecondOfMinute
Chronologyyearyear
ChronologyyearOfCenturyyearOfCentury
ClockcurrentClockcurrentClock
ClocknowcurrentTime
ClocknowMillisResolutioncurrentTimeMillis
ClocknowSystem-
ClocknowSystemMillisResolution
ClocksetClock-
ClocktodaycurrentDate
Clock-clock
ComparisonsisAfterisAfter
ComparisonsisAfterOrEqual-
ComparisonsisBeforeisBefore
ComparisonsisBeforeOrEqual-
ConstantsDAYDAY
ConstantsDAYS_PER_NANO-
ConstantsHOURHOUR
ConstantsHOURS_PER_NANO-
ConstantsMICRO-
ConstantsMILLI-
ConstantsMINUTEMINUTE
ConstantsMINUTES_PER_NANO-
ConstantsSECONDSECOND
ConstantsSECONDS_PER_NANO-
ConstantsWEEKWEEK
ConsantsYEAR_365YEAR
ConstantsYEAR_AVG-
ConstantsYEARS_PER_NANO_365-
ConstantsYEARS_PER_NANO_AVG-
ConstantsZERO_LENGTH_INSTANT_ARRAYZERO_LENGTH_DATETIME_ARRAY
Conversions: Date-time typestoDate-
Conversions: Date-time typestoInstanttoDateTime
Conversions: Date-time typestoLocalDate-
Conversions: Date-time typestoLocalTime-
Conversions: Date-time typestoZonedDateTimegetZonedDateTime
Conversions: EpochepochAutoToEpochNanosautoEpochToNanos
Conversions: EpochepochAutoToInstantautoEpochToTime
Conversions: EpochepochAutoToZonedDateTime-
Conversions: EpochepochMicros-
Conversions: EpochepochMicrosToInstantmicrosToTime
Conversions: EpochepochMicrosToZonedDateTime-
Conversions: EpochepochMillismillis
Conversions: EpochepochMillisToInstantmillisToTime
Conversions: EpochepochMillisToZonedDateTime-
Conversions: EpochepochNanosnanos, toEpochNano
Conversions: EpochepochNanosToInstantnanosToTime, makeInstant
Conversions: EpochepochNanosToZonedDateTimemakeZonedDateTime
Conversions: EpochepochSeconds-
Conversions: EpochepochSecondsToInstantsecondsToTime
Conversions: EpochepochSecondsToZonedDateTime-
Conversions: ExcelexcelToInstant-
Conversions: ExcelexcelToZonedDateTime-
Conversions: ExceltoExcelTimegetExcelDateTime
Conversions: Time unitsmicrosToMillis-
Conversions: Time unitsmicrosToNanosmicrosToNanos
Conversions: Time unitsmicrosToSeconds-
Conversions: Time unitsmillisToMicros-
Conversions: Time unitsmillisToNanosmillisToNanos
Conversions: Time unitsmillisToSeconds-
Conversions: Time unitsnanosToMicrosnanosToMicros
Conversions: Time unitsnanosToMillisnanosToMillis
Conversions: Time unitsnanosToSeconds-
Conversions: Time unitssecondsToMicros-
Conversions: Time unitssecondsToMillis-
Conversions: Time unitssecondsToNanossecondsToNanos
FormatformatDateformatDate
FormatformatDateTimeformat
FormatformatDurationNanos-
Format-getPartitionFromTimestampMillis
Format-getPartitionFromTimestampMicros
Format-getPartitionFromTimestampNanos
Format-getPartitionFromTimestampSeconds
Format-createFormatter
Format-DateStyle
Overflow / UnderflowDateTimeOverflowExceptionDateTimeOverflowException
ParseDateTimeParseException-
ParseparseDurationconvertPeriod
ParseparseDurationNanosexpressionToNanos
ParseparseDurationNanosQuiet-
ParseparseDurationQuietconvertPeriodQuiet
ParseparseEpochNanos-
ParseparseEpochNanosQuiet-
ParseparseInstantconvertDateTime
ParseparseInstantQuietconvertDateTimeQuiet
ParseparseLocalDateconvertDate
ParseparseLocalDateQuietconvertDateQuiet
ParseparseLocalTimeconvertLocalTime
ParseparseLocalTimeQuietconvertLocalTimeQuiet
ParseparsePeriodconvertPeriod
ParseparsePeriodQuietconvertPeriodQuiet
ParseparseTimePrecisiongetFinestDefinedUnit
ParseparseTimePrecisionQuiet-
ParseparseTimeZone-
ParseparseTimeZoneQuiet-
ParseparseZonedDateTime-
ParseparseZonedDateTimeQuiet-
Parse-convertTime
Parse-convertTimeQuiet
Time ZonetimeZone-
Time ZonetimeZoneAliasAdd-
Time ZonetimeZoneAliasRm-
Query LanguageTimeLiteralReplacedExpressionconvertExpression

Python API changes

TypeNew Method / ConstantOld Method / Constant
Arithmeticdiff_days-
Arithmeticdiff_micros-
Arithmeticdiff_millis-
Arithmeticdiff_minutes-
Arithmeticdiff_nanosdiff_nanos, minus
Arithmeticdiff_seconds-
Arithmeticdiff_years_365-
Arithmeticdiff_years_avg-
Arithmeticminus_periodminus_period, minus_nanos
Arithmeticplus_periodplus_period, plus_nanos
Chronologyat_midnightdatetime_at_midnight
Chronologyday_of_monthday_of_month
Chronologyday_of_weekday_of_week
Chronologyday_of_yearday_of_year
Chronologyhour_of_dayhour_of_day
Chronologymicros_of_millimicrosOfMilli
Chronologymicros_of_second-
Chronologymillis_of_daymillis_of_day
Chronologymillis_of_secondmillis_of_second
Chronologyminute_of_dayminute_of_day
Chronologyminute_of_hourminute_of_hour
Chronologymonth_of_yearmonth_of_year
Chronologynanos_of_daynanos_of_day
Chronologynanos_of_milli-
Chronologynanos_of_secondnanos_of_second
Chronologysecond_of_daysecond_of_day
Chronologysecond_of_minutesecond_of_minute
Chronologyyearyear
Chronologyyear_of_centuryyear_of_century
Clocknownow
Clocktoday-
Comparisonsis_afteris_after
Comparisonsis_after_or_equal-
Comparisonsis_beforeis_before
Comparisonsis_before_or_equal-
ConstantsDAYDAY
ConstantsDAYS_PER_NANO-
ConstantsHOURHOUR
ConstantsHOURS_PER_NANO-
ConstantsMICRO-
ConstantsMILLI-
ConstantsMINUTEMINUTE
ConstantsMINUTES_PER_NANO-
ConstantsSECONDSECOND
ConstantsSECONDS_PER_NANO-
ConstantsWEEKWEEK
ConsantsYEAR_365YEAR
ConstantsYEAR_AVG-
ConstantsYEARS_PER_NANO_365-
ConstantsYEARS_PER_NANO_AVG-
Conversions: Date-time typesmake_instant-
Conversions: Date-time typesmake_zdt-
Conversions: Date-time typesto_instantto_datetime
Conversions: Date-time typesto_local_date-
Conversions: Date-time typesto_local_time-
Conversions: Date-time typesto_zdt-
Conversions: Epochepoch_auto_to_epoch_nanos-
Conversions: Epochepoch_auto_to_instant-
Conversions: Epochepoch_auto_to_ztd-
Conversions: Epochepoch_micros-
Conversions: Epochepoch_micros_to_instant-
Conversions: Epochepoch_micros_to_zdt-
Conversions: Epochepoch_millismillis
Conversions: Epochepoch_millis_to_instantmillis_to_datetime
Conversions: Epochepoch_millis_to_zdt-
Conversions: Epochepoch_nanosnanos, toEpochNano
Conversions: Epochepoch_nanos_to_instantnanos_to_datetime
Conversions: Epochepoch_nanos_to_ztd-
Conversions: Epochepoch_seconds-
Conversions: Epochepoch_seconds_to_instant-
Conversions: Epochepoch_seconds_to_zdt-
Conversions: Excelexcel_to_instant-
Conversions: Excelexcel_to_ztd-
Conversions: Excelto_excel_time-
Conversions: Time unitsmicros_to_millis-
Conversions: Time unitsmicros_to_nanos-
Conversions: Time unitsmicros_to_seconds-
Conversions: Time unitsmillis_to_micros-
Conversions: Time unitsmillis_to_nanosmillis_to_nanos
Conversions: Time unitsmillis_to_seconds-
Conversions: Time unitsnanos_to_micros-
Conversions: Time unitsnanos_to_millisnanos_to_millis
Conversions: Time unitsnanos_to_seconds-
Conversions: Time unitsseconds_to_micros-
Conversions: Time unitsseconds_to_millis-
Conversions: Time unitsseconds_to_nanos-
Formatformat_dateformat_date
Formatformat_datetimeformat
Formatformat_duration_nanosformat_nanos
Parseparse_durationto_period
Parseparse_duration_nanosto_nanos
Parseparse_epoch_nanos-
Parseparse_instantconvertDateTime
Parseparse_local_date
Parseparse_local_time-
Parseparse_periodto_period
Parseparse_time_precisiongetFinestDefinedUnit
Parseparse_time_zone-
Parseparse_ztd-
Parse-convertTime
Parse-convertTimeQuiet
Table operationtime_tabletime_table
Time Zonetime_zone-
Time Zonetime_zone_alias_add-
Time Zonetime_zone_alias_rm-
Time Zone-TimeZone

Type Changes

Deephaven date-time types (DateTime, TimeZone, Period) have been replaced with standard Java date-time types (Instant, LocalDate, LocalTime, ZoneId, ZonedDateTime, Period, Duration).

In most cases, type changes will require few or no code changes for Python and Groovy users. If your code directly interacts with one of the old date-time types, it will need to be ported to the new types.

In the new library, the key types are:

  • Instant - An instant in time. This is analogous to DateTime in the old date-time library.
  • LocalDate - A date with no time zone. No analog in the old date-time library.
  • LocalTime - A time with no date or time zone. No analog in the old date-time library.
  • ZoneId - A time zone identifier. This is analogous to TimeZone in the old date-time library.
  • ZonedDateTime - An instant with a time zone. No analog in the old date-time library.
  • Duration - A time period expressed in clock time (e.g., 1 second). This is analogous to Period in the old date-time library.
  • Period - A time period expressed in calendar time (e.g., 1 month). This is analogous to Period in the old date-time library.

Note that the old Period has been replaced by both Duration and Period. Durations are elapsed clock time (hours, minutes, seconds, nanoseconds), while Periods are elapsed calendar time (days, weeks, months, years). If you are using an old Period that contains both clock time and calendar time, it will need to be broken into both a period and a duration. For example:

from deephaven.time import parse_period, parse_duration

print("original period: 1YT21H5M")

period = parse_period("P1Y")
duration = parse_duration("PT21H5M")
print(period)
print(duration)

If you are using an old Period to express clock time, that Period will need to be converted to a Duration.

tip

In old Period strings, the T character divided what is now a Period and Duration.

Literal changes

As part of the transition to ISO formats, the literal values of some types have changed.

  • Existing DateTime literal formats can still be parsed as Instant or ZonedDateTime. You will need to change the name of the parsing method.
    • Python:
      • OLD: to_datetime("2021-01-23T12:34 MT")
      • NEW: parse_instant("2021-01-23T12:34 MT")
    • Java / Groovy:
      • OLD: convertDateTime("2021-01-23T12:34 MT")
      • NEW: parseInstant("2021-01-23T12:34 MT")
  • Old Period now becomes Duration and/or Period. Formatting has changed to be ISO-compliant. Details on the new literal formats can be found in the Oracle docs for Duration and Period. In most cases, you will simply need to select between Duration and Period and then place the appropriate "P" or "PT" prefix at the front of the literal string.
    • Python:
      • Duration
        • OLD: to_period("T12h34m")
        • NEW: parse_duration("PT12h34m")
      • Period
        • OLD: to_period("1y2d")
        • NEW: parse_period("P1y2d")
    • Java / Groovy:
      • Duration
        • OLD: convertPeriod("T12h34m")
        • NEW: parseDuration("PT12h34m")
      • Period
        • OLD: convertPeriod("1y2d")
        • NEW: parsePeriod("P1y2d")
  • Nanosecond duration literals now start with PT to be consistent with other durations.
    • OLD: 2:15:23
    • NEW: PT2:15:23
  • Nanosecond duration literals no longer support days.
    • OLD: "2DT01:02:13"
    • NEW: "PT49:02:13"
  • Local time literals follow the ISO standard and no longer need an L prefix. - OLD: 'L12:43:15' - NEW: '12:43:15'

Time zones

Time zone handling has been changed to be more flexible and customizable.

Full-length time zone names are now valid; e.g., "2021-04-23T11:23 America/New_York".

Hard-coded time zones (e.g., TZ_NY) have been removed. To access a time zone, use the time_zone(name) or timeZone(name) functions in Python and Java, respectively.

  • Python:
    • OLD: tz = TZ_NY
    • NEW: tz = time_zone(“America/New_York”)
  • Java / Groovy:
    • OLD: tz = TZ_NY
    • NEW: tz = timeZone(“America/New_York”)

Time zone aliases have replaced a specified list of hard-coded time zones (e.g., NY). Some of the most common aliases are provided in the default configuration. If you are using any of the existing short time zone names not in the default configuration, you will need to either (1) change to full-length time zone names or (2) add the short name to the alias list. To specify an alias list, use the property timezone.aliases=/path_to_your_file.csv. The alias file is a CSV where the first column is the alias, and the second column is the full-length time zone name.

Default aliases can be found at in our GitHub repo.

The default time zone is used for formatting output when no time zone is provided. The default is now configurable via the user.timezone=<full_length_timezone> property. The default is set to Eastern Standard Time (America/New_York).