Persistent Query XML Schema

This guide discusses the XML schema files used by Persistent Queries (PQs) to define the structure of tables and their columns. Like tables in Deephaven's database, PQs can be defined with XML files. These XML files are stored in etcd, but can be exported from Deephaven.

Create a new PQ

To illustrate the XML schema of a PQ, first create a PQ in the UI.

  • In the Settings tab, name the PQ Test_PQ_for_XML. The rest of the settings can be left as the default.
  • In Scheduling, select Disabled.
  • In Script, set Runtime to Python and enter the following code:
from deephaven import time_table

my_table = time_table("PT10s").update(
    ["X = 0.1 * ii", "SinX = sin(X)", "CosX = cos(X)"]
)

eod_trades = db.historical_table("LearnDeephaven", "EODTrades")
stock_quotes = db.historical_table("LearnDeephaven", "StockQuotes")
stock_trades = db.historical_table("LearnDeephaven", "StockTrades")
  • Keep the default load balancing and permissions.

Then, head back to the Settings tab and click Save in the bottom right corner.

Get PQ XML schema

There are two ways to get the XML schema for a PQ. Each is described in a subsection below.

Query Monitor

The Query Monitor allows you to see the PQs you have access to. It also allows you to export the XML schema for a PQ. Highlight one or more PQs in the Query Monitor table, and click the More Actions... menu (three vertical dots) on the top right of the Query Monitor table. The option Export Selected Queries downloads the XML schemas for the selected PQs.

dhconfig

Note

This is only an option for users with administrative permissions and command line access to a Deephaven server.

You can also use the dhconfig pq command to export the XML schema for a PQ.

Inspect the XML

Here's the XML for that query, with placeholders in place of usernames:

<?xml version="1.0" encoding="UTF-8"?>
<PersistentQueryRoot>
  <PersistentQuery>
    <Serial>1741283026663000365</Serial>
    <Version>1</Version>
    <Owner xml:space="preserve">dhuser</Owner>
    <Name xml:space="preserve">Test_PQ_for_XML</Name>
    <Enabled>true</Enabled>
    <HeapSizeInGB>4.0</HeapSizeInGB>
    <AdditionalMemoryInGB>0.0</AdditionalMemoryInGB>
    <DataBufferPoolToHeapSizeRatio>0.25</DataBufferPoolToHeapSizeRatio>
    <DetailedGCLoggingEnabled>true</DetailedGCLoggingEnabled>
    <DbServerName xml:space="preserve">AutoQuery</DbServerName>
    <RestartUsers>1</RestartUsers>
    <ScriptCode xml:space="preserve">from deephaven import time_table

my_table = time_table("PT10s").update(
    ["X = 0.1 * ii", "SinX = sin(X)", "CosX = cos(X)"]
)

eod_trades = db.historical_table("LearnDeephaven", "EODTrades")
stock_quotes = db.historical_table("LearnDeephaven", "StockQuotes")
stock_trades = db.historical_table("LearnDeephaven", "StockTrades")</ScriptCode>
    <ScriptLanguage xml:space="preserve">Python</ScriptLanguage>
    <ConfigurationType xml:space="preserve">Script</ConfigurationType>
    <Scheduling>
      <StringArrayValue xml:space="preserve">SchedulerType=com.illumon.iris.controller.IrisQuerySchedulerDaily</StringArrayValue>
      <StringArrayValue xml:space="preserve">Calendar=USNYSE</StringArrayValue>
      <StringArrayValue xml:space="preserve">BusinessDays=false</StringArrayValue>
      <StringArrayValue xml:space="preserve">Days=true=true=true=true=true=true=true</StringArrayValue>
      <StringArrayValue xml:space="preserve">StartTime=07:55:00</StringArrayValue>
      <StringArrayValue xml:space="preserve">StopTime=23:55:00</StringArrayValue>
      <StringArrayValue xml:space="preserve">TimeZone=America/New_York</StringArrayValue>
      <StringArrayValue xml:space="preserve">SchedulingDisabled=true</StringArrayValue>
      <StringArrayValue xml:space="preserve">Overnight=false</StringArrayValue>
      <StringArrayValue xml:space="preserve">RepeatEnabled=false</StringArrayValue>
      <StringArrayValue xml:space="preserve">SkipIfUnsuccessful=false</StringArrayValue>
      <StringArrayValue xml:space="preserve">StopTimeDisabled=false</StringArrayValue>
      <StringArrayValue xml:space="preserve">RestartErrorCount=0</StringArrayValue>
      <StringArrayValue xml:space="preserve">RestartErrorDelay=0</StringArrayValue>
      <StringArrayValue xml:space="preserve">RestartWhenRunning=Yes</StringArrayValue>
    </Scheduling>
    <Timeout>0</Timeout>
    <JVMProfile xml:space="preserve">Default</JVMProfile>
    <LastModifiedByAuthenticated xml:space="preserve">dhuser</LastModifiedByAuthenticated>
    <LastModifiedByEffective xml:space="preserve">dhuser</LastModifiedByEffective>
    <LastModifiedTime>1741698148197000000</LastModifiedTime>
    <ExpirationTime>-9223372036854775808</ExpirationTime>
    <WorkerKind xml:space="preserve">DeephavenCommunity</WorkerKind>
    <CreatedTime>1741698148197000000</CreatedTime>
    <ReplicaCount>1</ReplicaCount>
    <SpareCount>0</SpareCount>
  </PersistentQuery>
</PersistentQueryRoot>

PQ configuration

Atop the file is the configuration, which defines important details including:

  • Serial number - a system-generated number that uniquely identifies the PQ.
  • Owner - The username of the PQ's owner.
  • Name - The PQ's name.
  • Heap size - The heap size selected when the PQ was created or most recently edited.
<?xml version="1.0" encoding="UTF-8"?>
<PersistentQueryRoot>
  <PersistentQuery>
    <Serial>1741283026663000365</Serial>
    <Version>1</Version>
    <Owner xml:space="preserve">dhuser</Owner>
    <Name xml:space="preserve">Test_PQ_for_XML</Name>
    <Enabled>true</Enabled>
    <HeapSizeInGB>4.0</HeapSizeInGB>
    <AdditionalMemoryInGB>0.0</AdditionalMemoryInGB>
    <DataBufferPoolToHeapSizeRatio>0.25</DataBufferPoolToHeapSizeRatio>
    <DetailedGCLoggingEnabled>true</DetailedGCLoggingEnabled>
    <DbServerName xml:space="preserve">AutoQuery</DbServerName>
    <RestartUsers>1</RestartUsers>

Script code

Below the configuration is the script code. It should look familiar, since it's identical to the PQ code entered previously. Additionally, the language and configuration type are specified.

    <ScriptCode xml:space="preserve">from deephaven import time_table

my_table = time_table("PT10s").update(["X = 0.1 * ii", "SinX = sin(X)", "CosX = cos(X)"])

eod_trades = db.historical_table("LearnDeephaven", "EODTrades")
stock_quotes = db.historical_table("LearnDeephaven", "StockQuotes")
stock_trades = db.historical_table("LearnDeephaven", "StockTrades")
</ScriptCode>
    <ScriptLanguage xml:space="preserve">Python</ScriptLanguage>
    <ConfigurationType xml:space="preserve">Script</ConfigurationType

Scheduling

The scheduling section is defined as an array of values that match the parameters entered when creating the PQ. Because the scheduling was disabled, note the SchedulingDisabled=true, meaning the PQ does not have a schedule, despite the default parameters entered.

    <Scheduling>
      <StringArrayValue xml:space="preserve">SchedulerType=com.illumon.iris.controller.IrisQuerySchedulerDaily</StringArrayValue>
      <StringArrayValue xml:space="preserve">Calendar=USNYSE</StringArrayValue>
      <StringArrayValue xml:space="preserve">BusinessDays=false</StringArrayValue>
      <StringArrayValue xml:space="preserve">Days=true=true=true=true=true=true=true</StringArrayValue>
      <StringArrayValue xml:space="preserve">StartTime=07:55:00</StringArrayValue>
      <StringArrayValue xml:space="preserve">StopTime=23:55:00</StringArrayValue>
      <StringArrayValue xml:space="preserve">TimeZone=America/New_York</StringArrayValue>
      <StringArrayValue xml:space="preserve">SchedulingDisabled=true</StringArrayValue>
      <StringArrayValue xml:space="preserve">Overnight=false</StringArrayValue>
      <StringArrayValue xml:space="preserve">RepeatEnabled=false</StringArrayValue>
      <StringArrayValue xml:space="preserve">SkipIfUnsuccessful=false</StringArrayValue>
      <StringArrayValue xml:space="preserve">StopTimeDisabled=false</StringArrayValue>
      <StringArrayValue xml:space="preserve">RestartErrorCount=0</StringArrayValue>
      <StringArrayValue xml:space="preserve">RestartErrorDelay=0</StringArrayValue>
      <StringArrayValue xml:space="preserve">RestartWhenRunning=Yes</StringArrayValue>
    </Scheduling>

Other settings

Other settings can be changed when defining the PQ and are stored in the XML. They include:

  • Timeout (as defined on the scheduling tab)
  • JVM profile
  • Last modification details
  • Worker kind
    <Timeout>0</Timeout>
    <JVMProfile xml:space="preserve">Default</JVMProfile>
    <LastModifiedByAuthenticated xml:space="preserve">dhuser</LastModifiedByAuthenticated>
    <LastModifiedByEffective xml:space="preserve">dhuser</LastModifiedByEffective>
    <LastModifiedTime>1741698148197000000</LastModifiedTime>
    <ExpirationTime>-9223372036854775808</ExpirationTime>
    <WorkerKind xml:space="preserve">DeephavenCommunity</WorkerKind>
    <CreatedTime>1741698148197000000</CreatedTime>
    <ReplicaCount>1</ReplicaCount>
    <SpareCount>0</SpareCount>
  </PersistentQuery>
</PersistentQueryRoot>