dhconfig checkpoint

Checkpoint files are special files written by Deephaven to periodically record the state of tables on disk. These files allow the system to efficiently recover or reconstruct table data after a restart or failure, by resuming from the last checkpoint rather than replaying all historical data. Checkpoints are particularly important for large or persistent tables, as they significantly reduce recovery time and resource usage. Administrators may need to examine checkpoint files to verify table persistence, troubleshoot data issues, or audit system state.

The checkpoint configuration data type of the dhconfig tool is used to examine table checkpoint files on disk. Authentication options do not apply to this command; the executing user must have read access to the files to be examined. Checkpoint files are named table.size by default. File arguments may be the checkpoint file itself, or the directory containing table.size.

dhconfig checkpoint [list|help] [arguments]

checkpoint has only one action: list. list lists checkpoint configurations to a file or to stdout. The --help argument provides detailed information on list's available options and arguments:

dhconfig checkpoint list --help

usage: dhconfig checkpoint list [--configfile <arg>] [-csv] [-d <arg>] [--diskprops] [-e] [--etcd] [-f <arg>] [-h]
       [-head] [-l <arg>] [-v]
Print the contents of checkpoint files ('table.size' by default).

...followed by a description of all of the subcommand's options and some usage examples.

list examples

Display a specific checkpoint file:

/usr/illumon/latest/bin/dhconfig checkpoint list --file /db/path.to.table/table.size

/usr/illumon/latest/bin/dhconfig checkpoint list --file /db/Intraday/DbInternal/ProcessEventLog/some_partition/some_date/ProcessEventLog

/usr/illumon/latest/bin/dhconfig checkpoint list /db/Intraday/DbInternal/ProcessEventLog/some_partition/some_date/ProcessEventLog

/usr/illumon/latest/bin/dhconfig checkpoint list --file /tmp/any.file.name

Display all checkpoint files starting at a directory (recursively searching):

/usr/illumon/latest/bin/dhconfig checkpoint list --directory /db/Intraday/DbInternal/ProcessEventLog

/usr/illumon/latest/bin/dhconfig checkpoint list /db/Intraday/DbInternal/ProcessEventLog

Display a summary of all checkpoint files starting at a directory in CSV format:

/usr/illumon/latest/bin/dhconfig checkpoint list --directory /db/Intraday/DbInternal/ProcessEventLog --csv

Display a summary of all checkpoint files in a list file in CSV format:

find /db/Intraday/DbInternal/ProcessEventLog -name table.size > /tmp/list.txt
/usr/illumon/latest/bin/dhconfig checkpoint list --csv --list /tmp/list.txt

Examples with output:

$ /usr/illumon/latest/bin/dhconfig checkpoint list /db/Intraday/DbInternal/ProcessEventLog/db_query_server_qa-treasureplus-cluster-query-1_int_illumon_com/2022-07-08/ProcessEventLog/

Output:

$ /usr/illumon/latest/bin/dhconfig checkpoint list /db/Intraday/DbInternal/ProcessEventLog --csv --header

Checkpoint file '/db/Intraday/DbInternal/ProcessEventLog/db_query_server_qa-treasureplus-cluster-query-1_int_illumon_com/2022-07-08/ProcessEventLog/table.size':

CheckpointRecord[
	version=2,
	TableLocationState[size=7015, lastModificationTime=2022-07-08T17:32:28.691-0400],
	DataFileSizeRecords[
		[name=AuthenticatedUser.dat, size=28060],
		[name=AuthenticatedUser.sym, size=8],
		[name=AuthenticatedUser.sym.bytes, size=4],
		[name=EffectiveUser.dat, size=28060],
		[name=EffectiveUser.sym, size=8],
		[name=EffectiveUser.sym.bytes, size=4],
		[name=Host.dat, size=28060],
		[name=Host.sym, size=8],
		[name=Host.sym.bytes, size=46],
		[name=Level.dat, size=28060],
		[name=Level.sym, size=40],
		[name=Level.sym.bytes, size=24],
		[name=LogEntry.bytes, size=1107823],
		[name=LogEntry.dat, size=56120],
		[name=Process.dat, size=28060],
		[name=Process.sym, size=32],
		[name=Process.sym.bytes, size=32],
		[name=Timestamp.dat, size=56120]
	],
	SourceFileSizeRecord[name=DbInternal.ProcessEventLog.System.db_query_server_qa-treasureplus-cluster-query-1_int_illumon_com.2022-07-08.bin.2022-07-08.170000.000-0400, size=80605],
	ImportState[
		class=class com.illumon.iris.db.tables.dataimport.logtailer.ImportStateRowCounter,
		details=[
			[nRows, 7015]
		]
	]
]

Output:

filename, version, tableSize, lastModificationTime, lastModificationTimeStr, columnCount, sourceFile, sourceBytes
"/db/Intraday/DbInternal/ProcessEventLog/db_query_server_qa-treasureplus-cluster-query-1_int_illumon_com/2022-07-08/ProcessEventLog/table.size", 2, 33919, 1657316070141, 2022-07-08T21:34:30.141Z, "DbInternal.ProcessEventLog.System.db_query_server_qa-treasureplus-cluster-query-1_int_illumon_com.2022-07-08.bin.2022-07-08.210000.000+0000", 351464
"/db/Intraday/DbInternal/ProcessEventLog/db_merge_server_qa-treasureplus-cluster-infra-1_int_illumon_com/2022-07-08/ProcessEventLog/table.size", 2, 4575, 1657316016391, 2022-07-08T21:33:36.391Z, "DbInternal.ProcessEventLog.System.db_merge_server_qa-treasureplus-cluster-infra-1_int_illumon_com.2022-07-08.bin.2022-07-08.210000.000+0000", 3355
"/db/Intraday/DbInternal/ProcessEventLog/qa-treasureplus-cluster-infra-1_int_illumon_com/2022-07-08/ProcessEventLog/table.size", 2, 50146, 1657316070557, 2022-07-08T21:34:30.557Z, "DbInternal.ProcessEventLog.System.qa-treasureplus-cluster-infra-1_int_illumon_com.2022-07-08.bin.2022-07-08.210000.000+0000", 475383
"/db/Intraday/DbInternal/ProcessEventLog/qa-treasureplus-cluster-query-1_int_illumon_com/2022-07-08/ProcessEventLog/table.size", 2, 474510, 1657316070432, 2022-07-08T21:34:30.432Z, "DbInternal.ProcessEventLog.System.qa-treasureplus-cluster-query-1_int_illumon_com.2022-07-08.bin.2022-07-08.210000.000+0000", 2731761
"/db/Intraday/DbInternal/ProcessEventLog/db_query_server_qa-treasureplus-cluster-query-2_int_illumon_com/2022-07-08/ProcessEventLog/table.size", 2, 46555, 1657316070029, 2022-07-08T21:34:30.029Z, "DbInternal.ProcessEventLog.System.db_query_server_qa-treasureplus-cluster-query-2_int_illumon_com.2022-07-08.bin.2022-07-08.210000.000+0000", 925046
"/db/Intraday/DbInternal/ProcessEventLog/qa-treasureplus-cluster-query-2_int_illumon_com/2022-07-08/ProcessEventLog/table.size", 2, 51152, 1657316073318, 2022-07-08T21:34:33.318Z, "DbInternal.ProcessEventLog.System.qa-treasureplus-cluster-query-2_int_illumon_com.2022-07-08.bin.2022-07-08.210000.000+0000", 184474