Performing a thread dump

There are times during incident management when the Deephaven Support team may ask you to perform a Thread Dump of a particular Deephaven process. This does not impact the system in any way, but records valuable diagnostic information into the logs to be used in troubleshooting.

Performing a Thread Dump with jstack

To perform a Thread Dump, simply run:

sudo jstack -F <pid> > <file-path>
  • pid is the Process Id of the Deephaven process.
  • file-path is the file path where the thread dump will be written.

For example, to perform a Thread Dump of Worker 1, use ps, grep, and awk to capture the process id:

sudo jstack -F $(ps -ef | grep worker_1 | awk '{print $2}' | head -n 1) > /tmp/threadDumpWorker1.txt

Performing a Thread Dump with kill -3 (SIGQUIT)

In cases where you want the thread dump to go to the process logs, kill -3 can be used and the thread dump will be sent to the standard output stream of the process.

To perform a Thread Dump with kill -3, simply run:

sudo kill -3 <pid>
  • pid is the Process Id of the Deephaven process.

For example, to perform a Thread Dump of Worker 1, use ps, grep, and awk to capture the process id:

sudo kill -3 $(ps -ef | grep worker_1 | awk '{print $2}' | head -n 1)

By default, Worker logs are sent to the Deephaven database along with stderr and stdout, where stderr is redirected to the stdout for the process.

To view stdout logs for Worker 1, the following query can be used:

t=db.i("DbInternal",  "ProcessEventLog").where("Date=`" + today + "`").where("Process=`worker_1`").where("Level=`STDOUT`")