Appendices

Appendix: Server hardware examples

The example servers come from a cross-section of other large data installations running Deephaven.

Infrastructure Server

The Infrastructure Server handles the authentication, acl and other tasks of running Deephaven. This server requires very few compute, disk or memory resources.

Hardware and Operating Systems

  • 6 cores @ 3.0 Ghz Intel
  • 10GB RAM
  • Linux - RedHat 7 or derivative
  • Network: 1g
  • Storage:
    • Root - sufficient local disk space

Intraday Import Server

The Data Import Server is mainly a write-centric server with access to fast storage. Typically, this will be an SSD or volumes of SSDs, depending on the data speed and storage size required.

Hardware and Operating Systems

  • 40 cores @ 3.0 Ghz Intel
  • 256GB RAM
  • Linux - RedHat 7 or derivative
  • Network: 10g mtu 8192
  • Storage:
    • Root - sufficient local disk space
    • Database mount - 250GB SSD mirror local
    • Intraday data - Sufficient space consisting of SSD either local or iScsi
    • Historical data - 16 NFS mounts @ 35TB each

Query Servers

Query servers provide compute, memory and query access to storage. They host the memory-intensive applications.

Hardware and Operating Systems

  • 40 cores @ 3.0 Ghz Intel
  • 512GB RAM
  • Linux - RedHat 7 or derivative
  • Network: 10g mtu 8192
  • Storage:
    • Root - sufficient local disk space
    • Database mount - 250GB SSD mirror local
    • Historical data - 16 NFS mounts @ 35TB each

Batch Import Servers

Import servers are similar to Intraday Import Servers, but are less write-intensive. They are used to perform large-batch imports of data, typically overnight.

Hardware and Operating Systems

  • 24 cores @ 3.0 Ghz Intel
  • 192GB RAM
  • Linux - RedHat 7 or derivative
  • Network: 10g mtu 8192
  • Storage:
    • Root - sufficient local disk space
    • Database mount - 250GB SSD mirror local
    • Intraday data - Sufficient space for the import via local, iScsi, nfs or other
    • Historical data - 16 NFS mounts @ 35TB each

Appendix: Methodologies for mounting historical NFS partitions

  • Historical data is stored in /db/Systems/<DATABASE_NAMESPACE>.
  • File read operations for a particular namespace are performed through the file systems mounted under /db/Systems/<DATABASE_NAMESPACE>/Partitions/0..N.
  • File write operations for a particular namespace are performed through the filesystems mounted under /db/Systems/<DATABASE_NAMESPACE>/WritablePartitions/0..N.
  • The two sets of directories can be on local disk, on a shared filesystem, an NFS volume or be a set of soft links to other directories
  • It should be noted that the WritablePartitions can be a subset of the Partitions directories if you only require a small amount of IO bandwidth

There are three methodologies for mounting NFS partitions. By default, Deephaven uses the Partial Indirect Mounting option that links the WritablePartitions as a subset of the NFS volumes mounted under each Partitions directory.

In large deployments, customers often use the Indirect Mounting option to allow for maximum flexibility. By soft linking both the Partitions and the WritablePartitions to the mount point of the NFS volume. This allows changing the NFS mounts to which Deephaven is writing data by simply removing a soft link or changing the soft link to point to another NFS mount.

Historical data methodologies

There are three options for creating historical data directories and mounts.

Direct Mounting

Direct Mounting should only be used in small data installations where only a handful of mounts are required.

Adding additional storage would require creating similarly numbered directories and mounting more NFS volumes.

As an example, the namespace-0 NFS volume is mounted directly to the Partitions and WritablePartitions sub-directories:

sudo mount -t nfs -o ro <NFSSERVER_ADDRESS>:/namespace-0 /db/Systems/DbInternal/Partitions/0

sudo mount -t nfs <NFSSERVER_ADDRESS>:/namespace-0 /db/Systems/DbInternal/WritablePartitions/0

If additional storage beyond this is required, a second partition namespace-1 would be created.

Two more mount points would be created and the new NFS volume mounted under each:

sudo mkdir -p /db/Systems/<NAMESPACE>/Partitions/1

sudo mkdir -p /db/Systems/<NAMESPACE>/WritablePartitions/1

sudo mount -t nfs -o ro <NFSSERVER_ADDRESS>:/namespace-1 /db/Systems/<NAMESPACE>/Partitions/1

sudo mount -t nfs <NFSSERVER_ADDRESS>:/namespace-1 /db/Systems/<NAMESPACE>/WritablePartitions/1

Additional partitions would be added in this manner as the system expands.

Partial Indirect Mounting

With Partial Indirect Mounting, each NFS volume is only mounted once under /db/Systems/<NAMESPACE>/Partitions/0..N. The /db/Systems/<NAMESPACE>/WritablePartitions/0..N are then soft linked to their counterparts under /db/Systems/<NAMESPACE>/Partitions/.

sudo mount -t nfs <NFSSERVER_ADDRESS>:/namespace-0 /db/Systems/<NAMESPACE>/Partitions/0

sudo ln -s /db/Systems/<NAMESPACE>/Partitions/0 /db/Systems/<NAMESPACE>/WritablePartitions/0

Additional file systems would be added under /db/Systems/<NAMESPACE>/Partitions/ and then soft links would be created under the WritablePartitions directory.

Full Indirect Mounting

The most flexible methodology, Full Indirect Mounting, allows each individual mount to be customized, or moved with minimal impact to Deephaven. By soft linking, both /db/Systems/<DATABASE_NAMESPACE>/Partitions/0..N and /db/Systems/<DATABASE_NAMESPACE>/WritablePartitions/0..N the underlying NFS file systems can be switched out by changing a link, or removed from service by deleting the link altogether.

The namespace-0 NFS volume is mounted elsewhere on the server and soft links are created to both the Partitions and WritablePartitions directory.

sudo mkdir /srv/<NAMESPACE>/0

sudo mkdir /srv/<NAMESPACE>/read-only/0

sudo mount -t nfs <NFSSERVER_ADDRESS>:/namespace-0 /srv/<NAMESPACE>/0

sudo mount -t nfs -o ro <NFSSERVER_ADDRESS>:/namespace-0 /srv/<NAMESPACE>/read-only/0

sudo ln -s /srv/<NAMESPACE>/0 /db/Systems/<NAMESPACE>/WritablePartitions/0

sudo ln -s /srv/<NAMESPACE>/read-only/0 /db/Systems/<NAMESPACE>/Partitions/0

Adding additional storage or IO bandwidth is mounted under /srv/<NAMESPACE> directories, and new soft links are created for Partitions or WritablePartitions.