Modularizing Queries (Python)

Modularization (breaking code into loosely coupled, self-contained pieces) is a common practice in software development. It should be used to improve the organization and readability of Deephaven queries in both Python and Groovy. This guide covers best practices for importing Deephaven scripts into Core+ queries from both notebooks and Git repositories.

In Python, you can import notebook scripts with the deephaven_enterprise.notebook package, or Git repository scripts with the deephaven_enterprise.controller_import package.

Create a notebook to import

Create a new notebook in the File Explorer and add the following code:

def my_method(x) -> int:
    return x + 1


class MyClass:
    def __init__(self, x):
        self.x = x

    def my_class_method(self) -> int:
        return self.x + 1

Save the notebook as my_notebook.py.

Import the notebook

In Python, you import other scripts by adding their filepath to the Python path. However, Deephaven Core+ workers have the deephaven_enterprise.notebook package as a more secure and convenient alternative.

Importing a Deephaven notebook only requires the Deephaven database object (db). By default, once imported, meta_import makes Deephaven notebooks available in the notebook package:

from deephaven_enterprise.notebook import meta_import

meta_import(db)

# Import the whole package
import notebook.my_notebook

# Import specific variables/classes/methods
from notebook.my_notebook import my_method, MyClass

You may optionally change the module name that prefixes all web notebooks. The following example uses the mynb prefix:

from deephaven_enterprise.notebook import meta_import

meta_import(db, "mynb")

# Import the whole package
import mynb.my_notebook

# Import specific variables/classes/methods
from mynb.my_notebook import my_method, MyClass

Packages imported this way are used the same as any other Python package. For more on Python packages in table operations, see Python packages in Deephaven and the Python-Java boundary.

Execute a notebook

In Python, you can also execute any other Deephaven notebook found in the File Explorer via exec_notebook. In order to do so, you must specify:

  • The Deephaven database object (db).
  • The name of the notebook, including any path prefixes. This should always be prefixed by a /.
  • The Python directory of global variables. In most cases, this is globals().

The following example attempts to execute the notebook my_notebook.py:

from deephaven_enterprise.notebook import exec_notebook

exec_notebook(db, "/my_notebook.py", globals())

The following example executes my_other_notebook.py, which is found in the notebooks subdirectory:

from deephaven_enterprise.notebook import exec_notebook

exec_notebook(db, "/notebooks/my_other_notebook.py", globals())

Import from Git repositories

Importing scripts from an integrated Git repository uses a different approach than notebooks. Unlike notebook imports, controller imports:

  • Use the deephaven_enterprise.controller_import package instead of deephaven_enterprise.notebook.
  • Do not require passing a database object.
  • Use controller as the default module prefix instead of notebook.
  • Don't necessarily require script package names to match every directory.

For example, if the iris.scripts.repo.<repo>.paths property is set:

iris.scripts.repo.<repo>.paths=module/deephaven

The package found at module/deephaven/com/example/compute will be imported as com.example.compute, leaving out the module/deephaven prefix:

import deephaven_enterprise.controller_import

deephaven_enterprise.controller_import.meta_import()

# Import the whole package
import controller.com.example.compute as compute

# Import specific variables/classes/methods
from controller.com.example.compute import my_method, MyClass

You can optionally change the module prefix from the default controller. The following example uses the git prefix:

import deephaven_enterprise.controller_import

deephaven_enterprise.controller_import.meta_import("git")

# Import with custom prefix
import git.com.example.compute as compute
from git.com.example.compute import my_method, MyClass

To execute a single Python file from the controller without importing it as a module:

import deephaven_enterprise.controller_import

deephaven_enterprise.controller_import.exec_script("com/example/script_to_execute.py")