Version: Python

Use jpy

jpy is a bi-directional Java-Python bridge that facilitates calling Java from Python and vice versa.

Python programs that use jpy can access Java functionalities.
Java programs that use jpy can access Python functionalities.

For more details on jpy, see the jpy GitHub project, or use help("jpy") to see the package documentation.

The Deephaven query engine is implemented in Java, making it relatively easy to use Java from within Python. jpy is used as the bridge between the two languages. This guide covers how to use Java from Python. Calling Python from Java is much less common and won't be covered here.

caution

jpy is a lower-level tool and is generally only used when Deephaven does not provide Python wrappings for Java objects to accomplish the same tasks. Many of the things done in this document would be better accomplished with the deephaven.dtypes Python module, which supports the creation and manipulation of Java primitives, arrays, and many different Java object types. However, much of this document will demonstrate jpy's functionality in the most straightforward cases, with primitive types and simple objects. This is to explain jpy's usage rather than encourage its use in all of these cases.

note

To request a Python wrapping of a particular Java type or library, file a ticket in the Deephaven Core GitHub repository.

Get Java types

One of the primary functions of jpy is to retrieve Java types from Python. This is useful when the desired types do not have an existing Python wrapping. jpy provides the ability to get Java types with jpy.get_type.

Use `jpy.get_type`

To get a Java type from Python, pass the fully qualified class name of the desired type as a string argument to jpy.get_type. Here's a simple example using java.lang.String.

import jpy

# use get_type to get a Python wrapping of a java.lang.String
_JString = jpy.get_type("java.lang.String")

jpy.get_type can get any valid Java type that is present in the Java classpath, including Deephaven-specific types. Here is an example using Deephaven's DoubleVector.

# get the Deephaven representation of a vector of doubles
_JDoubleVector = jpy.get_type("io.deephaven.vector.DoubleVector")

What does `get_type` return?

When jpy.get_type is called, it returns a unique Python class that wraps the underlying Java type. You can inspect this class with familiar Python functions.

import jpy

_JURL = jpy.get_type("java.net.URL")

# the print function is overloaded to print the classpath of the underlying Java type
print(_JURL)
# the class gets recognized as a Python type, confirming that this is a Python object
print(type(_JURL))

When this class is created, it defines methods that mirror those of the underlying Java type. Use Python's help function to print out all the available methods.

# print all of the methods with help
help(_JURL)

Check the Javadoc for java.net.URL to confirm that all of its methods are represented by the Python wrapping.

Call methods on Java types

Python code can use Java methods directly. This means you can call any Java function from your Python program.

Call static methods

Static methods are methods that belong to the type rather than to an instance of the type. Static methods can be called directly from the return value of jpy.get_type. This example uses the valueOf static method from the java.lang.String type.

import jpy

_JString = jpy.get_type("java.lang.String")

# the valueOf method is static, so it can be called directly from the type
int_as_string = _JString.valueOf(123)

# verify that the returned object is really a string
print(int_as_string)
print(type(int_as_string))

Create objects and call instance methods

jpy.get_type can be used to create instances of Java data types. The code block below creates an instance of a java.lang.Float.

import jpy

_JFloat = jpy.get_type("java.lang.Float")

# calling the type like a function with an appropriate argument invokes its constructor
float_instance = _JFloat(123.456)

Unlike static methods, instance methods do require data stored in a type to perform their function. Therefore, they belong to the instance, rather than the type itself.

float_instance is an object and provides many instance methods for operating on the value 123.456. The Javadoc for java.lang.Float details all of the available methods. This example uses the toString, compareTo, and equals instance methods.

# the toString method returns the float represented as a string
float_instance_as_string = float_instance.toString()
print(float_instance_as_string)
print(type(float_instance_as_string))

# the compareTo method returns -1 if float is less than the argument, 0 if equal, and 1 if greater
less_than_or_grearter_than = float_instance.compareTo(654.321)
print(less_than_or_grearter_than)
print(type(less_than_or_grearter_than))

# the equals method returns True if the float is equal to the argument, and False otherwise
are_floats_equal = float_instance.equals(123.456)
print(are_floats_equal)
print(type(are_floats_equal))

Return type conversion

When Java methods are called through jpy, jpy attempts to convert the return values to Python types if possible. Here is an example of this behavior.

import jpy

# import java math library
_JMath = jpy.get_type("java.lang.Math")

# generate random float with random method
float_return = _JMath.random()

# verify that the returned type is correct
print(type(float_return))

When the return type of a Java method is a Java object that cannot be directly converted to Python, jpy wraps the result so that it can be used in Python.

import jpy

# get Deephaven date-time utils
_JTime = jpy.get_type("io.deephaven.time.DateTimeUtils")

# create a Java Instant using parseInstant
java_instant = _JTime.parseInstant("2023-06-30T17:00:00Z")

# the return type is a Java instant, so it is returned in a Python wrapping
print(type(java_instant))

Argument type conversion

In the previous examples, Java methods were called with Python objects as arguments. For this to work, jpy auto-converts the Python objects to Java objects with the appropriate types. Here is another example of this conversion.

import jpy

_JInteger = jpy.get_type("java.lang.Integer")

# construct a java.lang.Integer - the constructor is called with a Python object
int_instance = _JInteger(4)

# call compareTo instance method - also called with a Python object
int_compare = int_instance.compareTo(5)

Here, 4 and 5 are Python objects, so jpy must convert them to Java objects before calling methods.

Generally, this kind of auto-conversion is only possible with primitive types. To call a Java method with more complex argument types, objects of those types must first be created with jpy.

Here is an example of calling a method with a Java object argument. The dayOfWeek method from Deephaven's DateTimeUtils accepts a Java LocalDate as an argument. jpy cannot auto-convert any Python object to a Java LocalDate, so use jpy to create an instance of that type directly.

import jpy

# want to use the dayOfWeek static method from DateTimeUtils
_JTime = jpy.get_type("io.deephaven.time.DateTimeUtils")

# create LocalDate argument to pass to dayOfWeek method and get result
j_local_date = _JTime.parseLocalDate("2021-05-04")
j_day_of_week = _JTime.dayOfWeek(j_local_date)

note

This example is only illustrative, and this particular use case is best served by deephaven.time.

Method overloading

Java (unlike Python) allows method overloading, where multiple methods can have the same name but different signatures.

This example calls Java's max method on two different sets of inputs. When the inputs are integers, max(int, int) is invoked. When the inputs are floats, max(float, float) is invoked.

import jpy

# get the Java math library
_JMath = jpy.get_type("java.lang.Math")

# create an int and float representations of 5 and 7
x_int, y_int = int(5), int(7)
x_float, y_float = float(5), float(7)

# call overloaded 'max' method on both types
max_int = _JMath.max(x_int, y_int)
max_float = _JMath.max(x_float, y_float)

# the int overload returns an int
print(type(max_int))
# and the float overload returns a float
print(type(max_float))

The same concept applies when both Python and Java objects are passed in as inputs. Here's an example that calls two overloads of the minus method with different combinations of Python and Java arguments.

import jpy

# want to use the minus method to get time differences
_JTime = jpy.get_type("io.deephaven.time.DateTimeUtils")

# create the initial instant and two different types representing 12 hours
initial_instant = _JTime.parseInstant("2024-02-01T12:30:00Z")
diff_nanos = int(10e8 * 60 * 60 * 12)  # 12 hours in nanoseconds as a long
diff_duration = _JTime.parseDuration("PT12h")

# invoke two overloads - note that the first involves a Python conversion
res_1 = _JTime.minus(initial_instant, diff_nanos)
res_2 = _JTime.minus(initial_instant, diff_duration)

# verify that results are the same
print(res_1)
print(res_2)

Common errors

If jpy cannot find an overload matching the argument types provided, a "no matching Java method overloads found" error will be raised:

Type: <class 'RuntimeError'>
Value: no matching Java method overloads found

This error may arise from arguments having mismatched types, calling a method with the wrong number of arguments, or errors in auto-converting from a Python type. Here are examples of each scenario.

import jpy

_JMath = jpy.get_type("java.lang.Math")

# addExact takes two arguments, but is only given one
too_few_args = _JMath.addExact(5)

# hypot takes two arguments, but is given three
too_many_args = _JMath.hypot(3, 4, 5)

# abs has overloads for int, long, float, and double, not string
wrong_type_given = _JMath.abs("hello!")

# the most subtle error
# incrementExact only accepts an int, but a float cannot be auto-converted to an int
wrong_type_converted = _JMath.incrementExact(5.0)

An "ambiguous Java method call" error is thrown if the argument type is too ambiguous to select a particular overload:

Type: <class 'RuntimeError'>
Value: ambiguous Java method call, too many matching method overloads found

This can happen when passing non-native Python types, such as numpy types, to methods with multiple overloads. Here is an example of such behavior.

import numpy as np
import jpy

_JMath = jpy.get_type("java.lang.Math")

# the operation is defined, but jpy does not know which type to infer
ambiguous_types = _JMath.min(np.int32(4), np.int32(5))

Create Java arrays

jpy.array creates Java arrays directly from Python iterables of the appropriate types or an integer indicating the new array length.

Scalar arrays

To create arrays of scalars with jpy.array, pass in the name of the type and a Python iterable containing elements that can be converted to the target type.

import numpy as np
import jpy

# any Python iterable can be used, like tuples...
int_array = jpy.array("int", (1, 2, 3, 4, 5))

# lists...
bool_array = jpy.array("boolean", [True, False, True, False])

# or numpy arrays
double_array = jpy.array("double", np.array([4.5, 5.6, 6.7]))

Arrays can also be specified by providing the number of elements rather than a Python iterable. This yields an array containing all zeros for primitives or nulls for objects.

# creates an unitialized byte array of 100 elements
byte_array_100_elements = jpy.array("byte", 100)

If the Python iterable contains elements that cannot be converted to the requested Java type, the conversion will fail.

# 3.3 cannot be converted to an int
failed_int_array = jpy.array("int", [1, 2, 3.3, 4, 5])

jpy.array can also be used to create arrays of non-primitive types, such as java.lang.String and java.lang.Float. The fully-qualified class name must be specified:

import jpy

string_array = jpy.array("java.lang.String", ["I", "am", "a", "string!"])
float_array = jpy.array("java.lang.Float", [1.2, 2.3, 3.4, 4.5])

Nested arrays

Multi-dimensional arrays, or nested arrays, are common in Java. jpy can be used to create nested arrays by using nested calls to jpy.array. However, the classpath names do not work as they did previously. Consider the following example that attempts to create a nested array of integers.

# this won't work!
nested_int_array = jpy.array(
    "int",
    [  # this is the 'outer' array
        jpy.array("int", [1, 2, 3]),  # these are the 'inner' arrays
        jpy.array("int", [4, 5, 6]),
    ],
)

This code will fail because the outer array's specified type is an int but should be an array of ints. Here are the rules for specifying nested array types, each followed by simple examples.

For primitive types, use the type signature. The type signatures for each primitive type are:
Primitive Type Type Signature
boolean Z
byte B
char C
short S
int I
long J
float F
double D

Primitive Type	Type Signature
`boolean`	`Z`
`byte`	`B`
`char`	`C`
`short`	`S`
`int`	`I`
`long`	`J`
`float`	`F`
`double`	`D`

simple_primitive_array = jpy.array("[I", [jpy.array("int", [1])])

For object types, use the full class name, prepended with L and ending with ;.

simple_object_array = jpy.array(
    "[Ljava.lang.String;", [jpy.array("java.lang.String", ["Hello!"])]
)

The number of open square brackets determines the array's dimension. An n-dimensional array requires n-1 open square brackets.

simple_primitive_array_3d = jpy.array(
    "[[D", [jpy.array("[D", [jpy.array("double", [1.234])])]
)

See the Java documentation for more information on type signatures.

To clarify these rules, here are some concrete examples.

Example 1: 2D `boolean` array

This example creates a 2D array of boolean values. Since it is a nested array of primitives, the type signature Z for the boolean type must be used. It is a 2D array, so the type signature must be prepended by a single open square bracket [.

bool_array_2d = jpy.array(
    "[Z",
    [
        jpy.array("boolean", [True, False, True, False]),
        jpy.array("boolean", [False, True, False]),
    ],
)

Example 2: 3D `int` array

This example creates a 3D array of integer values. The int is primitive, so the type signature I must be used, along with two open square brackets since the array is three-dimensional.

int_array_3d = jpy.array(
    "[[I",
    [
        jpy.array(
            "[I",
            [
                jpy.array("int", [1, 2, 3, 4, 5]),
                jpy.array("int", [6, 7, 8]),
            ],
        ),
        jpy.array(
            "[I",
            [
                jpy.array("int", [9, 10, 11, 12]),
                jpy.array("int", [13, 14]),
            ],
        ),
    ],
)

Example 3: 2D `java.lang.String` array

This example creates a 2D array of Strings. Since this is a nested array of objects, the class name must be prepended with an L and end with a ;. Since this array is 2D, only one open square bracket is required.

string_array_nested = jpy.array(
    "[Ljava.lang.String;",
    [
        jpy.array("java.lang.String", ["I", "am", "a", "string!"]),
        jpy.array("java.lang.String", ["I", "am", "also" "a", "string!"]),
    ],
)

Example 4: 3D `java.lang.String` array

Finally, this example creates a 3D array of Strings. This will require L and ;, as well as two open square brackets.

string_array_nested_deeper = jpy.array(
    "[[Ljava.lang.String;",
    [
        jpy.array(
            "[Ljava.lang.String;",
            [
                jpy.array("java.lang.String", ["I", "am", "a", "string!"]),
                jpy.array("java.lang.String", ["I", "am", "also" "a", "string!"]),
            ],
        ),
        jpy.array(
            "[Ljava.lang.String;",
            [
                jpy.array("java.lang.String", ["I", "am", "a", "string", "too!"]),
                jpy.array(
                    "java.lang.String", ["I", "guess", "I", "am", "a", "string!"]
                ),
            ],
        ),
    ],
)

Use `jpy` with Deephaven

Because the Deephaven query engine is implemented in Java, all of its core data structures and operations are implemented in Java. Deephaven makes it easy to use Java objects directly with jpy.

The `j_object` property

To access the Java objects underlying Deephaven's Python API, use the j_object property.

from deephaven import empty_table

# create a new table
t = empty_table(10).update("X = ii")

# get the underlying Java object with j_object
t_j_object = t.j_object

t
t_j_object

Use Java libraries with Deephaven tables

Many use cases for jpy involve operating on Deephaven tables with Java libraries or methods that have not been wrapped in Python. Use jpy.get_type to get the desired class. Then, pass Deephaven tables to Java methods with the j_object property.

from deephaven import empty_table
import jpy

t = empty_table(5).update(["X1 = i", "X2 = X1 + 2", "X3 = X1 + 3", "X4 = X1 + 4"])

# get the TableTools type
_TableTools = jpy.get_type("io.deephaven.engine.util.TableTools")

# use the show method to print the names of columns in the table, note the j_object in the argument
_TableTools.show(t.j_object)

Deephaven libraries often return new Deephaven tables. These Deephaven tables are Java objects. To make them usable with the rest of the Python API, they must be explicitly wrapped in Python using the Table constructor from deephaven.table.

from deephaven.table import Table

new_t = empty_table(3).update(["X1 = 10", "X2 = 20", "X3 = 30", "X4 = 40"])

# merge tables with TableTools.merge - this is illustriative, as merge is wrapped in Python
# _j_merged_t is a Java Deephaven table, prepended with _j by convention
_j_merged_t = _TableTools.merge(t.j_object, new_t.j_object)

# wrap the result in a Python Deephaven table
merged_t = Table(_j_merged_t)

new_t
merged_t

Use Java values in query strings

Java objects created with jpy can be used directly in Deephaven's query strings.

import jpy
from deephaven import empty_table

# create a string
_JString = jpy.get_type("java.lang.String")
string_instance = _JString("I am a string!")

t = empty_table(10).update("StringCol = string_instance")

# check the metadata of the table, confirming that the column is understood as a string
t_meta = t.meta_table

t_meta
t

When a Java object is constructed, its methods can be called inside and outside query strings, depending on the use case.

import jpy
from deephaven import empty_table

# get URL type and create instance
_JUrl = jpy.get_type("java.net.URL")
url_instance = _JUrl("https://deephaven.io")

# call the methods outide of the query strings
protocol = url_instance.getProtocol()
host = url_instance.getHost()
port = url_instance.getPort()
uri = url_instance.toURI()

outside_calls = empty_table(1).update(
    [
        "URL = url_instance",
        "Protocol = protocol",
        "Host = host",
        "Port = port",
        "URI = uri",
    ]
)

# alternatively, call the methods inside of the query strings
inside_calls = empty_table(1).update(
    [
        "URL = url_instance",
        "Protocol = URL.getProtocol()",
        "Host = URL.getHost()",
        "Port = URL.getPort()",
        "URI = URL.toURI()",
    ]
)

outside_calls
inside_calls

When Java values are used directly in query strings, they do not require any Python-Java boundary crossings during their execution. Thus, such patterns will be very performant relative to other patterns that require boundary crossings. Here is an example that compares the efficiency of generating random numbers using a Python or a Java approach.

import time
from deephaven import empty_table

### Python approach
import random
from numba import jit

random.seed(12345)


@jit
def generate_gaussian() -> float:
    return random.gauss(0.0, 1.0)


python_start = time.time()
python_gauss = empty_table(10_000_000).update("X = generate_gaussian()")
python_end = time.time()

### Java approach
import jpy

_Rng = jpy.get_type("java.util.Random")
rng_instance = _Rng(12345)

java_start = time.time()
java_gauss = empty_table(10_000_000).update("X = rng_instance.nextGaussian()")
java_end = time.time()

### evaluate time difference
print("Python execution time: ", python_end - python_start)
print("Java execution time: ", java_end - java_start)

Log
python_gauss
java_gauss

Tie it all together

Finally, here's an example that ties all of these concepts together.

import jpy
from deephaven import empty_table
from deephaven.table import Table

# create table where column names are 'labels', and entries are 'values'
t = empty_table(3).update(["Value1 = ii", "Value2 = ii + 3"])

# get the ColumnsToRowsTransform to flip this table's rows and columns
_ColsToRowsTransform = jpy.get_type(
    "io.deephaven.engine.table.impl.util.ColumnsToRowsTransform"
)

# call the static columnsToRows method
_j_transformed_t = _ColsToRowsTransform.columnsToRows(
    # must use j_object to get the underlying Java object, otherwise method will not be found
    t.j_object,
    # labelColumn and valueColumn are String arguments, which jpy knows how to convert
    "LabelCol",
    "ValueCol",
    # labels and transposeColumns are String[] arguments, which jpy does not know how to convert, so use jpy.array
    jpy.array("java.lang.String", ["Value1", "Value2"]),
    jpy.array("java.lang.String", ["Value1", "Value2"]),
)

# transform return value to Python value, so that Deephaven Python API can be used
transformed_t = Table(_j_transformed_t)

transformed_t
t

Get Java types​

Use jpy.get_type​

What does get_type return?​

Call methods on Java types​

Call static methods​

Create objects and call instance methods​

Return type conversion​

Argument type conversion​

Method overloading​

Common errors​

Create Java arrays​

Scalar arrays​

Nested arrays​

Example 1: 2D boolean array​

Example 2: 3D int array​

Example 3: 2D java.lang.String array​

Example 4: 3D java.lang.String array​

Use jpy with Deephaven​

The j_object property​

Use Java libraries with Deephaven tables​

Use Java values in query strings​

Tie it all together​

Related documentation​