Recipes, not loops!

If you're coming from pandas, traditional Python, or other data processing tools, you're likely accustomed to writing loops to transform data. Stop! Deephaven works fundamentally differently, and understanding this difference early will save you countless hours of frustration and help you write better, faster code.

The fundamental paradigm shift: recipes, not loops

How you might be thinking

In pandas or traditional Python, you tell the computer exactly how to process each row:

# A loop applied to data — common habit, but wrong for Deephaven!
values = [0, 1, 2, 3, 4]
values_squared = [v * v for v in values]

This loop processes one element at a time and builds a new list. You're giving step-by-step instructions for how to process the data.

How to think in Deephaven

In Deephaven, you specify what you want, not how to compute it. You write a recipe that describes the transformation, and the Deephaven engine figures out the optimal way to execute it:

from deephaven import empty_table

# No loop — just describe what you want
result = empty_table(5).update(["X = i", "XSquared = X * X"])

Notice:

No loops — you describe the relationship (XSquared = X * X) and the engine applies it to every row.
You specify what to compute, not how to iterate.
The engine applies this recipe to all rows automatically.
This works the same way for arbitrarily complex operations.

Why this matters

For static data

Even for static, one-time calculations, the recipe approach has advantages:

Clearer code - Declarative recipes are easier to read than imperative loops.
Faster execution - The engine can optimize vectorized operations.
Less error-prone - No manual loop management or index tracking.

For real-time data

This is the critical difference. Loops execute once and stop. Recipes update automatically.

from deephaven import time_table

# This table adds a new row every second
source = time_table("PT1s").update(["X = i", "XSquared = X * X", "XCubed = X * X * X"])

Watch what happens:

The table keeps updating - new rows appear every second.
Your recipe runs automatically on every new row.
You wrote it once, but it executes forever.

With a loop approach:

# This would only work ONCE and never update!
for row in source.iter_tuple():
    x = row.X
    x_squared = x * x  # ❌ Where would this even go?

The recipe paradigm explained

Recipes are specifications, not instructions

When you write:

t.update("Y = X * 2")

You're not saying:

"Start at row 0"
"Read X from row 0"
"Multiply by 2"
"Store in Y at row 0"
"Go to row 1"
"Repeat..."

You're saying:

"For every row, Y should equal X times 2"

The engine decides:

How to chunk the data for optimal performance.
Whether to parallelize the operation.
How to handle updates efficiently.
What rows need recomputation when data changes.

The engine is smart about updates

Tracks dependencies - It knows that Y depends on X.
Computes incrementally - Only new or changed rows are processed.
Updates automatically - Results update without you doing anything.

This requires significant additional infrastructure with loops — a loop executes once and stops, so you would need to build your own subscription and recomputation logic.

Bridging pandas and Deephaven

Many users need to work with both pandas and Deephaven. Here's how to think about the transition:

import pandas as pd
import deephaven.pandas as dhpd

# Create data in pandas
time_index = pd.date_range(start="2025-01-01 00:00:00", periods=5, freq="h")
df = pd.DataFrame(
    {
        "time": time_index,
        "value": range(5),
    }
)

print("Original pandas DataFrame:")
print(df)

# Convert to Deephaven
t1 = dhpd.to_table(df)

# Check the column types
m = t1.meta_table

# Now use Deephaven recipes (NOT loops!)
t2 = t1.update("TsEpochNs = epochNanos(time)")

# More time operations using recipes
t3 = t2.update(
    [
        "TS3 = epochNanosToInstant(TsEpochNs + 2*SECOND)",
        "TS4 = time + 'PT2s'",
        "D3 = TS3-time",
        "D4 = TS4-time",
    ]
)

# Convert back to pandas if needed
df2 = dhpd.to_pandas(t3)
print("Result DataFrame:")
print(df2)

Key principle: Once you're in Deephaven, think in recipes. Save loops for when you convert back to pandas.

When loops ARE appropriate

There are valid uses for loops in Deephaven:

✅ Extracting data from Deephaven

from deephaven import empty_table

source = empty_table(5).update(["X = i", "Y = X * 2"])

# This is fine - you're extracting, not transforming
for row in source.iter_tuple():
    print(f"X={row.X}, Y={row.Y}")

See the table iteration guide for details.

✅ Control flow in your Python code

from deephaven import empty_table

source = empty_table(100).update(
    ["Symbol = (i % 3 == 0) ? `AAPL` : (i % 3 == 1) ? `GOOGL` : `MSFT`", "X = i"]
)

# Creating multiple similar tables - fine!
tables = []
for symbol in ["AAPL", "GOOGL", "MSFT"]:
    t = source.where(f"Symbol = `{symbol}`")
    tables.append(t)

❌ Transforming table columns

from deephaven import empty_table

source = empty_table(10).update("X = i")

# NEVER do this!
result_data = []
for row in source.iter_tuple():
    result_data.append(row.X * 2)  # ❌ Use .update() instead!

Common patterns: Wrong vs Right

Pattern: Create a column from another column

❌ Wrong (loop approach):

from deephaven import empty_table

source = empty_table(10).update("X = i")

# Don't do this!
values = []
for row in source.iter_tuple():
    values.append(row.X * row.X)
# Now what? How do you get this back into a table?

✅ Right (recipe approach):

from deephaven import empty_table

result = empty_table(10).update(["X = i", "XSquared = X * X"])

Pattern: Conditional logic

❌ Wrong (loop approach):

from deephaven import empty_table

source = empty_table(10).update("X = i")

# Don't do this!
results = []
for row in source.iter_tuple():
    if row.X % 2 == 0:
        results.append("Even")
    else:
        results.append("Odd")

✅ Right (recipe with ternary operator):

from deephaven import empty_table

result = empty_table(10).update(["X = i", "Label = (X % 2 == 0) ? `Even` : `Odd`"])

Pattern: Running calculations

❌ Wrong (loop with accumulator):

from deephaven import empty_table

source = empty_table(10).update("X = i")

# Don't do this!
running_sum = 0
results = []
for row in source.iter_tuple():
    running_sum += row.X
    results.append(running_sum)

✅ Right (use update_by):

from deephaven import empty_table
from deephaven.updateby import cum_sum

result = empty_table(10).update("X = i").update_by(cum_sum("SumX = X"))

Quick reference: Migration guide

pandas/Python Pattern	Deephaven Recipe
`df.apply(func)`	`.update("Y = func(X)")`
`for row in df.iterrows():`	❌ Don't! Use `.update()`
`df['Y'] = df['X'] * 2`	`.update("Y = X * 2")`
`df[df['X'] > 5]`	`.where("X > 5")`
`df.rolling(window=10).mean()`	`.update_by(rolling_avg_tick(...))`
`df.groupby('G').sum()`	`.sum_by("G")`

Next steps

Read Think like a ninja for more examples.
Learn about table operations to see recipes in action.
Understand the query engine for technical details.
See update_by operations for powerful recipes.