Syntax errors (Groovy)
Deephaven Persistent Queries are programs or "scripts" written in either Python or Groovy. As with any programming language, the interpreter has specific syntax rules around quotes, valid tokens, and sometimes even whitespace. Within your Deephaven query, the select, update, updateView, view, where, and other operations use Formulas. The Deephaven engine interprets formulas to produce efficient Java code; however, formulas may also have syntax errors.
A treatment of Groovy syntax is beyond the scope of this guide; however, inevitably, your queries are going to have a missing brace or an unknown variable. Diagnosing a syntax error revolves around reading the stack trace. For example, the following notebook assigns a value to the variable named val but then references value:
val = 2
if (value > 1) {
print(value)
} else {
print("value is too small")
}
When executed from a Code Studio or Notebook, the snippet produces the following stack trace:
r-Scheduler-Serial-1 | .c.ConsoleServiceGrpcImpl | Error running script: groovy.lang.MissingPropertyException: No such property: value for class: io.deephaven.dynamic.Script_7
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:65)
at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:51)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:341)
at io.deephaven.dynamic.Script_7.run(Script_7.groovy:4)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:427)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:461)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:436)
...
Each command entered into a console or the script of a Persistent Query is assigned a unique class name by the Groovy interpreter, in this case io.deephaven.dynamic.Script_7. The line number referenced is 4, but the line numbers in Groovy stack traces are inaccurate. In this case, the actual error is on the third line of the script.
Similarly, Groovy can raise syntax errors for problems such as missing braces, as in the example below:
value = 2
if (value > 1) {
print(value)
} else
print("value is too small")
}
This results in the following stack trace:
r-Scheduler-Serial-1 | .c.ConsoleServiceGrpcImpl | Error running script: io.deephaven.engine.util.GroovyExceptionWrapper: startup failed:
Script_8: 8: Unexpected input: '("value is too small")\n}' @ line 8, column 1.
}
^
1 error
Again, the line numbers are shifted by 1, but the stack trace includes a pointer to the ending brace }, which is the unexpected token. To correct this error, the missing brace on line 5 must be added or the brace on line 7 removed. Using an editor with syntax highlighting like the Deephaven Code Studio or IntelliJ IDEA can help identify these kinds of errors.
Formula syntax errors
Deephaven formulas and filters use a Java-like syntax to express operations on table columns, variables from your session, and globally imported functions. As in any programming language, proper syntax is necessary. In the following example, the Groovy syntax is correct, but the closing parenthesis inside the formula is missing:
x = emptyTable(1).update("X=2 * (3 + 4")
This results in a stack trace of:
1 r-Scheduler-Serial-1 | .c.ConsoleServiceGrpcImpl | Error running script: io.deephaven.engine.table.impl.select.FormulaCompilationException: Formula compilation error for: 2 * (3 + 4
2 at io.deephaven.engine.table.impl.select.DhFormulaColumn.initDef(DhFormulaColumn.java:201)
3 at io.deephaven.engine.table.impl.select.SwitchColumn.initDef(SwitchColumn.java:64)
4 at io.deephaven.engine.table.impl.select.analyzers.SelectAndViewAnalyzer.createContext(SelectAndViewAnalyzer.java:128)
5 at io.deephaven.engine.table.impl.QueryTable.lambda$selectOrUpdate$34(QueryTable.java:1527)
6 at io.deephaven.engine.table.impl.perf.QueryPerformanceRecorder.withNugget(QueryPerformanceRecorder.java:369)
7 at io.deephaven.engine.table.impl.QueryTable.lambda$selectOrUpdate$35(QueryTable.java:1509)
8 at io.deephaven.engine.table.impl.QueryTable.memoizeResult(QueryTable.java:3646)
9 at io.deephaven.engine.table.impl.QueryTable.selectOrUpdate(QueryTable.java:1508)
10 at io.deephaven.engine.table.impl.QueryTable.update(QueryTable.java:1487)
11 at io.deephaven.engine.table.impl.QueryTable.update(QueryTable.java:98)
12 at io.deephaven.api.TableOperationsDefaults.update(TableOperationsDefaults.java:94)
13 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
14 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
15 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
16 at java.base/java.lang.reflect.Method.invoke(Method.java:568)
17 at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:48)
18 at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSite.invoke(PojoMetaMethodSite.java:186)
19 at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:51)
20 at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
21 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
22 at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
23 at io.deephaven.dynamic.Script_7.run(Script_7.groovy:2)
24 at groovy.lang.GroovyShell.evaluate(GroovyShell.java:427)
25 at groovy.lang.GroovyShell.evaluate(GroovyShell.java:461)
26 at groovy.lang.GroovyShell.evaluate(GroovyShell.java:436)
27 at io.deephaven.engine.util.GroovyDeephavenSession.lambda$evaluate$0(GroovyDeephavenSession.java:352)
28 at io.deephaven.util.locks.FunctionalLock.doLockedInterruptibly(FunctionalLock.java:51)
29 at io.deephaven.engine.util.GroovyDeephavenSession.evaluate(GroovyDeephavenSession.java:352)
30 at io.deephaven.engine.util.AbstractScriptSession.lambda$evaluateScript$0(AbstractScriptSession.java:165)
31 at io.deephaven.engine.context.ExecutionContext.lambda$apply$0(ExecutionContext.java:196)
32 at io.deephaven.engine.context.ExecutionContext.apply(ExecutionContext.java:207)
33 at io.deephaven.engine.context.ExecutionContext.apply(ExecutionContext.java:195)
34 at io.deephaven.engine.util.AbstractScriptSession.evaluateScript(AbstractScriptSession.java:165)
35 at io.deephaven.enterprise.dnd.modules.GroovyConsoleSessionWithDatabaseModule$ScriptSessionWrapper.evaluateScript(GroovyConsoleSessionWithDatabaseModule.java:117)
36 at io.deephaven.engine.util.DelegatingScriptSession.evaluateScript(DelegatingScriptSession.java:72)
37 at io.deephaven.engine.util.ScriptSession.evaluateScript(ScriptSession.java:75)
38 at io.deephaven.server.console.ConsoleServiceGrpcImpl.lambda$executeCommand$4(ConsoleServiceGrpcImpl.java:193)
39 at io.deephaven.server.session.SessionState$ExportBuilder.lambda$submit$2(SessionState.java:1537)
40 at io.deephaven.server.session.SessionState$ExportObject.doExport(SessionState.java:995)
41 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
42 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
43 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
44 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
45 at io.deephaven.server.runner.scheduler.SchedulerModule$ThreadFactory.lambda$newThread$0(SchedulerModule.java:100)
46 at java.base/java.lang.Thread.run(Thread.java:840)
47 Caused by: io.deephaven.engine.table.impl.lang.QueryLanguageParser$QueryLanguageParseException:
48 Having trouble with the following expression:
49 Full expression : 2 * (3 + 4
50 Expression having trouble :
51 Exception type : java.lang.IllegalArgumentException
52 Exception message : Invalid expression 2 * (3 + 4: [Encountered unexpected token:<EOF>
53 at line 1, column 10.
54 Was expecting one of:
55 "!="
56 "%"
57 "%="
58 "&"
59 "&&"
60 "&="
61 ")"
62 "*"
63 "*="
64 "+"
65 "+="
66 "-"
67 "-="
68 "->"
69 "/"
70 "/="
71 "::"
72 "<"
73 "<<="
74 "<="
75 "="
76 "=="
77 ">"
78 ">="
79 ">>="
80 ">>>="
81 "?"
82 "^"
83 "^="
84 "instanceof"
85 "|"
86 "|="
87 "||"
88 Problem stacktrace :
89 com.github.javaparser.GeneratedJavaParser.generateParseException(GeneratedJavaParser.java:13708)
90 com.github.javaparser.GeneratedJavaParser.jj_consume_token(GeneratedJavaParser.java:13554)
91 com.github.javaparser.GeneratedJavaParser.PrimaryPrefix(GeneratedJavaParser.java:4760)
92 com.github.javaparser.GeneratedJavaParser.PrimaryExpression(GeneratedJavaParser.java:4548)
93 com.github.javaparser.GeneratedJavaParser.PostfixExpression(GeneratedJavaParser.java:4412)
94 com.github.javaparser.GeneratedJavaParser.UnaryExpressionNotPlusMinus(GeneratedJavaParser.java:4382)
95 com.github.javaparser.GeneratedJavaParser.UnaryExpression(GeneratedJavaParser.java:4255)
96 com.github.javaparser.GeneratedJavaParser.MultiplicativeExpression(GeneratedJavaParser.java:4152)
97 com.github.javaparser.GeneratedJavaParser.AdditiveExpression(GeneratedJavaParser.java:4070)
98 com.github.javaparser.GeneratedJavaParser.ShiftExpression(GeneratedJavaParser.java:4025)
99 com.github.javaparser.GeneratedJavaParser.RelationalExpression(GeneratedJavaParser.java:3965)
100 com.github.javaparser.GeneratedJavaParser.InstanceOfExpression(GeneratedJavaParser.java:3884)
101 com.github.javaparser.GeneratedJavaParser.EqualityExpression(GeneratedJavaParser.java:3816)
102 com.github.javaparser.GeneratedJavaParser.AndExpression(GeneratedJavaParser.java:3771)
103 com.github.javaparser.GeneratedJavaParser.ExclusiveOrExpression(GeneratedJavaParser.java:3731)
104 com.github.javaparser.GeneratedJavaParser.InclusiveOrExpression(GeneratedJavaParser.java:3691)
105 com.github.javaparser.GeneratedJavaParser.ConditionalAndExpression(GeneratedJavaParser.java:3651)
106 com.github.javaparser.GeneratedJavaParser.ConditionalOrExpression(GeneratedJavaParser.java:3606)
107 com.github.javaparser.GeneratedJavaParser.ConditionalExpression(GeneratedJavaParser.java:3574)
108 com.github.javaparser.GeneratedJavaParser.Expression(GeneratedJavaParser.java:3387)
109 com.github.javaparser.GeneratedJavaParser.ExpressionParseStart(GeneratedJavaParser.java:8192)
110 com.github.javaparser.JavaParser.parse(JavaParser.java:123)
111 io.deephaven.engine.table.impl.lang.JavaExpressionParser.parseExpression(JavaExpressionParser.java:27)
112 io.deephaven.engine.table.impl.lang.QueryLanguageParser.<init>(QueryLanguageParser.java:311)
113 io.deephaven.engine.table.impl.lang.QueryLanguageParser.<init>(QueryLanguageParser.java:212)
114 io.deephaven.engine.table.impl.select.codegen.FormulaAnalyzer.parseFormula(FormulaAnalyzer.java:240)
115 io.deephaven.engine.table.impl.select.codegen.FormulaAnalyzer.parseFormula(FormulaAnalyzer.java:122)
116 io.deephaven.engine.table.impl.select.DhFormulaColumn.initDef(DhFormulaColumn.java:181)
117 io.deephaven.engine.table.impl.select.SwitchColumn.initDef(SwitchColumn.java:64)
118 io.deephaven.engine.table.impl.select.analyzers.SelectAndViewAnalyzer.createContext(SelectAndViewAnalyzer.java:128)
119 io.deephaven.engine.table.impl.QueryTable.lambda$selectOrUpdate$34(QueryTable.java:1527)
120 io.deephaven.engine.table.impl.perf.QueryPerformanceRecorder.withNugget(QueryPerformanceRecorder.java:369)
121 io.deephaven.engine.table.impl.QueryTable.lambda$selectOrUpdate$35(QueryTable.java:1509)
122 io.deephaven.engine.table.impl.QueryTable.memoizeResult(QueryTable.java:3646)
123 io.deephaven.engine.table.impl.QueryTable.selectOrUpdate(QueryTable.java:1508)
124 io.deephaven.engine.table.impl.QueryTable.update(QueryTable.java:1487)
125 io.deephaven.engine.table.impl.QueryTable.update(QueryTable.java:98)
126 io.deephaven.api.TableOperationsDefaults.update(TableOperationsDefaults.java:94)
127 java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
128 java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
129 java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
130 java.base/java.lang.reflect.Method.invoke(Method.java:568)
131 org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:48)
132 org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSite.invoke(PojoMetaMethodSite.java:186)
133 org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:51)
134 org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
135 org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
136 org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
137 io.deephaven.dynamic.Script_7.run(Script_7.groovy:2)
138 groovy.lang.GroovyShell.evaluate(GroovyShell.java:427)
139 groovy.lang.GroovyShell.evaluate(GroovyShell.java:461)
140 groovy.lang.GroovyShell.evaluate(GroovyShell.java:436)
141 io.deephaven.engine.util.GroovyDeephavenSession.lambda$evaluate$0(GroovyDeephavenSession.java:352)
142 io.deephaven.util.locks.FunctionalLock.doLockedInterruptibly(FunctionalLock.java:51)
143 io.deephaven.engine.util.GroovyDeephavenSession.evaluate(GroovyDeephavenSession.java:352)
144 io.deephaven.engine.util.AbstractScriptSession.lambda$evaluateScript$0(AbstractScriptSession.java:165)
145 io.deephaven.engine.context.ExecutionContext.lambda$apply$0(ExecutionContext.java:196)
146 io.deephaven.engine.context.ExecutionContext.apply(ExecutionContext.java:207)
147 io.deephaven.engine.context.ExecutionContext.apply(ExecutionContext.java:195)
148 io.deephaven.engine.util.AbstractScriptSession.evaluateScript(AbstractScriptSession.java:165)
149 io.deephaven.enterprise.dnd.modules.GroovyConsoleSessionWithDatabaseModule$ScriptSessionWrapper.evaluateScript(GroovyConsoleSessionWithDatabaseModule.java:117)
150 io.deephaven.engine.util.DelegatingScriptSession.evaluateScript(DelegatingScriptSession.java:72)
151 io.deephaven.engine.util.ScriptSession.evaluateScript(ScriptSession.java:75)
152 io.deephaven.server.console.ConsoleServiceGrpcImpl.lambda$executeCommand$4(ConsoleServiceGrpcImpl.java:193)
153 io.deephaven.server.session.SessionState$ExportBuilder.lambda$submit$2(SessionState.java:1537)
154 io.deephaven.server.session.SessionState$ExportObject.doExport(SessionState.java:995)
155 java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
156 java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
157 java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
158 java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
159 io.deephaven.server.runner.scheduler.SchedulerModule$ThreadFactory.lambda$newThread$0(SchedulerModule.java:100)
160 java.base/java.lang.Thread.run(Thread.java:840)]
161 at io.deephaven.engine.table.impl.lang.JavaExpressionParser.parseExpression(JavaExpressionParser.java:30)
162 at io.deephaven.engine.table.impl.lang.QueryLanguageParser.<init>(QueryLanguageParser.java:311)
163 at io.deephaven.engine.table.impl.lang.QueryLanguageParser.<init>(QueryLanguageParser.java:212)
164 at io.deephaven.engine.table.impl.select.codegen.FormulaAnalyzer.parseFormula(FormulaAnalyzer.java:240)
165 at io.deephaven.engine.table.impl.select.codegen.FormulaAnalyzer.parseFormula(FormulaAnalyzer.java:122)
166 at io.deephaven.engine.table.impl.select.DhFormulaColumn.initDef(DhFormulaColumn.java:181)
167 ... 44 more
The summary information is on line 1, indicating an error with the formula 2 * 3 ( + 4.
1 r-Scheduler-Serial-1 | .c.ConsoleServiceGrpcImpl | Error running script: io.deephaven.engine.table.impl.select.FormulaCompilationException: Formula compilation error for: 2 * (3 + 4
The stack traces indicate where the error occurred, but the relevant details begin on line 47 and continue until line 87:
47 Caused by: io.deephaven.engine.table.impl.lang.QueryLanguageParser$QueryLanguageParseException:
48 Having trouble with the following expression:
49 Full expression : 2 * (3 + 4
50 Expression having trouble :
51 Exception type : java.lang.IllegalArgumentException
52 Exception message : Invalid expression 2 * (3 + 4: [Encountered unexpected token:<EOF>
53 at line 1, column 10.
...
In this case, the message indicates that the <EOF> or end of file token was encountered at column 10, but another token was expected. Many options are listed, including the correct closing parenthesis on line 61. In practice, with so many options, you must carefully examine the formula around the indicated position to determine what syntax is incorrect.
Invalid operator overloads
You may also have errors related to operators. The Deephaven formula parser converts operators to method calls to permit users to naturally express mathematic operations between various types unsupported by Java (for example, Deephaven permits you to add two BigIntegers together using the plus (+) operator). This is similar to operator overloading in many programming languages. In the example below, the formula attempts to multiply an integer and a string, which is not supported by Deephaven formulas:
x = emptyTable(1).update("X=2 * `abc`")
The following error is produced:
Error running script: io.deephaven.engine.table.impl.select.FormulaCompilationException: Formula compilation error for: 2 * `abc`
...
Caused by: io.deephaven.engine.table.impl.lang.QueryLanguageParser$QueryLanguageParseException:
Having trouble with the following expression:
Full expression : 2 * "abc"
Expression having trouble :
Exception type : io.deephaven.engine.table.impl.lang.QueryLanguageParser$ParserResolutionFailure
Exception message : Cannot find method multiply(int, java.lang.String)
...
The relevant detail is that no method is named multiply for int and Strings. The following operators may also be present in your stack trace, corresponding to an operator used in your formula:
| Operator | Method |
|---|---|
+ | plus |
- | minus |
* | multiply |
/ | divide |
% | remainder |
= or == | eq |
< | less |
<= | lessEquals |
> | greater |
>= | greaterEquals |
^ | xor |
| | binaryOr |
^ | binaryAnd |
|| | or |
&& | and |
Invalid use of in
The in keyword can only be used in a standalone expression and cannot be combined with other operators or formula elements (even if surrounded in parenthesis). For example, this is valid syntax:
a = [1, 2, 3]
x = emptyTable(10).update("X=ii").where("X in a")
However, in this example in is combined with another expression:
a = [1, 2, 3]
x = emptyTable(10).update("X=ii").where("X % 2 == 0 && X in a")
This results in an error:
Having trouble with the following expression:
Full expression : X % 2 == 0 && X in a
Expression having trouble :
Exception type : java.lang.IllegalArgumentException
Exception message : Invalid expression X % 2 == 0 && X in a: [Encountered unexpected token: "in" <IDENTIFIER>
at line 1, column 17.
When the in operator begins the combined expression, then an alternative error is produced indicating that the expression is not a valid long value:
RuntimeError: java.lang.IllegalArgumentException: Failed to convert literal value <a && X % 2 == 0> for column "X" of type long
...
caused by java.lang.NumberFormatException: For input string: "a && X % 2 == 0"