Query Failure Due to Server Connection

I launched two queries this afternoon that failed with the following errors: "No valid origins exist," "Unable to validate authentication token," and "Server marked token invalid." When I ran the same queries earlier in the day, they completed just fine. Why am I getting these errors?

In Deephaven, client processes maintain connections to one or more authentication servers. When client processes need to initiate connections to a Deephaven service, the client requests a token from one of the authentication servers and sends it to the Deephaven service. The server process then verifies the token with the authentication server. If the authentication server which provided the token is down for some reason (e.g., for maintenance), then the server process will not be able to verify the token, and your connection cannot be established.

To ensure that your application is fault tolerant, when performing an operation that requires sending an authentication token from the client to the server, you must handle errors verifying your token, which will propagate up the stack as a com.fishlib.auth.AuthException. When you receive an AuthException, you must proceed to the next authentication server.

To automate this process, you may use the WAuthenticationClientManager.TokenFactory class. The token factory is initialized with the service you are connecting to, and then you should invoke the tryActionUntil with a lambda:

final WAuthenticationClientManager.TokenFactory tokenFactory = WAuthenticationClientManager.DEFAULT.getTokenFactory("PersistentQueryController");
tokenFactory.tryActionUntil(token -> getControllerClient().authenticateToController(token))

This will automatically try each of the authentication servers. If all of the authentication servers are exhausted, your application will receive an AuthException with the message: "No more valid origins exist".

In this case, you are creating a RemoteProcessingRequest, which automatically includes an AuthToken, but you can specify the AuthToken as one of the constructor parameters. You are then calling RemoteQueryClient.sendNewQuery using that RemoteProcessingRequest. You should include both the creation of the RemoteProcessingRequest and the RemoteQueryClient.sendNewQuery inside of a TokenFactory lambda.

The creation of the RemoteProcessingRequest creates the token, and there are constructors with the token as an argument. You'll need to use one of those constructors in the same lambda function that creates your RemoteProcessingRequest. If you create the request in the lambda, but use it outside the lambda, the retry logic will be ineffective.