IAP Integration

Cloud Identity-Aware Proxy (IAP) is a Google Cloud service that adds an additional layer of security to your applications running on Google Cloud. IAP uses Google Cloud Identity and Access Management (IAM) to authenticate users and then uses role-based access control (RBAC) to determine what resources users have access to. In this example, we integrate Deephaven, deployed to GCE Kubernetes, to GCE-enabled IAP.

Enable IAP API for the project. Search "IAP" in the GCE console bar to acess the page to enable IAP for your current project:

Configure the Consent Screen:

Once IAP is enabled, you will be prmompted to configure the OAuth Consent Screen.

Use User Type “Internal”.
Fill in the authorized domain.

In API’s & Services, configure OAuth 2.0 Client ID, which will have a client ID and and a client secret.
Tie that client id and secret to the secret in your K8S cluster by creating a new secret:

kubectl create secret generic oauth-client-secret \
   --from-literal=client_id=client_id \
   --from-literal=client_secret=client_secret

Create an ssl certificate and key secret in your cluster for your ingress load balancer:

kubectl create secret tls my-ingress-tls-secret --cert=mydomain.name.crt --key=mydomain.name.key

We use `mydomain.name' for the key and certificate.

Create an ingress load balancer in your cluster and connect it to the tls secret you created above and to the k8s service to which you want that load balancer to redirect external requests. In our case, we tie it to the envoy service becausse we are building a Deephaven application.

kind: Ingress
metadata:
name: my-ingress
annotations:
  kubernetes.io/ingress.class: 'gce'
  kubernetes.io/ingress.global-static-ip-name: 'my-iap-ingress'
spec:
  tls:
    - secretName: my-ingress-tls-secret
  rules:
    - http:
        paths:
          - path: '/*'
        pathType: ImplementationSpecific
        backend:
          service:
            name: envoy
            port:
              number: 80

Next, create the backend service to which IAP binds, using the OAuth id/secret k8s secret we created earlier.

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
name: my-iap-backend
namespace: my-namespace
spec:
iap:
  enabled: true
  oauthclientCredentials:
    secretName: oauth-client-secret

Apply the ingress and backend config yaml files using the command kubectl apply -f filename.yaml. You should now be able to enable IAP for that backend service in the IAP page (“deephaven-enterprise/envoy” in the example).

Allow particular domain users to use the application. Create an IAM policy binding to that backend service:

gcloud iap web add-iam-policy-binding --resource-type=backend-services --service my-backend-service \
--member=user:user1@mydomain.name --role='roles/iap.httpsResourceAccessor'

To give Deephaven workers more time to initialize and report back to the Envoy service to which our backend service connects, increase the timeout for your backend service from the default 30 seconds to 660 seconds.

gcloud compute backend-services update my-backend-service  --global --timeout 660

Known Issues and Caveats:

The Envoy URL must match Google URL

The port of the frontend Google load balancer must match the port used by the Envoy layer. Normally, we expose our Envoy port directly, but with IAP, there is an additional load balancer in front. When the final layer runs on a different port than the Envoy layer, the worker URLs try to connect to a port that triggers a different origin / lack of IAP cookies and fails. By forcing both the external and internal load balancer to run on port 443, the socket URLs are same-origin, and the system can connect.

The project should have `iris` property

Set this property in iris-environment.prop to enable our CORS filter. This allows IAP to intercept the request and authenticate you:

cors.allowedOrigins=https://full.domain.name

Note that if the final load balancer is served on a given port, you MUST include this port (as well as the https:// protocol) in the allowed origin property.

Users must be explicitly granted permission to access IAP resources

As per https://cloud.google.com/iap/docs/enabling-kubernetes-howto#iap-access, we must add an IAM principal allowing users to access IAP-protected resources. The necessary permission is "Cloud IAP > IAP-secured Web App User". To add:

Click on the IAP application you want.
In the right-hand side menu, click Add Prinicipal This allows you to add a user for a specific IAP application rather than all IAP applications (which is what you get when granting through regular IAM permissions).

The Google backend must have correct oauth clientId/secret attached

In order for IAP to work on any Google Compute Engine deployment (kube or otherwise), the backend service must have the correct-for-IAP OAuth clientId + secret attached. This can only be altered manually via command line as there is no GUI exposing these values. By default, you should be configuring this via the Kubernetes configuration listed here: https://cloud.google.com/iap/docs/enabling-kubernetes-howto#add-iap-to-backendconfig

You can check/update oauth settings manually. Find your Kubernetes backend by navigating to https://console.cloud.google.com/kubernetes/discovery?project=deephaven-dev and clicking on the Envoy marked Internal Load Balancer:

The Deployments section lists the Load Balancer Backends:

You can check the oauth settings with:

gcloud compute backend-services --project deephaven-dev describe --global my-backend-service | grep oauth -B 3

This will return nothing if no oauth key is set. You can remove the grep to see the rest of the yaml.

Finally, you can set the oauth clientId+secret by following the Google Cloud CLI Documentation.

gcloud compute backend-services --project my-project update \
  --iap=enabled,oauth2-client-id=client_id,oauth2-secret=secret_id