Gateway and certificates

This page helps you put Goiabada on a cluster behind Envoy Gateway, with certificates from Let’s Encrypt through cert-manager.

It’s the recipe the project tests. Any Gateway API implementation and certificate issuer work too: the Goiabada side is the same, and Overview lists what it needs.

For a cluster without a gateway or cert-manager

The steps install Envoy Gateway and cert-manager, and create cluster-wide resources. If your cluster already runs either, skip its installation, check that what it runs fits the steps, and never apply these files over it. Ask whoever runs the cluster when in doubt.

Set it up

1. Install Envoy Gateway and its GatewayClass. Envoy Gateway’s install brings the Gateway API’s resources; wait for it:

   kubectl apply --server-side -f https://github.com/envoyproxy/gateway/releases/download/v1.9.1/install.yaml
   kubectl wait --timeout=5m -n envoy-gateway-system \
     deployment/envoy-gateway --for=condition=Available

   Check the Envoy Gateway releases for a newer version.

   Then create the eg GatewayClass the manifest names, with an EnvoyProxy that sets the traffic policy of the load balancer in front of Envoy. This is the Cluster policy, the wizard’s default, which works behind every load balancer; for Local, use the file in The gateway’s traffic policy instead. Save it as gatewayclass.yaml:

   apiVersion: gateway.envoyproxy.io/v1alpha1
   kind: EnvoyProxy
   metadata:
     name: goiabada-proxy
     namespace: envoy-gateway-system
   spec:
     provider:
       type: Kubernetes
       kubernetes:
         envoyService:
           externalTrafficPolicy: Cluster
   ---
   apiVersion: gateway.networking.k8s.io/v1
   kind: GatewayClass
   metadata:
     name: eg
   spec:
     controllerName: gateway.envoyproxy.io/gatewayclass-controller
     parametersRef:
       group: gateway.envoyproxy.io
       kind: EnvoyProxy
       name: goiabada-proxy
       namespace: envoy-gateway-system

   kubectl apply -f gatewayclass.yaml

2. Install cert-manager, after Envoy Gateway, since it looks for the Gateway API’s resources when it starts. Turn on its Gateway API support, and wait for it:

   kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.21.2/cert-manager.yaml
   kubectl -n cert-manager patch deployment cert-manager --type=json \
     -p='[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--enable-gateway-api"}]'
   kubectl -n cert-manager rollout status deployment/cert-manager --timeout=5m
   kubectl -n cert-manager rollout status deployment/cert-manager-webhook --timeout=5m
   kubectl -n cert-manager rollout status deployment/cert-manager-cainjector --timeout=5m

   Check the cert-manager releases for a newer version.

3. Prepare the database, MySQL, PostgreSQL or SQL Server, reachable from the cluster’s pods, as Database describes. Only trying Goiabada out? A PostgreSQL to try Goiabada on Kubernetes runs one in the cluster.

4. Generate the files and deploy them. Run the setup wizard and choose Kubernetes cluster. Then answer, in order: the database, the two public URLs (such as https://auth.example.com and https://admin.example.com), the namespace (goiabada unless you change it), the gateway’s traffic policy, whether NetworkPolicies restrict who reaches the pods, the rate limiter, whether to expose metrics, the administrator, and the database’s connection details.

   The wizard offers to test the database connection from your machine, which can’t reach a host only the cluster resolves, such as postgres.db.svc.cluster.local: for one, it says so and defaults to No. It warns if the database isn’t empty, and writes goiabada-k8s.yaml and goiabada-secrets.yaml. Back up the AES key from goiabada-secrets.yaml before anything else, as Back up the AES key says, and never commit that file; the wizard warns when it writes into a git working tree.

   Deploy both files, the Secrets first:

   kubectl apply -f goiabada-secrets.yaml -f goiabada-k8s.yaml

   A container reads its Secrets once, when it starts. Applied after the manifest, Secrets that replace older ones of the same names, from an earlier attempt in the same namespace, would arrive after the new pods had started with the old ones, and the database would be seeded with secrets the cluster no longer holds. Both files create the namespace.

   For production, you can create the Secrets another way instead of from the file: see Secrets.

5. Point DNS at the Gateway. Read its address, which can take a minute to appear:

   kubectl get gateway goiabada -n goiabada -o jsonpath='{.status.addresses[0].value}'

   Then create an A record for each host name, auth.example.com and admin.example.com, with that address, or a CNAME when the address is a host name, as on AWS. If your DNS provider can proxy traffic, as Cloudflare does, leave both records unproxied, DNS only: Let’s Encrypt has to reach the Gateway itself. Wait until both names resolve, for instance with nslookup auth.example.com.

6. Create a ClusterIssuer for Let’s Encrypt, which answers its challenge through Goiabada’s Gateway. Put your namespace in if it isn’t goiabada, then save it as letsencrypt-issuer.yaml:

   apiVersion: cert-manager.io/v1
   kind: ClusterIssuer
   metadata:
     name: letsencrypt-prod
   spec:
     acme:
       server: https://acme-v02.api.letsencrypt.org/directory
       privateKeySecretRef:
         name: letsencrypt-prod
       solvers:
       - http01:
           gatewayHTTPRoute:
             parentRefs:
             - name: goiabada
               namespace: goiabada
               kind: Gateway

   kubectl apply -f letsencrypt-issuer.yaml

   Create it only once both names resolve. Until it exists, cert-manager waits; from then on, it looks the names up. A name looked up before its record exists can stay missing for the cluster’s resolver for as long as your zone lets it remember a missing name, often 30 minutes. Let’s Encrypt needs no email address: it sends no expiry notices since June 2025.

7. Check it’s up, and sign in.

   kubectl get pods -n goiabada                    # both Running, 1/1
   kubectl get gateway,httproute -n goiabada       # the Gateway PROGRAMMED True, with an ADDRESS
   kubectl get certificates -n goiabada            # both READY True, within a few minutes

   Then sign in at https://admin.example.com, as First sign-in describes. The wizard prints no secret, so read the administrator’s password back out of the cluster:

   kubectl get secret goiabada-secrets -n goiabada -o jsonpath='{.data.admin-password}' | base64 -d

If something goes wrong:

• The certificates stay not ready: Certificates are not issued.
• The auth server’s pod is in CrashLoopBackOff, or can’t reach the database: CrashLoopBackOff or unable to create the database connection.
• The admin console’s pod is in CrashLoopBackOff, or signing in to it fails: Locked out of the admin console.
• A start waits for the migration lock, or refuses a dirty schema: Waiting for the migration lock, or marked dirty.
• Signing in fails with “Invalid redirect_uri”: Invalid redirect_uri.

The route in

The Gateway has three listeners: HTTP on port 80, and HTTPS on port 443 for each host name, each ending TLS with a certificate of its own, goiabada-tls-auth and goiabada-tls-admin. One HTTPRoute per host name sends its traffic to the server’s Service, and a third answers every plain HTTP request with a 301 to HTTPS.

The Gateway carries the annotation cert-manager.io/cluster-issuer: "letsencrypt-prod". From it, cert-manager creates one Certificate per HTTPS listener, and the ClusterIssuer proves you control each host name by answering Let’s Encrypt’s HTTP-01 challenge through a temporary route on the Gateway’s port 80 listener. cert-manager renews the certificates itself.

The gateway’s traffic policy

The traffic policy of the load balancer in front of Envoy decides which address Envoy sees for a client, and so what Goiabada’s rate limits, audit records and session records count. The wizard asks which one you use, writes its comments to match, and prints the GatewayClass file for it.

Cluster, the wizard’s default, works behind every load balancer: the load balancer may send a connection to any node, and the node forwards it to an Envoy pod wherever one runs. The node replaces the client’s address with its own, so Goiabada sees a node’s address for every client.

Local, with Envoy on every node (--gateway-traffic-policy=local), keeps the client’s address behind a load balancer that passes each connection through: a node delivers a connection only to an Envoy pod on that same node, so nothing rewrites its source. A load balancer that proxies connections instead, opening its own to the nodes, as some clouds’ do, shows Goiabada its own address under either policy, and only Proxy Protocol, below, carries the client’s. Check which you have once Goiabada runs: request any page, then read that request’s ip= in the auth server’s log with kubectl logs -n goiabada deployment/goiabada-authserver. Your own address means Local works; one address for every client is the load balancer’s. A node with no Envoy pod drops what the load balancer sends it, which is why Envoy runs as a DaemonSet here, at the cost of one Envoy pod per node. Save this as gatewayclass.yaml instead:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: goiabada-proxy
  namespace: envoy-gateway-system
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyDaemonSet: {}
      envoyService:
        externalTrafficPolicy: Local
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: goiabada-proxy
    namespace: envoy-gateway-system

An empty envoyDaemonSet is a DaemonSet with every default; an EnvoyProxy takes it or envoyDeployment, not both. This variant follows Envoy Gateway v1.9.1’s API. On a managed cluster whose load balancer proxies connections, applying it over the Cluster file replaced Envoy’s Deployment with a DaemonSet, the load balancer kept its address and traffic flowed, and Goiabada saw the load balancer’s address for every client. Check that a pod runs on every node after applying it: kubectl get daemonset -n envoy-gateway-system. Changing the policy on a running cluster replaces Envoy rather than rolling it: switching back to Cluster there failed requests for about 15 seconds, so change it when an interruption is acceptable.

A third way keeps the client’s address without an Envoy pod per node: Proxy Protocol, where the load balancer prepends the client’s address to each connection. It needs annotations on the load balancer’s Service that differ from cloud to cloud, so the wizard doesn’t offer it; Envoy Gateway’s client traffic policy documentation covers the Envoy side.

The EnvoyProxy and GatewayClass are cluster-wide

Every Gateway of class eg shares them. On a cluster that already runs Envoy Gateway, the traffic policy belongs to whoever runs it: agree it with them rather than apply either file over theirs, and answer the wizard with the policy the cluster actually uses.

Client IP and proxy trust has how Goiabada reads the address Envoy forwards.

Next steps

High availabilityRun more than one pod, and size the database for it.

SecurityWho can reach the pods, and what they may do.

SecretsThe Secrets the manifest reads, and other ways to create them.