Skip to content
Goiabada

Upgrade Goiabada

This page helps you move a deployment to a new release of Goiabada, and back again if you need to.

You don’t migrate the database yourself. The auth server brings the schema up to date when it starts, before it listens. The admin console holds no database connection of its own, since it reaches the auth server through its API, so restarting it migrates nothing.

Update to a new release

Section titled “Update to a new release”

  1. Read the release notes on the releases page. Anything a release asks of an upgrade beyond these steps, its notes state.

  2. Back up the database, and the AES key if you haven’t: see Back up the AES key.

  3. Install the new release, and restart. With images from your own registry, copy the release’s two images there first.

    Run the new release’s setup wizard over your answers file, from the directory holding docker-compose.yml. It writes the release’s version and digests into both image lines, and anything else the release changed in what it generates, and leaves docker-compose.override.yml, with your secrets, as it is. Review the diff, then start the new release:

    Terminal window
    ./goiabada-setup-linux-amd64 --answers=goiabada-setup.yaml --keep-secrets
    git diff -- docker-compose.yml goiabada-setup.yaml
    docker compose pull
    docker compose up -d

    Docker Compose recreates the containers whose image or configuration changed, the two servers among them. The database’s data stays in its volume.

    Without an answers file, set both image lines in docker-compose.yml to the new release yourself, the two references its notes list under Pinned image references, then run the last two commands.

    A setup wizard built from source writes the moving latest tag instead of a version, so docker compose pull fetches whatever release latest names that day. Replace it with a release version, so that you choose when to upgrade.

    A deployment pinned to a digest changes the tag and the digest together: the digest alone decides what runs, so a tag changed on its own keeps the old release running while docker compose up -d reports success.

  4. Check the start. The auth server writes database migrated once it has run the release’s migrations, or no need to migrate the database when it had none to run, and then starts its listeners. The build information record it writes as it starts names the release actually running in its version attribute: confirm it is the one you installed, which is where a stale pin shows.

Pin a deployment to an image digest

Section titled “Pin a deployment to an image digest”

A tag is mutable in the registry: a re-push under the same name, by mistake or through a compromised credential, changes what every node pulls next. A digest names the image content itself, and the release’s attestations are made against it, so pinning a deployment to a digest makes the registry unable to change what runs.

A release of the setup wizard pins both image lines: each names the release’s version tag and the digest the release pushed, docker.io/leodip/goiabada:authserver-<version>@sha256:<digest>. Read the rule at the end of this section before you change one by hand.

To pin an image line that names the tag alone, written by a wizard built from source or with --image-digest=false, add the digest after the tag: leodip/goiabada:authserver-<version>@sha256:<digest>, keeping the registry your line names in front.

Where the digest comes from:

  • The release’s notes. A release’s notes list both images under Pinned image references, each as a full reference, tag@sha256:digest, with the digests that release pushed and attested.
  • The registry, read through buildx. docker buildx imagetools inspect leodip/goiabada:authserver-<version> prints the digest on its Digest: line, and the same holds for the admin console’s image. That line names the multi-platform image index, the digest the release attests. A per-platform digest, the kind a registry’s tags page lists beside each architecture, selects that one architecture, so a node on another one can’t run it natively, and it won’t verify against the release’s attestation, which names the index.

Verify the digest against the release’s attestation before you trust it:

Terminal window
gh attestation verify oci://docker.io/leodip/goiabada@sha256:<digest> --repo leodip/goiabada --source-ref refs/tags/v<version> --signer-workflow leodip/goiabada/.github/workflows/release.yml

--source-ref binds the digest to the release you’re installing: an image an earlier release attested, re-pushed under this version’s tag, fails it, where --repo alone accepts any image this repository attested.

A reference carrying both a tag and a digest is pulled by the digest alone, so once a deployment is pinned, the tag and the digest change together, on an upgrade and on a rollback. A tag changed on its own keeps the old release running while the platform reports success, and a digest left in place on a rollback starts the newer server again, which migrates the schema back up.

How migrations run

Section titled “How migrations run”

The auth server applies whatever migrations its release carries when it starts, between opening the database and starting its listeners. It refuses to start on a database that is half migrated, and on one a newer release has already migrated, since it can’t know what that release changed. Waiting for the migration lock, or marked dirty covers both refusals.

A start writes these Info records about the schema:

Record When Attributes
waiting for the migration lock Another process holds the lock. Written once, before the wait none
migrating the database Before the first migration runs from_version, to_version, pending
database migrated After the last migration has run from_version, to_version, applied, duration
no need to migrate the database The schema is already current none
database migration stopped A stop signal ended the migrations between two files from_version, reached_version, applied, remaining

from_version is the schema version the database recorded, 0 for one that was never migrated, and to_version the one this release expects. pending and applied count migration files. A replica that waited behind another one’s migration writes waiting for the migration lock and then no need to migrate the database.

The migration lock

Section titled “The migration lock”

Only one process migrates a database at a time. A process takes the migration lock before it migrates and releases it when it’s done, so with several replicas the first to start migrates, and the others wait for it, then find the schema current and start. The migrate command takes the same lock, and waits the same way.

A waiting process waits for as long as the lock is held, on every engine; nothing in Goiabada gives up. A start that writes waiting for the migration lock and nothing more is waiting for a process that’s still migrating, not hung. The lock belongs to the holder’s database session, so a holder that dies releases it and can’t strand the others. On SQLite the lock covers one process only, so a start there never waits for it.

Your platform decides how long a start may take:

  • Kubernetes: the generated manifest’s startup probe allows 5 minutes, and restarts the container after that. An upgrade whose migrations take longer needs a higher failureThreshold, as First start and upgrades explains.
  • Docker Compose: the auth server’s healthcheck gives a start 5 minutes, start_period: 300s, before its failures count, and the admin console waits for that healthcheck to pass before it starts.

A start that’s stopped

Section titled “A start that’s stopped”

A stop signal (SIGTERM, which Kubernetes and Compose send, or SIGINT) that reaches the auth server while it’s still starting stops it cleanly, whatever it’s doing:

  • A wait ends at once: for the database connection, for the database to be created, or for the migration lock.
  • A migration file already running runs to its end, and the next one doesn’t start. The lock is released and the schema is left clean at the version it reached, which the next start carries on from, so a start stopped again and again still moves forward file by file.
  • The AES key’s re-encryption and a first start’s seed each commit whole: one under way completes, and one not yet begun doesn’t begin.

The auth server then writes shutdown signal received, database migration stopped if migrations were under way, and auth server stopped, and exits 0. In database migration stopped, reached_version is where the schema now is, and remaining how many files the next start will run.

The one thing the auth server can’t finish is a migration file that runs longer than the grace period the platform allows a stop, terminationGracePeriodSeconds on Kubernetes or stop_grace_period in Compose. The platform then ends it with SIGKILL, which no process can catch, and that file is left marked dirty: on MySQL and SQL Server, which run a file without an enclosing transaction, partly applied. The next start refuses until the schema is repaired by hand, as A migration was cut short walks through. Before an upgrade whose release notes announce a long migration, give the start enough time that the platform doesn’t stop it, and don’t stop it yourself.

Roll back to an earlier release

Section titled “Roll back to an earlier release”

The migrate command steps the schema without starting the server, against the database in GOIABADA_DB_*. The --db-* flags override those variables, given before or after migrate; any other flag goes before it. A mistyped command, such as migrat, is refused rather than starting the server.

Command What it does
goiabada-authserver migrate version Prints the schema version this binary expects and the one the database records
goiabada-authserver migrate to <version> Steps the schema to that version, up or down

  1. Back up the database. A rollback discards whatever the newer release wrote into columns and tables the older one doesn’t have.

  2. Step the schema down with the current binary, naming the schema version the target release expects, which its release notes state. The current binary is the one that must run it, because it’s the only one carrying the migrations being rolled back.

    Terminal window
    docker compose run --rm goiabada-authserver migrate to <version>

  3. Install the target release straight away, as in Update to a new release, with its version in place of the new one. A deployment pinned to a digest changes the tag and the digest together here too: a digest left in place starts the newer server again, which migrates the schema back up.

migrate to refuses any version below 000044: releases older than the one carrying that schema changed stored data in ways no migration reverses, so no rollback reaches them.

migrate to stops cleanly, as a stopped start does, on Ctrl-C (SIGINT), SIGTERM, SIGQUIT (Ctrl-\) and SIGHUP, which closing the terminal or losing an SSH session sends: a migration file already running finishes, and no other starts. It then exits 1, since it didn’t reach the version you asked for. Run the same command again to carry on. Don’t end it with SIGKILL (kill -9): that stops it mid-file and leaves the schema dirty.

Next steps

Section titled “Next steps”
Waiting for the migration lock, or marked dirtyWhen a start waits, or refuses the database.
DatabasePrepare the database for each engine.
SecretsWhat each secret protects, and backing up the AES key.