Skip to main content

Uninstall and cleanup

This page describes how to remove a Foundation4 installation, which resources remain after each step, and how to clean up after an interrupted installation. Operators who retire an installation, reinstall Foundation4 or recover from a failed installation use this page.

Removal order​

An installation is removed in the reverse order of installation:

  1. Application release. foundation4ai first, so that the API server and the workers stop before the services they use.
  2. Core release. foundation4ai-core, which removes NATS JetStream, Valkey, Prometheus and the bundled PostgreSQL.
  3. Remaining resources. The namespace, or the resources that remain in the namespace.
  4. External resources. The external database and the local files, when the installation is retired.

A production installation is backed up before removal, as described in Backup, restore and upgrades, unless the data is to be destroyed.

Resources after uninstall​

helm uninstall removes the resources that a release manages. Resources that the releases create as Helm hooks, resources that the operator creates, and the volumes of the NATS JetStream and bundled PostgreSQL StatefulSets remain.

ResourceRemains after helm uninstallRemoved by namespace deletion
Deployments, Services, StatefulSets, PodDisruptionBudgets, Ingress or HTTPRoute of both releasesNoNot applicable
Secret foundation4ai-core-nats-authNoNot applicable
Prometheus volume claim foundation4ai-core-prometheus-serverNoNot applicable
Prometheus ClusterRole and ClusterRoleBinding foundation4ai-core-prometheus-serverNoNot applicable; cluster-scoped
A failed Job foundation4ai-api-server-db-migration, foundation4ai-api-server-license-check or foundation4ai-api-server-create-admin-api-key, and the Job pod. Helm deletes each Job after the Job succeeds.YesYes
Secrets foundation4ai-core, foundation4ai-core-redis and foundation4ai-api-serverYesYes
ConfigMap foundation4ai-api-serverYesYes
Secret foundation4ai-secrets and image pull secretsYesYes
NATS JetStream volume claims foundation4ai-core-nats-js-foundation4ai-core-nats-0, -1 and -2YesYes
Bundled PostgreSQL volume claim postgres-data-foundation4ai-core-postgres-0 (evaluation profile)YesYes
External database: schemas public and embeddings and every stored objectYesNo

The remaining Secrets hold every credential of the installation: foundation4ai-secrets holds the master key secret and the application secret, and foundation4ai-core holds the connection URLs with the passwords. The remaining NATS JetStream volumes hold queued document text for up to 24 hours after submission, and the remaining volume claim of the bundled PostgreSQL holds every pipeline and document of an evaluation installation.

Complete removal​

The procedure removes every resource of an installation from the cluster. The commands match the removal of Install for evaluation.

  1. Uninstall the application release:

    helm uninstall foundation4ai -n foundation4ai

    Expected result: release "foundation4ai" uninstalled.

  2. Uninstall the core release:

    helm uninstall foundation4ai-core -n foundation4ai

    Expected result: release "foundation4ai-core" uninstalled.

  3. List the resources that remain:

    kubectl get jobs,secrets,configmaps,pvc -n foundation4ai

    Expected result: any failed installation Job, the Secrets foundation4ai-secrets, foundation4ai-core, foundation4ai-core-redis and foundation4ai-api-server, any image pull secrets, the ConfigMap foundation4ai-api-server, the 3 NATS JetStream volume claims and, for an evaluation installation, the volume claim postgres-data-foundation4ai-core-postgres-0. Kubernetes also lists the ConfigMap kube-root-ca.crt, which every namespace contains.

  4. Delete the namespace:

    kubectl delete namespace foundation4ai

    Expected result: namespace "foundation4ai" deleted. The namespace deletion also removes the volume claims, the Secrets and the remaining Jobs.

  5. Confirm that no cluster-scoped resources remain:

    kubectl get clusterrole,clusterrolebinding | grep foundation4ai

    Expected result: no output.

When the namespace is deleted without the two uninstall steps, the ClusterRole and the ClusterRoleBinding foundation4ai-core-prometheus-server remain. The following command removes both:

kubectl delete clusterrole,clusterrolebinding foundation4ai-core-prometheus-server

Expected result: clusterrole.rbac.authorization.k8s.io "foundation4ai-core-prometheus-server" deleted and the matching line for the ClusterRoleBinding.

Removal within a shared namespace​

When the namespace holds other workloads and stays in place, the remaining resources are deleted by name after the two uninstall steps:

kubectl delete job -n foundation4ai --ignore-not-found foundation4ai-api-server-db-migration \
foundation4ai-api-server-license-check foundation4ai-api-server-create-admin-api-key
kubectl delete secret -n foundation4ai --ignore-not-found foundation4ai-api-server foundation4ai-core \
foundation4ai-core-redis foundation4ai-secrets
kubectl delete configmap -n foundation4ai foundation4ai-api-server
kubectl delete pvc -n foundation4ai --ignore-not-found foundation4ai-core-nats-js-foundation4ai-core-nats-0 \
foundation4ai-core-nats-js-foundation4ai-core-nats-1 foundation4ai-core-nats-js-foundation4ai-core-nats-2 \
postgres-data-foundation4ai-core-postgres-0

Expected result: one deleted line for each resource that exists. --ignore-not-found skips the Jobs that Helm deleted after success, foundation4ai-core-redis when the bundled Valkey was disabled, and the volume claim of the bundled PostgreSQL when the bundled PostgreSQL was disabled. Image pull secrets created for Foundation4 are deleted by name as well.

External resources​

An installation with an external database keeps every pipeline, document, API key and the migration history in that database after the uninstall. A database administrator removes the database and the user when the installation is retired:

psql "$ADMIN_DATABASE_URL" -c 'DROP DATABASE foundation4ai'
psql "$ADMIN_DATABASE_URL" -c 'DROP ROLE foundation4ai'

Expected result: DROP DATABASE, then DROP ROLE. PostgreSQL refuses to drop a database with open connections, so the Foundation4 pods are removed first.

The following local files hold credentials and are deleted, or moved to the organization's secret store, when the installation is retired: foundation4ai.secrets.env, any backup of the Secret, and database dumps with the secrets that belong to each dump.

Reinstallation​

A reinstallation against the same external database keeps every stored object and the system ID, so the existing license remains valid. The reinstallation uses the application secret and the master key of the previous installation: another application secret makes the stored LLM API keys and embedding model credentials unreadable, and another master key identifier or secret replaces the master key of the previous installation, as Secrets and keys describes.

The following rules apply to a reinstallation:

  • NATS JetStream volumes. Volumes kept from the previous installation hold jobs for documents of the previous installation. The volumes are kept for a reinstallation against the same database, and deleted when the database is new.
  • Remaining Jobs. Helm replaces a failed Job of the previous installation when the new installation runs the hooks, so the Jobs need no removal.
  • Evaluation profile. A reinstallation of the core release reuses the remaining volume claim of the bundled PostgreSQL, with the database, the passwords and the system ID of the previous installation. After the volume claim is deleted, the core release creates a new, empty bundled database with a new system ID, which needs a new license.

Interrupted installation​

An installation that stops at a failed Job or at the Helm timeout, or a Helm command that is interrupted, leaves a release in the status failed, pending-install or pending-upgrade, and can leave a failed or running Job. The following procedure brings the namespace back to a state from which the installation starts again.

  1. Show the status of both releases and the Jobs:

    helm list -n foundation4ai --all
    kubectl get jobs -n foundation4ai

    Expected result: the STATUS column of each release, and the completions of each Job.

  2. Remove a release with the status failed or pending-install that never reached deployed:

    helm uninstall foundation4ai -n foundation4ai

    Expected result: release "foundation4ai" uninstalled. The same command with foundation4ai-core removes an interrupted core release.

  3. Return a release with the status pending-upgrade, or failed after an upgrade, to the last deployed revision:

    helm history foundation4ai -n foundation4ai
    helm rollback foundation4ai <last deployed revision> -n foundation4ai

    Expected result: Rollback was a success! Happy Helming!. When the interrupted upgrade applied new migrations, the rollback leaves the pods unable to start, as described in Backup, restore and upgrades; the upgrade is completed instead.

  4. Correct the cause of the interruption and repeat the installation step that failed.

    Expected result: Helm reports STATUS: deployed for the repeated step.

Troubleshooting​

Namespace remaining in terminating status​

  • Symptoms. kubectl delete namespace foundation4ai does not return, and kubectl get namespace foundation4ai shows the status Terminating.
  • Diagnosis. kubectl get namespace foundation4ai -o jsonpath='{.status.conditions}' names the resources that block the deletion. kubectl get pvc,pods -n foundation4ai shows the remaining volume claims and pods.
  • Cause. Resources with finalizers wait for a controller, for example volume claims that wait for the storage provisioner to release the volumes.
  • Resolution. Wait for the pods to stop and for the storage provisioner to delete the volumes. When the provisioner is unavailable, restore the provisioner first.
  • Actions to avoid. Removing finalizers by hand, which can leave volumes with document text in the storage backend.

Release reinstall blocked by another operation​

  • Symptoms. helm install or helm upgrade fails with a message that another operation is in progress.
  • Diagnosis. helm list -n foundation4ai --all shows the release in pending-install or pending-upgrade.
  • Cause. An earlier Helm command was interrupted before the command recorded the result.
  • Resolution. Follow the steps of Interrupted installation.
  • Actions to avoid. Deleting the Helm release Secrets in the namespace by hand, which removes the release history that a rollback needs.