Pre-installation checks

This page lists the checks that an operator runs before installing Foundation4 in the production profile, with the command for each check and the result that confirms the check. The checks catch the problems that otherwise surface as a stalled Helm installation: missing permissions, unbound volume claims, image pull failures, database privileges and malformed secrets. Operators run the checks once per cluster and again after any change to the cluster, the registry or the database.

Scope and order

Run the checks after steps 1 to 5 of Install on Kubernetes, which create the namespace, the image pull secret, the values file foundation4ai.values.yaml, the secrets file foundation4ai.secrets.env and the Secret foundation4ai-secrets. Run every command from the directory that holds the installation files. The test objects carry the label app.kubernetes.io/part-of: foundation4ai-checks, and the last section removes them.

Cluster access and permissions

Check	Command	Expected result
Target cluster	`kubectl config current-context`	The context of the target cluster
Kubernetes version	`kubectl version`	`Server Version` 1.27 or later; a version with image volume support when models or packages are used
Node architecture	`kubectl get nodes -L kubernetes.io/arch`	`amd64` in the `ARCH` column of the nodes that run Foundation4
Namespace creation	`kubectl auth can-i create namespaces`	`yes`
Cluster roles for Prometheus	`kubectl auth can-i create clusterroles`	`yes`
Cluster role bindings for Prometheus	`kubectl auth can-i create clusterrolebindings`	`yes`
Secret reads during rendering	`kubectl -n foundation4ai auth can-i get secrets`	`yes`
Job logs	`kubectl -n foundation4ai auth can-i get pods --subresource=log`	`yes`
Port forward	`kubectl -n foundation4ai auth can-i create pods --subresource=portforward`	`yes`
Pod security	`kubectl get namespace foundation4ai --show-labels`	No `pod-security.kubernetes.io/enforce=restricted` label
Namespaced resources	Resource loop below	`yes` on every line

The resource loop checks each kind of object that the two releases create:

for r in pods secrets configmaps serviceaccounts services deployments statefulsets jobs \
  persistentvolumeclaims poddisruptionbudgets horizontalpodautoscalers ingresses; do
  printf '%s: ' "$r"
  kubectl -n foundation4ai auth can-i create "$r"
done

Expected result: 12 lines, each ending in yes.

Workstation tools

Check	Command	Expected result
Helm version	`helm version --short`	A version that starts with `v3.`
Kustomize	`kubectl kustomize --help`	The help text of the `kustomize` subcommand
Secret generation	`openssl version` and `uuidgen`	A version string and a universally unique identifier (UUID)
Recipe tools, when `just` is used	`just --version`, `kustomize version` and `yq --version`	A version string from each tool

Storage and images

Check	Command	Expected result
Default StorageClass	`kubectl get storageclass`	One class marked `(default)`, or a class named in the values for NATS JetStream and Prometheus
No placeholders in the values file	`grep -n '<' foundation4ai.values.yaml`	No output
Image tags set	`grep -n -e serverImageTag: -e grpcImageTag: -e dashboardImageTag: foundation4ai.values.yaml`	3 lines, each with a tag
Pull secret type	`kubectl -n foundation4ai get secret <pull secret> -o jsonpath='{.type}'`	`kubernetes.io/dockerconfigjson`
Volume binding and image pulls	Installation test	Pod `Succeeded` and volume claim `Bound`
Registry token lifetime	The registry's documentation	Tokens that do not expire, or a refresh mechanism, as described in Charts, images and installation bundle

Installation test

The installation test requests a 10 GiB volume claim, the size of one NATS JetStream claim, and starts one container from each Foundation4 image. Save the following manifest as install-check.yaml, with the image repositories and tags from the values file. Remove the imagePullSecrets entry when the registry needs no credentials, and add storageClassName to the claim when the values name a storage class.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: install-check
  namespace: foundation4ai
  labels:
    app.kubernetes.io/part-of: foundation4ai-checks
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: install-check
  namespace: foundation4ai
  labels:
    app.kubernetes.io/part-of: foundation4ai-checks
spec:
  restartPolicy: Never
  imagePullSecrets:
  - name: <pull secret>
  containers:
  - name: server
    image: <api server repository>:<api server image tag>
    command: ["df", "-h", "/check"]
    volumeMounts:
    - name: check
      mountPath: /check
  - name: grpc
    image: <grpc service repository>:<grpc service image tag>
    command: ["true"]
  - name: dashboard
    image: <dashboard repository>:<dashboard image tag>
    command: ["true"]
  volumes:
  - name: check
    persistentVolumeClaim:
      claimName: install-check

Run the test:

kubectl apply -f install-check.yaml
kubectl -n foundation4ai wait --for=jsonpath='{.status.phase}'=Succeeded pod/install-check --timeout=300s
kubectl -n foundation4ai logs install-check -c server
kubectl -n foundation4ai get pvc install-check

Expected result: pod/install-check condition met, a df line for the file system mounted on /check, and the claim in Bound status. When the wait times out, kubectl -n foundation4ai describe pod install-check shows the cause: ErrImagePull or ImagePullBackOff for an image or credential problem, exec format error in a container log for an image built for another architecture, or an unbound claim for a storage problem.

For a mirrored registry, repeat the image test with the core release images listed in Charts, images and installation bundle. The core release images take pull credentials from separate values, so a successful test of the Foundation4 images does not cover the core release images.

Database

The database checks apply to the production profile. The evaluation profile installs PostgreSQL from the bundled chart.

Check	Command	Expected result
Connection URL format	`grep -Ec '^POSTGRES_URL=postgres(ql)?://[A-Za-z0-9_.-]+:[A-Za-z0-9]+@[^/]+/[A-Za-z0-9_]+' foundation4ai.secrets.env`	`1`
Reachability, version and privileges	Database test	Pod `Succeeded` with the values described below

The URL check confirms a postgres:// URL with a user, a password of letters and digits only, a host and a database name. The charts write the URL into a configuration file without quoting or encoding, so other characters in the password can break the configuration.

Database test

The database test runs psql inside the cluster with the URL from the Secret foundation4ai-secrets. The test uses the PostgreSQL image of the bundled chart, pgvector/pgvector:pg18-trixie, or the mirrored copy of that image. Save the following manifest as database-check.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: database-check
  namespace: foundation4ai
  labels:
    app.kubernetes.io/part-of: foundation4ai-checks
spec:
  restartPolicy: Never
  containers:
  - name: psql
    image: pgvector/pgvector:pg18-trixie
    env:
    - name: POSTGRES_URL
      valueFrom:
        secretKeyRef:
          name: foundation4ai-secrets
          key: POSTGRES_URL
    command: ["psql"]
    args:
    - "$(POSTGRES_URL)"
    - "-x"
    - "-c"
    - "SELECT current_setting('server_version') AS server_version, (SELECT default_version FROM pg_available_extensions WHERE name = 'vector') AS vector_available, (SELECT extversion FROM pg_extension WHERE extname = 'vector') AS vector_installed, has_database_privilege(current_database(), 'CREATE') AS create_on_database, has_schema_privilege('public', 'CREATE') AS create_in_public, (SELECT rolsuper FROM pg_roles WHERE rolname = current_user) AS superuser"

Run the test:

kubectl apply -f database-check.yaml
kubectl -n foundation4ai wait --for=jsonpath='{.status.phase}'=Succeeded pod/database-check --timeout=120s
kubectl -n foundation4ai logs database-check

Expected result: pod/database-check condition met and one record. The values differ by installation; the following output meets every requirement:

-[ RECORD 1 ]------+-------
server_version     | 18.1
vector_available   | 0.8.1
vector_installed   | 0.8.1
create_on_database | t
create_in_public   | t
superuser          | f

The fields are read as follows:

Server version. PostgreSQL 18, the version that the bundled chart and the Foundation4 test environment use.
Vector available. pgvector 0.8.0 or later is installed on the database server. An empty value means that the server has no pgvector package.
Vector installed. The extension exists in the database. An empty value is acceptable when the database user can create the extension, for example when superuser is t. Otherwise a database administrator runs CREATE EXTENSION vector; in the database before installation.
Create privileges. Both values are t, because Foundation4 creates the schemas public and embeddings and the tables in both.

When the pod ends in Failed status, the log shows the psql connection error. Database describes the database settings.

Secrets file

The secrets file checks read foundation4ai.secrets.env and print counts, key names or unchanged template lines, never the new secret values.

Check	Command	Expected result
Template passwords replaced	`grep -Fx -f foundation4ai.secrets.env.template foundation4ai.secrets.env`	No line that starts with `POSTGRES_SUPERUSER_PASSWORD`, `POSTGRES_PASSWORD` or `REDIS_PASSWORD`
Hexadecimal passwords	`grep -Ec -e '^POSTGRES_SUPERUSER_PASSWORD=[0-9a-f]{48,}$' -e '^POSTGRES_PASSWORD=[0-9a-f]{48,}$' -e '^REDIS_PASSWORD=[0-9a-f]{48,}$' foundation4ai.secrets.env`	`3`
Master key identifier is a UUID	`grep -Ec '^FOUNDATION4AI_APP_MASTER_KEY=[0-9a-fA-F]{8}(-[0-9a-fA-F]{4}){3}-[0-9a-fA-F]{12}$' foundation4ai.secrets.env`	`1`
Master secret is hexadecimal	`grep -Ec '^FOUNDATION4AI_APP_MASTER_SECRET=[0-9a-f]{48,}$' foundation4ai.secrets.env`	`1`
Application secret is a Fernet key	`grep -Ec '^FOUNDATION4AI_APP_SECRET=[A-Za-z0-9_-]{43}=$' foundation4ai.secrets.env`	`1`
License value present	`grep -Ec '^FOUNDATION4AI_APP_LICENSE=.+' foundation4ai.secrets.env`	`1`, with `pending` until the license is received
Secret matches the file	`kubectl -n foundation4ai describe secret foundation4ai-secrets`	Every key of the secrets file with a size in bytes, and no values

A Fernet key is 32 random bytes encoded in URL-safe Base64: 44 characters that end in =. The command openssl rand -base64 32 | tr '+/' '-_' produces a Fernet key. Secrets and keys describes each secret.

Ingress and Gateway API

These checks apply when the values enable ingress or httpRoute.

Check	Command	Expected result
Ingress controller	`kubectl get ingressclass`	The class named in `ingress.className`
Gateway API resources	`kubectl get crd httproutes.gateway.networking.k8s.io`	The resource definition is listed
Gateway	`kubectl get gateways -A`	The Gateway named in `httpRoute.parentRefs`
Transport Layer Security (TLS) certificate Secret	`kubectl -n foundation4ai get secret <tls secret> -o jsonpath='{.type}'`	`kubernetes.io/tls`

API and dashboard access describes both access methods.

Cleanup

Remove the test objects before installing the releases:

kubectl -n foundation4ai delete pod,pvc -l app.kubernetes.io/part-of=foundation4ai-checks

Expected result: pod "install-check" deleted, pod "database-check" deleted and persistentvolumeclaim "install-check" deleted.

Scope and order​

Cluster access and permissions​

Workstation tools​

Storage and images​

Installation test​

Database​

Database test​

Secrets file​

Ingress and Gateway API​

Cleanup​