Skip to main content

Personal project spaces

A personal project space holds the documents that one end user loads for one project, such as specifications, research notes or meeting transcripts, so that searches and answers for the project draw on that content. Foundation4 stores the spaces of every user in one pipeline and places each space at a document path that the client application builds from the user and the space. The client application provides the user accounts, the screens that create spaces and add documents, and the conversion of files to text. This page describes the division of work, the path layout, the pipeline, the Foundation4 request behind each user action and the two ways to keep the spaces of different users apart. Integrators who build a client application with project spaces and security reviewers who assess one need this page.

Division of work​

ComponentResponsibility
Client applicationSigns in users, stores the name of each space and folder, converts files to text, builds the path of every request from the signed-in user, and displays results and answers
Foundation4Stores and processes the documents of every space, limits each search and answer to the path pattern of the request, and enforces the permissions and classification allow-list of each API key
Model serverGenerates the answers to questions about a space, for the agent that the client application runs

The user interface, the supported file formats and the records of spaces and folders belong to the client application. The following table lists each user action and the Foundation4 request that the client application sends for the action:

User actionClient applicationFoundation4 request
Create a spaceAssigns a space label and stores the space nameNone
Add a documentConverts the file to text and builds the path from the user, the space and the folderPOST /pipelines/{id}/documents
Browse a spaceLists the folders and the documents of the spaceGET /pipelines/{id}/paths and GET /pipelines/{id}/documents with a pattern
Search a spaceSends the query with the pattern of the spacePOST /pipelines/{id}/search
Ask about a spaceSends the question with the pattern of the spacePOST /agents/{id}/execute
Replace or move a documentSubmits a new version with the same external identifierPOST /pipelines/{id}/documents
Remove a documentDeletes the documentDELETE /pipelines/{id}/documents/{document_id}
Rename a spaceChanges the stored space nameNone
Delete a spaceLists the documents of the space and deletes each documentGET /pipelines/{id}/documents, then DELETE for each document

Path layout​

The path of each document has four labels, spaces.<user>.<space>.<folder>, as defined in Document paths:

  • User label. An opaque label derived from the account identifier in the client application, such as u-1842. A label holds only lowercase letters, digits, hyphens and underscores, and every key that lists the pipeline's paths reads the labels, so a path never carries a user name or an email address.
  • Space label. An opaque label that the client application assigns when the user creates the space, such as p-07. The client application stores the space name, so renaming a space changes no document.
  • Folder label. The folder that the user selects when adding a document, such as notes or specs, or a default folder such as main. Every document sits in a folder and never at the space path itself.

A folder label converted from a folder name changes when the user renames the folder, and the client application then resubmits every document of the folder with the new path. A client application that lets users rename folders therefore assigns opaque folder labels as well.

With this layout, one pattern covers each scope of a request:

ScopePattern
One folderspaces.u-1842.p-07.specs
One spacespaces.u-1842.p-07.>
Every space of one userspaces.u-1842.>

Each document also carries the following values:

  • External identifier. The client application's own identifier of the document, such as doc-58213, unique across the pipeline. Submissions with the same external identifier in the same pipeline create versions of one document, so an identifier that two users can share, such as a file name, lets one user's upload replace and move the other user's document.
  • Title. The file name or document title in the title metadata field, so that search results and answer sources name the document.

Pipeline​

One pipeline holds the spaces of every user, with the following settings:

  • Classifications. spaces, the classification of every document when the client application separates users, as described in Separation by the client application. Separation by Foundation4 adds one classification for each user.
  • Metadata schema. title as a string.
  • Full-text index. full_text_search is fts, so that searches match exact terms, such as part numbers, and hybrid searches combine full-text and vector results.
  • Text splitter. A text splitter with fragments of up to 1,000 characters, created as described in Knowledge base question answering.
curl -X POST "$FOUNDATION4_URL/pipelines" \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
-H "Content-Type: application/json" \
-d '{
"name": "project-spaces",
"description": "Personal project spaces",
"embedding_model_id": "'"$EMBEDDING_MODEL_ID"'",
"default_text_splitter_id": "'"$TEXT_SPLITTER_ID"'",
"classifications": ["spaces"],
"full_text_search": "fts",
"schema": {
"type": "object",
"properties": {
"title": {"type": "string"}
},
"required": ["title"]
}
}'

Expected result: status 201 and the new pipeline. Set $PIPELINE_ID from the id field.

Documents​

Foundation4 accepts documents as plain text and accepts no files. The client application converts each file to text, such as the text of a PDF or a word processing file, and sends the text with the path of the folder that the user selected:

curl -X POST "$FOUNDATION4_URL/pipelines/$PIPELINE_ID/documents" \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
-H "Content-Type: application/json" \
-d '{
"classification": "spaces",
"external_identifier": "doc-58213",
"path": "spaces.u-1842.p-07.specs",
"metadata": {"title": "Pump controller requirements"},
"contents": "Pump controller requirements\n\nThe controller stops the pump when the inlet pressure stays below 0.8 bar for more than 5 seconds."
}'

Expected result: status 201 and the new document, with status set to pending and path set to the path of the request. Set $DOCUMENT_ID from the id field. The client application retrieves the document with GET /pipelines/{id}/documents/{document_id} until status is success, and shows the document as ready from then on, as described in Ingest documents reliably. A request body is limited to 10 MiB by default, so the text of a large file is divided into several documents, as described in Request size.

Each version carries its own path, so every new version of a document sends path. A new version without path has no path and leaves every space. The client application changes a document in the following ways:

  • Replace. A new version with the same external identifier, the same path and the new text. The previous version expires at submission, and the new version is searchable once the new version reaches success.
  • Move. A new version with the same external identifier and text and the path of another folder or another space of the same user. Foundation4 processes the moved document again, including new embeddings.
  • Remove. A delete request, which removes every version and every fragment of the document permanently.
curl -X DELETE "$FOUNDATION4_URL/pipelines/$PIPELINE_ID/documents/$DOCUMENT_ID" \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET"

Expected result: status 204.

Space contents​

The path list returns the folders of a space that hold at least one current document, with the number of documents in each folder. The -G and --data-urlencode options of curl encode the pattern as a query parameter:

curl -G "$FOUNDATION4_URL/pipelines/$PIPELINE_ID/paths" \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
--data-urlencode "path=spaces.u-1842.p-07.>"

Expected result: status 200 and the folders of the space in byte order:

{
"data": [
{"path": "spaces.u-1842.p-07.notes", "documents": 4},
{"path": "spaces.u-1842.p-07.specs", "documents": 2}
],
"next": null
}

The path list omits a folder or a space without a current document, so the client application keeps the spaces and folders of each user in its own records. The document list returns the documents of one folder, or of the whole space with the pattern of the space:

curl -G "$FOUNDATION4_URL/pipelines/$PIPELINE_ID/documents" \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
--data-urlencode "path=spaces.u-1842.p-07.specs" \
--data-urlencode "first=100"

Expected result: status 200 and the documents of the folder in data, each with id, path, status and metadata, and the cursor of the next page in page_info, as described in Paginate and filter lists.

Search in a space​

The client application searches one space with the pattern of the space. A hybrid search matches the meaning of the query and the exact terms of the query:

curl -X POST "$FOUNDATION4_URL/pipelines/$PIPELINE_ID/search" \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
-H "Content-Type: application/json" \
-d '{
"query": "When does the controller stop the pump?",
"type": "hybrid",
"classification": "spaces",
"path": "spaces.u-1842.p-07.>",
"params": {"k": 5}
}'

Expected result: status 200 and up to 5 fragments of the space, ordered by descending fused score. The following response is shortened to one fragment:

[
{
"id": "019a7c2e-4b1d-7e3f-9a20-6c5d8e7f1a2b",
"classification": "spaces",
"metadata": {"title": "Pump controller requirements"},
"path": "spaces.u-1842.p-07.specs",
"document_id": "019a7c2d-9f3e-7a41-8b52-3d4e5f6a7b8c",
"page_content": "Pump controller requirements\n\nThe controller stops the pump when the inlet pressure stays below 0.8 bar for more than 5 seconds.",
"version": 1794668400000000,
"expired_at": null,
"score": 0.0328,
"order_id": 0
}
]

A search request without path covers only the documents without a path, so a request that omits the pattern returns no fragment of any space. Set up hybrid search describes the fused score, and Search requests lists the other search types and fields.

Questions about a space​

An agent answers questions from the documents of one space when the execution request names the pattern of the space. The agent of Answers from Foundation4 serves project spaces without change, because each execution request names the pipeline, the LLM and the path:

curl -N -X POST "$FOUNDATION4_URL/agents/$AGENT_ID/execute" \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
-H "x-pipeline-id: $PIPELINE_ID" \
-H "x-llm-id: $LLM_ID" \
-H "Content-Type: application/json" \
-d '{
"prompt": {"question": "When does the controller stop the pump?"},
"classification": "spaces",
"path": "spaces.u-1842.p-07.>",
"stream": "sse",
"tracing": true
}'

Expected result: status 200, the x-foundation4ai-tracing-id response header and the answer as server-sent events. The pattern applies to every retrieval placeholder of the agent, so the answer draws only on the documents of the space. Sources describes how the client application lists the documents behind an answer from the execution trace. A client application that calls its own LLM instead searches the space and builds the prompt from the search results, as described in Retrieval only.

Space deletion​

Foundation4 has no request that deletes the documents under a path. The client application deletes a space in the following steps:

  1. List the documents of the space with path set to spaces.u-1842.p-07.> and first set to 1000.
  2. Delete each listed document with DELETE /pipelines/{id}/documents/{document_id}.
  3. Repeat from step 1 until the list returns an empty data array. Deleted documents leave the list, so each pass reads the first page again.

Expected result: an empty document list for the pattern of the space, and no folder of the space in the path list. The client application then removes the space from its own records. The same steps with the pattern spaces.u-1842.> delete every space of a user, for example when the user's account closes.

Separation of users​

Document paths organize the spaces and restrict no access: every key that can search the pipeline can search under any path, as described in Document paths. A client application keeps the spaces of different users apart in one of two ways:

ApproachEnforced byAPI keysAI assistants
Separation by the client applicationThe code of the client applicationOne service key on the client application's serverNot supported
Separation by Foundation4The classification allow-list of each user's keyOne key for each user, and a provisioning key on the client application's serverSupported with the user's key

Separation by the client application​

One service key and the classification spaces serve every user. Foundation4 searches under any pattern that the service key names and holds no record of which user a path belongs to, so the client application applies the following rules to every request:

  • Server-side key. The service key stays on the client application's server and never reaches a browser, a mobile application or an AI assistant.
  • User label from the session. The client application derives the user label from the signed-in account and never from a value in the user's request.
  • Patterns from the user label. Every pattern that the client application sends for a user starts with spaces.<user>., and no request for a user names >, * or spaces.>.
  • Document ownership. Before acting on a document identifier received from the user's browser, the client application reads the document and confirms that the document's path starts with the user's prefix.
  • No AI assistant access. An AI assistant connected over the Model Context Protocol (MCP) chooses the path of each tool call itself, so an assistant with the service key reaches every space.

A defect in these rules exposes the documents of other users, because every space shares the service key and the classification.

Separation by Foundation4​

Each user receives a classification of the pipeline and an API key whose classification allow-list holds only that classification. Foundation4 then refuses each request of the user's key that reaches the documents of another user, whatever path the request names. Access control describes allow-lists.

A provisioning key on the client application's server prepares each new user in three requests. The first request adds the user's classification to the pipeline:

curl -X POST "$FOUNDATION4_URL/pipelines/$PIPELINE_ID/classifications" \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
-H "Content-Type: application/json" \
-d '{"add": ["u-1842"], "remove": []}'

Expected result: status 200 and an empty body. The second request creates the user's key:

curl -X POST "$FOUNDATION4_URL/api-keys" \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
-H "Content-Type: application/json" \
-d '{
"name": "spaces-u-1842",
"description": "Project spaces of user u-1842"
}'

Expected result: status 201 and the new key. Foundation4 returns the secret only in this response, so the client application stores the secret in its secret store. Set $USER_KEY_ID from the id field. The third request grants the user's key the permissions of the API keys table, with the allow-list u-1842:

curl -X POST "$FOUNDATION4_URL/api-keys/$USER_KEY_ID/permissions" \
-H "x-api-key: $FOUNDATION4_API_KEY" \
-H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
-H "Content-Type: application/json" \
-d '{
"permissions": [
{"object_type": "pipelines", "object_id": "'"$PIPELINE_ID"'", "permission": 5},
{"object_type": "text-splitters", "object_id": "'"$TEXT_SPLITTER_ID"'", "permission": 1},
{"object_type": "agents", "object_id": "'"$AGENT_ID"'", "permission": 5},
{"object_type": "llms", "object_id": "'"$LLM_ID"'", "permission": 5}
],
"classifications": ["u-1842"]
}'

Expected result: status 200 and an empty body. The client application then sends the user's requests with the user's key, and every document and request names the classification u-1842 in place of spaces. Foundation4 applies the following rules to the user's key:

  • Search and answers. A search or an agent execution that names another user's classification returns HTTP 403 and no fragment, whatever the path.
  • Lists. The path list and the document list include only documents with the user's classification, so the key lists only the spaces of the user.
  • Other users' documents. Reading another user's document returns HTTP 403. A new version with the external identifier of another user's document returns HTTP 400 Classification mismatch and leaves the document unchanged. A document that the key adds under another user's path carries the key's own classification, so the other user's requests never return the document.
  • Pipeline changes. The key holds no write permission, so the key cannot change or delete the pipeline or delete documents.

The approach has the following consequences for the client application:

  • AI assistants. A user can connect an AI assistant with the user's key, as described in Connect an AI assistant over MCP. The assistant reaches only the user's spaces, whatever paths the assistant names.
  • Deletions. Write permission on a pipeline also allows changing and deleting the pipeline, so the user's key holds none. The client application removes documents and spaces with the provisioning key, after confirming that each document carries the user's classification.
  • Account removal. Removing the classification with {"add": [], "remove": ["u-1842"]} deletes every document and fragment of the user, and the classification's encryption key, in one request, as described in Classifications. DELETE /api-keys/{id} then deletes the user's key.
  • Encryption. Each classification has a dedicated encryption key, so the fragment text of each user is encrypted under a separate key, as described in Data protection.
  • License. A license can limit the number of API keys, and each user's key counts toward the limit, as described in Licensing.
  • Secrets. The client application stores one secret for each user and replaces a key as described in Key lifecycle.

API keys​

The keys are created as described in Scoped keys:

KeyApproachPermissions
Service keySeparation by the client applicationAll permissions (7) on the pipeline with the allow-list spaces, execute (1) on the pipeline's text splitter, and read and execute (5) on the agent and the LLM
User keySeparation by Foundation4Read and execute (5) on the pipeline with the allow-list of the user's classification, execute (1) on the pipeline's text splitter, and read and execute (5) on the agent and the LLM
Provisioning keySeparation by Foundation4All permissions (7) on the pipeline with the allow-list *, write (2) on the api-keys object type, execute (1) on the pipeline's text splitter, and read and execute (5) on the agent and the LLM

The provisioning key holds the permissions that the key grants to each user's key, because a key can grant only what the key holds. The service key and the provisioning key stay on the client application's server.