LLM Knowledge base support¶

It is possible to use a knowledge base to improve the accuracy of the LLM's responses. Knowledge bases can be configured in the prompts file, under the knowledge key.

Each knowledge base has its own retrieval strategy. Depending on the retrieval strategy, the knowledge base will function differently.

Configuring a knowledge base¶

WARNING: This strategy is for internal use only. It is not possible to use Assistants from a "bring your own" OpenAI account.

In an LLM prompts file, a knowledge base is configured by adding a knowledge key:

prompts:
  - id: faq
    text: |
      system: use the given knowledge base to answer the users question.
      [!kb]
      user: {{ question }}
knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    managed_openai_assistant: {}

This can then be used in bubblescript as follows:

dialog main do
  ask "What is the capital of France?"
  answer = LLM.complete(@prompts.faq, kb=@knowledge.my_knowledge_base, question=answer.text)
  say answer.text
end

In this case the AI section of the studio will have a 'knowledge bases' section, where you can upload files to the knowledge base.

Retrieval strategies¶

There are several strategies to use a knowledge base.

Managed OpenAI Vector store (Responses API)¶

The managed_openai_responses strategy automatically creates and manages an OpenAI Vector store for your knowledge base. Files uploaded to the bot's file system are automatically synchronized with the vector store.

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    managed_openai_responses: {}

This strategy will: - Automatically create an OpenAI Vector store for your bot - Synchronize files from the bot's file system to the vector store - Use the OpenAI Responses API with the file_search tool to search the vector store and incorporate the results in the LLM response.

It is possible to configure the number of results to return with the max_num_results field:

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    managed_openai_responses:
      max_num_results: 5

This example will retrieve up to 5 results from the vector store, instead of the default of 3.

Managed OpenAI Assistant (to be deprecated)¶

The managed_openai_assistant strategy automatically creates and manages an OpenAI Assistant for your knowledge base. Files uploaded to the bot's file system are automatically synchronized with the assistant's vector store.

WARNING: This strategy is deprecated and will be removed in the future. From release 2.46, newly created bots will use the managed_openai_responses strategy instead.

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    managed_openai_assistant:
      provider: "openai"

This strategy will: - Automatically create an OpenAI Assistant for your bot - Create a vector store attached to the assistant - Synchronize files from the bot's file system to the vector store - Use the assistant's file search capabilities to find relevant documents

The provider field should be set to openai or microsoft_openai and must match the provider of the LLM prompt that uses this knowledge base.

Files are automatically synchronized when: - Files are uploaded to the bot's file system - Files are renamed or moved - Files are deleted

The assistant and vector store are created automatically when first needed, and are named based on your bot ID and knowledge base ID.

Scripts collection¶

The scripts_collection strategy uses a collection of scripts as a knowledge base:

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    scripts_collection:
      collection: "kb/"

This will read all the scripts in the kb/ directory and use them as a knowledge base, by directly including the content of the scripts in the prompt, on the place where the [!kb] tag is set in the prompt definition.

Vertex AI RAG Engine (fixed corpus)¶

The fixed_vertex_rag strategy uses a Vertex AI RAG Engine corpus you manage in Google Cloud. When the LLM runs, the app calls Vertex AI generateContent (Google AI / Gemini) with a retrieval tool that points at your rag_corpus (vertexRagStore), so the model retrieves from the corpus during generation. Authentication uses the same service account JSON as other Google Cloud features via BotsiWeb.Service.Google.

Requirements:

A RAG corpus already exists in Vertex AI; use the full resource name, which must include the same location segment you configure.
The configured GCP project in the service account credentials must match the projects/{project_id} prefix of rag_corpus.

knowledge:
  - id: "my_vertex_kb"
    label: "Vertex RAG"
    fixed_vertex_rag:
      location: "europe-west4"
      rag_corpus: "projects/my-gcp-project/locations/europe-west4/ragCorpora/my-corpus"

Optional fields:

Field	Description
`context_role`	Reserved for YAML compatibility; not used (retrieval is not injected as a separate message).
`top_k`	Passed to the RAG store as `similarityTopK` when set.
`vector_distance_threshold`	Passed in the RAG store `ragRetrievalConfig` filter as `vectorDistanceThreshold` when set.
`metadata_filter`	CEL metadata filter string in `ragRetrievalConfig` when set.

RAG Engine is only available in certain regions; some US regions may require allowlisting per Google’s documentation.

Vertex AI RAG Engine (managed corpus)¶

The managed_vertex_rag strategy creates (or reuses) a Vertex RAG corpus per bot and knowledge base id, syncs files from the bot filesystem into that corpus via Cloud Storage and the Import RAG files API, then uses the same Gemini generateContent + vertexRagStore retrieval tool flow as fixed_vertex_rag when the LLM runs.

Configuration in config.exs (and per-environment overrides) for BotsiCommon.LLM.Knowledge.GoogleVertexRagManager:

Key	Description
`prefix`	Prefix for corpus `displayName`, similar to managed OpenAI resources (e.g. `dev` or `test`).
`import_bucket`	GCS bucket name where bot files are staged as `gs://…` objects before import. The bucket must be readable by Vertex RAG in the same GCP project as the corpus.
`enabled`	When `false`, BuildHooks for this listener are skipped (used in the test environment). Defaults to `true` when omitted.

knowledge:
  - id: "my_vertex_kb"
    label: "Vertex RAG managed"
    managed_vertex_rag: {}

Optional fields match fixed_vertex_rag where applicable: context_role, top_k, vector_distance_threshold, metadata_filter.

File metadata keys written by the sync hooks: vertex_rag_file_resource (full RagFile resource name for deletion) and vertex_rag_file_error on failure.

Imports run as long-running operations; indexing may lag slightly behind upload. See the RAG Engine overview for quotas and regions.

Internal retrieval strategies¶

WARNING: These strategies are for internal use only. It is not possible to use Assistants from a "bring your own" OpenAI account.

OpenAI Responses API (hardcoded)¶

The fixed_openai_responses strategy is used to connect to an OpenAI Vector store by hardcoding the vector store ID.

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    fixed_openai_responses:
      vector_store_id: "vs_1234567890"
      max_num_results: 1

OpenAI Assistant through integration¶

The external_openai_assistant strategy is used to connect to an OpenAI Assistant by configuring the assistant ID in an integration secret.

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    external_openai_assistant:
      assistant_integration_alias: "my_assistant"
      # optional reference to alias where the project ID is stored
      project_integration_alias: "my_project"

And then in the integrations file, add a new entry like this:

- provider: secret
  alias: my_assistant
  context: bot
  description: Assistant ID for 'My Knowledge Base'
- provider: secret
  alias: my_project
  context: bot
  description: Project ID for 'My Knowledge Base'

No automatic synchronization of files is done from the platform to the assistant. With this retrieval strategy, you need to manually add files to the assistant on the OpenAI platform.

OpenAI Assistant (hardcoded)¶

The "fixed" OpenAI Assistant strategy. This strategy is configured using a hardcoded OpenAI Assistant ID to find the most relevant documents to include in the prompt.

knowledge:
  - id: "my_knowledge_base"
    label: "My Knowledge Base"
    fixed_openai_assistant:
      provider: openai
      assistant_id: "asst_1234567890"

To use this strategy, you need to provide the assistant_id of an OpenAI Assistant. For the openai provider, you can create an assistant here. For the microsoft_openai provider, you can create an assistant here. The provider should be set to openai or microsoft_openai; it decides which API endpoint to use. The provider of the knowledge base must match the provider of the LLM prompt that is using the knowledge base.

No automatic synchronization of files is done from the platform to the assistant. With this retrieval strategy, you need to manually add files to the assistant on the OpenAI platform.