Knowledge Base

A common bottleneck of LLMs is imposed by their limited context windows. In medium to big coding projects this limitation can be easily observed through the high number of hallucinations the models make when dealing with complex tasks.

Project specific information such as architecture documents, coding standards, and design guidelines are often too large to fit in the context window, yet they are essential for the LLM to deliver accurate responses.

The Knowledge Base addresses this problem by acting as an external, searchable memory for your project documents. Instead of stuffing everything into the prompt, documents are indexed once and made available to the LLM on demand through dedicated search tools that have RAG capabilities.

Usage

The Knowledge Base is populated manually by the user. Documents such as architecture docs, coding standards, design guidelines are indexed once, then made available to the LLM on demand.

The indexing pipeline follows these steps:

• Documents are converted to plain text using a converter.

• The text is split into chunks and each chunk is transformed into a vector embedding using the configured Embedding Model.

• The resulting index is stored locally and persists across sessions.

Adding documents

To add documents to the knowledge base, click the Load Files… button located in the dedicated chat page. Documents are indexed automatically upon addition if an embedding model is configured.

Note

If no embedding model is configured, documents are still added to the knowledge base, but only the textual_search_knowledge_base tool will be available during retrieval.

Managing the embedding model

• Use the DVT AI: Set Embedding Model command to change the embedding model.

• You can enforce the usage of a specific embedding model (See Advanced Configuration).

Note

Changing the embedding model requires reindexing all documents.

Advanced Configuration

The Knowledge Base can be customized to support more advanced use cases. For instance:

• Selecting which converter to use for each document format

• Adjusting how documents are split into chunks (size, overlap)

• Sharing a knowledge base with team members

Converters

A converter transforms a file from its original format into plain text (usually Markdown), making its content available for indexing and retrieval.

Converter Configuration

Converters can be configured through a dedicated configuration object that defines specific options for each converter. AI Assistant looks for the converter configuration file in the following locations:

• $DVT_HOME/config/ai/converters.{json,jsonc,js,ts}

• $DVT_AI_DIR/converters.{json,jsonc,js,ts}

• $HOME/.dvt/ai/converters.{json,jsonc,js,ts}

• <project>/.dvt/ai/converters.{json,jsonc,js,ts}

JSON / JSONC

{
    "version": 10,
    "configs": [
        {
            "strategy": "tika",
            "from": ["pdf", "docx", "pptx", "xlsx"],
            "maxFileSizeBytes": 52428800
        },
        {
            "strategy": "internal-converter",
            "from": ["pdf", "docx"],
            "maxFileSizeBytes": 52428800,
            "strategyOptions": {
                "endpoint": "http://parsing.internal.service/parse"
            }
        }
    ]
}

TypeScript

import { ConvertersConfig } from "./@api/v10"

export default {
    version: 10,

    configs: [
        {
            strategy: "tika",
            from: ["pdf", "docx", "pptx", "xlsx"],
            maxFileSizeBytes: 50 * 1024 * 1024
        },
        {
            strategy: "internal-converter",
            from: ["pdf", "docx"],
            maxFileSizeBytes: 50 * 1024 * 1024,
            strategyOptions: {
                endpoint: "http://parsing.internal.service/parse"
            }
        }
    ]
} satisfies ConvertersConfig

JavaScript

export default {
    version: 10,

    configs: [
        {
            strategy: "tika",
            from: ["pdf", "docx", "pptx", "xlsx"],
            maxFileSizeBytes: 50 * 1024 * 1024
        },
        {
            strategy: "internal-converter",
            from: ["pdf", "docx"],
            maxFileSizeBytes: 50 * 1024 * 1024,
            strategyOptions: {
                endpoint: "http://parsing.internal.service/parse"
            }
        }
    ]
}

Properties

• version - required, number

• configs - required, array of dynamic configuration

Each dynamic configuration has the following properties:

• strategy - required, string representing the converter strategy name. Must correspond to an available predefined or custom converter

• from - required, list of file formats (extensions) this converter handles

• maxFileSizeBytes - required, number representing the maximum file size in bytes that this converter will process

• strategyOptions - optional, strategy-specific options passed to the converter factory at runtime. Converter factories can use this to receive additional data they require (e.g. API keys, endpoint URLs).

Predefined Converters

AI Assistant comes with a set of predefined converters for converting popular file formats (e.g pdf, docx, pptx). Use the following values to set the converters through file-base dynamic configurations.

Tika

• strategy: tika

• from: defaults to pdf, xlsx, pptx, docx

• maxFileSizeBytes: defaults to 50MB

Llama-Cloud

This converter uses the LlamaParse cloud API. See the Official Documentation for more details.

• strategy: llama-cloud

• from: defaults to pdf,

• maxFileSizeBytes: defaults to 50MB

• strategyOptions:

  • apiKey: defaults to the value of LLAMA_CLOUD_API_KEY environment variable

Note

All of the converters work out of the box except Llama-Cloud, which requires the LLAMA_CLOUD_API_KEY to be set.

Custom Converters

AI Assistant allows you to define custom converters by providing a converter factory. A converter factory is a module that registers a new conversion strategy and provides the logic for converting documents into Markdown.

AI Assistant looks for converter factories in the following locations:

• $DVT_HOME/config/ai/converter-factories/*.{ts,js}

• $DVT_AI_DIR/converter-factories/*.{ts,js}

• <project>/.dvt/ai/converter-factories/*.{ts,js}

• $HOME/.dvt/ai/converter-factories/*.{ts,js}

The strategy field exported by the factory must match the strategy value used in the converter configuration.

TypeScript

import { ConvertedDocument, Converter, ConverterFactory, DefaultConverterConfig } from "../@api/v10"

const INTERNAL_SERVERL_URL = "http://parsing.internal.service";

export default {
    version: 10,

    strategy: 'internal-converter',

    getDefaultConverterConfig: async (): Promise<DefaultConverterConfig> => {
        return {
            from: ["pdf", "docx"],
            maxFileSizeBytes: 50 * 1024 * 1024
        }
    },

    getConverter: async (config, abortSignal): Promise<Converter> => {
        return {
            config,
            convert: async (inputDocument): Promise<ConvertedDocument> => {
                const filePath = inputDocument.filePath;

                const response = await fetch(`${INTERNAL_SERVERL_URL}/parse`, {
                    method: 'POST',
                    headers: { 'Content-Type': 'application/json' },
                    body: JSON.stringify({ filePath }),
                    signal: abortSignal,
                });

                if (!response.ok) {
                    const errorBody = await response.json().catch(() => ({}));
                    throw new Error(
                        errorBody.error ?? `Parsing server error: ${response.status} ${response.statusText}`
                    );
                }

                const result = await response.json();
                return {
                    text: result,
                    filePath: inputDocument.filePath,
                    documentId: inputDocument.documentId
                };
            },
        }
    }
} satisfies ConverterFactory

JavaScript

const INTERNAL_SERVERL_URL = "http://parsing.internal.service";

export default {
    version: 10,

    strategy: 'internal-converter',

    getDefaultConverterConfig: async () => {
        return {
            from: ["pdf", "docx"],
            maxFileSizeBytes: 50 * 1024 * 1024
        }
    },

    getConverter: async (config, abortSignal) => {
        return {
            config,
            convert: async (inputDocument) => {
                const filePath = inputDocument.filePath;

                const response = await fetch(`${INTERNAL_SERVERL_URL}/parse`, {
                    method: 'POST',
                    headers: { 'Content-Type': 'application/json' },
                    body: JSON.stringify({ filePath }),
                    signal: abortSignal,
                });

                if (!response.ok) {
                    const errorBody = await response.json().catch(() => ({}));
                    throw new Error(
                        errorBody.error ?? `Parsing server error: ${response.status} ${response.statusText}`
                    );
                }

                const result = await response.json();
                return {
                    text: result,
                    filePath: inputDocument.filePath,
                    documentId: inputDocument.documentId
                };
            },
        }
    }
}

Knowledge Base Pipeline Configuration

The Knowledge Base can be configured through a dedicated configuration object. AI Assistant looks for the knowledge base pipeline configuration file in the following locations in this exact order:

• $DVT_HOME/config/ai/rag_pipeline.{json,jsonc,js,ts}

• $DVT_AI_DIR/rag_pipeline.{json,jsonc,js,ts}

• <project>/.dvt/ai/rag_pipeline.{json,jsonc,js,ts}

• $HOME/.dvt/ai/rag_pipeline.{json,jsonc,js,ts}

JSON / JSONC

{
    "version": 10,

    "converters": [
        { "from": "pdf", "strategy": "tika" },
        { "from": "docx", "strategy": "tika" },
        { "from": "xlsx", "strategy": "tika" }
    ],

    "embeddingConfig": {
        "maxChunkSize": 1000,
        "chunkOverlap": 40,
        "embedder": {
            "providerName": "openai",
            "modelName": "text-embedding-3-small"
        }
    },
    "storageConfig": {
        "storagePath": "$HOME/.dvt/ai/vector_db"
    }
}

TypeScript

import { RAGPipelineConfig } from "./@api/v10"

export default {
    version: 10,

    converters: [
        { from: "pdf", strategy: "tika" },
        { from: "docx", strategy: "tika" },
        { from: "xlsx", strategy: "tika" },
    ],

    embeddingConfig: {
        maxChunkSize: 1000,

        chunkOverlap: 40,

        embedder: {
            providerName: "openai",
            modelName: "text-embedding-3-small"
        }
    },

    storageConfig: {
        storagePath: "$HOME/.dvt/ai/vector_db"
    }
} satisfies RAGPipelineConfig

JavaScript

export default {
    version: 10,

    converters: [
        { from: "pdf", strategy: "tika" },
        { from: "docx", strategy: "tika" },
        { from: "xlsx", strategy: "tika" },
    ],

    embeddingConfig: {
        maxChunkSize: 1000,

        chunkOverlap: 40,

        embedder: {
            providerName: "openai",
            modelName: "text-embedding-3-small"
        }
    },

    storageConfig: {
        storagePath: "$HOME/.dvt/ai/vector_db"
    }
}

Properties

version - required, number

converters - optional, array of objects having the following properties

• from (string) - The source file format (e.g. pdf, docx, md).

• strategy (string) - The converter strategy name. Must correspond to an available configured converter. If multiple entries match the same format, the first one takes precedence.

embedding - optional, object

• maxChunkSize (number) - The maximum size of a chunk in tokens. Controls the granularity of the input data. This value must be less than the max input size of the configured embedding model.

• chunkOverlap (number) - The number of tokens that overlap between consecutive chunks. Overlapping chunks help preserve context across boundaries.

• embedder.providerName (string) - The provider name. Must match a provider configured in the model configuration.

• embedder.modelName (string) - The embedding model name. Must exist in the provider’s configured models list.

storage - optional, object

• storagePath (string) - Absolute file path to the locally stored database file. Supports environment variable expansion (e.g. $HOME, $DVT_HOME).

Note

When you change the embedding model using the DVT AI: Set Embedding Model command or by modifying the configuration, all previously indexed documents must be reindexed to remain compatible with the new model.

Examples

Configuring The Knowledge Base to point to a shared location

JSON / JSONC

{
    "version": 10,

    "storageConfig": {
        "storagePath": "$SHARED_DIRECTORY/.dvt/ai/vector_db"
    }
}

TypeScript

import { RAGPipelineConfig } from "./@api/v10"

export default {
    version: 10,

    storageConfig: {
        storagePath: "$SHARED_DIRECTORY/.dvt/ai/vector_db"
    }
} satisfies RAGPipelineConfig

JavaScript

export default {
    version: 10,

    storageConfig: {
        storagePath: "$SHARED_DIRECTORY/.dvt/ai/vector_db"
    }
}

Configuring the Knowledge Base to use a custom converter

rag_pipeline.json

{
    "version": 10,

    "converters": [
        { "from": "pdf", "strategy": "docling" },
        { "from": "docx", "strategy": "tika" },
        { "from": "xlsx", "strategy": "tika" }
    ],

    "storageConfig": {
        "storagePath": "$SHARED_DIRECTORY/.dvt/ai/vector_db"
    }
}

converter-factories/docling.ts

import { ConvertedDocument, Converter, ConverterFactory, DefaultConverterConfig } from "../@api/v10";


const DOCLING_SERVER_URL = "http://127.0.0.1:8001";

export default {

    version: 10,

    strategy: 'docling',

    getDefaultConverterConfig: async (): Promise<DefaultConverterConfig> => {
        return {
            from: ['pdf'],
            maxFileSizeBytes: 50 * 1024 * 1024,
        }
    },

    getConverter: async (config, abortSignal): Promise<Converter> => {
        return {
            config,

            convert: async (inputDocument): Promise<ConvertedDocument> => {
                const filePath = inputDocument.filePath;

                const response = await fetch(`${DOCLING_SERVER_URL}/parse`, {
                    method: 'POST',
                    headers: { 'Content-Type': 'application/json' },
                    body: JSON.stringify({ filePath }),
                    signal: abortSignal,
                });

                if (!response.ok) {
                    const errorBody = await response.json().catch(() => ({}));
                    throw new Error(
                        errorBody.error ?? `Docling server error: ${response.status} ${response.statusText}`
                    );
                }

                const result = await response.json();
                return {
                    text: result.text,
                    filePath: inputDocument.filePath,
                    documentId: inputDocument.documentId
                }
            },
        }

    },


} satisfies ConverterFactory;

doclingServer.py

from flask import Flask, request, jsonify
from pathlib import Path
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions, TableStructureOptions, TableFormerMode

import hashlib

app = Flask(__name__)

SUPPORTED_EXTENSIONS = {".pdf"}
PAGE_BREAK = "<!-- page break -->"
ARTIFACTS_PATH = "~/.cache/docling/models"

def get_pdf_pipeline_options():
    return PdfPipelineOptions(
        enable_remote_services=False,
        do_ocr=False,
        do_table_structure=True,
        table_structure_options=TableStructureOptions(
            mode=TableFormerMode.ACCURATE
        ),
        artifacts_path=ARTIFACTS_PATH
    )

def convert_document(file_path: str) -> dict:
    """Convert a document to markdown and return a ConvertedDocument dict."""
    path = Path(file_path).resolve()
    if not path.exists():
        raise FileNotFoundError(f"File not found: {path}")

    converter = DocumentConverter(
        format_options={
            InputFormat.PDF: PdfFormatOption(
                pipeline_options=get_pdf_pipeline_options(),
            )
        }
    )

    result = converter.convert(str(path))
    doc = result.document

    content = doc.export_to_markdown(
        image_placeholder="",
        page_break_placeholder=PAGE_BREAK,
        include_annotations=True,
        mark_annotations=True
    )

    document_id = hashlib.sha256(content.encode("utf-8")).hexdigest()

    if path.suffix.lower() == ".pdf" and PAGE_BREAK in content:
        text = [page.strip() for page in content.split(PAGE_BREAK)]
    else:
        text = content

    return {
        "documentId": document_id,
        "filePath": str(path),
        "text": text,
    }


@app.route("/parse", methods=["POST"])
def parse():
    data = request.get_json(silent=True)
    if not data or "filePath" not in data:
        return jsonify({"error": "Missing 'filePath' in JSON body"}), 400

    file_path = data["filePath"]
    path = Path(file_path)

    if not path.exists():
        return jsonify({"error": f"File not found: {file_path}"}), 404

    if path.suffix.lower() not in SUPPORTED_EXTENSIONS:
        return jsonify({"error": f"Unsupported file type: {path.suffix}"}), 400

    try:
        result = convert_document(file_path)
        return jsonify(result), 200
    except Exception as e:
        return jsonify({"error": str(e)}), 500


if __name__ == "__main__":
    app.run(host="127.0.0.1", port=8001, debug=False)