Skip to content

AI_Model_API_Settings_Guide.md

Latest commit

 

History

History
56 lines (36 loc) · 1.73 KB

AI_Model_API_Settings_Guide.md

File metadata and controls

56 lines (36 loc) · 1.73 KB
Raw

AI Model API Settings Guide

CodeSentinel uses the OpenAI Python SDK, making it compatible with any API that adheres to the OpenAI chat completions specification.

For accurate prompt sizing, export the model's Hugging Face-compatible tokenizer.json and set tokenizer_path in config.yaml. Without one, CodeSentinel uses a conservative local estimate powered by tokenizers and does not download a tokenizer at runtime.

Local LLMs (Recommended)

Running a local LLM ensures your code never leaves your machine.

LM Studio

  1. Load a model (e.g., gpt-oss-20b).

  2. Go to the Local Server tab and click Start Server.

  3. In config.yaml, set:

    openai_base_url: "http://localhost:1234/v1"
ai_model: "your-model-id" # Copy from LM Studio

llama.cpp

  1. Start a simple service using llama-server, with the following recommended parameters:

    llama-server -m ~/model.gguf --ctx-size 32768 --parallel 1 --port 8900

  2. In config.yaml, set:

    openai_base_url: "http://localhost:8900/v1"
ai_model: "your-model-id" # Copy from llama.cpp

    Actually, for the llama-server service started in the manner described above, you can enter any model name you want.

    However, you cannot leave it blank, as the system will prevent it from running with an empty model name.

Cloud Providers

OpenAI

  1. Get an API key from the OpenAI Dashboard.

  2. Set your environment variable: export OPENAI_API_KEY="sk-...".

  3. In config.yaml, set:

    openai_base_url: "https://api.openai.com/v1"
ai_model: "gpt-4o"

Others

Simply update the openai_base_url and ai_model in config.yaml to match the provider's documentation.