> ## Documentation Index
> Fetch the complete documentation index at: https://prompt-lockbox.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Configure LLM

> Step-by-step guide to LLM configuration

<Info>
  This is a must-to-do setup to use any AI feature in the toolkit.
</Info>

This guide will walk you through setting up LLMs for use within the toolkit. Whether you're connecting to a popular **cloud-based provider**, **running a local model**, or **integrating your own custom model**. Follow the steps below to get started with the setup that best fits your workflow.

<Note>
  **Install Provider Libraries**

  Before configuring a provider, make sure you have its Python library installed.

  * For **OpenAI**: `pip install openai`
  * For **Anthropic**: `pip install anthropic`
  * For **HuggingFace**: `pip install transformers torch litellm`
  * For **Ollama**: `pip install ollama`
  * ...
</Note>

There are two ways you can configure LLMs in this toolkit -

1. **Automated Setup (Recommended)**: Use the interactive `plb configure-ai` command.
2. **Manual Setup**: Manually create a `.env` file for secrets and edit your `plb.toml` for settings.

We recommend using the command for a guided, error-free experience.

## Automated Setup

**This interactive command is the easiest and safest way to set up your LLM**. It will guide you through selecting a provider, entering your API key securely, choosing a model and configuring additional settings.

Here's the steps -

<Steps>
  <Step title="Start the Configuration Panel">
    In your terminal, run the command:

    ```bash theme={null}
    plb configure-ai
    ```
  </Step>

  <Step title="Choose Your Provider">
    The command will present a list of common providers. Use the arrow keys to make a selection and press Enter.

    ```shell theme={null}
    ? Select your LLM Provider: 
    > OpenAI
      Anthropic
      Ollama (Local)
      Huggingface
      ...
    ```
  </Step>

  <Step title="Configure Your Settings">
    Based on your choice of the provider, follow the guided instructions in the CLI.

    <Note>
      Configuring mainstream, lightweight and cloud based LLMs is straightforward, but local models and Hugging Face setups require few additional steps. Please see the details below.
    </Note>

    Once you select the provider, follow these steps:

    1. **Enter your API key**:

    ```bash theme={null}
    ? Please enter your OpenAI API Key (will be stored in .env): 
    sk-********************
    ```

    2. **Select a Model**:

    ```bash theme={null}
    ? Enter the default model name (e.g., gpt-4o-mini): gpt-4o-mini
    ```

    You can also set custom model parameters like `max_tokens`, `temperature`, etc. in `plb.toml` file.

    **For Local or Huggingface Setup**:

    <AccordionGroup>
      <Accordion title="Local Model Configuration (Ollama)">
        <Info>
          You can checkout this awesome [blog post](https://medium.com/@arunpatidar26/run-llm-locally-ollama-8ea296747505) to learn how to download and use ollama.
        </Info>

        Using a local model via Ollama requires an extra step:

        1. **Ensure Ollama is Running**: The wizard will first remind you to start the Ollama server in a separate terminal.

        ```bash theme={null}
        ? To use Ollama, first run `ollama serve` in a new terminal.
        Press Enter to continue once your server is running...
        ```

        2. **Set the Model Name**: Enter the name of the Ollama model you have pulled (e.g., llama3). No API key is needed.

        ```bash theme={null}
        ? Please enter the Ollama model name to use: llama3
        ```
      </Accordion>

      <Accordion title="Huggingface Model Configuration">
        <Info>
          Using huggingface model requires you to create an account at [huggingface](https://huggingface.co/) and generate an access token. You can checkout this awesome [blog post](https://medium.com/@aroman11/how-to-use-hugging-face-api-token-in-python-for-ai-application-step-by-step-be0ed00d315c) to learn more.
        </Info>

        To use models from Huggingface:

        1. **Enter Your HF Token**: Provide your HF Token when prompted. This will be securely stored in your `.env` file.

        ```bash theme={null}
        ? Please enter your Hugging Face Token (will be stored in .env): hf_********************
        ```

        2. **Set the Model Repository ID**: Enter the full repository ID of the model you want to use.

        ```bash theme={null}
        ? Enter the Hugging Face model repo ID (e.g., mistralai/Mistral-7B-Instruct-v0.2): mistralai/Mistral-7B-Instruct-v0.2
        ```
      </Accordion>
    </AccordionGroup>
  </Step>

  <Step title="Configuration Complete!">
    Once you've completed the steps for your chosen provider, the command will save all settings to your `plb.toml` and `.env files` and confirm that the setup is complete.

    ```bash theme={null}
    Success! Configuration saved. You are now ready to use AI features!
    ```
  </Step>
</Steps>

## Manual Setup

If you prefer to manage your configuration files directly, you can set them up manually.

<Warning>
  Always add .env to your .gitignore file to prevent your secret keys from being committed to version control.
</Warning>

<Steps>
  <Step title="Create the .env file for API Keys/Tokens">
    In your project's root directory, create a `.env` file and add the corresponding environment variable for your provider.

    ```powershell theme={null}
    # Inside your .env file

    # For OpenAI
    OPENAI_API_KEY="sk-..."

    # For Hugging Face
    HUGGING_FACE_HUB_TOKEN="hf_..."
    ```
  </Step>

  <Step title="Edit the plb.toml file">
    Next, open your project's plb.toml file. Add or edit the \[ai] section to specify the provider and model you intend to use.

    <Tabs>
      <Tab title="OpenAI Example">
        ```toml theme={null}
        [ai]
        # In plb.toml
        [ai]
        provider = "openai"
        model = "gpt-4o-mini"    
        ```
      </Tab>

      <Tab title="Ollama Example">
        ```toml theme={null}
        [ai]
        # In plb.toml
        [ai]
        provider = "ollama"
        model = "llama3"    
        ```
      </Tab>

      <Tab title="Huggingface Example">
        ```toml theme={null}
        [ai]
        # In plb.toml
        [ai]
        provider = "huggingface"
        model = "mistralai/Mistral-7B-Instruct-v0.2"    
        ```
      </Tab>
    </Tabs>

    Once both files are created and saved, your manual configuration is complete.

    However, you can check for success using `plb configure-ai` status.
  </Step>
</Steps>

## Advanced: Using a Custom or Self-Hosted Model

<Warning>
  **Under Development**

  The feature is currently under development! Checkout our [GitHub](https://github.com/ananya868/prompt-lockbox/) for updates.
</Warning>

## Next Steps

You can now checkout the AI Features page!

<CardGroup>
  <Card title="AI Features" icon="stars" href="/ai_features">
    Get a quick look on AI features.
  </Card>
</CardGroup>