Skip to content

Integrate with vLLM

1 Add Model

Select the model provider as vLLM, and enter the following necessary information in the model addition dialog box:

  • Model Name: The custom model name in MaxKB.
  • Permissions: Divided into private and public permissions. Private models are available only to the current user, while public models can be used by all users within the system, but other users cannot edit or delete them.
  • Model Type: Large Language Model.
  • Base Model: The base model name under different model types. The dropdown options include some commonly used base model names, and custom input is supported.
  • API Domain: vLLM service address, e.g., http://192.168.20.242:8000/v1.
  • API Key: If there is no API Key, you can enter any characters.

2 Configuration Example

vLLM-Large Language Model configuration example illustration:

vLLM LLM Model