Skip to main content
Version: 6.19

LLM connections

Introduction to LLM connections​

The LLM connections page allows to manage the connections to various language models. You can use a model provided by a company such as OpenAI, or a self-hosted model.

Settings for LLM connections​

ConSol CM supports both standard language models and models with reasoning capabilities. If reasoning is enabled for a connection, ConSol CM automatically handles the token configuration required for communicating with the model.

The following settings can be made:

  • Provider type: Select OpenAI, Azure, Ollama or Anthropic to use an external provider which comes with a built-in configuration. Select Custom if you need to connect another type of provider or if you want to use a self-hosted model. In this case, you need to create a script that implements the communication.
  • Model URL: Enter the URL where the model can be accessed. This URL acts as the bridge between your system and the AI model.
  • Model name: Specify the exact name of the model which you want to connect to.
  • Reasoning enabled: Determines whether the connected model has reasoning capabilities. Set this option to True when using a reasoning model. The default value is False. This setting is not available for the provider type Ollama.
  • Reasoning effort: Specifies the computational effort used by the model for reasoning. Higher effort levels usually consume more tokens. The available values, e.g. low, medium, high or xhigh, depend on the model provider. Refer to the documentation of your provider for the supported values. This setting is displayed only if Reasoning enabled is set to True.
  • API key: Enter your API key for the model. It is provided by the provider of the LLM.
  • Token limit: Tokens are units of text processed by the AI. Enter the maximum number of tokens which the response from the AI should return. The default value is 500. You can increase the value if the responses are cut off. Please note that this can also increase the cost of the request.
  • Use Privacy Purger: A feature to enhance data privacy by removing personal information from texts before they are sent to the AI model. Select True if personal data should be removed from the text before sending it to the AI. The text will first be sent to the Privacy Purger application. The Privacy Purger returns text where personal data has been replaced by general tags. This text is then sent to the AI.
  • Privacy Purger URL: Enter the URL where the Privacy Purger can be accessed. On cloud systems, you can obtain it from the ConSol CM support team. For an on-premise system, you need to install the Privacy Purger application.
  • Privacy Purger password: Enter the Privacy Purger password.
  • Script: You can select an AI integration type script if you want to use your own code to send data to the LLM. This might be necessary when adding Custom models which expect a syntax not covered by the standard implementation.
Token limit for reasoning models

Reasoning models typically require more tokens than standard models. If requests fail because of token limits, increase the Token limit setting for the LLM connection.

Basic tasks​

Working with LLM connections​

You can perform the following actions on LLM connections:

  • Configure the default LLM connection: The default connection is created automatically. Provide the connection settings and click the Update button to save the settings. This action is mandatory if you want to use the prompt wizard, see Using the prompt wizard.
  • Create a new LLM connection: Click the New configuration in the header and enter the name of the configuration. Afterwards you can provide the connection settings.
  • Update an LLM connection: Modify the connection settings and click the Update button.
  • Test an LLM connection: Click the Test connection button to check if the connection works correctly.
  • Rename an LLM connection: Click the Rename configuration button in the header to change the name of the connection.
  • Remove an LLM connection: Click the Remove configuration button in the header to delete the connection.

The settings are saved to system properties.

Advanced tasks​

Using LLM connections in scripts​

Use the methods from AIClientService to send requests to an LLM. You need to pass the name of the configuration, and the system and user message as parameters:

aiClientService.send("myConfiguration", "mySystemMessage", "myUserMessage")

If the request should be sent to the Privacy Purger first, use the following method:

aiClientService.send("myConfiguration", "mySystemMessage", "myUserMessage", true)