Update a recommended model configuration
Update or archive a recommended model configuration from the Models page in the CosmicAC web interface.
Update the values in a recommended model configuration, or archive a configuration that you no longer need.
Editing a recommended model configuration doesn't change a job that already serves the model. CosmicAC applies the new values to the next job that anyone creates for that model.
Prerequisites
Before you start, make sure that you have the following.
- A running CosmicAC deployment. See Set up CosmicAC.
- The platform administrator role. See Teams and roles.
Steps
Open the recommended configurations
In the left navigation, click Models, and then click Recommended configurations.
Open the configuration
Find the model. To narrow the list, select a status, such as Active or Archived, select a job type, or search by name, job type, quantisation, or runtime image.
In the model's row, click the pencil icon. The Edit recommended configuration page opens.
Edit the default parameters
Under Default parameters, edit the values that CosmicAC prefills into a new job's Serving configuration. For the recommended values, see Recommended configuration values.
For a Managed Inference model, you can edit the following fields.
- CUDA version: the CUDA driver version.
- Runtime image: the vLLM serving image.
- Data type: the numeric precision that the model runs at.
- Quantisation: the method that compresses the model weights.
- Reasoning parser: the parser that separates thinking tokens from the final response.
- Replicas: the number of model copies to serve.
- GPUs per replica: the number of GPUs that each replica uses.
- GPU memory utilisation: the fraction of GPU memory to use.
- Max model length: the maximum context length.
- Max concurrent sequences: the maximum number of requests handled at the same time.
- Root disk (GB): the root disk size in GB.
- Environment variables: additional
vllm serveoptions, as name and value pairs. - Video and image input: whether the model accepts multimodal input. This setting applies to vision-language models only.
Set the job creation warnings
Under Job creation warnings, select when CosmicAC warns about a new job that differs from a recommended value. The person who creates the job can acknowledge the warning and continue.
For a Managed Inference model, select a setting for each of the following fields.
- GPU memory utilisation
- Max model length
- Max concurrent sequences
- Replicas
- Root disk
Each field offers the following settings. Root disk offers only No warning and Warn above.
- No warning: CosmicAC never warns about this field.
- Warn below: CosmicAC warns when the job's value is lower than the recommended value.
- Warn above: CosmicAC warns when the job's value is higher than the recommended value.
- Warn when different: CosmicAC warns when the job's value differs from the recommended value.
Save the configuration
Click Save.
Archive a configuration
To archive a configuration, open it, and then click Archive. To find an archived configuration later, select Archived on the Recommended configurations page.