10 lines
2.0 KiB
Markdown
10 lines
2.0 KiB
Markdown
# `InferenceCohereServiceSettings` [interface-InferenceCohereServiceSettings]
|
|
|
|
| Name | Type | Description |
|
|
| - | - | - |
|
|
| `api_key` | string | A valid API key for your Cohere account. You can find or create your Cohere API keys on the Cohere API key settings page. IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key. |
|
|
| `embedding_type` | [InferenceCohereEmbeddingType](./InferenceCohereEmbeddingType.md) | For a `text_embedding` task, the types of embeddings you want to get back. Use `byte` for signed int8 embeddings (this is a synonym of `int8`). Use `float` for the default float embeddings. Use `int8` for signed int8 embeddings. |
|
|
| `model_id` | string | For a `completion`, `rerank`, or `text_embedding` task, the name of the model to use for the inference task. * For the available `completion` models, refer to the [Cohere command docs](https://docs.cohere.com/docs/models#command). * For the available `rerank` models, refer to the [Cohere rerank docs](https://docs.cohere.com/reference/rerank-1). * For the available `text_embedding` models, refer to [Cohere embed docs](https://docs.cohere.com/reference/embed). The default value for a text embedding task is `embed-english-v2.0`. |
|
|
| `rate_limit` | [InferenceRateLimitSetting](./InferenceRateLimitSetting.md) | This setting helps to minimize the number of rate limit errors returned from Cohere. By default, the `cohere` service sets the number of requests allowed per minute to 10000. |
|
|
| `similarity` | [InferenceCohereSimilarityType](./InferenceCohereSimilarityType.md) | The similarity measure. If the `embedding_type` is `float`, the default value is `dot_product`. If the `embedding_type` is `int8` or `byte`, the default value is `cosine`. |
|