1.2 KiB
1.2 KiB
InferenceRequestChatCompletion [interface-InferenceRequestChatCompletion]
| Name | Type | Description |
|---|---|---|
max_completion_tokens |
long | The upper bound limit for the number of tokens that can be generated for a completion request. |
messages |
InferenceMessage[] | A list of objects representing the conversation. Requests should generally only add new messages from the user (role user). The other message roles ( assistant, system, or tool) should generally only be copied from the response to a previous completion request, such that the messages array is built up throughout a conversation. |
model |
string | The ID of the model to use. |
stop |
string[] | A sequence of strings to control when the model should stop generating additional tokens. |
temperature |
float | The sampling temperature to use. |
tool_choice |
InferenceCompletionToolType | Controls which tool is called by the model. |
tools |
InferenceCompletionTool[] | A list of tools that the model can call. |
top_p |
float | Nucleus sampling, an alternative to sampling with temperature. |