OptionalapiSome APIs allow an API key instead
OptionalapiThe version of the API functions. Part of the path.
OptionalauthOptionalconvertOptionalendpointHostname for the API call (if this is running on GCP)
OptionallocationRegion where the LLM is stored (if this is running on GCP)
OptionalmaxMaximum number of tokens to generate in the completion.
OptionalmodelModel to use
OptionalmodelModel to use
Alias for model
OptionalplatformWhat platform to run the service on. If not specified, the class should determine this from other means. Either way, the platform actually used will be in the "platform" getter.
OptionalresponseAvailable for gemini-1.5-pro.
The output format of the generated candidate text.
Supported MIME types:
text/plain: Text output.application/json: JSON response in the candidates.OptionalsafetyOptionalsafetyOptionalstopOptionalstreamWhether or not to include usage data, like token counts in the streamed response chunks.
OptionaltemperatureSampling temperature to use
OptionaltopKTop-k changes how the model selects tokens for output.
A top-k of 1 means the selected token is the most probable among all tokens in the model’s vocabulary (also called greedy decoding), while a top-k of 3 means that the next token is selected from among the 3 most probable tokens (using temperature).
OptionaltopPTop-p changes how the model selects tokens for output.
Tokens are selected from most probable to least until the sum of their probabilities equals the top-p value.
For example, if tokens A, B, and C have a probability of .3, .2, and .1 and the top-p value is .5, then the model will select either A or B as the next token (using temperature).
Input to chat model class.