pyflink.dataframe.OpenAICompatProvider#

class OpenAICompatProvider(*, endpoint: Optional[str] = None, api_key: Optional[str] = None, task: Optional[str] = None, model: Optional[str] = None, system_prompt: str = 'You are a helpful assistant.', user_prompt: Optional[str] = None, temperature: Optional[float] = None, top_p: Optional[float] = None, max_tokens: Optional[int] = None, stop: Optional[str] = None, presence_penalty: Optional[float] = None, n: Optional[int] = None, seed: Optional[int] = None, content_type: Union[Literal['TEXT', 'IMAGE_URL', 'MULTI_IMAGE_URLS'], List[Literal['TEXT', 'IMAGE_URL', 'MULTI_IMAGE_URLS']], Tuple[Literal['TEXT', 'IMAGE_URL', 'MULTI_IMAGE_URLS'], ...]] = 'TEXT', response_format: Optional[str] = None, dimension: Optional[int] = None, max_context_size: Optional[int] = None, context_overflow_action: str = 'truncated-tail', error_handling_strategy: str = 'RETRY', retry_num: int = 100, retry_backoff_strategy: str = 'FIXED', retry_backoff_base_interval: str = '1s', retry_fallback_strategy: str = 'FAILOVER', extra_header: Optional[str] = None, extra_body: Optional[str] = None, **extra_options: Any)[source]#

Provider for all OpenAI-compatible endpoints (openai-compat).

Covers OpenAI, DeepSeek, Bailian, and any other service that implements the OpenAI chat/completions or embeddings API.

Parameters

endpoint – The endpoint to connect to. Required with api_key.
api_key – The key used to authorize the access to the endpoint. Required when using BYOK models.
task – The model task. Supported values are "chat/completions", and "embeddings". Required when using Flink AI Model Service.
model – The version of the model to use.
system_prompt – The system message of a chat. Can be disabled by setting to empty string. Defaults to "You are a helpful assistant.".
user_prompt – The prompt of a chat, passed to the model service through user’s role. Can be disabled by setting to empty string.
temperature – Controls the randomness or “creativity” of the output. Typical values are between 0.0 and 1.0.
top_p – The probability cutoff for token selection. Usually either temperature or top_p are specified, but not both.
max_tokens – The maximum number of tokens that can be generated in the chat completion.
stop – A CSV list of strings to pass as stop sequences to the model.
presence_penalty – Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics.
n – How many chat completion choices to generate for each input message. Keep n as 1 to minimize costs.
seed – If specified, the model platform will make a best effort to sample deterministically. Determinism is not guaranteed.
content_type – Content type of the input columns. Pass a string for a single input column, or a list/tuple of strings for multiple input columns. Multiple content types are positional: their count and order must match the input columns. Supported values are "TEXT" (default), "IMAGE_URL", and "MULTI_IMAGE_URLS". Task-specific limitations are enforced by the model provider.
response_format – The format of the response ("text" or "json_object").
dimension – The size of the embedding result array.
max_context_size – Max number of tokens for context. context_overflow_action is triggered if this threshold is exceeded.
context_overflow_action – Action to handle context overflows. One of "truncated-tail", "truncated-tail-log", "truncated-head", "truncated-head-log", "skipped", or "skipped-log" (case-insensitive). Defaults to "truncated-tail".
error_handling_strategy – Strategy for handling errors during model requests. "RETRY" retries the request (limited by retry_num, retry_fallback_strategy, etc.); "FAILOVER" throws exceptions and fails the job; "IGNORE" skips the error input and continues. Defaults to "RETRY".
retry_num – Number of retries for client requests. Defaults to 100.
retry_backoff_strategy – The strategy to use for retry backoff. "FIXED" or "EXPONENTIAL". Defaults to "FIXED".
retry_backoff_base_interval – The base interval for retry backoff, used as the initial delay before the first retry and as the base for calculating subsequent retry delays. Defaults to "1s".
retry_fallback_strategy – Fallback strategy to employ if the retry attempts are exhausted. "FAILOVER" or "IGNORE". Defaults to "FAILOVER".
extra_header – Additional headers for the requests. Should be a JSON-format string whose values are strings or list of strings.
extra_body – Additional parameters to pass through the requests’ body. Should be a JSON-format string.
**extra_options – Additional options passed through as-is (keys are not translated).

Example::

>>> provider = OpenAICompatProvider(
...     endpoint="https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions",
...     api_key="sk-...",
...     temperature=0.7,
... )

Methods

`model_option_key`()	Return the Java-side option key used for a per-call model name.
`provider_name`()	Return the provider identifier recognized by Flink's Java runtime.
`to_options`([input_columns])	Return Java-side options for `ModelDescriptor`.