pyflink.dataframe.ai.providers.OpenAICompatProvider

class OpenAICompatProvider(*, endpoint: str | None = None, api_key: str | None = None, task: str | None = None, model: str | None = None, system_prompt: str = 'You are a helpful assistant.', user_prompt: str | None = None, temperature: float | None = None, top_p: float | None = None, max_tokens: int | None = None, stop: str | None = None, presence_penalty: float | None = None, n: int | None = None, seed: int | None = None, content_type: str = 'TEXT', response_format: str | None = None, dimension: int | None = None, max_context_size: int | None = None, context_overflow_action: str = 'truncated-tail', error_handling_strategy: str = 'RETRY', retry_num: int = 100, retry_backoff_strategy: str = 'FIXED', retry_backoff_base_interval: str = '1s', retry_fallback_strategy: str = 'FAILOVER', extra_header: str | None = None, extra_body: str | None = None, **extra_options: Any)

Provider for all OpenAI-compatible endpoints (openai-compat).

Covers OpenAI, DeepSeek, Bailian, and any other service that implements the OpenAI chat/completions or embeddings API.

Parameters:

• endpoint – The endpoint to connect to. Required with api_key.

• api_key – The key used to authorize the access to the endpoint. Required when using BYOK models.

• task – The model task. Supported values are "chat/completions", and "embeddings". Required when using Flink AI Model Service.

• model – The version of the model to use.

• system_prompt – The system message of a chat. Can be disabled by setting to empty string. Defaults to "You are a helpful assistant.".

• user_prompt – The prompt of a chat, passed to the model service through user’s role. Can be disabled by setting to empty string.

• temperature – Controls the randomness or “creativity” of the output. Typical values are between 0.0 and 1.0.

• top_p – The probability cutoff for token selection. Usually either temperature or top_p are specified, but not both.

• max_tokens – The maximum number of tokens that can be generated in the chat completion.

• stop – A CSV list of strings to pass as stop sequences to the model.

• presence_penalty – Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics.

• n – How many chat completion choices to generate for each input message. Keep n as 1 to minimize costs.

• seed – If specified, the model platform will make a best effort to sample deterministically. Determinism is not guaranteed.

• content_type – Content type of the input string. Supported types: "TEXT" (default), "IMAGE_URL".

• response_format – The format of the response ("text" or "json_object").

• dimension – The size of the embedding result array.

• max_context_size – Max number of tokens for context. context_overflow_action is triggered if this threshold is exceeded.

• context_overflow_action – Action to handle context overflows. One of "truncated-tail", "truncated-tail-log", "truncated-head", "truncated-head-log", "skipped", or "skipped-log" (case-insensitive). Defaults to "truncated-tail".

• error_handling_strategy – Strategy for handling errors during model requests. "RETRY" retries the request (limited by retry_num, retry_fallback_strategy, etc.); "FAILOVER" throws exceptions and fails the job; "IGNORE" skips the error input and continues. Defaults to "RETRY".

• retry_num – Number of retries for client requests. Defaults to 100.

• retry_backoff_strategy – The strategy to use for retry backoff. "FIXED" or "EXPONENTIAL". Defaults to "FIXED".

• retry_backoff_base_interval – The base interval for retry backoff, used as the initial delay before the first retry and as the base for calculating subsequent retry delays. Defaults to "1s".

• retry_fallback_strategy – Fallback strategy to employ if the retry attempts are exhausted. "FAILOVER" or "IGNORE". Defaults to "FAILOVER".

• extra_header – Additional headers for the requests. Should be a JSON-format string whose values are strings or list of strings.

• extra_body – Additional parameters to pass through the requests’ body. Should be a JSON-format string.

• **extra_options – Additional options passed through as-is (keys are not translated).

Example::

>>> provider = OpenAICompatProvider(
...     endpoint="https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions",
...     api_key="sk-...",
...     temperature=0.7,
... )

Methods

model_option_key()	Return the Java-side option key used for a per-call model name.
provider_name()	Return the provider identifier recognized by Flink's Java runtime.
to_options()	Return all configured options as a dict with Java-side key names.