pyflink.dataframe.OpenAICompatProvider#

class OpenAICompatProvider(*, task: Optional[str] = None, endpoint: Optional[str] = None, api_key: Optional[str] = None, model: Optional[str] = None, system_prompt: str = 'You are a helpful assistant.', user_prompt: Optional[str] = None, temperature: Optional[float] = None, top_p: Optional[float] = None, max_tokens: Optional[int] = None, stop: Optional[str] = None, presence_penalty: Optional[float] = None, n: Optional[int] = None, seed: Optional[int] = None, content_type: Union[Literal['TEXT', 'IMAGE_URL', 'MULTI_IMAGE_URLS'], List[Literal['TEXT', 'IMAGE_URL', 'MULTI_IMAGE_URLS']], Tuple[Literal['TEXT', 'IMAGE_URL', 'MULTI_IMAGE_URLS'], ...]] = 'TEXT', response_format: Optional[str] = None, dimension: Optional[int] = None, max_context_size: Optional[int] = None, context_overflow_action: str = 'truncated-tail', error_handling_strategy: str = 'RETRY', retry_num: int = 100, retry_backoff_strategy: str = 'FIXED', retry_backoff_base_interval: str = '1s', retry_fallback_strategy: str = 'FAILOVER', extra_header: Optional[str] = None, extra_body: Optional[str] = None, **extra_options: Any)[source]#

Model provider for all OpenAI-compatible endpoints (openai-compat).

Covers endpoints that implement the OpenAI chat/completions or embeddings API.

Note

We recommend using Flink AI service. With Flink AI service, you do not need to configure endpoint or api_key.

Parameters

task – The model task. Supported values are "chat/completions", and "embeddings". Required when using Flink AI Model Service.
endpoint – The endpoint to connect to. Required with api_key.
api_key – The key used to authorize the access to the endpoint. Required when using BYOK models.
model – The version of the model to use.
system_prompt – The system message of a chat. Can be disabled by setting to empty string. Defaults to "You are a helpful assistant.".
user_prompt – The prompt of a chat, passed to the model service through user’s role. Can be disabled by setting to empty string.
temperature – Controls the randomness or “creativity” of the output. Typical values are between 0.0 and 1.0.
top_p – The probability cutoff for token selection. Usually either temperature or top_p are specified, but not both.
max_tokens – The maximum number of tokens that can be generated in the chat completion.
stop – A CSV list of strings to pass as stop sequences to the model.
presence_penalty – Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics.
n – How many chat completion choices to generate for each input message. Keep n as 1 to minimize costs.
seed – If specified, the model platform will make a best effort to sample deterministically. Determinism is not guaranteed.
content_type – Content type of the input column(s). For a single input column or multiple input columns that all use the same content type, pass one string, for example "TEXT". For multiple input columns with different content types, pass a list or tuple of strings; the count and order must match the input columns. Supported values are "TEXT" (default), "IMAGE_URL", and "MULTI_IMAGE_URLS". Task-specific limitations are enforced by the model provider.
response_format – The format of the response ("text" or "json_object").
dimension – The size of the embedding result array.
max_context_size – Max number of tokens for context. context_overflow_action is triggered if this threshold is exceeded.
context_overflow_action – Action to handle context overflows. One of "truncated-tail", "truncated-tail-log", "truncated-head", "truncated-head-log", "skipped", or "skipped-log" (case-insensitive). Defaults to "truncated-tail".
error_handling_strategy – Strategy for handling errors during model requests. "RETRY" retries the request (limited by retry_num, retry_fallback_strategy, etc.); "FAILOVER" throws exceptions and fails the job; "IGNORE" skips the error input and continues. Defaults to "RETRY".
retry_num – Number of retries for client requests. Defaults to 100.
retry_backoff_strategy – The strategy to use for retry backoff. "FIXED" or "EXPONENTIAL". Defaults to "FIXED".
retry_backoff_base_interval – The base interval for retry backoff, used as the initial delay before the first retry and as the base for calculating subsequent retry delays. Defaults to "1s".
retry_fallback_strategy – Fallback strategy to employ if the retry attempts are exhausted. "FAILOVER" or "IGNORE". Defaults to "FAILOVER".
extra_header – Additional headers for the requests. Should be a JSON-format string whose values are strings or list of strings.
extra_body – Additional parameters to pass through the requests’ body. Should be a JSON-format string.
**extra_options – Additional options passed through as-is (keys are not translated).

Note

When a configured provider is used, DataFrame AI function calls can override the provider’s constructor parameters for that call.

LLM function config supports these runtime prediction options with this provider:

async: Execution-mode hint. The provider supports async prediction only, so omit this key or set it to "true". Setting "false" is not supported.
output-mode: Output ordering for async prediction. Allowed values are "ORDERED" and "ALLOW_UNORDERED". Default: table config table.exec.async-ml-predict.output-mode ("ORDERED" by default).
max-concurrent-operations: Maximum number of concurrent async prediction calls. Allowed values are positive integer strings. Default: table config table.exec.async-ml-predict.max-concurrent-operations ("10" by default).
timeout: Timeout for async prediction calls. Allowed values are duration strings such as "30s" or "3 min". Default: table config table.exec.async-ml-predict.timeout ("3 min" by default).

Example:

>>> provider = OpenAICompatProvider(task="chat/completions")
>>> pf.set_model_provider(provider)
>>> df = pf.from_dict({
...     "review_text": ["The shoes fit well but the sole arrived scratched."],
...     "product_image_url": ["https://example.com/product.jpg"],
... })
>>> predictions = df.llm.predict(
...     "review_text",
...     "product_image_url",
...     model="qwen3.6-plus",
...     content_type=["TEXT", "IMAGE_URL"],
...     system_prompt="Respond to the product review and image together.",
...     temperature=0.3,
...     config={
...         "max-concurrent-operations": "20",
...         "timeout": "30s",
...     },
... )
>>> classified = df.llm.ai_classify(
...     "review_text",
...     ["positive", "negative"],
...     model="qwen3.6-plus",
...     temperature=0.1,
... )

Methods

`model_option_key`()	Return the Java-side option key used for a per-call model name.
`provider_identifier`()	Return the provider identifier recognized by Flink's Java runtime.
`to_options`([input_columns, overrides])	Return Java-side options for `ModelDescriptor`.