pyflink.multimodal.operators.image_embedding#

image_embedding(*columns, model='ViT-B/32', pretrained='openai', model_sharing=None, concurrency=None, batch_size=None, num_gpus=None, gpu_type=None)[source]#

Create an image embedding UDF (CLIP / open_clip-based).

Requires pip install open_clip_torch torch Pillow. This is a pandas batch UDF that supports GPU acceleration via num_gpus / gpu_type.

Parameters

*columns – Optional image column(s). When provided, the UDF is applied directly instead of returning a factory.
model – CLIP model architecture name. Default "ViT-B/32".
pretrained – Pretrained weights checkpoint. Default "openai".
model_sharing – Model sharing mode across parallel subtasks. None uses per-process caching.
concurrency – UDF concurrency. None uses the framework default.
batch_size – Pandas batch size. None uses the framework default.
num_gpus – Fractional GPU count per subtask, e.g. 0.5. None runs on CPU.
gpu_type – Required GPU type, e.g. "A10". None accepts any available GPU.

Returns

A UDF that returns a L2-normalized list[float32] embedding vector, or None for null image inputs. The vector dimension depends on the model (e.g. 512 for ViT-B/32).

Example::

>>> # As a reusable variable
>>> embed = image_embedding(model="ViT-B/32")
>>> df = df.with_column("vector", embed(col("img")))
>>>
>>> # Inline
>>> df = df.with_column("vector", image_embedding(col("img")))