PyFlink Multimodal#

PyFlink Multimodal Operators.

Operators live under pyflink.multimodal.operators and are organized by functionality:

image_transform — codec boundary (decode / encode / compress / to_tensor), pixel transforms (resize, crop, blur, flip, color adjust, …), and model-backed ops (remove background).
image_info — metadata extraction (aspect ratio, sharpness, perceptual hash) and filters (size, shape, file size, validity).
image_detect — object detection (YOLO), instance segmentation (FastSAM), subplot detection, and OCR.
image_face — face detection, face counting, and face blurring.
image_quality — NSFW filtering, aesthetic scoring, composite quality scoring, and watermark filtering.
image_embed — CLIP embedding and image-text similarity scoring.
video_frames — video metadata probing, segment splitting, and frame extraction.

Quick Start:

from pyflink.multimodal.operators import (
    image_decode, image_resize, image_encode,
)

# Curried — configure first, apply later
resizer = image_resize(width=512, height=512)
df = df.with_columns(resized=resizer(col("img")))

# Direct — configure and apply in one step
df = df.with_columns(out=image_encode(col("resized"), format="JPEG"))

Dependencies#

Built-in VVR images install the multimodal runtime dependencies from flink-python/dev/multimodal/requirements-multimodal.in. The same version ranges are checked lazily by model-backed operators when they open, so this file remains the compatibility contract for user-overridden dependencies.

For local source development, install from the same file instead of copying package versions into job code:

pip install -r flink-python/dev/multimodal/requirements-multimodal.in

Or for Flink runtime:

env.set_python_requirements("path/to/requirements.txt")

When a job overrides a built-in package through Python requirements or Python archives, keep the override inside the supported range reported by pyflink.model.check_dependencies().