pyflink.multimodal.operators.image_detect_objects#

image_detect_objects(*columns, model='yolov8n', confidence=0.05, imgsz=640, iou=0.5, model_sharing=None, concurrency=None, batch_size=None, num_gpus=None, gpu_type=None)[source]#

Create an object detection UDF (YOLO-based).

Requires pip install ultralytics. This is a pandas batch UDF that supports GPU acceleration via num_gpus / gpu_type.

Parameters

*columns – Optional image column(s). When provided, the UDF is applied directly instead of returning a factory.
model – YOLO model name, e.g. "yolov8n", "yolov8s". Default "yolov8n".
confidence – Minimum confidence threshold for detections. Default 0.05 (aligned with Data-Juicer).
imgsz – Inference resolution in pixels. Larger values improve small-object detection at the cost of speed and memory. Default 640.
iou – NMS IoU threshold. Overlapping boxes above this ratio are suppressed. Default 0.5.
model_sharing – Model sharing mode across parallel subtasks. None uses per-process caching.
concurrency – UDF concurrency. None uses the framework default.
batch_size – Pandas batch size. None uses the framework default.
num_gpus – Fractional GPU count per subtask, e.g. 0.5. None runs on CPU.
gpu_type – Required GPU type, e.g. "A10". None accepts any available GPU.

Returns

A UDF that returns a list of dicts with label, x, y, w, h, confidence keys.

Example:

>>> # As a reusable variable
>>> detect = image_detect_objects(model="yolov8n", confidence=0.05)
>>> df = df.with_column("objects", detect(col("img")))
>>>
>>> # Inline
>>> df = df.with_column("objects", image_detect_objects(col("img")))