pyflink.multimodal.operators.audio_split_by_timestamp#

audio_split_by_timestamp(*columns, timestamps=None, segment_type='audio', concurrency=None)[source]#

Split audio by explicit timestamp ranges into one output row per segment.

Timestamp ranges are interpreted as millisecond start_ms/end_ms pairs. They may be provided as a literal Python sequence via timestamps=[...] or as a second DataFrame column expression. segment_type="audio" materializes waveform slices. segment_type="ref" returns lazy reference slices without opening the encoded audio object. For URI input, timestamp ranges are not clipped to the actual audio duration; for bounded AUDIO_CLIP_REF input, ranges are clipped only to the existing reference boundary. That boundary is trusted as caller metadata and is not validated against the real audio duration. SQL module calls use segment_type as the third argument, for example audio_split_by_timestamp(uri, ranges, 'ref').

Parameters

*columns – Optional audio columns. segment_type="audio" supports waveform rows. segment_type="ref" supports URI STRING and reference rows.
timestamps – Optional literal sequence of ranges. Each range may be a Row/object/dict with start_ms and end_ms fields, or a two-item sequence. Leave this unset when passing ranges as the second input column.
segment_type – "audio" for materialized waveform rows or "ref" for reference rows.
concurrency – UDF concurrency. None uses the framework default.

Returns

A UDTF producing one segment column per emitted segment. For segment_type="audio", segment is a waveform row with data TENSOR('float32'), sample_rate INT, channels INT, frames BIGINT, sample_format STRING, layout STRING, duration_ms BIGINT, source_uri STRING, start_time_ms BIGINT, end_time_ms BIGINT. For segment_type="ref", segment has uri STRING, start_time_ms BIGINT, end_time_ms BIGINT.

Raises

ValueError – If the input does not match segment_type, if a timestamp range is invalid, or if segment_type is unsupported.

Examples::

>>> ranges = [{"start_ms": 1000, "end_ms": 3500}]
>>> # Usage 1: create a reusable UDTF and join it laterally.
>>> split = audio_split_by_timestamp(timestamps=ranges)
>>> segments = df.join_lateral(split(col("waveform")).alias("segment"))
>>>
>>> # Usage 2: pass the column directly when joining laterally.
>>> segments = df.join_lateral(
...     audio_split_by_timestamp(
...         col("waveform"),
...         timestamps=ranges,
...     ).alias("segment")
... )
>>>
>>> # Dynamic timestamp ranges can be read from a column.
>>> segments = df.join_lateral(
...     audio_split_by_timestamp(
...         col("waveform"),
...         col("ranges"),
...     ).alias("segment")
... )