pyflink.multimodal.operators.audio_decode#

audio_decode(*columns, on_error='raise', max_decoded_bytes=1073741824, concurrency=None)[source]#

Decode audio into a waveform row.

The waveform row contains data plus audio layout fields: sample_rate, channels, frames, sample_format, layout, duration_ms, and optional source/segment fields. data stores a variable-shape float32 tensor exposed to Python as a numpy.ndarray with shape (frames, channels). Downstream operators should consume the row through audio operators rather than parsing data manually.

For remote URI inputs, large encoded files may be copied to a worker-local temporary file after metadata probing confirms that the decoded waveform fits max_decoded_bytes. The temporary file is an internal optimization to avoid moving large byte payloads across the JVM/Python boundary; it is cleaned up by the operator and is never returned to downstream operators.

Parameters

*columns – Optional audio columns. Supported inputs are BYTES, URI STRING, and AUDIO_CLIP_REF.
on_error –
Handling strategy for corrupt supported audio inputs:
- "raise" (default): propagate decode failures.
- "null": return NULL for rows that cannot be decoded.
URI access errors, unsupported input kinds, and invalid AUDIO_CLIP_REF values still raise.
max_decoded_bytes – Maximum decoded PCM payload allowed in one AUDIO_WAVEFORM row. The default is 1 GiB. Values above the PyFlink tensor payload limit are rejected. This guard is checked before materializing decoded samples and prevents compressed audio from expanding into an unexpectedly large per-row waveform.
concurrency – UDF concurrency. None uses the framework default.

Returns

A UDF producing ROW<data TENSOR('float32'), sample_rate INT, channels INT, frames BIGINT, sample_format STRING, layout STRING, duration_ms BIGINT, source_uri STRING, start_time_ms BIGINT, end_time_ms BIGINT>.

Raises

ValueError – If on_error is unsupported or a row value is not one of the supported input forms. on_error="raise" also propagates row-level media failures, including decoded-size guard failures and short reads from truncated inputs.

Examples::

>>> # Usage 1: create a reusable UDF and apply it to a column.
>>> decode = audio_decode(
...     on_error="null",
...     max_decoded_bytes=1024 * 1024 * 1024,
... )
>>> df = df.with_column("waveform", decode(col("audio_bytes")))
>>>
>>> # Usage 2: pass the column directly when building the expression.
>>> df = df.with_column(
...     "waveform",
...     audio_decode(
...         col("audio_bytes"),
...         on_error="null",
...         max_decoded_bytes=1024 * 1024 * 1024,
...     ),
... )