pyflink.dataframe.DataFrame.join_asof#

DataFrame.join_asof(other: pyflink.dataframe.dataframe.DataFrame, how: str = 'inner', on: Optional[Union[str, pyflink.table.expression.Expression, pyflink.dataframe.dataframe._ProctimeMarker]] = None, by: Optional[Union[str, pyflink.table.expression.Expression, List[Union[str, pyflink.table.expression.Expression]]]] = None, left_by: Optional[Union[str, pyflink.table.expression.Expression, List[Union[str, pyflink.table.expression.Expression]]]] = None, right_by: Optional[Union[str, pyflink.table.expression.Expression, List[Union[str, pyflink.table.expression.Expression]]]] = None) → pyflink.dataframe.dataframe.DataFrame[source]#

ASOF join with a dimension table at a time attribute.

join_asof can be used to perform a lookup join. To perform a lookup join, use one of three forms:

Do not pass on.
Pass on=pyflink.dataframe.proctime().
Pass on as a proctime column on this DataFrame.

The right DataFrame must be a direct source DataFrame created by a reader whose connector is lookup-capable.

Shared string join keys are internally renamed and deduplicated. Other same-name columns are not handled automatically; rename or alias non-key column conflicts before calling join_asof.

Parameters

other – Right DataFrame (the dim table).
how – "inner" or "left".
on – ASOF time attribute. Defaults to pyflink.dataframe.proctime() when omitted or None.
by – Join key(s) shared between both DataFrames. See join for accepted shapes (str, Expression, or list thereof).
left_by – Left-side join key(s); use together with right_by when the join keys have different names on each side.
right_by – Right-side join key(s); see left_by.

Returns

A new DataFrame with the ASOF join result.

Raises

ValueError – If how is not "inner" or "left", or if join keys are not provided.

Example:

>>> from pyflink.dataframe import col, proctime
>>> result_df = probe_df.join_asof(
...     dim_df,
...     by=col('id') == col('d_id'),
... )
>>> result_df = probe_df.join_asof(
...     dim_df,
...     on=proctime(),
...     by=col('id') == col('d_id'),
...     how='left',
... )
>>> result_df = probe_df.join_asof(
...     dim_df,
...     on=col('proc_time'),
...     left_by='id',
...     right_by='d_id',
... )