Skip to main content

Row Distributor

Create multiple DataFrames based on provided filter conditions from an input DataFrame.

This is useful for cases where rows from the input DataFrame needs to be distributed into multiple DataFrames in different ways for downstream Gems.


DataFrameInput DataFrame for which rows needs to be distributed into multiple DataFramesTrue
Filter ConditionsBoolean Type column or boolean expression for each output tab. Supports SQL, Python and Scala expressionsTrue


Row distributor 1


Number of outputs can be changed as needed by clicking the + button.

Generated Code

def RowDistributor(spark: SparkSession, in0: DataFrame) -> (DataFrame, DataFrame, DataFrame):
df1 = in0.filter((col("order_status") == lit("Started")))
df2 = in0.filter((col("order_status") == lit("Approved")))
df3 = in0.filter((col("order_status") == lit("Finished")))

return df1, df2, df3