Limit gem

ProphecySparkBasicsPython 0.0.1+ProphecySparkBasicsScala 0.0.1+UC Dedicated Cluster 14.3+UC Standard Cluster 14.3+Livy 3.0.1+

Limits the number of rows in the output.

Parameters

Parameter	Description
DataFrame	Input DataFrame
Limit	Number of rows required in output. Allowed range: [0, 231 -1]

Limit to 10 rows

If you want to limit your output to 10 rows, you can input 10 in the Limit gem.

note

Data samples generated before the Limit gem might also be limited. This is because Spark tries to push the limit down to earlier stages of execution to minimize data processing. This means Spark may reduce the number of rows fetched from the source or processed in earlier transformations.

Code

Python

def limit(spark: SparkSession, in0: DataFrame) -> DataFrame:
    return in0.limit(10)

Scala

object limit {
  def apply(spark: SparkSession, in: DataFrame): DataFrame =
    in.limit(10)
}