Low Code Spark

OrderBy

Order by is a simple sort operation that sorts the incoming data on one more columns or expressions.

.

Example

If we have the following incoming data, that we want to sort by customer_id .

.

We add the OrderBy component SortByCustomer as shown below

.

.

The outgoing data from the sort is in ascending order of customer_id

.

.

Example Code

The code is fairly straigtforward as well.


object SortBrandsByPrice {

  def apply(spark: SparkSession, in: DataFrame): OrderBy = {
    import spark.implicits._

    val out = in.orderBy(col("ext_price").desc,
                         col("brand").asc,
                         col("brand_id").asc,
                         col("i_manufact_id").asc,
                         col("i_manufact").asc
    )

    out

  }

}
    

# Python code coming soon!