Script
Provides a SparkSession
and allows you to run custom code
Parameters
Parameter | Meaning | Required |
---|---|---|
Input DataFrame(s) | Input DataFrame(s) | False |
Output DataFrame(s) | Output DataFrame(s) | False |
Code | Custom code to be executed | True |
info
To remove input/output DataFrame(s), simply click icon on the left sidebar
Examples
Script Gem with Input and Output: Un-pivoting a DataFrame
We'll perform the unpivot
operation using our custom code
Script Gem with only Output: Generating a DataFrame
We'll use the provided SparkSession
to create and return a DataFrame
note
Since we removed the input port, we don't see input DataFrame in the method signature