Script
Provides a SparkSession and allows you to run custom code
Parameters
| Parameter | Meaning | Required |
|---|---|---|
| Input DataFrame(s) | Input DataFrame(s) | False |
| Output DataFrame(s) | Output DataFrame(s) | False |
| Code | Custom code to be executed | True |
info
To remove input/output DataFrame(s), simply click icon on the left sidebar
Examples
Script Gem with Input and Output: Un-pivoting a DataFrame
We'll perform the unpivot operation using our custom code
Script Gem with only Output: Generating a DataFrame
We'll use the provided SparkSession to create and return a DataFrame
note
Since we removed the input port, we don't see input DataFrame in the method signature
