Custom

Includes a set of Gems that do not fall into the other available categories of Gems.

📄️ SQLStatement

Create DataFrames based on custom SQL queries

📄️ Script

Provide a place to use a SparkSession and whatever code you wish to use

📄️ FileOperation

Perform file operations on different file systems

📄️ DeltaTableOperations

Gem that encompasses some of the import side operations of Delta

📄️ RestAPIEnrich

Enrich DataFrame with content from rest API response based on configuration