59 docs tagged with "gems"

Group data and apply aggregation methods or pivot operations

Avro

BigQuery

Change the data type of multiple columns at once.

Rename multiple columns in your Dataset in a systematic way.

Compare columns between two dataframes

CosmosDB

CSV

Standardize data formats and address missing or null values in the data.

DB2

Remove rows with duplicate values of specified columns

Remove rows with duplicate values of specified columns

Delta

Delta Table

Dynamically filter columns of your dataset based on a set of conditions.

Filter your data based on a custom filter condition

Filter your data based on a custom filter condition

Fixed Format

Flatten nested data

Flatten nested data

Transforming your data

Read from or write to Tables managed by a Hive metastore

Iceberg

JDBC

Join one or more DataFrames on conditions

JSON

Reading and writing data from Apache Kafka in batch mode

Limit the number of rows

Limit the number of rows

Lookup

MongoDB

Oracle

ORC

Sort your data based on one or more Columns

Sort your data based on one or more Columns

Parquet

Redshift

Select one or more columns or values using expressions and functions.

Select one or more columns or values using expressions and functions.

Repartition or coalesce a DataFrame

Enrich DataFrame with content from rest API response based on configuration

Create multiple DataFrames based on filter conditions

Salesforce

Add, Edit, Rename or Drop Columns

Union, Intersect and Difference

Union, Intersect and Difference

Snowflake

Prophecy Streaming Gems

Gems are data seeds, sources, transformations, and targets

Create DataFrames based on custom SQL queries

Create Models based on custom SQL queries

Teradata

Text

Data transformation steps in SQL

Use the Unpivot Gem to transform your data from a wide format to a long format.

Use the Unpivot Gem to transform your data from a wide format to a long format.

Aggregate and transform Windowed data

Aggregate and transform Windowed data

XLSX (Excel)