Prophecy 3.4.3.x

January 8, 2025

Features

The following are features and improvements available in this release.

SQL UDFs stored in Databricks Unity Catalog are now available for use in Prophecy Python Projects.
There are now three ways to upload a file and incorporate it into your Spark pipeline:
- Drag and drop the file directly onto the pipeline canvas.
- Open the Source/Target gem drawer and click Upload file.
- Create a new Source gem, click + New Dataset, and then select Upload file.
When SCIM is enabled, there is a new option under Settings > SSO that lets you easily generate a SCIM token.
We now offer M2M functionality with Databricks OAuth for scheduled job and deployment authentication. Visit the Databricks documentation to learn more about access to Databricks with a Service Principal using OAuth.

The following are the newest available Prophecy Library versions in this release.