Prophecy 4.1.1 Extended Maintenance (EM) Release

June 20, 2025

Prophecy 4.1.1.0 is an extended maintenance release. Learn more in Prophecy versions support.

Features

The following enhancement is available in this release.

Databricks connections in Prophecy fabrics now accept service principal credentials for OAuth.

Now you can deploy scheduled pipelines using the service principal credentials. To learn more, visit the Databricks connection documentation.
You can now fetch and reference the hidden _metadata column from input files in Databricks. Spark projects only.
The Run History tab in the Observability interface now displays the Run Type and Run ID for each historical pipeline run.
For target models running on BigQuery SQL fabrics, you can now configure overwrite modes to partition by a specific column and appropriate partitions (Added in 4.1.1.1).
We've added the ability to reindex your SQL warehouse for the knowledge graph. This ensures that Prophecy's AI agent has the latest information about your datasets (Added in 4.1.1.2).

The following fixes are available in this release.

We updated tornado and jupyter_core in the Python sandbox to address CVE-2025-47287 and CVE-2025-30167, respectively.
SparkEdge could not authenticate with Artifactory when secrets were stored as files because we required the master password to be available as an environment variable. We now support mounting the encrypted master password as a file via a Kubernetes secret. This fix applies to self-hosted deployments only.
Spark project deployment is no longer blocked when you are missing service principal on an unrelated fabric using Databricks OAuth (Fixed in 4.1.1.1).

The following are the newest available Prophecy Library versions in this release.