Skip to main content

Prophecy 3.4.4.x

January 31, 2025

  • Prophecy Python libs version: 1.9.33
  • Prophecy Scala libs version: 8.8.0

Features​

Expanded business rules capabilities​

The new business rule capabilities include:

  • Setting a timeframe in which business rules should apply.
  • Automatic detection of conflicting rule logic.
  • Defining multiple output columns per business rule.

Updates​

New custom gem function​

When you are building a custom Spark gem, you can include the customOutputSchemaEnabled function in your gem code to enable or disable to Custom Schema toggle by default. This toggle can be found in the Output tab of a gem.

Advanced mode for SQL target model​

When you want to overwrite the default database for your target model, you can now use advanced mode to dynamically populate the field. Rather than inputting a simple string, you can now use functions, values, and variables in your database definition.

Use multiple unique keys​

Previously when writing a SQL model using an SCD 2 merge approach, you could only define one unique key. This update lets you define one or more unique keys.

Multi-tenancy support for Hashicorp Vault​

When you set up Hashicorp Vault as a secret provider, a new optional Namespace field lets you specify the isolated environment in your Vault.

Customer-managed encryption keys (CMEK) for BigQuery​

BigQuery SQL fabrics now include a field that lets you include your CMEK. This way, you can access datasets protected by CMEK in Prophecy.

Fixes​

Improved performance on UC shared clusters​

We have removed some redundancy and have fixed how partial runs execute on Databricks UC shared clusters.

Allow use of tables with quotes​

This fix lets you use Snowflake tables that include quotation marks in the table name.

Better error detection in Gem Builder​

Errors captured from sandbox are now displayed with a stacktrace showing the lines in your code which caused an exception to be thrown.