MongoDB connection
MongoDB is a NoSQL database designed to store and retrieve unstructured or semi-structured data using BSON documents.
Prerequisites
When you create a MongoDB connection in Prophecy, access permissions are tied to the credentials you use. This is because Prophecy uses your credentials to execute all data operations, such as reading from or writing to collections.
To fully leverage a MongoDB connection in Prophecy, you need the following MongoDB permissions:
Read
from the collection defined in the connectionWrite
to the collection defined in the connection
Feature support
The table below outlines whether the connection supports certain Prophecy features.
Feature | Supported |
---|---|
Read data with a Source gem | Yes |
Write data with a Target gem | Yes |
Browse data in the Environment browser | Yes |
Connection parameters
To create a connection with MongoDB, enter the following parameters:
Parameter | Description |
---|---|
Connection Name | Name to identify your connection |
Protocol | Protocol to use to communicate to the database Example: mongodb+srv for cloud-hosted clusters |
Host | Where your MongoDB instance runs Example: cluster0.<cluster-name>.mongodb.net for cloud-hosted clusters |
Username | Username for your MongoDB instance |
Password (Secret required) | Password for your MongoDB instance |
Database | Default database for reading and writing data |
Collection | Collection to use for the connection |
Data type mapping
When Prophecy processes data from MongoDB using SQL warehouses, it converts MongoDB-specific data types to formats compatible with your target warehouse. This table shows how MongoDB data types are transformed for Databricks and BigQuery.
MongoDB | Databricks | BigQuery |
---|---|---|
32-bit integer | INT Alias: Integer | INT64 Alias: Integer |
64-bit integer | BIGINT Alias: Bigint | INT64 Alias: Integer |
Double | DOUBLE Alias: Double | FLOAT64 Alias: Float |
Boolean | BOOLEAN Alias: Boolean | BOOL Alias: Boolean |
String | STRING Alias: String | STRING Alias: String |
Date | TIMESTAMP Alias: Timestamp | TIMESTAMP Alias: Timestamp |
Timestamp | TIMESTAMP Alias: Timestamp | TIMESTAMP Alias: Timestamp |
Binary | BINARY Alias: Binary | BYTES Alias: Bytes |
ObjectId | STRING Alias: String | STRING Alias: String |
Decimal128 | DECIMAL(34,2) Alias: Decimal | BIGNUMERIC(34, 2) Alias: Numeric |
Min key | STRING Alias: String | STRING Alias: String |
Max key | STRING Alias: String | STRING Alias: String |
Regular Expression | STRING Alias: String | STRING Alias: String |
DBPointer | STRING Alias: String | STRING Alias: String |
JavaScript | STRING Alias: String | STRING Alias: String |
JavaScript with scope | STRING Alias: String | STRING Alias: String |
Symbol | STRING Alias: String | STRING Alias: String |
Embedded Document | STRUCT Alias: Struct | STRUCT Alias: Struct |
Array | ARRAY Alias: Array | ARRAY Alias: Array |
Empty Array | ARRAY Alias: Array | ARRAY Alias: Array |
Null | STRING Alias: String | STRING Alias: String |
Unidentified | STRING Alias: String | STRING Alias: String |
Learn more in Supported data types.
Sharing connections within teams
Connections in Prophecy are stored within fabrics, which are assigned to specific teams. Once a MongoDB connection is added to a fabric, all team members who have access to the fabric can use the connection in their projects. No additional authentication is required—team members automatically inherit the access and permissions of the stored connection credentials.
Be mindful of the access level granted by the stored credentials. Anyone on the team will have the same permissions—including access to sensitive data if allowed.
To manage this securely, consider creating a dedicated fabric and team for high-sensitivity connections. This way, only approved users have access to those credentials.
Fetching data
Prophecy fetches data from MongoDB connections in the following ways:
-
When you browse a MongoDB connection in the Environment browser, Prophecy fetches data on demand as you expand folders. You can manually refresh the Environment browser to see updated files.
-
When a pipeline runs, Source gems will read the latest available version of the data. Keep in mind that schema evolution may or may not be picked up automatically depending on the type of Source gem used.