Data Flows
Data Flows is a visual tool within the Zeta Marketing Platform (ZMP) that can ingest and integrate any data source either one time or as an ongoing feed. With Data Flows, you can seamlessly manage data workflows and conduct master data management tasks like cleansing, accuracy checks, resolving duplicates, and flexible extract, transform and load (ETL) functions through a straightforward, low-code user interface. Furthermore, use this powerful feature to:
Accelerate the speed of data integration and onboarding to drive timelier customer engagement
Minimize or eliminate the need for technical intervention to manage your own data on your own terms
Harmonize your data so that your systems are connected and are speaking the same language
Simplify your data operations and gain complete transparency into your most valuable asset—your customer data
From the menu on the left, navigate to Data > Data Flows.
Click on New Data Flow to open the canvas.
Data Conductor’s data flows are currently comprised of these three 3 components.
Click here to see how the Amazon S3 Buckets, Google Buckets, Microsoft Azure Blob Storage and External SFTP can be configured for self-serve connection management.
You can also schedule the execution of Snowflake queries and Google BigQuery queries. The schedule options available are: Every 5 minutes, every 15 minutes, hourly, daily, weekly, and monthly.
Any queries created for either Snowflake or BigQuery from within Data Flows can only be used in those flows. Query Segments created from Segments and Lists page can be used in Data Flows as well.
Source Of Data (required) | Destination of Data (required) | Actions on the Data (optional) |
|---|---|---|
This is where you will be pulling the data from. | This is where you will be sending the data to. | These are various actions that can be taken on the data to ensure that the data has everything it needs for use by the destination of the data. |
Everyone to have access to all sources | Everyone to have access to all sources | Some Data Actions will be restricted to certain accounts if built custom for the client, or if those actions require payment. |
Types of Sources
| Types of Destinations
| Available Actions (changes often)
|
Visualize Output Data
You can upload a sample file in order to visualize the end result file, which will have been modified after passing through the data flow’s action nodes.
1. Navigate to Data > Data Flows within ZMP. Click on New Data Flow.
2. Within the data flow canvas, finish the entire flow and click on Save.
3. You can select the Activate after save toggle or Upload sample file after save or both.
The allowed file types for upload are CSV and JSON.
2. Based on the sample file uploaded:
the action cards will be stacked,
source summary will be displayed at the top-right, and
the destination card will show that the data is being processed.
 |
You can hover over the stacked Action cards to spread them apart for viewing. 3. While the summary stats of uploaded source files and destination files are displayed, click on View Data to view the source data OR destination data in the table view mode.
The data flow canvas is NOT editable if you’ve chosen the Activate after save toggle since you will need to deactivate the flow before editing (existing behavior).
4. If you had chosen the Upload sample file after save, the data flow is editable and in a Draft state. Click on Save/Activate.
5. Go back to the data flow list view to see the status of the sample flow and get notified when the sample file processing is complete.
6. Click on the flow to navigate to the canvas.
Share Data Flows
Reach out to you account team if you want this functionality enabled for your ZMP instance.
You can share data flows across accounts and copy feeds.
1. Navigate to Data > Data Flows within ZMP.
2. Click on the action menu against the data flow you wish to share and select Share.
3. As the Share Data Flow panel slides in from the right, click on Share to Another Account.
4. Click on Add Account and select the destination account from the scrollable list that appears.
5. Once the destination account has been selected, navigate to the Approvals section and enter the Approver email address(es) to add people who can review the flow in the destination account.
6. Click on Share Data Flow. In the pop-up, click on Share to Account.
   Once approved, the approver can Activate the data flow in the destination account.  |
Meanwhile, the added approver can go into the flow from the destination account and click on Data Flow Feed Schedule
As per the data flow feed schedule, if the start date is not set, the data flow will pick up the data. However, once the start date is set, the flow will no longer search for any files uploaded prior to that date. Therefore, uploading a file before the scheduled date would not result in processing it; it should be done after the start date.
Once the start date is set, the data flow won't look for any files uploaded earlier than the start date.
Data Flow State Change Notifications
Email notifications are sent out under the following scenarios:
Data Flow is Activated
Data Flow is Deactivated
Data Flow is Shared from one account to another
Data Flow is requested to be approved
Data Flow is Approved
Data Flow is Rejected
In all these cases, users who are on the ZMP notification list will receive these emails as well as users whose emails were added in the Notifications or Approval steps when a data flow is shared.
Users who do not want to receive these emails will be able to remove themselves and even add back on in the future if their needs change.