Exporting Incremental Logs to Databases via Remote Target

With this update, database export capability in the Automated Flow module has been enhanced. You can now export Incremental Logs directly to supported databases such as MySQL and others via Remote Target integration.

Prerequisites

• A Remote Target must already be defined. The flow can only export data to databases that have been configured as Remote Targets.
  For setup instructions, refer to the Remote Targets.
• Target tables must exist in your database. Before activating the flow, the tables where incremental data will be written must be created in your database.
  The table names, column names, and data types should match the structure shown under the Incremental Logs section.

How to Access

Navigation Path:
Data Space > Data Management > Automated Flows

Go to Data Space, then click Data Management, and open the Automated Flows section. Here, you can create and configure flows that automatically export incremental logs to external databases via Remote Target.

How to Use

1. Create a New Automated Flow

In the Automated Flows section, click New, then select Automated Flow.
Enter a name for your flow, set the Start Date and End Date, then click Next to continue.

2. Add the Recurring Trigger

Drag and drop the Recurring Trigger node onto the canvas.
Set the desired frequency for the export (e.g., daily, weekly).

Incremental Logs can only be used with the Recurring Trigger set to a Daily period.

• You can use either the "Occurs Once" or "Occurs Every" options. When using "Occurs Every", the minimum occurrence time can be 15 minutes.

3. Add the Data Export Node

In the Actions section, drag the Data Export node into the flow.
This node defines what data will be exported and where it will be sent. Click the node to configure its settings.

4. Choose the Data Type

In the configuration window, select the data type as Incremental Logs.

5. Select the Log Type

When you select the Incremental Logs, choose the relevant log type under Transactional or Marketing, depending on the data you want to export. Click Next to proceed.

6. Select the Remote Target (Database)

In the final step, select the Remote Target database from the list of available options.

• Google BigQuery
• Microsoft SQL
• PostgreSQL
• ODBC (Generic) - used for connecting to MySQL and other databases
• Oracle
• Amazon S3
• Amazon Redshift

The same steps apply for all supported database types.

Enter your database connection details, then save and activate your flow.