Importing Events

Overview

Capturing your users’ behavior through events is important for enabling a complete view of your users’ experience with your brand. Events enable Zeta to power 1:1 triggered experiences, capture user interest signals for personalization, and to give you a holistic view of your users’ actions. This article outlines details of Zeta’s File-Based Event Imports to help you set up your data integration.

Why Use File-Based Event Imports?

Integration of event information into the Zeta Marketing Cloud can be accomplished easily in near real-time with implementation of Zeta’s JavaScript library or Event Tracking REST API. Sometimes, though, there are situations where offline or other user behavioral data is most easily delivered in data files. Zeta makes it easy to set up automated ingestion of these data feeds, delivered to a Zeta-hosted FTP server, for immediate use in Zeta.

Enabling FTP Imports

Your Zeta Solutions Architect or Account Manager can provision and enable file imports for your account.

If your site already has enabled file imports for users or exports of event or user data via FTP, this step is already done!

Get Your FTP Credentials

Once enabled, you’ll find an FTP username and password in the Settings → Integrations section of the application. You will use these credentials to access the Zeta-Hosted FTP server for delivering your data file.

Connecting to the FTP Server

Connect to the Zeta FTP server, with your FTP credentials and find the “events” folder:

Protocol: SFTP
Hostname: sftp.boomtrain.net
Logon Type: User/Password Authentication
Directory/events

All properly-formatted data files dropped into this directory will be start being processed into your Zeta account within a couple of minutes. Zeta’s systems will start processing your file within five minutes of a completed upload to this server. To ensure that your file has been completely transferred to Zeta’s servers, the server implements a a short delay from the time your file finishes uploading until the start of processing.

Formatting Your Files

  • Each file dropped into the FTP servers should be a UTF-8 encoded JSONLines file with extension .jsonl.

  • Each file dropped must have a unique filename. This is in place to protect against accidentally dropping the same file twice, which could result in duplicate campaigns being sent to your users.

  • Each line in the file should be a JSON object that represents a single event to be imported into your account.

Three fields will be required in each line:

  1. Either user_id or email string. These identifiers are used to determine which user the event will be tracked for. Note that if both user_id and email are included, and each are mapped to different users currently, the event will be tracked to the user who matches the user_id field.

  2. A timestamp for the event represented as an ISO-8601 timestamp string or an epoch time integer in milliseconds.  that represents a time within one year in the past and one day in the future. Note that we recommend delivering event files as soon as possible; this enables more real-time actions on the events. Note that triggered messages will not send for events that are processed more than 24 hours after their timestamp.

  3. An event_type string, representing the type of the event, e.g. “purchased”, “viewed”, “liked”, “shared”.

If this is a user who has never been seen before, we will create a new user for the event with those identifiers.

Events must not contain the following fields, as they are reserved by Zeta Events system:

  • enriched

  • identity

  • status

  • identified

  • properties

  • event_id

  • is_test

  • bsin

  • site_id

Fields using the following keys will be inserted into the top level field of the event of the same name, failing if they don’t meet the following validation:

Field

Type

Description

url

String

The URL of the resource being acted upon.

resource_id

String

The ID for the resource being acted upon.

resource_type

String

The type for the resource being acted upon.

All other fields will be inserted into the properties object of the event.

Sample JSONL File

{"user_id": "1234","event_type": "purchased", "timestamp": "2018-09-01T18:41:44", "shopping_cart_items": [{"item_name": "Shoes","price": 123.99},{"item_name": "Pants","price": 34.99},{"item_name": "Shirt","price": 47.00}], "keywords": ["clothing", "footwear"]}

{"user_id": "1235","event_type": "purchased", "timestamp": "2018-09-01T18:46:12", "shopping_cart_items": [{"item_name": "Shoes","price": 123.99}],"keywords": ["clothing", "footwear", "My \"Quoted\" Value"]}

Uploading Multiple Files

Archive/Compressed Files

Files may be dropped individually or bundled into an archive/compressed file that has one of the following compression formats:

  • zip (.zip)

  • gzip (.gz, .tar.gz, .tgz)

  • bzip2 (.bz2, .tbz2, .tar.bz2)

Each item (with the exception of an optional manifest file) in the archive should be a valid file at the top-level of the archive (no sub-directories will be traversed). Any files that fail validation in an archive will trigger a warning state in the status log (see 3 and 5 for details) indicating that the specific files failed to process, but the rest of the successfully-processable files in the archives will proceed.

Manifest File for Guaranteed Processing Order

Optionally, an archive may contain at the top level a single manifest file with the exact filename manifest.json that specifies the order of processing for the data files within the archive. Note that if no manifest file is included, the files within the archive will be processed in a non-guaranteed order (likely in parallel).

This manifest must contain an ordered list of files in the archive that are to be processed sequentially. If the manifest is present but not in a valid format, the entire import will fail with message “Invalid manifest file format.”

The format of the manifest is a JSON file as follows:

JSON Object Manifest

// JSON Dictionary with a single contents array with an ordered list of files to be found in the archive by filename. This comment is optional.

{

    "contents": [

        {"filename": "file1.jsonl"},

        {"filename": "file2.jsonl"}

    ]

}

  • If the manifest lists a file that is not included in the archive, the entire import will fail with message “File <missing_file.ext> was included in the manifest but not found in the archive.”

  • If the manifest does not list a file that is in the archive, the entire import will fail with message “File <missing_from_manifest.ext> was found in the archive but not included in the manifest.”

See Progress of Your File Import

Within a couple of minutes, you’ll see your file begin to be processed in the following areas of the Zeta application:

Feed Module

On the Dashboard page (https://boomtrain.net/dashboard), a feed item will appear which shows the status of the upload. This will update with progress in real time and indicate the result of the file processing once complete.

File History Page

A full listing of previous file drops can be found in the File Import History page, found in Settings, or at https://boomtrain.net/settings/file_imports. On this page, you’ll find a log of all event and user imports with their status. This page will update in real-time without the need to refresh for updates.

In the event that a file was unable to process due to one or more errors, those errors will be delivered here as a feedback file that can be accessed once the file has completely finished being processed. Simply click on the error icon to download the error file.