Marketing that Learns

Overview

The Marketing that Learns chart is a real-time visualization of Zeta's Moment Scoring technology. As more and more Ads are served and scored, our models learn about the user attributes that best drive performance to your campaign's goal.
This report contains three charts:

  • The Marketing that Learns Chart

  • The Model Management Chart: Shows how your campaign's models are affecting various attributes over time

  • The Clicks Chart: Shows the number of clicks (or conversions, or other goals) of your models over time.

Currently, Viewability and Video Completion Models are not compatible with this report. This feature will be added in a future release.

How it Works

Zeta's Moment Scoring gains insight every time an impression is served – that's over 100 billion bid requests, or moments, per day, from which our models learn. Our technology scores each of those moments and uses that information to improve its model for predicting the moments that will drive impact against the campaign's objective. In turn, the model seeks more such moments – moments to engage your customers, and drive conversions. This ability to learn, adapt, and evolve is called "Marketing that Learns". The result is better campaign performance that drives better results, such as a lower CPA.

Predictive models combine "data mining" and statistical modeling techniques to predict whether the display of an ad will have the desired result. They are more sophisticated than "bidding rules", which typically include just a few considerations (e.g. bid $3 CPM on women ages 35-55 in the Northeast US). Our models consider millions of possible attributes, including, for example:

  • Page visits (previous and current)

  • Actions taken

  • Device and browser details

  • Location

But at the start of a campaign, the amount a model knows is limited. A model begins to learn, initially, when users visit the advertiser's site, picking up information from the site's Pixels. This is why thorough and early installation of Pixels is so important. As a rule of thumb, Zeta recommends placing Pixels at least two weeks prior to the start of your campaign.

Employing Pixel data, a model begins to form hypotheses about who is likely to convert. Once conversions begin a model typically begins a steep learning curve that accompanies a significant lowering of CPA and an ever-increasing number of conversions. This feature is not limited to CPA campaigns, but works for other types of campaigns as well.

Zeta has created models in various categories such as Conversion, Click, and Viewability. There are multiple types in each category. For example, model types in the Conversion category include those intended for new campaigns, for difficult conversion situations, for click-through campaigns, etc.

Opening the Marketing that Learns Report

  1. Navigate to the DSP logo in the upper left corner, and then click the menu icon. The DSP shows a menu for Advertisers and Admin.

  2. Select the Account that contains your Advertiser, and then select your Advertiser. The DSP shows all the Advertiser's campaigns.

  3. Select the campaign you would like to view from the list.

Opening the Report

  1. With the campaign open, click the caret next to Manage Campaign (atop the page), and then click Campaign Analytics. The Report Builder appears. From this report, you can access all other Insights reports.

2. Click the caret next to Report Builder, and select Models > Marketing That Learns from the drop-down menu.

Analyzing Model

If the campaign is using multiple models, select which model to analyze the campaign against. This gives you the story for significant attributes over time by the different model IDs. Each model uses a different color scheme in the charts.

A model is listed as "Inactive" if it has not served any impressions in the last seven days. This can happen if the campaign is paused or if the budget of the campaign was spent over seven days ago.

Date Range

Click the Date Range to view the date selector pop-up. Click two dates in the pop up, which form the starting and end dates for the charts.
To include the entire campaign:

  1. Click the Date Selector.

  2. Click Use all Campaign Dates.

  3. Click the X.

Marketing That Learns Chart

This chart displays the relationship between what the campaign's model(s) have learned and how this has affected CPA.

  • The blackline, with the axis at left, measures the campaign's overall CPA.

  • The colored line, with the axis at right, measures the number of significant attributes the campaign's model used on each date. The color of this line corresponds to the model you have selected.

In the example above, note how over time as the model learned more, CPA dropped considerably.

Model Management Chart

When you have multiple models running behind your campaign, you can see how each model affects your campaign. This also helps us decide which is the best model to run on a campaign. The color of the lines on the Marketing that Learns chart corresponds to the color used on the models in this chart.

Clicks Chart

The Clicks Chart appears below the main chart. This shows your goals over the same time period as the first graph.

Hover Display

Hover over any graph to display detailed information from the date you are hovering over.

Which Metrics Appear

The performance metrics displayed in the DSP depend on the campaign objective declared.

Campaign Objective

Performance Metric

Results Graph Metric

CPA

Cumulative CPA

Conversions

Conversions

Cumulative CPA

Conversions

CPC

Cumulative CPC

Clicks

CPM

Cumulative CPC

Clicks

Clicks

Cumulative CPC

Clicks

CTR

Cumulative CPC

Clicks

Impression Day ROAS

Cumulative ROAS

Client Revenue

Export Image

Click Export Image in the upper-right corner to save the graphs to a PNG (image) file.

FAQ's

Question

Answer

On some campaigns, I notice that the attributes curve goes up and down quite a bit. What is the reason for that?

The AI optimizes its models constantly as new data arrives, meaning that from day to day it can add or remove dozens of attributes as it tries to zone in on the sweet spot for your campaign.

In addition, sometimes these changes can be the result of changes you make in your campaign. For example, suppose initially your Ads feature famous basketball players endorsing your product, but later your change your Ads to show dolls and ponies. As the new Ads will probably appeal to a different sort of customer the model must try a new approach to find a new sweet spot.

I understand that the DSP AI employs millions of attributes. Why am I only seeing hundreds in use for my campaign?

Although our models evaluate millions of possible attributes to find those most relevant for a campaign, this screen highlights only the one hundred or so that matter most.

How often does the chart update with data for the model attributes?

The platform generates the Marketing that Learns chart once a day, and with a two-day lag on model data.

I changed my campaign to try improving learning, but when I came in the next day nothing seems to have changed? Do I need to make more dramatic changes?

When significant changes (e.g. changing goals or tactics) are made to a campaign, the model usually needs time to re-learn and optimize performance for the new setup. Reaching an optimal state can take several weeks, but initial model learning will often be visible after 3-4 days.

Why did the attributes curve begin high from day one?

This likely indicates that someone managing the campaign has chosen a Model Group (see figure below) on the Campaign Setup screen. This functionality groups the current campaign with another campaign for the same advertiser, which enables borrowing what the model learned. If you later remove the campaign from the grouping, you may see a brief dip in the curve until the models retrain themselves.

My campaign's Pixels have been live for two weeks and have met Zeta's recommended 1000 fires and 200 conversions guidelines. Why are there no significant attributes?

The default model shown in the chart learns from conversions driven by Zeta, and Therefore, it usually starts with zero attributes at the campaign launch. In addition, we have other models learning from unattributed conversions that help drive performance for new campaigns. Therefore, the early placement of Pixels remains critical to campaign success.

Why does the attributes curve end on the last day of the campaign, but the performance curve continues for another thirty days?

For certain types of campaign goals, e.g. CPA, Conversions, Zeta's technology continues to attribute conversions to Ads for up to thirty days after the end of a campaign.

Where can I view the details of what the model has learned about my customers?

Please see the Insights screen to learn more details about the specifics of your overall campaign. Note that Insights data is not based directly on our models but is generated by matching campaign conversions to data provided by third-party data providers.

At this time, we do not show specific model attributes with the chart. Our models evaluate millions of different attributes at any given time making the data complex. As part of future enhancements, we are evaluating the best way to bring more context to what the graph shows.

Why is my campaign not performing well right after launch?

The DSP's AI is marketing that learns, not marketing that knows. This means:

  • Our models require live campaign data to refine their learning{color} and make optimization decisions. Specifically, our models learn from conversions, and the factors in a campaign that drive conversions.

  • Ideal times of day or times of week for displaying Ads are examples of attributes that can only be learned over time.

  • Zeta's performance generally improves after two weeks of data, after learning has occurred.

How many significant attributes should I expect for a campaign w a given number of conversions?

There is no rule. Generally, our models gain a high level of precision after you've hit at least 100 conversions in the last 56 days. (The model only looks at conversions from prior 56 days)

How long does it take a model to reach optimal learning?

The model typically reaches a "sweet spot" of intelligence after about one month.

Are the shown conversions Zeta conversions or third-party conversions?

If third-party conversion data is provided, then that data is displayed.

Why does the CPA in the graph not match the daily CPA displayed in the Analyze tab in the DSP?

The Analyze tab in the platform provides a view of your daily CPA, whereas the chart shows the cumulative CPA since the beginning of your campaign.

Zeta's display targeting should be viewed as a distinct element within an advertiser's broader marketing mix. As a result, Zeta and paid search are not comparable for the following reasons:

  • Paid search represents a pull marketing tactic, in which users are actively seeking information on your product or service and Ads are served.

  • Zeta provides technology to support push marketing tactics, in which we actively direct Ads pertaining to your product or offer toward a targeted audience based on AI bidding decisioning.

  • In push marketing the audience may not actively be shopping for your product at that time, making users more passive and a more challenging conversion than paid search.

  • A diversified media mix is critical to driving strong ROAS. As a result, including both push and pull tactics in your marketing mix, with the tactics working together to drive stronger returns for the business by. pulling in both familiar, active shoppers and engaging passive new customers, who might convert if presented with the right message at the right time.

What is the difference between the attributes we show in Insights and what we represent in the Marketing That Learns graph?

Insights should be used in conjunction with the Marketing That Learns Graph, not as a replacement.
The Insights data may display related data on demographics, campaign performance, etc., but is not based directly on our models. The Marketing that Learns chart better displays what our AI is learning and doing and is based on AI.