Configure A4T reports in Analysis Workspace for auto-target activities | Adobe goals (2023)

Prize

The Analytics for Target (A4T) integration for auto-targeting activities uses Adobe Target ensemble machine learning (ML) algorithms to select the best experience for each visitor based on their profile, behavior and context, all while using an Adobe Analytics measure metric.

While comprehensive analytics are available in the Adobe Analytics Analysis Workspace, there are some changes to the standardAnalysis for goalspanel is required to correctly interpret Auto-Target activities due to differences between experiment activities (manual A/B testing and auto-assignment) and personalization activities (Auto-Target).

This guide reviews the recommended customizations for analyzing Auto-Target activities in Analysis Workspace, which are based on the following key concepts:

• OfControl versus targetingdimension can be used to distinguish control experiences from those in the Auto-Target ensemble ML algorithm.
• Visits should be used as the normalizing measure when looking at performance breakdowns by experience level. Besides,The default counting method in Adobe Analytics may include visits where the user cannot actually see the content of the activity, but this default behavior can be changed by using a properly scoped segment (details below).
• Visit-lookback-scoped attribution, also known as the "visit-lookback window" on the prescribed attribution model, is used by the Adobe Target ML models during their training phases, and the same (non-default) attribution model must be used when decomposing the target metrics.

Create A4T for the Auto-Target panel in the Analysis Workspace

To create an A4T for Auto-Target report, start withAnalysis for goalspanel in the Analysis Workspace, as shown below, or start with a freeform table. Then make the following selections:

1. control experience: You can choose any experience; however, you will override this choice later. Note that for Auto-Target activities, the control experience is actually a control strategy, namely a) Serve randomly between all experiences, or b) Provide a single experience (this choice is made during activity creation in Adobe Target). Even if you chose option (b), your automatic target activity has designated a specific experience as a control. You still need to follow the procedure described in this guide to analyze A4T for auto-target activities.

2. normalizing metric: Select Visit.

3. Success statistics: While you can choose any metric to report on, you should generally see reports on the same metric selected for optimization when you create activities in Target.

   

   Figure 1: Configuring the Analytics for Target panel for Auto-Target activities.

Use the Control vs. dimension. Targeted to compare the Target Ensemble ML model with your control

The standard A4T panel is designed for classic (manual) A/B testing or automated assignment activities where the goal is to compare the performance of individual experiences to the control experience. However, in Auto-Target activities, the first-order comparison must be between the controlstrategyand the targetedstrategy. In other words, to determine the influence of the overall performance of the Auto-Target ensemble ML model on the control strategy.

To perform this comparison, you must useControl vs. targeted (Analytics for Target)dimension. Drag and drop to replaceTargeted experiencesdimension in the standard A4T report.

Note that this replacement invalidates the default lift and confidence calculations on the A4T panel. To avoid confusion, you can remove these statistics from the default panel and leave the following report:

Figure 2: The recommended baseline report for Auto-Target activities. This report is configured to compare targeted traffic (provided by the ensemble ML model) to your control traffic.

Add breakdowns of stats by experience level

To better understand how the ensemble ML model performs, see experience level breakdownsControl versus targetingdimension. Drag it into the Analysis WorkspaceTargeted experiencesdimension to your report, then split each of the control and target dimensions separately.

Figure 3: Breakdown of the targeted dimension by target experiences

An example of the resulting report is shown here.

Figure 4: A standard Auto-Target report with experience level breakdowns. Remember that your target value may be different and your control strategy may have a single experience.

Why 'Visit' is the right default for Auto-Target activities

When analyzing an automated goal activity, always select Visits as the default normalization metric. Auto-Target Personalization selects an experience for a visitor once per visit (formally once per Target session), meaning that the experience presented to a visitor may change with each visit. So if you use Unique Visitors as a normalizing metric, the fact that a single user can see multiple experiences (on different visits) will lead to confusing conversion rates.

A simple example illustrates this point: consider a scenario where two visitors participate in a campaign that only has two experiences. The first visitor comes by twice. They are assigned to experience A on the first visit, but to experience B on the second visit (because their profile status changes on the second visit). After the second visit, the visitor converts by placing an order. The conversion is attributed to the most recently viewed experience (experience B). The second visitor also walks by twice and is shown experience B both times, but never converts.

Let's compare visitor and visitor level reports:

Table 1: Example of comparison of visit normalized and visit normalized reports for a scenario where decisions are linked to a visit (and not a visitor, as in regular A/B testing). Normalized visitor metrics are confusing in this scenario.

As can be seen in the table, there is a clear discrepancy between the visitor figures. Although there are two unique visitors in total, this is not a sum of individual unique visitors for each experience. While visit-level conversion rates aren't necessarily wrong, visit-level conversion rates probably make a lot more sense when comparing individual experiences. Formally, the unit of analysis ("visit") is the same as the unit of resolution, meaning that breakdowns of metrics can be added and compared at the experience level.

Filter by actual visits to the activity

Adobe Analytics' default counting method for visits to a Target activity may include visits where the user has not interacted with the Target activity. This is due to how target activity mappings are held in the Analytics visit context. As a result, the number of visits to the Target activity can sometimes be inflated, resulting in a decrease in conversion rates.

If you prefer to report on visits where the user actually interacted with the auto-target activity (whether through an access to the activity, a view or visit event, or a conversion), you can:

1. Create a specific segment with hits from that target activity, and then
2. Filter the Visits metric using this segment.

To create the slice:

1. chooseComponents > Create Segmentoption on the Analysis Workspace toolbar.
2. Specify onetitlefor your segment. In the example below, the segment is called "Hit with specific Auto-Target activity".
3. DragsTarget activitiesdimension to the segmentDefinitionsection.
4. Usecorresponding tooperator.
5. Search for your specific target activity.
6. Click the gear icon, then selectAttribution model > Instanceas shown in the image below.
7. ClickRedden.

Figure 5: Use a segment like the one shown here to filter the Visits metric in your A4T for Auto Target report

Once the segment is created, use it to filter the Visits metric so that the Visits metric only includes visits where the user interacted with the target activity.

To filter visits with this segment:

1. Drag the newly created segment from the Components toolbar and hover over the bottom of itVisitmetric label for a blueSort byprompt vises.
2. Drop the fragment. The filter is applied to this metric.

The final panel looks like this:

Figure 6: Reporting panel with the 'Hit with specific automatic target activity' segment applied to the Visits metric. This segment ensures that only visits where a user actually interacted with that target activity are included in the report.

Make sure the target metric and attribution are aligned with your optimization criteria

The A4T integration enables the Auto-Target ML modelTraineduses the same conversion event data that Adobe Analytics usesgenerate performance reports. However, there are certain assumptions that must be used when interpreting this data when training the ML models, which are different from the default assumptions made during the reporting phase of Adobe Analytics.

In particular, the Adobe Target ML models use a visit-based attribution model. That is, the ML models assume that a conversion must occur on the same visit as a content view for the activity in order to "attribute" the conversion to the decision made by the ML model. This is necessary for Target to ensure timely training of its models; Target cannot wait up to 30 days for a conversion (the default attribution period for reports in Adobe Analytics) before including it in the training data for its models.

Thus, the difference between the attribution used by the Target models (during training) and the default attribution used when querying the data (during report generation) can lead to inconsistencies. It may even appear that the ML models perform poorly when it is really about attribution.

The exact metric definition and attribution settings dependthe optimization criterionthat you entered when you created the activity.

Target defined conversions or Analytics metrics withMaximize the metric value per visit

When metric is a goal conversion or an Analytics metric withMaximize the metric value per visit, the definition of goal metrics allows multiple conversion events to occur during the same visit.

To view target metrics that have the same attribution method used by the Target ML models, follow these steps:

1. Hover over the gear icon for the target stats:

   

2. Scroll to the resulting menuData settings.

   (Video) Adobe Analytics Activity Conversions

3. chooseUse a non-standard attribution model(if not already selected).

   

4. ClickEdit.

5. chooseModel:Participation, iReview window:Visit.

   

6. ClickTo apply.

These steps ensure that your report attributes the target metric to the experience view if the target metric event occurredevery moment("participation") in the same visit that an experience was displayed.

Analytics statistics withUnique Visitor Conversion Rates

Define the visit with a positive metric segment

In the scenario you selectedMaximize the conversion rate of unique visitsas an optimization criterion, the correct definition of conversion rate is the proportion of visits where the metric value is positive. This can be achieved by creating a segment that filters for visits with a positive value of the metric and then filtering the visit metric.

1. As before, you must chooseComponents > Create Segmentoption on the Analysis Workspace toolbar.

2. Specify onetitlefor dit segment.

   In the example below, the segment is called "Visit with an order".

3. Drag the base metric you used in your optimization goal into the segment.

   In the example below we useOrder: % pmetric, so the conversion rate measures the number of visits where an order was registered.

4. In the top left of the segment definition container, selectInvolving Visit.

5. Useis greater thanoperator and set the value to 0.

   Setting the value to 0 means that this segment includes visits where the order metric is positive.

6. ClickRedden.

Figure 7: Segment definition filtering for visiting positive orders. Depending on your activity's optimization metric, you should replace orders with an appropriate metric

Apply this to the visits in the activity filtered metric

This segment can now be used to filter by visits with a positive order count and where there was a hit for the Auto-Target activity. The procedure for filtering a metric is similar to before, and after applying the new segment to the already filtered visitor metric, the report window should look like Figure 8

Figure 8: The report window showing the correct unique visit conversion metric: the number of visits that recorded a hit of the activity and where the conversion metric (orders in this example) was not zero.

Final step: Create a conversion rate that captures the magic above

With the changes to the Visits and Goals metrics in the previous sections, the last change you need to make to your standard A4T auto-targeting reporting window is to create conversion rates that have the right ratio: the corrected Goals metric to a properly filtered 'Visits' metric.

Do this by creating a calculated metric using the following steps:

1. chooseComponents > Create Metricsoption on the Analysis Workspace toolbar.
2. Specify onetitlefor your measurements. For example, "Visit-adjusted conversion rate for activity XXX."
3. chooseFormat= Percent medecimals= 2.
4. Drag the appropriate goal metric for your activity (eg, Activity Conversions) into the definition and use the gear icon on that goal metric to adjust the attribution model to (Attendance|Visit), as described earlier.
5. chooseAdd > Containerfrom the top right corner ofDefinitionsection.
6. Select the division operator (÷) between the two containers.
7. Drag your previously created segment named "Hit with specific Auto-Target activity" into this tutorial for that specific Auto-Target activity.
8. DragsVisitmetric in the segment container.
9. ClickRedden.

The full calculated metric definition is shown here.

Figure 7: The Visit Adjusted and Attribution Adjusted Model Conversion Rate Statistics Definition. (Note: this metric is dependent on your target metric and activity. In other words, this metric definition cannot be reused across activities).

Summary: Final example of the Analysis Workspace pane for auto-targeting reports

Combining all the above steps into one panel, the image below shows a complete overview of the recommended report for Auto-Target A4T activities. This report is the same as the one used by the Target ML models to optimize your target metrics. The report includes all the nuances and recommendations discussed in this tutorial. This report is also the closest to the counting methods used in traditional Target report-driven Auto-Target activities.

Click to expand the image.

Figure 10: The final A4T Auto-Target report in Adobe Analytics Workspace, which combines all the metric definition adjustments described in the previous sections of this tutorial.