Adobe Target Flashcards by Anja Pedersen

A popular Travel and Hospitality website will occasionally change home page banner promotion for hotel rooms with seasonal deals and special rates for certain customers.

Choose the activity type that are best suited to meet the business needs of the above situation (Choose all that apply)

a) A/B test
b) Experience Targeting
c) recommendations
d) AP
e) Multivariate Test

b) Experience Targeting
d) AP
e) Multivariate Test

How well did you know this?

Not at all

Perfectly

Consider the following:

A financial services mobile app has recently started optimizing there digital properties with Adobe Target. Their Android and iOS apps have been using a very stale splash page to welcome users and stakeholders doesn’t agree on the changes that should be made to the images and messaging on the page.

Choose the activity type that are best suited to meet the business needs of the above situation (Choose all that apply)

a) A/B test
b) Experience Targeting
c) recommendations
d) AP
e) Multivariate Test

a) A/B test

e) Multivariate Test

How well did you know this?

Not at all

Perfectly

Consider additonal aspects of the same situation:

Choose the best approach to configuring the activity:

a) MVT test with VEC
b) A/B test with VEC using custom code option
c) A/B testing using form based workflow
d) A/B testing with Target Classic

c) A/B testing using form based workflow

How well did you know this?

Not at all

Perfectly

Mr. Singh has to setup a test to validate changes to two different elements of the home page which is receiving lots of traffic. Element A – Hero BannerElement B – Header Link In total, each of the elements had two versions including the default control version. Key stakeholders from the marketing team want to understand whether change in hero banner had a greater impact on the conversions than changing the header link? In this situation, which of the following test would be appropriate to identify the impact of elements.

a) Experience Targeting

b )A/B

c) MVT
d) Automated Personalization

c) MVT

How well did you know this?

Not at all

Perfectly

Our client XYZ.com ran a campaign on website to improve their order conversion with 95% of the confidence. For the next year, client had estimated total # of orders 10,000. Based on the results of the existing test (Table below) they have decided to go ahead with winner. As a Target BP you are tasked to calculate the incremental number of orders for next year.

a 13,000

b 3,000

c 23,000

d I need more information for this.

b 3,000

How well did you know this?

Not at all

Perfectly

You are currently running one MVT test where you have different offer combinations or discount which you would like to show to users at different paces on the page. You have configured and activated the campaign. One of your stakeholders saw one experience where users were seeing 2 contradicting offers. He raised this issue with your, what you will do or forget to do while configuring the test.

a It is expected behaviour in an MVT test

b You must have forgot to exclude the audience for that experience.

c You forgot to exclude the experience from ‘preview section’

d You forgot to exclude the experience within “Exclusion Group” section of test

c You forgot to exclude the experience from ‘preview section’

How well did you know this?

Not at all

Perfectly

Boss at Travel company wants to increase the booking numbers. A product manager suggesting that we should have the Flight and Hotel booking module on Hero Banner, however, UX manager is saying that this will impact the user experience of the customers and will increase the bounce rate from the page.Which test type you would recommend in this scenario?

a MVT

b A/B Test

c Behavioural Targeting

d Recommendation

b A/B Test

How well did you know this?

Not at all

Perfectly

Run an experiment targeting test with various set of segments.

Run an AB test to find the best layout and then run Experience Targeting test

Run a recommendation Test

Run and MVT followed by a Recommendation test

Run a recommendation Test

How well did you know this?

Not at all

Perfectly

A test was configured to show the experience for a new users who started his website journey from home.html with tablet device.
Below are 4 journey of 4 users, please choose who will be qualified for the experience?

Paul entered on website via home.html using mobile phone and added products to the cart in past but never made a purchase.
Simon is a new visitor and currently on home.html page of the website using Tablet devices and used paid search while landed at product.html page.
Rose is using a tablet device and visited few videos of the brand on youtube in the past. She started her journey from home.html
Charlie never visited the website in the past and started her journey on website first time from home.html using a mobile device.

Simon

Rose

Charlie

Paul

Rose

How well did you know this?

Not at all

Perfectly

Mr. John, an optimization practice lead had setup an A/B test 2 weeks back, which had an estimated test duration of three weeks. One of the experience reached confidence 95%+. Stakeholders approved to stop the test and analyze the results. The test showed a 10% increase in overall orders. The sample size was reached, which was set to prove up to a 20% difference in revenue per visitor (RPV) The following effects were observed:• 10% increase in order rate• 3% increase in page views per visit• 3% increase in average order size• 10% increase in product views A stakeholder asks the consultant to annualize the impact on overall revenue projected through the next year to present as program ROI. How should the optimization consultant proceed?

Inform the stakeholder that the projected revenue would increase by 10% as the order rate is directly proportional to revenue

Inform the stakeholder that the projected revenue would increase by 3% as the order size is directly proportional to revenue

Inform the stakeholder that the projected revenue cannot be determined as the test was designed to prove RPV

Inform the stakeholder that the test would need to run two times as long to prove difference in RPV

Inform the stakeholder that the projected revenue cannot be determined as the test was designed to prove RPV

How well did you know this?

Not at all

Perfectly

Verify the Activity Preview page to validate if that particular experience is getting delivered.

Validate the experience delivery using a browser incognito mode.

Validate the reports to check if that particular experience has any data against it

Use Adobe Debugger VEC setting for quick validation of the experience.

Validate the reports to check if that particular experience has any data against it

How well did you know this?

Not at all

Perfectly

Mr. Singh has setup an A/B test on the home page of a leading e-Commerce site to validate two different versions of a page element. Also there is global test running across the site which is not a key test for the top management as compared to the test setup on home page. What needs to be done by the optimization consultant to ensure that there is no conflict in the delivery of the content of both the tests? Select all that applies

Disable the global test as this is not a key test for stakeholders

Validate selector (elements on the page) collision

Validate priority levels for the test & ensure the home page test has priority over global test

Validate audience criteria

Validate selector (elements on the page) collision

Validate priority levels for the test & ensure the home page test has priority over global test

Validate audience criteria

How well did you know this?

Not at all

Perfectly

Review the activity url and template rules

Validate the Audience setup

Pause this activity and re-activate to resolve any backend issue

Will do a browser functionality testing

Validate data with Analytics for the same dimension/segment

Review the activity url and template rules

Validate the Audience setup

Will do a browser functionality testing

Validate data with Analytics for the same dimension/segment

How well did you know this?

Not at all

Perfectly

What are the key parameters required/recommended to implement the order confirmation mbox on the confirmation page.

1) orderId, orderTotal, productPurchaseId
2) orderNumber, revenue, productId
3) purchaseId, orderId, totalRevenue
4) order, units, & revenue

1) orderId, orderTotal, productPurchaseId

How well did you know this?

Not at all

Perfectly

You will setup an experience targeting activity

You will create an audience with mBox call is not triggered

You will create an audience with “not in other test”

How well did you know this?

Not at all

Perfectly

Based on the calculations derived using sample size calculator, Mr. Singh figured out that to run an A/B test on the campaign landing page to validate two different versions of CTA text would take 21 days. The activity was configured and was set to live. After two weeks it was assessed that the test duration is actually increasing and it would take more time than initially estimated. What are the possible reasons for increase in the test duration?

Under estimation of traffic numbers

Over estimation of traffic numbers

Increased estimated lift

Due to high volume of traffic observed while promoting special offers during the last two weeks

Over estimation of traffic numbers

How well did you know this?

Not at all

Perfectly

An AP test was setup by Mr. Singh on the home page of a leading financial institution. The test audience included users from geo location North America & the configuration included variations across five different elements i.e. CTA, Hero Banner, Header Link, Footer Link, & Banner Message. After setting up the test Mr. Singh had shared the Preview URL links with leading marketing folks across each region to get a sign-off before going live with the activity. One of the stakeholders intended to validate the activity to understand whether someone from North America was actually qualifying for the test. What needs to be done in this case to meet the stakeholder expectation?

Preview URL shared will allow you to validate audience qualification by default

Set the activity to live with actual audience to validate

Set the activity to live with QA audience(URL param=”target”) to validate

Set the activity to live & use multiple audiences which includes QA & actual audience to validate.

How well did you know this?

Not at all

Perfectly

Mr. Singh has setup an experience targeting test which has four different experiences. The activity went live last week and based on the observation from one of the key marketing folks, there was an issue where he was qualifying incorrectly. Below is the audience criteria along with the experience. How should the order of the experience-audience pair be configured while setting up the test?
Experience A Users coming from “Gurgaon” location
Experience B Users coming from “Haryana”
Experience C Users coming from “India”

Exp. B | Exp. C | Exp. A

Exp. C | Exp. B | Exp. A

Exp. A | Exp. C | Exp. B

Exp. A | Exp. B | Exp. C

How well did you know this?

Not at all

Perfectly

“Recently viewed content”, “Recently watched videos”, “Most viewed Items across the site” & “Top selling products from across the site” represent an example of which of the following?

Search & Promote Recommendations

Recommendations Criteria

Target Classic Product Rules

Recommendations Criteria

How well did you know this?

Not at all

Perfectly

Mr. Adam is an optimization consultant for a leading financial institution, who had setup an A/B test two weeks back. The test has reached significance which was designed to prove up to a 10% difference in orders at 95% confidence. The sample size was 50,000 visitors per variation. Based on the below results, what should Adam recommend keeping the primary KPI in mind.
Experience Visitors Orders Order Rate Lift Confidence
Control 23,075 846 3.666%
Experience B 23,085 1000 4.331% 18.3% 96%

Experience B is confidently better than Control. Use Experience B moving forward.

Continue to monitor change in lift until the lift observed is closer to estimated lift 10%.

Continue running the activity to set sample size because the results are inconclusive.

Divert 100% to experience B as the it is 96% confident of delivering 18.3% lift.

Continue running the activity to set sample size because the results are inconclusive.

How well did you know this?

Not at all

Perfectly

While using Adobe Target Sample Size calculator, the optimization consultant derived that the expected traffic would be 10,000 visitors per experience per day. But when observed in the ongoing activity this wasn’t the case and the traffic numbers were 50% less than the expected numbers. What is the first activity that should be performed by the consultant?

Re-validate the input data & derive a new conversion rate using the sample size calculator

End the activity immediately to avoid any further incorrect reporting data

Re-validate the activity setup, metrics & ensure that data is getting into reports

Create a duplicate test, make the required metric level changes based on new conversion rate derived & end the previous activity.

Re-validate the activity setup, metrics & ensure that data is getting into reports

How well did you know this?

Not at all

Perfectly

Disable the global test as this is not a key test for stakeholders

Validate selector (elements on the page) collision

Validate priority levels for the test & ensure the home page test has priority over global test

Validate audience criteria

Validate selector (elements on the page) collision

Validate priority levels for the test & ensure the home page test has priority over global test

Validate audience criteria

How well did you know this?

Not at all

Perfectly

Automated personlization activity is recommended only for the ecommerce websites as this is heavily dependent on conversion data.

True

False

How well did you know this?

Not at all

Perfectly

Mr. Smith, an optimization consultant had setup an A/B auto-allocate test three weeks back, which was planned to be run for 2 weeks. Due to over estimation of traffic during the initial phase, an additional week was required to reach the test completion based on pre-defined significance and sample size. The test showed a 10% increase in RPV, which was set to prove up to a 20% difference in number of orders The following effects were observed:
10% increase in order rate
3% increase in page views per visit
20% increase in number of orders
3% increase in average order size
10% increase in product views
A stakeholder asks the consultant to annualize the impact on projected orders for year 2018, which is estimated to be 160000 . How should the consultant proceed?

Inform the stakeholder that the projected incremental orders for 2018 would be 192000

Inform the stakeholder that the projected orders cannot be determined as the increase in RPV is less than the estimated lift 20% for number of orders

Inform the stakeholder that the projected orders for 2018 would be 192000

Inform the stakeholder that the projected orders for 2018 would be 32000

Inform the stakeholder that the projected orders for 2018 would be 192000

How well did you know this?

Not at all

Perfectly

P should circulate the preview experience URL's, so that stakeholders can validateMr. Singh has setup and A/B test to validate two different versions of campaign landing page on a leading cruise lines site. The audience criteria to qualify for this test is if the user is returning visitor with a prior booking on the site. The activity is planned to be live next week on PROD site and the stakeholders want to validate if the activity actually delivers the expected experience to the booked users. What needs to be done by BP to address the requirement? BP should circulate the preview experience URL’s, so that stakeholders can validate BP should setup QA audience and validate the content delivery on PROD BP should setup QA audience combined with AND returning booked guest audience to validate the content delivery on PROD BP can validate the delivery on staging site & assume the content will be delivered on PROD

BP should setup QA audience combined with AND returning booked guest audience to validate the content delivery on PROD

You configured a test and running this test for a week time. First weekly report was sent to stakeholders and they found that visitors number in the report are lower then the expected numbers and saying the configuration of test is not correct. what you will do to ensure that everything is correct? Check that all apply. Review the activity url and template rules Validate the Audience setup Pause this activity and re-activate to resolve any backend issue Will do a browser functionality testing Validate data with Analytics for the same dimension/segment

Review the activity url and template rules Validate the Audience setup Will do a browser functionality testing Validate data with Analytics for the same dimension/segment

BP wants to setup an A/B test across all the product detail pages. Sample URL for a product detail page is www.abc.com/products/men/123.html. How should the test be setup so that the correct audience qualifies for the test & meet the configuration requirements? Create an audience ->current page URL contains www.abc.com/products Create an audience ->landing page URL contains www.abc.com/products Configure-> Page Delivery-> Setup template rule & URL include www.abc.com/products/men/123.html A/B test can be setup only on a single page.

Create an audience ->current page URL contains www.abc.com/products

BP has created an AP test and is setting up the required audience segment for the activity. Audience criteria is dependent on Analytics variable eVar66 which has the bookingStatus of the user(booked/Non-booked). BP has created the audience with the below configuration. The expected content is not getting delivered on the page. What are the possible concerns? Mbox parameter is not firing with the required Target global mbox request. EVar66 is not enabled in Analytics configuration EVar values cannot be mapped into Target for defining audience criteria

Mbox parameter is not firing with the required Target global mbox request.

Which activity has a similar interface like MVT type activity? A/B Auto Target A/B Manual Recommendation Activity Automated Personalization

Automated Personalization

On the home page of a leading e-commerce site, BP want to validate the changes on a CTA & Hero Banner. There are two versions of CTA & Hero banner, with a total of 4 combinations. Along with this, BP wants to perform actions like changing the CTA position & adding an element prior to the carousel hero banner. Which of the activity type is recommended in this case A/B Activity MVT Activity AP Activity Recommendations Activity

A/B Activity

Mr. Singh is working as on optimization consultant for a leading financial institution where he had setup an A/B test to validate two different versions of the campaign landing page. He had shared the QA links internally with the marketing & UX leads to validate the user experience. One of the leads discovered an issue which doesn't comply with organization UX policy & hence wanted Mr. Singh to make appropriate corrections. Which of the access level would suffice to make the required corrections to the A/B test? (Select all that applies) Editor Approver Target Observer Marketing Cloud Admin

Editor Approver

Mr. Singh had created an MVT test on the home page of a leading cruise lines site, where the stakeholders requested to have three variations each across four different page elements. Post configuration, the preview URL links were distributed to UX & marketing lead for validation from user experience standpoint. Based on the guidelines from UX team, some of the combinations were impacting & recommended to be excluded from the activity. How can Mr. Singh achieve that? Under preview mode, select the combination to be excluded Under configure section, select the combination to be excluded Under combinations->experience, select the combination to be excluded Under compose mode, select the combination to be excluded

Under preview mode, select the combination to be excluded

You had configured a test with 4 variants for a retail website which is being running since last few days now. Stakeholders has accessed the results from the reporting console and found that Variant 3 has very low conversion. Stakeholders are worried as this is impacting the business metric. You had calculated that we need 30 K/variant to get the required confidence level. Currently all the variant have reached around 30% of the sample size and will take another few weeks to reach there. What recommendation as a Target consultant you will provide to your stakeholders. We can reduce the allocation to the the Variant 3. Variant 3 is the least performing so remove this experience from all users. Stop this test and create another test with learning from other variants into 1. Wait for the test to provide conclusive results

Wait for the test to provide conclusive results

You have configured an AP activity on a website with a "Random Forest" algorithm selected. This test has been running for quite a few days. Ecommerce improvement is the key success criteria for this. One of the experiences is receiving maximum traffic and has very low conversion, stakeholder asked you look into the issue and fix that. What you will do in that situation. You will change the configuration to stop traffic from that experience. You will let stakeholders know that Adobe Target is currently learning and let it run. You should exclude that experience from your experience so that it should not impact business metric. Automated personalization is being managed by Adobe consultant so let them know about this.

You will let stakeholders know that Adobe Target is currently learning and let it run.

``` elow are the results from an A/B/C/D test, which was run for the duration of 3 weeks. BP is asked to interpret the results. Which of the statements is true based on the below result. Exp Visitors RPV Revenue p-value Control 251 11.9 $3000 0.5 Exp B 244 16.3 $4000 0.47 Exp C 249 20.8 $5000 0.95 Exp D 253 23.7 $6000 0.04 ``` Control is better than Exp B with an RPV of 11.9 with 50% confidence Exp. B is better than Exp. D with an RPV of 16.3 with greater than 60% confidence Exp. D is better than control with an RPV of 23.7 with greater than 95% confidence Control is better than Exp. C with RPV of 20.8 with 0.95 p-value.

Exp. D is better than control with an RPV of 23.7 with greater than 95% confidence

The optimization team for a very hip and trend fashion blog want to improve the depth of the visit and page consumption of the site. Bounce rate of the website is very high and stakeholders are asking for improvements that will encourage visitors to read more articles, view additional slideshows, and watch more videos.As an optimization consultant, what will you suggest from below? Run an experiment targeting test with various set of segments. Run an AB test to find the best layout and then run Experience Targeting test Run a recommendation Test Run and MVT followed by a Recommendation test

Run a recommendation Test

Over estimation of traffic numbers

BP has setup an A/B test, where the user lands on the test page from a specific display campaign. The primary goal of the activity is to measure the increased lift in revenue. The volume of transactions is very high and hence the stakeholders want to know the number of orders which has value greater than $200 in the reporting interface. How should the BP setup the test to achieve this? Add Audience/Target condition to include visitors with order value > $200 Add reporting segment to include visitors with order value > $200 While reviewing reporting data, add custom segment to filter orders with value > $200 Order value > $200 decision cannot be taken at goal level and hence all the values will be present in the reports.

Add reporting segment to include visitors with order value > $200

An A/B/C test is running since two weeks and the stakeholders realized that experience C is unable to generate the required conversions and want to stop the experience for all the new & returning visitors. What should be done by the BP to address the issue? Reduce the Audience traffic to 50% and divert all the traffic to experience A & B. Reduce the Audience traffic to 75% and divert all the traffic to experience A & B. Deactivate the activity, reduce the traffic to 0% for experience C & then reactivate the activity. D) Deactivate the activity, create a new activity with only A & B experience & set that to live.

Deactivate the activity, create a new activity with only A & B experience & set that to live.

Mr. Singh has setup an A/B test for users who are not part of any other test within the same domain. In which part of the audience section can you create this segment? Custom Visitor Profile Audience Exclusion User Profile

Visitor Profile

A/B Auto Allocate activity uses enhanced version of the alogirithm used by Automated Personalization activity type. True False

False

Mr. Smith, an optimization consultant was trying to rearrange a header nav. element with other header nav. element, but was unable to find the default VEC capability to make the required changes to the test. Which of the following activity type is he using to build the test case? choose all that applies Automated Personalization Experience Targeting MVT A/B Auto Target Testing

Automated Personalization MVT

MVT

In which of the activity type, offer level targeting can be acheived? MVT Automated Personalization Search & Promote Recommendations

Automated Personalization

An A/B test is live since a week now. One of the stake holders mentioned that he is unable to see a particular experience and wants the BP to address the scenario. What is the quickest way for BP to verify if a particular experience is actually getting delivered? Verify the Activity Preview page to validate if that particular experience is getting delivered. Validate the experience delivery using a browser incognito mode. Validate the reports to check if that particular experience has any data against it Use Adobe Debugger VEC setting for quick validation of the experience.

Validate the reports to check if that particular experience has any data against it

There are two campaign banners set up on the home page of a leading e-Commerce site which also has Adobe Analytics configured on the site. Mr. Singh has setup an A/B test to validate the campaign performance where adding product to cart is the primary goal of the test. Key stakeholders want to understand users from which campaign contributed more to the conversions? How can you setup the goal to understand the expected behavior? Setup link tracking in Analytics & use the click event as goal Add campaign click as dependency in advanced settings while creating the activity In global configuration under setup, add campaign click as dependency for primary goal Add a secondary goal for campaign click and configure order of goal execution

Add campaign click as dependency in advanced settings while creating the activity

Optimization consultant has set up an A/B test with Target as a reporting source & circulated the activity details to get a sign-off from UX & marketing leads. One of the leads proposed to use A4T reporting source. What is the possible reasons to use A4T over Target reporting source? Stakeholders want to create more than one custom metrics in Target A4T reporting is much faster than Target Default A4T is flexible to add more data sources for robust analysis. Analysis needs to include historical data for comparison

Stakeholders want to create more than one custom metrics in Target

Which type of test you should use for the below?Product manager, UX manager and Marketing manager was brainstorming for improving the performance for one of the key landing pages. Product manger has researched and found that competitors use an image which suggest users which product they are about to purchase as compared to company's own brand. At the same time UX manager says that images doesn't help in decision making but Text on CTA and colour of CTA has the highest impact on user's decision. Marketing manager says we are using different copy on our advertisements on TV and other offline channels and this is why we are not getting orders. MVT A/B Automated Personlization Recommendations

MVT

Your client is very advance in testing cycle and have been running multiple tests/section which are receiving high traffic. He submitted a test request that to be configured for a section where 3 more tests are currently running. He wanted to show the newly configured test for only customers who have not been exposed to any other test. What you should do to show this new test for anybody who hasn't seen the previous test? You will setup an experience targeting activity You will create an audience with mBox call is not triggered You will create an audience with "not in other test" If it will be an AB test then it will automatically will be handled by the Adobe Target.

You will create an audience with "not in other test"

Optimization consultant wants to setup to deliver 4 different retail promo banners to individual groups of people that came through a specific email campaigns. Which of the activity is best suited in this scenario? A/B test Experience targeting Automated Personalization Recommendations

Experience targeting

``` 1. Which activity type should be used to deliver a temporary promotion banner to all site visitors that clicked through an email campaign? A. A/B Test B. Experience Targeting C. Automated Personalization D. Recommendations ```

B. Experience Targeting

A Target Business Practitioner (BP) needs to create an activity. The Target BP chooses Adobe Target as a reporting source instead of Adobe Analytics. What is one reason that the Target BP chooses Adobe Target? A. The BP wants to create multiple custom metrics using the VEC. B. Adobe Analytics causes a 24-hour delay in results. C. An unusually high number of visitors must be tracked in the activity. D. Analysis needs to include trends over time.

A. The BP wants to create multiple custom metrics using the VEC.

A Target Business Practitioner needs to make sure an activity is free of any collisions with any previously made activities. How should the Target Business Practitioner confirm that this is true? A. Launch the activity and rely on Target to block any colliding activities. B. Review the Status, Priority, and Audience of every activity on the same URL to make sure there is no undesired overlap. C. Confirm that the activity has a unique name that is NOT duplicated or closely resembled elsewhere. D. Set the priority level to "high" on the new activity.

B. Review the Status, Priority, and Audience of every activity on the same URL to make sure there is no undesired overlap.

A stakeholder asks the Target Business Practitioner (BP) to create an activity that tests the addition of a pop-up message after a visitor adds an item to their cart. Which of the following would allow the BP to correctly target this audience? A. Add a custom mbox parameter for users with cart additions and target this audience to users with that parameter B. Create an Analytics segment published to the Marketing Cloud for visitors with cart additions and target this audience to that segment C. Select Visitor Profile > "Cart Additions" in audience creation D. Target the activity to the cart URL

A. Add a custom mbox parameter for users with cart additions and target this audience to users with that parameter

A particular search engine marketing campaign is driving users to a landing page URL with tracking codes appended. The tracking codes must be present in the URL for the search team's tracking, which occurs in a separate tool. A Target Business Practitioner (BP) created an activity experience that redirects incoming users from that default landing page to a new page. After the activity has been running for 1 week, the search team notes that the tracking codes are NOT included in the URL when users arrive at the new page. What should the BP do to fix this? A. Create a profile script that will store the tracking codes for each user B. Change the activity to exclude users without a tracking code C. Include the new page in the activity URL targeting D. Check the "Include Current Query Parameters" checkbox on the experience

D. Check the "Include Current Query Parameters" checkbox on the experience

6. A Target Business Practitioner (BP) reviews the ongoing results of an activity with four experiences. The BP notes that there are about 400-500 visitors for each experience, but the clickthrough metric shows 0 for two of the experiences. Which step should the BP take first? A. Verify the metric is set up properly in the activity B. Remove any activity experiences with data discrepancies C. Pause and restart the activity D. Stop the test and report on all current findings

A. Verify the metric is set up properly in the activity

The conversion rate in the results data for an A/B Test activity is much lower than expected. A Target Business Practitioner (BP) must validate the ongoing activity and ensure proper performance. What should the BP do first? A. Monitor data of all live activities B. End the activity immediately C. Re-examine the activity setup and metrics D. Reduce traffic exposure to 50%

C. Re-examine the activity setup and metrics

An A/B Test activity with three experiences reaches its sample size. Stakeholders want to know the activity's results. Consider the following details: • 100% of mobile site traffic allowed into activity • 120,000 visitors per experience • 3,000-4,000 conversions per experience Preliminary results are shown. Experience Lift Confidence Experience A [Control] Experience B -2% 76% Experience C 17% 99% Which action should a BP take when analyzing the current results of this activity? A. Eliminate Experience B as an under-performing challenger and continue to run the test. B. Declare Experience C the winner with positive lift and high significance. C. Allow the activity to continue running to gather more data. D. Expose winning experience to all desktop users due to positive trends.

B. Declare Experience C the winner with positive lift and high significance.

A Target Business Practitioner (BP) is reviewing the results from an A/B activity with two experiences that was launched yesterday. The sample size was 500,000 visitors per variation. The test was designed to prove up to a 1.5% difference in orders at 95% confidence. Experience Visitors Orders Order Rate Lift Confidence Control 23,075 846 3.666% Experience A 23,085 765 3.331% -9.614% 96% Which recommendation should the BP make? A. Continue running the activity to set sample size because the results are inconclusive. B. Experience B is confidently worse than Control. Use Control moving forward. C. Experience B is confidently better than Control. Use Experience B moving forward. D. Continue to monitor change in lift until the lift observed is greater than the minimum detectable lift.

A. Continue running the activity to set sample size because the results are inconclusive.

A Target Business Practitioner (BP) is reviewing test results after an activity has ended. The test showed a 5% increase in revenue per visitor. The sample size was reached, which was set to prove up to a 2% difference in order rate. The following effects were observed: • 5% increase in revenue per visitor (RPV) • 0.5% increase in orders per visitor • 3% increase in average order value • 3.5% increase in average order size • 0.5% decrease in page views • 0.8% increase in product views A stakeholder asks the BP to annualize the impact on revenue projected through the next year to present as program ROI. How should the BP proceed? A. Inform the stakeholder that the result would NOT be trustworthy because the test was sized to prove differences in order rate. B. Project annual revenue attributed to the audience in the test over the next year and multiply by the lift observed. C. Project annual revenue attributed to the audience in the test over the next year and multiply by the lift observed. D. Inform the stakeholder that the test would need to run four times as long to prove differences in order rate.

A. Inform the stakeholder that the result would NOT be trustworthy because the test was sized to prove differences in order rate.

What is Fractional vs Full-factorial MVT

Full-Factorial = testing all combinations. e.g. With 7 elements (banner, headline, call-to-action, etc.) having 2 combinations each you have 2^7 = 128 variations to be tested. Issues : -requires more time -requires more traffic -possibly at the expense of segmentation -if tests run too long, the result may be outdated when it finally arrives Fractional-testing = testing a subset to indicate directional change. “regardless of methodology, conversions and revenues are less dependent on statistics, and more creativity, innovation, and the ability to listen and react to your customer."

where do you define actity url for a new test settings pop-up box in configuration section under browse after viewing

settings pop-up box in configuration section under browse after viewing

Pausing and reactivating a A/B test will keep the existing visitors in the same experience

true

AP activitves need less traffic and data to reach significance

false

MVT cannot be configured through form-based workflow

true

ET work in the same manner as A/B test or Auto-allocate

false

VEC can be used to create activities for web, mobile web and mobile app enviroments

false

form-based workflow cannot be used for web-based activitites

false

VEC can be used to create activities for web, mobile web and mobile app environments

false

form-based workflow cannot be used for web-based activities

false

only VEC activities can be delivered for each page

false

How is lift calculated

calculated the increase and decrease of the experience against the control