E10 - Testing Campaigns and Assets Flashcards
What % of all strategic marketers use A/B testing to learn about customer behavior?
75%
what is the typical email survey response rate?
15%
What % of organizations believe email marketing ROI will increase?
64%
What is the first reason to test?
To identify customer behaviors.
What’s a good analogy for testing?
Photography - keep taking photos to get the best image possible.
What’s the second reason for testing?
Reveal actual vs. stated customer actions
True or false: Customer behaviors may differ from their stated preferences and opinions
True
What are 3 reasons surveys can be interpreted incorrectly?
1 - don’t reach enough of the intended targets; 2 - include questions that can be influential towards a desired outcome; 3 - include answers influenced by group behavior
What is the third reason to test?
Increase ROI and conversions
Testing emails this week will yield the best choice for what timeframe?
This week
What time based factors can affect test results?
Current events and approaching holidays.
What can be tested?
Any element in a campaign canvas.
Give 9 examples of things that can be tested
(1) CTA (2) Subject line (3) Copy (4) Conversion path (5) Button design (6) Form location (7) Offer (8) total redesign (9) images / icons
Name the 3 steps to test
(1) develop a plan (2) execute (3) measure
3 steps to develop a plan
(1) Baseline (2) Hypothesis (3) Methodology
3 steps to execute
(1) Assets (2) Campaign (3) Edits
3 steps to measure
(1) Analyze (2) Optimize (3) Repeat
What is the lesson from the test where a complex page resulted in a 38% increase in trial sign-ups?
Pretty does not always equal winner.
Where can you find use cases of A/B comparisons?
WhichTestWon.com
//delete after class//
User912 EPITECA Eloqua1 // next slide, 22
With every test you gain insights into customer’s behavior _____
for that period in time.
1 reason that comes back as a hurdle to slow testing is what?
Timing
Some test winners might result in more opens but less ___ or ___
clickthroughs or revenue.
If you’re wondering what you should be testing, go on Topliners and search for…
Email features to test
In the form test, which test won?
The version with more checkboxes instead of more text-based fields.
In a test with and without testimonials, what was the result?
The one with testimonials led to a 34% increase in sales.
What kind of campaign do you create to do an A/B email test?
Simple email campaign
What option do you choose for Type of campaign to do an A/B email test?
A/B Test
Where do you configure the parameters o the A/B test?
In step 4 of the simple campaign wizard.
What’s the default % to test on?
50%.
Best practice is to test on at least __% of your list
10%.
What parameters can you configure in an email A/B test?
Percentage to test, winning metric, testing time, whether to automatically send the winning version after test.
What winning metrics can you choose for an email A/B test?
Total opens, unique opens, total clicks, unique clicks, total conversions
Best practice testing time is…
at least 24-48 hours.
3 good in-depth reports include…
Email Analysis Overview Report, Form Submission Overview Report, and Landing Page Activity for Date Range Report
When looking at your metrics, don’t forget to look at the….
big picture.
Form Submission Overview - what metrics does this report measure?
Submissions, Conversions Views and Abandonmnet
What things would you test by looking at the landing page overview report?
Form position, banner, layout
//slide 37//
//slide 37//
What should you use Eloqua as?
A reusable testing environment.
What should you test?
As many features as possible – and measure results.
Can you use an A/B test only with a simple campaign or can you send the winner back into a multi-step campaign?
You can send the winner back into a multi-step campaign, but you have to do it manually.
What campaign workflow steps can you test?
Segments, filters, assets, decision steps
True or false? you should copy a Campaign Canvas before testing a workflow.
True.
Why copy a campaign for testing?
Because you can run a separate report on your testing campaign.
Where can you decide not to sync a campaign with CRM?
Campaign settings.
What are the 6 elements on a QA campaign testing checklist?
Copy campaign, shorten wait steps and evaluation periods, create segment of QA testers, run test, make edits, save and activate.
Who should you send your tests to in a QA segment? - step 1
- Myself. It’s fine, but not optimal because only one test case.
Who should you send your tests to in a QA segment? - step 2
- Free email accounts for testing.
Who should you send your tests to in a QA segment? - step 3
- Internal global contacts
What are some workflow issues you might find in a QA campaign test?
Incorrect links, email fails to send, repeat emails, emails sent at wrong time
Also not to be forgotten when thinking about timing in QA testing.
you have the ability to control timing on your email scheduling.
If you’re going to do repeat testing, what should you remember to do?
Allow email to be sent to recipients more than once, and allow contacts to enter the campaign more than once.
What are 9 triggers that can be used for dynamic content?
1 - Purchase, 2 - renewal, 3 - webinar registration, 4 - location, 5 - birth date, 6 - survey submit, 7 - title, 8 - sequence, 9 - custom
Pro’s of dynamic content
Increased loyalty, higher conversion rates
Cons and/or needs of dynamic content
Limited reporting, requires clean data, needs analysis
What can you control?
Campaigns, campaign workflow, assets
What don’t you control?
Email programs, web browsers, mobile browsers.
How can you test assets in different browsers?
ARE font differences acceptable?
What should you look for in mobile?
Fonts, Content order
What should you look for when testing email services?
broken image links
What should you test in form submissions?
Does the button lead to the correct form? When you fill it out, does the information go to the right place?
//may have missed some of the testing before slide 61
//slide 61 - look out to test
To do a test send, what should you do?
From Email Easy Editor, choose Gear menu, and select Test Content and Deliverability.
Should you ever prepopulate forms with a default value?
No.
Key Point #1 for testing a campaign
Create a contacts list for testing.
Key Point #2 for testing a campaign
Test every asset
Key Point #3 for testing a campaign
Test dynamic content
What do embarrassing mistakes cause?
Brand dilution.
What are some reasons to edit an active campaign?
Contacts, Current Events, Team Members
REasons for adding and moving contacts
Stuck in step, requests to add individuals (from sales or customers)
Why add segments instead of adding copy campaigns?
Unified reporting, less work, more organized campaign area.
Keep in mind when deactivating campaigns…
Overlapping campaigns, pausing during deactivation, resend option checkbox
Eloqua’s default is to send emails how many times?
Only once.
When a campaign is reactivated, when should campaign steps be?
In the future.
4 steps to deactivate capaign
Open, deactivate, make changes, save and reactivate
If you add additional segments to a campaign, best practice is to…
feed it into a wait step before the email, not directly into an email.
Why feed additional segments into a wait step before the email?
In case there is a technological hiccup preventing them from getting the email.
What do you do with soft bouncebacks?
Analyze them!
How can you take care of soft bouncebacks at the campaign level?
Use the email Routing tab to automatically route contacts excluded from email send to a wait step. Then you can add to a shared list, analyze, re-send, etc.
Reactivating - 4 steps.
(1) Check dates on campaign canvas AND emails (2) Check contacts (3) Add segments to wait steps (4) Check the resend box.
How many emails were sent out by marketers in 2013?
over 838 billion emails
What can you search for in Topliners for more info?
Testing
What QuickCasts can you watch for help with adding contacts to campaign canvas and moving contacts in the campaign canvas?
Adding a Contact to the Campaign Canvas and Moving Contacts in the Campaign Canvas within Eloqua University
(further reference)
http://docs.oracle.com/cloud/latest/marketingcs_gs/OMCAA/index.html#Help/SimpleEmailCampaigns/Tasks/CreatingSimpleEmailCampaigns.htm
What should you look for when testing mobile email services?
Broken text links
