Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

SMM > Using Experiments > Flashcards

Using Experiments Flashcards

1
Q

What is the point of using experiments?

A

One control group and one group exposed to your ads helps you see if your ads are driving the results you want, and verify that your results are indeed from your ad and not chance, organic results that would have happened anyway.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Randomization?

A

Process of assigning people to different groups by chance, or randomly. That way you create groups that are as similar as possible.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

RCT?

A

Randomized Controlled Trials

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is a Randomized Controlled Trial?

A

A study design that randomly assigns participants to a treatment group or to a control group.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

ITT?

A

Intention To Treat

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is Intention to Treat?

A

Keeps all people in the same group they were originally assigned to, regardless of what they see or do. This minimizes the effect of potential bias.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Keeping people in the same group because of Intention to Treat has what effect?

A

Keeps bias out, though it may make your results seem slightly less effective than if only people who took the treatment were kept in the Treatment group (instead of allowing to let go of people who choose not to use the Treatment)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What 2 components make up a Good Experiment?

A
  1. Uses RCT: Randomized Controlled Trial (randomly assigns participants to a Treatment Group or to a Control Group).
  2. Uses ITT (Intention to Treat) for the test group.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Experiments let you prove what?

A

A causal relationship between an ad and their results

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

How can you set up these experiments and evaluate their results?

A

Either through FB official tools, or through third party options.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What are the FB official tools that help you measure RCT’s? (Randomized Controlled Trials)? (2)

A
  1. Holdout Tests ACTION (measure “conversion lift”)

2. Brand Survey Tests ATTITUDE (measure “brand recall lift”)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What do Holdout Tests measure?

A

ACTIONS: Measure conversion lift or conversions, measures actions people took: could be purchases, add to cart, etc.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What do Brand Survey Tests measure?

A

ATTITUDES: Brand Recall Lift, measures changes in people’s attitudes about brands: changes in brand recall and other brand-related measures.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

3 Ways to Pass Data about Test Group for Holdout Actions:

A
  1. Actions on Website (via Facebook Pixel)
  2. Actions in an App (via Facebook SDK)
  3. Actions Offline (via Conversions API)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What does a Conversions API do generally?

A

Connects data from a system that monitors offline sales to Facebook’s system.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Instead of measuring conversions, how is Brand Recall Lift tested for Brand Survey Tests?

A

People are shown a small survey that asks them if they recall seeing an ad for a brand.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What stage of the Customer Journey do you run a Brand Survey Test?

A

Awareness & Consideration

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What stage of the Customer Journey do you run a Holdout Test?

A

Conversions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Why would you want to run these HoldOut or Brand Survey tests? (2)

A
  1. to prove the effectiveness of your campaigns

2. to optimize your advertising spend (more of what is working, etc.)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

A/B Testing: What are the variables that you can test in an A/B test?

A
  1. Creative (image/video), copy, etc.
  2. Audience
  3. Placement
  4. Delivery Optimization (WOW you can compare your manual optimization vs CBO/automated optimization)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Best Practices for A/B tests:

A
  1. Use enough budget
  2. Have at least 80% Power (ideally 90% power if really important for future decisions)
  3. Run long enough, ideally at least 2 weeks.
  4. Use the Winner in the next test. (aka keep testing as you build new campaigns)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

The Two Main Parts to your Results:

A
  1. The winner (aka which ad had the lowest cost per result)

2. The Confidence Level

23
Q

Two ways to increase Confidence Level of your A/B tests:

A
  1. Increase budget

2. Increase duration that test runs for so it can achieve more results.

24
Q

After you get the results from an A/B test, what is next?

A
  1. FB gives you the option to shift all of your budget from the loser over to the winner (and stop running the loser ad)
  2. KEEP TESTING! try another new iteration with only one changed variable.
25
Q

What are the two methods of Cross-Platform Measurement?

A
  1. Media Mix Modeling (sometimes called Marketing Mix Modeling)
  2. Attribution Modeling
26
Q

What does the Marketing Mix Model do?

A

It takes the data (results from marketing campaigns on various platforms [sales and views gained from money spent]), cleans it up, and creates an ROI from each channel to the percent contribution of each channel’s marketing contribution on my final result.

27
Q

What is a Marketing Mix Model (official definition):

A

A mathematical relationship showing the statistical relationship between variables and a KPI. (basically between the money you spent and the results you got).

Uses linear regression.

28
Q

What is Attribution?

A

The process of assigning credit to touchpoints along the customer’s conversion path.

  • By understanding which of your ads deserves credit for inspiring your customer to take an action, you can better measure the effectiveness of your ads and make decisions for future planning and optimization.
  • This helps you understand if you have invested in the right channels, by understanding which channels worked best.
29
Q

Attribution helps you to:

A
  • Understand the ROI for different channels

- Optimize budgets for the future

30
Q

ROI is different from weight because?

A

Bceause it takes into account how much you paid for the ads.

31
Q

Can Attribution be used to evaluate Offline Marketing?

A

It’s mostly used for Online Marketing, but yes it can be used for Offline channels too.

32
Q

Is Attribution only for paid advertising?

A

Nope. You can use Attribution modeling to evaluate the effectiveness of paid as well as non-paid social media posts.

33
Q

So far this sounds like Media Mix Modeling (Marketing Mix Modeling), So how is ATTRIBUTION DIFFERENT?

A
  • With Attribution, you choose a MODEL. You will think about the customer journey and make some assumptions about which part of your marketing you think had the biggest effect on inspiring the sale.
  • The MODEL you decide on is the Attribution Model.
34
Q

What is the Attribution Model?

A

The model you decide on for being the main marketing points along your customer journey that you believe most influenced the sale.

35
Q

Elements of Attribution:

A
  1. Input Data (information about the ads they saw and whether they clicked on them)
  2. Attribution Model (Model that is used to assign value to the touch points). The output is a value for each touchpoint that reflects the impact it had on the conversion.
36
Q

Accuracy of Attribution Depends on:

A
  1. Quality of Data Input

2. Accuracy of the Model (how much it reflects reality)

37
Q

Rules Based Models

A

Used by most Attribution Analysis.

38
Q

What are Rules Based Models?

A

A rule is selected that determines how conversions should be allocated to add touch points.

39
Q

Name the commonly used Attribution Models:

A

Last Click Attribution Model.
Last Touch Attribution Model
Even TouchPoints Attribution Model

40
Q

What is the Last Click Attribution Model?

A

Gives 100% credit to the last click a person made before the conversion took place. (even if a person saw and clicked on many ads before that).

41
Q

Problem with The Last Click Attribution Model?

A

It doesn’t reflect reality. Most marketers agree that many touchpoints along the way probably contributed to the conversion.

42
Q

What is the Last Touch Attribution Model?

A

Similar to Last Click Attribution Model, it is a one-touch-point attribution, except this doesn’t even happen with a click.

Last Touch Point Attribution Model gives 100% credit to the last ad a person interacted with before they took the conversion (whether they clicked it or just saw it).

43
Q

Even Credit Attribution Model

A

Even credit to all touchpoints. (impressions and clicks)

44
Q

Time Decay Attribution Model

A

More credit to more recent clicks/impressions (the ones closer to the conversion).

45
Q

Positional Attribution Model

A

First and last touchpoints (clicks or impressions) given more credit. Middle touchpoints are given all equal remaining credit.

46
Q

First Click Attribution Model

A

100% credit given to first click.

47
Q

First Touch Attribution Model

A

100% credit to first click (or first impression if no click)

48
Q

How do you choose which Rule Based Attribution Model to apply?

A

You’d really have to know your customers. This can be hard to decide, so sometimes it’s best to rely on a Statistical Model instead of a Rule-Based model (that you decide on).

49
Q

Statistical Models have the benefit of doing what?

A

Letting the data speak. You don’t decide on rules ahead of time.

50
Q

Statistical Models:

A

Assigns fractional credit for a conversion to touchpoints based on their incremental estimated impact.

51
Q

Measurement Challenges when measuring across different Channels (2):

A
  1. Measure across platforms and devices

2. Connecting touchpoints to one person.

52
Q

Facebook connects touchpoints to one person by combining data sets from:

A 
unique Facebook ID
Pixel data
SDK
Conversion API
a persistent login
53
Q

Why can’t you just use cookies?

A

They don’t work well on mobile devices

They don’t work in apps

54
Q

Difference between Rule Based Attribution Models and Statistical Models:

A

Rule Based is based on a rule you choose (ie: Last Click: you decide 100% credit goes to whatever touchpoint the user clicked on to convert), vs:
Statistical Models are data driven, using learning from actual data observations. More accurate.

SMM (16 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions