SS17 - Evaluating Portfolio Performance

Question 1

Why is performance evaluation important from the fund sponsor’s perspective?

Answer 1

Performance evaluation improves the effectiveness of the IPS by acting as a feedback and control mechanism.

1. Shows where the policy and allocation is effective and where it isn’t.
2. Directs management to areas of value added and lost.
3. Quantifies the results of active management and other policy decisions.
4. Indicates where other, additional strategies can be successfully applied.
5. Provides feedback on the consistent application of the policies set forth in the IPS.

Question 2

Why is performance evaluation important from the fund manager’s perspective?

Answer 2

As is the case with the fund sponsor, performance evaluation serves as a feedback and control mechanism.

Allows manager to:

• compare investment returns to a benchmark
• investigate the effectiveness of each component of their investment process

Question 3

What are the three components of performance evaluation?

Answer 3

1. Measurement
   
   • calculate rates of return based on changes in value over some time period
2. Attribution
   
   • determine the sources of account performance
3. Appraisal
   
   • draw conclusions regarding whether performance was affected by investment decisions, the market, or chance.

Question 4

What is time-weighted rate of return (TWRR)?

Answer 4

TWRR calculates the compounded rate of growth over a stated evaluation period of one unit of money initially invested in the account.

TWRR reflects what would have happened to the beginning value of the account if no external cash flows occurred.

1. calculate subperiod returns covering each period that has an external cash flow
2. compound subperiod results together

Question 5

What is the money-weighted rate of return (MWRR)?

Answer 5

MWRR is an internal rate of return on all funds invested during the evaulation period.

It is the rate that solves the equation:

MV_1 = MV_0(1+R)^m + SUM_cashflows( CF_i * (1 + R)^L(i) )

where L(i) is the number of days the cash inflow is in the portfolio (or number of days an outflow is absent from the portfolio)

Question 6

What are the differences between TWRR and MWRR?

Answer 6

• MWRR is an *average *growth rate for all funds in the acount
• MWRR is affected by the timing of external cash flows
  
  • example: consider a large cash flow that is exposed to a large increase or decrease in asset values
• TWRR is only a linking of subperiod returns. Not affected by external cash flows

Question 7

Which rate of return method should be used for manager evaluation?

Answer 7

**TWRR **should generally be used for manager evaluation and GIPS because it is not affected by client decisions to add or subtract funds.

Question 8

If a manager controls the timing of cash flows, which return method should be used?

Answer 8

If the manager controls timing of cash flows, we’d use MWRR for performance reporting and GIPS. We want to reward or penalize the manager for their inflow/outflow timing decisions.

Question 9

What are the differences in **data requirements **for calculating TWRR vs MWRR?

Answer 9

• MWRR only requires beginning and end of period market value
• TWRR can be more data intensive because you need the market value of the account on every date on which there are cash flows

Question 10

What are the common **data quality issues **faced when calculating returns?

Answer 10

• estimates on values of assets must be used when an account contains illiquid/infrequently traded securities
• estimates on values of thinly traded fixed income securities may need to be obtained from a matrix of similar securities
• for securities that are very illiquid, carrying value might be at the price of the last trade which might not accurately reflect current value
• accounting valuations should include:
  
  • accrued interest
  • dividends

Question 11

What are the three components that portfolio returns can be decomposed into?

Answer 11

1. market (M)
2. style (S)
   
   • S = B - M
   • difference between manager’s style index and the market return
   • example: B = SP Value Index; M = SP500
3. active management (A)
   
   • A = P - B
   • difference between the managers overall portfolio return and the manager’s style benchmark return

P = M + S + A

Question 12

What are the properties of a valid benchmark?

Answer 12

S-A-M-U-R-A-I

1. specified in advance
2. appropriate
   
   • consistent with the managers investment approach and style
3. measurable
4. unambiguous
5. reflective of managers current investment opinions
   
   • means the manager has knowledge and expertise on securities in the benchmark
6. accountable
   
   • manager should agreee that the difference between his portfolio and the index is due to active management (his actions)
7. investable

Question 13

What are the seven primary types of benchmarks?

Answer 13

1. Absolute
2. Manager universes
3. Broad market indices
4. Style indices
5. Factor-model-based
6. Returns-based
7. Custom security-based

Question 14

What are the advantages and disadvantages of an absolute benchmark?

Answer 14

Advantages

• simple, straightforward

Disadvantages

• not investable (since it is just a target return)

Question 15

What are the advantages and disadvantages of using manager universes as a benchmark?

Answer 15

Advantages

• measurable

Disadvantages

• subject to survivorship bias (underperforming managers go out of business and are no longer in the index)
• if a fund sponsor uses a manager’s universe, the sponsor has to rely on the fact that it was accurately compiled
• cannot be identified in advance (so it is not investable)

Question 16

What are the advantages and disadvantages of broad market indices?

Answer 16

Advantages

• well-recognized, easy to understand
• unambiguous, investable (generally), measurable, specified in advance
• appropriate if it reflects the approach of the manager

Disadvantages

• managers style may deviative from the style of the index. example: not appropriate to use SP500 for a small cap US growth manager

Question 17

What are the advantages and disadvantages of style indices?

Answer 17

Advantages

• widely available, understood, and accepted
• appropriate if it reflects the manager’s style and is investable

Disadvantages

• some may contain weights of individual securities that are larger than is considered prudent (too concentrated)
• different defintions of a “style” can produce quite different benchmark returns

Question 18

What are the advantages and disadvantages of factor-model-based benchmarks?

Answer 18

Advantages

• useful in performance evaluation
• provides managers and sponsors with insight into the managers style by capturing factor exposures that affect account performance

Disadvantages

• focusing on factors is not intuitive to all managers or sponsors
• data and modeling not always available or expensive
• can be ambiguous; different factor models can produce different output

Question 19

What are the advantages and disadvantages of a returns-based benchmark?

Answer 19

Advantages

• generally easy to use and intuitive
• meets the criteria of valid benchmark (SAMURAI)
• useful only when the only information available is account returns

Disadvantages

• the style indices used may not reflect what the manager actually owns
• need a sufficient number of monthly returns
• will not work with managers who change style

Question 20

What are the advantages and disadvantages of a custom security-based benchmark?

Answer 20

Advantages

• meets all requirements of a valid benchmark
• allows continual monitoring of investment process
• allows fund sponsors to effectively allocate risk across investment management teams

Disadvantages

• can be expensive to construct and maintain
• lack of transparency by the manager may make it impossible to construct the benchmark

Question 21

How does a **factor-model-based benchmark **work?

Answer 21

Factor models *relate *a specified set of factor exposures to returns on an account

Well known 1-factor example is CAPM (one factor is market return)

Generalized:

R_p = a_p + b₁ * F₁ + b₂ * F₂ + b_k * F_k + epsilon

a_p = expected return if all factor values were 0

F_k = factors that have a systematic effect on portfolio performance

b_i = sensitivity of the portfolio returns to returns generated from factor i

epsilon = error term. return not explained by the model

Question 22

How does a **returns-based benchmark **work?

Answer 22

Constucted using:

• manager’s returns over specificied periods
• corresponding returns on several style indices for the same period

These return series are submitted to an allocation algorithm that solves for the combination of investment style indices that most closely track the account’s returns.

Question 23

How can a custom security-based benchmark be constructed?

Answer 23

1. Identify the important elements of the manager’s investment process
2. select securities that are consistent with that process
3. weight he securities (including cash) to reflect the manager’s process
4. review and adjust as needed to replicate the manager’s process and results
5. rebalance the custom benchmark on a predetermined schedule

Question 24

What are some drawbacks of using manager universes as benchmarks?

Answer 24

It is measurable but doesn’t have serveral other properties of a valid benchmark:

• known in advance (median manager is only known after)
• also means that it is not unambiguous
• it is impossible to verify the benchmark’s appropriatness due to the ambiguity of median manager

Fund sponsor’s need to rely on benchmark compiler’s representations

Subject to survivorship bias as underperforming managers are dropped

Question 25

What are the main issues that should be considered in assessing benchmark quality?

Answer 25

1. systematic bias
2. tracking error
3. risk characteristics
4. coverage
5. turnover
6. positive active positions

Question 26

How can you test an index for systematic bias?

Answer 26

Beta Method

Calculate the *historical beta *of the account relative to the benchmark (regression of portfolio returns on benchmark returns).

The beta should be close to 1. If it is not, that is an indication the benchmark may be responding to different factors and thus has a different set of risk factor exposures

Correlation Method

Returns due to the manager’s active decision making (A) should be *uncorrelated *with the manager’s investment style (S).

Question 27

How can you use **tracking error **to assess the quality of an index?

Answer 27

If the appropriate benchmark has been selected:

• tracking error will be **smaller **than that of the difference between the portfolio and a market index.
• this implies that the benchmark is capturing important elements of the investment manager’s style. This is good, as it suggests the benchmark was the right one.

Question 28

How can you assess the quality of a benchmark using risk characteristics?

Answer 28

The account’s exposure to systematic sources of risk should be very similar to those of the benchmark.

During individual periods of time there may be deviations, but the longer term average should be the same as the benchmark. If it is not, that indicates a systematic bias.

Question 29

What is the coverage ratio and how is it a measure of benchmark quality?

Answer 29

The coverage ratio is the market value of securities in **both the portfolio and the benchmark **as a percentage of the total market value of the portfolio.

A higher coverage ratio means the manager is replicating the benchmark more closely.

Question 30

What is turnover and how is important in indexing?

Answer 30

Turnover *of an index *is the proportion of the benchmark’s total market value that is bought or sold during periodic rebalancing.

A passively managed fund should use a benchmark with low turnover.

Question 31

What are positive active positions and how can they be used as a measure of benchmark quality?

Answer 31

Example of a postive position: GOOG = 2% in benchmark and 4% in portfolio means we have a 2% active position.

If the manager doesn’t have a view on a benchmark name, he will have a negative position in that benchmark name.

If the number/proportion of negative positions is large, it may mean that the benchmark does not reflect the manager’s investment process.

Question 32

What are the major reasons that assigning a benchmark to hedge funds is difficult?

Answer 32

In general, difficulties aries due to lack of transparency and diversity of hedge funds.

1. Return can be hard to calculate (because net market value/capital is often close to zero)
   
   • one way to address this is to evaluate performance in terms of value-added return:
     R_v= R_p - R_b
2. Some hedge funds target an absolute return and say comparison with arbitrary benchmarks is irrelevant
3. Some funds have no definable style so it would be hard to pick a benchmark for comparison
4. Because of the difficulties above, some people have used Sharpe ratio to evaluate hedge funds instead of benchmarks. However, this is questionable because many hedge funds show skewed returns.

Question 33

What is the difference between **macro **and **micro **performance attribution?

Answer 33

The two basic forms of performance attribution are:

• macro - done at the fund sponsor level
• **micro **- done at the investment manager level

Question 34

What are the three main inputs to macro performance attribution?

Answer 34

1. Policy allocations
   
   • it is up to the sponsor to determine asset categories and weights
   • it is up to the sponsor to allocate total funds across managers
2. Benchmark portfolio returns
   
   • a fund sponsor may use broad market indices for asset categories and narrower focused indices for individual managers
3. Fund returns, valuations, and external cash flows
   
   • When using percentage terms, calculate at the manager level. This helps make decisions regarding manager selection
   • When using monetary terms, you need to take account valuation and external cash flow data into account

Question 35

How is macro performance attribution done at a high level?

Answer 35

Start with beginning and ending market value.

Analyze six levels of attribution that can explain the change in market value.

1. Net contributions
2. Risk-free asset
3. Asset categories
4. Benchmarks
5. Investment managers
6. Allocation effects

Question 36

How should net contributions be interpreted in a macro performance attribution analysis?

Answer 36

Net contributions are net sum of external cash flows made by the client.

They increase or decrease ending market value but are not investment value added or lost.

Question 37

How should the risk-free investment be incorportated into a macro perfomance analysis?

Answer 37

The risk free asset simulates what the fund’s ending market value would have been if the following had been invested at the risk-free rate.

• fund’s beginning market value
• any external cash flows

Question 38

How are asset categories taken into account in a macro performance attribution analysis?

Answer 38

Sponsors may realize that the risk-free investment is too conservative. Thus, in addition to risk-free, they may use an investment in a passively replicated strategic asset allocation with index funds.

The incremental return above the risk free rate is the sum of (R_i - R_F) * w_i for each asset category i.

Question 39

What is the **benchmark level **of macro performance attribution analysis?

Answer 39

At the benchmark level, the sponsor can select and assign a manager a benchmark different from the policy benchmark.

Example: SP500 as the strategic/policy benchmark, but SP Value Index as a manager-specific benchmark.

Question 40

What is the investment manager or active management level of macro performance attribution?

Answer 40

This level simulates the results of investing the fund’s beginning value (and external cash flows) and earning the return actually produced by managers.

Note: this assumes the sponsor actually allocated funds in accord with the *policy allocations. *This might not actually be perfectly true.

Question 41

What is the allocation effects level of macro performance attribution?

Answer 41

The allocation effects level is just a plug so the attribution sums to the portfolio ending value.

Question 42

What is micro performance attribution?

Answer 42

Micro performance attribution analyzes individual portfolios relative to designated benchmarks. The value added return can be broken into three parts:

1. pure sector allocation
2. allocation/selection interaction
3. within-sector selection

Question 43

What is the **pure sector selection **term of micro performance attribution?

Answer 43

Pure sector selection is the sum over all sectors j:

(P_w,j - B_w,j) * (B_r,j - B_r)

• w* = weight
• r* = return
• j* = sector

Assumes:

• manager holds the same sectors as benchmark
• the same securities within each sector are held in the same proportion as the benchmark

Question 44

What is the **allocation/selection interaction **term of micro performance attribution?

Answer 44

The allocation/selection interaction is the sum over all sectors j:

(P_w,j - B_w,j) - (P_r,j - B_r,j)

This component will often be small/close to zero if the benchmark is appropriate.

Joint effect of assigning weights to both sectors and individual securities. A decision to increase the allocation of a security will also increase the weight of that security’s sector.

Question 45

What is the **within-sector selection **term of micro performance attribution?

Answer 45

The within sector selection term is sum over all sectors j:

B_w,j * (P_r,j - R_j)

Assumes:

• manager weights each sector in the portfolio in the same proportion as in the overall benchmark

Question 46

How can you construct a **multifactor model **for micro performance attribution?

Answer 46

1. identify fundamental factors that will generate systematic returns
2. determine the exposures of the portfolio and benchmark to the factors at the start of the evaluation period
3. determine the manager’s active exposure to each factor
4. determine the active impact, which is the added return due to the manager’s active exposures

Question 47

Results from a multifactor model micro performance analysis will look similar to a returns-based style analysis. What is the primary difference?

Answer 47

The multifactor model uses seveal other factors such as use of:

• leverage
• market timing
• sector rotation
• size of firm

that would not be used in a returns-based style analysis.

Question 48

What are the **strengths and limitations **of micro performance attribution?

Answer 48

Strengths

• disaggregates performance effects of manager’s decisions between *sectors *and securities
• fairly easy to calculate

Limitations

• need to identify an appropriate benchmark with specified securities and weights at the start of the period
• security selection decisions will affect sector weighting (allocation/selection interaction)

Question 49

What are the strengths and limitations of fundamental factor model attribution?

Answer 49

Strengths

• identifies factors other than just security selection or sector allocation

Limitations

• exposures to the factors need to be determined at the start of the period
• can prove to be quite complex and lead to potential spurious correlations

Question 50

What is the primary difference between attribution analysis in fixed-income vs equity?

Answer 50

In fixed-income, duration and interest rates are typically the dominant factor in returns.

Question 51

In fixed-income performance attribution, what are the two major components?

Answer 51

1. effects from the external interest rate environment
   
   • ​​shifts, twists in the yield curve
   • beyond manager’s control. manager shouldn’t be penalized or rewarded
   • based on a term structure analysis of default-free securities
2. effects from the manager’s contribution

Question 52

What are the **two components **of the external interest rate effect?

Answer 52

1. A simulation of what the manager’s benchmark would have returned if interest rates had moved in the manner of the forward curve.
   
   • called the expected interest rate effect​
2. ​A simulation of the benchmark based on what actually happened to interest rates. The difference between this and (1) is due to changes in forward rates
   
   • called the unexpected interest rate effect

The sum of (1) and (2) are the external interest rate effect. The portfolio could have passively earned this return.

Question 53

What are the four components of fixed-income attribution that capture the actions of the manager?

Answer 53

1. Interest rate management effect
   
   • measures managers ability to anticipate changes in rates and adjust duration and convexity accordingly
2. Sector/quality management effect
   
   • considers what happened to the yield spreads on the actual sectors and quality of assets held in the portfolio
3. Security-selection effect
   
   • examines the actual securities selected by the manager
4. Trading effect
   
   • a plug figure

Question 54

How can you measure the interest rate management effect?

Answer 54

• Price each asset in the portfolio as if it were a default free bond
• Compare this to another simulation that also uses Treasury rates but includes:
  
  • changes manager made to duration and positioning on the yield curve

You can further break this down into components:

• duration
• convexity
• yield-curve shape

Question 55

What is an example of the sector/quality management effect?

Answer 55

If the manager holds corporate bonds and the corporate spreads narrow, the portfolio will outperform the Treasury-only simulation.

Question 56

What is an example of the security-selection effect?

Answer 56

For example: if corporate bond spreads narrowed 20 bps and the corporate bonds held by the manager narrowed more, the manager’s selection effect is positive for corporate bonds.

Question 57

What are the five common risk-adjusted performance measures?

Answer 57

1. Jensen’s alpha
2. The Information ratio
3. The Treynor measure
4. The Sharpe ratio
5. M^2 (Modigliani and Modigliani)

Question 58

What is the formula for Jensen’s Alpha?

Answer 58

R_A = R_F + Beta_A * (R_M - R_F)

expected return on account = RFR + beta of account * (expected return on market - RFR)

A linear regression** **can be used to calculate ex-post alpha:

alpha_A = R_At - R_A

ex-post alpha on the account = actual return on account in period t - predicted return on account

Question 59

What is the formula for the Treynor measure?

Answer 59

T_A= (R_A - R_F) / Beta_A

Uses beta as a measure of systematic risk

If a manager has positive alpha, will plot above the SML.

Treynor is slope of the line drawn from RFR to the portfolio on the SML.

Question 60

What is the formula for the Sharpe ratio?

Answer 60

S_A= (R_A - R_F) / sigma_A

Note that sharpe uses total risk whereas Jensen’s alpha and IR use systematic risk

Sharpe ratio of the *market *is slope of CML.

CAL is the line between the risk-free rate and the intersection of portfolio return and std deviation. Slope of CAL is the Sharpe ratio.

Question 61

What is the formula for the M² measure?

Answer 61

M_p²= R_F+ [(R_P- R_F) / sigma_P] * sigma_M

M² measures the value added or lost relative to the market if the portfolio had the same risk (std dev) as the market

It measures the result of a hypothetical portfolio that:

• uses leverage to increase risk and return if the portfolio has less risk than the market
• or lends at the risk free rate to lower risk and return if the portfolio has more risk than the market

Question 62

Which measures uses systematic risk and which use total risk?

Answer 62

Systematic Risk

• Alpha
• Treynor

Total Risk

• Sharpe
• M²

Question 63

What are criticisms of Alpha and Treynor?

Answer 63

Criticisms come from the fact that they both depend on beta and the assumptions of the CAPM.

1. assumption of single price risk rather than multifactor risk pricing
2. use of market proxy like SP500
   
   • small changes to what is considered to be the market can significantly change Jensen and Treynor measures

Question 64

What is a criticism of M²?

Answer 64

M² is criticized because it uses a benchmark that may not be precisely replicable.

Also, transaction costs to replicate the market or a custom benchmark are not considered.

Question 65

How are a portfolio’s alpha and beta incorporated into IR, Treynor, and Sharpe?

Answer 65

• Postive alpha will directly correlate to a Treynor ratio that is greater than the *market *Treynor ratio
• Beta is directly used in the Treynor measure
• Beta is indirectly used in IR because IR uses a benchmark to calculate excess return

Question 66

What is a quality control chart?

Answer 66

A quality control chart is a way of evaluating performance results.

X-axis: Time

Y-axis: Manager’s cumulative value-added return

• Horizontal line at zero (benchmark return).
• Cone shaped upper and lower 95% confidence intervals. (narrows as time goes on)

If cumulative returns were consistently above or below the line, that would indicate superior or inferior performance (not random performance).

Question 67

What are the **three assumptions **made about the **distribution of a manager’s value-added returns **in a quality control chart?

Answer 67

1. null hypothesis states that the expected value-added return is zero
2. value-added returns are independent and normally distributed
3. investment process is consistent (more or less constant variability of the value-added returns)

Question 68

What are the costs associated with hiring/firing a manager?

Answer 68

1. a proportion of the existing manager’s portfolio may need to be liquidated if the new manager’s style is significantly different
2. replacing managers involves a significant amount of time and effort for fund sponsor

Question 69

What are some elements of a formalized manager continuation policy?

Answer 69

• replace managers only when justified
  
  • short periods of underperformance are not enough
• develop formal policies and apply them consistently to all managers
• use portfolio performance and other information in evaluating managers
  
  • make sure manager’s strategy is appropriate and consistent
  • relevane benchmark (style) selections
  • personnel turnover
  • growth of account

Question 70

What is a Type I error?

Answer 70

Type I error: rejecting the null when it is true.

Keeping managers who are not adding value.

“Type I horn”

Question 71

What is a Type II error?

Answer 71

Type II error: failing to reject the null when it is false.

Firing good managers who are adding value.