Reading 31 Evaluating Portfolio Performance

Question 1

Three questions naturally arise in examining the investment performance of an account

Answer 1

Three questions naturally arise in examining the investment performance of an account:

• What was the account’s performance?
• Why did the account produce the observed performance?
• Is the account’s performance due to luck or skill?

The first issue is addressed by performance measurement, which calculates rates of return based on investment-related changes in an account’s value over specified time periods. Performance attribution deals with the second issue. It extends the results of performance measurement to investigate both the sources of the account’s performance relative to a specific investment benchmark and the importance of those sources. Finally, performance appraisal tackles the third question. It attempts to draw conclusions concerning the quality (that is, the magnitude and consistency) of the account’s relative performance.

Question 2

Performance measurement

Answer 2

Performance measurement is a component of performance evaluation. Performance measurement is the relatively simple procedure of calculating returns for an account. Performance evaluation, on the other hand, encompasses the broader and much more complex task of placing those investment results in the context of the account’s investment objectives.

Performance measurement is the first step in the performance evaluation process. Yet it is a critical step, because to be of value, performance evaluation requires accurate and timely rate-of-return information. Therefore, we must fully understand how to compute an account’s returns before advancing to more involved performance evaluation issues.

Performance appraisal involves interpretation of performance attribution.

Question 3

The emphasis on income-related return measures was due what factors?

Answer 3

Performance was typically measured on an income-only basis, thus excluding the impact of capital appreciation. For example, current yield (income-to-price) and yield-to-maturity were commonly quoted return measures.

The emphasis on income-related return measures was due to several factors:

• Portfolio management emphasis on fixed-income assets. Particularly in the low-volatility interest rate environment that existed prior to the late 1970s, bond prices tended to be stable. Generally high allocations to fixed-income assets made income the primary source of investment-related wealth production for many investors.
• Limited computing power. Accurately accounting for external cash flows when calculating rates of return that include capital appreciation requires the use of computers. Access to the necessary computing resources was not readily available. The income-related return measures were simpler and could be performed by hand.
• Less competitive investment environment. Investors, as a whole, were less sophisticated and less demanding of accurate performance measures.

Question 4

The time-weighted rate of return (TWR)

Answer 4

The time-weighted rate of return (TWR) reflects the compound rate of growth over a stated evaluation period of one unit of money initially invested in the account. Its calculation requires that the account be valued every time an external cash flow occurs.

The subperiod returns can be combined through a process called chain-linking. Chain-linking involves first adding 1 to the (decimal) rate of return for each subperiod to create a set of wealth relatives. A wealth relative can be thought of as the ending value of one unit of money (for example, one dollar) invested at each subperiod’s rate of return. Next, the wealth relatives are multiplied together to produce a cumulative wealth relative for the full period, and 1 is subtracted from the result to obtain the TWR.

Question 5

The money-weighted rate of return (MWR)

Answer 5

The money-weighted rate of return (MWR) measures the compound growth rate in the value of all funds invested in the account over the evaluation period. In the corporate finance literature, the MWR goes by the name internal rate of return, or IRR. Of importance for performance measurement, the MWR is the growth rate that will link the ending value of the account to its beginning value plus all intermediate cash flows. With MV1 and MV0 the values of the account at the end and beginning of the evaluation period, respectively, in equation form the MWR is the growth rate R that solves

MV₁ = MV₀(1 + R)^m + CF₁(1 + R)^m–L(1) + … + CF_n(1 + R)^m–L(n)

where

m = number of time units in the evaluation period (for example, the number of days in the month)

CF_i = the ith cash flow

L(i) = number of time units by which the ith cash flow is separated from the beginning of the evaluation period

Question 6

TWR versus MWR

Answer 6

• The MWR represents the average growth rate of all money invested in an account, while the TWR represents the growth of a single unit of money invested in the account. Consequently, the MWR is sensitive to the size and timing of external cash flows to and from the account, while the TWR is unaffected by these flows. Under “normal” conditions, these two return measures will produce similar results.
• When external cash flows occur that are large relative to the account’s value and the account’s performance is fluctuating significantly during the measurement period, then the MWR and the TWR can differ materially.
• If funds are contributed to an account prior to a period of strong performance, then the MWR will be positively affected compared to the TWR, as a relatively large sum is invested at a high growth rate.
• Conversely, if funds are withdrawn from the account prior to the strong performance, then the MWR will be adversely affected relative to the TWR. (The opposite conclusions hold if the external cash flow occurred prior to a period of weak performance.)
• The TWR is unaffected by external cash flow activity. Valuing the account at the time of each external cash flow effectively removes the impact of those flows on the TWR. Consequently, the TWR accurately reflects how an investor would have fared over the evaluation period if he or she had placed funds in the account at the beginning of the period.
• In most situations, an investment manager has little or no control over the size and timing of external cash flows into or out of his or her accounts. Therefore, practitioners generally prefer a rate-of-return measure that is not sensitive to cash flows if they want to evaluate how a manager’s investment actions have affected an account’s value.

Question 7

The Linked Internal Rate of Return (LIRR)

Answer 7

• Despite its useful characteristics, the TWR does have an important disadvantage: It requires account valuations on every date that an external cash flow takes place. Thus, calculation of the TWR typically necessitates the ability to price a portfolio of securities on a daily basis.
• The MWR, on the other hand, despite its sensitivity to the size and timing of external cash flows, requires only that an account be valued at the beginning and end of the evaluation period and that the amounts and dates of any external cash flows be recorded.
• The complementary advantages and disadvantages of the TWR and the MWR led to make an important recommendation: The TWR should be approximated by calculating the MWR over reasonably frequent time intervals and then chain-linking those returns over the entire evaluation period. This process is referred to as the Linked Internal Rate of Return (LIRR) method.
• The study concluded that only under unusual circumstances would the LIRR fail to provide an acceptable representation of the TWR. Specifically, the LIRR would fail if both large external cash flows (generally over 10% of the account’s value) and volatile swings in subperiod performance occurred during the evaluation period. With an evaluation period as short as one month, the chances of such a joint event occurring for an account are low.

Question 8

Annualized Return

Answer 8

In general, with measurement periods shorter than a full year, it is inadvisable to calculate annualized returns. Essentially, the person calculating returns is extrapolating the account’s returns over a sample period to the full year. Particularly for equity accounts, returns can fluctuate significantly during the remaining time in the evaluation period, making the annualized return a potentially unrealistic estimate of the account’s actual return over the full year.

Question 9

Matrix pricing

Answer 9

Many thinly traded fixed-income securities, a current market price may not always be available. In that case, estimated prices may be derived based on dealer-quoted prices for securities with similar attributes (for example, a security with a similar credit rating, maturity, and economic sector). This approach is referred to as matrix pricing.

Question 10

Properties of a Valid Benchmark

Answer 10

A valid benchmark is:

• Specified in advance. The benchmark is specified prior to the start of an evaluation period and known to all interested parties.
• Owned. The investment manager should be aware of and accept accountability for the constituents and performance of the benchmark. It is encouraged that the benchmark be embedded in and integral to the investment process and procedures of the investment manager.
• Measurable. The benchmark’s return is readily calculable on a reasonably frequent basis.
• Unambiguous. The identities and weights of securities or factor exposures constituting the benchmark are clearly defined.
• Reflective of current investment opinions. The manager has current investment knowledge (be it positive, negative, or neutral) of the securities or factor exposures within the benchmark.
• Appropriate. The benchmark is consistent with the manager’s investment style or area of expertise.
• Investable. It is possible to forgo active management and simply hold the benchmark.

Question 11

Seven primary types of benchmarks

Answer 11

There are seven primary types of benchmarks in use.

1. Absolute
2. Manager Universes
3. Broad Market Indexes
4. Style Indexes
5. Factor-Model-Based
6. Returns-Based
7. Custom Security-Based

Question 12

Manager Universes Benchmark

Answer 12

Consultants and fund sponsors frequently use the median manager or fund from a broad universe of managers or funds as a performance evaluation benchmark. A median manager benchmark fails all the tests of benchmark validity except for being measurable.

Question 13

Broad Market Indexes Benchmark

Answer 13

• Market indexes are well recognized, easy to understand, and widely available, and satisfy several properties of valid benchmarks. They are unambiguous, generally investable, and measurable, and they may be specified in advance.
• In certain situations, market indexes are perfectly acceptable as benchmarks, particularly as benchmarks for asset category performance or for “core” type investment approaches in which the manager selects from a universe of securities similar in composition to the benchmark.
• However, in other circumstances, the manager’s style may deviate considerably from the style reflected in a market index. For example, assigning a micro-capitalization US growth stock manager an S&P 500 benchmark clearly violates the appropriateness criterion.

Question 14

Style Indexes Becnhmark

Answer 14

• Broad market indexes have been increasingly partitioned to create investment style indexes that represent specific portions of an asset category: for example, subgroups within the US common stock asset category. Four popular US common stock style indexes are 1) large-capitalization growth, 2) large-capitalization value, 3) small-capitalization growth, and 4) small-capitalization value.
• Similar to broad market indexes, investment style indexes are often well known, easy to understand, and widely available. However, their ability to pass tests of benchmark validity can be problematic. Some style indexes contain weightings in certain securities and economic sectors that are much larger than what many managers consider prudent. Further, the definition of investment style implied in the benchmark may be ambiguous or inconsistent with the investment process of the manager being evaluated.
• Users of style indexes should closely examine how the indexes are constructed and assess their applicability to specific managers.

Question 15

Factor-Model-Based Benchmark

Answer 15

The simplest form of factor model is a one-factor model, such as the familiar market model. In that relationship, the return on a security, or a portfolio of securities, is expressed as a linear function of the return on a broad market index, established over a suitably long period (for example, 60 months):

R_p = a_p + β_pR_I + ε_p

These benchmarks are not always intuitive to the fund sponsor and particularly to the investment managers (who rarely think in terms of factor exposures when designing investment strategies), are not always easy to obtain, and are potentially expensive to use. In addition, they are ambiguous. We can build multiple benchmarks with the same factor exposures, but each benchmark can earn different returns. For example, we can construct two different portfolios, each with a beta of 1.2 (“normal beta”), but the portfolios can have materially different returns. Also, because the composition of a factor-based benchmark is not specified with respect to the constituent securities and their weights, we cannot verify all the validity criteria (the benchmark may not be investable, for example).

Question 16

Returns-Based Benchmark

Answer 16

These benchmarks are constructed using:

1) the series of a manager’s account returns (ideally, monthly returns going back in time as long as the investment process has been in place) and
2) the series of returns on several investment style indexes over the same period.

These return series are then submitted to an allocation algorithm that solves for the combination of investment style indexes that most closely tracks the account’s returns.

Returns-based benchmarks are generally easy to use and are intuitively appealing. They satisfy most benchmark validity criteria, including those of being unambiguous, measurable, investable, and specified in advance. Returns-based benchmarks are particularly useful in situations where the only information available is account returns. One disadvantage of returns-based benchmarks is that, like the style indexes that underlie the benchmarks, they may hold positions in securities and economic sectors that a manager might find unacceptable. Further, they require many months of observation to establish a statistically reliable pattern of style exposures. In the case of managers who rotate among style exposures, such a pattern may be impossible to discern.

Question 17

Custom Security-Based Benchmark

Answer 17

• A custom security-based benchmark is simply a manager’s research universe weighted in a particular fashion. Most managers do not use a security weighting scheme that is exactly an equal weighting across all securities or one that exactly assigns weights according to market capitalization. Consequently, a custom benchmark reflecting a particular manager’s unique weighting approach can be more suitable than a published index for a fair and accurate appraisal of that manager’s performance.
• The overwhelming advantage of a custom security-based benchmark is that it meets all of the required benchmark properties and satisfies all of the benchmark validity criteria, making it arguably the most appropriate benchmark for performance evaluation purposes. In addition, it is a valuable tool for managers to monitor and control their investment processes and for fund sponsors to effectively allocate or budget risk across teams of investment managers.
• One major disadvantage is that custom security-based benchmarks are expensive to construct and maintain. In addition, as they are not composed of published indexes, the perception of a lack of transparency can be of concern.
• In a custom-security-based benchmark, ther will be and should be misfit risk if the managers style is different than the broad market and if the custom benchmark accurately reflects the managers style.

The three steps reuired to construct a custom-security benchmark are as follows:

1. Identify the manager’s investment process including asset selection and weighting
2. Use the same assets and weighting for the benchmark
3. Assess and rebalance the benchmark on a predetermined schedule

Question 18

Building Custom Security-Based Benchmarks

Answer 18

The construction of such a benchmark involves the following steps:

1. Identify prominent aspects of the manager’s investment process.
2. Select securities consistent with that investment process.
3. Devise a weighting scheme for the benchmark securities, including a cash position.
4. Review the preliminary benchmark and make modifications.
5. Rebalance the benchmark portfolio on a predetermined schedule.

Question 19

Critique of Manager Universes as Benchmarks

Answer 19

• With the exception of being measurable, the median account in a typical commercially available universe does not have the properties of a valid benchmark.
• One of the most significant deficiencies is that, although the universe can be named, the median account cannot be specified in advance. Universe compilers can only establish the median account on an ex post basis, after the returns earned by all accounts have been calculated and ranked. Prior to the start of an evaluation period, neither the manager nor the fund sponsor has any knowledge of who the median manager will be at period end.
• In addition, different accounts will fall at the median from one evaluation period to another. For these reasons, the benchmark is not investable and cannot serve as a passive alternative to holding the account that is under analysis.
• Even after the evaluation period concludes, the identity of the median manager typically remains unknown, preventing the benchmark from satisfying the unambiguous property. The ambiguity of the median manager benchmark makes it impossible to verify its appropriateness by examining whether the investment style it represents adequately corresponds to the account being evaluated.
• Because fund sponsors terminate underperforming managers, universes are unavoidably subject to “survivor bias.”
• Placing above the median of a universe of investment managers or funds may be a reasonable investment objective, but the performance of a particular manager or fund is not a suitable performance benchmark that can be used to assess investment skill.

Question 20

Set of benchmark quality criteria

Answer 20

Systematic Biases

Over time, there should be minimal systematic biases or risks in the benchmark relative to the account. One way to measure this criterion is to calculate the historical beta of the account relative to the benchmark; on average, it should be close to 1.0.

Tracking Error

We define tracking error as the volatility of A or (P – B). A good benchmark should reduce the “noise” in the performance evaluation process. Thus, the volatility (standard deviation) of an account’s returns relative to a good benchmark should be less than the volatility of the account’s returns versus a market index or other alternative benchmarks.

Risk Characteristics

An account’s exposure to systematic sources of risk should be similar to those of the benchmark over time. The objective of a good benchmark is to reflect but not to replicate the manager’s investment process. Because an active manager is constantly making bets against the benchmark, a good benchmark will exhibit risk exposures at times greater than those of the managed portfolio and at times smaller. Nevertheless, if the account’s risk characteristics are always greater or always smaller than those of the benchmark, a systematic bias exists.

Coverage

Benchmark coverage is defined as the proportion of a portfolio’s market value that is contained in the benchmark. For example, at a point in time, all of the securities and their respective weights that are contained in the account and the benchmark can be examined. The market value of the jointly held securities as a percentage of the total market value of the portfolio is termed the coverage ratio. High coverage indicates a strong correspondence between the manager’s universe of potential securities and the benchmark. Low coverage indicates that the benchmark has little relationship, on a security level, with the opportunity set generated by the manager’s investment process.

Turnover

Benchmark turnover is the proportion of the benchmark’s market value allocated to purchases during a periodic rebalancing of the benchmark. Because the benchmark should be an investable alternative to holding the manager’s actual portfolio, the benchmark turnover should not be so excessive as to preclude the successful implementation of a passively managed portfolio.

Positive Active Positions

An active position is an account’s allocation to a security minus the corresponding weight of the same security in the benchmark. When a good custom security-based benchmark has been built, the manager should be expected to hold largely positive active positions for actively managed long-only accounts.

Note that when an account is benchmarked to a published index containing securities for which a long-only manager has no investment opinion and which the manager does not own, negative active positions will arise. A high proportion of negative active positions is indicative of a benchmark that is poorly representative of the manager’s investment approach.

Question 21

Hedge Funds and Hedge Fund Benchmarks

Answer 21

• Hedge funds attempt to expose investors to a particular investment opportunity while minimizing (or hedging) other investment risks that could impact the outcome. In most cases, hedging involves both long and short investment positions.
• The ambiguity of hedge fund manager opportunity sets has led to the widespread use of the Sharpe ratio to evaluate hedge fund manager performance.
• Typically, a hedge fund’s Sharpe ratio is compared to that of a universe of other hedge funds that have investment mandates assumed to resemble those of the hedge fund under evaluation. Unfortunately, this approach is exposed to the same benchmark validity criticisms leveled against standard manager universe comparisons. Further, the standard deviation as a measure of risk (the denominator of the Sharpe ratio) is questionable when an investment strategy incorporates a high degree of optionality (skewness), as is the case for the strategies of many hedge funds.
• Problems with using traditional techniques to assess long-short hedge funds include:
  
  • It is possible for MV₀ (market value at the beginning of the period) to be zero for a long-short portfolio, making the return calculation nonsensical.
  • Many hedge funds use “absolute return” approach, which makes relative performance comparison with a traditional benchmark less useful.
• Alternative performance methods that can be used instead:
  
  • Value-added return
  • Creating separate long/short benchmarks
  • the Sharpe ratio

Question 22

Impact Equals Weight Times Return

Answer 22

A manager can have a positive impact on an account’s return relative to a benchmark through two basic avenues:

1) selecting superior (or avoiding inferior) performing assets and
2) owning the superior (inferior) performing assets in greater (lesser) proportions than are held in the benchmark.

This simple concept underlies all types of performance attribution. The assets themselves may be divided or combined into all sorts of categories, be they economic sectors, financial factors, or investment strategies. In the end, however, the fundamental rule prevails that impact equals (active) weight times return.

Question 23

Conducting a Macro Attribution Analysis

Answer 23

Six levels or components of investment policy decision making into which the Fund’s performance might be analyzed.

Specifically, those levels (which we later refer to as “investment strategies” for reasons to become apparent shortly) are:

1. Net Contributions
2. Risk-Free Asset
3. Asset Categories
4. Benchmarks
5. Investment Managers
6. Allocation Effects

Macro attribution starts with the fund’s beggining value and ends with its ending market value. In between six levels of analysis that attribute the change in market value to sources of increase or decrease in market value.

Question 24

Micro Attribution

Answer 24

• Pure Sector Allocation return: difference between the weight of the Portfolio to a given sector and the Portfolio’s benchmark weight for that sector
• Within-Sector Selection return: stock picking
• Allocation/Selection Interaction return: equals out all values

Question 25

Level 1 of macro attribution, Net contributions

Answer 25

Is the sim of external cash flows made by the client into or withdawn from the portfolio. Net contributions increase or decrease ending market value but are not investment value added or lost.

Question 26

Level 2 of macro attribution, Risk-free investment

Answer 26

Simulates what fund’s ending value would havfe been if the beggining value and external cash flows had earned the risk-free return.

Question 27

Level 3 of macro attribution, Asset categories

Answer 27

Recognizes that most sponsors will consider risk-free investments as too conservative. It simulates the ending value of beggining value and external cash flows if funds had been invested in asset category benchmarks weighted in accord with the fund’s strategic policy (in other words, passively replicating the strategic asset allocation with index funds).

The incremental return of the asset category level is the weighted average of the categories’ returns over the risk-free asset. It could be calculated as:

R_AC=Σ _{(i=1; A)} (w_i)(R_i-R_F)

where:

R_AC = incremental return (above the risk-free rate) for the asset category strategy

(R_i-R_F) = excess return (above the risk-free rate) for asset category i

w_i = weight of asset category i

A = number of asset categories

Up to this point, all results could havfe been achieved by passively implementing the fund’s strategic asset allocation.

Question 28

Level 4 of macro attribution, the benchmark level

Answer 28

Allows the sponsor to select and assign managers a benchmark different from the policy benchmark. This is tactical asset allocation by the sponsor. For example, 60% in the S&P 500 might fit the fund’s strategic ibjective but the sponsor may expect value stocks to outperform the S&P. The sponsor could direct the manager to use the S&P value index as the manager’s target or manager benchmark. Level 4 simulates the returns of the beginning market value and external cash flows if invested in manager benchmarks. The Level 4 result can also be passively achieved but reflects active decision making by the sponsor to deviate from strategic benchmarks. The level 4 incremental return could be calculated as:

R_B = Σ_(i=1;A)Σ_(j=1;M)(w_i)(w_i,j)(R_B,i,j - R_i)

where:

R_B = incremental return fot benchmark strategy

w_i = policy weight for asset category i

w_i,j = weight assigned to manager j in asset category i

R_B,i,j = return for manager j’s benchmark in category i

R_i = return on asset category i

A = number of asset categories

M = number of managers in asset category i

Question 29

Level 5 of macro attribution, Investment managers or active managers

Answer 29

Simulates the results of investing the fund’s beginning value and external cash flows and earning the returns actually produced by the managers. The simulation has actually allocated funds in accord with the policy allocation, an assumption that is usually not perfectly implemented. The level 5 incremental return could be calculated as:

R_IM = Σ_(i=1;A)Σ_(j=1;M) (w_i)(w_i,j)(R_A,i,j - R_B,i,j)

where:

R_IM = incremental return for the investment manager level

w_i = policy weight of asset category i

w_i,j = policy weight of manager j in asset category j

RA,i,j = return for manager j’s portfolio in category i

R_B,i,j = return for jth manager’s benchmark for asseet category i

A = number of asset categories

M = number pf managers in asset category i

Question 30

Level 6 of macro attribution, Allocation effects

Answer 30

Is simply a plug to sum to the portfolio ending value. If all policies were perfectly implemented, the allocation effect would be zero.

Question 31

Fundamental Factor Model Micro Attribution

Answer 31

Economic sectors and industries represent only one potential source of common factor returns. There are other common factor return sources. For example, with respect to common stocks, a company’s size, its industry, its growth characteristics, its financial strength, and other factors seem to have an impact on account performance. Often these factors are referred to as fundamental factors.

Question 32

Sources of the Total Return of a Fixed-Income Portfolio

Answer 32

The total return of a fixed-income portfolio can be attributed to the external interest rate effect, on one hand, and the management effect, on the other. The return due to the external interest rate environment is estimated from a term structure analysis of a universe of Treasury securities and can be further separated into:

• the return from the implied forward rates (the expected return) and
• the difference between the actual realized return and the market implied return from the forward rates (the unexpected return).

The overall external interest rate effect represents the performance of a passive, default-free bond portfolio.

The management effect can be decomposed into four components:

• Interest rate management effect: Indicates how well the manager predicts interest rate changes. To calculate this return, each security in the portfolio is priced as if it were a default-free security. The interest rate management contribution is calculated by subtracting the return of the entire Treasury universe from the aggregate return of these repriced securities. The interest rate management effect can be further broken down into returns due to duration, convexity, and yield-curve shape change.
• Sector/quality effect: Measures the manager’s ability to select the “right” issuing sector and quality group. The sector/quality return is estimated by repricing each security in the portfolio using the average yield premium in its respective category. A gross return can be then calculated based on this price. The return from the sector/quality effect is calculated by subtracting the external effect and the interest rate management effect from this gross return.
• Security selection effect: Measures how the return of a specific security within its sector relates to the average performance of the sector. The security selection effect for each security is the total return of a security minus all the other components. The portfolio security selection effect is the market-value weighted average of all the individual security selection effects.
• Trading activity: Captures the effect of sales and purchases of bonds over a given period and is the total portfolio return minus all the other components.

Question 33

Investment skill

Answer 33

Investment skill - the ability to outperform an appropriate benchmark consistently over time.

Question 34

Risk-Adjusted Performance Appraisal Measures

Answer 34

Three risk-adjusted performance appraisal measures have become widely used:

• ex post alpha (also known as Jensen’s alpha),
• the Treynor measure (also known as reward-to-volatility or excess return to nondiversifiable risk), and
• the Sharpe ratio (also known as reward-to-variability).

Another measure, M², has also received some acceptance.

Question 35

Ex Post Alpha

Answer 35

The ex post alpha uses the ex post Security Market Line (SML) to form a benchmark for performance appraisal purposes.

R_At – r_ft = α_A + β_A(R_Mt – r_ft) + ε_t

where for period t, R_At is the return on the account, r_ft is the risk-free return, and R_Mt is the return on the market proxy (market index). The term α_A is the intercept of the regression, β_A is the beta of the account relative to the market index, and ε is the random error term of the regression equation. The estimate of the intercept term α_A is the ex post alpha (Jensen measure). We can interpret ex post alpha as the differential return of the account compared to the return required to compensate for the systematic risk assumed by the account during the evaluation period. The level of the manager’s demonstrated skill is indicated by the sign and value of the ex post alpha.

Question 36

Treynor Measure

Answer 36

The Treynor measure is closely related to the ex post alpha. Like the ex post alpha, the Treynor measure relates an account’s excess returns to the systematic risk assumed by the account. As a result, it too uses the ex post SML to form a benchmark, but in a somewhat different manner than the ex post alpha. The calculation of the Treynor measure is

T_A=(R_A−r_f)/β_A

R_A and r_f are the average values of each variable over the evaluation period. The Treynor measure has a relatively simple visual interpretation, given that the beta of the risk-free asset is zero. The Treynor measure is simply the slope of a line, graphed in the space of mean ex post returns and beta, which connects the average risk-free return to the point representing the average return and beta of the account. When viewed alongside the ex post SML, the account’s benchmark effectively becomes the slope of the ex post SML. Thus, a skillful manager will produce returns that result in a slope greater than the slope of the ex post SML.

Both the ex post alpha and the Treynor measure will always give the same assessment of the existence of investment skill. This correspondence is evident from the fact that any account with a positive ex post alpha must plot above the ex post SML. Therefore, the slope of a line connecting the risk-free rate to this account must be greater than the slope of the ex post SML, the indication of skill under the Treynor measure.

The Treynor measure uses only systematic risk (i.e., beta) in the denominator, so lowering the unsystematic risk of the asset will have no affect on the Treynor measure.

Alpha and Treynor both measure risk as systematic risk (beta). They will agree in that a manager with positive alpha will have a Treynor in excess of the market Treynor. They may not always agree in relative ranking. A manger with the highest alpha may not have the highest Treynor.

Both Alpha and Treynor are criticized because they depend on beta and assumptions of the CAPM. The criticism include:

1. the assumption of a single priced risk rather than some form of multifactor risk pricing and
2. the use of a market proxy, such as the S&P 500, to stand for the market. Roll’s critque shows that small changes in what is assumed to be the market can significantly change tha alpha and Treynor calculations and even reverse the conclusions of superior or inferior perforance and rankings.

Question 37

Sharpe Ratio

Answer 37

Both the ex post alpha and Treynor measure compare excess returns on an account relative to the account’s systematic risk. In contrast, the Sharpe ratio compares excess returns to the total risk of the account, where total risk is measured by the account’s standard deviation of returns. The ex post Sharpe ratio is traditionally given by:

S_A=(R_A−r_f)/σ_A

The benchmark in the case of the Sharpe ratio is based on the ex post capital market line (CML). The ex post CML is plotted in the space of returns and standard deviation of returns and connects the risk-free return and the point representing the mean return on the market index and its estimated standard deviation during the evaluation period.

The Sharpe ratio uses standard deviation, which includes both unsystematic and systematic risk. Lowering unsystematic risk, therefore, will lower the denominator and increase the Sharpe ratio.

Question 38

M²

Answer 38

Like the Sharpe ratio, M² uses standard deviation as the measure of risk and is based on the ex post CML. M² is the mean incremental return over a market index of a hypothetical portfolio formed by combining the account with borrowing or lending at the risk-free rate so as to match the standard deviation of the market index. M² measures what the account would have returned if it had taken on the same total risk as the market index. To produce that benchmark, M² scales up or down the excess return of the account over the risk-free rate by a factor equal to the ratio of the market index’s standard deviation to the account’s standard deviation.

M^{<span>2</span>}_{<span>A</span>}=r_f+(R_A−r_f) * (σ_M/σ_A)

M² will evaluate the skill of a manager exactly as does the Sharpe ratio. However, it is possible for the Sharpe ratio and M² to identify a manager as not skillful, although the ex post alpha and the Treynor measure come to the opposite conclusion. This outcome is most likely to occur in instances where the manager takes on a large amount of nonsystematic risk in the account relative to the account’s systematic risk.

Question 39

Information Ratio

Answer 39

IR_A=(R_A−R_B)/σ_A-B

where σ_A−B is the standard deviation of the difference between the returns on the account and the returns on the benchmark. The Sharpe ratio in this form is commonly referred to as the information ratio, defined as the excess return of the account over the benchmark relative to the variability of that excess return. The numerator is often referred to as the active return on the account, and the denominator is referred to as the account’s active risk. Thus, from this perspective, the information ratio measures the reward earned by the account manager per incremental unit of risk created by deviating from the benchmark’s holdings.

Question 40

Criticisms of Risk-Adjusted Performance Appraisal Methods

Answer 40

• Perhaps the most prominent criticisms have involved the reliance of the ex post alpha and the Treynor measure on the validity of the CAPM.
• Critics have also pointed to problems raised by the use of surrogates (such as the S&P 500) for the true market portfolio. Roll showed that slight changes in the market portfolio surrogate can yield significantly different performance appraisal answers.
• Use of a market index or custom benchmark in the appraisal of investment performance is open to criticism in that it is difficult in most cases for the account manager to replicate precisely the benchmark’s return over time. Transaction costs associated with initially creating and then later rebalancing the benchmark, as well as the costs of reinvesting income flows, mean that the benchmark’s reported returns overstate the performance that a passive investor in the benchmark could earn.
• Stability of the parameters and the estimation error involved in the risk-adjusted appraisal measures is also an issue. Even if the assumptions underlying the appraisal measures hold true, the ex post calculations are merely estimates of the true parameters of the actual risk–return relationships. If the estimates are recalculated over another period, they may well show conclusions that conflict with the earlier estimates, even if those relationships are stable over time. Further, that stability cannot be taken for granted; the aggressiveness of the account manager may change rapidly over time in ways that cannot be captured by the estimation procedures.

Question 41

Quality Control Charts

Answer 41

• One effective means of presenting performance appraisal data is through the use of quality control charts.
• Underlying the quality control chart’s construction are three assumptions concerning the likely distribution of the manager’s value-added returns.
• The primary assumption (and one that we will subsequently test) is referred to as the null hypothesis. The null hypothesis of the quality control chart is that the manager has no investment skill; thus, the expected value-added return is zero.
• Our second assumption states that the manager’s value-added returns are independent from period to period and normally distributed around the expected value of zero.
• The third assumption is that the manager’s investment process does not change from period to period. Among other things, this third assumption implies that the variability of the manager’s value-added returns remains constant over time.
• Given this information, we can compute a confidence band associated with the expected distribution of the manager’s value-added returns. Based on our three assumptions, the confidence band indicates the range in which we anticipate that the manager’s value-added returns will fall a specified percentage of the time.
• Type I error is when the null hypothesis is rejected when it is true (not fire bad managers)
• Type 2 error is failure to reject the null when it is false (fire a good manager)

Question 42

The Practice of Performance Evaluation

Answer 42

In summary, using past performance to evaluate existing managers is statistically problematic. In the long run, superior managers will outperform inferior managers. However, due to the inherent uncertainty of investment management, over typical evaluation periods (3–5 years) the odds that superior managers will underperform their benchmarks (and, conversely, that inferior managers will outperform their benchmarks) are disturbingly high. Expensive, incorrect decisions may frequently result from relying on past performance to evaluate investment managers.

Question 43

Manager Continuation Policy

Answer 43

In an attempt to reduce the costs of manager turnover yet systematically act on indications of future poor performance, some fund sponsors have adopted formal, written manager continuation policies (MCP) to guide their manager evaluations. The purpose of an MCP is severalfold:

• to retain superior managers and to remove inferior managers, preferably before the latter can produce adverse results;
• to ensure that relevant nonperformance information is given significant weight in the evaluation process;
• to minimize manager turnover; and
• to develop procedures that will be consistently applied regardless of investment committee and staff changes.

An MCP can be viewed as a two-part process. The first part we refer to as manager monitoring, while the second part we call manager review.

Question 44

Manager Monitoring

Answer 44

• The goal of MCP manager monitoring is to identify warning signs of adverse changes in existing managers’ organizations. It is a formal, documented procedure that assists fund sponsors in consistently collecting information relevant to evaluating the state of their managers’ operations. The key is that the fund sponsor regularly asks the same important questions, both in written correspondence and in face-to-face meetings.
• As part of the manager monitoring process, the fund sponsor periodically receives information from the managers, either in written form or through face-to-face meetings. This information is divided into two parts.
• The first part covers operational matters, such as personnel changes, account growth, litigation, and so on. The staff should flag significant items and discuss them in a timely manner with the respective managers.
• The second part of the responses contains a discussion of the managers’ investment strategies, on both a retrospective and a prospective basis. The fund sponsor should instruct the managers to explain their recent investment strategies relative to their respective benchmarks and how those strategies performed.
• The goal of these discussions is to assure the fund sponsor that the manager is continuing to pursue a coherent, consistent investment approach. Unsatisfactory manager responses may be interpreted as warning signs that the manager’s investment approach may be less well-defined or less consistently implemented than the staff had previously believed.
• As part of the manager monitoring process, the staff should regularly collect portfolio return and composition data for a performance attribution analysis. The purpose of such a periodic analysis is to evaluate not how well the managers have performed, but whether that performance has been consistent with the managers’ stated investment styles.

Question 45

Manager Review

Answer 45

The manager review closely resembles the manager selection process, in both the information considered and the comprehensiveness of the analysis. The staff should review all phases of the manager’s operations, just as if the manager were being initially hired. We can view this manager review as a zero-based budgeting process (a budgeting process in which all expenditures must be justified each new period). We want to answer the question, “Would we hire the manager again today?”

The primary differences between hiring a new manager and retaining a manager under review are that the fund sponsor once had enough confidence in the manager to entrust a large sum of money to the manager’s judgment and that there is a sizable cost associated with firing the manager. Thus, the fund sponsor should address the following questions:

• What has fundamentally changed in the manager’s operation?
• Is the change significant?
• What are the likely ramifications of the change?
• Are the costs of firing the manager outweighed by the potential benefits?

Question 46

Manager Continuation Policy as a Filter

Answer 46

Two types of decision errors may occur:

• Type I error—keeping (or hiring) managers with zero value-added. (Rejecting the null hypothesis when it is correct.)
• Type II error—firing (or not hiring) managers with positive value-added. (Not rejecting the null hypothesis when it is incorrect.)

A coarse filter will be conducive to Type I errors. Conversely, a fine filter will lead the sponsor to commit more Type II errors.

If in truth the manager has no skill and we reject the null hypothesis because the manager’s value-added returns fall outside of the confidence band (particularly, in this case, on the upside), then we have committed a Type I error. Conversely, if the manager is indeed skillful yet we fail to reject the null hypothesis because the manager’s value-added returns fall inside the confidence band, then we have committed a Type II error.

Both Type I and Type II errors are expensive.

Question 47

The challenges inherent in performance measurement and performance evaluation for a long–short hedge fund

Answer 47

There are performance measurement issues that are created when there are short positions in a portfolio. The basic equation for the return on an account is

r_t=(MV₁−MV₀)/MV₀

In theory, the net assets of a long-short portfolio could be zero. If the value of the portfolio’s long positions is equal to the value of the portfolio’s short positions, then the beginning market value, MV₀, would be zero and the account’s rate of return would be either positive infinity or negative infinity.

To address this problem, we need to revise our performance measurement methodology. One approach would be to determine returns by summing the performance impacts of the individual security positions (both long and short). A return could be calculated for the period that the individual security positions were maintained. Once an individual security position changed, the return period would end and a new return period would start.

Regarding performance evaluation, if we have information regarding the historical returns and holdings of a long–short equity manager’s long and short portfolios, we could use either returns-based or security-based benchmark building approaches to construct separate long and short benchmarks for the manager. These benchmarks could then be combined in appropriate proportions to create an appropriate benchmark for the manager.

Another possible option for performance evaluation is the use of the Sharpe ratio to evaluate hedge fund manager performance. It can be calculated without reference to the manager’s underlying investment universe. Typically, a hedge fund manager’s Sharpe ratio is compared to that of a universe of other hedge fund managers whose investment mandates are similar to those of the manager under evaluation.

Question 48

Compare and contrast macro attribution with micro attribution. What is the difference between using a return metric and using a dollar metric?

Answer 48

Both macro attribution and micro attribution are different facets of performance attribution. The basic tenet behind performance attribution is that an account’s performance is compared to a designated benchmark, then the sources of differential returns are identified and quantified. The main difference between macro and micro attribution is the definition of which “account’s” performance we are analyzing. Macro attribution is done at the fund sponsor level; that is, analysis is typically done for a grouping of investment managers or investment accounts. Micro attribution is carried out at the level of the individual investment manager.

There are three main inputs to the macro attribution approach:

1. policy allocations;
2. benchmark portfolio returns; and
3. fund returns, valuations, and external cash flows.

Fund sponsors determine policy allocations, or “normal” weightings, for each asset class and individual manager. These are typically determined after some sort of asset liability analysis and/or determination of the risk tolerance of the governing body of the fund.

Benchmark portfolio returns are an important factor in determining the value added by the fund. If the benchmarks do not adequately match the managers’ investment styles, the performance attribution will have little value. Fund sponsors may use broad market indexes as the benchmarks for asset categories (the Wilshire 5000 as the benchmark for overall US domestic equities, for example) and may use more focused indexes to represent managers’ investment styles (such as the Russell 2000 Value Index for a small-cap value manager).

Fund returns, valuations, and external cash flows are all critical elements for determining the relevant performance for the portfolio as a whole and for each individual investment manager’s account.

A return metric implies that fund returns are used at the level of the individual management account to allow an analysis of the fund sponsor’s decisions regarding manager selection. A dollar-metric approach uses account valuation and external cash flow data to calculate rates of return and also to compute the dollar impacts of the fund sponsor’s investment policy decision making.

Question 49

Coverage ratio

Answer 49

The coverage ratio is the market value of securities that are both in the portfolio and the benchmark. Specified as a percentage of the total market value of the portfolio. The higher the coverage ratio, the more closely the manager is replicating the benchmark. Benchmark turnover is the proportion of the benchmark`s total market value that is bought or sold (turned over) during periodic rebalancing. Passive-managed portfolios should utilize benchmarks with low turnover.

Question 50

Specify one way for fundamental factor model micro attribution is similiar to and different from a returns-based style analysis

Answer 50

The methods are similiar in that they both use the following initial steps in constructing a suitable factor model:

• Identify the fundamental factors that will generate systimatic returns
• Determine the exposures of the portfolio and the benchmark to the fundamental factors
• Determine the performance of each of the factor

In this way, the fundamental factor model micro attribution results will look very similiar to a return-based style analysis and can be determined the same way (e.g., the returns to the portfolio are regressed against the returns to several different indices to determine the factor exposures).

The primary difference between them is the use of other fundamental factors (e.g., management`s use of leverage, market timing, sector rotation, the size of the firm, and so on) that would not ordinarily be used in a returns-based style analysis.

Question 51

Matrix pricing

Answer 51

Matrix pricing is using the quoted price of a similiar asset as a proxy for the market value of thinly traded (illiquid) fixed income securities.

Question 52

Trade accounting vs. settlement accounting

Answer 52

The use of trade date accounting is regarded to be a key feature of a good return measurement process.

Question 53

In global performance evaluation, performance attribution seeks to?

Answer 53

Performance attribution seeks to identify the sources between portfolio and benchmark return.

Note that performance measurement involves the calculation of risk and return, while performance appraisal seeks to identify whether returns are a result of a manager`s luck or skill.

Question 54

Factors on which can be based the test of benchmark’s quality

Answer 54

1. Over time, there should be minimal systematic biases or risks in the benchmark relative to the portfolio. One measure of this criterion is the historical beta of the portfolio relative to the benchmark; on average it should be close to 1.0
2. A high quality benchmark should reduce the “noise” in the performance evaluation process. Therefore, the tracking error of the portfolio relative to a hich quality benchmark should be lower than the tracking error relative to alternative benchmarks.
3. ! Market cap is used as a method of evaluation the appropriateness of a benchmark given a manager’s investment style, rather than as a test of benchmark quality.

Question 55

Inputs to macro attribution approach

Answer 55

1. Policy allocations
2. Benchmark portfolio returns
3. Fund returns, valuations, external cash flows

Question 56

Performance Attribution

Answer 56

• Account’s performance is compared to a designated benchmark
• Sources of differential return are identified and quantified