Topics 40-42 Flashcards
Poor Data Quality
A list of negative impacts on a business from poor data quality.
- Financial impacts:*
- Businesses may experience lower revenues (e.g., lost sales), higher expenses (e.g., penalties, re-work costs), and lower cash flows as a result of inaccurate or incomplete data.
- Confidence-based impacts:*
- Managers may make incorrect business decisions based on faulty data.
- Poor forecasting may result due to input errors.
- Inaccurate internal reporting may occur with unreliable information.
Satisfaction impacts:
- Customers may become dissatisfied when the business processes faulty data (e.g., billing errors).
- Employees may become dissatisfied when they are unable to properly perform their job due to flawed data.
Productivity impacts:
- Additional (corrective) work may be required, thereby reducing production output.
- Delays or increases in processing time.
Risk impacts:
- Underestimating credit risks due to inaccurate documentation, thereby exposing a lender to potential losses (e.g., Basel II Accords for quantifying credit risk).
- Underestimating investment risk, thereby exposing an investor to potential losses.
- Compliance impacts:*
- A business may no longer be in compliance with regulations (e.g., Sarbanes-Oxley) if financial reports are inaccurate.
Identify the most common issues that result in data errors
The most common data issues that increase risk for an organization are as follows:
- Data entry errors.
- Missing data.
- Duplicate records.
- Inconsistent data.
- Nonstandard formats.
- Complex data transformations.
- Failed identity management processes.
- Undocumented, incorrect, or misleading metadata (description of content and context of data files).
From a financial perspective, such data errors (accidental or not) may lead to inconsistent reporting, incorrect product pricing, and failures in trade settlement.
Examples of risks arising out of data errors include:
- Fraudulent payroll overpayments to fictitious employees or those who are no longer employed by the firm.
- Underbilling for services rendered.
- Underestimating insurance risk due to missing and inaccurate values (e.g., insured value).
Explain how a firm can set expectations for its data quality and describe some key dimensions of data quality used in this process
The important (but not complete) set of dimensions that characterize acceptable data include accuracy, completeness, consistency, reasonableness, currency, and uniqueness.
Accuracy
The concept of accuracy can be described as the degree to which data correctly reflects the real world object.
Completeness
Completeness refers to the extent to which the expected attributes of data are provided. There may be mandatory and optional aspects of completeness. Note that although data may be complete, it may not necessarily be accurate.
Consistency
Consistency refers to reasonable comparison of values between multiple data sets.
Note that consistency does not necessarily imply accuracy.
There are three types of consistency:
- Record level: consistency between one set of data values and another set within the same record.
- Cross-record level: consistency between one set of data values and another set in different records.
- Temporal level: consistency between one set of data values and another set within the same record at different points in time.
Reasonableness
Reasonableness refers to conformity with consistency expectations. For example, the income statement value for interest expense should be consistent or within an acceptable range when compared to the corresponding balance sheet value for long-term debt.
Currency
Currency of data refers to the lifespan of data. In other words, is the data still considered relevant and useful, given that the passage of time will gradually render it less current and less correct? Measurement of currency would consist of determining the frequency in which the data needs to be updated, and determining whether the existing data is still up-to-date.
Uniqueness
Uniqueness of data is tied into the data error involving duplicate records. Uniqueness suggests that there can only be one data item within the data set.
Describe the operational data governance process, including the use of scorecards in managing information risk
Operational data governance refers to the collective set of rules and processes regarding data that allow an organization to have sufficient confidence in the quality of its data.
Specifically, a data governance program should exist that clarifies the roles and responsibilities in managing data quality. A data quality scorecard could be used to monitor the success of such a program.
In short, operational data governance aims to detect data errors early on and then set into motion the steps needed to sufficiently deal with the errors on a timely basis. As a result, there should be minimal or no subsequent impact on the organization.
Data Quality Inspection vs. Data Validation
Data validation is a one-time step that reviews and assesses whether data conforms to defined business specifications. In contrast, data quality inspection is an on-going set of
steps aimed to:
- reduce the number of errors to a tolerable level,
- spot data flaws and make appropriate adjustments to allow data processing to be completed, and
- solve the cause of the errors and flaws in a timely manner.
The goal of data quality inspection is to catch issues early on before they have a substantial negative impact on business operations.
Data Quality Scorecard
- A base-level metric is straightforward in that it is measured against clear data quality criteria. It is relatively easy to quantify whether the criteria is met in terms of arriving at a data quality score.
- In contrast, a complex metric is a combined score that could be a weighted average of several different metrics (customized to the specific user(s)). Such a combined metric allows for a qualitative reporting of the impact of data quality on the organization. A data quality scorecard could report the metric in one of three ways: by issue, by business process, or by business impact.
Complex Metric Scorecard Viewpoints
Data quality issues view :
- Considers the impact of a specific data quality problem over multiple business processes.
- The scorecard shows a combined and summarized view of the impacts for each data problem.
Business process view :
- For each business process, the scorecard has complex metrics that quantify the impact of each data quality problem.
- It allows for the ability to determine exactly where in the business process the data problem is originating.
Business impact view :
- The scorecard provides a high-level understanding of the risks embedded in data quality problems (i.e., a combined and summarized view).
- By going into more detail, one can identify the business processes where the problems occur.
Describe the seven Basel II event risk categories
- It is important to recognize that the severity and frequency of losses can vary dramatically among the categories. For example, loss events are small but occur very frequently in the Execution, Delivery, and Process Management category.
- Whereas, losses are much less frequent but typically have a large dollar amount in the Clients, Products, and Business Practices category as these loss events commonly arise from substantial litigation suits.
- The modeling of loss event data differs for each category. Thus, it is important to make sure every event is placed in the appropriate group. When assigning loss events, consistency is more important than accuracy.
- The process of identifying and classifying risks is commonly referred to as OpRisk taxonomy.
- There are roughly three ways the firms drive risk taxonomy exercise: cause-driven, impact-driven, event driven. The last one (event-driven) is the superior one, the first one (cause-driven) is inferior. A mixture of the three method should never by applied!
Six level 2 categories for the event type identified as Execution, Delivery, and Process Management (EDPM)
Figure 2 identifies the six level 2 categories for the event type identified in level 1 as Execution, Delivery, and Process Management (EDPM).
For financial firms, the EDPM category typically has the highest frequency of occurrence compared to the other categories.
Subcategories with examples for the Clients, Products, and Business Practices (CPBP) category
The second Basel II category listed in Figure 1 is Clients, Products, and Business Practices (CPBP). The most common type of loss events in this category arise from disagreements between clients and counterparties, as well as regulatory fines for negligent business practices and advisory fiduciary duties.
Examples of operational risk events for all categories except EDPM and CPBP
- The Business Disruption and System Failures (BDSF) category is far less common than the first two Basel II categories. A system crash will result in substantial losses for a firm, but most of these losses would be categorized under the EDPM category. Basel II defines failed activity examples leading to loss events in the BDSF category as hardware, software, telecommunications, and utility outage.
- The Basel II level 1 External Fraud category has only two sub categories: (1) theft and fraud and (2) systems security. Examples of activities that are classified under the systems security subcategory are hacking damage and theft of information with monetary losses.
- The Basel II level 1 Internal Fraud category also has only two subcategories: (1) unauthorized activity and (2) theft and fraud. Examples of activities that are classified under unauthorized activity are intentionally not reporting transactions, unauthorized transaction type, and the intentional mismarking of positions.
- The Basel II level 1 Employment Practices and Workplace Safety {EPWS) category has three subcategories: (1) employee relations, (2) safe environment, and (3) diversity and discrimination.
- The last Basel II level 1 category for Op Risk loss events is Damage to Physical Assets (DPA). The only subcategory is disasters and other events.
Summarize the process of collecting and reporting internal operational loss data, including the selection of thresholds
- The foundation of an OpRisk framework is the internally created loss database. Any event that meets a firm’s definition of an operational risk event should be recorded in the loss event database and classified based on guidelines in the operational risk event policy. A minimum of five years of historical data is required to satisfy Basel II regulatory guidelines.
- Basel II requirements allow financial institutions to select a loss threshold for loss data collection. OpRisk managers should not set the threshold for collecting loss data too low (e.g., $0) if there are business units that have a very large number of smaller losses, because it would require a very high amount of reporting. OpRisk managers should also not just think in terms of large OpRisk threshold amounts.
- When quantifying capital requirements, Basel II does not allow recoveries of losses to be included in the calculation. Regulators require this rule because gross losses are always considered for capital calculations to provide a more realistic view of the potential of large losses that occur once every 1,000 years.
Issue of timeframe for recoveries in collecting loss data and
reporting expected operational losses
Another important issue to consider in the process of collecting loss data is the timeframe for recoveries. The financial crisis of 2007—2009 illustrated that the complexity of some loss events can lead to very long time horizons from the start of the loss event to the final closure. It is important for firms to have a policy in place for the processing of large long timeframe losses.
To help firms know what to report, the International Accounting Standards Board (IASB) prepared IAS37, which establishes guidelines on loss provisions or the reporting of expected operational losses after the financial crisis in 2007—2009. Three important requirements for the reporting of expected operational losses are as follows:
- Loss provisions are not recognized for future operating losses.
- Loss provisions are recognized for onerous contracts where the costs of fulfilling obligations exceed expected economic benefits.
- Loss provisions are only recognized for restructuring costs when a firm has a detailed restructuring plan in place.
The IAS37 report states that loss provisions of restructuring costs should not include provisions related to relocation of staff, marketing, equipment investments, or distribution investments. Loss provisions must be recognized on the balance sheet when the firm has a current obligation regarding a past loss event. Balance sheet reporting of loss events is required when the firm is likely to be obligated for a loss and it is possible to establish a reliable estimate of the amount of loss. Gains from the disposal of assets or expected reimbursements linked to the loss should not be used to reduce the total expected loss amount. Reimbursements can only be recognized as a separate asset.
Explain the use of a Risk Control Self-Assessment (RCSA) and key risk indicators (KRIs) in identifying, controlling, and assessing operational risk exposures
A risk control self-assessment (RCSA) requires the documentation of risks and provides a rating system and control identification process that is used as a foundation in the OpRisk framework. Once the RCSA is created, it is commonly performed every 12—18 months to assess the business unit’s operational risks.
The following four steps are commonly used in designing an RCSA program:
- Identify and assess risks associated with each business unit’s activities.
- Controls are then added to the RCSA program to mitigate risks identified for the firm. The manager also assesses any residual risk which often remains even after controls are in place.
- Risk metrics, such as key risk indicators or internal loss events, are used to measure the success of OpRisk initiatives and are linked to the RCSA program for review. These risk metrics would also include all available external data and risk benchmarks for operational risks.
- Control tests are performed to assess how effective the controls in place mitigate potential operational risks.
Key risk indicators (KRIs) are identified and used to quantify the quality of the control environment with respect to specific business unit processes. KRIs are used as indicators for the OpRisk framework in the same way that other quantitative measures are used in market and credit risk models. Even though KRIs may be costly to measure, they provide the best means for measuring and controlling OpRisk for the firm.
External data such as stock market indices and market interest rate levels are also used in RCSA frameworks.
Three common methods of gathering external data are: internal development, consortia, and vendors. Under the internal development method, the firm gathers and collates information from media such as news or magazines. This may be the least expensive method, but it may not be as accurate and has the potential to overlook large amounts of relevant data. The most popular consortium for banks is the Operational Riskdata eXchange Association (ORX), which contains large banks in the financial industry. While this consortium has a relatively low loss reporting threshold, there are often no details on the losses and therefore this data can only be used for measurement. There are a number of vendors who provide detailed analysis on losses that can be used for scenario analysis. However, the loss threshold for vendor data is often much higher and the information may not always be accurate.
Describe and assess the use of scenario analysis in managing operational risk
- Scenario analysis models are especially useful tools for estimating losses when loss experiences related to emerging risks are not available to the financial institution. Inputs to scenario analysis models are collected from external data, expert opinions, internal loss trends, or key risk indicators (KRIs).
- Studies suggest that most financial firms analyze between 50 and 100 scenarios on an annual basis.
- One of the challenges in scenario analysis is taking expert advice and quantifying this advice to reflect possible internal losses for the firm.
Biases and Challenges of Scenario Analysis
- One of the biggest challenges of scenario analysis is the fact that expert opinions are always subject to numerous possible biases. There is often disparity of opinions and knowledge regarding the amount and frequency of losses. Expert biases are difficult to avoid when conducting scenario analysis.
- Examples of possible biases are related to presentation, context, availability, anchoring, confidence, huddle, gaming, and inexpert opinion.
- Presentation bias occurs when the order that information is presented impacts the expert’s opinion or advice. Another similar type of bias is context bias. Context bias occurs when questions are framed in a way that influences the responses of those being questioned.
- Another set of biases are related to the lack of available information regarding loss data for a particular expert or for all experts. Availability bias is related to the expert’s experience in dealing with a specific event or loss risk. The availability bias can result in over or under estimating the frequency and amount of loss events. A similar bias is referred to as anchoring bias. Anchoring bias can occur if an expert limits the range of a loss estimate based on personal experiences or knowledge of prior loss events. The availability an expert has to information can also result in a confidence bias. The expert may over or under estimate the amount of risk for a particular loss event if there is limited information or knowledge available for the risk or the probability of occurrence.
- Expert opinions are often obtained in structured workshops that have a group setting. This group setting environment can lead to a number of biases. Huddle bias (also known as anxiety bias) refers to a situation described by behavioral scientists where individuals in a group setting tend to avoid conflicts and not express information that is unique because it results from different viewpoints or opinions. An example of a huddle bias would be a situation where junior experts do not voice their opinions in a structured workshop because they do not want to disagree in public with senior experts. Another concern for group environments is the possibility of gaming. Some experts may have ulterior motives for not participating or providing useful information in workshops. Another problem with workshop settings is the fact that top experts in the field may not be willing to join the workshop and prefer to work independently. The lack of top experts then attracts less experienced or junior experts who may have an inexpert opinion. These inexpert opinions can then lead to inaccurate estimates and poor scenario analysis models.
Compare the typical operational risk profiles of firms in different financial sectors
Basel II defines level 1 business units into the following categories:
- Trading and Sales,
- Corporate Finance,
- Retail Banking,
- Commercial Banking,
- Payment and Settlement,
- Agency Services,
- Asset Management, and
- Retail Brokerage.
Explain the role of operational risk governance and explain how a firms organizational structure can impact risk governance
There are four main organizational designs for integrating the OpRisk framework within the organization. Most large firms start at design 1 and progress to design 4 over time.
Design 1: Central Risk Function Coordinator
- The risk manager gathers all risk data and then reports directly to the Chief Executive Officer (CEO) or Board of Directors. Regulators believe there exists a conflict of interest for reporting risk data directly to management or stakeholders that are primarily concerned with maximizing profits. Thus, this design can only be successful if business units are responsive to the Central Risk function without being influenced by upper management who controls their compensation and evaluates their performance.
Design 2: Dotted Line or Matrix Reporting
This type of framework is only successful if there is a strong risk culture for each business unit that encourages collaboration with the Central Risk function. Furthermore, this dotted line structure is preferred when there is a culture of distrust of the Central Risk function based on some historical events.
Design 3: Solid Line Reporting
For larger firms that have centralized management, the solid line reporting is more popular. The solid line indicates that each business unit has a risk manager that reports directly to the Central Risk function. This design enables the Central Risk function to more effectively prioritize risk management objectives and goals for the entire firm. The solid line reporting also creates a more homogeneous risk culture for the entire organization.
Design 4: Strong Central Risk Management
Many large firms have evolved into a strong central risk management design either voluntarily or from regulatory pressure. Under this design, there is a Corporate Chief Risk Officer who is responsible for OpRisk management throughout the entire firm. The Central Risk Manager monitors OpRisk in all business units and reports directly to the CEO or Board of Directors. Regulators prefer this structure as it centralizes risk data which makes regulatory supervision easier for one direct line of risk management as opposed to numerous risk managers dispersed throughout various business units of the firm.
