PMI-PBA Domain Area Test: Evaluation Flashcards
You have recently delivered a product to a customer that took you over 18 months to build. The product was built specifically for this customer under a contract with specific product specifications. Testing missed a defect and the product went into production causing customer complaints. The product meets the specifications in the contract as the contract only specified high-level product requirements and didn’t specify design details. What should you do first?
Investigate the issue and determine the root cause.
If there is a defect in the product, it is your responsibility to fix it even if the product meets the documented specifications. You must issue a change request to fix this defect but prior to that, you need to investigate the issue and determine the root cause. Only once the root cause is known, you will be able to issue a change request. [PMBOK® Guide 6th edition, page 292]
Regardless of the project lifecycle, scope and magnitude, a business analyst performs a number of tasks during the Solution Evaluation stage. All of the following are tasks that are typically performed during this stage EXCEPT:
Collecting and analyzing functional and non-functional solution requirements.
Collecting and analyzing functional and non-functional solution requirements are part of the Elicitation and Analysis stages. The rest of the tasks are all typically performed during the Solution Evaluation stage. [The PMI Guide to Business Analysis, page 278]
You are conducting the post-implementation solution evaluation of a recently released system. You can use all the following ways to demonstrate value delivery to the business EXCEPT:
Approach to risk management during development
You will assess the risk management approach to assess the efficiency of the project team and not the product. The rest of the choices can all be used to demonstrate value delivery by the system to the business. [The PMI Guide to Business Analysis, page 282]
A product team is developing a new information management system for your organization. The project is 75% complete and has delivered a minimum viable product. You are conducting the solution evaluation for the released project and find that the system has reached a point of diminishing returns. What should you do next?
Terminate the project and reallocate the remaining funds to other higher-priority projects.
A point of diminishing returns is reached when additional value that could be obtained from the solution does not justify the additional effort needed to achieve that value. At this point, it is recommended to “end early” and reallocate remaining funds to other higher-priority projects. [The PMI Guide to Business Analysis, page 279]
You are analyzing your business products to identify which products need to retire and which products require further investment. Which of the following tools would provide you with valuable information regarding your products to help make such decisions?
Product portfolio matrix
A Product portfolio matrix, also known as a growth-share matrix, is a market analysis quadrant diagram used by organizations to qualitatively analyze their products and product lines. One axis reflects market growth (or demand for a product) while the other reflects the market share of the organization. The matrix provides a quick visual way to evaluate which products are meeting or exceeding performance expectations in the marketplace. [The PMI Guide to Business Analysis, page 283]
Your team has recently released a new software application in the business. The application has been well-received by the business stakeholders. You want to conduct a formal solution evaluation at this stage. Which of the following will you use as a baseline to evaluate the application?
Acceptance criteria
Solution evaluation activities are performed to assess whether a solution has achieved the desired business results. Solution evaluation consists of the work done to analyze measurements obtained for the solution by comparing the actual results of acceptance testing to the expected or desired values, as defined by the acceptance criteria. [The PMI Guide to Business Analysis, page 278]
During a post-implementation solution evaluation exercise, you would use the documented business goals and objects as an input to provide:
Context for evaluating solution performance.
Business goals and objectives specify stated targets that the busines is seeking to achieve. They provide the context for evaluating solution performance because they are a measurable description of the expected business value. [The PMI Guide to Business Analysis, page 281]
Your product team has recently launched a new solution in the business. The benefits delivered by the solution are intangible in nature and extremely difficult and costly to measure. The internal auditor has been assigned to conduct the post-implementation solution evaluation. You have been notified that the auditor is using proxies for measuring business value delivered by the system. What should be your reaction?
Determine the effectiveness of the proxy values and support the internal auditor with the solution evaluation activities.
When measuring benefits is difficult or costly, organizations could use next-best-alternative ways to measure business value. Proxies for value might be used in these situations. You need to support the auditor with her assignments. [The PMI Guide to Business Analysis, page 281]
You are conducting a post-implementation solution evaluation of a product one year after its release. The product was expected to generate an ROI of 12%. Reviewing the actual financial performance, you find out that the product actual ROI is 9%. How would you document this finding in your final report?
The product could not meet its target financial performance.
The higher the ROI, the better. During the post-implementation review we consider the result of an actual versus expected cost-benefit analysis to assess the business value of the solution. In this case, the product could not meet its target financial performance. [The PMI Guide to Business Analysis, page 283]
Solution evaluation activities can be carried out either qualitatively or quantitatively. The choice of the appropriate approach varies from project to project. All of the following are examples of performing qualitative or coarsely quantitative solution evaluation EXCEPT:
Analyzing the data created by the solution.
Analyzing the results from surveys, focus groups, or the results of exploratory testing of functionality are examples of qualitative or coarsely quantitative evaluation activities. Other evaluation activities involve obtaining more precise quantitative measurements, such as directly looking at data from a solution. [The PMI Guide to Business Analysis, page 278]
A business analyst is responsible for carrying out Solution Evaluation processes for each product delivery project. Solution Evaluation comprises both short-term and long-term evaluations of the product/solution. Which of the following statements is correct in regard to the long-term evaluation of a product or solution?
Long-term evaluation focuses on evaluating the business value delivered by the product or solution.
Long-term evaluation focuses on evaluating the business value delivered by the product or solution. Product or solution evaluation during the acceptance testing is the short-term focus. The other two choices are irrelevant. [The PMI Guide to Business Analysis, page 275]
The products categorized as “high market share” and “low market growth rate” products on a growth-share matrix implies that these products:
Provide dependable income stream.
A Product portfolio matrix, also known as a growth-share matrix, is a market analysis quadrant diagram used by organizations to qualitatively analyze their products and product lines. One axis reflects market growth while the other reflects the market share of the organization. The matrix provides a quick visual way to evaluate which products are meeting or exceeding performance expectations in the marketplace. The products that provide the most significant benefits to the organization would be found in the upper left quadrant, because these are the products where the organization has a high market share in a market with a high growth rate. Those in the upper right quadrant are regarded as having good potential because, although they have a low market share, they are in a market that is continuing to grow. Those in the lower left quadrant, with a high market share in a low growth market, are considered a dependable income stream. [The PMI Guide to Business Analysis, page 284]
Some of the benefits and value of the solution recently released by your team are intangible in nature. What would be your approach regarding evaluation of these benefits?
Define measurements that provide indirect evidence of benefits.
Some of the benefits and value of the solution may seem to be intangible, and therefore, not possible to measure. For intangible benefits, it may be necessary to define measurements that provide indirect evidence that the benefits have been achieved. [The PMI Guide to Business Analysis, page 279]
You are leading an Agile project developing a new manufacturing control system for your firm. During the Evaluate Acceptance Results and Address Defects process, you have identified a number of performance issues in the system. What should you do next?
Add the defects to the backlog.
Adaptive projects do not have formal change control processes. On such projects, all product defects are added to the product backlog. [The PMI Guide to Business Analysis, page 294]
You are conducting the solution evaluation for a newly released information management system. During the evaluation you discovered new and changed requirements. What should you do to address this issue?
Issue a change request for solution refinement.
Evaluation of an implemented solution may be used to identify new or changed requirements, which may lead to solution refinement or new solutions. Doing nothing and ignoring the new and changed requirements is not recommended. The other two choices are extremes. [The PMI Guide to Business Analysis, page 279]
