Chapter 3 - Static and Dynamic Analysis

Question 1

What is Control Flow Analysis and what types of defects can it detect?

Answer 1

Control flow analysis is the static technique where the steps followed through a program are analyzed through the use of a control flow graph, usually with the use of a tool.

• loops that are badly designed (e.g., having multiple entry points or that do not terminate)
• ambiguous targets of function calls in certain languages
• incorrect sequencing of operations
• unreachable code
• uncalled functions

Question 2

What measure can Control Flow Analysis be used to generate for a component under test?

Answer 2

Cyclomatic complexity. The cyclomatic complexity is a positive integer which represents the number of independent paths in a strongly connected graph. Cyclomatic complexity is generally used as an indicator of the complexity of a component.

Question 3

What is Cyclomatic Complexity and what can it tell us?

Answer 3

Cyclomatic complexity is a positive integer which represents the number of independent paths in a strongly connected graph.

Cyclomatic complexity is generally used as an indicator of the complexity of a component. Any component that is measured with a higher complexity should be reviewed for possible refactoring, for example division into multiple components.

Question 4

Data flow analysis covers a variety of techniques which gather information about the use of variables in a system, investigating the lifecycle of each variable along a control flow path. What is one common technique to classify the use of a variable?

Answer 4

One common technique classifies the use of a variable as one of three atomic actions:
* when the variable is defined, declared, or initialized (e.g., x:=3)
* when the variable is used or read (e.g., if x > temp)
* when the variable is killed, destroyed, or goes out of scope (e.g., text_file_1.close, loop control variable (i) on exit from loop)

Question 5

Sequences of actions taken on variables that indicate potential anomalies include:
* definition followed by another definition or kill with no intervening use
* definition with no subsequent kill (e.g., leading to a possible memory leak for dynamically allocated variables)
* use or kill before definition
* use or kill after a kill

What static test technique is used to detect these types of anomalies?

Answer 5

Data Flow Analysis. The lifecycle of each variable along a control flow path is investigated, (i.e., where it is declared, defined, used, and destroyed), since potential anomalies can be identified if these actions are used out of sequence.

Question 6

What is a weakness of Data Flow Analysis as a static test technique?

Answer 6

The use of control flow paths to determine the sequence of actions for a variable can lead to the reporting of potential anomalies that cannot occur in practice.

For instance, static analysis tools cannot always identify if a control flow path is feasible, as some paths are only determined based on values assigned to variables at run time.

There is also a class of data flow analysis problems that are difficult for tools to
identify, when the analyzed data are part of data structures with dynamically assigned variables, such as records and arrays.

Static analysis tools also struggle with identifying potential data flow anomalies when variables are shared between concurrent threads of control in a program as the sequence of actions on data becomes difficult to predict.

Question 7

Why is static analysis so important to improving the maintainability of code, architecture, and websites?

Answer 7

Poorly written, uncommented, and unstructured code tends to be harder to maintain. It may require more effort for developers to locate and analyze defects in the code, and the modification of the code to correct a defect or add a feature is likely to result in further defects being introduced.

Static analysis is used to verify compliance with coding standards and guidelines; where non-compliant code is identified, it can be updated to improve its maintainability.

Question 8

Modular designs generally result in more maintainable code. True or false?

Answer 8

True.

Question 9

Static analysis tools support the development of modular code in several ways. One is by searching for repeated code.

How does this help?

Answer 9

These sections of code may be candidates for refactoring into components (although the runtime overhead imposed by component calls may be an issue for real-time systems).

Question 10

Static analysis tools support the development of modular code in several ways. One is by generating metrics which are valuable indicators of code modularization. These include measures of coupling and cohesion.

How does this help?

Answer 10

A system that has good maintainability is more likely to have a low measure of coupling (the degree to which components rely on each other during execution) and a high measure of cohesion (the degree to which a component is self-contained and focused on a single task).

Question 11

What are some security vulnerabilities that status testing analysis tools can identify in websites?

Answer 11

• code injection
• cookie security
• cross-site scripting
• resource tampering
• SQL code injection

Question 12

Why is it recommended to start dynamic analysis early in the project?

Answer 12

Failures that are not immediately reproducible (intermittent) can have significant consequences on the testing effort and on the ability to release or productively use software. Such failures may be caused by memory or resource leaks, incorrect use of pointers and other corruptions. Due to the nature of these failures, which may include the gradual worsening of system performance or even system crashes, these failures often are the most expensive failures to find and to correct.

Question 13

What type of analysis may be applied to accomplish the following?
* Prevent failures from occurring by detecting memory leaks and wild pointers;
* Analyze system failures which cannot easily be reproduced;
* Evaluate network behavior;
* Improve system performance by using code profilers to provide information on runtime system behavior which can be used to make informed changes.

Answer 13

Dynamic Analysis.

Dynamic analysis is used to detect failures where the symptoms are only visible when the code is executed. For example, the possibility of memory leaks may be detectable by static analysis (finding code that allocates but never frees memory), but a memory leak is readily apparent with dynamic analysis.

Question 14

True or False: Dynamic Analysis requires that the Technical Test Analyst has the needed technical and analytical skills to review logs and diagnose the cause of a failure.

Answer 14

False.
Dynamic analysis tools can be used even if the Technical Test Analyst has minimal technical skills; the tools used usually create comprehensive logs which can be analyzed by those with the needed technical and analytical skills.

Question 15

What are memory leaks and why do they occur?

Answer 15

A memory leak occurs when areas of memory (RAM) are allocated to a program but are not subsequently released when no longer needed. This memory area is not available for re-use. When this occurs frequently or in low memory situations, the program may run out of usable memory.

Question 16

True or False: Memory leaks typically cause problems after some time. When software is newly installed, or when the system is restarted, memory is reallocated, and so memory leaks are not noticeable.

Answer 16

True.
Memory leaks typically cause problems after some time – when a significant amount of memory has leaked and become unavailable. When software is newly installed, or when the system is restarted, memory is reallocated, and so memory leaks are not noticeable; testing is an example where frequent memory allocation can prevent the detection of memory leaks. For these reasons, the negative effects of memory leaks may first be noticed when the program is in production.

Question 17

What is the primary symptom of a memory leak?

Answer 17

The primary symptom of a memory leak is a steadily worsening system response time which may ultimately result in system failure. While such failures may be resolved by re-starting (re-booting) the system, this may not always be practical or even possible for some systems.

Question 18

Wild pointers within a program are pointers that are no longer accurate and must not be used. For example, a wild pointer may have “lost” the object or function to which it should be pointing or it does not point to the area of memory intended (e.g., it points to an area that is beyond the allocated boundaries of an array). When a program uses wild pointers, what are some of the consequences that may occur?

Answer 18

• The program may perform as expected. This may be the case where the wild pointer accesses memory which is currently not used by the program and is notionally “free” and/or contains a reasonable value.
• The program may crash. In this case the wild pointer may have caused a part of the memory to be incorrectly used which is critical to
  the running of the program (e.g., the operating system).
• The program does not function correctly because objects required by the program cannot be accessed. Under these conditions the program may continue to function, although an error message may be issued.
• Data in the memory location may be corrupted by the pointer and incorrect values subsequently used (this may also represent a security threat).

Question 19

With the dynamic analysis of program performance, tools help identify performance efficiency bottlenecks and generate a wide range of performance metrics which can be used by the developer to
tune the system performance. For example, information can be provided about the number of times a component is called during execution. Why is this useful?

Answer 19

Components which are frequently called would be likely candidates for performance enhancement. Often the Pareto rule holds here: a program spends a disproportionate part (80%) of its run time in a small number (20%) of components.