FX

Question 1

What is software verification?

Answer 1

ensure that the actual implementation meets the requirements set forth at the beginning

Question 2

The main thrust of Software Engineering II is on Testing. If you had to hazard a guess, which of the five phases above do you think Testing falls into?

Answer 2

If you answered Verification you would be correct. But if you think Verification is something that happens after all the other phases are complete, you would be incorrect. Other than the Requirements phase, testing can play a crucial role throughout the software development process.

Question 3

What is unit testing?

Answer 3

Unit Testing is when individual units of the software are tested…

Question 4

What is a failure?

Answer 4

Failure - a deviation from the expected behavior

These failures occur because there exists a “bug” in the code (a fault).

Question 5

What is the a fault

Answer 5

Fault: - an instance of incorrect code that can lead to a failure

A fault is introduced to the program when a programmer makes a mistake (an error).

Question 6

What is a error?

Answer 6

Error - a mistake that introduces a fault (e.g. typo and conceptual misunderstanding)

Question 7

Functional Testing

Answer 7

These types of tests are used to verify that the software meets the requirement specifications when it comes to functionality; does the software do the things it is expected to do?
Examples:

Does clicking “save” actually save the changes to the hard drive?
Does the search algorithm actually find the shortest path?
Does the isItPrime function correctly identify prime numbers?
Does the database correctly supply responses to the client?

Question 8

Non-functional Testing

Answer 8

These types of tests are used to verify that the software performs at the required levels. This can literally mean performance, but also usability, reliability, and robustness.
Examples

Does the webpage load in less than 2 seconds?
Does the interface provide enough clues for easy navigation?
Does the system successfully recover from a catastrophic failure?
Does the service support 10,000+ active users at a time?

Question 9

You are interested in learning how user friendly your software is.

Answer 9

Manual - This type of task really requires a human to interact with the software.

Question 10

Scenarios 2: You want to make sure your new changes didn’t break your existing features

Automated or Manual

Answer 10

Automated - Using pre-written tests that were known to pass, you can quickly verify that your changes didn’t cause any of them to fail. This is called Regression Testing.

Question 11

Scenarios 3: You want to test for game breaking bugs in a massively online multiplayer game.

Answer 11

Manual While automated tests could be used to test components of the game, testing the combined product really requires humans actively trying to break the game while it is running.

Question 12

Scenarios 4: You want to verify your system can handle a large range of possible inputs.

Answer 12

Automated - Instead of having a human try to input thousands of possible inputs, automated tests can randomly generate inputs in a fraction of the time. This is called Random Testing.

Question 13

When is black box testing useful?

Answer 13

Black Box Testing is when you write tests based purely on the description provided for the software (a.k.a. the specification).

Question 14

What are the limitations of black box testing?

Answer 14

It isn’t possible to test every possible input, so tests may miss logic branches/program paths untested (more on this in the White Box module)
There is no way to know why the failure occurs, just that the failure indicates a fault
Poorly written specifications can lead to inaccurate tests

Question 15

Random Testing - Advantages

Answer 15

Advantages
Quick to write
Can cover large portions of the input domain with very little code
Tests have no bias
Can potentially generate an input that no one considered

Question 16

Random Testing Disadvantages

Answer 16

Many random inputs could fall under the same “test case”, so become redundant and a waste of resources
May not test the tricky parts of the input domain like edge cases
Without any targeting, random inputs could easily miss glaring errors
Randomly selecting inputs for each test run could see some runs pass while others fail

Question 17

How does Partition Testing improve a testing suite?

Answer 17

partition Method is a way of reducing the potentially thousands of different combinations of testable features (i.e. test cases) down to a number that can be realistically implemented.

Question 18

When is white box testing useful?

Answer 18

White box testing focuses on verifying that all the code works as intended by trying to have as much of it run as possible during testing. This is done through something called coverage.

Definition Time

Code coverage is the extent to which a given test suite executes the source code of the software.

Question 19

Why is 100% statement coverage not enough?

Answer 19

All the non-executed statements are inside conditionals that aren’t triggered:

Question 20

Do you think 100% branch coverage means 100% statement coverage? Why?

Answer 20

100% branch coverage guarantees 100% statement coverage. In this way we can say branch coverage subsumes statement coverage.

Question 21

What is the input domain for my_abs?

def my_abs(val):
    if(val > 0):
        return val
    else
        return -val

Answer 21

Answer-The input domain consists of all real numbers!

Question 22

What is an oracle?

Answer 22

The oracle’s sole job is to watch for error states in the software and save the random inputs that generated those states for later inspection. . An oracle can be as simple as displaying error generating input to the screen or it could be an elaborate piece of software that generates formal bug reports.

Question 23

How can we make our random testers “smarter”?

Answer 23

With guided random testing, we use what is called a heuristic to help us make our random generation smarter

Question 24

How does random testing help with large input domains?

Answer 24

Can cover large portions of the input domain with very little code

Question 25

value in dynamically adding tests to a testing suite

Answer 25

rror recovery, an accurate number of tests run, and the ability to use asserts with random tests

Question 26

What software design approach is most directly associated with TDD?

Answer 26

Agile

Question 27

What does it mean to write the “bare minimum” of code?

Answer 27

Write the bare minimum of code to make the test pass

Question 28

When do we know we are done?

Answer 28

stop development after adding “enough” tests without triggering a new failure

Question 29

can you identify where in the process we refactored?

Answer 29

We refactored everytime we added test cases that broke our implementation. So when we changed return ‘1’ to return str(num), that was refactoring. We did this repeatedly as we went along and fixed our failing tests.

Question 30

Define Continuous Integration

Answer 30

The primary goal of CI is to have many developers working on the same codebase at the same time. CI as a set of guiding principles

Question 31

Continuous Integration Principles

Answer 31

Use a VCS like Git to maintain a central codebase.
Building the software should be automated and easily triggered. Once built, the software should be able to test itself against a provided test suite. Everyone needs to commit work to the shared codebase at least once a day. Every commit to master should be built and tested. Mandatory Code Review when requesting changes to be merged into the shared codebase

Question 32

What is a Fagan Inspection?

Answer 32

A process of trying to find defects in documents (such as source code or formal specifications) during various phases of the software development process

Planning - gathering participants and resources needed for the inspection process

Overview - meeting to discuss the important aspects of the project

Preparation - participants must review the material before the meeting

Inspection meeting - the actual moment the code is physically inspected for defects

Rework - any defects that are discovered are “fixed”

Follow-up - someone follows up with the developer to ensure the reworks have been done correctly

Question 33

How can I be a good Code Reviewer?

Answer 33

Read every line you are assigned
Put effort into your feedback
Think about how you would do it

Question 34

How can I be a good Code Reviewee?

Answer 34

Proofread
Make it worth it
Shine a light on it

Question 35

What is the difference between a Stub and a Mock?

Answer 35

Stubs

Stubs contain predefined data that is returned when called, but do not imitate behavior.

Mocks

Mocks simulate the behavior of a service and its actions can be verified.

Question 36

What does Unittest’s patch do?

Answer 36

which looks up an object in a given module and replaces that object with a Mock . Usually, you use patch() as a decorator or a context manager to provide a scope in which you will mock the target object.

Question 37

When is it appropriate to employ MC/DC?

Answer 37

Modified Condition/Decision Coverage’s purpose is to only test the important conditions to limit the number of test cases required.

Question 38

Which type of testing requires the input of the client/user?

Answer 38

Acceptance Testing

Question 39

Which type of testing requires the input of the client/user?

Answer 39

Acceptance Testing

Question 40

What are the differences between a Mutation-Based and Generation-Based Fuzzer?

Answer 40

Mutation-Based - This type of fuzzer starts by selecting a valid input. It then mutates the input in some random way and throws it at the software under test. The tester can achieve decent coverage by curating a varied list of valid inputs that would trigger different states in the software.

Question 41

Generation-Based -

Answer 41

As you may have guessed, generation fuzzers do not start with valid inputs. These fuzzers use some “knowledge” of the input domain to create the random inputs. This is similar to how we used “rules” to generate random inputs to test a password validator. The difference here is that in Exploration: Random Testing we were unit testing and now we are more concerned about how these random inputs affect the system.

Question 42

What are the differences between Load and Stress Testing?

Answer 42

Load Testing is where software is tested for performance under expected operating conditions.
Stress Testing is where software is tested to ensure it functions under extreme operating conditions.

Question 43

Fuzzing

Answer 43

aims to discover errors in programs by feeding them random inputs

Question 44

Fuzzer

Answer 44

is the tool/software used to perform Fuzzing

Question 45

What happens during the Requirements phase?

Answer 45

Interviews with stakeholders

Craft user stories and use cases

Question 46

What happens during the Design phase?

Answer 46

Layout the architecture of the program

Produce a design document

Question 47

Which of the following are examples of Functional Testing?

Answer 47

Regression Testing

Unit Testing

Question 48

Identify the 4 elements to a testing framework.

Answer 48

Test runner
Test case
Test fixture
Test suite

Question 49

Each unittest test case must include at least one ________.

Answer 49

Assertion

Question 50

Context: Mocks

Given the following code snippet, what goes in the blank?

from unittest.mock import Mock

mock = Mock()

# Set return_value
mock.abs.\_\_\_\_\_\_ = "7"

Answer 50

return_value

Question 51

Given the following code snippet, what is the output of the print statement?

mock = Mock()

mock. func(“Hello World”)
mock. func(“Greetings Planet”)

print(mock.func.call_args)

Answer 51

call(‘Greetings Planet’)

Question 52

Given the following snippet. If we wanted to use a mock to trigger an exception, what would we use in the blank?

Mock file reader to control its behavior

open = Mock()

def load_file():
# Does absolutely nothing other than raise the desired exception
    content = open('temp_file.txt')
    if content.length != 0:
        return content
    return None

class TestCase(unittest.TestCase):
    def test_read_file(self):
        # Test for IOError
        open.\_\_\_\_\_\_ = IOError
        # This is a context manager that allows us to test for exceptions
        with self.assertRaises(IOError):
            load_file()

Answer 52

side_effect

Question 53

Which of the following is called before each test case in a TestCase object?

Answer 53

setUp()

Question 54

Which of the following is called after all the test cases in a TestCase Object are run (not after each test)?

Answer 54

tearDownClass()