Software systems and engineering

Question 1

What is the testing process?

Answer 1

Unit test- if parts don’t work by themselves they won’t work together
Integration (component) testing- do they work together
Validation and verification- it works but does it meet user’s functional requirements
Functional acceptance tests
Non-functional acceptance tests (scalability, security)
Usability testing, A/B testing

Question 2

What do you test?

Answer 2

Resource exhaustion, errors, recovery- does it work in the real world? does it run out of memory/disk space/CPU or network bandwidth?
Performance/stress testing- is it scalable?
Usability testing- does it work with real users under real conditions, is it intuitively clear to use?
Tests

Question 3

What are regression tests and test data?

Answer 3

Regression tests- make sure that new code to fix bugs doesn’t break old code
Test data- real world and synthetic (can’t get enough real, need certain statistical properties) to expose all bugs

Question 4

How do you plan testing?

Answer 4

Testing quadrant!
Q1: technology facing/supporting development- unit/component/deployment tests, automated
Q2: business facing/supporting development- functional tests, prototypes, simulations, acceptance tests (written by users before development starts listing acceptance criteria of requested application)
Q3: business facing/critiquing product- exploratory, usability, user acceptance, scenarios, alpha/beta, checking that spec is correct
Q4: technology facing/critiquing the product- performance and load tests, security
functional acceptance tests vs non-functional (capability, usability, security, modifiability, availability)

Question 5

Why live with bugs?

Answer 5

Bugs are expensive- direct;y through fixing and indirectly through damaged reputation and lost time
There will always be bugs- unknown unknowns, unexpected circumstances, new situation

Question 6

When should you test?

Answer 6

ASAP and frequently to reduce costs and defects

Question 7

Who should test?

Answer 7

Automate when possible so tedious parts done by machine- programmers gentle on their own code and get bored- BUT still need someone to define test cases and implement them in automated testing framework

Question 8

What should automated testing include?

Answer 8

Cover max no of test cases- unit tests, regression tests, UI testing
Test automation cloud for large distributed cloud services

Question 9

What tools can be used for testing?

Answer 9

xUnit- for unit tests
KIF- iOS UI testing framework, test scenarios
Selenium- automates browsers
Robotium- selenium for android
Static code analysers- check for code inconsistency eg FxCOP

Question 10

What are profiling tools?

Answer 10

Allow the analysis of software performance, e.g. tracking of memory allocation to spot leaks and inefficient use, looking for bottlenecks in need of optimisation

Question 11

What are analytics?

Answer 11

Collecting and analysing data once product is live with users to try and increase market performance, metrics include DAU (daily active users), conversion (how many users signed up/made purchase), engagement (hours spent), sales by territory, crashes, etc
Tools: fabric.io, Appsee

Question 12

What is usability testing?

Answer 12

Method

• define user interactions to be tested, how system is implemented (real product/prototype/simulator), baso test setup
• how user feedback captured: interviews, scoring cards
• session time
• how user interactions observed and analysed: video/screen capture, eye tracking

User group

• choose user group to provide representative set of product/service users
• segment users by characteristics e.g. demographic, age, gender, level of experience

Scenarios
-specific actions users asked to perform

Results presentation
report to include
-conclusions based on observations to identify e.g. common source of confusion
-qualitative user feedback
-quantitative results- rankings/scorings by user groups (charts/tables)

Question 13

Whats the best number of test users?

Answer 13

Nielson- 5
but that was with 5 users of same kind, would be better to have 5 users for each user segment?
Faulkner- 20
depends on diversity of target users and complexity of product

Question 14

When to user test?

Answer 14

ASAP
Make prototype with UI flow, main controls, layouts and data presentation
Option A: real app
real app and real feel BUT higher effort
Option B: digital prototyping platform
accessible, easy collaboration, easy release to users/feedback BUT needs graphic design of fake UI elements

Question 15

What is the software engineering process?

Answer 15

analysis
design- how goals achieved
implementation
building
testing
deployment
maintenance (80-90% of total cost of ownership)

Question 16

Who’s in a traditional team?

Answer 16

Architect- spec and high level design
Project manager- scheduling, tracking progress so on time and budget
Lead programmer
Programmer
Tester- test and validation procedures

Question 17

What are the pros and cons of the waterfall model?

Answer 17

Pros

• early clarification of system goals
• can charge for changes in requirements
• works well with management tools

Cons

• no iterations to adjust to changing requirements
• lot of time getting system right from start but right today doesn’t mean right tomorrow

Question 18

What is the spiral model?

Answer 18

Decide fixed no of iterations
Allows risk management
Iterative development with systematic aspects of waterfall
Iterative incremental refinement each time around spiral

Question 19

What are lightweight/agile methods?

Answer 19

Contrast to single cycle regulated methods
e.g. extreme programming (XP)
driving cost to reduce the cost of change
‘Stay aware. Adapt. Change’

Question 20

What does XP involve?

Answer 20

Software development framework
-Practices
-Values
Principles bridge the gap between them

Question 21

What are the XP values?

Answer 21

Communication
Courage
Feedback
Simplicity
Respect

Question 22

What are the XP principles?

Answer 22

Flow
Improvement
Reflection
Economics
Baby steps
Opportunity
Accepted responsibility
Redundancy
Diversity
Self-similarity

Failure
Mutual benefit

Humanity
Quality

Question 23

What are the XP practices?

Answer 23

Sit together to communicate with all our senses
Pair programming
Include the whole team
Continuous integration
Informative workspace
Energised work
Stories>requirements
Test-first programming

Weekly cycle and quarterly planning
Incremental design
Slack (minor tasks to be dropped)
10 minute build- automatically build whole system and run all tests in 10 mins

Question 24

What is continuous integration?

Answer 24

Introduced by Kent Beck as part of XP
Integration unpredictable so has to become part of every baby step- integrate and test changes after no more than a couple of hours
Involves source repository, an automated build, agreement of team

Question 25

Why does CI need a source repository?

Answer 25

Git/Github, Subversion, Mercurial, Perforce
Central place to store source code
Historical record of what’s been done over time
Facility to record a set of sources as a ‘release’
Ability to reconstruct a project as it was at any time in the past
Facility to create separate code branches and merge them later
PUSH EVERYTHING

Question 26

Why does CI need an automated build?

Answer 26

• Turning code into a running system can be waste of time and source of mistakes
• Automated environments e.g. Ant for Java, Nant/MSBuild for .Net
• Should include everything so can run using single command line on new machine and have everything
• Integrated development Environments (IDEs) such as Eclipse, Microsoft visual code, Xcode have build management process
• BASICALLY run via command line without IDE to automate, treat build scripts as codebase (test and refactor), be independent of IDE configurations, etc

Question 27

What are the CI key practices?

Answer 27

Maintain single source repository
Automate the build
Make build self-testing
Make it easy for anyone to get the latest executable
Automate deployment
Test in a clone of the production environment
Everyone commits to mainline everyday
Every commit should build the mainline on an integration machine
Everyone can see what’s happening
Keep build fast

Question 28

What is CI software?

Answer 28

Basic functionality to check repo for commits, if there are new ones run build script to compile software and run the tests to notify about the results
Components
-long running process that executes workflow at regular intervals
-view of results, success or failure
-web server to show list of builds that run and to show the report
Tools: Jenkins, TravisCI, CircleCI (being enriched to do continuous delivery too)

Question 29

What’s wrong with the traditional release candidate?

Answer 29

A change to the code may or may not be releasable, release candidate identified at the end of the process, testing significantly delayed which is expensive and stressful

Question 30

What is continuous delivery?

Answer 30

Alternative to traditional release candidate
Build software that’s always in production ready state
Achieved via CI and deployment pipeline to test whether the deployment is in a state to be delivered

Question 31

What is the deployment pipeline?

Answer 31

Automated software delivery system- from source control into users hands
1. developers commit changes to source repo- compile code, run tests, creates installers- if all ok assemble executable code into binaries and store them in an artefact repository
2. triggered by successful stage 1- longer running automated acceptance tests
3. pipeline branches to enable independent deployment of build in various environments (user acceptance tests, capacity tests, production).
Testers should see available release candidates, their status and comments so can deploy them in a relevant environment at the press of a button

Question 32

What are the pros and cons of continuous delivery?

Answer 32

+more frequent feedback from business colleagues, users, etc
+can get edge over competitors by releasing important features early
+better understanding users may improve product
+better reliability and stability
+automating repeated tasks saves time
-more intense collaboration between departments
-more automation investment
-more effort to deploy regularly

Question 33

What is the Cambridge cluster?

Answer 33

Technology is brains behind smartphones, magic behind Siri, making computing cool again

Question 34

What is innovation?

Answer 34

research is money into knowledge, innovation is knowledge into money

Question 35

What considerations need to be taken regarding innovation?

Answer 35

• the more innovation, the longer to bring to market
• timing important, is market ready for innovation, what is required to bring it to market?
• the innovation and market position both need to be protected

Question 36

What is software innovation?

Answer 36

• provide software systems which change the practices of their user communities
• innovative product displays novelty and utility
• product innovation at a point in time and with respect to a particular user segment
• start ups efficient at getting to new markets: teams of dedicated people, nothing to loose, no established business models to protect, no targets to meet

Question 37

Why start a software technology company?

Answer 37

high impact with few resources- scalable
can validate idea quicker
high gross profit margins
(although no work/life balance, binary outcome, stressful)

Question 38

Start-up team- why you?

Answer 38

Need drive, commitment, resources
Experience helps
Need engineers and business co-founders (fill gaps with advisory board, board of directors)

Question 39

Defining the idea- why this and why now?

Answer 39

ALL about execution, doesn’t have to be ‘the first of’
Need a good idea so understand
-product or service?
-established or new market? if established is it growing or saturated?
-does someone else do it already? are the successful? what can be learnt from their experience?
-how big is the total addressable market?
-how can the market be accessed?

Question 40

Is it a technology, a feature or a product?

Answer 40

e.g. blockchain, face detection, electric car

Question 41

How can the Gartner Curve (Hype Cycle) be used for market research?

Answer 41

shows global trends, predictions, key players

• innovation trigger
• peak of inflated expectations
• trough of disillusionment
• slope of enlightenment
• plateau of productivity

Question 42

What does ‘competitive landscape’ mean?

Answer 42

need to understand the competitors, their size and scale etc
(public companies- annual reports, private companies- annual accounts)
need to understand market trends, ie growth, proliferation, consolidation

Question 43

What are the combinations of product/market type?

Answer 43

1. new product, existing market- improving on well defined value proposition of competing offers, eg LED bulb
2. new product, existing market and re-segmenting as a low-cost proposition- improving on value for money, eg cheap android
3. new product, existing market and re-segmenting as a niche proposition- improving on well-defined value-proposition by focusing on that segment of the market, eg craft beer
4. new product, new market- customers don’t know they need it or what price they would pay, eg instant messaging
5. international idea arbitrage- implementing a proven successful value proposition in a new territory, adapting to cultural differences, eg LinkedIn with better privacy

Question 44

What is SWOT?

Answer 44

Strengths, weaknesses, opportunities, threats

Question 45

What is a business plan?

Answer 45

Two purposes:
internal- convince yourself and team
external- raise funds
show how business will be developed into a profitable company bc they need external capital to start
Present market analysis, prove the ability to execute and forecast the revenue and resources required
All plans include hockey stick forecast where profits over time rise

Question 46

What is the business model canvas?

Answer 46

Dynamic view of business model, relies on constant iterations through the cycle of testing a company’s hypothesis
-key partners, key activities, key resources, value propositions, customer relationships, channels, customer segments, cost structure, revenue streams

Question 47

How do we develop ideas via customer development?

Answer 47

prove the product/market fit

• who are the customers
• does the product solve their problem
• are they prepared to pay the price for the value they’re getting
• is business sustainable if selling product to customers at this price

Question 48

What is a lean start-up?

Answer 48

Build- unit and usability tests, CI, incremental deployment, free and open source, cloud computing, refactoring
Code
Measure- continuous deployment, usability tests
Data
Learn- 5 whys
Ideas
all in a loop faster