Software systems and engineering Flashcards
What is the testing process?
Unit test- if parts don’t work by themselves they won’t work together
Integration (component) testing- do they work together
Validation and verification- it works but does it meet user’s functional requirements
Functional acceptance tests
Non-functional acceptance tests (scalability, security)
Usability testing, A/B testing
What do you test?
Resource exhaustion, errors, recovery- does it work in the real world? does it run out of memory/disk space/CPU or network bandwidth?
Performance/stress testing- is it scalable?
Usability testing- does it work with real users under real conditions, is it intuitively clear to use?
Tests
What are regression tests and test data?
Regression tests- make sure that new code to fix bugs doesn’t break old code
Test data- real world and synthetic (can’t get enough real, need certain statistical properties) to expose all bugs
How do you plan testing?
Testing quadrant!
Q1: technology facing/supporting development- unit/component/deployment tests, automated
Q2: business facing/supporting development- functional tests, prototypes, simulations, acceptance tests (written by users before development starts listing acceptance criteria of requested application)
Q3: business facing/critiquing product- exploratory, usability, user acceptance, scenarios, alpha/beta, checking that spec is correct
Q4: technology facing/critiquing the product- performance and load tests, security
functional acceptance tests vs non-functional (capability, usability, security, modifiability, availability)
Why live with bugs?
Bugs are expensive- direct;y through fixing and indirectly through damaged reputation and lost time
There will always be bugs- unknown unknowns, unexpected circumstances, new situation
When should you test?
ASAP and frequently to reduce costs and defects
Who should test?
Automate when possible so tedious parts done by machine- programmers gentle on their own code and get bored- BUT still need someone to define test cases and implement them in automated testing framework
What should automated testing include?
Cover max no of test cases- unit tests, regression tests, UI testing
Test automation cloud for large distributed cloud services
What tools can be used for testing?
xUnit- for unit tests
KIF- iOS UI testing framework, test scenarios
Selenium- automates browsers
Robotium- selenium for android
Static code analysers- check for code inconsistency eg FxCOP
What are profiling tools?
Allow the analysis of software performance, e.g. tracking of memory allocation to spot leaks and inefficient use, looking for bottlenecks in need of optimisation
What are analytics?
Collecting and analysing data once product is live with users to try and increase market performance, metrics include DAU (daily active users), conversion (how many users signed up/made purchase), engagement (hours spent), sales by territory, crashes, etc
Tools: fabric.io, Appsee
What is usability testing?
Method
- define user interactions to be tested, how system is implemented (real product/prototype/simulator), baso test setup
- how user feedback captured: interviews, scoring cards
- session time
- how user interactions observed and analysed: video/screen capture, eye tracking
User group
- choose user group to provide representative set of product/service users
- segment users by characteristics e.g. demographic, age, gender, level of experience
Scenarios
-specific actions users asked to perform
Results presentation
report to include
-conclusions based on observations to identify e.g. common source of confusion
-qualitative user feedback
-quantitative results- rankings/scorings by user groups (charts/tables)
Whats the best number of test users?
Nielson- 5
but that was with 5 users of same kind, would be better to have 5 users for each user segment?
Faulkner- 20
depends on diversity of target users and complexity of product
When to user test?
ASAP
Make prototype with UI flow, main controls, layouts and data presentation
Option A: real app
real app and real feel BUT higher effort
Option B: digital prototyping platform
accessible, easy collaboration, easy release to users/feedback BUT needs graphic design of fake UI elements
What is the software engineering process?
analysis design- how goals achieved implementation building testing deployment maintenance (80-90% of total cost of ownership)
Who’s in a traditional team?
Architect- spec and high level design Project manager- scheduling, tracking progress so on time and budget Lead programmer Programmer Tester- test and validation procedures
What are the pros and cons of the waterfall model?
Pros
- early clarification of system goals
- can charge for changes in requirements
- works well with management tools
Cons
- no iterations to adjust to changing requirements
- lot of time getting system right from start but right today doesn’t mean right tomorrow
What is the spiral model?
Decide fixed no of iterations
Allows risk management
Iterative development with systematic aspects of waterfall
Iterative incremental refinement each time around spiral
What are lightweight/agile methods?
Contrast to single cycle regulated methods
e.g. extreme programming (XP)
driving cost to reduce the cost of change
‘Stay aware. Adapt. Change’
What does XP involve?
Software development framework
-Practices
-Values
Principles bridge the gap between them
What are the XP values?
Communication Courage Feedback Simplicity Respect
What are the XP principles?
Flow Improvement Reflection Economics Baby steps Opportunity Accepted responsibility Redundancy Diversity Self-similarity
Failure
Mutual benefit
Humanity
Quality
What are the XP practices?
Sit together to communicate with all our senses Pair programming Include the whole team Continuous integration Informative workspace Energised work Stories>requirements Test-first programming
Weekly cycle and quarterly planning
Incremental design
Slack (minor tasks to be dropped)
10 minute build- automatically build whole system and run all tests in 10 mins
What is continuous integration?
Introduced by Kent Beck as part of XP
Integration unpredictable so has to become part of every baby step- integrate and test changes after no more than a couple of hours
Involves source repository, an automated build, agreement of team
Why does CI need a source repository?
Git/Github, Subversion, Mercurial, Perforce
Central place to store source code
Historical record of what’s been done over time
Facility to record a set of sources as a ‘release’
Ability to reconstruct a project as it was at any time in the past
Facility to create separate code branches and merge them later
PUSH EVERYTHING
Why does CI need an automated build?
- Turning code into a running system can be waste of time and source of mistakes
- Automated environments e.g. Ant for Java, Nant/MSBuild for .Net
- Should include everything so can run using single command line on new machine and have everything
- Integrated development Environments (IDEs) such as Eclipse, Microsoft visual code, Xcode have build management process
- BASICALLY run via command line without IDE to automate, treat build scripts as codebase (test and refactor), be independent of IDE configurations, etc
What are the CI key practices?
Maintain single source repository
Automate the build
Make build self-testing
Make it easy for anyone to get the latest executable
Automate deployment
Test in a clone of the production environment
Everyone commits to mainline everyday
Every commit should build the mainline on an integration machine
Everyone can see what’s happening
Keep build fast
What is CI software?
Basic functionality to check repo for commits, if there are new ones run build script to compile software and run the tests to notify about the results
Components
-long running process that executes workflow at regular intervals
-view of results, success or failure
-web server to show list of builds that run and to show the report
Tools: Jenkins, TravisCI, CircleCI (being enriched to do continuous delivery too)
What’s wrong with the traditional release candidate?
A change to the code may or may not be releasable, release candidate identified at the end of the process, testing significantly delayed which is expensive and stressful
What is continuous delivery?
Alternative to traditional release candidate
Build software that’s always in production ready state
Achieved via CI and deployment pipeline to test whether the deployment is in a state to be delivered
What is the deployment pipeline?
Automated software delivery system- from source control into users hands
1. developers commit changes to source repo- compile code, run tests, creates installers- if all ok assemble executable code into binaries and store them in an artefact repository
2. triggered by successful stage 1- longer running automated acceptance tests
3. pipeline branches to enable independent deployment of build in various environments (user acceptance tests, capacity tests, production).
Testers should see available release candidates, their status and comments so can deploy them in a relevant environment at the press of a button
What are the pros and cons of continuous delivery?
+more frequent feedback from business colleagues, users, etc
+can get edge over competitors by releasing important features early
+better understanding users may improve product
+better reliability and stability
+automating repeated tasks saves time
-more intense collaboration between departments
-more automation investment
-more effort to deploy regularly
What is the Cambridge cluster?
Technology is brains behind smartphones, magic behind Siri, making computing cool again
What is innovation?
research is money into knowledge, innovation is knowledge into money
What considerations need to be taken regarding innovation?
- the more innovation, the longer to bring to market
- timing important, is market ready for innovation, what is required to bring it to market?
- the innovation and market position both need to be protected
What is software innovation?
- provide software systems which change the practices of their user communities
- innovative product displays novelty and utility
- product innovation at a point in time and with respect to a particular user segment
- start ups efficient at getting to new markets: teams of dedicated people, nothing to loose, no established business models to protect, no targets to meet
Why start a software technology company?
high impact with few resources- scalable
can validate idea quicker
high gross profit margins
(although no work/life balance, binary outcome, stressful)
Start-up team- why you?
Need drive, commitment, resources
Experience helps
Need engineers and business co-founders (fill gaps with advisory board, board of directors)
Defining the idea- why this and why now?
ALL about execution, doesn’t have to be ‘the first of’
Need a good idea so understand
-product or service?
-established or new market? if established is it growing or saturated?
-does someone else do it already? are the successful? what can be learnt from their experience?
-how big is the total addressable market?
-how can the market be accessed?
Is it a technology, a feature or a product?
e.g. blockchain, face detection, electric car
How can the Gartner Curve (Hype Cycle) be used for market research?
shows global trends, predictions, key players
- innovation trigger
- peak of inflated expectations
- trough of disillusionment
- slope of enlightenment
- plateau of productivity
What does ‘competitive landscape’ mean?
need to understand the competitors, their size and scale etc
(public companies- annual reports, private companies- annual accounts)
need to understand market trends, ie growth, proliferation, consolidation
What are the combinations of product/market type?
- new product, existing market- improving on well defined value proposition of competing offers, eg LED bulb
- new product, existing market and re-segmenting as a low-cost proposition- improving on value for money, eg cheap android
- new product, existing market and re-segmenting as a niche proposition- improving on well-defined value-proposition by focusing on that segment of the market, eg craft beer
- new product, new market- customers don’t know they need it or what price they would pay, eg instant messaging
- international idea arbitrage- implementing a proven successful value proposition in a new territory, adapting to cultural differences, eg LinkedIn with better privacy
What is SWOT?
Strengths, weaknesses, opportunities, threats
What is a business plan?
Two purposes:
internal- convince yourself and team
external- raise funds
show how business will be developed into a profitable company bc they need external capital to start
Present market analysis, prove the ability to execute and forecast the revenue and resources required
All plans include hockey stick forecast where profits over time rise
What is the business model canvas?
Dynamic view of business model, relies on constant iterations through the cycle of testing a company’s hypothesis
-key partners, key activities, key resources, value propositions, customer relationships, channels, customer segments, cost structure, revenue streams
How do we develop ideas via customer development?
prove the product/market fit
- who are the customers
- does the product solve their problem
- are they prepared to pay the price for the value they’re getting
- is business sustainable if selling product to customers at this price
What is a lean start-up?
Build- unit and usability tests, CI, incremental deployment, free and open source, cloud computing, refactoring
Code
Measure- continuous deployment, usability tests
Data
Learn- 5 whys
Ideas
all in a loop faster
