API / End-to-End Testing Flashcards
Do you have any experience with web service testing?
Yes, at Neat we used many distributed microservices that communicated through various protocols, mostly over HTTP through REST APIs.
At Comcast we had an IoT API that interacted with devices, and was consumed by a main API and exposed to our mobile clients.
We used REST because it’s lightweight and less standardized, and exposes more targeted functionality.
What are some benefits of a tool like SoapUI?
Open source
JIRA integration (JIRA exposes a REST API)
SOAP exposes the complete application using WSDL; REST exposes pieces via URI resources. (i.e., Twitter only exposes the Timeline for the particular user after they’ve authorized, etc.).
SoapUI supports Groovy and JavaScript.
It looks like a comprehensive tool for designing, packaging and running functional, performance, and security tests.
Do you have experience with end-to-end testing?
Yes — end-to-end testing is after all just testing the WHOLE feature.
Yes, for example we had a new feature where we created a new account type (Teflon). This account type had particular limitations (like metering of services in our web app — got a particular number of ‘credits’ to some services, or was restricted from using other services).
For example, they had unlimited data, but they could only “OCR/Parse” 5 items a month. They were also restricted from other services (like email-in, NeatScan, etc.).
Obviously, this had implications across all or most of our services. At user sign up at the e-commerce level, at the central web application level, at the database level.
Had to test with new users being created, making sure the functionality was correct.
Also had to test with existing users, after data migrations were applied (post-deploy). Had to test upgrade and downgrade workflows. Had to test account expirations (and whether those accounts would transition). Testing all from multiple clients (3rd party scanners, mobile devices, different flavors of desktop applications - both legacy OS-specific and the light-weight Node apps), across all the different services.
I could test this at the Black Box functional level, and we had to do that at first, just to get a feel for the different flows.
I also wrote scripts that ran on Rails console (to make sure ActiveRecord models were functioning properly), or mongo instances (to make sure migrations were correct, and that the data was being replicated across the primary-secondary sets).
But then we could write a series of API requests to POSTs and GETs to the various services, mocking the user-agent in the request headers to simulate multiple clients), and we would package these requests up into individual user scenarios and automate them as part of end-to-end regression.
How did you go about testing an API?
Read the documentation, if it’s available.
Start exploring it using some of my tools:
Charles Proxy, Fiddler (configured for mobile, desktop)
Postman, Cocoa Rest Client
cURL, HTTPie
I have some Ruby scripts that exercise an API via HTTParty and open-uri gems.
Start by consuming the API with happy path data (error codes, required params).
Try from a variety of clients (modify user-agent headers, etc.).
Try under a variety of network conditions using Network Link Conditioner (Hardware IO Tools for Xcode).
Security - try passing flags into POST methods, testing OAuth or Basic Authentication (username/password).
Testing whether the queries are indexed or not.
Stress testing, performance testing.
What tools did you use to automate the testing?
Ruby scripts using HTTParty and open-uri.
In Postman you can define a collection of API calls.
SoapUI
Authorization (basic auth, OAuth), cookie manager, etc.
There is a Tests tab where you can set up assertions on the responses you get from your API calls. These are written in JS.
Postman has a CLI (newman), where you can import a collection of API calls from Postman as JSON.
JMeter can also be useful.
What is a web service?
A web service allows a program to talk to an engine/server instead of using a browser. Services share business logic, data, and processes through various protocols (JSON, XML, over HTTP) (unlike GUI).
Example: user can go to maps.google.com and enter their address and get a map view. Or, I can write a web service that takes the user’s geolocation and interacts directly with the maps API and shows them the map without any user interaction.
What are some differences between SOAP and REST?
SOAP is a protocol, REST is an architectural style (SOA).
SOAP uses services interfaces to expose business logic, REST uses URI to expose business logic.
REST has less standards. SOAP defines its own security standards, whereas REST inherits from underlying transport protocols.
