Mock Interview I

Question 1

What is health?

Answer 1

Health refers to the overall stability of an area relating to the bugs/issues found within it. If there are more bugs or bugs with higher severity, that would be an indication of an unhealthy area.

Question 2

What shows coverage?

Answer 2

Coverage refers to the amount of comprehensive testing for an area or feature, and shows where testers have focused on testing. Telemetry can assist in tracking the current amount of coverage.

Question 3

How would you determine the health of an area?

Answer 3

Bugs are the largest indicator of area health. Through the number of bugs, rate of discovery, and severity, we are able to interpret the current stability of an area.

Another thing to consider is your Mean Time to Failure. How long does a player test the title before they encounter heavy issues like crashes? Over x amount of time, how many times are they hitting crashes or P1/P2 issues?

Last thing would be to understand area according to the title’s current stage of development. Is the area WIP? Is the product in sustainment?

Question 4

How would you test the health of an area depending on the production timeline?

Answer 4

While early in the development cycle and with new features being implemented, scripted or white box testing should be used to test the core features of the game.

As development slows/reaches sustainment/if scripted testing is complete, ad-hoc or grey/black box testing can be used to highlight issues occurring from player perspective.

Question 5

How would you test an area if you were told it was Code Complete?

Answer 5

An area at Code Complete should be tested using ad-hoc as most, if not all, functionality has been implemented.

Question 6

What is Scripted Testing?

Answer 6

Scripted testing is testing completed under a set of conditions, usually in the form of test cases containing specific repro steps.

Question 7

How is Scripted Testing used?

Answer 7

Scripted testing is used to test functionality of an area.

Question 8

What is Non-Scripted Testing?

Answer 8

Non-scripted testing is testing around variables for issues that a player might encounter during normal gameplay. Often times this would include halo-testing and ad-hoc.

Question 9

How is Non-Scripted Testing used?

Answer 9

Non-scripted testing is used to identify issues with an area that might appear during normal gameplay from the perspective of a player.

Question 10

What would you change about your project?

Answer 10

While onboarding Robert, I had realized that one of the things that may make it easier to absorb the amount of information presented would be to attempt to present it differently - maybe as an interactive experience.

Question 11

What do you think is a critical skill to have?

Answer 11

Effective communication.

Question 12

How would you handle a belligerent tester?

Answer 12

Keep communication flowing and open with the lead. STE’s are not the authority in discipline and there are times where a lead should step in to assist with the situation, however you can keep others in the loop of anything that may be occurring.

Question 13

New tester asks what the main requirements of the job are. What do you tell them?

Answer 13

1. A passion for video games.
2. The ability to work as part of a team.
3. Curiosity and drive to keep learning.

Question 14

If you see a tester is struggling how would you help them out?

Answer 14

Observe the tester’s behavior and see how they approach a given task. Depending on the nature of their struggles, we can work with them directly to better understand the issue.

While we can phrase descriptions in different ways or further explaining the purpose of completing the task they are working on, we can also think of different testing methodologies to provide to the tester. If their struggle pertains to a specific area, we can assist in finding informative documentation or guide them to the correct connection to seek the information themselves.

Question 15

How would you train a new hire vs someone who has been here a while and just transferred to your team?

Answer 15

Regardless of if they are a new hire or not, the first thing I would focus on is increasing their familiarity with the title through gameplay/ad-hoc. Familiarity with the title is needed before they can test it. This will also include any team onboarding processes.

If they are a new tester, we can discuss more about what makes a bug as well as the purpose behind scripted and non-scripted testing. Along with the title, we can also work on getting used to the tools we use.

If the tester is a transfer from another project, we can show them our processes and discuss where we are within our development process. After that, we can gauge their knowledge and compare what they know to what we do on this project.

Question 16

If you trained a new tester in a few weeks and they aren’t performing to your standards what would you do?

Answer 16

The first thing I would do is to check my standards. Are my expectations in line with their TA level? Are they continuing to lack knowledge for resources I am able to provide to them? Am I offering enough guidance for them to build their own independence?

The second thing I would do is to sit down and discuss any thoughts that they might be thinking about. Have a conversation regarding what their side or perspective is vs. mine.

The third thing would be to observe the tasking they are taking on or have on their plate. Are they encountering issues with understanding the task, or are they not producing results in a timely fashion? If something is buggable or not as intended, are they able to notice it?

Depending on the timeframe and their progress from time A to time B, a lead would eventually need to step in to connect with the tester.

Question 17

What if you expect a certain amount of bugs and your tester writes one with problems like: my internet/power went out. What would you do?

Answer 17

Check to see if the tester was able to escalate their hardware issue effectively. Get in touch with the tester to see if an IT ticket needs to be submitted. Would this be a hardware or PC component-related issue?

Question 18

How do you gauge a testers performance? What data would you use to back that up?

Answer 18

We can gauge a tester’s performance by examining their bugs for counts and resolutions, looking into tasks to check their time taken to completion, and monitoring their communication within different channels to see if they are informing you with what they are doing or for any issues that arise.

Question 19

If a tester is doing really bad but previously were the rock star tester how would you handle it?

Answer 19

Observe the tester and gather data that supports their recent decline in performance. Ask to speak with the tester privately wherever they are comfortable doing so, this can be in direct messages in Teams, on a call, or in an office.

During the conversation explain the observation of decreased performance and ask general questions to see if you can find a cause to the change, such as a shift in mood, uncomfortable interaction with a coworker, or if something else has occurred that influences their work. Then work with them to find a solution to their issue, provided they are also interested in working through it.

If the tester is not comfortable discussing a specific issue with me directly, offer them other ways to communicate their concerns. This can include contacting leads or submitting an anonymous form to a party with the ability to act on the information.

Question 20

What are the traits of a good tester?

Answer 20

Adaptive, flexible, able to communicate clearly and when needed, inquisitive, ability to follow direction, and observant.

Question 21

In what situations should the lead step in when dealing with a tester?

Answer 21

If the tester continues to struggle and improvement is not visible after a substantial period of time, using communication that is harsh or combative, or is not responsive to messages/updates.

Question 22

What makes a good STE?

Answer 22

Communication ability (spoken and written), people management skills, analytical ability, and a strong understanding of the skills used to be a successful TA.

Question 23

What’s different for STE?

Answer 23

Outside of being in build and testing things along with testers, they also create collateral for test passes and are able to communicate health/coverage for their area. While questions regarding current health and coverage might not be related to how it’ s doing, they will need to make educated guesses on what the plans are for the future of the title based on where there area is sitting.

Question 24

What is the purpose of a report?

Answer 24

Reports provide a high-level break down of information relating to an area that is easy to understand, while giving full context of what is happening in an area to developers/respective contacts.

Question 25

What is the difference between what a producer and developer would want from a report?

Answer 25

A producer might want to see information relating to blocking issues, plans for test and timelines, and current schedule. If the schedule for tasking is longer than expected, this may also include another plan to fix it or alleviate the problem.

A developer would want to know the current work they need to do, including information of area health and a list of what needs to be worked on.

Question 26

What is an inside report?

Answer 26

An inside report is a report that communicates area coverage information, tester performance metrics, and project updates to contacts within the company.

Question 27

What is automation?

Answer 27

Automation is testing conducted on the title through the use of scripts or pre-programmed instructions.

Question 28

What would you automate?

Answer 28

Extremely repetitive tasks to test function of the title, soak tests, and testing to gather performance data.

Question 29

In what situation would you not use automation?

Answer 29

Testing where human judgement or variable is required. While you can create automation based on runs of specific tasks, it would inefficient and not useful to replicate non-scripted testing.

Question 30

What is Telemetry?

Answer 30

Telemetry is recorded material that relates to the operation of software as it runs. This can include a game’s interaction with various types of hardware as well as its communication with servers.

Question 31

What is Telemetry?

Answer 31

Telemetry is recorded material that relates to the operation of software as it runs. This can include a game’s interaction with various types of hardware as well as its communication with servers.