July 2009 Monthly Meeting Summary
Test Automation Management via ROI and Scorecards - Rick VanKeuren, VeriSign Inc.
* Automation tracking and management via scorecards
* Determining automation Return on Investment
* Other issues in managing and tracking test automation
Bio: Rick has worked in the software industry for over 18 years. He has
extensive management and software engineering experience spanning a variety of roles but focused primarily on software development and testing. Rick is currently a QA Director with VeriSign where he manages a team of QA Engineers and Analysts responsible for the software quality assurance of the company's naming registration systems that include all com, net, edu, and other top level domain registrations. Rick holds a B.S. degree in Computer Science from Texas Christian University and an M.S. degree in Technology Management from George Mason University.
Took place on: Wed. July 8 2009 6:30 PM
Rick VanKeuren presented on some innovative and interesting
approaches to test automation management, and there was extensive discussions
at the meeting about challenges and experiences of other attendees and pros and cons
of various approaches. A 'scorecard' approach to tracking and managing
were presented, including tracking long term and short term targets/goals, quarterly
metrics, and semi annual resetting of targets. ROI reports and calculations were
discussed; such calculations were noted as providing visibility re QA efforts
in a language that senior management is often more familiar with.
Some points made/discussions included: that trends and relative values were more useful
than specific numbers in metrics; that target automation 'coverage' could vary
from project to project or business unit to business unit depending on a variety of
factors; that terms like 'coverage' and 'test case' etc can have varying meanings;
maintenance of automation suites can be a challenge to track and manage.
Some questions discussed also included:
- How 'successful' are your automation efforts?
- What does 'success' mean?
- What should your priorities be?
- How can management keep track of the progress of automation efforts?
- What kinds of metrics can be used to track/manage automation efforts?
- What kind of information is helpful in justifying and priortiziing automation efforts?
NoVaTAIG Home Page
Copyright 2009 Northern Virginia Test Automation Interest Group
Test Automation Interest Group