March 2014 Monthly Meeting Summary
Test Automation in Continuous Delivery - roundtable discussion
Potential subtopics included:
* Continuous Delivery (CD) vs Continuous Integration (CI) vs 'Other'
* How does a CD approach affect test automation?
* Automation considerations in each part of the delivery pipeline
* Configuration items, feature toggles, etc and test automation impacts/considerations
* In CD should all testing be automated or not - and what are the impacts/considerations?
* Tools for test automation in CD
* Experience reports
Meeting took place on: Wed. March 19 2014 6:30 PM
- For those unfamiliar with Continuous Delivery concepts, there were initial short presentations by Gopal Addada and Estaban Sancho
discussing the basic concepts and vocabulary
- Some resources suggested by Gopal included:
* a 2-minute YouTube video on Intro to Continuous Delivery
* Old but still interesting CD tools listing
* Helpful website: step by step lessons in implementing CD using AWS & Jenkins http://www.devopscloud.com/
* 24-minute YouTube video on Transformation to Continuous Delivery at Ancestry
- There was some discussion about the challenges in implementing CD in an organization, and it was mentioned that
the non-technical aspects such as organizational and individual change were often a large part of the challenges
- One of the methods of influencing change that was emphasized was by example - such as having a small group have a successful
initial implementation before trying to change the entire organization.
- Test automation at different points in the delivery pipeline included static tests such as those for code analysis and security static tests.
- Manual testing can still be a part of the CD approach and there was discussion of doing manual testing in production systems via 'feature toggles'
so that testing could be done without exposing new functionality to the entire production environment.
- Unit testing was discussed as the responsbility of dev and only including testing that did not need a deployment
- There was some discussion of the term 'Component testing' for mid-level testing and it included things like API testing, command line/ no-UI.
It was mentioned that sometimes these tests were the responsibility of 'automation specialists'
- The 'high end' of test automation was mentioned as UI-type testing such as testing with Selenium
- There was discussion of the automation level terminology used in the 'How Google Tests Software' book - small/medium/large/enormous,
based on how long a test too to run, if it had any dependencies, etc
- There was some discussion of CD tools. It was noted that while Jenkins is oriented to Continuous Integration, open source tool Go from
Thoughtworks is oriented to Continuous Delivery.
- In discussing deployments it was mentioned that there are really 2 parts to deployments - provisioning + deploying. Puppet and chef
were discussed and comments were made that they had limitations for use in CD and needed Fabric or something similar. Ansible was
mentioned as a good CD tool - according to the readme on Github it is a "radically simple configuration-management, application deployment,
task-execution, and multinode orchestration engine."
- Some challenging aspects of CD were discussed, including the following points:
* There was a comment that schema changes can be a challenge to handle in CD and require careful consideration.
* It was suggested that a CD pipeline be for a well-defined project, and that dependencies across pipelines can be a challenge.
NoVaTAIG Home Page
Copyright 2014 Northern Virginia Test Automation Interest Group
Test Automation Interest Group