Before releasing Mitaka, we agreed on a tecnical definition of done1. This can always evolve to add more coverage, but this is what we currently test when deciding whether a release is ready from a technical perspective:
- Packstack all-in-one deployments testing the 3 upstream scenarios
- TripleO single contoller and single compute deployment validated with Tempest smoke tests
- TripleO three controller and single compute deployment with pacemaker validated using a Heat template based ping test
These same tests are used to validate our trunk repos2.
Update: This definition of done has been updated in recent releases and is now published in https://www.rdoproject.org/rdo/release-checklist/