DevOpRET: continuous reliability testing in DevOps

To enter the production stage, in DevOps practices candidate software releases have to pass quality gates, where they are assessed to meet established target values for key indicators of interest. We believe software reliability should be an important such indicator, as it greatly contributes to the end-user satisfaction. We proposeDevOpRET, an approach for reliability testing as part of the acceptance testing stage in DevOps.DevOpRETrelies on operational-profile-based testing, a common reliability assessment technique.DevOpRETleverages usage and failure data monitored in operations to continuously refine its estimate. We evaluate accuracy and efficiency ofDevOpRETthrough controlled experiments with a real-world open source platform and with a microservice architectures benchmark. The results show thatDevOpRETprovides accurate and efficient estimates of the true reliability over subsequent DevOps cycles.