Cloud Native Automation for DevOps Performance Evaluation: Key Challenges and Potential Solutions

This thesis investigates how cloud native tools can be used to automatically collect and analyze DevOps performance using the four DORA metrics: deployment frequency, lead time for changes, change failure rate, and mean time to restore. The work focuses on what is required in practice to obtain these metrics from a Kubernetes-based CI/CD pipeline that uses GitHub Actions for continuous integration and Argo CD for continuous delivery. A prototype framework was implemented on a local single-node Kubernetes cluster. GitHub Actions builds, tests, scans, and publishes container images, while Argo CD continuously synchronizes updated manifests to the cluster using a pull-based GitOps workflow. A dedicated collector component receives deployment events from Argo CD and incident alerts from Alertmanager via webhooks, combines them with additional metadata retrieved from the container registry and GitHub, and exposes derived time-series metrics to Prometheus for visualization in Grafana. Lead time for changes is decomposed into four segments (preparation, tests, build and push, deployment) using image labels and commit metadata from both application and manifest repositories. The framework was evaluated using functional tests, where simulated deployments and failure scenarios were executed for multiple applications, and performance tests, where synthetic deployment events were generated at increasing rates while external APIs were replaced by a mock service. The functional tests showed that the collected metrics matched the expected deployment and incident behavior, and that the segmented lead times highlighted where delays occur in the pipeline. The performance tests indicated that the collector remained stable under the tested loads, with predictable resource usage. Overall, the results suggest that the proposed approach can provide automated and reasonably accurate DORA metrics in a Kubernetes environment. At the same time, the work shows that reliable metrics require combining telemetry from several components and introducing a dedicated collector that correlates deployment events and incident alerts with CI run data, registry image metadata, and Git commit metadata.