Automated reliability testing for Kubernetes controllers
To use Sieve, developers need to port their controllers and provide end-to-end test cases (see getting started for more information). Sieve will automatically instrument the controller by intercepting the event handlers in
controller-runtime. Sieve runs in two stages. In the learning stage, Sieve will run a test case and identify promising points in an execution to inject faults. It does so by analyzing the sequence of events traced by the instrumented controller. The learning produces test plans that are then executed in the testing stage. A test plan tells Sieve of the type of fault and the point in the execution to inject the fault. These steps are detailed in https://github.com/sieve-project/sieve/blob/main/docs/port.md.
Key resultsSee the following URL for a list of all controllers we have tested and found bugs in: https://github.com/sieve-project/sieve/blob/main/docs/bugs.md. As of now, Sieve has been applied to ten different Kubernetes controllers and it has found more than 46 bugs that have severe consequences like security vulnerabilities, data loss, resource leaks and more.