Here is the new full-page test job after the conversion: Here’s a short example of our full-page test jobs before the conversion: To try out Knapsack Pro, all we did was follow the docs and afterward merge all separate Cypress jobs into one and set up some reporterOptions in Cypress to ensure that we’ll collect all of the test results as JUnit reports from all the parallel runs. Whenever we had an issue, we could just write to him, and we would figure out a solution together. The founder of Knapsack Pro, Artur Trzop, has been helping us proactively, sometimes noticing and notifying us that something went wrong with our pipeline even before we did. The dashboard also suggests if you’d benefit from less or more parallel runners as well. It comes with a dashboard to see the distribution of test subsets for each parallel run of CI runners, which is useful when splitting test suites or to see if something is going wrong with the runs. Its name comes from the “ knapsack problem”, a problem in combinatorial optimization, to figure out how to take things (test suites) with certain attributes (number of tests, the time it took on average to run, etc.) and determine how best to divide them to get the most value (all CI nodes are running tests from boot until shutdown, and they all finish at the same time). Knapsack Pro is a tool that helps split up test suites in a dynamic way across parallel CI runners to ensure that each runner finishes work simultaneously. List of Cypress test files, showing that tests are nested two levels: by module and by feature or area List of Cypress test Gitlab CI jobs, showing that there are many separate jobs for each of the test subfolders and that job runtimes vary from 53 seconds to 5 minutes Pipeline summary What is Knapsack Pro? But, as the tests’ size grew asymmetrically, we had to shift around which suites should run together continuously. Manual splitting of Cypress testsĪfter seeing that running tests in series takes extremely long, we tried to split some areas up and run them on separate Cypress runners, and this worked for a while. If our pipelines are fast, then we have to wait for less while they are running after applying some of the changes suggested during code reviews and getting hotfixes out to production if necessary. If our pipelines are slow, developers could forget that they had something running in the background and jump between tasks, or worse, somebody else merges code into the main branch, and the pipeline has to be run again… We want our pipelines and tests to run faster but not spend too much time nor money on doing it, so running each test on its own server is probably not the best way to go, not to mention the overhead of starting up the runners and checking out the source code. We use Cypress for end-to-end and integration testing with separate tests for both types of deployment. We use GitLab for code reviews and version control and GitLab CI to run our build pipelines. Most of the code is shared between these units, but there are some minor differences in design and major differences in flows/functionality, so they need to be tested separately. Our application is deployed in two distinct ways: as an npm package integrated into other modules within as a sidebar (called sidebar ), and as a standalone web application (called full-page ) which is dockerized and then deployed on GKE ( Google Kubernetes Engine) for our production environment and using GCR ( Google Cloud Run) for staging envs. Our team is responsible for the Help Center, where users can go to find articles and chat with support before, during, and after their trips. Knapsack Pro, Gitlab, and Cypress logos flying around in a speed frenzy What is our setup? We wanted to optimize everything we could within the pipeline a way of doing this was to parallelize our tests with Knapsack Pro to speed up our Cypress integration tests which would save us a sizable amount of time. When the runtime of our build pipeline had reached 20 minutes on average for each commit on each Merge Request, we knew we needed a solution to speed things up.
0 Comments
Leave a Reply. |