Author here. To provide some context, this article was written as a follow up to our previous benchmark (https://blog.checklyhq.com/puppeteer-vs-selenium-vs-playwrig...) after a lot of folks requested we add Cypress. Initially I was not sure this would be a good idea, as Cypress has a different focus and scope, but the sheer number of requests convinced me some folks out there might benefit. I hope this is helpful.
The scope of the benchmark is narrower than that, as stated in the article. It is not our aim to talk features, community, reliability (etc.) in this one article - that rather sounds like a book, and a long one at that.
It is indeed hard to have an all-encompassing comparison that will take everything into account. Unfortunately, to have precise answers in feasible time you have to narrow the scope. Here we looked at one parameter of a complex equation.
Author here - I'd be keen to go into further detail in a follow-up article if people are interested. Out of curiosity, what details would be most interesting for you about system and browser config?
I'd be curious for you to try performance of Puppeteer and Playwright running in a lightweight Alpine container using Chromium on Linux so that you get native support. Let me know if you need assistance.
This is highly opinionated, but I have been using and teaching how to use Selenium and Appium for a few years so that is unavoidable :) I would say the pros of Puppeteer & Playwright:
- generally higher speed
- higher reliability (specifically lower base false-positive rate)
These are mainly due to architectural choices (less moving parts between script and browser).
That being said, Selenium has been the open-source standard in cross-browser testing for a long time now, and is more polished and feature-rich. Also, multi-language support makes it an easier choice for non-JS teams. I would suggest a quick hands-on POC if you want to use these tools in a project.
The other answers pretty much sum this up already: the topic is not so straightforward as many think. I believe having a dedicated post on this might be beneficial, so I will put this on my list of articles for the basics on theheadless.dev (I am one of the initial contributors).
