PHP benchmarks across major CMSs and frameworks

Compare each platform only against itself

I treated every platform as its own test. WordPress against WordPress across PHP versions, Drupal against Drupal, Laravel against Laravel.

I picked one page per platform and wrote down what was on it, so each number stayed tied to a known page and no chart could read as a ranking between unrelated systems.

Build a page worth testing on all 20 platforms

I had to build 20 installations, each one rebuildable a year later and comparable to itself. Installing them was the quick part. Working out what belonged on each page took far longer.

Some platforms came with a demo site, so the work was choosing which page to test. Drupal came with Umami, a food magazine, and I tested an article page from it. Craft, Kirby, Symfony, concrete5 and PyroCMS all came with starter content I could use as installed.

The two stores needed their products brought in. For WooCommerce I installed the Storefront theme and imported the official sample products, then tested a single product page that loads an image, a description, an add-to-cart button, reviews, sidebar widgets, a related-products block and a pullout widget. For Easy Digital Downloads I installed Themedd and its demo content, then tested a download page with an image, a description, a purchase button and some category links. Two WordPress product pages, and one does several times the work of the other. This is why a leaderboard would have been useless.

Laravel, CodeIgniter and CakePHP open to a welcome screen, which is a sensible thing for a framework to do and an unhelpful one for a benchmark. There was no page until I built one. For Laravel I built a small app against a database of 1,000 posts, for CodeIgniter a news table of 1,000 rows, and for CakePHP a list of 400 generated posts. I published those schemas in the report so a reader could rebuild the test instead of trusting my numbers.

Then I wrote down every tested page element by element. That record is what let me install a newer version in 2022 and know I was measuring the same kind of page. No single platform was a disaster. The frameworks took the most fiddling, and that was about the same work as setting up the two stores with their products imported. I’m going through all of this to drive the point home that most of a benchmark happens before it runs.

Hold the environment still

Every setup ran on the same Google Cloud machine, an Intel Xeon with 30 cores and 120GB of RAM in an isolated container, on Ubuntu 20.04 with Nginx and MariaDB. I kept the application, the content, the tested page and every server setting fixed while switching PHP versions.

Full-page caching stayed off. A cached page never reaches PHP, and a benchmark where PHP does no work measures nothing.

OPcache stayed on. It stores precompiled bytecode in shared memory, so PHP stops reparsing scripts on every request. Every properly configured PHP server has it enabled, and turning it off would have measured a setup that doesn’t exist in production.

I raised opcache.max_accelerated_files from 4000 to 50000 so the larger platforms wouldn’t hit the cache limit. Every other value stayed at the recommended default, so the one deliberate change from stock is documented in the report.

Warm up first, then measure twice

I ran warm-ups on every setup before recording anything. A cold PHP process with an empty OPcache answers the first requests slowly, and measuring that tells you about startup. The warm-ups never appeared in the published reports, which is one of the things I’d change.

Then ApacheBench sent a fixed workload at 15 concurrent users and I recorded requests per second. I ran each benchmark several times to catch results that moved between runs, and from 2022 I tightened that into a stated rule, averaging only the top three.

Publish the results that argued against the article

Drupal 9.1 was the first one. It scored 363.06 req/sec on PHP 7.3, 328.08 on 7.4 and 304.07 on 8.0. Newer PHP was slower, on the flagship CMS of the report, in a series whose entire purpose was convincing people to upgrade.

So I ran it again. Then I tore the Drupal install down and built it from scratch. The third run gave the same answer, so it went into the report with a note saying the result was strange and that later Drupal versions might behave differently.

That one got written up as the surprise of 2021. Going back through the full dataset for this page, it wasn’t a surprise at all. A newer PHP version scored lower than the version before it in 20 places across the series, on 15 of the 34 setups. I published every one of those 20 numbers, with the percentages worked out so a reader could see how big each drop was.

Where a platform couldn’t run on a PHP version at all, I recorded it as unsupported and invented no number for it. 67 of the 210 cells in the series have no score for that reason. So when you see a blank in the chart, you know the platform couldn’t run that version. Leave those out and an incompatible combination starts looking like a slow one.

Work out every percentage for the reader

Every successful test needed the platform, its release, the PHP version, the tested page, the value and the unit before it went anywhere. I organized and labelled all of it, then handed it to Kinsta’s graphics team, who built the two combined charts from what I supplied.

I worked out every percentage in the article, so a reader doesn’t have to do arithmetic off a chart to find out what an upgrade buys them.