Slow Performance with Slim PHP on Amazon Linux
Recently I have had some projects that involved creating RESTful APIs. I wanted something lightweight but also performant. After some research, I settled on using PHP with the Slim microframework. I was worried about slow performance, but reading a bunch of benchmarks it showed it could handle 2000+ req/sec with a basic test.
For most of the projects, I was able to utilize AWS ec2 instances. I decided to use the Amazon Linux AMI and installed Apache 2.4 with PHP 7. For the most part, I was happy with the setup and everything appeared to be fine. Responses were pretty snappy and they returned in a matter of milliseconds.
Slow performance issues arise
With one of the projects, we had some changes to implement in our API so we rolled out a new release. At the same time, we also received a higher amount of traffic. All of a sudden the response times became really slow. Of course, most people wanted to blame the code changes that were in the new release for the slow response time. Going through the changes, I couldn’t see anything that would really affect performance. To ease everyone’s mind though, we rolled back the updates.
The rollback didn’t clear up the issue, so I immediately started looking up performance tuning for PHP and Linux. I found a bunch of kernel tweaks that were recommended to allow more concurrent connections and I implemented them. This still did not fix the issue with slow performance. I was puzzled at what would be limiting the performance. We were not hitting the memory limit and the CPU was not being taxed heavily.
Troubleshooting the issue
So I decided to do some benchmark tests on a non-production node. Using the Apache Benchmark tool, I did a series of tests on our API. I was only able to get 17 req/sec. This was pitiful. My first reaction was to blame our code, but I resisted and instead read up more on the benchmarks of the Slim microframework. My setup was similar to the servers they used for the benchmarks. The only difference was that most of the people were using Ubuntu for their operating system. I didn’t think anything of that since I couldn’t imagine that the flavor of Linux would be the problem.
Surprise results
I then did benchmarks against a clean install of the Slim microframework to make sure it wasn’t our code. When I ran the tests it exhibited the slow performance issue and was only able to get around 250 req/sec. This didn’t match the 2000+ req/sec that I saw from other benchmarks. Again I was puzzled. It had me stumped for a little bit. The more I read on other’s benchmarks, the more the fact they were using Ubuntu stood out to me. So I decided to spin up an AWS instance with Ubuntu on the same size instance as we used before.
This time when I ran the tests I was able to get over 3000 req/sec. This gave me some relief because it proved that the framework was capable of handling that type of load. I was still confused on why the Amazon Linux instances weren’t able to get that type of performance. So I started looking up the problem and found other people had issues with PHP performance on CentOS. Amazon Linux is based on CentOS so I started to wonder if there was something different in the Linux Distros. I couldn’t find a smoking gun, so I ran some benchmarks on an instance running CentOS 7.
Performance issues with CentOS variants
The test results were very similar to what I was getting with Amazon Linux. This still had me wondering why this was the case? What exactly was so different for CentOS variants? This got me wondering if it was only an issue with Slim or if other frameworks had this issue. So I did a round of benchmarks for Silex and Lumen. The outcome was the same as with Slim. The CentOS variants had significantly slower response times.
The only conclusion I can come up with is that PHP has some type of performance defect on CentOS variants. I can’t even begin to wonder what is causing it. I reached out to AWS support to see if they had heard anyone else complain about it, but they just said they don’t support Slim and PHP instead of actually understanding the issue was trying to bring up.
This took me a long time to figure out, so hopefully, this post will help others that may be having the same issues.
The benchmark results for Slim PHP on various Linux distros
