Tuesday 27 August 2013

Doing it wrong again - this time with Zend Optimizer plus

The guys at PHP have now committed to shipping Zend Optimizer Plus with future releases of PHP, I thought I'd have a play around with it.

tl;dr

While normally I run my PHP from Apache + mod_php, for the purposes of this exercise, it was easier for me to set up nginx / php-fpm using PHP 5.3.3/APC 3.1.2 and 5.5.1/Zend Opcache 7.0.2dev. All were compiled from source using default settings on a 32-bit PCLinuxOS 2012 distribution (kernel 2.6.38.8) on a dual AMD 4200 machine with 2Gb of memory.

Tests were run using ab on localhost, each test was run by first seeding the opcocde cache (ab -n 2 -c 2) then taking a measurement (ab -n 1000 -c 100).

For both APC and ZOP+ tests were run with an without checking for modified timestamps on PHP source files.

Final scores

My results showed Zend Optimizer Plus to be no faster than APC. Although the numbers are within the margin of error, if anything ZOP+ was slower.

Config Application Timestamps time per request (ms)
Php 5.3.3 + APC Dokuwiki yes 6.274
Php 5.3.3 + APC Dokuwiki no 6.271
Php 5.5.1, ZOP Dokuwiki yes 6.643
Php 5.5.1, ZOP Dokuwiki no 6.293
Php 5.3.3 + APC + mysql Wordpress yes 36.012
Php 5.3.3 + APC + mysql Wordpress no 35.868
Php 5.5.1, ZOP, mysqlnd Wordpress yes 35.905
Php 5.5.1, ZOP, mysqlnd Wordpress no 35.978
static HTML from Dokuwiki Dokuwiki n/a 0.171
static HTML from Wordpress Wordpress n/a 0.182

This seems to fly in the face of what is currently being reported elsewhere.

More stuff about my methodology

I wanted to create a reasonably realistic test, hence using 2 off the shelf Content Management Systems. Wordpress uses a MySQL backend for its data: and there is a further difference beteween the APC and ZOP+ configurations: the former uses libmysqlclient while the latter was built with MySQLnd (which meant I had to rewrite the database class to use the mysqli_ functions in place of mysql). The effects on performance are complex and tied to the level of concurrency but at 100 concurrent HTTP requests I was expecting this to be minimal. Dokuwiki, on the other hand, uses file based storage.

Other reviews

The PHP wiki page about the change links to a spreadsheet showing what look like impressive stats. The stats are reported as requests per second for various configurations.


Some more reviews:

https://managewp.com/boost-wordpress-performance-zend-optimizer - doesn't show response times / requests/second but does say that load and memory usage were lower implying greater capacity using ZOP+ compared with APC

http://halfelf.org/2013/trading-apc-for-zend/ reports a similar reduction in CPU and memory, but again no response times.

http://www.ricardclau.com/2013/03/apc-vs-zend-optimizer-benchmarks-with-symfony2/ results again in req/s, and showing an improvement of around 10-15%

http://massivescale.blogspot.co.uk/2013/06/php-55-zend-optimiser-opcache-vs-xcache.html compared ZOP+ and Xcache finding approx 15-20% improvement in req/s and similar reduction in response times with Joomla.

The optimizer bit(s)

PHP opcode caches have been around for a long time. ZOP+ brings something new: a code optimizer. Since it is still generating pcode it doesn't apply the CPU specific tweaks that a native code compiler does. Despite the 32 bits integer used to set the optimizer flag, only 6 flags are recognized by the optimizer (and the last pass only cleans up the debris left by the first 5). The optimizations are mostly substitutions - replacing PHP's builtin constants with literals, post increment with pre-increment, compile time type-juggling of built-in constants and such like. There is no inlining of functions in loops. No branch order prediction. Having dug through the code, I was not expecting it the optimizer to deliver revolutionary speed improvements.

And yet, Dimitry's spreadsheet shows 'ZF Test (ZF 1.5)' going from 158 req/s to 217 req/s!

I presume this refers to the Zend Framework. While this far from speedy I find it astonishing that the performance of the code should improve so much with a few relatively simple tweaks to the opcodes - it rather suggests that there is huge scope for optimizing the code by hand. Although I also note that the performance of 'Scrom (ZF App)' only improves by around 8%.

What am I doing wrong?

The consistent difference (apart from the opcode cache) in my experiment was using different versions of PHP - to a certain extent I'm not really comparing like-for-like. I can only assume that if I ran APC against PHP 5.5.1 and/or ZOP+ with PHP 5.3.3 I would see a very different story. However if you are seeking optimal performance at low load levels (rather than optimal capacity) then there seems to be little incentive to apply this upgrade.

There are anecdotal reports of stability issues with APC on PHP 5.4+; there may be sound technical and economic reasons why APC is not being actively maintained and for ZOP+ to be a better strategic choice.

A clear choice?

I can live without APC's support for user-data caching. But the elephant in the room is the fact that ZOP+ does not reclaim memory: if your code base is larger than the cache size, or the cache fills up with old verions of code, it forces a full flush and re-initialization. This should not be a problem for sites with a dedicated devops personnel managing releases to a small number of applications using a continuous deployment strategy. However for the rest of us there needs be a significant performance advantage with ZOP+ to make this a price worth paying.

Saturday 17 August 2013

Starting a new website - part 3

So the decision was made, I would stick with Dokuwiki and use PJAX for loading the pages.

A bit of coding and hey presto...www.scottishratclub.co.uk

(live site is sill running an incomplete version of the code - note to self - get the current version deployed)

In order to structure the Javacript changes nicely, keep everything tidy and fit in with Dokuwiki too, the functionality is split across a syntax plugin (for implementing widgets, including initializing PJAX, fixing the problems introduced by defering loading of the javascript and accomodating a strict Content Security Policy). This then places some constraints on how further widgets are implemented so it's really a framework (yeuch!). Anyway the plugin is called Jokuwiki

In order to use PJAX, the source page needs to be slightly modified (but it's JUST 5 LINES OF CODE!):


if ('true'!=$_SERVER['HTTP_X_PJAX']){
....top part of page
} ?><div id="pjax-container">
....stuff loaded via pjax
</div><?php
if ('true'!=$_SERVER['HTTP_X_PJAX']){
....bottom part of page
}

But just to make it real easy, I published a template too. Not the one I used on my website - but if anyone want it....let me know.

The impact of PJAX on performance is rather large:


Of course it had to be deployed to the site. So I dusted down pushsite and fired it up with a recipe for deploying the site. About 50 files loaded then I stopped getting responses from the remote system. I ran it again.....a further 20. The socket was still connected but nothing happening. Switching to passive mode didn't help. Adding throttling didn't help. I spent several hours battling with it and gave up. Same story the following day - so I logged a call with the service provider. The following day, they suggested using a different FTP server.....same problem. They said they'd get back to me.

Since I had no ssh access, I couldn't unpack a tarball from the shell - doing it via a PHP script invoked from the web would have meant I'd have to spend just as much time fixing the permissions as uploading the stuff by hand. But a bit of rummaging around in cpanel and I found that there was a back/restore option running over HTTP - so I download a backup, unpacked it, overwrote the backed-up website with my new site, packed it up and restored it onto the server. Job done.