ocPortal Tutorial: Optimising
Written by Chris Graham, ocProducts
ocPortal is very heavily optimised so that pages load as quickly as possible. This tutorial will provide information on techniques and issues for increasing throughput, so that your site may support more visitors.If you have need to be able to take particularly high load, considering getting the support of the experts.
Table of contents
Making PHP run faster
Speed can be approximately doubled if an "opcode cache" is installed as a PHP extension. These caches cache compiled PHP code so that PHP does not need to re-compile scripts on every page view. The 3 main solutions for this are:- Zend Optimizer (part of it is free but mostly it is commercial; made by the company behind PHP)
- eAccelerator (free, carried on from Turck mmcache)
- ionCube PHP Accelerator (free)
- APC (free)
We recommend using eAccelerator, for reasons that will become apparent.
Only one opcode cache can be used, and they often need to be manually compiled against the PHP version on the server and then installed as an extension. Such system administation is beyond the scope of this tutorial.
ocPortal cacheing
ocPortal provides many forms of cache, to allow the system to run as quickly as is possible.The ocPortal caches are:
- language cache: this removes the need for ocPortal to parse the .ini language files on each load
- template cache: this removes the need for ocPortal to parse the .tpl template files on each load
- Comcode page cache: this removes the need for ocPortal to parse the .txt Comcode pages on each load
- Comcode cache: this removes the need for ocPortal to parse Comcode whenever it is used
- block cache: this removes the need for many blocks to be fully executed whenever they are viewed – they are cached against the parameters they are called up with using a per-block tailored scheme
- theme image cache: this removes the need for ocPortal to search for theme image files whenever they are referenced by code (a code could translate to perhaps 10 different URLs, due to the flexibility of the system)
- values caches: this isn't a specific cache, but cacheing of values such as member post counts removes the need for ocPortal to recalculate them on-the-fly
- persistent cache: this is a very special cache that is explained in the next section
Technical note for programmers
ocPortal is not designed to "cache everything as it will be displayed" because the high dynamic nature of the system makes that impossible. Instead, Tempcode is typically cached against relevant parameters, which provides a "half way" between determined output and neutral data.
General tips
- If you are using a particularly visually complex block (e.g. deep pop-out menus) then use the 'quick_cache' block option on it if possible.
- Avoid making very long Comcode pages if they are just going to be static for all users. Ideally for usability pages should not be very long, but there are cases where you may choose for them to be (e.g. large glossaries). You can put {$,page hint: no_smart_conversion} into a Comcode page to prevent the WYSIWYG editor converting it to Comcode and otherwise use the Comcode 'html' tag to include pure HTML content.
- If you get a lot of 404 errors, it is best to make a static 404 page instead of using ocPortal's which can be relatively intensive. You activate a custom 404 page by putting a reference to it in the .htaccess file (our recommended.htaccess file does this for ocPortal's).
Persistent cache
The persistent cache is a cache that aims to store regularly-accessed data in-memory between requests, so that it does not actually need to be loaded from the database or re-calculated on each page load. This cache removes about 30% of the page load time, but most importantly, takes load away from the database, allowing the database to become less of a limiting factor in high throughput situations.The cache is implemented to work with either:
- eAccelerator/mmcache, which provides in-memory storage features as well as an opcode cache (hence why we recommend it as the opcode cache to use)
- memcache ('memcache' is the PHP extension for the 'memcached' server), which provides a heavyweight solution to memory sharing – it is not recommended that this be used, as it requires additional configuration and PHP memcached support (by the mainstream extension) is limited, leading ocPortal to not be able to make use of its advantage over eAccelerator (a network of memory servers)
- disk cache – whilst this does increase disk usage, it still provides a performance boost over not having a persistent cache. Don't use the disk cache is a production situation as it is not designed to work on busy servers (file access conflicts)
ocPortal will use a persistent cache if the following is in info.php:
Code
$SITE_INFO['use_mem_cache']='1';
ocPortal does not cache processed content in memory that has no special featured status, as this would only trade reliance on CPU for reliance on memory in a non-productive fashion.
We cannot guarantee that the persistent cache will operate correctly as we do not formally test it. If you encounter problems, however, we will be happy to help correct them.
Aggressive cacheing for bots
If you want to serve cached pages to bots, put a line like this into your info.php file:Code
$SITE_INFO['fast_spider_cache']=3;
The cache lifetime in this example would be 3 hours, but you can change it to whatever you require.
The cache files are saved under the persistant_cache directory.
Server
A faster and more dedicated server will make ocPortal run faster. This may seem obvious, but in the efforts of optimisation, is easily forgotten.CPU speed will be the most limiting factor for most websites: so this is the first thing that should be considered.
The Cloud
ocPortal runs perfectly on Rackspace Cloud Sites hosting (formerly named Mosso Cloud), which will spread requests across machines automatically, and handle data synching faultlessly (we have been told that behind-the-scenes there is database replication, but it is unnoticeable). You could also run on your own Amazon Cloud Server Instances, but there is probably no reason to – it's a lot more work and ocPortal is designed runs on commodity machines that require no custom configuration.Server farms (custom)
ocPortal has special (but unofficial) support for running on server farms:- ocPortal file changes are architectured so that any changes call up a synchronisation function which can be used to distribute filesystem changes across servers. As ocPortal requires no usage of FTP once installed, it presents the ideal managed solution for automated mirroring, once the initial network and synchronisation hook are created
- the ocPortal database system supports replication
In order to implement file change synchronisation, you need to create a simple PHP file in data_custom/sync_script.php that defines these functions:
PHP code
/**
* Provides a hook for file synchronisation between mirrored servers. Called after any file creation, deletion or edit.
*
* @param PATH File/directory name to sync on (may be full or relative path)
*/
function master__sync_file($filename)
{
// Implementation details up to the network administrators; might work via NFS, FTP, etc
}
/**
* Provides a hook for file-move synchronisation between mirrored servers. Called after any rename or move action.
*
* @param PATH File/directory name to move from (may be full or relative path)
* @param PATH File/directory name to move to (may be full or relative path)
*/
function master__sync_file_move($old,$new)
{
// Implementation details up to the network administrators; might work via NFS, FTP, etc
}
In order to implement replication, just change the 'db_site_host' and 'db_forums_host' values using config_editor.php (or editing info.php by hand in a text editor) so that they contain a comma-separated list of host names. The first host in the list must be the master server. It is assumed that each server has equivalent credentials and database naming. Due to the master server being a bottleneck, it will never be picked as a read-access server in the randomisation process, unless there is only one slave.
Round-Robin DNS could be used to choose a frontend server from the farm randomly, or some other form of load balancing such as one based on a non-cacheing proxy server.
Content Delivery Networks
You can have randomised content delivery networks used for your theme images and CSS files.To activate you would type a command like this into OcCLE:
Code
:set_value('cdn','example1.example.com,example2.example.com');
As you can see, it is a comma-separated list of domain names. These domain names must be serving your files from a directory structure that mirrors that of your normal domain precisely.
ocPortal will randomise what CDN's it uses, so parallisation can work more efficiently (this is the only reason for the comma-separation). Web browsers have a limit on how many parallel web requests may come from a single server, and this works around that.
DNS may be used to route the CDN servers based on geographic location.
Facebook's HipHop PHP (hphp)
Facebook released an alternative implementation of PHP that works as a compiler (compiles to an executable, via C++) instead of an interpreter. It more than doubles performance, and is set to improve further as it is developed.This is not suitable for most people for the following reasons:
- It requires server admin access
- The compiled app takes over the whole server
- It only runs on 64 bit systems running Linux (Fedora, CentOS, Ubuntu and Kubuntu are all the ones known to work at the time of writing)
- You can't make changes to your PHP code without doing a full recompile and redeployment (e.g. you can't just install and use new addons)
- It requires some advanced skills
ocPortal fully supports hphp since version 5.
The 'roadsend' addon (shipped by default) provides a build script for hphp (we used to use this with the Roadsend PHP compiler, but this is no longer maintained and was always rather complex and buggy).
To invoke a compile (we are assuming that hphp has been properly installed):
- Open a shell to the ocPortal directory
- Type ./hphp.sh (this is in the 'roadsend' addon)
- Make sure data_custom/fields.xml is deleted (at the time of writing there is a critical bug in hphp's XML support that any XML parsing will lead to request failure)
In order to support hphp (which imposes some language restrictions) we made some changes to ocPortal, and updated our coding standards. There's no guarantee that any individual non-bundled addon will meet the coding standards however.
We expect Facebook will improve hphp performance will increase over time, as the current 50% performance improvement seems low compared to what could be possible, especially if they can implement better 'type hinting' support (or some other type detection support, such as type database building via gathering at run-time); currently type detection (needed for the significant performance boosts) is very limited. hphp is an Open Source project, so anybody can contribute.
To support short URLs it is necessary to turn on the hidden 'ionic_on' option, which tells ocPortal that short URLs are supported even though Apache is not running. Of course, you also would need to enable the regular configuration options to enable short URLs also.