Things have been quiet recently, so I wanted to make a very quick and very informal update. I don't really have time to proof-read it carefully, it's somewhere between "announcement" and "brain dump".
v10 development is going well, but it's also a slog at the moment, as some major rearchitecting work is happening. I'll step through some of that in this post. I still expect v10 to be out next year at some point, but I'm still not ready to give any firm commitment or release date . Our stable focus is still very much on v9 and maintaining v8, while we ensure v10 really does deliver in a big way.
We'll need to make another v9 patch release soon, as it's been a while now. That said, nobody seems to be reporting major problems, so it'll be more of a minor fix roundup.
PerformanceMost of our major performance ideas are now implemented.
These include various optimisations in various areas, some small, some large. My favourite is "output streaming" which allows ocPortal to start sending page output before the full page is generated. This better fills the Internet 'pipe', reduces latency, and allows the browser to detect and load dependent resources sooner. It makes things feel a lot snappier because you get a quicker response. Related to output streaming changes are the ability to disable the ocPortal 'pre-processor', which gives a significant boost (OTTOMH about 7%) – we no longer need the pre-processor as we've thought of faster ways to do what it used to be needed for.
I have big news: we have beaten the 0.1 second page generation threshold on my development machine on a regular LAMP install. IIRC v9 got to about 0.18 seconds, but it's hard to be scientific with these numbers as there are so many variables. The key takeaway is there are good speed boosts, made possible via some clever design changes or optimisations (we're long past the point where optimisation can result in "easy wins").
We have also updated ocPortal to work with the latest Facebook HHVM virtual machine, which is providing ever-increasing page-load speeds. We provide scripts and documentation. We have a way of running on their "strict typed" version of HHVM (called 'hack'), which is a great way of ensuring the integrity of our codebase, similar to what the ocProducts version of the traditional PHP can do.
ScalingWe've theoretically made ocPortal work on Google AppEngine, although currently I am waiting on someone from Google to get back to me with resolution of a server crash in the live environment, so practically it doesn't work yet. I've been doing a fair bit of back and forth with Google on various issues, so it's very early days for this really. Google sent me a T-shirt for helping them . Once the issues are fixed, I'm not sure how well it will work, but it's been a useful process because it's led to a number of ocPortal improvements and changes in approach that lead to ocPortal being able to scale better, including:
- Support for task queues. Long running tasks can now be put into a queue, which runs via CRON, and results are delivered via e-mail notification. This reduces the chance of uneven server load, or timed-out long running tasks.
- The ocPortal writable filesystem can be separated completely from the ocPortal application files. This is generally very useful for setting up ocPortal to run across multiple servers.
- Many improvements to the persistent cache.
- Reduction in queries and disk writes.
So, focus on what really useful things Google AppEngine has taught us, not whether it will be a viable thing to deploy ocPortal on (as that is very much up in the air).
Addon architecturev9 has two separate ways of dealing with addons. Bundled addons are done via "addon-registry hooks", and non-bundled ones are built from various separate indexes in our git repository. v10 now only has a single system, combining the best of both worlds. We have also added various automated tests to our test platform that ensure we keep things neat and tidy.
General cleanupsFew will care, but programmers do care. For example, do we write "pagelink" or "page_link" in our code. A whole load of things have been made consistent. When working on a large codebase, having everything perfectly consistent goes a long way to making other kinds of necessary complexity tolerable. Small things like this matter.
Sitemap, menus, icons, buttons, retina displays, design changesThe above title kind of explains why things are bogged down right now. All these things are blending into each other, so are being developed simultaneously.
The Sitemap is a piece of new critical infrastructure to v10, that has been partly sponsored (thanks Fletch and sholzy who have generously contributed). I'm putting in my own time to get it done, and it's proven a real monster. The Sitemap replaces various systems that already existed, with one new system that blends the features of all that came before it and it much more robust and flexible. It allows us to extend our functionality, while greatly simplifying things.
It's hard for me to quickly summarise what the Sitemap can do, but I'll give some cases:
- You can now generate drop-down menus that cover everything, from the zone level, to the entry-level. So, multiple-level dropdown menus that cover the whole site.
- You can configure things to define exactly what parts of the Sitemap you want.
- You can combine parts of the Sitemap with stored menus easily – e.g. add a forum link to your menu, and automatically show all the forums under it.
- The actual sitemap block, will now just be a menu type. You can therefore customise it via normal menu editing. You can splice things together, as described above.
- Things like the gallery list, or Wiki+ tree view, are no longer necessary – these are just instances of parts of the Sitemap being fed through a menu. This is what I mean by simplification, I've been able to erase various old ocPortal systems, hence simplifying the product.
Everything on the Sitemap has an icon. We have an icon pool of 323 icons, available in both 24x24 and 48x48 sizes. So, for example, the recommend site feature now has an icon, each type of virtual forum has an icon, the default Comcode pages have icons.
As discussed recently on the forum, buttons are being unified (View topic: Thoughts on buttons - ocPortal.com). These will draw from our new icon set, and is the other reason why we have such a large number of icons now. We have all the icons designed, but the actual button work isn't done yet.
I mentioned our icon pool is in 24x24 and 48x48. This roughly corresponds to the icon sized currently used for menu icons and "bigicons", but note two things:
- Industry-standard sizes
- There is a 2x multiple
Various other theme images have been updated to have 2x varieties.
I'm not sure if this will make the v10 cut yet, but it is definitely something being considered.
The plan is to make the default ocPortal use case very simple, and attractive. Menus will be auto-generated from the Sitemap (in fact, a single dropdown menu – goodbye having separate menus in different zones and a zone menu, unless you choose this). A simple single navigation around the website makes things a whole lot simpler for users, but of course you can still do exactly what you do now, have any number of menus in any number of zones. I also am considering making block layout automatic, by default – anything you have content for, will have a block automatically put into the layout for it. Then only power-users need to worry about customising block layout. To summarise – everything should be automatically linked together out of the box, with no need to consider site structure or navigation unless you plan to customise it. This should make ocPortal significantly simpler for new users.
Various other design changes are being considered. We're currently in the process of going through some of the less attractive areas of ocPortal, and designing out some improvements. Probably some will be put onto the tracker for future considerations, while others just implemented right away for quick wins.
Phew, I hope that gives people an indication of what is going on, and keeps people patient for quite a while longer .