HTML Logo by World Wide Web Consortium (www.w3.org). Click to learn more about our commitment to accessibility and standards.

Moving forward with Composr

ocPortal has been relaunched as Composr CMS, which is now in beta. ocPortal 9 will be superseded by Composr 10.

Head over to compo.sr for our new site, and to our migration roadmap. Existing ocPortal member accounts have been mirrored.


Where's the load?

Login / Search

 [ Join | More ]
 Add topic 
Posted
Rating:
#21128 (In Topic #5099)
Avatar

Well-settled

I'm, writing a quick topic in the hopes that Chris or one of the otherstaff can explain which bits of the system use the most systemresources.

On my current 2.6.x install, I'm running PHP 5. My server load is low(0.2 or something similar). It has quad Intel Xeon 2.6ghz processorsand a heap of RAM. When I try and enter the Admin area it willsporadically timeout and just display a white screen. This actually happens throughout the admin area and is a little infuriatingwhen adding pages, so I'm guessing there may be a bit of code in theregobbling up the memory or else it's trying to do a lot of things atonce.

So… what are the most memory intensive things happening there andI'll see if I can track down the problem by disabling stuff bit by bitin the source code. I just don't see how multiple database queriesalone can slow the system this much.

I'm sure there must be something similar happening in the installer as well as certain steps will either occasionally timeout for me or just go really slowly.

Any ideas would be greatly appreciated :)

 The Last Outpost - Entertainment news, reviews, previews & interviews. No holds barred - big boys' language in use!

Back to the top
 
Posted
Rating:
#21150
Avatar

Hi Pete,

Sorry to hear you're getting these problems. I know this is stock advice based on our minimum requirements, but please make sure ocPortal is running with at least 12MB, preferrably 16MB.

Now, to answer your question:

The core of ocPortal is really really heavily optimised, so what's left that takes up load is either one of:
  • some page/block that itself has not been well optimised yet is resource hungry (this shouldn't happen, but if you can narrow down a problem to something, we'll certainly look into it). This is most likely to happen when ocPortal is processing heaps of data (such as a long list of content) without using an efficient algorithm.
  • something giving ocPortal a bottleneck, such as an overloaded disk, or when ocPortal has to open up network connections to an overloaded/dodgy server.
  • Tempcode. ocPortal is a bit slower than some other well-optimised systems due to the flexibility we designed in the templating system. Basically ocPortal could be considered a programming language that runs on top of PHP. It takes templates and various input, compiles it, processes it all together, and then evaluates it all as one huge blob of code. The PHP code behind Tempcode is extremely tightly optimised, but it is generally the root cause of most memory issues.

Whether ocPortal is timing out (the PHP time limit setting) or running out of memory (the PHP memory limit setting), or if PHP is just generally failing due to some other fault or resource instability (e.g. usage spikes), is crucial in identifying problems.

One small tip is you could place a .htaccess file under the Admin Centre directory that defines a higher PHP memory limit.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#21154
Avatar

Well-settled


It's all done with customised php.ini's on my server, but here's my limits:

max_execution_time = 3600
post_max_size = 55M
upload_max_filesize = 50M
max_input_time = 3600
memory_limit = 256M

Nice 'n big ;)

I wonder if it might actually be the tempcode y'know… there's so much that goes into each page. I know this is going to be a silly question, but could something be developed using XML and XSLT? Would that not be heaps faster?

Also, would it not be more efficient to save the custom comcode pages as formatted xhtml rather than running the pages through a lot of regular expression code (I presume) every time they're loaded?

Anywho - back to the particular example of logging into the Admin Centre - I'm assuming it grabs the user's details, the list of admin actions, the admin notes, the admin menus and renders the admin centre homepage, but is it doing anything else that's a particular strain? I've turned off logging which helped somewhat when the site was receiving a LOT of visitors (it's amazing how large that mysql table can get - I think it was at 52mb before I noticed ;)).

EDIT: I think I may have found the problem - when I try to view the templates used to load the admin centre it stops loading before it can render the table. I've tried this half a dozen times and it won't render the list before the page stops responding.

I'm guessing there's quite a few templates to load in 2.6.x's admin centre main page ;)

FURTHER EDIT: Just got it to load - there's a lot of instances of different template bits:

BLOCK_MAIN_STAFF_NEW_VERSION (1)
BLOCK_SIDE_STORED_MENU (3)
BOTTOM (1)
COMCODE_PAGE_STAFF_FOOTER (2)
CSS_NEED (4)
CSS_NEED_INLINE (2)
FOOTER (1)
GLOBAL (1)
HEADER (2)
HYPERLINK (27)
JAVASCRIPT_NEED (4)
MENU_BRANCH_tree (33)
MENU_NODE_BRANCH_tree (5)
MENU_STAFF_LINK (3)
MENU_tree (3)
NOTES_BLOCK (1)
PARAGRAPH (1)
TABLE_TITLED_END_side (4)
TABLE_TITLED_START_side (4)
TOP (2)

If it's got to do str_replace or preg_replace on all of those it's going to slow things down a little. That said, I wouldn't have thought it would be slowing it down quite that much…

 The Last Outpost - Entertainment news, reviews, previews & interviews. No holds barred - big boys' language in use!

Back to the top
 
Posted
Rating:
#21160
Avatar

I know this is going to be a silly question, but could something be developed using XML and XSLT? Would that not be heaps faster?

It's definitely not a silly question - I've certainly considered it too. I've also considered doing the processing using Javascript. However it's not practical because Tempcode is so dynamic that it's unseparable from the data that only the server can know. For example, if an 'IMG' tag is inside the Tempcode, the client couldn't really deal with that; if the server does any pre-filtering, then a high enough proportion of the work has to be done just for that as to marginalise any client-side-processing improvement.

Also, would it not be more efficient to save the custom comcode pages as formatted xhtml rather than running the pages through a lot of regular expression code (I presume) every time they're loaded?

They're converted to Tempcode and saved in the Comcode Page cache. Further processing would remove the ability for dynamic elements to update. For example, if '$USER' was present in a page (there are many ways for this kind of thing to appear, even as the result of Comcode being compiled), then that would not update when being viewed by different users. This is the sophistication Tempcode gives that I referred to in the last post, and indeed the biggest performance penalty, because it stretches across ocPortal.

but is it doing anything else that's a particular strain

Are the Comcode Page and block caches on? If they are, then no, there shouldn't really be much strain.
I highly doubt this is the cause of the problem, but try commenting out line 411 of sources/site.php (I give you permission). You'll see what I mean when you get there, but we very carefully designed that line to only execute after the page itself is output.

<list>

If I remember correctly, in 2.6 the template tree showed what was used in the generation of the current page. In version 3.0, it actually shows what would be used to generate the current page, if the caches are off.
Therefore because for 2.6 it's a direct representation, it strongly suggests to me you have some caches off.

If it's got to do str_replace or preg_replace on all of those it's going to slow things down a little

It's actually done by a custom designed parser, that goes through character by character. But if you have the Template cache enabled, it actually saves the parsed files to disk using the PHP 'serialize' function, which is super-fast to load up.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#21161
Avatar

Well-settled

Hmm… everything caching-wise was turned on. I also just turned on gzipped output and it apers to be having much more success at loading the admin centre homepage first time now :)

Having said that, things like the serttings page load very quickly and did do before - the admin centre homepage seems to be the slow one.

 The Last Outpost - Entertainment news, reviews, previews & interviews. No holds barred - big boys' language in use!

Back to the top
 
Posted
Rating:
#21178
Avatar

I suppose another possibility is the new-version check from our server is slow. Perhaps the connection between our servers is dodgy? You can edit the Admin Centre front page to remove blocks, trial and error.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
There are too many online users to list.
Control functions:

Quick reply   Contract

Your name:
Your message: