HTML Logo by World Wide Web Consortium (www.w3.org). Click to learn more about our commitment to accessibility and standards.

Moving forward with Composr

ocPortal has been relaunched as Composr CMS, which is now in beta. ocPortal 9 will be superseded by Composr 10.

Head over to compo.sr for our new site, and to our migration roadmap. Existing ocPortal member accounts have been mirrored.


Unable to save files using code_editor.php

Login / Search

 [ Join | More ]
 Add topic 
Posted
Rating:
#82803 (In Topic #17259)
Avatar

Community saint

When I open a php file in the root of my site with code_editor.php I get a "This file is not overridable - it will be edited directly." message at the bottom of the screen and I can not save my changes.

I even tried it with a file in the sources directory and although I don't get the same message as above, I still can't save any changes.

It shouldn't be an ftp account permissions issue as I am using the same account that I use when I upload source files.

Any ideas?

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#82836
Avatar

temp1024 said

When I open a php file in the root of my site with code_editor.php I get a "This file is not overridable - it will be edited directly." message at the bottom of the screen and I can not save my changes.

I even tried it with a file in the sources directory and although I don't get the same message as above, I still can't save any changes.

It shouldn't be an ftp account permissions issue as I am using the same account that I use when I upload source files.

Any ideas?

I presume the button just isn't doing anything?

Try opening up the network tab of developer tools in Chrome/Safari/Firefox, and seeing how the save request previews. What it is doing is it saves into a hidden frame, and the result of that frame puts out a Javascript alert. If there is some kind of error it may be hidden inside that frame.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#82845
Avatar

Community saint

Chris Graham said

temp1024 said

When I open a php file in the root of my site with code_editor.php I get a "This file is not overridable - it will be edited directly." message at the bottom of the screen and I can not save my changes.

I even tried it with a file in the sources directory and although I don't get the same message as above, I still can't save any changes.

It shouldn't be an ftp account permissions issue as I am using the same account that I use when I upload source files.

Any ideas?

I presume the button just isn't doing anything?

Try opening up the network tab of developer tools in Chrome/Safari/Firefox, and seeing how the save request previews. What it is doing is it saves into a hidden frame, and the result of that frame puts out a Javascript alert. If there is some kind of error it may be hidden inside that frame.

After I press the "edit" button the network tools reports a 403 forbidden on the iframe:

Code

http://mydomain.com/mysite/code_editor.php?type=edit

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#82846
Avatar

That makes me thing "mod_security".


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#82848
Avatar

Community saint

Chris Graham said

That makes me thing "mod_security".

I have tried it with and without:

Code

<IfModule mod_security.c>
SecFilterEngine Off
SecFilterScanPOST Off
</IfModule>
And it makes no difference.

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#82849
Avatar

Unfortunately mod_security 2 cannot be configured with .htaccess. I brought this up with them and got into a bit of an argument – they were well on the side of server admins at large companies, not on the side of users who don't control the software and servers they use.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#82850
Avatar

Community saint

So how is what the code editor does different to say what the upgrader tool does (i.e. both modify php files)?


Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#82853
Avatar

Editor pipes it through HTTP (before it is relayed through FTP), upgrader copies it. i.e. PHP code in upgrader never touches mod_security.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#82854
Avatar

Community saint

I see. Thanks!

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#83001
Avatar

If you happen to get a hold of the mod_security rule details that are failing, please post here. Sometimes these are written into Apache error logs.

I'm collecting a list of known problematic rule IDs.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Item has a rating of 5 (Liked by Chris Graham)  
Rating:
#83042
Avatar

Community saint

Contacted my host and got the details:
[Sat Apr 07 12:11:17 2012] [error] [client xxx.xxx.xxx.xxx] ModSecurity: Access denied with code 403 (phase 2). Pattern match "; ?(?:cat|ls|perl|uname|pwd|cp|kill|echo|tclsh8?|cpp|python|ch
own|rm|kill|ping|rsync|rdiff-backup|ssh|scp|wget|curl|links|g
\\+\\+|ch(?:grp|own)|passwd|bash|telnet) " at ARGS:file. [file "/etc/httpd/modsecurity.d/10_asl_rules.conf"] [line "430"] [id "340029"] [rev "12"] [msg "Atomicorp.com WAF Rules: Possible command in REQUEST_URI or Argument"] [data "; echo "] [severity "CRITICAL"] [hostname "mysite.com"] [uri "/mysite/code_editor.php"] [unique_id "T3@iP3CMtOEACcG1wIAAAAA1"]

I've updated Configuring mod_security - ocPortal.com with a summary.

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#83192
Avatar

Community saint

I just got my host to whitelist to rule #340029 above and I'm still getting 403-forbidden errors. Looks like there will be multiple rules to get around this one.

As a side note I'm also getting 403-forbidden errors in the comcode editor while editing comcode that contains <script>. I never had problems before, so it must be new rule updates.

I'm slooooowly working through these as each rule is identified.

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#83205
Avatar

Community saint

Well, after many emails back and forth with my host, we have identified rules #340029, #340128, #380018, #380020 that got the code editor to save the php page I was working on.

We also worked on the template editor because I was no longer able to edit templates I created months ago.

In order to edit HEADER.TPL we needed to whitelist #340147, #340149, #340148 & #350148 . I thought great, done!

Then I thought I'd try and edit JAVASCRIPT.TPL and bang! doesn't save!!!. Ended up having to add rules #340095 & #340118 to get that to save.

OK, now that should be all….WRONG! Unable to save changes to JAVASCRIPT_CUSTOM_GLOBALS.tpl .

At this point I can see that this Identify and whitelist approach ain't gonna cut-it as anything could be lurking around the corner to trip one rule or another.

Chris, its looking like editors are going to have to obfuscate the document payloads in some way. I can't see any other sane way around it. And you won't be able to use straight hex or base64 encoding as mod_rewrite is aware of those.

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#83206
Avatar

Ok so these are an extra set of modsecurity rules, they are all in here:

http://updates.atomicorp.com/channels/rules/delayed/modsec/10_asl_rules.conf

I think you are close to getting through it and I'd encourage you to keep pushing. I am studying the rules that failed and I'll document them.

Regarding payload obfuscation, I'd rather not have to do it because it means putting a dependency on JS in there, and also I can see there are rules here such as "code injection attempt base64encoded" designed to thwart this, so we'd be going to war with them.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#83207
Avatar

I've updated the page with these.

Configuring mod_security - ocPortal.com

I'd selfishly like you to keep picking out issues but also see the note I added about "SecFilterScanPOST On".


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#83208
Avatar

Community saint

Chris Graham said

Ok so these are an extra set of modsecurity rules, they are all in here:

http://updates.atomicorp.com/channels/rules/delayed/modsec/10_asl_rules.conf
Yep, found that doc also and that's where I got my hex/base64 info.
I think you are close to getting through it and I'd encourage you to keep pushing.
I have a feeling that I will always be close but never reach the end :'( . They will always be adding rules so we will always be playing catch-up, and as I found out, something that works one day may not work the next.
I am studying the rules that failed and I'll document them.
F.Y.I. The template editor problems can of course also occur in the comcode editor. But I'm sure you new that ;) .
Regarding payload obfuscation, I'd rather not have to do it because it means putting a dependency on JS in there
There are already plenty of JS dependencies. I can't even get to select a template/css to edit with JS off. A quick look around with JS off and I can't leave feedback/comment or send a chat message, and the shoutbox and polls side boxes won't render properly.

And the code/template/comcode editors are really advanced features so that I don't think JS dependency is unrealistic for it.
and also I can see there are rules here such as "code injection attempt base64encoded" designed to thwart this, so we'd be going to war with them.
I thought that you already declared ware on them with the .htaccess issues.


Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#83215
Avatar

There are already plenty of JS dependencies. I can't even get to select a template/css to edit with JS off. A quick look around with JS off and I can't leave feedback/comment or send a chat message, and the shoutbox and polls side boxes won't render properly.

Actually there aren't, but if ocPortal thinks you have Javascript it'll use that. It does that with URL parameters and cookie settings. If the Javascript detection is turned on, there'll be a cookie still around saying you have Javascript. Otherwise there is a special URL to access without Javascript.

I'm generally concerned about complexity. If ocPortal has to intercept a form before posting, rewriting post variables to do it, I just see a lot of points of failure doing that. E.g. some connection issue happening, the user using the back button, and finding their content garbled. A lot could go wrong so it would be a lot of careful work and testing to engineer away those points of failure.

And the code/template/comcode editors are really advanced features so that I don't think JS dependency is unrealistic for it.

At this time we support without Javascript, and it's kind of an accessibility point. E.g. someone can theoretically use these editors using something like a Braille reader with no Javascript support. It is likely in the future we will drop non-JS support given that ARIA now makes Javascript more accessible, but it would be a planned affair rather than an erosion of what we currently support.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#83218
Avatar

Community saint

Chris Graham said

There are already plenty of JS dependencies. I can't even get to select a template/css to edit with JS off. A quick look around with JS off and I can't leave feedback/comment or send a chat message, and the shoutbox and polls side boxes won't render properly.
Actually there aren't, but if ocPortal thinks you have Javascript it'll use that. It does that with URL parameters and cookie settings. If the Javascript detection is turned on, there'll be a cookie still around saying you have Javascript. Otherwise there is a special URL to access without Javascript.
Fair point. Its probably some lingering memories then.
I'm generally concerned about complexity. If ocPortal has to intercept a form before posting, rewriting post variables to do it, I just see a lot of points of failure doing that. E.g. some connection issue happening, the user using the back button, and finding their content garbled. A lot could go wrong so it would be a lot of careful work and testing to engineer away those points of failure.
I understand your concerns. My concerns are that these darn security walls can creep up when you least expect them and then be stumped as to why thing aren't saving/working.

At the moment I can Identify them readily because I'm actively working the issues, but a month+ down the track if something happens who know how easily (or even if) I'll remember this stuff and be able identify the cause.

Chris Graham said

And the code/template/comcode editors are really advanced features so that I don't think JS dependency is unrealistic for it.

At this time we support without Javascript, and it's kind of an accessibility point. E.g. someone can theoretically use these editors using something like a Braille reader with no Javascript support. It is likely in the future we will drop non-JS support given that ARIA now makes Javascript more accessible, but it would be a planned affair rather than an erosion of what we currently support.
Of course accessibility is a very important factor and the standard from fall-back can be used in those situations as it is now.

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#83331
Avatar

Community saint

Chris Graham said

I think you are close to getting through it and I'd encourage you to keep pushing.
Well, with the discovery and whitelisting of the latest rule (#390715 for code_editor.php), my host is no longer willing to help identify new mod_security rules.

I can't say I can really blame them too much as we have already done quite a few test/whitelist cycles.

I had already gone so far as consolidating 1700 .php files and ~500 .tpl files into 25 super-files to simplify testing but even that is too much for them.  :'(

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#83337
Avatar

Sigh :(.

I just had another look at the rules. There is a lot in there about exceptions for various web systems, so I looked at the exceptions and found the code editor would also need these ones banning: 340011, 340021, 340027, 340131, 340133
I've updated the community docs document.

I thought again about trying to do payload obfuscation but without major reengineering that's not going to be viable. Due to limitations in Javascript we can't reliably intercept form submissions (form.onsubmit is not called if form.submit is called directly). We also can't automatically rewrite form fields because it'll trigger onchange events which could have consequences, and also isn't suitable for use with HTML5's inbuilt field validation. So we'd need to start writing a submission pipeline for all the forms, copying fields into a secondary 'safe' form, and then submitting that. It'd take ages and really over-complicate the code.

I think a web host has a few choices:
  • Help people with mod_security, disabling rules as required
  • Use mod_security v1, or get v2 modified, such that the user can disable it
  • Disable mod_security for users if they ask
  • Don't use mod_security at all
  • Advertise the hosting as limited

Did you try asking them to disable completely, or turn SecFilterScanPOST off?


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
There are too many online users to list.
Control functions:

Quick reply   Expand