HTML Logo by World Wide Web Consortium (www.w3.org). Click to learn more about our commitment to accessibility and standards.

Moving forward with Composr

ocPortal has been relaunched as Composr CMS, which is now in beta. ocPortal 9 will be superseded by Composr 10.

Head over to compo.sr for our new site, and to our migration roadmap. Existing ocPortal member accounts have been mirrored.


404 errors.

Login / Search

 [ Join | More ]
 Add topic 
Posted
Rating:
#76325 (In Topic #15751)
Avatar

Fan in training

Under details of the who's online block i often see 404 errors regarding mostly guests but occasionally members too.

Nobody is reporting any problems about accessing the website but i am curious as to why these error messages are being thrown up. I have attached an image of what i see on the site.

Grateful for any info or advice.

Many thanks…Ni3k:thumbs:
Back to the top
 
Posted
Rating:
#76330
Avatar

Community saint

I've noticed the same thing - many 404 errors even from my own IP address. I check the against spammer databases and they are sometimes listed there. I think these might be the result of the "gootkit" hack - hopefully the change the next release identifying browsers will help to address this issue.

Bob
Back to the top
 
Posted
Rating:
#76331
Avatar

Fan in training

Thxs BobS,

Kind of explains. But as you say hopefully it can be resolved in a later release.

Thanks
 ;)
Back to the top
 
Posted
Rating:
#76340
Avatar

Community saint

Well, identifying the browser will surely be a benefit in at least identifying those cases where the hacker is not trying to hide his footprints.

I'm not sure why I have 404s from my IP address. I run a Mac so gootkit is not a concern from my address. There is something else going on as well as I never see the 404 page yet these entries are in the logs.

Bob
Back to the top
 
Posted
Rating:
#76341
Avatar

It's pretty easy to generate 404 errors, e.g. a broken image. ocPortal's htaccess redirects them to 404.htm.

To debug, look at the web servers error logs. ocPortal doesn't actually know what's going on regarding why the 404.htm page gets called up.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#76343
Avatar

Community saint

Chris-

What's strange is that I get 404s from my IP address although I never am redirected to the 404 page.

I've gone through the server logs and can explain pretty much any other type of 404 (I know I am getting quite a few from gootkit) but I still don't see why I am getting them from my address.

I had a 404 error show up for my IP address this afternoon but it did not even show up in the server log files as a 404 - just some 200 redirected to the 404 page.

Bob
Back to the top
 
Posted
Rating:
#76347
Avatar

Possibly missing resource screens might log in ocPortal as 404's too, I'm not sure.

If it was a broken image, you might not even notice.

One thing you could do to see if the browser is really opening it is look at the 'Network' tab in the Chrome developer tools, it'll say exactly what URLs get loaded up.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#76361
Avatar

Community saint

I'll take a look in Chrome. Whatever it is does not seem critical, it's just odd seeing the 404s in the Users Online when I know I haven't seen a 404 screen.

Bob
Back to the top
 
Posted
Rating:
#76594
Avatar

Community saint

Well, I am seeing a lot of 404s due to bingbot's appending "Forcerecrawl" to URLs and from malformed URLs from Purebot from puritysearch.net.

It looks like the latter is a scraper and just wasting my bandwidth. How can I ban this search bot?

Thanks for any help.

Bob
Back to the top
 
Posted
Rating:
#76686
Avatar

Community saint

Well, I am getting tired of dealing with these 404s piecemeal. These are not missing images or broken links as they are failing on initial entry to the site. They could be bad links indexed in Google but I doubt it.

When I checked the stats for the 404 page (which accounts for about 15% of my total page views), I find the following:

Browsers
=========
Unknown 54.10%
Other      20.30%
gootkit     14.50%


Operating Systems
============
Unknown 72.20%
Other       8.40%


In other words, I have a lot of 404s coming from seemingly nefarious places. Does anyone else have number like this for their 404 page view stats in ocPortal?

I've identified the puritysearch bot as one offender, but other than a few bad incoming links which have been corrected and the bingbot "ForceRecrawl", there are no where near the number of 404s indicated in the ocPortal stats in the cPanel logs so I am really confused.

At this point I just want to head off any malicious characters so I am thinking of using the blacklist I found at Block Website Harvesters | Proxies | Scrapers | Server Exploiters | With Apache Web Server .htaccess IP Blocklist to see how much that might reduce the 404s. Has anyone used this blacklist and, if so, what were your results?

Also, how would you add this to the .htaccess file such that it does not interfere with ocPortals "Banned IP addresses" function?

Thanks for any help.

Bob
Back to the top
 
Posted
Rating:
#76707
Avatar

Community saint

No one with any input or suggestions to integrate the blacklist above into ocPortal's .htaccess file?

Today has been a particularly bad day for spammer bots hitting the site and I am looking for some relief.

Thanks for any help.

Bob
Back to the top
 
Posted
Rating:
#76725
Avatar

Community saint

Really trying to figure out how I can use this blacklist without screwing up ocPortal's IP bans.

Do I just add the above blacklist as a separate section of the .htaccess file or do I need to integrate ocPortal's IP bans into this? I'm just not sure what happens if there are two separate chunks of code dealing with the same issue although I noticed that the blacklist also employes the <Files> tag to encapsulate its list of bans.

If anyone can shed some light on this, I'd be most grateful.

Thanks for any help.

Bob
Back to the top
 
Posted
Rating:
#76896
Avatar

Community saint

Well,, I checked with the my hosting company and they said that you can have multiple blocks of blocklists which will all be executed independently. So I have implemented the exploited servers list linked above and am seeing some relief already. In particular, anonymous proxies are now gone.

I am thinking that I am going to also deny IPSs from Russia, Ukraine and Latvia - these are the countries that have the most activity with what appears to be malicious intent. Hate to block whole countries but it's just taking too much of my time to run these down every day and block them. I had sort of anticipated this and had noted where the IP was located when banning in ocPortal so I will be able to remove the IPs from the list maintained by ocPortal.

Bob
Back to the top
 
Posted
Rating:
#76901
Avatar

Community saint

New twist on the IP address lookup. Today, I had someone with the address 2a01:0e34:ef0d:e7b0:590f:705a:ebd4:7eb9. I'm guessing that this might be an IPv6 address. At any rate, the address lookup does not work. It uses the first bit as a username.

Not sure why this odd address occurred but thought I should report that the lookup doesn't work for it.

Bob
Back to the top
 
Posted
Rating:
#76902
Avatar

Community saint

BobS said

 I had someone with the address 2a01:0e34:ef0d:e7b0:590f:705a:ebd4:7eb9. I'm guessing that this might be an IPv6 address.
Looks like to me.
Not sure why this odd address occurred
Just someone sitting behind an IPv6 network most likely.

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#76903
Avatar

Community saint

temp-

My real point is that the address lookup did not work. Chris should make sure that the code and service that they use can handle IPv6 addresses as it is just a matter of time until they will become more commonplace.

Bob
Back to the top
 
Posted
Rating:
#76905
Avatar

Seemed to work when I just tested it, maybe it's already fixed for v8.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#76915
Avatar

Community saint

That would be good. Don't see too many IPv6 addresses yet but they will become more common in the not so distant future.

Bob
Back to the top
 
Posted
Rating:
#77190
Avatar

Community saint

BobS said

Well, I am seeing a lot of 404s due to bingbot's appending "Forcerecrawl" to URLs and from malformed URLs from Purebot from puritysearch.net.
Well, a month later and MS has been investigating to see if "ForceRecrawl" is one of their directives (huh?). Of course, they could also check the IPs which all emanate from their IP ranges).

I don't get MS…they say that want to be in the search business but their actions display something less than a half-heartedness. And while I am over-indexed at Google with 2900 links, Bing can only come up with 11 out of a sitemap including some 360 or so urls). Neither Google nor Bing have me happy in this particular instance.

Bob
Back to the top
 
Posted
Rating:
#77218
Avatar

Fan in action

A couple months back I was getting pelted with 404 error pages showing up in the logs, along with repeated hacking attempts. First thing everyone does is point the fingure, or at least think of the CMS must be the cause.

When I got around to looking in to it, here I had forgotten that I had registered with SiteLock. Part of the certification process to gain certification, for six months they bot hit your site like something fierce. 

"You Can't Always Get What You Want"
Mick Jagger, Rolling Stones: 1969~Let It Bleed Album 
Back to the top
 
1 guests and 0 members have just viewed this: None
Control functions:

Quick reply   Expand