HTML Logo by World Wide Web Consortium (www.w3.org). Click to learn more about our commitment to accessibility and standards.

Moving forward with Composr

ocPortal has been relaunched as Composr CMS, which is now in beta. ocPortal 9 will be superseded by Composr 10.

Head over to compo.sr for our new site, and to our migration roadmap. Existing ocPortal member accounts have been mirrored.


Spam block mod / notifications

Login / Search

 [ Join | More ]
 Add topic 
Posted
Rating:
#85555 (In Topic #17575)
Avatar

Community saint

FYI Chris

I wasn't sure if I should have posted this in the spammer database thread or start a new one. I didn't know how closely you were monitoring that topic with Bob over there talking to himself, and I figured this would get lost in his conversation. ;)

I'm just passing on my thoughts and info to you for reference for future versions.

I have "Perceived spammer blocked" notification set to email when one is banned, and it seems that is working way too good. What triggers a notification when an IP ban is triggered? I've noticed I am receiving multiple emails from the same IP ban just a few seconds apart. Some of these bans have triggered 20 or more emails for the same IP ban in just a few seconds.

I was away for 7 hours today and received 133 IP ban emails during that time. In 9 hours I have received 318 emails (159 + 159 carbon copies). If some unsuspecting person chooses one of the email digests or a PT he/she could be in for a surprise.  O_o

Steve
Back to the top
 
Posted
Rating:
#85557
Avatar

Community saint

sholzy-

I didn't have any notification set until just now but it will be interesting to see what emails I get.

Regarding talking to myself, I just posted in the other thread that I don't expect Chris to address this necessarily since the feature is not formally released. However, I have taken to reporting both banned and unbanned IPs with their response from HTTP:BL in the hopes of providing plenty of data for Chris to analyze when he gets around to it.

I think I figured out that getting a response of 127.0.0.67 from Tornevall will only result in a ban if the "Implied confidence level" exceeds the "Spammer ban threshold".

Anyway, I'll let you know if I have any issues with email here.

Bob
Back to the top
 
Posted
Rating:
#85559
Avatar

Community saint

I was just giving Chris something to think about.

In 20 hours I've received a total of 634 emails (317 + 317 carbon copies) for bans.

I'll probably turn emails back off since I know bad IP's are being caught.

Steve
Back to the top
 
Posted
Rating:
#85561
Avatar

Community saint

Wow…that's a lot. I've only had two bans so far today but both were before I turned notifications on. We'll see what happens with notifications on 8.0 once I get another automatic ban.

Bob
Back to the top
 
Posted
Rating:
#85564
Avatar

Community saint

BobS said

Wow…that's a lot. I've only had two bans so far today but both were before I turned notifications on. We'll see what happens with notifications on 8.0 once I get another automatic ban.

Bob

I'm switching notifications settings to the email digest.

In the 24 hours I've had notifications turned on, I've had:
  • 52 banned IP
  • 350 emails
    • 141 was for a single IP  O_o
And since I have carbon copy turned, I get double the emails. So those 350 emails are actually 700 emails landing in my in-box.

This is why I wanted to bring this to Chris' attention for consideration for tweaking in the release version.  :)

Steve
Back to the top
 
Posted
Rating:
#85566
Avatar

Community saint

Good to have this reported as it serves to as it renders the notifications pretty useless. It's a bit like U.S. intelligence – more raw data than they can process in a timely manner.

Bob
Back to the top
 
Posted
Rating:
#85568
Avatar

Community saint

I think 1 notification for each IP that hits the ban list would be perfect. Then each reoccurring ban after the IP was released from being banned would generate notifications again.

Steve
Back to the top
 
Posted
Rating:
#85586
Avatar

Community saint

sholzy said

I think 1 notification for each IP that hits the ban list would be perfect. Then each reoccurring ban after the IP was released from being banned would generate notifications again.
I definitely agree with this.

I just got my first notification which was duplicated just as yours are.

The notification subject contains "Notification: Perceived spammer banned (194.167.115.18)" but the IP does not appear in my ban list. My block list cache time is set to 1080 (18 hours) and I received this notification at 6:45AM, less than 2 hours ago.

Bob

Back to the top
 
Posted
Rating:
#85592
Avatar

Community saint

I want to throw this bug in here since it's related to both spamblock mod and notifications.

I switched from individual emails to "daily digest" and I'm still receiving individual emails. (Or maybe I'm misunderstanding how the digest option works?)

Steve
Back to the top
 
Posted
Rating:
#85612
Avatar

I think I figured out that getting a response of 127.0.0.67 from Tornevall will only result in a ban if the "Implied confidence level" exceeds the "Spammer ban threshold".

Yes, however only in the case of HTTP:BL not having any data, otherwise tornevall would never be checked. i.e. HTTP:BL takes precedence because it has confidence data which tornevall does not. Putting this another way, a higher implied confidence level will never take precedence over the real confidence level that HTTP:BL reports.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Item has a rating of 5 (Liked by sholzy)  
Rating:
#85613
Avatar

BobS said

Good to have this reported as it serves to as it renders the notifications pretty useless. It's a bit like U.S. intelligence – more raw data than they can process in a timely manner.

Bob


And the NSA now have another 700 emails from Sholzy to check for indications of terrorism ;).


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#85616
Avatar

Ok I have noticed 3 problems here…

1…

The internal IP ban list is not checked if the .htaccess is writable. This was a (small) optimisation but now we have these temporary bans, we need to take that off. Permanent bans will still get written to .htaccess, which is the main performance boost (stops a DOS attack pulling up requests that have to bootstrap ocPortal).

In sources/global2.php change:

Code

   if (((!isset($SITE_INFO['known_suexec'])) || ($SITE_INFO['known_suexec']=='0')) && (!is_writable_wrap(get_file_base().'/.htaccess'))) // If we have to run this in software
to:

Code

   require_code('config'); // Config is needed for much active stuff

2…

The subject lines of the notifications all say ban, which is wrong. I'll fix, but check the body of the message for whether it is really a ban, a block, or an approve. It is possible in the flood case some say block…

3…

…There's no way of monitoring what people have been blocked within a time threshold, so no way I can really limit these. So I do suggest putting the notification on digest mode. I'll make that the default for this notification.


I'll look at the digest apparently not working soon.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#85618
Avatar

Community saint

I've just received 9 emails, all with subjects containing "Notification: Perceived spammer banned (85.236.65.186)".

It appears that the IP is checked at each page load (my setting) and the email sent if it the response indicates the IP shuld be banned. It seems like the actual email sending code should be part of "ad ban" code and that a check for an existing ban should be made prior to doing the add or will this produce too much overhead?

Bob

Back to the top
 
Posted
Rating:
#85619
Avatar

See '1' above.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#85621
Avatar

Community saint

Chris Graham said

See '1' above.
Got it – we crossed due to my being one of the slowest typists in the world.

Thanks for your help.

Bob
Back to the top
 
Posted
Rating:
#85622
Avatar

sholzy said

I want to throw this bug in here since it's related to both spamblock mod and notifications.

I switched from individual emails to "daily digest" and I'm still receiving individual emails. (Or maybe I'm misunderstanding how the digest option works?)


A little from column A and a little from column B.

I just tested this and found the digests weren't working for system-originating emails. This OcCLE command should fix it:

Code

:$GLOBALS['SITE_DB']->alter_table_field('digestives_tin','d_from_member_id','?USER');

I think you were seeing a lack of digests, but also seeing the CC emails you mentioned earlier.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#85633
Avatar

Community saint

Chris Graham said

See '1' above.
I just had an IP banned and received two notification emails. I made the changes in '1' above but it seems I am still receiving two emails.

Bob
Back to the top
 
Posted
Rating:
#85634
Avatar

Just 2, or a stream? 2 could happen due to a CC address, or due to them flooding (hence parallel requests before the block kicks in).


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#85636
Avatar

Community saint

It was just two, both to the same address.

Also, the body of the email contains no reference to 'ban', 'block' or 'approve' – just the subject which contains "Notification: Perceived spammer banned (94.102.48.116)".

Bob
Back to the top
 
Posted
Rating:
#85638
Avatar

What does the body say?


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
There are too many online users to list.
Control functions:

Quick reply   Expand