HTML Logo by World Wide Web Consortium (www.w3.org). Click to learn more about our commitment to accessibility and standards.

Moving forward with Composr

ocPortal has been relaunched as Composr CMS, which is now in beta. ocPortal 9 will be superseded by Composr 10.

Head over to compo.sr for our new site, and to our migration roadmap. Existing ocPortal member accounts have been mirrored.


Require admin approval if IP already listed in db

Login / Search

 [ Join | More ]
 Add topic 
Posted
Rating:
#111261 (In Topic #22353)
TQ
Avatar

Honoured member

Help stop individuals scraping the site using multiple usernames

I know I've broached this subject before in a slightly different way but I want to stop persistent users from creating multiple user accounts so that they can gain enough points to download everything on my site without 'Giving Back'!

When I last discussed this, I suggested that the user should not be able to create a new account from an IP already in the db. The argument against this was that there may be more than one legitimate member using a given IP from a dynamic range so the new user would be unfairly block, which, frankly, is a reasonable argument.

That said, when I was checking on a user recently I discovered that they had registered 16 accounts from the same IP address. Further investigation revealed that 69 IP's had 3 or more users associated with them (a total of 267 accounts). Looking at a few of these account showed that these people had exhausted their 'joining' points then created a new account to continue downloading.

I have since banned each member when the IP is used by more than 3 members and added them to the usersubmitban_ip table.

My revised suggestion is that any new account created is checked against the existing members IP addresses and, if the IP is already in the db, the member is flagged for Admin approval before they can sign in. I will then check the db to learn more about that IP's usage.

I'm aware that I can already 'Require member validation' but with 50-70 new users per day this would not only create a lot of un-necessary work but also stop legitimate members from accessing the site until I had time to approve their account. Knowing how ridiculously behind I am with validating uploads, this would be a recipe for disaster.

Is my suggestion possible? Would it require a lot of code changes? Can someone suggest a better method of resolving this issue?

All feedback will be answered and any suggestions gratefully received.

TQ
Back to the top
 
Posted
Rating:
#111262
Avatar

Try this change…

In sources/ocf_join.php:

Change:

Code

$require_new_member_validation=get_option('require_new_member_validation')=='1';

To:

Code

$require_new_member_validation=get_option('require_new_member_validation')=='1';
if ($GLOBALS['FORUM_DB']->query_value('f_members','COUNT(*)',array('m_ip_address'=>get_ip_address()),' AND id>1')>=1)
    $require_new_member_validation=true;



Tracker issue for cleaner implementation:
0002413: More flexibility in email_confirm_join option, IP uniqueness - Composr CMS feature tracker


Last edit: by Chris Graham


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#111263
TQ
Avatar

Honoured member

Thank you Chris, added to the appropriate script and waiting to catch the first one out!

Much appreciated.

TQ
Back to the top
 
Posted
Rating:
#111308
TQ
Avatar

Honoured member

Chris Graham said

Try this change…

In sources/ocf_join.php:

Change:

Code

$skip_confirm=(get_option('skip_email_confirm_join')=='1');<br />
To:

Code

$skip_confirm=(get_option('skip_email_confirm_join')=='1');<br /> if ($GLOBALS['FORUM_DB']->query_value('f_members','COUNT(*)',array('m_ip_address'=>get_ip_address()))>=1)<br />     $skip_confirm=false;<br />
Tracker issue for cleaner implementation:
0002413: More flexibility in email_confirm_join option, IP uniqueness - Composr CMS feature tracker

Can you think of any reason why this doesn't work? I've rolled up a quick table (in my admin zone) of users that have 3 or more accounts on the same IP. A couple have turned up but I never received the validation email and they managed to validate the accounts themselves.

While writing this I had a thought; I've now tried to preceed

Code

    // Send 'validate this member' notification
    if ($require_new_member_validation)
etc ...
with

Code

    if ($GLOBALS['FORUM_DB']->query_value('f_members','COUNT(*)',array('m_ip_address'=>get_ip_address()))>1)
    $require_new_member_validation=true;
It seems to work but I'm not sure if it's the right thing to do?

TQ

Edit: As a footnote to the above, I changed >=1 to >1 at the end of the IF statement as it appears that the new user is already in the database before the IP check is processed.


Last edit: by TQ
Back to the top
 
Posted
Rating:
#111309
TQ
Avatar

Honoured member

Hi Chris,

So I said it didn't work when I should have said I didn't get a Admin validation email.

Of course my variation didn't work either as the user was still able to validate their account themselves.

When you have time, could you have a look at how I can use you code change above and receive the email.

Thanks

TQ
Back to the top
 
Posted
Rating:
#111316
Avatar

Whoops. You're right but didn't get the code in the right order, I am editing my original post.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#111317
TQ
Avatar

Honoured member

Hi Chris,

Welcome back from your vacation, I hope it gave you some respite from my never ending barrage of question.

The latest version of the change worked like a charm. I could see why v1 didn't work but didn't have the courage to keep editing the script after killing it on one occasion. I did however learn a little more about the way the site is coded which pleases me immensely (how sad am I?).

In you absence I was triggering the admin validation email so I could at least examine the user (after the event) and bouncing them to 'Probation' status and sending them a friendly email. I wonder if your tracker might include this method rather the complete denial of membership (maybe even as an Admin option). As part of my probation membership I have the site setup to disallow the use of points (which they need to download). If that idea doesn't hold water, it is perfect as it is.

I know I'm never going to catch everyone as one of the people I put on probation then tried to use a VPN, his failing was that he used minor variation of his first & second email address and it stood out like a sore thumb, otherwise I'd never have caught him.

Thank you Chris, very grateful for your help as always.

TQ
Back to the top
 
Posted
Rating:
#111318
Avatar

Thanks, and glad to hear :).

Probation - good idea.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#111319
TQ
Avatar

Honoured member

Back to my machine after a few hours away and find 3 email for user validation. On checking, all are fist-time-callers (neither ocP or my db check shows their IP more than once).

Could it be that the member is already in the db by the time this code runs?

I've changed the members count to >=2 for now and will monitor the db to see if it works as expected.

Thanks

TQ
Back to the top
 
Posted
Rating:
#111320
Avatar

I don't think so, but you need to have removed all your code and made only the change I posted above.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#111321
TQ
Avatar

Honoured member

That was the first thing I checked before I wrote back.

I've now deleted the edited version and replaced it with a fresh copy in sources_custom (unadulterated).

Thanks

TQ
Back to the top
 
Posted
Rating:
#111322
TQ
Avatar

Honoured member

It's still not right but I may have an inkling why.

The problem is, from time to time, a new member gets flagged as sharing the same IP as another user even though that is not the case.

My table, which I'm still using to cross-check, produced the same result on one occasion tonight. This is the first time I've seen this happen but it did help me investigate.

I search the db and found that Guest had the same IP as the wrongly flagged user, understandable if someone is just joining.

I'm able to eliminate Guest from my table results but I don't know how to fix it ocP style.

Do you think this is why I'm getting about 10% of legitimate first-time-callers flagged as having 2 accounts?

TQ
Back to the top
 
Posted
Rating:
#111323
Avatar

Ok, I edited my post. The only change is ,' AND id>1' now in there.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#111324
TQ
Avatar

Honoured member

Thanks Chris
TQ
Back to the top
 
Posted
Rating:
#111325
TQ
Avatar

Honoured member

Hi Chris,

Just a quick note to let you know that it seems to working as expected.

In the past 12 hours I've had 29 new members, 2 were IP dups - correctly flagged and NO wrongly flagged members.

Thank you.

TQ
Back to the top
 
Posted
Rating:
#111326
TQ
Avatar

Honoured member

Latest Report

past 24 Hours ...
New Users = 66
Duplicate users on single IP = 8 (One had a total of 8 accounts so IP is now banned)
Erroneous flagged users = 1

3 of the 8 created a second account within minutes of the first. (Didn't wait long enough for the user validation email, CRON every 2 minutes & no delayed delivery). Maybe the join script could stop a second join action if within a given time.

Nothing untoward about the erroneously flagged user except the 'Investigate IP' showed a number of 'Flood' entries which I suspect was them refreshing the page on their iPhone but I don't understand how the IP got into the mix with the above script.

Basically, it's working like a charm and it proves that there are a lot of people out there willing to go out of their way to circumvent the site rules. A quick tally of my membership (19,553) shows about 6% have more than 1 user account (based on IP). I wonder what the real number is for those on dynamic IP's?

The technique of putting the duplicates on probation and sending them a friendly email offering to help resolve a forgotten IP/ID seems to work well. One has replied the rest have either stopped trying or found a way of joining with a different IP.

As a footnote Chris, thanks to your help regarding the sitemap etc., my sites position in the search engines is almost recovered. Traffic is up by 650% from the low at this time last month and not far from it's old position.

TQ
Back to the top
 
Posted
Rating:
#111327
Avatar

Thanks.

Maybe for that erroneous user, there was another account they since deleted. Or another account whose IP since changed.

Join confirm mails skip CRON, so should be no issue there.

If join screen is refreshed it invisibly skips adding the member, as it sees it's already there - it just shows the join messages again. So they'd have to go back and join again with other details, would be a pain to try and track that in a new way - I think the system you have now, essentially that's what it is.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#111328
TQ
Avatar

Honoured member

Chris said

Maybe for that erroneous user, there was another account they since deleted. Or another account whose IP since changed.
I've already (temporarily) disabled members ability to delete their own account as I did wonder if this was happening a couple of days ago.

Chris said

So they'd have to go back and join again with other details,
That's exactly what they are doing.

Chris said

would be a pain to try and track that in a new way
All I had in mind was a check to see if the IP was already in the db and the join time was within the past hour but I'll take whatever I can get.

I'm certain no other CMS would be able to defend against the abuse that gets thrown at my site. It could be coincidence but failed path redirects are up since I implemented this action.

Thanks

TQ
Back to the top
 
Posted
Rating:
#111444
TQ
Avatar

Honoured member

Further update

I want to start with the fact that this has been extremely useful to me, it's an absolute must-have on my site.

I'm still getting a (very) few false positives but it now appears to happen when a real dupe user IP is in process. Let me explain;
  • Firstly, the false positive member has actually been able to verify their own account, it's just I get the email message saying they need to be verified. In short, it's not interrupting their registration.
     
  • It seems to happen when someone else is trying to register a new username with an IP that is already in the database ie. 2 users registering at the same time, 1 dupe, one not.
Next, is it possible to extend this feature to actually put the dupe IP member into the probation group? I'd also like the system to send them a specific email message but that's probably asking too much.

Another related question, where are the past IP's stored in the db, I've yet to find the location?

Thanks again, this has returned control over some of my greedy users.

TQ
Back to the top
 
Posted
Rating:
#111452
Avatar

Past IPs are stored via the stats table for as long as the hit data is retained; there's no dedicated table for them. Perhaps there should be, it would be useful.

I'll have to bow out of this topic I'm afraid, it would take quite some time for me to get back into it.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
There are too many online users to list.
Control functions:

Quick reply   Expand