Search engines and profile tabs
|
Posted
( )
#89327
(In Topic #18064)
|
|
|---|---|
Community saint |
The first 3.5 pages of results are all from my member profile screen. The only thing that should be indexed is the members profile tab and not the Posts/Points/Blog/Activity/Friends tabs. I'm not aware of any meta-tag/search directive that can cause a part of a page to be ignored (other then googleoff which is only valid for Google Search Appliance and not googlebot). I think the only practical way around this is to create a {$NOT_BOT} tempcode symbol that we can surround the undesirable tabs with so that the bot's can't even see the content. This will also allow us to use it selectively on any other content that we may not want to allow bots to search. And while on the topic of bots, it appears that the "msnbot" identifier is being phased out in favour of "Bingbot" which is not in bots.txt (at least not in v7.0.1).
Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
|
|
|
| Important! |
Posted
|
|
ocStaff (admin) |
Automated fix messageThis issue has been filed on the tracker as issue #736, with a fix. If I answered something that you think should be in the documentation, please take the initiative and add it to the community documentation. We really need people to help out here and build a well-organised large support resource. |
| Important! | |
|
Posted
|
|
|
ocStaff (admin) |
If I answered something that you think should be in the documentation, please take the initiative and add it to the community documentation. We really need people to help out here and build a well-organised large support resource. |
|
|
|
|
Posted
|
|
|
ocStaff (admin) |
 .
If I answered something that you think should be in the documentation, please take the initiative and add it to the community documentation. We really need people to help out here and build a well-organised large support resource. |
|
|
|
| Important! |
Posted
|
|
ocStaff (admin) |
Automated fix messageThis issue has been filed on the tracker as issue #737, with a fix. If I answered something that you think should be in the documentation, please take the initiative and add it to the community documentation. We really need people to help out here and build a well-organised large support resource. |
| Important! | |
|
Posted
|
|
|
Community saint |
Thanks Chris. Note that the tempcode tutorial does not mention bot as a valid option for $BROWSER_MATCHES . Bummer. I just whish they would come up with a standard way to allow it. I've got no problem with browsers seeing the content, I just don't want them to index it. And ocportal.com  .Thanks. Just had a look at get_bot_type() in support.php and it seems to only be reading bots.txt from text_custom folder and not text folder. It also has the default bot list hard coded  .
Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
|
|
|
|
|
Posted
|
|
|
ocStaff (admin) |
Whoops, forgot about that. Yes bots.txt is only a template, not actually used. The list is hard-coded for performance reasons as quite a few servers have very saturated disks and we try to minimise disk reads. I'll fix that and I'll add a unit test to check it is kept in sync. Will also fix the tutorial. If I answered something that you think should be in the documentation, please take the initiative and add it to the community documentation. We really need people to help out here and build a well-organised large support resource. |
|
|
|
1 guests and 0 members have just viewed this: None
Control functions: