HTML Logo by World Wide Web Consortium (www.w3.org). Click to learn more about our commitment to accessibility and standards.

Moving forward with Composr

ocPortal has been relaunched as Composr CMS, which is now in beta. ocPortal 9 will be superseded by Composr 10.

Head over to compo.sr for our new site, and to our migration roadmap. Existing ocPortal member accounts have been mirrored.


Allow exclusion of contents from sitemap for all content types

Login / Search

 [ Join | More ]
 Add topic 
Posted
Rating:
#82925 (In Topic #17271)
Avatar

Community saint

Currently, the sitemap generated by ocPortal includes all content with Guest view access. There are situations where you want the content available for guests on the site but that you do not want that content indexed by the search engines. The only solution to this currently is to place a restriction in the robots.txt file which works but which also causes an inconsistently: Google warns that items in the sitemap are excluded by entries in robots.txt. For Bing, I suspect that this would result in "errors" which, if over a certain percent of your sitemap entries, has them reject your sitemap.

I think the better approach towards addressing this is to allow the admin to mark content as included/excluded when it is created and that the sitemap generation script would respect the exclude in addition to any current logic. This would provide the means to have a "custom" sitemap automatically generated by ocPortal. This should work for all content types and include category and individual entry exclusions.

Bob

0000415: Allow exclusion of contents from sitemap for all content types - ocPortal feature tracker
Back to the top
 
Posted
Rating:
#82927
Avatar

Community saint

Another option might be to check the url's against robots.txt and exclude it if it matches.

I might actually tinker around with that myself.

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#82936
Avatar

Community saint

Good idea, temp. It's a little more involved for someone new to maintaining a site but would solve the situation nicely.

Bob
Back to the top
 
Posted
Rating:
#82938
Avatar

I agree


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#82939
Avatar

Community saint

BobS said

It's a little more involved for someone new to maintaining a site but would solve the situation nicely.
True, and your suggestion is still a good one. This way we get the best of both worlds.

Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Item has a rating of 5 (Liked by Harry-S)  
Rating:
#85106
Avatar

Community saint

This one reared its ugly head today. I've suddenly gone from 3 warnings to 82 in Google's sitemap report because I have /polls/* blocked in robots.txt while I have those three URLs in my sitemap.

This just seems insane and I don't know if I should attribute it to changes made on the Google Webmaster Tools site (which I mostly hate because it puts pretty much everything one extra click away). They show only the three URLs but they say that there are 82 warnings. What are the other 79? Or are they just telling me how many times they have encountered this issue?

I don't think this is impacting me on Google in any significant way but I know that Bing will delist your site if you have too many errors against your sitemap. Google calls these warnings , not errors, so perhaps it is not much an issue.

Anyway, I am done ranting on this.

Note that there is a tracker item to deal with this: 0000415: Allow exclusion of contents from sitemap for all content types - ocPortal feature tracker

Bob

Back to the top
 
There are too many online users to list.
Control functions:

Quick reply   Contract

Your name:
Your message: