This is an archived forum and the content is probably no longer relevant, but is provided here for posterity.
The active forums are here.
March 09, 2012 2:56am
Subscribe [2]#1 / Mar 09, 2012 2:56am
I need someone to give me a general run down on how to setup a robots.txt for my expression engine site. Can someone give me an example that I can use as a guide?
Also, I see in other threads that you recommend telling the googlebots not to spider your system folder - but that doesn’t that require you naming your system folder? Isn’t that dangerous?
I am using MSM, and need to setup robots.txt for main site and additional site.
Thanks!
#2 / Mar 09, 2012 4:48pm
Hello stephanie23,
The Robots.txt file is going to vary from site to site depending on what you would like to accomplish. This is actually a better topic for the How-To Forum.
A good resource on robots.txt is the wiki.
Have a great weekend.
Cheers,
#3 / Mar 09, 2012 10:01pm
Thanks - I may be blind, but I cant see how to create a new topic in “how to” area…. can you please direct me. Usually i get to new topic by clicking on a post and “new topic” button comes up, but i cant see it anywhere in the how to forums…
In fact, i cant even see the “how to Forum” listed on the forum homepage where all the different forums are listed…
#4 / Mar 11, 2012 11:59pm
Hi stephanie23,
The “how to” forum is a bit on the “classic” side 😉
The Community Help forums are the place to be these days.
No need to start a new thread, I’ll go ahead and move this one on over there for you.
Cheers!