Thread

Hi, after searching around the forums and web for some time, I still cant find an answer to this question:

How do you set up your robots.txt file when you are running Multi Site Manager with your site on an addon domain, and are there directories people generally block? (im looking for an example). My system folder is above root level and the site i am looking to create robots for is not the MSM main site. There is not much in the folder, just:
- images
- templates (where I store my templates as files)
- themes (css, javascripts etc)
- user images
admin.php
index.php
htaccess
error_log
ftp_quota

Which files should I block?

I am wondering if i need to block things like member data - or does this no longer apply because my site is above root?

Any help on this would be greatly appreciated. I know what robots are so please no links to generic “abouts robots” pages please! 😊 Im sorry if this seems obvious, I just need some reassurance that I am doing it right.

Hi Stefani Ann,

Im sorry if this seems obvious, I just need some reassurance that I am doing it right.

No worries! We are here to help 😊

Being an MSM site doesn’t really change much. Just consider the site’s containing folder like you would any web-root.

I am wondering if i need to block things like member data - or does this no longer apply because my site is above root?

Check this out by visiting member related URLs. If they are accessible via a browser, then the robots will pick up on them too.

Let me know if you have any other questions.

Cheers,