Basic Question re: Search Engine Bots
I've got what is I am sure a very basic question with an even more basic answer... however... I don't know it!
Is there a way to stop Search Engine Spiders/bots from crawling entire directories?
I know how to stop them on individual pages... but have thought that there must be an easier way if you don't want anything in a directory to be crawled.
I have found some info on a robots.txt file but I'm not sure on how I would construct it, where I'd put it or what! Would this be the way to go for what I'm looking for?
Thanx
