Question:
If you have a robots.txt file and a meta tag within an HTML web page, that contradict each other, which has precedence?
Answer:
If there is a conflict between the directive in robots.txt and the meta tag directive (e.g. if your robots.txt allows spidering but your page meta tag does not) the robots.txt directive has precedence.
great post as usual!
Good dispatch and this enter helped me alot in my college assignement. Say thank you you for your information.
What a great resource!