|
I have a section on one of my sites that I set up for testing. When I did it I forgot to block the bots and I ended up with my test stuff in the bing index. I later changed out the robots.txt but bing kept on coming back and ignored the robots file. So, I changed out the name of the folder and put an htpasswd in front of it.
Bing came by for months, multiple times a day, trying to get into the original section even though it was now getting 404 on everything. It kept that up for a good long while, I think at least a year before I saw it slack off.
|