robot.txt

what is robot.txt file?
waht is its content
and can i include in my site?

[cvvikram]

Check this site

http://www.searchengineworld.com/rob...s_tutorial.htm

OK BUT IS IT HARMFUL

[Deep]

Sometimes yes, specially when you deny some directories which you do not want other ppl to see...

for example...

u have denied directory name admin

so google wont scan but hacker or anything will check robots.txt and see what is denied and then try to play with it...

so if you are denying any directory then password protect that directory using .htaccess rule so that no one can play with it much..

Deep

and how to add a log file on the server to find out stats of site

and how to add a log file on the server to find out stats of site

[Deep]

can you be specific? I didnt get your question

[crashuniverse]

robots.txt also helps in bots to remove ur website totally from their indexing so dat ur website doesn't come in search results.

[sting]

Originally Posted by nik_for_you

and how to add a log file on the server to find out stats of site

U mean the ur site statistics about your visitors ? if yes try this http://allcount.com/

i mean i heard many people talking that googlebot came to my site from seeing the log file . so i m talking about that log file..

[Deep]

hmm no google bot cannot scan any file unless you have linked it anywhere on the page...

Deep

ok .. and allcount.com is fine . i used it.. is ther any other way i can record all stats on my site in text file

[debiprasad_sahoo]

Originally Posted by cvvikram

Check this site

http://www.searchengineworld.com/rob...s_tutorial.htm

The page does not exist. Please check.

[Charan]

WTH!!!! you bumped a two year old thread

reported...