Anmelden | Registrieren | FAQ
Anonymous

robots.txt

+ Antwort erstellen

6 Beiträge Seite 1 von 1


robots.txt

von iDana » Fr Mai 20, 2011 4:04 pm

Hello,

I know about robots.txt but can anyone give me any more information on it?

Thanks.
Benutzeravatar

iDana

  • Beiträge: 60
  • Registriert: Di Okt 26, 2010 10:05 am
--
iDana

Re: robots.txt

von iDana » Fr Mai 20, 2011 4:28 pm

Thank you!:)
Benutzeravatar

iDana

  • Beiträge: 60
  • Registriert: Di Okt 26, 2010 10:05 am
--
iDana

Re: robots.txt

von nigerianprince » Di Mai 24, 2011 7:53 pm

thank you as well! I was looking for robot.txt info before but the linked site is definitely a better resource than what i was referring to before
Benutzeravatar

nigerianprince

  • Beiträge: 1
  • Registriert: Di Mai 24, 2011 7:44 pm

Re: robots.txt

von markomako79 » Fr Aug 05, 2011 4:30 am

I added a sitemap to my wordpress blog and when I ran an page analyzer it told me that robots.txt was now present. Does this term have something to do with the sitemap, or is it totally unrelated. It would be great if someone could tell me if having robots.txt benefits my website when it comes to SEO?
Benutzeravatar

markomako79

  • Beiträge: 11
  • Registriert: Di Aug 02, 2011 6:08 pm
--
Creating a survival seeds website, and need some basic HTML help.

Re: robots.txt

von HTML.net » Fr Aug 05, 2011 6:48 am

Robots.txt is a file you can upload to your web site to guide search bots (Google etc.) how to index your site.

For example, if you write in a file robot.txt:

User-agent: *
Disallow: /

no page will be indexed.

You can read more at the link above: http://www.robotstxt.org

- Andreas, HTML.net
Benutzeravatar

HTML.net
Site Admin

  • Beiträge: 1194
  • Registriert: Sa Sep 11, 2010 11:24 am
--
Show some love for HTML.net on Twitter, Facebook and Google: Use the buttons on top of all pages.

Re: robots.txt

von Preeju M S » Fr Jun 09, 2017 4:22 am

Robots.txt file is a text file for restricting bots (robots, search engine crawlers ) from a website or certain pages on the website. Using a robots.txt file and with a disallow direction, we can restrict bots or search engine crawling program from websites and or from certain folders and files.
Benutzeravatar

Preeju M S

  • Beiträge: 6
  • Registriert: Do Jun 08, 2017 7:40 am
--
Data Entry Outsourcing Company - http://www.onlinedataentryoutsourcing.com


+ Antwort erstellen

Seite 1 von 1