Robots.txt Generator
Create a robots.txt file to guide search engine crawlers, with your sitemap link.
How to use the Robots.txt Generator
- 1Choose what to allowAllow crawling for normal websites.
- 2List folders to blockAdmin, cart, checkout and similar.
- 3Add your sitemap URLThen download robots.txt and upload it to public_html.
What is the Robots.txt Generator?
robots.txt is a small file in your website's root that tells search engine crawlers which areas they may visit. A correct file keeps bots out of admin pages and carts, and points them to your sitemap.
This generator builds a clean robots.txt with folders to block, an optional WordPress rule, an option to block AI crawlers and your sitemap link.
Choose what bots may crawl, add folders to block and your sitemap URL, then copy or download robots.txt.
When to use it
- New websites.
- WordPress sites.
- Stopping crawlers from wasting time on private pages.
Questions
Does robots.txt hide pages from Google?
It stops crawling, but a blocked page can still appear if other sites link to it. Use a noindex tag to keep a page out of results.
Is the Robots.txt Generator free?
Yes. It is completely free with no sign-up and no watermark. Use it as often as you like.
Are my files uploaded to your server?
No. This tool runs inside your browser, so your files and text stay on your own phone or computer.
Does it work on mobile?
Yes, it works in Chrome, Safari and other modern browsers on Android, iPhone, Windows and Mac.
Need the full premium tools?
Get Ahrefs, Semrush, ChatGPT Plus, Canva Pro and 50+ more premium tools at group-buy prices. Pay with bKash, Nagad, Rocket or card and get access in minutes.
Ahrefs$7.38/30 Days
semrush$2.45/30 Days
ubersuggest$2.70/30 Days
brightlocal$3.69/30 Days
neuronwriter$3.69/30 Days
morningscore$2.46/30 Days