Create a robots.txt File
Generate a Valid robots.txt for Your Site
Need to use Create a robots.txt File right now?
Builds a correctly formatted robots.txt controlling crawler access, without needing to memorize the directive syntax.
robots.txt
User-agent: * Disallow: /admin/
Features
- Runs entirely in your browser
- Privacy-first — your data is never uploaded
- Real-time, instant results
- 100% free, no sign-up required
- Works on desktop, tablet, and mobile
- No installation needed
Who uses this tool?
About Create a robots.txt File
robots.txt tells search engine crawlers which parts of your site they should and shouldn't access, and it has to sit at your domain's root with exact syntax — a typo in the directive name or a misplaced rule can accidentally block your entire site from search engines, or fail to block the pages you actually wanted hidden. This tool builds the file through a simple form instead of hand-editing plain text.
Add as many User-agent/Disallow rule pairs as you need, include any extra lines for Allow exceptions or crawl-delay directives, and specify your sitemap URL — the tool assembles everything into correctly formatted, ready-to-deploy robots.txt content. Multiple rules for different crawlers (like blocking one bot while allowing others) are fully supported.
This matters most for anyone managing a site without a CMS plugin generating this automatically: staging environments you want to keep out of search results, admin sections that shouldn't be crawled, or simply making sure your sitemap is being pointed to correctly so search engines discover your pages faster.
How it works
- Add User-agent rules. Specify which crawlers and which paths to disallow.
- Add your sitemap URL. Helps search engines discover all your pages.
- Copy the generated file. Save as robots.txt at your site's root.
Examples
Blocking an admin section
Input
User-agent: *, Disallow: /admin/
Output
User-agent: *\nDisallow: /admin/\n\nSitemap: https://example.com/sitemap.xml