Skip to main content

Robots.txt Generator

Build a valid robots.txt with allow/disallow rules, multiple user-agents, sitemaps, and crawl-delay. Free.

100% client-side · no data leaves your browser
User-agent: *

About this tool

Create a correct robots.txt without memorizing the syntax. Add per-user-agent allow and disallow rules, list your sitemaps, and set an optional crawl-delay. The output updates live and can be copied or downloaded as robots.txt.

How to create a robots.txt file

Generate a search-engine robots.txt for your website.

  1. 1

    Add rules

    Set a user-agent and the paths to allow or disallow.

  2. 2

    Add sitemaps

    List your sitemap URLs and an optional crawl-delay.

  3. 3

    Download

    Copy or download robots.txt and place it at your site root.