# About the Robots Txt API

With the Robots Txt API, Wix users can define which crawlers are allowed to access and index their website content. This API allows you to control search engine visibility, while ensuring proper SEO management and content protection.

You can view the site's `robots.txt` file by adding `/robots.txt` to your root domain. For example: https://www.gardencenter.com/robots.txt.

[Learn more](fallback::https://support.wix.com/en/article/editing-your-sites-robotstxt-file) about `robots.txt` in Wix.

The Robots Txt API allows you to:

- [Retrieve](fallback::https://dev.wix.com/docs/rest/business-management/marketing/txt-file-server/robots-txt/get-robots-txt) an existing `robots.txt` file.
- [Update](fallback::https://dev.wix.com/docs/rest/business-management/marketing/txt-file-server/robots-txt/update-robots-txt) the `robots.txt` file.

## Terminology

- **robots.txt**: A simple text file placed in the root directory of a website, that provides instructions to web crawlers about which parts of the site they should access.


@sdk_package_setup

Coding Examples

Sled Tests

Go Headless

Develop Websites (SDK)

Build Apps

Client (Deprecated)

API Preview

Wix CLI

Ricos

Develop Websites

API Reference

Velo

Picasso

GraphQL (Alpha)

ID of the Wix account associated with the event.

accountId

ID of the parent Wix account. Only included when accountId belongs to a child account.

parentAccountId

ID of the Wix site associated with the event. Only included when the event is tied to a specific site.

siteId

AccountInfo

identityType

IdentificationData

WebhookIdentityType

ID of a site visitor that has not logged in to the site.

anonymousVisitorId

appId

ID of a site visitor that has logged in to the site.

memberId

ID of a Wix user (site owner, contributor, etc.).

wixUserId

data

eventType

The identification type and identity data.

instanceId

MessageEnvelope

The request to get the Robots.txt file content

Subdomain of the requested `robots.txt file`. For example, `www`, `es`, `fr`. Default: `www`.

subdomain

GetRobotsTxtRequest

The response of the Robots.txt file request

GetRobotsTxtResponse

Represents a `robots.txt` file entity that controls web crawler access to a website or subdomain. It defines crawling rules and directives for search engines and other automated agents, ensuring proper indexing behavior and preventing unauthorized access to specific site areas.

Full text content of the `robots.txt` file.

content

Whether this `robots.txt` file uses Wix's [default content](https://support.wix.com/en/article/editing-your-sites-robotstxt-file).

default

If applicable, target subdomain for the `robots.txt` file. For example, a site can have multiple subdomains like `www.example.com`, `es.example.com`, `fr.example.com`. Each subdomain can have its own `robots.txt` file with different rules. Default: 'www'

RobotsTxt

The request to update the content of the Robots.txt file

UpdateRobotsTxtRequest

UpdateRobotsTxtResponse

value

BoolValue

Int64Value

values

ListValue

NullValue

StringValue

fields

Struct

nanos

seconds

Timestamp

Value

boolValue

listValue

nullValue

numberValue

stringValue

structValue

kind

bodyAsJson

ActionEvent

A sequence number that indicates the order of updates to an entity. For example, if an entity was updated at 16:00 and then again at 16:01, the second update will always have a higher sequence number.
You can use this number to make sure you're handling updates in the right order. Just save the latest sequence number on your end and compare it to the one in each new message. If the new message has an older (lower) number, you can safely ignore it.

entityEventSequence

Fully Qualified Domain Name of an entity. This is a unique identifier assigned to the API main business entities.
For example, `wix.stores.catalog.product`, `wix.bookings.session`, `wix.payments.transaction`.

entityFqdn

ID of the entity associated with the event.

entityId

Event timestamp in [ISO-8601](https://en.wikipedia.org/wiki/ISO_8601) format and UTC time. For example, `2020-04-26T13:57:50.699Z`.

eventTime

Event ID. With this ID you can easily spot duplicated events and ignore them.

If present, indicates the action that triggered the event.

originatedFrom

Event action name, placed at the top level to make it easier for users to dispatch messages.
For example: `created`/`updated`/`deleted`/`started`/`completed`/`email_opened`.

slug

Whether the event was triggered as a result of a privacy regulation application
(for example, GDPR).

triggeredByAnonymizeRequest

DomainEvent

body

entityAsJson

Indicates the event was triggered by a restore-from-trashbin operation for a previously deleted entity

EntityCreatedEvent

deletedEntityAsJson

EntityDeletedEvent

Since platformized APIs only expose PATCH and not PUT we can't assume that the fields sent from the client are the actual diff.
This means that to generate a list of changed fields (as opposed to sent fields) one needs to traverse both objects.
We don't want to impose this on all developers and so we leave this traversal to the notification recipients which need it.

currentEntityAsJson

EntityUpdatedEvent

deletedDate

RestoreInfo

Updates the `robots.txt` file entity.

The content can be set to an empty string to create a blank file.

To restore Wix's default content, set the `default` field to `true` and omit the `content` field.

Update robots.txt

UpdateRobotsTxt

import { robotsTxt } from '@wix/seo';

async function updateRobotsTxt(options) {
  const response = await robotsTxt.updateRobotsTxt(options);
};

updateRobotsTxt

import { robotsTxt } from '@wix/seo';
import { auth } from '@wix/essentials';

async function myUpdateRobotsTxtMethod(options) {
  const elevatedUpdateRobotsTxt = auth.elevate(robotsTxt.updateRobotsTxt);
  const response = await elevatedUpdateRobotsTxt(options);
}

updateRobotsTxt (with elevated permissions)

import { createClient } from '@wix/sdk';
import { robotsTxt } from '@wix/seo';
// Import the auth strategy for the relevant access type
// Import the relevant host module if needed

const myWixClient = createClient ({
  modules: { robotsTxt },
  // Include the auth strategy and host as relevant
});


async function updateRobotsTxt(options) {
  const response = await myWixClient.robotsTxt.updateRobotsTxt(options);
};

Self-hosted SDK calls require you to [create a client](https://dev.wix.com/docs/sdk/articles/work-with-the-sdk/about-the-wix-client).

updateRobotsTxt (self-hosted)

Triggered when the `robots.txt` file entity is updated.

Robots Txt Updated

Get robots.txt file

GetRobotsTxt

View SEO Settings

import { robotsTxt } from '@wix/seo';

async function getRobotsTxt(options) {
  const response = await robotsTxt.getRobotsTxt(options);
};

getRobotsTxt

import { createClient } from '@wix/sdk';
import { robotsTxt } from '@wix/seo';
// Import the auth strategy for the relevant access type
// Import the relevant host module if needed

const myWixClient = createClient ({
  modules: { robotsTxt },
  // Include the auth strategy and host as relevant
});


async function getRobotsTxt(options) {
  const response = await myWixClient.robotsTxt.getRobotsTxt(options);
};

getRobotsTxt (self-hosted)

User-agent: *
Allow: /

# Optimization for Google Ads Bot
User-agent: AdsBot-Google-Mobile
User-agent: AdsBot-Google
Disallow: /_api/*
Disallow: /_partials*
Disallow: /pro-gallery-webapp/v1/galleries/*

Robots Txt

RobotsTxtUpdated

wix-seo.read

import { robotsTxt } from '@wix/seo';

robotsTxt.onRobotsTxtUpdated((event) => {
    // handle your event here
});

onRobotsTxtUpdated

import { createClient, AppStrategy } from '@wix/sdk';
import { robotsTxt } from '@wix/seo';

const wixClient = createClient({
  auth: AppStrategy({
    appId: 'MY-APP-ID',
    publicKey: 'YOUR_APP_PUBLIC_KEY',
  }),
  modules: {
      robotsTxt,
    },
});

wixClient.robotsTxt.onRobotsTxtUpdated((event) => {
    // handle your event here
});

Update Robots Txt