logitly.ai

Bot & Agent Policy

Last updated: October 23, 2026

This page describes how our AI agents access the web, how to verify that traffic really comes from us, and how website owners can control or block our access.

1. About Our Agents

Logitly AI provides AI data services. Traffic from us comes from AI agents, which may visit publicly accessible web pages while performing tasks. We use the results of this work to build, test, and improve datasets, benchmarks, and AI agents.

Our agents are designed to access only publicly accessible pages. They are not designed to log in to accounts, get past paywalls or CAPTCHAs, or get around any other access controls.

We do not run a search engine or a public archive, and we do not republish the web pages our agents visit. Before we release or deliver any dataset or benchmark, we filter personal and sensitive data from it (see our Privacy Policy).

2. How Our Agents Access the Web

All of our agents run in isolated sandboxes. Every outbound request from a sandbox goes through network infrastructure that we control. Agents can fetch web content in two ways:

Registered tools. These are web-access tools that we build and maintain. Requests from them identify themselves with this user agent:

Mozilla/5.0 (compatible; LogitlyAI/1.0; +https://www.logitly.ai/bot)

Standard command-line tools. Agents may also use common tools available in the sandbox, such as curl or wget. These requests may not carry our user agent string; they may show the tool's default user agent instead (for example, curl/8.x). To identify this traffic, use the method in Section 3.

Our registered tools check robots.txt before fetching; standard command-line tools do not.

By default, our agents make no more than 2 requests per second to any single domain. If you see unusually high activity from our IP addresses, contact us at the address in Section 5 and we will look into it.

3. Verifying Our Traffic

All traffic from our agents comes from IP addresses we control. To verify a request:

  1. Run a reverse DNS lookup on the IP address. The hostname should end in .logitly.ai.
  2. Run a forward DNS lookup on that hostname. It should return the original IP address.

If a request claims to be from us but fails this check, it is not from us. Please report it to the address in Section 5.

4. Opting Out

By robots.txt. To block us from your entire site, add the following to your robots.txt file:

User-agent: LogitlyAI
Disallow: /

By firewall. Command-line tools do not read robots.txt, so to stop that traffic, block our IP ranges on your server or firewall.

By email. If you can't edit your robots.txt file, email us at the address in Section 5 with your domain name. We will process your opt-out within 10 business days.

Content we have already collected. Opting out stops future access. To ask us to remove content we have already collected, see our DMCA & Content Removal Policy. For personal data, see our Privacy Policy.

5. Contact

For agent questions, including opt-out requests, rate concerns, and reports of traffic falsely claiming to be from us, email admin@logitly.ai.