WordPress.org

Plugin Directory

Dark Visitors – AI Agent Analytics (Bot Traffic), LLM Referral Tracking, Automatic Robots.txt

Dark Visitors – AI Agent Analytics (Bot Traffic), LLM Referral Tracking, Automatic Robots.txt

Description

For full details, visit the Dark Visitors website.

Track AI agents, bot traffic, and LLM referrals

Bots now make up half of your website’s traffic. Harness the activity of AI agents, crawlers, scrapers, and LLM assistants to grow your business and protect your content from unwanted bot access.

1. Agent Analytics

Get realtime visibility into the hidden ecosystem of crawlers, scrapers, AI agents, and other bots browsing your website.

  • Monitor traffic from crawlers and scrapers: Track the hidden activity of all known artificial agents visiting your website in real time.
  • Capture sessions from browser-using AI agents: Analyze the behavior of AI models autonomously navigating and interacting with your website. Optimize their experience to increase conversions and human referrals.
  • Spot spoofed visits with agent verification: Detect bad actors pretending to be legitimate agents.
  • Get spike alert notifications: Get notified when an agent’s traffic starts surging or overloading your server.
  • Identify top agents and pages: Understand which agents visit your website the most, and which pages they’re most interested in.
  • Export reports to CSV: Download agent activity reports for custom analysis.

2. LLM Referral Tracking

Measure human traffic coming from AI chat and search platforms like ChatGPT, Perplexity, Claude, Gemini, and Copilot.

  • Find out which AI platforms are mentioning you: See which LLMs are including or recommending your website in their responses.
  • Export reports to CSV: Download LLM referral reports for custom analysis.

3. Automatic Robots.txt

Protect sensitive content from unwanted scraping with a continuously updating robots.txt.

  • Generate your robots.txt automatically: Opt out of LLM training by generating a robots.txt that blocks new agents automatically. No need to make manual updates.
  • Block agents who try to break your rules: Block badly-behaving agents from accessing your website.

Demos

Demos can be found on the home page.

Further Details

This plugin connects your website to Dark Visitors. It uses the Dark Visitors API to connect to your account and provide functionality.

By signing up, you agree to the Terms of Service and Privacy Policy.

Are you finding this plugin helpful? Please give us a ★★★★★ review!

Screenshots

  • Insights into crawler and scraper activity
  • Insights into AI agent activity
  • The realtime timeline of all agent visits to the website
  • Insights into AI chat and search LLM referrals
  • Insights into blocked agents who tried to ignore robots.txt rules

Installation

  1. Install and activate the Dark Visitors plugin from the WordPress Plugin Directory
  2. Sign up for Dark Visitors and create a new project for your website.
  3. Copy your access token from the project’s settings page, then paste it into the access token field on the plugin’s configuration page.

Your website is now connected to Dark Visitors.

FAQ

Do I need a Dark Visitors account to use this plugin?

Yes. Creating an account is free and only takes a few seconds. This is where you’ll be able to see dashboards and insights into agent activity on your website.

Is Dark Visitors free?

Dark Visitors is a freemium service. The free tier has an extremely generous event limit that’s more than enough for 99% of websites. You can choose to upgrade if you want to unlock premium features or need events beyond the free tier. More details can be found on the pricing page.

What are some examples of agents (bots)?

Examples include Operator, GPTBot, CCBot, Claudebot, PerplexityBot, Googlebot, Bytespider, etc. See the full agent list for more.

Why should I care about visits from agents?

There are many reasons to care. Their activity affects:

  • Whether your content is used in AI model training
  • The success rate of autonomous AI agents executing tasks for human users (e.g. making purchases)
  • If potential users can use AI assistants to analyze and ask questions about your website
  • How your pages appear in search engine results
  • Server resource usage
  • Etc.

With ongoing advancements in AI, their activity is becoming more meaningful every day.

Is my website really being visited by agents?

Almost certainly. In fact, about half of your traffic is likely coming from them right now. This has been observed across websites of all categories and sizes.

Reviews

November 3, 2024 5 replies
Bingbot and googlebot are hitting my site with hundreds of visits per minute and Gavin from support spotted the problem immediately!
August 21, 2024
An easy improvement would be to link to the settings page from the plugin entry like other plugins, other than that, in general this plugin and its online dashboard is generally well thought out and useful. My website is small and has limited resources that get overrun by crawlers making it unavailable. This plugin helps identify the culprits and decide if I should block them or not. For free you can block them in robots.txt, as I have a physical file, the plugin won’t touch that but its easy to add manually. It will report those crawlers that ignore the robots file and the paid version will enforce those rules, but I just update “.htaccess” myself. I’m reducing my rating by a star since you won’t get much “Agent Analytics” (per month) and for my tiny site it quickly ran out, we are a NFP so don’t have money to burn on subscriptions. The the description for the plan incorrectly states “Unlimited Agent Analytics” when the plan only includes 100,000 events.
July 29, 2024 1 reply
Please remind me in a month to provide an update on the performance of this plugin on a heavily visited website.
June 9, 2024
This is a great plugin. I had no idea how many bots were visiting my website every day. Super interesting insights, and a handy way to keep my robots.txt up to date. Thanks!
Read all 5 reviews

Contributors & Developers

“Dark Visitors – AI Agent Analytics (Bot Traffic), LLM Referral Tracking, Automatic Robots.txt” is open source software. The following people have contributed to this plugin.

Contributors

Changelog

1.20.0

  • Screenshot and copy updates

1.19.0

  • Code, design, screenshot, and copy updates

1.18.0

  • Code updates

1.17.0

  • Code updates

1.16.0

  • Code updates

1.15.0

  • Code updates

1.14.0

  • Code updates

1.13.0

  • Code updates

1.12.0

  • Code updates

1.11.0

  • Code updates

1.10.0

  • Code updates

1.9.0

  • Code updates

1.8.0

  • Copy changes

1.7.0

  • Design updates

1.6.0

  • Code cleanup

1.5.0

  • Simplified the analytics code

1.4.0

  • Added the option to block agents

1.3.0

  • Added a small edge case fix

1.2.0

  • Updated copy

1.1.0

  • Added the ability to disable analytics

1.0.0

  • Initial release