About MusingWebDesign-AgentPostureBot

Last updated: August 12, 2026

Musing Web Design operates an automated client that measures how public websites respond to automated visitors — AI training crawlers, AI shopping agents, and data harvesters. This page exists so that anyone who sees our requests in their server logs can find out exactly who we are, what we do, and how to stop us.

How to identify us

Every request we make identifies itself. Our user-agent string always begins with MusingWebDesign-AgentPostureBot and always contains a link back to this page. For example:

MusingWebDesign-AgentPostureBot/0.3 (+https://musingwebdesign.com/bot)

When we are testing whether a site has a rule about a specific third-party crawler, we say so honestly rather than pretending to be that crawler:

MusingWebDesign-AgentPostureBot/0.3 (probing-as: GPTBot; +https://musingwebdesign.com/bot)

We never impersonate Googlebot, GPTBot, ClaudeBot, PerplexityBot, or any other crawler. If a user-agent string claims to be one of those and does not name us, it is not us.

What we do

• Request a small number of public, unauthenticated pages — typically a homepage, robots.txt, llms.txt, a sitemap, and a few product pages.
• Record whether each request succeeded, and what kind of response came back.
• Record whether standard machine-readable files and structured data are present.
• Read sitemaps to find example product pages — the purpose sitemaps exist for.

What we do not do

• We do not store page content, product listings, prices, or catalog data. We record measurements, presence, and counts only.
• We do not collect personal data.
• We do not attempt to access anything behind a login, paywall, or authorization check.
• We do not try to get around blocks. If your server refuses our request, we record that refusal as our result and move on. We do not rotate user-agents, use proxy networks to disguise our origin, or retry in order to defeat a block.
• We do not attempt to log in, submit forms, place orders, or change anything on your site.

How we behave

• We honor robots.txt rules that apply to our own user-agent.
• We make roughly one request at a time per site, spaced about a second apart.
• A typical visit is fewer than 25 requests and is over in well under a minute.
• We visit infrequently — a given site is usually measured no more than monthly.

How to block us

Add the following to your robots.txt and we will stop:

User-agent: MusingWebDesign-AgentPostureBot
Disallow: /

How to opt out entirely

Email bot@musingwebdesign.com from an address at the domain in question, or from an address that can be verified as representing it, and we will add the domain to our permanent exclusion list within 48 hours. No explanation is required. Once excluded, we stop measuring the domain and remove it from any dataset we have not already delivered.

How we use what we measure

We use these measurements to advise site owners about their own sites, and to publish aggregate statistics about how online retail responds to automated clients.

In anything we publish publicly, figures are aggregated and individual domains are not named. We identify a specific domain only in a report delivered to that domain's own owner, or in a paid engagement for a client — and in the latter case only where doing so is lawful and consistent with our agreements.

Questions or complaints

Email bot@musingwebdesign.com. A real person reads it. If our bot has caused a problem on your site, tell us and we will fix it.

---

Musing Web Design · Waco, Texas