We tried to make our Shopify SEO agent break a store.
These 18 deterministic fixtures target mistakes that can turn an approved fix into merchant harm: duplicate schema, invented commerce facts, heading errors, rendering gaps, and crawler-policy mix-ups.
Specific failures. Specific checks.
Open a case to see the merchant risk and the exact behavior the test requires.
HTML and page classification
Empty H1 is not a missing H1
Risk: An empty H1 needs a different fix from a missing H1.
Required: Report the existing H1 as empty and do not claim it is absent.
PassedMissing H1 stays distinct
Risk: A page with no H1 needs one clear main heading.
Required: Report the H1 as missing.
PassedReadable and empty H1s are separated
Risk: Adding another H1 would worsen a page that already has a readable H1 plus an empty one.
Required: Keep the readable H1 and report the empty duplicate separately.
PassedHomepage product-signal leak
Risk: Theme-wide product metadata can leak onto a homepage and create an invalid Product patch.
Required: Do not propose Product schema when the page is identified as the homepage.
PassedJavaScript-only critical content
Risk: A browser-rendered page can look complete while critical initial HTML is empty.
Required: Report the initial-HTML blocker without claiming every AI system is unable to understand the site.
PassedStructured data and commerce facts
Existing Product schema
Risk: A second Product entity can create conflicting product facts.
Required: Suppress a Product proposal when Product JSON-LD already exists.
PassedExisting ProductGroup schema
Risk: ProductGroup already represents a product family and must not trigger a duplicate Product proposal.
Required: Suppress a Product proposal when ProductGroup JSON-LD already exists.
PassedExisting organization-level entity
Risk: Duplicate Organization entities can introduce conflicting brand facts.
Required: Suppress an Organization proposal when an organization-level entity already exists.
PassedUnknown currency is never guessed
Risk: Defaulting to USD can publish false pricing information for a non-US store.
Required: Require merchant input and never default an unknown currency to USD.
PassedUnknown availability is never guessed
Risk: Defaulting to InStock can publish false inventory information.
Required: Require merchant input and never default unknown availability to InStock.
PassedKnown GBP currency is preserved
Risk: A valid store currency must survive proposal generation unchanged.
Required: Preserve GBP from observed product metadata.
PassedMalformed JSON-LD
Risk: Malformed JSON-LD may be ignored by crawlers.
Required: Report the parse failure separately from schema presence.
PassedSchema absence claim boundary
Risk: Missing JSON-LD does not prove search or AI systems cannot understand a business.
Required: Recommend accurate markup without making an absolute invisibility claim.
PassedCanonical behavior
Canonical appears only after JavaScript
Risk: A JavaScript-added canonical can be delayed or conflict with fetched HTML.
Required: Report that the canonical is absent from the initial HTML.
PassedCanonical is missing
Risk: URL variants can split signals when no canonical is present.
Required: Report the canonical as missing.
PassedCrawler policy
Search crawler is blocked
Risk: Blocking a search crawler prevents search indexing and related AI search visibility.
Required: Classify a Googlebot homepage block as high severity.
PassedAnswer-time citation crawler is blocked
Risk: Blocking an answer-time citation crawler prevents that surface from fetching the page.
Required: Classify an OAI-SearchBot homepage block as high severity.
PassedTraining crawler is blocked
Risk: A training preference must not be mislabeled as a current search-indexing failure.
Required: Treat a GPTBot-only block as informational, not as a search blocker.
PassedA regression suite, not a trust shortcut.
What passed
The named deterministic rule produced its required result for the published synthetic fixture.
What it does not prove
It does not cover every theme, certify security, guarantee accuracy, or predict rankings, traffic, citations, or sales.
What still gates action
Live evidence, exact-store authorization, merchant approval, current provider state, execution support, and read-back verification remain separate.
Test the live page, not a promise.
Run one read-only audit before connecting Shopify or approving a change.
Audit my Shopify store