Origin Story and AI's Impact on Web Interaction 00:15
- Scrunch AI was founded after recognizing the disruptive potential of LLMs on how people interact with businesses online.
- Early on, businesses mainly wanted chatbot plugins, but the founders realized most consumers don't want to engage with site-specific chatbots.
- The core insight was that AI is fundamentally altering online discovery and customer journeys, not just traditional SEO.
Monitoring AI Search and Content Discovery 06:52
- Scrunch monitors how major AI platforms (ChatGPT, Gemini, Perplexity, Claude, Meta AI) surface and reference businesses in their search and conversational interfaces.
- Focus is on platforms with significant user adoption; the coverage will expand as new platforms emerge.
- Many brands are more concerned with AI search features (search box enabled) because user actions with commercial intent increasingly involve search, which pulls real-time or recent web content.
- For pre-training crawlers, brands can control what is exposed, particularly avoiding including sensitive or short-lived promotion info.
Web Content, Paywalls, and the AI Content Ecosystem 09:53
- There is growing debate and experimentation around compensating content creators for AI's use of their data (e.g., Cloudflare's paywall for bots).
- Businesses must balance content restrictions with their need for exposure and traffic via AI/search engines.
- New browsing paradigms (AI browsers, agent-based retrieval) are emerging, making it harder to entirely block LLMs from consuming web content.
- The shift is not just AI replacing search but AI potentially replacing much of traditional web browsing for many user intents.
How AI Search Works & Optimization Challenges 16:04
- Most AI search platforms still use traditional ranking algorithms (BM25, PageRank, etc.) before applying AI-based re-ranking and summarization.
- Traditional SEO practices continue to matter, but effective re-ranking depends on having descriptive, relevant page metadata and content.
- Structurally clear, informative pages perform better in AI-generated responses than those optimized just for human clickthrough or visuals.
Strategies for Content Exposure and Personalization 19:59
- Businesses must decide how much product/context detail to publish for AI consumption—more detail improves discovery and inference by AI models.
- AI search does not yet pass user query data to destination sites, mainly for privacy reasons, but inbound traffic analysis can guide content development.
- Real-world example: Monitoring what topics or products inbound AI-driven visitors engage with helps refine content for future queries.
Hallucinations, Content Clarity, and SEO Evolution 22:42
- AI models sometimes hallucinate features or information; this can reveal demand for non-existent features, which can guide product/content development.
- Clear, straightforward descriptions of offerings on product and homepage pages are crucial, especially for startups and B2B SaaS providers.
- There is growing attention on how different AI systems re-rank and select content, akin to historical SEO tracking of Google algorithm updates.
Naming the Space and the 'Agent Experience' Era 26:35
- The field is sometimes termed AEO (AI Engine Optimization) or GEO (Generative Engine Optimization), but Scrunch prefers to frame it as “agent experience.”
- Unlike classic SEO, the goal expands from being discovered to managing the full AI-mediated user journey, as more customer actions complete within AI tools.
Personalization, Prompt Clusters, and Use Case Observations 29:04
- Personalization and user profiles (memories, preferences) already influence AI search results and content selection, with increasing effect.
- Common high-value prompt clusters: coding, shopping, B2B software comparisons, and broader problem-solving rather than classic “search” intent.
- Many business users leverage AI to directly solve problems (“do the thing for me”), which is higher intent than information gathering alone.
Tactics: Programmatic Content, Technical Dos and Don'ts 39:00
- LLMs and AI search engines do not execute JavaScript when indexing; ensure key content is server-rendered for inclusion.
- File formats like LLM.txt (or similar) are not yet major factors in discoverability—traditional, well-structured HTML content remains primary.
- Over-focusing on embeddings and “semantic” hacks yields little benefit for exposure; clarity and structure are more effective.
- Programmatic SEO/content generation is widely used; it works best if content brings unique, helpful insights rather than just regurgitated web material.
- Effective cases include turning support tickets into structured how-to content for AI and users.
Case Studies and Uplifts 47:30
- Clerk (developer authentication) saw a 6x increase in AI-driven traffic and a 9x increase in conversions by optimizing docs and content for AI platforms.
- Scrunch acts as a feedback/experimentation tool rather than a content generation service, supporting customers to iterate based on data.
- Typical uplift for actively engaged clients is double-digit percentage traffic growth within 1-2 months.
Market Share and Platform Dynamics 51:38
- ChatGPT is the dominant AI platform in terms of traffic, engagement, and brand relationship, with AI overviews (Google, Bing) gaining exposure.
- Perplexity follows as a strong, niche favorite, especially in tech/early adopter circles, though still smaller in volume.
- Meta AI and Claude have passionate user bases but lower market share compared to ChatGPT.
- Market shares are volatile; new entrants can spike in usage but ChatGPT has had the most enduring traction.
Best Practices and Closing Thoughts 55:05
- The web is rapidly shifting toward AI as a primary interface; being “AI compatible” is increasingly critical for exposure and conversion.
- Avoid “snake oil” solutions; focus on clear, honest, well-structured content that helps both humans and AI agents.
- Scrunch is actively hiring, especially for those interested in AI/search/web intersections.