Books reimagined: use AI to create new experiences for things you know — Lukasz Gandecki, Xolvio

Project Background and Motivation 00:00

  • Lukasz Gandecki introduces himself and his background in programming
  • The project "Books Reimagined" began when reading a complex book about Donald Trump; confusion about characters prompted him to build an AI companion app
  • Initial AI tool provided summaries and images for characters within the book's context to aid understanding

Demonstrations of AI-enhanced Book Experiences 01:21

  • First experience created was with "The Snow Queen," featuring music, scene narration, and atmosphere, though initially only in Polish
  • A new English-version experience was built for the conference, based on "1984"
  • Users can send a voice note to the book and get contextual answers about specific scenes
  • The AI responds almost instantly (within 100 milliseconds) upon voice note completion, with smooth interaction and accurate scene summaries
  • Users can ask what happened since their previous question and receive a summary covering the intervening content

Enhanced Book Search and Deep Analysis 03:34

  • Traditional search (exact term match) is not effective for remembering scenes or details in books
  • Embeddings-based semantic search enables users to find scenes by their memory of the content, such as "where Winston met O'Brien"
  • Deep research feature allows users to ask complex questions, prompting the AI to analyze the book up to the current reading point and provide a synthesized answer

Iterative and AI-driven Development Workflow 04:38

  • The early development approach favored quickly iterating ideas by using AI rather than over-planning upfront
  • Throwing away AI-generated code is psychologically easier, supporting rapid prototyping
  • Describes a process of iterative "waves" of rewriting that become less drastic over time, allowing eventual transition to traditional engineering practices like adding tests and refactoring
  • Suggests focusing refactoring efforts on code that is both bad and painful to maintain, not just code that is merely poorly written or unattractive

Hiding AI Complexity and Enhancing Human Touch 06:24

  • Many existing AI book experiences are simple wrappers or have mediocre voice assistants
  • The team's approach is to obscure AI from the user, using it for first drafts and letting humans polish the final product
  • Human oversight ensures quality in music, graphics, and matching avatars to characters, as AI cannot fully assess these aspects

Magical Reading Experiences: Benefits and Novelty 07:10

  • Combining simple building blocks—such as music, visuals, and context-aware answers—creates a unique and immersive reading experience
  • Features like 24/7 “all-knowing” assistant, spoiler-free natural language search, and mood-matching music make books more engaging and cinematic
  • Such experiences were previously impossible due to high costs of custom content for each book; AI lowers these barriers

Technical Process and Open-Source Release 08:28

  • The workflow uses LLMs for scene and character analysis, mood detection for music, and generation of structured XML metadata for book content
  • AI automates mapping characters, scenes, and moods across books, streamlining the production of rich content
  • The book experience player is being open sourced, allowing broader creation of Netflix-style interactive books
  • Invitation to collaborate for those interested in more compelling, “magical” AI book experiences beyond chatbots

Closing and Contact Information 09:31

  • Lukasz Gandecki thanks the audience and provides contact details for further discussion about AI-enhanced book technologies