The speaker, founder of Lexica, emphasizes that demos are among the most important things today, especially for generative models.
Curiosity is presented as the primary driver for bringing ideas from the future into the present, surfacing as a subconscious feeling.
Creating demos is a process that often begins with an exciting idea, followed by challenges and iterations, leading eventually to a sense of pride once operational.
Demos serve as a tool to explore the possibilities of models, as true understanding often comes only through interaction.
The best demos come from following individual curiosity rather than set objectives.
GPT-3's release in 2020 was described as magical, given its novel capabilities, but it was expensive ($75 per million output tokens) and subject to usage restrictions.
The ability to create immediate feedback interfaces (inspired by Brett Victor) enabled more dynamic interaction with code, even when dealing with small context windows.
The progression from GPT-3's 2,000 token context to today’s models with much larger contexts makes today's possibilities even more remarkable.
Some demos required creative solutions, like splitting outputs across multiple prompts and joining them to overcome model limitations.
The expanding context window from 2,000 to 4,000 tokens enabled more ambitious demos, such as automated agents attempting tasks like buying AirPods online.
Feeding entire web pages exceeded context limits, so custom HTML parsers were built to condense pages into essential elements—though results were imperfect.
These experiments suggested models pre-trained on web text possessed latent "agency," a novel insight for the time.
Newer demos, such as a basketball shot tracker using Gemini 2.5 Pro, illustrate evolving capabilities, like video analysis and personalized feedback.
Real-world applications are expanding, with users achieving results equivalent to specialized human coaching using AI-driven feedback.
The speaker argues much untapped potential remains: even freezing current model weights could support new products for years, based solely on unexplored use cases.
Philosophies on AI Engineering and Exploration 06:35
Quoting Richard Hamming, the talk draws a distinction between traditional, goal-oriented engineering and the exploratory nature of AI engineering.
AI engineering is compared to excavation, where demos act as tools to unearth hidden capabilities, guided by curiosity.
Many capabilities remain unknown even to model creators, as evidenced by OpenAI staff discovering new model abilities through user demos.
Historical anecdotes, like Darwin's eight years studying barnacles, underscore that groundbreaking progress often comes from seemingly unimportant or lengthy exploration.
The Value and Responsibility of Sharing Demos 08:18
Today’s expanded model capacities—up to a million-token context window—offer unprecedented opportunities, but progress relies on iterative, curiosity-driven experimentation.
Progress is compared to crossing a foggy pond: each step reveals the next, but the full path is unpredictable.
Good demos reveal hidden abilities in models, often found by prioritizing exploration over certainty.
The unique perspective of each individual, shaped by their distinct experiences, means anyone could uncover novel applications.
Referencing Licklider's "man-machine symbiosis" paper from the 1960s, the speaker highlights how past pioneers could only dream of the computational power available today.
The speaker suggests a moral responsibility to honor that legacy by using and sharing the capabilities of modern AI, advancing the entire field.
Building and publicizing demos democratizes discovery and ensures progress is shared, not stifled.
Attendees reflect on the effectiveness of demos, with one noting AI coaching rivaled costly human trainers in providing feedback for running form.
Additional early demos, such as a simple banking app created shortly after GPT-3’s release, are shown to highlight the evolution from text completion to reasoning assistance.
Early experiences required working with base models and creative prompt engineering due to lack of instruct models.
The speaker recounts the frustration and motivation behind demos: that powerful capabilities exist largely undiscovered.
There's encouragement to revisit unrealized ideas from computing pioneers, as our current technology makes many of them practical.
The "multivac" demo illustrates breaking down complex problems using visual interfaces—suggesting that today's models are better viewed as reasoning assistants, not just text generators.
The talk ends with a call to action for attendees to build something new with existing technology and share it widely.