How agents broke app-level infrastructure - Evan Boyle

Introduction to AI Infrastructure Challenges 00:03

• Evan Boyle introduces the topic of how agents affect app-level infrastructure, particularly focusing on the compute layer.
• He shares his background as the founder of Genisx and experience with cloud and developer tools.

The Evolution of AI Engineering 00:15

• Engineers start with simple prompts, leading to successful funding rounds but eventually face complex workflows.
• Transitioning from AI engineering to data engineering is necessary due to the challenges of context management.

Infrastructure Limitations 01:10

• Traditional web infrastructure is ill-suited for modern AI applications which often require longer response times.
• Current AI applications can have latency of a few seconds, unlike the sub-second response times in web 2.0.

Workflow Management Challenges 03:36

• Users frequently encounter issues due to infrastructure limitations, leading to unreliable applications.
• Existing tools for long-running workflows often require complex setups that many developers find cumbersome.

Serverless Limitations 04:09

• Current serverless providers impose timeouts and limitations on long-running processes, complicating user experience.
• The importance of resumability in applications is highlighted, especially when users refresh or navigate away from the page.

User Experience in AI Applications 04:48

• Use cases demonstrate the need for background processing to avoid user drop-off during long tasks.
• The necessity of keeping users informed about task progress to enhance engagement and satisfaction is emphasized.

Development of an Open Source Library 07:28

• Evan discusses the creation of a component model for building workflows that are infrastructure-aware and reusable.
• The anti-framework design promotes composition and reusability over abstraction.

Workflow Execution and Debugging 09:29

• Workflows can automatically create REST APIs that support both synchronous and asynchronous calls.
• Detailed tracing and error handling are integrated, allowing developers to debug easily and manage retries.

Architectural Considerations 10:04

• Separation of API and compute layers is essential for scalability and flexibility in handling workflows.
• Use of Redis streams allows for resumability and easier management of user interactions with workflows.

Lessons for Building Agentic Workflows 12:11

• Start simple and plan for long-running processes without overcomplicating initial designs.
• Emphasizes the importance of error handling, user experience, and the intricacies of deploying workflows.

Conclusion and Resources 13:14

• Evan encourages developers to explore GenSx on GitHub for pre-built infrastructure solutions.
• He concludes with gratitude for the opportunity to present and wishes the audience success in their projects.