SUMM

Obie Fernandez, principal engineer at Shopify, introduces himself and his focus on augmented engineering, using AI to improve developer experience
Shopify is one of the largest Ruby on Rails organizations, working on their main application for nearly 20 years and managing millions of lines of code across about 5,000 repositories
The company processes roughly half a million pull requests per year, creating significant scale-related challenges for implementing AI

Two main AI approaches are discussed: agentic tools (ideal for adaptive, exploratory, and ambiguous tasks needing reasoning and autonomy) and structured workflow orchestration (ideal for predictable, repeatable tasks)
Agentic tools like Claude Code excel at debugging, ongoing adaptation, and iteration, while structured workflows are suited for tasks like codebase migrations or targeted refactoring
Combining both approaches provides flexibility, as some tasks benefit from nondeterministic agents and others from deterministic workflows

ROAST is an open source structured workflow orchestration tool developed at Shopify, particularly helpful for repeatable engineering tasks
Implemented in Ruby, ROAST links prompt-oriented tasks with conventional scripts, supporting integration across different platforms or tools
Initially developed to automate test generation and optimization for Shopify’s large codebase with over half a million tests
Cloud Code is heavily used, with about 500 daily active users and up to 250,000 requests per second at peak usage in Shopify

Shopify has a culture of tinkering and experimentation, further amplified by the rise of AI tools; this led to hundreds of custom workflow implementations across the company
ROAST was created to standardize and support cross-organizational needs for workflow orchestration, addressing the issue of redundant solutions
ROAST and Cloud Code have a bidirectional integration: ROAST can call Cloud Code for tasks, and Cloud Code workflows can invoke ROAST as a tool

Combining nondeterministic and deterministic workflows minimizes the accumulation of errors and inefficiencies in multi-step processes
Breaking large workflows into smaller, well-defined steps and limiting the task scope for agents improves accuracy and reliability
For example, in optimizing test suites, ROAST manages grading and divides work so that Claude Code handles just the necessary components
For dynamic language tasks (like introducing typing to Ruby code), workflows can consistently manage tool invocations and hand results to AI agents for targeted assistance

Enables workflow caching and session saving, which saves time during debugging and development by allowing partial workflows to be rerun
Incorporates function/tool caching for faster repeated runs on similar data sets
Allows for use of Cloud SDK in oneshot workflows, especially for tasks with clear, deterministic instructions
Practical features include inline prompts, bash command declarations, convention-oriented configuration, and output templating using ERB

ROAST has been used internally for about 5-6 weeks, launched as an open source project 2-3 weeks ago, and is rapidly gaining adoption within Shopify
The framework is inspired by Ruby on Rails conventions, making it particularly appealing for Rails enthusiasts
During the Q&A, topics include considerations around code generation for agents and how ROAST’s prompts and workflows support customization and AI integration for workflow development

Spotlight on Shopify