Building agent fleet architectures your CISO doesn't hate — Lou Bichard, Gitpod
Introduction to Gitpod and Secure Dev Environments 01:37
Gitpod is a platform providing secure, automated development environments, serving as a replacement for a local machine, with developers spending around 37 hours a week inside it.
As a highly mission-critical software, reliability is paramount, especially when working with highly secure, regulated organizations like banks, pharmaceutical, and healthcare companies.
The talk aims to discuss how technical architectures affect business, particularly for vendors, and for those building, consuming, or buying AI tools in secure environments.
Gitpod started in 2019 with a fully managed SaaS product, hosted on GCP Kubernetes, which offered a quick "time to wow" but faced significant issues with crypto mining, abuse, and being insufficient for enterprise needs.
The next iteration was a self-hosted model, packaging the existing architecture for customer installation across various cloud providers and Kubernetes flavors, but this led to significant "day two effects" and high operational overhead for customers, eroding ROI.
Gitpod then moved to a "managed substrate" model, focusing on AWS to reduce variance while still allowing customers to self-host, with Gitpod managing the service and receiving telemetry data, but source code remained on customer infrastructure.
Despite operational success, this model, still built on Kubernetes, remained complicated with high fixed costs and complexity, leading Gitpod to move away from Kubernetes for their specific use case.
The current architecture was designed from first principles to run securely within regulated companies without creating a huge operational burden.
It utilizes a "runner" component, which is a single ECS task costing single-digit dollars per month, to securely run source code and access data on the customer's infrastructure.
Core dev environments run on AWS EC2, backed up to EBS, leveraging native cloud provider functionalities instead of a portable but complex platform like Kubernetes.
Gitpod manages metadata (user IDs, not PII) on its side to reduce customer operational overhead, while sensitive data like IP and source code remain within the customer's infrastructure.
The deployment process for customers is streamlined, often taking as little as 3 minutes for technical setup, with the main challenge being network configuration and VPC details on the customer's side.
Gitpod recently launched its agent offering, which leverages the same existing infrastructure within customer environments.
Autonomous agents require the same access as human developers (source code, internal systems like databases or clusters) and can operate within the same secure dev environments with identical privileges.
The agent offering is privacy-first, running within the customer's infrastructure and benefiting from the API-first design that audit logs every action, providing a complete audit history for both human and agent tasks.
For those purchasing AI tools, it is crucial to consider the underlying architecture and infrastructure qualities, while vendors can learn from Gitpod's journey towards a more simplified technical architecture, moving away from complex platforms like Kubernetes to better serve customers.