SUMM

The speaker introduces themselves as an investment partner at Amplify, focusing on technical founders and AI engineers.
The 2025 State of AI Engineering Survey was launched to better understand the current AI engineering landscape.
500 respondents participated, with a majority identifying as engineers, but titles and roles were diverse.
Many attendees perform similar work regardless of official job titles, highlighting the broad and growing technical community.
The term "AI engineering" has gained significant traction only since late 2022, correlating with the launch of ChatGPT.
Many experienced software engineers are newcomers to AI, with nearly half of those with over 10 years of experience working in AI for three years or less; 10% started in the past year.

Over half of respondents use large language models (LLMs) for both internal and external applications.
OpenAI provides three out of the top five and half of the top ten models used in external, customer-facing products.
Main use cases are code generation, code intelligence, and content writing assistance.
94% of LLM users apply them to at least two use cases; 82% use them for three or more, indicating broad application across tasks.

Retrieval-augmented generation (RAG) is the most popular customization method after few-shot learning, used by 70% of respondents.
Fine-tuning is more common than expected, especially among researchers and research engineers.
About 40% of fine-tuners use parameter-efficient methods like LoRA or QLoRA; other techniques include DPO, reinforcement fine-tuning, and supervised fine-tuning, with hybrid methods also mentioned.
Over 50% update their models at least monthly, and 17% update weekly.
Prompt updates are even more frequent: 70% update monthly, and 10% do so daily.
31% have no prompt management system in place.

Text models are used much more widely than image, video, or audio models in production (multimodal production gap).
Audio has the highest intent for future adoption, with 37% of non-users planning to adopt it.
Adoption rates for modalities are expected to increase as models improve and become more accessible.

AI agents are defined as systems where LLMs control decision-making or workflow.
While 80% report LLMs work well at work, fewer than 20% say the same for agents.
Most respondents plan to use agents in the future; less than 10% do not plan to adopt them.
Current agents in production typically have write access, often with a human in the loop, and some have independent action capabilities.
Monitoring and permission systems for agents are areas of growing interest.

60% use standard observability tools to monitor AI systems; over 50% use offline evaluation methods.
Multiple methods, including human review and user data collection, are used to evaluate model accuracy and quality.
For monitoring model usage, internal metrics are most relied upon.
65% use dedicated vector databases for storage; within this group, 35% self-host and 30% use third-party providers.

Most believe AI agents should disclose their non-human status when interacting with users.
A small majority would pay more for faster inference compute.
Most respondents believe transformer-based models will remain dominant through 2030.
The majority expect convergence between open and closed source models.
The average prediction is that 26% of U.S. Gen Z will have AI girlfriends/boyfriends.
Evaluation is cited as the number one pain point in AI engineering today.

The survey identified the top ten podcasts and newsletters that AI engineers actively learn from monthly.
Multiple popular content creators are recognized, some present at the event.
The speaker announces a forthcoming full report with more details.
The presentation concludes with an invitation to continue discussions online or in person.

The 2025 AI Engineering Report — Barr Yaron, Amplify