AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff

Importance of Prompt Engineering 00:00

  • Prompt engineering remains crucial, with studies showing that good prompts can boost problem-solving success to 90%, while poor ones can drop it to 0%.
  • Despite claims of its obsolescence with newer AI models, prompt engineering continues to evolve and remain relevant.

Techniques for Effective Prompt Engineering 05:11

  • Sander Schulhoff, a pioneer in prompt engineering, shares key techniques for improving prompts, emphasizing the need for practical application.
  • The conversation covers basic techniques, advanced methods, and the concepts of prompt injection and red teaming.

Basic Techniques for Prompt Engineering 09:17

  • Trial and error is essential for learning effective prompting.
  • Few-shot prompting: Provide examples of desired outputs to guide the AI in generating similar responses.

Modes of Prompt Engineering 09:29

  • Two modes exist: conversational mode, which involves iterative prompting during interactions, and product-focused mode, which centers on optimizing prompts for consistent, high-volume results.

Advanced Techniques in Prompt Engineering 12:05

  • Self-criticism: Instruct the AI to evaluate its own responses, leading to improved outputs through iterative feedback.
  • Decomposition: Ask the AI to identify sub-problems before attempting to solve the main task.

Common Misconceptions in Prompt Engineering 18:05

  • Role prompting (assigning the AI a specific persona) has been found to lack significant impact on performance for accuracy-based tasks, though it may help with expressive tasks.

Challenges with AI Prompt Security 51:59

  • Prompt injection involves tricking AI into producing harmful or undesirable outputs. Techniques include obfuscation and using deceptive narratives.
  • Red teaming competitions gather data on prompt injection techniques, highlighting the ongoing security concerns in AI development.

Effective Defense Strategies Against Prompt Injection 73:23

  • Common defenses, like improved prompting and AI guardrails, often fall short against motivated attackers.
  • Effective strategies include safety tuning (training AI to recognize harmful prompts) and fine-tuning for specific tasks, reducing susceptibility to malicious inputs.

The Future of AI and Prompt Engineering 78:06

  • While AI will continue to develop, the challenge of securing these systems against misuse persists.
  • Collaboration between AI developers and researchers is essential to improve security and mitigate risks associated with advanced AI capabilities.