Kembali ke video

Warp

Your agent skills should improve themselves

Platform
YouTube
Durasi
3:24
Bahasa yang diucapkan
Inggris (Amerika Serikat)
Tontonan
800
Wajah di layar
89%
Frame gelap
0%
Warp Your agent skills should improve themselves at 0:00medium presenter
WarpYour agent skills should improve themselves
0:00
Warp Your agent skills should improve themselves at 0:04medium presenter
WarpYour agent skills should improve themselves
0:04
Warp Your agent skills should improve themselves at 0:05
WarpYour agent skills should improve themselves
0:05
Warp Your agent skills should improve themselves at 0:06
WarpYour agent skills should improve themselves
0:06
Warp Your agent skills should improve themselves at 0:06
WarpYour agent skills should improve themselves
0:06
Warp Your agent skills should improve themselves at 0:08medium presenter
WarpYour agent skills should improve themselves
0:08
Warp Your agent skills should improve themselves at 0:09medium presenter
WarpYour agent skills should improve themselves
0:09
Warp Your agent skills should improve themselves at 0:26corner inset
WarpYour agent skills should improve themselves
0:26
Warp Your agent skills should improve themselves at 0:32corner inset
WarpYour agent skills should improve themselves
0:32
Warp Your agent skills should improve themselves at 0:45corner inset
WarpYour agent skills should improve themselves
0:45
Warp Your agent skills should improve themselves at 0:46corner inset
WarpYour agent skills should improve themselves
0:46
Warp Your agent skills should improve themselves at 0:50corner inset
WarpYour agent skills should improve themselves
0:50
Warp Your agent skills should improve themselves at 0:52corner inset
WarpYour agent skills should improve themselves
0:52
Warp Your agent skills should improve themselves at 0:56corner inset
WarpYour agent skills should improve themselves
0:56
Warp Your agent skills should improve themselves at 0:57corner inset
WarpYour agent skills should improve themselves
0:57
Warp Your agent skills should improve themselves at 1:02
WarpYour agent skills should improve themselves
1:02
Warp Your agent skills should improve themselves at 1:06corner inset
WarpYour agent skills should improve themselves
1:06
Warp Your agent skills should improve themselves at 1:10corner inset
WarpYour agent skills should improve themselves
1:10
Warp Your agent skills should improve themselves at 1:14corner inset
WarpYour agent skills should improve themselves
1:14
Warp Your agent skills should improve themselves at 1:18
WarpYour agent skills should improve themselves
1:18
Warp Your agent skills should improve themselves at 1:22
WarpYour agent skills should improve themselves
1:22
Warp Your agent skills should improve themselves at 1:26corner inset
WarpYour agent skills should improve themselves
1:26
Warp Your agent skills should improve themselves at 1:34corner inset
WarpYour agent skills should improve themselves
1:34
Warp Your agent skills should improve themselves at 1:50corner inset
WarpYour agent skills should improve themselves
1:50
Warp Your agent skills should improve themselves at 1:58corner inset
WarpYour agent skills should improve themselves
1:58
Warp Your agent skills should improve themselves at 2:02corner inset
WarpYour agent skills should improve themselves
2:02
Warp Your agent skills should improve themselves at 2:06corner inset
WarpYour agent skills should improve themselves
2:06
Warp Your agent skills should improve themselves at 2:18corner inset
WarpYour agent skills should improve themselves
2:18
Warp Your agent skills should improve themselves at 2:26corner inset
WarpYour agent skills should improve themselves
2:26
Warp Your agent skills should improve themselves at 2:30corner inset
WarpYour agent skills should improve themselves
2:30
Warp Your agent skills should improve themselves at 2:30corner inset
WarpYour agent skills should improve themselves
2:30
Warp Your agent skills should improve themselves at 2:34corner inset
WarpYour agent skills should improve themselves
2:34
Warp Your agent skills should improve themselves at 2:38corner inset
WarpYour agent skills should improve themselves
2:38
Warp Your agent skills should improve themselves at 2:42corner inset
WarpYour agent skills should improve themselves
2:42
Warp Your agent skills should improve themselves at 2:46corner inset
WarpYour agent skills should improve themselves
2:46
Warp Your agent skills should improve themselves at 2:50corner inset
WarpYour agent skills should improve themselves
2:50
Warp Your agent skills should improve themselves at 2:58corner inset
WarpYour agent skills should improve themselves
2:58
Warp Your agent skills should improve themselves at 3:02corner inset
WarpYour agent skills should improve themselves
3:02
Warp Your agent skills should improve themselves at 3:06corner inset
WarpYour agent skills should improve themselves
3:06
Warp Your agent skills should improve themselves at 3:10corner inset
WarpYour agent skills should improve themselves
3:10
Warp Your agent skills should improve themselves at 3:14corner inset
WarpYour agent skills should improve themselves
3:14
Warp Your agent skills should improve themselves at 3:18medium presenter
WarpYour agent skills should improve themselves
3:18

Transkrip Your agent skills should improve themselves

Ditranskripsikan oleh model suara dari audio filmnya sendiri, bukan ditulis oleh mereknya. Klik baris mana pun untuk melompat ke momen itu.

  1. 0:00Let's talk about self-improvement loops for agents. The concept is pretty simple. We have a number of
  2. 0:05conversations with our agents every day across our team, and there are a number of failure points in
  3. 0:10those conversations where agents may hold the skills wrong or write inefficient code that we
  4. 0:15have to go revise, and it would be nice to have a system that's evaluating those conversations as
  5. 0:20they happen and suggesting improvements to our skills so that the agent performs better next time.
  6. 0:25To address this, we've created a three-step system that works off of all of your conversation data.
  7. 0:31It starts from aggregating all of your raw conversations, turning those into scored conversations,
  8. 0:37so defining rubrics of what success and failure look like, and then having an agent review all of those scores to suggest improvements.
  9. 0:45Here I'll show you what that looks like implemented as part of a larger software factory.
  10. 0:50This dashboard is part of Warp Factories, which you can get early access to right now.
  11. 0:55And you can see here we've set up a number of scorers that we run across all the conversations
  12. 1:00across our team.
  13. 1:01Here we've created a code quality agent that reviews all the conversations we have working
  14. 1:05on our main repositories, and it evaluates whether it has passed or failed our code quality
  15. 1:11test.
  16. 1:12And we can see the configuration for that here, where whenever an agent is implementing
  17. 1:17code on our repositories, we've given it some instructions on how to evaluate the performance.
  18. 1:22This code quality agent is going to assess the design and correctness of the code that
  19. 1:26agents write, and it's also going to look into the diffs and documentation to suggest
  20. 1:32improvements.
  21. 1:32And here we've set the model that's going to judge all these conversations, along with
  22. 1:36classifications, which are basically a rubric from zero to one where you can define how
  23. 1:42the agent should judge that conversation.
  24. 1:44For code quality, it's fairly simple.
  25. 1:46It's going to be approved if the code is correct and consistent with our conventions.
  26. 1:51If there's insufficient evidence, it should bail out without explicitly failing, and it
  27. 1:56should block that PR if it finds evidence that it wasn't up to our quality standards.
  28. 2:01This agent will run on an automation every time an agent tries to open a PR, and it will
  29. 2:06assess the pass or failure over time.
  30. 2:08Here as you can see, we've been fine-tuning the success and failure criteria to get the
  31. 2:12best quality output.
  32. 2:14And then once we have agents grading these conversations, we can have another agent review
  33. 2:18all of the runs that failed our quality test and suggest improvements to our skills so they don't
  34. 2:24fail next time. And we can see samples of all of those improvement runs right here in our
  35. 2:28self-improvement dashboard. This is also featured as baked into warp factories. We can see here
  36. 2:34that a self-improvement agent reviewed all of those scored conversations and made some PRs to
  37. 2:40actually improve our skills. In this example, our code review agent wasn't fetching all of the
  38. 2:44necessary PR feedback before writing a review.
  39. 2:48And down here, we can see direct references
  40. 2:50to the failing scores that prompted this PR.
  41. 2:53Here, it was prompted by our efficiency and task compliance
  42. 2:57scores, which are other rubrics that we have set up,
  43. 3:00where there was lower compliance than we would have liked.
  44. 3:03And here, we can see the changes that it suggested.
  45. 3:05Since our factory is configured as code,
  46. 3:07it's easy for agents to reach into the configuration
  47. 3:10and suggest ways to improve skills.
  48. 3:12And with that, we have a closed loop system.
  49. 3:15Bring tasks into the factory and let the code quality improve on its own.
  50. 3:20If that sounds interesting to you, check out Warp Factories in Early Access.
50 baris · 635 kata