Voltar aos vídeos

Warp

Your agent skills should improve themselves

Plataforma
YouTube
Duração
3:24
Idioma falado
inglês (Estados Unidos)
Visualizações
2 mil
Rosto em cena
91%
Quadros escuros
0%
Warp Your agent skills should improve themselves at 0:00medium presenter
WarpYour agent skills should improve themselves
0:00
Warp Your agent skills should improve themselves at 0:05
WarpYour agent skills should improve themselves
0:05
Warp Your agent skills should improve themselves at 0:06
WarpYour agent skills should improve themselves
0:06
Warp Your agent skills should improve themselves at 0:06
WarpYour agent skills should improve themselves
0:06
Warp Your agent skills should improve themselves at 0:08medium presenter
WarpYour agent skills should improve themselves
0:08
Warp Your agent skills should improve themselves at 0:09medium presenter
WarpYour agent skills should improve themselves
0:09
Warp Your agent skills should improve themselves at 0:26corner inset
WarpYour agent skills should improve themselves
0:26
Warp Your agent skills should improve themselves at 0:32corner inset
WarpYour agent skills should improve themselves
0:32
Warp Your agent skills should improve themselves at 0:45corner inset
WarpYour agent skills should improve themselves
0:45
Warp Your agent skills should improve themselves at 0:46corner inset
WarpYour agent skills should improve themselves
0:46
Warp Your agent skills should improve themselves at 0:50corner inset
WarpYour agent skills should improve themselves
0:50
Warp Your agent skills should improve themselves at 0:52corner inset
WarpYour agent skills should improve themselves
0:52
Warp Your agent skills should improve themselves at 0:56corner inset
WarpYour agent skills should improve themselves
0:56
Warp Your agent skills should improve themselves at 0:57corner inset
WarpYour agent skills should improve themselves
0:57
Warp Your agent skills should improve themselves at 1:02
WarpYour agent skills should improve themselves
1:02
Warp Your agent skills should improve themselves at 1:06corner inset
WarpYour agent skills should improve themselves
1:06
Warp Your agent skills should improve themselves at 1:10corner inset
WarpYour agent skills should improve themselves
1:10
Warp Your agent skills should improve themselves at 1:14corner inset
WarpYour agent skills should improve themselves
1:14
Warp Your agent skills should improve themselves at 1:18
WarpYour agent skills should improve themselves
1:18
Warp Your agent skills should improve themselves at 1:22
WarpYour agent skills should improve themselves
1:22
Warp Your agent skills should improve themselves at 1:30
WarpYour agent skills should improve themselves
1:30
Warp Your agent skills should improve themselves at 1:34corner inset
WarpYour agent skills should improve themselves
1:34
Warp Your agent skills should improve themselves at 1:50corner inset
WarpYour agent skills should improve themselves
1:50
Warp Your agent skills should improve themselves at 1:58corner inset
WarpYour agent skills should improve themselves
1:58
Warp Your agent skills should improve themselves at 2:02corner inset
WarpYour agent skills should improve themselves
2:02
Warp Your agent skills should improve themselves at 2:06corner inset
WarpYour agent skills should improve themselves
2:06
Warp Your agent skills should improve themselves at 2:10corner inset
WarpYour agent skills should improve themselves
2:10
Warp Your agent skills should improve themselves at 2:18corner inset
WarpYour agent skills should improve themselves
2:18
Warp Your agent skills should improve themselves at 2:26corner inset
WarpYour agent skills should improve themselves
2:26
Warp Your agent skills should improve themselves at 2:30corner inset
WarpYour agent skills should improve themselves
2:30
Warp Your agent skills should improve themselves at 2:30corner inset
WarpYour agent skills should improve themselves
2:30
Warp Your agent skills should improve themselves at 2:34corner inset
WarpYour agent skills should improve themselves
2:34
Warp Your agent skills should improve themselves at 2:38corner inset
WarpYour agent skills should improve themselves
2:38
Warp Your agent skills should improve themselves at 2:42corner inset
WarpYour agent skills should improve themselves
2:42
Warp Your agent skills should improve themselves at 2:46corner inset
WarpYour agent skills should improve themselves
2:46
Warp Your agent skills should improve themselves at 2:50corner inset
WarpYour agent skills should improve themselves
2:50
Warp Your agent skills should improve themselves at 2:58corner inset
WarpYour agent skills should improve themselves
2:58
Warp Your agent skills should improve themselves at 3:02corner inset
WarpYour agent skills should improve themselves
3:02
Warp Your agent skills should improve themselves at 3:06corner inset
WarpYour agent skills should improve themselves
3:06
Warp Your agent skills should improve themselves at 3:10corner inset
WarpYour agent skills should improve themselves
3:10
Warp Your agent skills should improve themselves at 3:14corner inset
WarpYour agent skills should improve themselves
3:14
Warp Your agent skills should improve themselves at 3:18medium presenter
WarpYour agent skills should improve themselves
3:18

Transcrição de Your agent skills should improve themselves

Transcrito por um modelo de fala a partir do áudio do próprio vídeo, não escrito pela marca. Clique em qualquer linha para ir até aquele momento.

  1. 0:00Let's talk about self-improvement loops for agents. The concept is pretty simple. We have a number of
  2. 0:05conversations with our agents every day across our team, and there are a number of failure points in
  3. 0:10those conversations where agents may hold the skills wrong or write inefficient code that we
  4. 0:15have to go revise, and it would be nice to have a system that's evaluating those conversations as
  5. 0:20they happen and suggesting improvements to our skills so that the agent performs better next time.
  6. 0:25To address this, we've created a three-step system that works off of all of your conversation data.
  7. 0:31It starts from aggregating all of your raw conversations, turning those into scored conversations,
  8. 0:37so defining rubrics of what success and failure look like, and then having an agent review all of those scores to suggest improvements.
  9. 0:45Here I'll show you what that looks like implemented as part of a larger software factory.
  10. 0:50This dashboard is part of Warp Factories, which you can get early access to right now.
  11. 0:55And you can see here we've set up a number of scorers that we run across all the conversations across our team.
  12. 1:01Here we've created a code quality agent that reviews all the conversations we have working on our main repositories.
  13. 1:07And it evaluates whether it has passed or failed our code quality test.
  14. 1:12And we can see the configuration for that here.
  15. 1:14Where whenever an agent is implementing code on our repositories, we've given it some instructions on how to evaluate the performance.
  16. 1:22This code quality agent is going to assess the design and correctness of the code that agents write.
  17. 1:27And it's also going to look into the diffs and documentation to suggest improvements.
  18. 1:32And here we've set the model that's going to judge all these conversations along with classifications,
  19. 1:37which are basically a rubric from zero to one where you can define how the agent should judge that conversation.
  20. 1:44For code quality, it's fairly simple.
  21. 1:46It's going to be approved if the code is correct and consistent with our conventions.
  22. 1:51If there's insufficient evidence, it should bail out without explicitly failing, and it
  23. 1:56should block that PR if it finds evidence that it wasn't up to our quality standards.
  24. 2:01This agent will run on an automation every time an agent tries to open a PR, and it will
  25. 2:06assess the pass or failure over time.
  26. 2:08Here as you can see, we've been fine tuning the success and failure criteria to get the
  27. 2:12best quality output.
  28. 2:14And then, once we have agents grading these conversations, we can have another agent review
  29. 2:18all of the runs that failed our quality test and suggest improvements to our skills so they don't fail next time.
  30. 2:25And we can see samples of all of those improvement runs right here in our self-improvement dashboard.
  31. 2:30This is also a feature that's baked into warp factories.
  32. 2:33We can see here that a self-improvement agent reviewed all of those scored conversations and
  33. 2:38made some PRs to actually improve our skills. In this example, our code review agent wasn't fetching all of the necessary PR
  34. 2:45feedback before writing a review. And down here we can see direct references to the failing scores
  35. 2:51that prompted this PR. Here it was prompted by our efficiency and task compliance scorers,
  36. 2:58which are other rubrics that we have set up, where there was lower compliance than we would
  37. 3:02have liked. And here we can see the changes that is suggested. Since our factory is configured as
  38. 3:07code, it's easy for agents to reach into the configuration and suggest ways to improve our
  39. 3:12skills. And with that, we have a closed loop system. Bring tasks into the factory and let
  40. 3:18the code quality improve on its own. If that sounds interesting to you, check out Warp
  41. 3:22Factories in Early Access.
41 linhas · 638 palavras