영상 목록으로

Warp

The secret to cheaper, smarter coding agents

플랫폼
YouTube
길이
3:44
사용 언어
영어(미국)
조회수
761
컷 간격 중앙값
23.27s
얼굴 노출
95%
어두운 프레임
3%
Warp The secret to cheaper, smarter coding agents at 0:00medium presenter
WarpThe secret to cheaper, smarter coding agents
0:00
Warp The secret to cheaper, smarter coding agents at 0:03medium presenter
WarpThe secret to cheaper, smarter coding agents
0:03
Warp The secret to cheaper, smarter coding agents at 0:03
WarpThe secret to cheaper, smarter coding agents
0:03
Warp The secret to cheaper, smarter coding agents at 0:05
WarpThe secret to cheaper, smarter coding agents
0:05
Warp The secret to cheaper, smarter coding agents at 0:07corner inset
WarpThe secret to cheaper, smarter coding agents
0:07
Warp The secret to cheaper, smarter coding agents at 0:11corner inset
WarpThe secret to cheaper, smarter coding agents
0:11
Warp The secret to cheaper, smarter coding agents at 0:17corner inset
WarpThe secret to cheaper, smarter coding agents
0:17
Warp The secret to cheaper, smarter coding agents at 0:18corner inset
WarpThe secret to cheaper, smarter coding agents
0:18
Warp The secret to cheaper, smarter coding agents at 0:22corner inset
WarpThe secret to cheaper, smarter coding agents
0:22
Warp The secret to cheaper, smarter coding agents at 0:23corner inset
WarpThe secret to cheaper, smarter coding agents
0:23
Warp The secret to cheaper, smarter coding agents at 0:25corner inset
WarpThe secret to cheaper, smarter coding agents
0:25
Warp The secret to cheaper, smarter coding agents at 0:26corner inset
WarpThe secret to cheaper, smarter coding agents
0:26
Warp The secret to cheaper, smarter coding agents at 0:26corner inset
WarpThe secret to cheaper, smarter coding agents
0:26
Warp The secret to cheaper, smarter coding agents at 0:27corner inset
WarpThe secret to cheaper, smarter coding agents
0:27
Warp The secret to cheaper, smarter coding agents at 0:29corner inset
WarpThe secret to cheaper, smarter coding agents
0:29
Warp The secret to cheaper, smarter coding agents at 0:32corner inset
WarpThe secret to cheaper, smarter coding agents
0:32
Warp The secret to cheaper, smarter coding agents at 0:33corner inset
WarpThe secret to cheaper, smarter coding agents
0:33
Warp The secret to cheaper, smarter coding agents at 0:37corner inset
WarpThe secret to cheaper, smarter coding agents
0:37
Warp The secret to cheaper, smarter coding agents at 0:38corner inset
WarpThe secret to cheaper, smarter coding agents
0:38
Warp The secret to cheaper, smarter coding agents at 0:57corner inset
WarpThe secret to cheaper, smarter coding agents
0:57
Warp The secret to cheaper, smarter coding agents at 1:02corner inset
WarpThe secret to cheaper, smarter coding agents
1:02
Warp The secret to cheaper, smarter coding agents at 1:06corner inset
WarpThe secret to cheaper, smarter coding agents
1:06
Warp The secret to cheaper, smarter coding agents at 1:10corner inset
WarpThe secret to cheaper, smarter coding agents
1:10
Warp The secret to cheaper, smarter coding agents at 1:14corner inset
WarpThe secret to cheaper, smarter coding agents
1:14
Warp The secret to cheaper, smarter coding agents at 1:22corner inset
WarpThe secret to cheaper, smarter coding agents
1:22
Warp The secret to cheaper, smarter coding agents at 1:30corner inset
WarpThe secret to cheaper, smarter coding agents
1:30
Warp The secret to cheaper, smarter coding agents at 1:38corner inset
WarpThe secret to cheaper, smarter coding agents
1:38
Warp The secret to cheaper, smarter coding agents at 1:58corner inset
WarpThe secret to cheaper, smarter coding agents
1:58
Warp The secret to cheaper, smarter coding agents at 2:02corner inset
WarpThe secret to cheaper, smarter coding agents
2:02
Warp The secret to cheaper, smarter coding agents at 2:10corner inset
WarpThe secret to cheaper, smarter coding agents
2:10
Warp The secret to cheaper, smarter coding agents at 2:14corner inset
WarpThe secret to cheaper, smarter coding agents
2:14
Warp The secret to cheaper, smarter coding agents at 2:18corner inset
WarpThe secret to cheaper, smarter coding agents
2:18
Warp The secret to cheaper, smarter coding agents at 2:26corner inset
WarpThe secret to cheaper, smarter coding agents
2:26
Warp The secret to cheaper, smarter coding agents at 2:30corner inset
WarpThe secret to cheaper, smarter coding agents
2:30
Warp The secret to cheaper, smarter coding agents at 2:38corner inset
WarpThe secret to cheaper, smarter coding agents
2:38
Warp The secret to cheaper, smarter coding agents at 2:42corner inset
WarpThe secret to cheaper, smarter coding agents
2:42
Warp The secret to cheaper, smarter coding agents at 2:46corner inset
WarpThe secret to cheaper, smarter coding agents
2:46
Warp The secret to cheaper, smarter coding agents at 2:51corner inset
WarpThe secret to cheaper, smarter coding agents
2:51
Warp The secret to cheaper, smarter coding agents at 2:58corner inset
WarpThe secret to cheaper, smarter coding agents
2:58
Warp The secret to cheaper, smarter coding agents at 3:02corner inset
WarpThe secret to cheaper, smarter coding agents
3:02
Warp The secret to cheaper, smarter coding agents at 3:06corner inset
WarpThe secret to cheaper, smarter coding agents
3:06
Warp The secret to cheaper, smarter coding agents at 3:14corner inset
WarpThe secret to cheaper, smarter coding agents
3:14
Warp The secret to cheaper, smarter coding agents at 3:22corner inset
WarpThe secret to cheaper, smarter coding agents
3:22
Warp The secret to cheaper, smarter coding agents at 3:24corner inset
WarpThe secret to cheaper, smarter coding agents
3:24
Warp The secret to cheaper, smarter coding agents at 3:31corner inset
WarpThe secret to cheaper, smarter coding agents
3:31
Warp The secret to cheaper, smarter coding agents at 3:38corner inset
WarpThe secret to cheaper, smarter coding agents
3:38
Warp The secret to cheaper, smarter coding agents at 3:42medium presenter
WarpThe secret to cheaper, smarter coding agents
3:42
Warp The secret to cheaper, smarter coding agents at 3:44
WarpThe secret to cheaper, smarter coding agents
3:44

The secret to cheaper, smarter coding agents 자막 전문

브랜드가 작성한 것이 아니라, 영상의 오디오를 음성 모델이 받아쓴 것입니다. 아무 줄이나 클릭하면 그 순간으로 이동합니다.

  1. 0:00Let's see a real benchmark to understand how warp benchmarks work and how we used it to reduce our cost to PR by 63%.
  2. 0:06This is the benchmark that we used.
  3. 0:08You can see that it's generated as a report with an overall recommendation of which model to select as your default for the ages that you tested,
  4. 0:16as well as a breakdown with a Pareto curve across metrics like correctness and average cost, the configuration, and the breakdown across all of your scoring criteria.
  5. 0:26Let's start by looking at the overall recommendation here, which is GBT 5.6 sold.
  6. 0:31This was generated based on the level of correctness that that model achieved relative to other models in the set.
  7. 0:38You also notice additional recommendations called out that could also be a viable choice.
  8. 0:43For example here, GLM 5.3 flash was also very performance and at a lower cost.
  9. 0:49So if cost is the metric that you primarily care about, you may reject the overall recommendation
  10. 0:54and go for the most efficient choice.
  11. 0:56You can also use the Pareto curve to inform that decision.
  12. 0:59We can see a breakdown using our scoring criteria.
  13. 1:03For this test, we just looked at the correctness
  14. 1:05of the model, could it complete all of the tasks
  15. 1:07that we set up for it,
  16. 1:08and we can see metrics that we may care about.
  17. 1:10Average cost is going to be the defaults,
  18. 1:12but you can also look at average time
  19. 1:14if you're more concerned with latency,
  20. 1:15as well as a breakdown of output tokens and tool calls,
  21. 1:18which are useful in debugging scenarios.
  22. 1:20You wanna find models that score highest
  23. 1:22on the correctness scale so further up is best. You also want to find the leftmost models which
  24. 1:27are going to have the lowest average cost. And we can imagine a curve that we could plot going from
  25. 1:32the bottom left of the chart to the top right. And the models that fall along this curve are going to
  26. 1:37be GLM 5.3 which achieves relatively high correctness at the lowest cost and GPT 5.6 Sol
  27. 1:43which achieves the highest correctness at a bump in cost. And all the models that fall below this
  28. 1:49curve which are Grok, Opus, and Gemini in this testing are dominated meaning we
  29. 1:54don't need to look at those. And down here we can see a breakdown across each
  30. 1:58of the tasks that were tested for all of our models. And all of these tasks are
  31. 2:02replaying real agent runs that we have put through the factory in the past.
  32. 2:06Clicking into the configurations up here we can see each of those runs. We can
  33. 2:11click into the trial which is our replay of that agent run with that model
  34. 2:15selection and a mock of the MCP servers and secrets that were available in that original
  35. 2:21agent run. We can also view the judges that reviewed that trial run to determine its correctness.
  36. 2:27We can see this scoring agent, which is sort of our review agent that's given a rubric of what
  37. 2:32correctness looks like, give it a pass. And these scores can use rubrics that you define as well as
  38. 2:38model selections to fine-tune the cost of running these benchmarks. Now to close the loop you can
  39. 2:43use the results of these benchmarks to change your default models or even configure custom routers to
  40. 2:48send different tasks to the right model for the job. With warp factories all of this is configured
  41. 2:53as code. In our case we set our implementation agent to use GPT 5.6 sole by default and now
  42. 2:59whenever we trigger a factory agent to work on this repository we're going to be using this model.
  43. 3:04And if you want to get fine grained with your selections you can create a custom model router.
  44. 3:09Here we have our default set to GPT 5.6, but if there are specific tasks that could be routed to a more efficient model like
  45. 3:16Grok 4.6 medium for example
  46. 3:19We could define a description of these sorts of tasks that should be routed to that model instead in our case
  47. 3:24We ran a benchmark specifically focused on front-end UI tasks and we found that grok 4.6 medium was a high performer
  48. 3:31So we added a routing rule to our agent saying that simple front-end and UI tasks should use this model instead
  49. 3:37If this seems useful to you, factory benchmarks are part of warp factories, and you can get early access here.
49줄 · 748단어