Kembali ke video

Warp

The secret to cheaper, smarter coding agents

Platform
YouTube
Durasi
3:44
Bahasa yang diucapkan
Inggris (Amerika Serikat)
Tontonan
761
Jeda potongan median
23.27s
Wajah di layar
95%
Frame gelap
3%
Warp The secret to cheaper, smarter coding agents at 0:00medium presenter
WarpThe secret to cheaper, smarter coding agents
0:00
Warp The secret to cheaper, smarter coding agents at 0:03medium presenter
WarpThe secret to cheaper, smarter coding agents
0:03
Warp The secret to cheaper, smarter coding agents at 0:03
WarpThe secret to cheaper, smarter coding agents
0:03
Warp The secret to cheaper, smarter coding agents at 0:05
WarpThe secret to cheaper, smarter coding agents
0:05
Warp The secret to cheaper, smarter coding agents at 0:07corner inset
WarpThe secret to cheaper, smarter coding agents
0:07
Warp The secret to cheaper, smarter coding agents at 0:11corner inset
WarpThe secret to cheaper, smarter coding agents
0:11
Warp The secret to cheaper, smarter coding agents at 0:17corner inset
WarpThe secret to cheaper, smarter coding agents
0:17
Warp The secret to cheaper, smarter coding agents at 0:18corner inset
WarpThe secret to cheaper, smarter coding agents
0:18
Warp The secret to cheaper, smarter coding agents at 0:22corner inset
WarpThe secret to cheaper, smarter coding agents
0:22
Warp The secret to cheaper, smarter coding agents at 0:23corner inset
WarpThe secret to cheaper, smarter coding agents
0:23
Warp The secret to cheaper, smarter coding agents at 0:25corner inset
WarpThe secret to cheaper, smarter coding agents
0:25
Warp The secret to cheaper, smarter coding agents at 0:26corner inset
WarpThe secret to cheaper, smarter coding agents
0:26
Warp The secret to cheaper, smarter coding agents at 0:26corner inset
WarpThe secret to cheaper, smarter coding agents
0:26
Warp The secret to cheaper, smarter coding agents at 0:27corner inset
WarpThe secret to cheaper, smarter coding agents
0:27
Warp The secret to cheaper, smarter coding agents at 0:29corner inset
WarpThe secret to cheaper, smarter coding agents
0:29
Warp The secret to cheaper, smarter coding agents at 0:32corner inset
WarpThe secret to cheaper, smarter coding agents
0:32
Warp The secret to cheaper, smarter coding agents at 0:33corner inset
WarpThe secret to cheaper, smarter coding agents
0:33
Warp The secret to cheaper, smarter coding agents at 0:37corner inset
WarpThe secret to cheaper, smarter coding agents
0:37
Warp The secret to cheaper, smarter coding agents at 0:38corner inset
WarpThe secret to cheaper, smarter coding agents
0:38
Warp The secret to cheaper, smarter coding agents at 0:57corner inset
WarpThe secret to cheaper, smarter coding agents
0:57
Warp The secret to cheaper, smarter coding agents at 1:02corner inset
WarpThe secret to cheaper, smarter coding agents
1:02
Warp The secret to cheaper, smarter coding agents at 1:06corner inset
WarpThe secret to cheaper, smarter coding agents
1:06
Warp The secret to cheaper, smarter coding agents at 1:10corner inset
WarpThe secret to cheaper, smarter coding agents
1:10
Warp The secret to cheaper, smarter coding agents at 1:14corner inset
WarpThe secret to cheaper, smarter coding agents
1:14
Warp The secret to cheaper, smarter coding agents at 1:22corner inset
WarpThe secret to cheaper, smarter coding agents
1:22
Warp The secret to cheaper, smarter coding agents at 1:30corner inset
WarpThe secret to cheaper, smarter coding agents
1:30
Warp The secret to cheaper, smarter coding agents at 1:38corner inset
WarpThe secret to cheaper, smarter coding agents
1:38
Warp The secret to cheaper, smarter coding agents at 1:58corner inset
WarpThe secret to cheaper, smarter coding agents
1:58
Warp The secret to cheaper, smarter coding agents at 2:02corner inset
WarpThe secret to cheaper, smarter coding agents
2:02
Warp The secret to cheaper, smarter coding agents at 2:10corner inset
WarpThe secret to cheaper, smarter coding agents
2:10
Warp The secret to cheaper, smarter coding agents at 2:14corner inset
WarpThe secret to cheaper, smarter coding agents
2:14
Warp The secret to cheaper, smarter coding agents at 2:18corner inset
WarpThe secret to cheaper, smarter coding agents
2:18
Warp The secret to cheaper, smarter coding agents at 2:26corner inset
WarpThe secret to cheaper, smarter coding agents
2:26
Warp The secret to cheaper, smarter coding agents at 2:30corner inset
WarpThe secret to cheaper, smarter coding agents
2:30
Warp The secret to cheaper, smarter coding agents at 2:38corner inset
WarpThe secret to cheaper, smarter coding agents
2:38
Warp The secret to cheaper, smarter coding agents at 2:42corner inset
WarpThe secret to cheaper, smarter coding agents
2:42
Warp The secret to cheaper, smarter coding agents at 2:46corner inset
WarpThe secret to cheaper, smarter coding agents
2:46
Warp The secret to cheaper, smarter coding agents at 2:51corner inset
WarpThe secret to cheaper, smarter coding agents
2:51
Warp The secret to cheaper, smarter coding agents at 2:58corner inset
WarpThe secret to cheaper, smarter coding agents
2:58
Warp The secret to cheaper, smarter coding agents at 3:02corner inset
WarpThe secret to cheaper, smarter coding agents
3:02
Warp The secret to cheaper, smarter coding agents at 3:06corner inset
WarpThe secret to cheaper, smarter coding agents
3:06
Warp The secret to cheaper, smarter coding agents at 3:14corner inset
WarpThe secret to cheaper, smarter coding agents
3:14
Warp The secret to cheaper, smarter coding agents at 3:22corner inset
WarpThe secret to cheaper, smarter coding agents
3:22
Warp The secret to cheaper, smarter coding agents at 3:24corner inset
WarpThe secret to cheaper, smarter coding agents
3:24
Warp The secret to cheaper, smarter coding agents at 3:31corner inset
WarpThe secret to cheaper, smarter coding agents
3:31
Warp The secret to cheaper, smarter coding agents at 3:38corner inset
WarpThe secret to cheaper, smarter coding agents
3:38
Warp The secret to cheaper, smarter coding agents at 3:42medium presenter
WarpThe secret to cheaper, smarter coding agents
3:42
Warp The secret to cheaper, smarter coding agents at 3:44
WarpThe secret to cheaper, smarter coding agents
3:44

Transkrip The secret to cheaper, smarter coding agents

Ditranskripsikan oleh model suara dari audio filmnya sendiri, bukan ditulis oleh mereknya. Klik baris mana pun untuk melompat ke momen itu.

  1. 0:00Let's see a real benchmark to understand how warp benchmarks work and how we used it to reduce our cost to PR by 63%.
  2. 0:06This is the benchmark that we used.
  3. 0:08You can see that it's generated as a report with an overall recommendation of which model to select as your default for the ages that you tested,
  4. 0:16as well as a breakdown with a Pareto curve across metrics like correctness and average cost, the configuration, and the breakdown across all of your scoring criteria.
  5. 0:26Let's start by looking at the overall recommendation here, which is GBT 5.6 sold.
  6. 0:31This was generated based on the level of correctness that that model achieved relative to other models in the set.
  7. 0:38You also notice additional recommendations called out that could also be a viable choice.
  8. 0:43For example here, GLM 5.3 flash was also very performance and at a lower cost.
  9. 0:49So if cost is the metric that you primarily care about, you may reject the overall recommendation
  10. 0:54and go for the most efficient choice.
  11. 0:56You can also use the Pareto curve to inform that decision.
  12. 0:59We can see a breakdown using our scoring criteria.
  13. 1:03For this test, we just looked at the correctness
  14. 1:05of the model, could it complete all of the tasks
  15. 1:07that we set up for it,
  16. 1:08and we can see metrics that we may care about.
  17. 1:10Average cost is going to be the defaults,
  18. 1:12but you can also look at average time
  19. 1:14if you're more concerned with latency,
  20. 1:15as well as a breakdown of output tokens and tool calls,
  21. 1:18which are useful in debugging scenarios.
  22. 1:20You wanna find models that score highest
  23. 1:22on the correctness scale so further up is best. You also want to find the leftmost models which
  24. 1:27are going to have the lowest average cost. And we can imagine a curve that we could plot going from
  25. 1:32the bottom left of the chart to the top right. And the models that fall along this curve are going to
  26. 1:37be GLM 5.3 which achieves relatively high correctness at the lowest cost and GPT 5.6 Sol
  27. 1:43which achieves the highest correctness at a bump in cost. And all the models that fall below this
  28. 1:49curve which are Grok, Opus, and Gemini in this testing are dominated meaning we
  29. 1:54don't need to look at those. And down here we can see a breakdown across each
  30. 1:58of the tasks that were tested for all of our models. And all of these tasks are
  31. 2:02replaying real agent runs that we have put through the factory in the past.
  32. 2:06Clicking into the configurations up here we can see each of those runs. We can
  33. 2:11click into the trial which is our replay of that agent run with that model
  34. 2:15selection and a mock of the MCP servers and secrets that were available in that original
  35. 2:21agent run. We can also view the judges that reviewed that trial run to determine its correctness.
  36. 2:27We can see this scoring agent, which is sort of our review agent that's given a rubric of what
  37. 2:32correctness looks like, give it a pass. And these scores can use rubrics that you define as well as
  38. 2:38model selections to fine-tune the cost of running these benchmarks. Now to close the loop you can
  39. 2:43use the results of these benchmarks to change your default models or even configure custom routers to
  40. 2:48send different tasks to the right model for the job. With warp factories all of this is configured
  41. 2:53as code. In our case we set our implementation agent to use GPT 5.6 sole by default and now
  42. 2:59whenever we trigger a factory agent to work on this repository we're going to be using this model.
  43. 3:04And if you want to get fine grained with your selections you can create a custom model router.
  44. 3:09Here we have our default set to GPT 5.6, but if there are specific tasks that could be routed to a more efficient model like
  45. 3:16Grok 4.6 medium for example
  46. 3:19We could define a description of these sorts of tasks that should be routed to that model instead in our case
  47. 3:24We ran a benchmark specifically focused on front-end UI tasks and we found that grok 4.6 medium was a high performer
  48. 3:31So we added a routing rule to our agent saying that simple front-end and UI tasks should use this model instead
  49. 3:37If this seems useful to you, factory benchmarks are part of warp factories, and you can get early access here.
49 baris · 748 kata

More from Warp

All of Warp →

The rest of their work we have taken apart.

Introducing the Warp Agent CLI: a coding agent that does what others can'tLaunch23K views
Warp
Introducing the Warp Agent CLI: a coding agent that does what others can't
2:17101 frames8.1s cut72% face
Orchestrate a team of agents in Warp4K views
Warp
Orchestrate a team of agents in Warp
2:58107 frames0% face
We fixed Claude Code's biggest problem...4K views
Warp
We fixed Claude Code's biggest problem...
4:20126 frames100% face
Your agent skills should improve themselves2K views
Warp
Your agent skills should improve themselves
3:24113 frames91% face
What's it like to work at Warp?662 views
Warp
What's it like to work at Warp?
1:3470 frames1.6s cut71% face

Cut at the same speed

Films whose median shot runs closest to 23.27s.

What is ClickUp?22M views
ClickUp
What is ClickUp?
1:2835 frames23.3s cut0% face
Agentic Videos: Turn Any Video into an Interactive AI Experience666 views
D-ID AI Video Platform
Agentic Videos: Turn Any Video into an Interactive AI Experience
1:0224 frames23.1s cut67% face
https://t.co/OVuIhnRm6v →685K views
Lovable
https://t.co/OVuIhnRm6v →
0:3749 frames23.0s cut0% face
Introducing Silent Speech interface for your existing devices! From ...Launch102K views
Adam
Introducing Silent Speech interface for your existing devices! From ...
2:43107 frames23.7s cut98% face
Anatoli Kopadze - Anthropic engineers just showed how they build a full app from scratc...1.9M views
Anatoli Kopadze
Anatoli Kopadze - Anthropic engineers just showed how they build a full app from scratc...
39:37243 frames23.8s cut44% face
Tutorial: Team Collaboration186 views
Infinity Maps
Tutorial: Team Collaboration
1:1530 frames22.7s cut7% face
Today we're announcing that hybrid agentic inference is coming to Per...Launch354K views
Perplexity
Today we're announcing that hybrid agentic inference is coming to Per...
0:6023 frames24.0s cut0% face
Meet folk MCP866 views
folk
Meet folk MCP
2:59109 frames24.1s cut49% face
ICYMI: Gemini 3.7 Flash is now available to all Google AI Pro and Ult...Launch350K views
Google
ICYMI: Gemini 3.7 Flash is now available to all Google AI Pro and Ult...
1:0126 frames24.2s cut0% face
Head of Engineering Shopify: "AI writes the code, AI reviews the co...715K views
Anatoli Kopadze
Head of Engineering Shopify: "AI writes the code, AI reviews the co...
26:40265 frames24.3s cut57% face
Peec AI: Track and Improve Your AI Brand Visibility1.3M views
Peec AI
Peec AI: Track and Improve Your AI Brand Visibility
0:2210 frames22.2s cut50% face
Introducing Socialite: the world’s first AI employee that works 24/7 ...Launch216K views
Sean Rich
Introducing Socialite: the world’s first AI employee that works 24/7 ...
1:2333 frames22.1s cut100% face

Lit and framed like this

Closest on how much of the running time has a face in it and how dark the frames are.

Make great video presentations with mmhmm1.5M views
Airtime
Make great video presentations with mmhmm
5:14119 frames8.9s cut95% face
How to build apps and automate tasks with Google Antigravity 2.060K views
Google
How to build apps and automate tasks with Google Antigravity 2.0
2:15106 frames7.1s cut95% face
Getting Started with ChatGPT Work23K views
OpenAI
Getting Started with ChatGPT Work
3:53156 frames4.2s cut94% face
ty for the love on this. insane... i wanna keep telling the story. i...38K views
Farza
ty for the love on this. insane... i wanna keep telling the story. i...
2:16180 frames8.4s cut92% face
Try Acapela.com - Product Hunt Launch Video2K views
Acapela
Try Acapela.com - Product Hunt Launch Video
2:26103 frames37.5s cut98% face
Benchling cuts migration time by 75% with cloud agents4K views
Cursor
Benchling cuts migration time by 75% with cloud agents
2:38134 frames4.7s cut95% face
Meet Airtime Camera | Professional video branding for Zoom, Teams & virtual meetings (2025)556K views
Airtime
Meet Airtime Camera | Professional video branding for Zoom, Teams & virtual meetings (2025)
1:5647 frames11.4s cut96% face
Spotlight what matters – New in Basecamp 51K views
37signals
Spotlight what matters – New in Basecamp 5
1:1024 frames96% face
Leave Me Alone v2.0 - Getting Started999 views
James Ivings
Leave Me Alone v2.0 - Getting Started
2:2497 frames96% face
Perfect lighting, zero-typing, instant recaps, and live translation? Am I dreaming"1K views
Google Workspace
Perfect lighting, zero-typing, instant recaps, and live translation? Am I dreaming"
2:0798 frames26.4s cut96% face
Wispr Flow + Cursor: Dictation Built for Developers1K views
Wispr Flow
Wispr Flow + Cursor: Dictation Built for Developers
1:2933 frames14.7s cut94% face
Community ownership made easy.3K views
Fairmint
Community ownership made easy.
2:05139 frames2.0s cut96% face

More social

Same category, other makers, most watched first.

Your ideas, your apps—Base44622M views
Base44
Your ideas, your apps—Base44
0:5651 frames1.1s cut6% face
Turn any idea into an app with Base44538M views
Base44
Turn any idea into an app with Base44
0:5538 frames2.2s cut50% face
Build Your Website with One Sentence | Base44154M views
Base44
Build Your Website with One Sentence | Base44
0:4835 frames0.9s cut31% face
Rip The Script | Nike Football81M views
Nike Football
Rip The Script | Nike Football
6:00310 frames1.1s cut60% face
PetSmart | "Pickup in...STORE!"75M views
PetSmart
PetSmart | "Pickup in...STORE!"
0:1511 frames1.4s cut64% face
NEO The Home Robot Order Today69M views
1X
NEO The Home Robot Order Today
9:53271 frames2.8s cut25% face
Apple Education: Ready for every learning opportunity69M views
Apple
Apple Education: Ready for every learning opportunity
1:4687 frames1.3s cut40% face
Phone calls suck. So, we bought a giant telephone car, raised $65,00...64M views
Bland
Phone calls suck. So, we bought a giant telephone car, raised $65,00...
1:4854 frames4.3s cut81% face
The Impossible Return – OIKOS 15s Commercial 202646M views
OIKOS Protein
The Impossible Return – OIKOS 15s Commercial 2026
0:1517 frames1.0s cut65% face
Oh wait there’s more: we’re building 3D Transforms, here’s a preview ...39M views
Figma
Oh wait there’s more: we’re building 3D Transforms, here’s a preview ...
1:0423 frames4.0s cut78% face
Getting Help is Simple, With Hims36M views
hims
Getting Help is Simple, With Hims
0:3023 frames2.0s cut70% face
Made in Figma (2025) Like this ❤️ we designed it for you36M views
Figma
Made in Figma (2025) Like this ❤️ we designed it for you
0:3872 frames0.5s cut7% face
Browse every frame in the library