Zurück zu den Videos

Warp

The secret to cheaper, smarter coding agents

Plattform
YouTube
Länge
3:44
Gesprochene Sprache
Englisch (Vereinigte Staaten)
Aufrufe
761
Median-Schnitt
23.27s
Gesicht im Bild
95%
Dunkle Bilder
3%
Warp The secret to cheaper, smarter coding agents at 0:00medium presenter
WarpThe secret to cheaper, smarter coding agents
0:00
Warp The secret to cheaper, smarter coding agents at 0:03medium presenter
WarpThe secret to cheaper, smarter coding agents
0:03
Warp The secret to cheaper, smarter coding agents at 0:03
WarpThe secret to cheaper, smarter coding agents
0:03
Warp The secret to cheaper, smarter coding agents at 0:05
WarpThe secret to cheaper, smarter coding agents
0:05
Warp The secret to cheaper, smarter coding agents at 0:07corner inset
WarpThe secret to cheaper, smarter coding agents
0:07
Warp The secret to cheaper, smarter coding agents at 0:11corner inset
WarpThe secret to cheaper, smarter coding agents
0:11
Warp The secret to cheaper, smarter coding agents at 0:17corner inset
WarpThe secret to cheaper, smarter coding agents
0:17
Warp The secret to cheaper, smarter coding agents at 0:18corner inset
WarpThe secret to cheaper, smarter coding agents
0:18
Warp The secret to cheaper, smarter coding agents at 0:22corner inset
WarpThe secret to cheaper, smarter coding agents
0:22
Warp The secret to cheaper, smarter coding agents at 0:23corner inset
WarpThe secret to cheaper, smarter coding agents
0:23
Warp The secret to cheaper, smarter coding agents at 0:25corner inset
WarpThe secret to cheaper, smarter coding agents
0:25
Warp The secret to cheaper, smarter coding agents at 0:26corner inset
WarpThe secret to cheaper, smarter coding agents
0:26
Warp The secret to cheaper, smarter coding agents at 0:26corner inset
WarpThe secret to cheaper, smarter coding agents
0:26
Warp The secret to cheaper, smarter coding agents at 0:27corner inset
WarpThe secret to cheaper, smarter coding agents
0:27
Warp The secret to cheaper, smarter coding agents at 0:29corner inset
WarpThe secret to cheaper, smarter coding agents
0:29
Warp The secret to cheaper, smarter coding agents at 0:32corner inset
WarpThe secret to cheaper, smarter coding agents
0:32
Warp The secret to cheaper, smarter coding agents at 0:33corner inset
WarpThe secret to cheaper, smarter coding agents
0:33
Warp The secret to cheaper, smarter coding agents at 0:37corner inset
WarpThe secret to cheaper, smarter coding agents
0:37
Warp The secret to cheaper, smarter coding agents at 0:38corner inset
WarpThe secret to cheaper, smarter coding agents
0:38
Warp The secret to cheaper, smarter coding agents at 0:57corner inset
WarpThe secret to cheaper, smarter coding agents
0:57
Warp The secret to cheaper, smarter coding agents at 1:02corner inset
WarpThe secret to cheaper, smarter coding agents
1:02
Warp The secret to cheaper, smarter coding agents at 1:06corner inset
WarpThe secret to cheaper, smarter coding agents
1:06
Warp The secret to cheaper, smarter coding agents at 1:10corner inset
WarpThe secret to cheaper, smarter coding agents
1:10
Warp The secret to cheaper, smarter coding agents at 1:14corner inset
WarpThe secret to cheaper, smarter coding agents
1:14
Warp The secret to cheaper, smarter coding agents at 1:22corner inset
WarpThe secret to cheaper, smarter coding agents
1:22
Warp The secret to cheaper, smarter coding agents at 1:30corner inset
WarpThe secret to cheaper, smarter coding agents
1:30
Warp The secret to cheaper, smarter coding agents at 1:38corner inset
WarpThe secret to cheaper, smarter coding agents
1:38
Warp The secret to cheaper, smarter coding agents at 1:58corner inset
WarpThe secret to cheaper, smarter coding agents
1:58
Warp The secret to cheaper, smarter coding agents at 2:02corner inset
WarpThe secret to cheaper, smarter coding agents
2:02
Warp The secret to cheaper, smarter coding agents at 2:10corner inset
WarpThe secret to cheaper, smarter coding agents
2:10
Warp The secret to cheaper, smarter coding agents at 2:14corner inset
WarpThe secret to cheaper, smarter coding agents
2:14
Warp The secret to cheaper, smarter coding agents at 2:18corner inset
WarpThe secret to cheaper, smarter coding agents
2:18
Warp The secret to cheaper, smarter coding agents at 2:26corner inset
WarpThe secret to cheaper, smarter coding agents
2:26
Warp The secret to cheaper, smarter coding agents at 2:30corner inset
WarpThe secret to cheaper, smarter coding agents
2:30
Warp The secret to cheaper, smarter coding agents at 2:38corner inset
WarpThe secret to cheaper, smarter coding agents
2:38
Warp The secret to cheaper, smarter coding agents at 2:42corner inset
WarpThe secret to cheaper, smarter coding agents
2:42
Warp The secret to cheaper, smarter coding agents at 2:46corner inset
WarpThe secret to cheaper, smarter coding agents
2:46
Warp The secret to cheaper, smarter coding agents at 2:51corner inset
WarpThe secret to cheaper, smarter coding agents
2:51
Warp The secret to cheaper, smarter coding agents at 2:58corner inset
WarpThe secret to cheaper, smarter coding agents
2:58
Warp The secret to cheaper, smarter coding agents at 3:02corner inset
WarpThe secret to cheaper, smarter coding agents
3:02
Warp The secret to cheaper, smarter coding agents at 3:06corner inset
WarpThe secret to cheaper, smarter coding agents
3:06
Warp The secret to cheaper, smarter coding agents at 3:14corner inset
WarpThe secret to cheaper, smarter coding agents
3:14
Warp The secret to cheaper, smarter coding agents at 3:22corner inset
WarpThe secret to cheaper, smarter coding agents
3:22
Warp The secret to cheaper, smarter coding agents at 3:24corner inset
WarpThe secret to cheaper, smarter coding agents
3:24
Warp The secret to cheaper, smarter coding agents at 3:31corner inset
WarpThe secret to cheaper, smarter coding agents
3:31
Warp The secret to cheaper, smarter coding agents at 3:38corner inset
WarpThe secret to cheaper, smarter coding agents
3:38
Warp The secret to cheaper, smarter coding agents at 3:42medium presenter
WarpThe secret to cheaper, smarter coding agents
3:42
Warp The secret to cheaper, smarter coding agents at 3:44
WarpThe secret to cheaper, smarter coding agents
3:44

Transkript von The secret to cheaper, smarter coding agents

Von einem Sprachmodell aus dem Ton des Films transkribiert, nicht von der Marke geschrieben. Klicke auf eine Zeile, um an diese Stelle zu springen.

  1. 0:00Let's see a real benchmark to understand how warp benchmarks work and how we used it to reduce our cost to PR by 63%.
  2. 0:06This is the benchmark that we used.
  3. 0:08You can see that it's generated as a report with an overall recommendation of which model to select as your default for the ages that you tested,
  4. 0:16as well as a breakdown with a Pareto curve across metrics like correctness and average cost, the configuration, and the breakdown across all of your scoring criteria.
  5. 0:26Let's start by looking at the overall recommendation here, which is GBT 5.6 sold.
  6. 0:31This was generated based on the level of correctness that that model achieved relative to other models in the set.
  7. 0:38You also notice additional recommendations called out that could also be a viable choice.
  8. 0:43For example here, GLM 5.3 flash was also very performance and at a lower cost.
  9. 0:49So if cost is the metric that you primarily care about, you may reject the overall recommendation
  10. 0:54and go for the most efficient choice.
  11. 0:56You can also use the Pareto curve to inform that decision.
  12. 0:59We can see a breakdown using our scoring criteria.
  13. 1:03For this test, we just looked at the correctness
  14. 1:05of the model, could it complete all of the tasks
  15. 1:07that we set up for it,
  16. 1:08and we can see metrics that we may care about.
  17. 1:10Average cost is going to be the defaults,
  18. 1:12but you can also look at average time
  19. 1:14if you're more concerned with latency,
  20. 1:15as well as a breakdown of output tokens and tool calls,
  21. 1:18which are useful in debugging scenarios.
  22. 1:20You wanna find models that score highest
  23. 1:22on the correctness scale so further up is best. You also want to find the leftmost models which
  24. 1:27are going to have the lowest average cost. And we can imagine a curve that we could plot going from
  25. 1:32the bottom left of the chart to the top right. And the models that fall along this curve are going to
  26. 1:37be GLM 5.3 which achieves relatively high correctness at the lowest cost and GPT 5.6 Sol
  27. 1:43which achieves the highest correctness at a bump in cost. And all the models that fall below this
  28. 1:49curve which are Grok, Opus, and Gemini in this testing are dominated meaning we
  29. 1:54don't need to look at those. And down here we can see a breakdown across each
  30. 1:58of the tasks that were tested for all of our models. And all of these tasks are
  31. 2:02replaying real agent runs that we have put through the factory in the past.
  32. 2:06Clicking into the configurations up here we can see each of those runs. We can
  33. 2:11click into the trial which is our replay of that agent run with that model
  34. 2:15selection and a mock of the MCP servers and secrets that were available in that original
  35. 2:21agent run. We can also view the judges that reviewed that trial run to determine its correctness.
  36. 2:27We can see this scoring agent, which is sort of our review agent that's given a rubric of what
  37. 2:32correctness looks like, give it a pass. And these scores can use rubrics that you define as well as
  38. 2:38model selections to fine-tune the cost of running these benchmarks. Now to close the loop you can
  39. 2:43use the results of these benchmarks to change your default models or even configure custom routers to
  40. 2:48send different tasks to the right model for the job. With warp factories all of this is configured
  41. 2:53as code. In our case we set our implementation agent to use GPT 5.6 sole by default and now
  42. 2:59whenever we trigger a factory agent to work on this repository we're going to be using this model.
  43. 3:04And if you want to get fine grained with your selections you can create a custom model router.
  44. 3:09Here we have our default set to GPT 5.6, but if there are specific tasks that could be routed to a more efficient model like
  45. 3:16Grok 4.6 medium for example
  46. 3:19We could define a description of these sorts of tasks that should be routed to that model instead in our case
  47. 3:24We ran a benchmark specifically focused on front-end UI tasks and we found that grok 4.6 medium was a high performer
  48. 3:31So we added a routing rule to our agent saying that simple front-end and UI tasks should use this model instead
  49. 3:37If this seems useful to you, factory benchmarks are part of warp factories, and you can get early access here.
49 Zeilen · 748 Wörter

More from Warp

All of Warp →

The rest of their work we have taken apart.

Introducing the Warp Agent CLI: a coding agent that does what others can'tLaunch23K views
Warp
Introducing the Warp Agent CLI: a coding agent that does what others can't
2:17101 frames8.1s cut72% face
Orchestrate a team of agents in Warp4K views
Warp
Orchestrate a team of agents in Warp
2:58107 frames0% face
We fixed Claude Code's biggest problem...4K views
Warp
We fixed Claude Code's biggest problem...
4:20126 frames100% face
Your agent skills should improve themselves2K views
Warp
Your agent skills should improve themselves
3:24113 frames91% face
What's it like to work at Warp?662 views
Warp
What's it like to work at Warp?
1:3470 frames1.6s cut71% face

Cut at the same speed

Films whose median shot runs closest to 23.27s.

What is ClickUp?22M views
ClickUp
What is ClickUp?
1:2835 frames23.3s cut0% face
Agentic Videos: Turn Any Video into an Interactive AI Experience666 views
D-ID AI Video Platform
Agentic Videos: Turn Any Video into an Interactive AI Experience
1:0224 frames23.1s cut67% face
https://t.co/OVuIhnRm6v →685K views
Lovable
https://t.co/OVuIhnRm6v →
0:3749 frames23.0s cut0% face
Introducing Silent Speech interface for your existing devices! From ...Launch102K views
Adam
Introducing Silent Speech interface for your existing devices! From ...
2:43107 frames23.7s cut98% face
Anatoli Kopadze - Anthropic engineers just showed how they build a full app from scratc...1.9M views
Anatoli Kopadze
Anatoli Kopadze - Anthropic engineers just showed how they build a full app from scratc...
39:37243 frames23.8s cut44% face
Tutorial: Team Collaboration186 views
Infinity Maps
Tutorial: Team Collaboration
1:1530 frames22.7s cut7% face
Today we're announcing that hybrid agentic inference is coming to Per...Launch354K views
Perplexity
Today we're announcing that hybrid agentic inference is coming to Per...
0:6023 frames24.0s cut0% face
Meet folk MCP866 views
folk
Meet folk MCP
2:59109 frames24.1s cut49% face
ICYMI: Gemini 3.7 Flash is now available to all Google AI Pro and Ult...Launch350K views
Google
ICYMI: Gemini 3.7 Flash is now available to all Google AI Pro and Ult...
1:0126 frames24.2s cut0% face
Head of Engineering Shopify: "AI writes the code, AI reviews the co...715K views
Anatoli Kopadze
Head of Engineering Shopify: "AI writes the code, AI reviews the co...
26:40265 frames24.3s cut57% face
Peec AI: Track and Improve Your AI Brand Visibility1.3M views
Peec AI
Peec AI: Track and Improve Your AI Brand Visibility
0:2210 frames22.2s cut50% face
Introducing Socialite: the world’s first AI employee that works 24/7 ...Launch216K views
Sean Rich
Introducing Socialite: the world’s first AI employee that works 24/7 ...
1:2333 frames22.1s cut100% face

Lit and framed like this

Closest on how much of the running time has a face in it and how dark the frames are.

Make great video presentations with mmhmm1.5M views
Airtime
Make great video presentations with mmhmm
5:14119 frames8.9s cut95% face
How to build apps and automate tasks with Google Antigravity 2.060K views
Google
How to build apps and automate tasks with Google Antigravity 2.0
2:15106 frames7.1s cut95% face
Getting Started with ChatGPT Work23K views
OpenAI
Getting Started with ChatGPT Work
3:53156 frames4.2s cut94% face
ty for the love on this. insane... i wanna keep telling the story. i...38K views
Farza
ty for the love on this. insane... i wanna keep telling the story. i...
2:16180 frames8.4s cut92% face
Try Acapela.com - Product Hunt Launch Video2K views
Acapela
Try Acapela.com - Product Hunt Launch Video
2:26103 frames37.5s cut98% face
Benchling cuts migration time by 75% with cloud agents4K views
Cursor
Benchling cuts migration time by 75% with cloud agents
2:38134 frames4.7s cut95% face
Meet Airtime Camera | Professional video branding for Zoom, Teams & virtual meetings (2025)556K views
Airtime
Meet Airtime Camera | Professional video branding for Zoom, Teams & virtual meetings (2025)
1:5647 frames11.4s cut96% face
Spotlight what matters – New in Basecamp 51K views
37signals
Spotlight what matters – New in Basecamp 5
1:1024 frames96% face
Leave Me Alone v2.0 - Getting Started999 views
James Ivings
Leave Me Alone v2.0 - Getting Started
2:2497 frames96% face
Perfect lighting, zero-typing, instant recaps, and live translation? Am I dreaming"1K views
Google Workspace
Perfect lighting, zero-typing, instant recaps, and live translation? Am I dreaming"
2:0798 frames26.4s cut96% face
Wispr Flow + Cursor: Dictation Built for Developers1K views
Wispr Flow
Wispr Flow + Cursor: Dictation Built for Developers
1:2933 frames14.7s cut94% face
Community ownership made easy.3K views
Fairmint
Community ownership made easy.
2:05139 frames2.0s cut96% face

More social

Same category, other makers, most watched first.

Your ideas, your apps—Base44622M views
Base44
Your ideas, your apps—Base44
0:5651 frames1.1s cut6% face
Turn any idea into an app with Base44538M views
Base44
Turn any idea into an app with Base44
0:5538 frames2.2s cut50% face
Build Your Website with One Sentence | Base44154M views
Base44
Build Your Website with One Sentence | Base44
0:4835 frames0.9s cut31% face
Rip The Script | Nike Football81M views
Nike Football
Rip The Script | Nike Football
6:00310 frames1.1s cut60% face
PetSmart | "Pickup in...STORE!"75M views
PetSmart
PetSmart | "Pickup in...STORE!"
0:1511 frames1.4s cut64% face
NEO The Home Robot Order Today69M views
1X
NEO The Home Robot Order Today
9:53271 frames2.8s cut25% face
Apple Education: Ready for every learning opportunity69M views
Apple
Apple Education: Ready for every learning opportunity
1:4687 frames1.3s cut40% face
Phone calls suck. So, we bought a giant telephone car, raised $65,00...64M views
Bland
Phone calls suck. So, we bought a giant telephone car, raised $65,00...
1:4854 frames4.3s cut81% face
The Impossible Return – OIKOS 15s Commercial 202646M views
OIKOS Protein
The Impossible Return – OIKOS 15s Commercial 2026
0:1517 frames1.0s cut65% face
Oh wait there’s more: we’re building 3D Transforms, here’s a preview ...39M views
Figma
Oh wait there’s more: we’re building 3D Transforms, here’s a preview ...
1:0423 frames4.0s cut78% face
Getting Help is Simple, With Hims36M views
hims
Getting Help is Simple, With Hims
0:3023 frames2.0s cut70% face
Made in Figma (2025) Like this ❤️ we designed it for you36M views
Figma
Made in Figma (2025) Like this ❤️ we designed it for you
0:3872 frames0.5s cut7% face
Browse every frame in the library