動画一覧へ戻る

Alex Albert

Prompt engineering is hard. That's why we've been adding tons of feat...

プラットフォーム
X
長さ
9:16
話されている言語(推定)
英語
再生数
12.2万
カット間隔の中央値
23.86s
顔の映る割合
100%
暗いフレーム
64%
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 0:00corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
0:00
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 0:20corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
0:20
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 1:07corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
1:07
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 1:16corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
1:16
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 1:31corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
1:31
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 1:36corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
1:36
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 1:48corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
1:48
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 2:52corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
2:52
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 3:00corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
3:00
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 3:24corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
3:24
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 3:32corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
3:32
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 4:04corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
4:04
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 4:12corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
4:12
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 4:20corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
4:20
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 5:08corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
5:08
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 5:12corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
5:12
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 5:15corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
5:15
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 5:24corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
5:24
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 5:32corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
5:32
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 5:56corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
5:56
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 6:04corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
6:04
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 6:20corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
6:20
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 6:36corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
6:36
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 6:52corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
6:52
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 7:16corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
7:16
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 7:40corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
7:40
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 7:56corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
7:56
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 8:12corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
8:12
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 8:28corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
8:28
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 8:36corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
8:36
Alex Albert Prompt engineering is hard. That's why we've been adding tons of feat... at 9:16corner inset
Alex AlbertPrompt engineering is hard. That's why we've been adding tons of feat...
9:16

Prompt engineering is hard. That's why we've been adding tons of feat... の文字起こし

ブランドが書いたものではなく、映像の音声から音声モデルが書き起こしたものです。行をクリックすると、その瞬間に移動します。

  1. 0:00Today, I want to show off a range of improvements that we've made recently to the Anthropic
  2. 0:05Console in Workbench and illustrate how we are trying to cut down the time that devs
  3. 0:12have to spend prompt engineering by up to 90%.
  4. 0:14Before I jump into the features, I want to quickly walk through how many devs currently
  5. 0:21do the prompt engineering process.
  6. 0:23It might look something like this.
  7. 0:25You were working within a spreadsheet, you have a prompt, you have another column with all your test cases for that prompt, and then you categorize all the different outputs from various models that you're evaluating within separate columns in this spreadsheet.
  8. 0:42Potentially, you have an ideal output column as well, and maybe you're even rating these outputs against the ideal output.
  9. 0:49This is very tedious, and it takes a lot of time.
  10. 0:51So we thought to ourself, well, we have this LLM that can help us write prompts.
  11. 0:58Maybe there's a way that we can integrate Claude into the actual prompt
  12. 1:02engineering process itself.
  13. 1:04So I'm going to show off what that looks like in a second here.
  14. 1:07But let's go back to this tab and let's click into write a prompt from scratch.
  15. 1:13That's going to take us into the workbench.
  16. 1:15And the workbench will be greeted with this interface.
  17. 1:18You can see at the top we have the optionality for you to include a system prompt,
  18. 1:22give me some examples which I'll touch on in a moment, and then our first user
  19. 1:27message to Claude. This would be the beginning of your prompt. I'm going to go
  20. 1:31back into the spreadsheet and copy paste this out of here, go back into the
  21. 1:37workbench, paste it in here, and now we can run this prompt in the workbench.
  22. 1:43When we try to run it, we detect that we've inserted a variable,
  23. 1:47as you can tell with these double curly brackets on both sides.
  24. 1:51In this case, we want an article that we can summarize within our prompt.
  25. 1:56But maybe in this case we don't have an article.
  26. 1:58We can actually use Claude to generate an article for us.
  27. 2:01So I'm going to hit this generate button up top.
  28. 2:04And Claude is now going to draft a test case for us right here.
  29. 2:10You can see that start to stream out.
  30. 2:12The cool part about this is that the generation logic behind generating that example is actually included right here.
  31. 2:19So if we wanted to change this, we can go in here and we can add maybe a new rule for Claude.
  32. 2:26Like the article must be greater than a thousand words.
  33. 2:32We can click regenerate and Claude will look at the new generation logic and then write a test case that conforms to those new rules for us.
  34. 2:40And that is streaming out and once it finishes we can run this. Great, so it's finished. Let's click run.
  35. 2:46And now we're running the article against our prompt.
  36. 2:50We have the model selected here so we're running against the latest 3.5 sonnet model.
  37. 2:55Now we have our prompt, but this is leaving some room for improvement.
  38. 3:00We could do a lot better with this prompt.
  39. 3:02Maybe there is something wrong with the output, or it's not formatted correctly, or it's too long, or it's too short.
  40. 3:09If we don't want to spend the time working through the prompt engineering process,
  41. 3:14we can actually just have Claude do it for us.
  42. 3:16So we'll click this Improve Prompt button.
  43. 3:18And if you had any sort of detailed feedback, you can provide it here.
  44. 3:21Of course, the more detailed the feedback, the more Claude will actually be able to help you.
  45. 3:26But in our case, we're just going to run it without any feedback and click Improve Prompt.
  46. 3:31This kicks off a process that is powered by Claude in the back end to actually draft the new improved prompt for you.
  47. 3:38It goes through a planning step, and then it writes an initial draft.
  48. 3:43And then once we have that initial draft, we actually pass it back to Claude again.
  49. 3:47and Claude will read through that draft
  50. 3:49and plan some more revisions.
  51. 3:52And then finally, it will write the final draft
  52. 3:55and that will stream out here.
  53. 3:58And then we can actually go ahead
  54. 4:00and open that new draft in the workbench.
  55. 4:04So Claude's right, this improved prompt for us.
  56. 4:06We can go in here.
  57. 4:07Of course, we can change things.
  58. 4:09We can add things, whatever we need to do.
  59. 4:11In this case, I'm just going to run it as is.
  60. 4:13We already have that test case generated.
  61. 4:16Let's just click run again.
  62. 4:17And this time you'll see that the output is completely different.
  63. 4:21Claude has rewritten this problem so that we use techniques like pre-fill and
  64. 4:25chain of thought to actually pick the main topic out of the article and
  65. 4:29the key points and then draft a summary.
  66. 4:32And then it's doing something very unique where it's numbering the words
  67. 4:35within that summary so that it stays within our 20 to 40 word constraint.
  68. 4:39These are all suggestions that the prompt writing Claude made to improve this prompt.
  69. 4:45And then finally, it spits out the summary here within these XML tags.
  70. 4:50So if we were a developer, we're writing an application,
  71. 4:53we can easily extract that out of the overall response.
  72. 4:57This is looking pretty good, but let's say that we wanted to take this step further and
  73. 5:01now evaluate it across different models and maybe across different prompt versions as well.
  74. 5:06We can click into the evaluate tab and you'll be greeted with this interface.
  75. 5:10You might realize that this looks pretty similar to something like this in the spreadsheet.
  76. 5:16The advantage of this though is that Claude is embedded everywhere.
  77. 5:21Let's actually add a comparison column right here by clicking Add Comparison,
  78. 5:25and we can select our old prompt version.
  79. 5:28Now we can compare the V2 to the V1, and we can easily see what the different outputs are.
  80. 5:34But maybe one case isn't enough here.
  81. 5:37Let's say we wanted another article to test both these prompts on.
  82. 5:40We could go add a row and pull in another article manually,
  83. 5:45or we could import one from maybe a spreadsheet that we have in the past,
  84. 5:49or we could just ask Claude to create another one for us.
  85. 5:52We're going to do that. Let's click Generate Test Case.
  86. 5:56Claude will spit out a new article for us.
  87. 6:00You can see that start to stream in here.
  88. 6:02And once that's done, we can then run this against our different problem versions
  89. 6:06directly in this Evaluate tab.
  90. 6:08I'm going to click run here and here, first running against the new v2.
  91. 6:13So this is a little bit of a longer output so it takes a second.
  92. 6:16Boom, there's our output.
  93. 6:18And then now it's running against v1.
  94. 6:23So now we have these new outputs on the new test case.
  95. 6:25We can go in here and we can rate these.
  96. 6:28Maybe this one is 5 out of 5 and this one's 1 out of 5.
  97. 6:31And then if we're satisfied with one of our responses, we can actually copy it, go into
  98. 6:37this ideal output column and paste it in.
  99. 6:40This allows us to have a saved response.
  100. 6:43If we're going to add more rows or more prompt versions or
  101. 6:47test against different models, we can then go back and
  102. 6:50compare to what we felt was our perfect output at the time.
  103. 6:53This is very, very powerful.
  104. 6:55Again, this mimics that same sort of workflow that you'd see when
  105. 6:58you're working within a spreadsheet.
  106. 7:00The only difference is Claude is embedded again everywhere within here,
  107. 7:04which cuts down the work on your end.
  108. 7:06The last thing I want to show is back in the prompt section around including examples.
  109. 7:13Examples are a very powerful prompt engineering technique where you give Claude a test case
  110. 7:17and then you give Claude an ideal output and you basically instruct Claude to follow that format
  111. 7:25or whatever it is, the response style that you showed it in the example in the prompt.
  112. 7:29We've made that something that's very easy to embed into your prompts now by just clicking this button.
  113. 7:36This pulls up a tab that allows you to input an article,
  114. 7:39input the ideal output for that test case article,
  115. 7:43and then add that to the prompt.
  116. 7:45But in our case, we don't have either of those.
  117. 7:48So again, I'm going to ask Claude to generate an article for us.
  118. 7:52Once that finishes, we can go in and we can, again,
  119. 7:56make changes to the article if we want.
  120. 7:58And then we can provide our ideal output,
  121. 8:00which I'm too lazy right now to write out.
  122. 8:02So I'm just going to have Claude draft me an ideal output.
  123. 8:06And it will stream something out here.
  124. 8:08And in most cases, you're actually going to want to go in and change this to be the ideal output for you and for your application.
  125. 8:17We could go in here.
  126. 8:18We could change some words around.
  127. 8:19We can modify this in whichever way we want.
  128. 8:22And then once we're satisfied with our output in our example, we can click add example.
  129. 8:28And that will insert it directly into our prompt.
  130. 8:30To take a look at what that looks like in the code, let's click this get code button.
  131. 8:36and you'll see this tab pop up.
  132. 8:38We support all the different ways you can access Claude,
  133. 8:41all the different code snippets.
  134. 8:43And in this one in Python,
  135. 8:44you can see that within this messages array
  136. 8:46in the user block,
  137. 8:47we've added in an examples text here.
  138. 8:51We wrap the examples for you in these XML tags.
  139. 8:54We have the article,
  140. 8:56and if you scroll all the way to the end,
  141. 8:57you'll see that we also have the ideal output
  142. 9:00pended to this as well.
  143. 9:01We can easily copy that code out of here,
  144. 9:04or if we're looking for even more information we can view our docs.
  145. 9:08Otherwise I'm just going to close it out here.
  146. 9:11So this is the workbench.
  147. 9:12Go try it out on your own prompts
  148. 9:14and let us know how it works for you in the comments below.
148行 · 1,719語