-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Commit from GitHub Actions (Render and deploy Blog)
- Loading branch information
1 parent
5232ad2
commit 9f6fd99
Showing
12 changed files
with
1,029 additions
and
230 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -59,6 +59,64 @@ Open AI released a great Prompt Engineering guide (link in comments). <br><br>He | |
| <guid>https://www.bryanwhiting.com/posts/2023-12-15-prompt-techniques-/</guid> | ||
| <pubDate>Fri, 15 Dec 2023 00:00:00 GMT</pubDate> | ||
| </item> | ||
| <item> | ||
| <title>LLMs can find a needle in the haystack</title> | ||
| <dc:creator>Bryan Whiting</dc:creator> | ||
| <link>https://www.bryanwhiting.com/posts/2023-12-15-llms-can-find-a-needle-in-the-haystack/</link> | ||
| <description><![CDATA[ | ||
| <p><img src="https://www.bryanwhiting.com/posts/2023-12-15-llms-can-find-a-needle-in-the-haystack/photo.jpeg" class="img-fluid"></p> | ||
| <div class="callout callout-style-default callout-note callout-titled"> | ||
| <div class="callout-header d-flex align-content-center"> | ||
| <div class="callout-icon-container"> | ||
| <i class="callout-icon"></i> | ||
| </div> | ||
| <div class="callout-title-container flex-fill"> | ||
| Is RAG necessary when you have incredible memory? | ||
| </div> | ||
| </div> | ||
| <div class="callout-body-container callout-body"> | ||
| </div> | ||
| </div> | ||
| <section id="context" class="level1"> | ||
| <h1>Context</h1> | ||
| <p>Check out this thread:</p> | ||
| <blockquote class="twitter-tweet blockquote"> | ||
| <p lang="en" dir="ltr"> | ||
| (1/8) The Needle in the Haystack done by <a href="https://twitter.com/GregKamradt?ref_src=twsrc%5Etfw"><span class="citation" data-cites="GregKamradt">@GregKamradt</span></a> was an amazing analysis of retrieval performance! Greg has graciously allowed us to build on his work with a repository that is now OSS.<a href="https://twitter.com/natfriedman?ref_src=twsrc%5Etfw"><span class="citation" data-cites="natfriedman">@natfriedman</span></a> We have a much more rigorous test we’ve put out based on this idea.… <a href="https://t.co/i5O8zrcwQT">pic.twitter.com/i5O8zrcwQT</a> | ||
| </p> | ||
| — Aparna Dhinakaran (<span class="citation" data-cites="aparnadhinak">@aparnadhinak</span>) <a href="https://twitter.com/aparnadhinak/status/1735678863814938695?ref_src=twsrc%5Etfw">December 15, 2023</a> | ||
| </blockquote> | ||
| <script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> | ||
| <p>This is a powerful analysis. Sure, Anthropic will find a way to improve or challenge the results. But the point is clear: these technologies can remember hyper specific 7-digit random numbers out of a batch of 126,000 tokens, where a token is roughly 4 characters. GPT is clear winner here, too.</p> | ||
| <p>Also, open source is getting incredibly good. This implies the future is open source.</p> | ||
| <blockquote class="twitter-tweet blockquote"> | ||
| <p lang="en" dir="ltr"> | ||
| Comparing <a href="https://twitter.com/OpenAI?ref_src=twsrc%5Etfw"><span class="citation" data-cites="OpenAI">@OpenAI</span></a> <a href="https://twitter.com/hashtag/GPT4?src=hash&ref_src=twsrc%5Etfw">#GPT4</a> Turbo to <a href="https://twitter.com/MistralAI?ref_src=twsrc%5Etfw"><span class="citation" data-cites="MistralAI">@MistralAI</span></a> <br><br>GPT-4 is pretty good in that region in general. Interesting to see how <a href="https://twitter.com/MistralAI?ref_src=twsrc%5Etfw"><span class="citation" data-cites="MistralAI">@MistralAI</span></a> scales to larger context windows <a href="https://t.co/WQo6MmGIHh">pic.twitter.com/WQo6MmGIHh</a> | ||
| </p> | ||
| — Aparna Dhinakaran (<span class="citation" data-cites="aparnadhinak">@aparnadhinak</span>) <a href="https://twitter.com/aparnadhinak/status/1735747087021600916?ref_src=twsrc%5Etfw">December 15, 2023</a> | ||
| </blockquote> | ||
| <script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> | ||
| </section> | ||
| <section id="impact" class="level1"> | ||
| <h1>Impact</h1> | ||
| <p>RAG can be used to make retrieval more efficient. But if retrieval is already super efficient maybe RAG is only a short term thing. Context lengths of 10m tokens…probably by next year right?</p> | ||
| <p>Start of the year we were at 4K tokens. Now there are 126,000 tokens. 30x improvement. So to do another 30x improvement is 3.76M. So yea, by next year you should be able to just load the entire RAG database into memory. But…gonna be super expensive.</p> | ||
| <p>Point is: would GPT be this effective if it was using RAG over a database? Or is it more effective loading it all into context?</p> | ||
| </section> | ||
| <p>_________________________ <br> <em>For coaching on data analytics or machine learning, <a href="mailto:[email protected]">reach out</a></em>.</p> ]]></description> | ||
| <category>ai</category> | ||
| <guid>https://www.bryanwhiting.com/posts/2023-12-15-llms-can-find-a-needle-in-the-haystack/</guid> | ||
| <pubDate>Fri, 15 Dec 2023 00:00:00 GMT</pubDate> | ||
| </item> | ||
| <item> | ||
| <title>Learn by doing</title> | ||
| <dc:creator>Bryan Whiting</dc:creator> | ||
|
|
@@ -1460,45 +1518,5 @@ Tip | |
| <pubDate>Wed, 22 Jun 2022 00:00:00 GMT</pubDate> | ||
| <media:content url="https://www.bryanwhiting.com/posts/2022-06-22-downstream-impact/photo.jpeg" medium="image" type="image/jpeg"/> | ||
| </item> | ||
| <item> | ||
| <title>The Power of Party Pay by Visible</title> | ||
| <dc:creator>Bryan Whiting</dc:creator> | ||
| <link>https://www.bryanwhiting.com/posts/2022-06-21-the-power-of-party-pay-by-visible/</link> | ||
| <description><![CDATA[ | ||
| <p><img src="https://www.bryanwhiting.com/posts/2022-06-21-the-power-of-party-pay-by-visible/photo.jpeg" class="img-fluid"></p> | ||
| <section id="visibles-party-pay-madness" class="level1"> | ||
| <h1>Visible’s party pay madness</h1> | ||
| <p>Visible wireless offers $25 off to users who join a party. So a user who signs up to pay $50 for unlimited data can click one button to join a party and only have to pay $25/mo from then on. Why not just offer users $25 to begin with?</p> | ||
| <p>It’s really confusing to me as a user - why not just give me the $25 automatically if it takes nothing to join a group? Because of gamification.</p> | ||
| </section> | ||
| <section id="why-it-works" class="level1"> | ||
| <h1>Why it works</h1> | ||
| <p>I think their strategy is as follows:</p> | ||
| <ol type="1"> | ||
| <li>They hope some people get brought in on marketing and never remember to activate a party pay. Let’s say this number is 20-40%. Perhaps these users fund the rest of the discounts.</li> | ||
| <li>Joining a party pay group for free is an easy way to make your customers win. I.e., a user gets $25 off for joining. Gimmicky, but I think it works. So a user has an immediate sense of gratification - a great way to build a strong emotional bond. “Click a button to get $25 off in perpetuity? Nice!” Anyone would agree this feels better than just starting off with $25.</li> | ||
| <li>This makes users want to share and tell others. Especially if customers feel they’ve gamed the system. Who today doesn’t like a hack?</li> | ||
| </ol> | ||
| <p>But even if (1) is wrong, and that 95% of people join a party I’m sure they’ve done the math to consider how many people they need to sign up to make the money work.</p> | ||
| </section> | ||
| <section id="the-larger-strategy" class="level1"> | ||
| <h1>The Larger Strategy</h1> | ||
| <p>Why would Verizon cannibalize their own sales? I switched my wife out of a $70 unlimited plan on Verizon to a $25 plan on visible for the same service. Well some customers will stay with Verizon because of its prestige and the add on benefits. But users who are cutting the cord are going somewhere else, I’m thinking Verizon wants to stay ahead of the curve. It’s a race to the bottom, sure. But it doesn’t hurt to try to win the race.</p> | ||
| </section> | ||
| <p>_________________________ <br> <em>For coaching on data analytics or machine learning, <a href="mailto:[email protected]">reach out</a></em>.</p> ]]></description> | ||
| <category>startups</category> | ||
| <category>strategy</category> | ||
| <guid>https://www.bryanwhiting.com/posts/2022-06-21-the-power-of-party-pay-by-visible/</guid> | ||
| <pubDate>Tue, 21 Jun 2022 00:00:00 GMT</pubDate> | ||
| <media:content url="https://www.bryanwhiting.com/posts/2022-06-21-the-power-of-party-pay-by-visible/photo.jpeg" medium="image" type="image/jpeg"/> | ||
| </item> | ||
| </channel> | ||
| </rss> | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes
Oops, something went wrong.