Generative AI Weekly - May 30, 2024
Google's search result issues, another Klarna cost-savings case study, a public opinion report, and new use cases for generative models.
This week:
- Why is Google's LLM search integration getting slammed? A good reminder of where LLMs should not be trusted. "These chatbots can still be incredibly useful. But assuming they’re wrong is a different mental model from most technologies you use regularly."
- Another Klarna case study - saving S&M spend of $10M a year. 87% adoption among their employees using LLMs daily. 85% of their employees use an internal-facing custom LLM daily.
- I was thinking recently about how to explain the core functionality of LLMs at its most basic level. I recently re-read a book about Claude Shannon ("A Mind at Play" - highly recommended) where he created a game showing how LLMs would eventually work with his wife back in 1950. Ethan Mollick recently pointed to a more recent reference that (some) folks may find more relevant at the highest level.
- Reuter's releases a global survey on public sentiment regarding LLMs.
- Sentiment is generally surprising, and the optimism levels are very different from that of the circles I follow. ChatGPT is the clear market leader and Claude (a tool I use as much as GPT at this point) only has about 5% awareness.
- "Answering factual questions" is the highest level use-case.. which is concerning given hallucination and some of the issues above!
- Michael Taylor is a favorite performance marketer and writer of mine who has written two great articles recently regarding Creativity and Learning with AI.
- Somewhat technical, but a great read from O'Reilly on building with LLMs. Some less technical takeaways around evaluating when LLMs should be used, and the value of adding more context to prompts.
- Github CEO gives a TED Talk on the future of coding with LLMs.
- Another example of open-source LLMs achieving similar performance to SOTA models.
- Keyword-stuffing for resumes - LLM edition.
- And for this weekend's paper - Financial Statement Analysis with Large Language Models. "Overall, our analysis suggests that GPT shows a remarkable
aptitude for financial statement analysis and achieves state-of-the-art performance without any specialized training."
