Große Sprachmodelle: GPT-4, LLaMA & Co 🎙️

<article class="post-detail"> <header> <h2 class="post-title"> <a href="https://python-podcast.de/show/grosse-sprachmodelle-gpt-4-llama-co/">Große Sprachmodelle: GPT-4, LLaMA &amp; Co 🎙️</a> (click here to comment) </h2> <!-- link is on one line to avoid underlined whitespace --> <div class="post-card-meta"> <a href="https://python-podcast.de/show/grosse-sprachmodelle-gpt-4-llama-co/"><time datetime="2023-04-06T15:00:00+02:00">6. April 2023</time>,</a> <span class="author">Jochen</span> </div> </header> <div class="post-body"> <section class="block-overview"> <section class="block-paragraph"> <p data-block-key="o45ar">Disclaimer: Sowohl Titel wie auch Beschreibung wurden von GPT-4 generiert. Kontext war die Liste der Links aus den Shownotes.</p><p data-block-key="3tv40"></p><p data-block-key="c3cn5"><a href="https://the.scapegoat.dev/">Manuel</a>, <a href="https://mastodon.world/@shezi">Johannes</a>, <a href="https://twitter.com/oryon_dominik">Dominik</a> &amp; <a href="https://fedi.wersdoerfer.de/@jochen">Jochen</a> tauschen sich über große Sprachmodelle (LLMs) wie GPT-4 und LLaMA aus 🌐. Sie besprechen faszinierende Anwendungen in Projekten wie GitHub Copilot und BlenderGPT 🛠️, sowie die Rolle von Word Embeddings und Reinforcement Learning from Human Feedback (RLHF) in der Modellentwicklung 📊. ChatGPT wird dabei als Beispiel hervorgehoben, das die Nützlichkeit von LLMs einer breiteren Öffentlichkeit verdeutlicht hat. Die Diskussion umfasst auch ethische Bedenken im Zusammenhang mit LLMs 🚨 und schließt mit Empfehlungen für vertiefende Ressourcen 📚🎧.</p> </section> <section class="block-audio"> </section> </section> <section class="block-detail"> <section class="block-paragraph"> <h2 data-block-key="34zwn">Shownotes</h2><p data-block-key="ek0qq">Unsere E-Mail für Fragen, Anregungen &amp; Kommentare: <a href="mailto:[email protected]">[email protected]</a></p><h3 data-block-key="52q2s">News Allgemeines Geplauder</h3><ul><li data-block-key="6p7is"><a href="https://futureoflife.org/open-letter/pause-giant-ai-experiments/">Pause Giant AI Experiments: An Open Letter</a></li><li data-block-key="fc8ra"><a href="https://aiguide.substack.com/p/thoughts-on-a-crazy-week-in-ai-news">Thoughts on a Crazy Week in AI News</a></li><li data-block-key="8asvi"><a href="https://github.com/features/copilot">GitHub Copilot</a></li><li data-block-key="75b1e"><a href="https://www.jetbrains.com/de-de/fleet/">JetBrains Fleet</a></li><li data-block-key="538mn"><a href="https://platform.openai.com/docs/models/gpt-3">GPT-3</a> (generative pre-trained transformer) / <a href="https://en.wikipedia.org/wiki/Few-shot_learning_(natural_language_processing)">Few-shot learning</a> / <a href="https://en.wikipedia.org/wiki/Prompt_engineering#Chain-of-thought">Chain-of-thought</a></li><li data-block-key="3l5hq"><a href="https://openai.com/product/gpt-4">GPT-4</a></li><li data-block-key="end87"><a href="https://cims.nyu.edu/~sbowman/eightthings.pdf">Eight Things to Know about Large Language Models</a> | Sehr interessante Zusammenfassung von Dingen, die man bisher weiß</li><li data-block-key="c3b4u"><a href="https://github.com/gd3kr/BlenderGPT">BlenderGPT - This addon allows you to use Blender with natural language commands using OpenAI&#x27;s GPT-3.5/GPT-4</a></li><li data-block-key="7ihac"><a href="https://ai.facebook.com/blog/large-language-model-llama-meta-ai/">Introducing LLaMA: A foundational, 65-billion-parameter large language model</a> / <a href="https://github.com/antimatter15/alpaca.cpp">Alpaca.cpp</a> / <a href="https://vicuna.lmsys.org">Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality</a></li><li data-block-key="26b8n">GPUs in der Cloud: <a href="https://www.beam.cloud">beam.cloud</a> / <a href="https://www.pipeline.ai">pipeline.ai</a> / <a href="https://www.cerebrium.ai/">crebrium.ai</a> / <a href="https://www.banana.dev">banana.dev</a></li><li data-block-key="7ff36"><a href="https://huggingface.co">Hugging Face</a> / <a href="https://transformersbook.com">Natural Language Processing with Transformers</a> (Book)</li><li data-block-key="1f00g"><a href="https://github.com/ggerganov/llama.cpp">Inference of LLaMA model in pure C/C++</a></li></ul><h3 data-block-key="6dl3e">Large Language Models</h3><ul><li data-block-key="3mjr6"><a href="https://youtu.be/kCc8FmEb1nY">Let&#x27;s build GPT: from scratch, in code, spelled out</a></li><li data-block-key="7tcgk"><a href="https://arxiv.org/abs/1706.03762">Attention Is All You Need</a> | Das ursprüngliche Transformer-Paper</li><li data-block-key="2or0c"><a href="https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluigi-effect-mega-post">The Waluigi Effect (mega-post)</a></li><li data-block-key="3eqh2"><a href="https://github.com/hwchase17/langchain">LangChain</a> | Building applications with LLMs through composability</li><li data-block-key="c3pqd"><a href="https://openai.com/blog/chatgpt-plugins">ChatGPT plugins</a></li><li data-block-key="1h14k"><a href="https://en.wikipedia.org/wiki/Zero-shot_learning">Zero-shot learning</a></li><li data-block-key="e9o9f"><a href="https://dl.acm.org/doi/pdf/10.1145/3442188.3445922">On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?</a></li><li data-block-key="eb256"><a href="https://link.springer.com/article/10.1007/s11229-022-03931-4">Understanding models understanding language</a> | Was Modelle aus Text über Farben lernen</li><li data-block-key="dnqav"><a href="https://en.wikipedia.org/wiki/Global_workspace_theory">Global workspace theory</a></li><li data-block-key="86j31"><a href="https://en.wikipedia.org/wiki/Bouba/kiki_effect">Bouba/kiki effect</a></li><li data-block-key="bdij9"><a href="https://github.com/lllyasviel/ControlNet">ControlNet</a></li><li data-block-key="3mdom"><a href="https://en.wikipedia.org/wiki/Word_embedding#:~:text=In%20natural%20language%20processing%20(NLP,to%20be%20similar%20in%20meaning.">Word embeddings</a></li><li data-block-key="baqd8"><a href="https://github.com/simonw/llm">llm command line tool</a></li><li data-block-key="9s842"><a href="https://github.com/go-go-golems/geppetto">Geppetto - go LLM and GPT3 specific prompting framework</a> | enthält das cli tool pinocchio</li><li data-block-key="bujlh"><a href="https://github.com/kovidgoyal/kitty">kitty - the fast, feature-rich, cross-platform, GPU based terminal</a></li><li data-block-key="2fqf4"><a href="https://github.com/asottile/pyupgrade">pyupgrade A tool (and pre-commit hook) to automatically upgrade syntax for newer versions of the language</a></li><li data-block-key="3lvvj"><a href="https://en.wikipedia.org/wiki/Weak_supervision">Semi supervised learning</a></li><li data-block-key="9r3rp"><a href="https://huggingface.co/blog/rlhf">Illustrating Reinforcement Learning from Human Feedback (RLHF)</a></li><li data-block-key="f1n8a"><a href="https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/">What Is ChatGPT Doing … and Why Does It Work?</a></li><li data-block-key="ejukp"><a href="https://jalammar.github.io/illustrated-transformer/">The Illustrated Transformer</a></li><li data-block-key="5ltc0"><a href="https://cims.nyu.edu/~sbowman/eightthings.pdf">Eight Things to Know about Large Language Models</a></li></ul><h3 data-block-key="991bk">Medien zum Vertiefen</h3><ul><li data-block-key="cplui"><a href="https://shows.acast.com/the-robot-brains/episodes/yoshua-bengio">Yoshua Bengio: large language models, higher cognition, causality, working memory, responsible AI</a> (<a href="https://shows.acast.com/the-robot-brains">The Robot Brains Podcast</a>) | Sehr gut!</li><li data-block-key="7m6tf"><a href="https://hotelbarpodcast.com/podcast/episode-88-chatgpt/">Episode 88: ChatGPT</a> (<a href="https://hotelbarpodcast.com">Hotel Bar Sessions</a>) | Meh, aber interessant wie daneben die liberal arts Leute liegen</li><li data-block-key="bfbd"><a href="https://www.nytimes.com/2023/03/21/opinion/ezra-klein-podcast-kelsey-piper.html">A.I. Is About to Get Much Weirder. Here’s What to Watch For.</a> (<a href="https://www.nytimes.com/column/ezra-klein-podcast">The Ezra Klein Show</a>) | Quite solid for a journalistic publication</li><li data-block-key="8abdl"><a href="https://www.latent.space/p/chatgpt-gpt4-hype-and-building-llm#details">ChatGPT, GPT4 hype, and Building LLM-native products — with Logan Kilpatrick of OpenAI</a> (<a href="https://www.latent.space/podcast">Latent Space Podcast</a>)</li><li data-block-key="c57bu"><a href="https://youtu.be/xKV8_N1b5RI">Prompt Engineering and AI Constitutions with Stephen Wolfram</a></li></ul><h3 data-block-key="c3bf6">Picks</h3><ul><li data-block-key="eddqt"><a href="https://streamlit.io/">streamlit.io - A faster way to build and share data apps</a></li><li data-block-key="fn17j"><a href="https://github.com/charliermarsh/ruff">ruff - An extremely fast Python linter, written in Rust</a></li><li data-block-key="48r6k"><a href="https://jamesturk.github.io/scrapeghost/">Scrapeghost is an experimental library for scraping websites using OpenAI&#x27;s GPT</a></li><li data-block-key="bju1m"><a href="https://github.com/gd3kr/BlenderGPT">BlenderGPT - This addon allows you to use Blender with natural language commands using OpenAI&#x27;s GPT-3.5/GPT-4</a></li><li data-block-key="a04nn"><a href="https://www.descript.com/">Descript is the simple, powerful, and fun way to edit</a></li></ul><h3 data-block-key="aeltm">Manuels Links</h3><ul><li data-block-key="6cotc"><a href="https://github.com/go-go-golems/geppetto">Geppetto - go LLM and GPT3 specific prompting framework</a></li><li data-block-key="7c09n"><a href="https://github.com/go-go-golems/glazed">glazed - Output structured data in a variety of formats</a></li><li data-block-key="9f3jn"><a href="https://github.com/go-go-golems/escuse-me">GO GO ESCUSE ME</a></li><li data-block-key="267nn"><a href="https://gist.github.com/wesen/e02efb6c374c426654a565abda5e5592">zsh ding</a></li><li data-block-key="3tvcl"><a href="https://gist.github.com/wesen/a4b96784f448685ec94afdfcd48bf0ec">kitty script</a></li></ul> </section> </section> </div> </article>