Raspberry Pi5 LLM performance
<!-- SC_OFF --><div class="md"><p>Hey all,</p> <p>To preface: A while ago I asked if anyone had benchmarks for the performance of larger (30B/70B) models on a Raspi: there were none (or I didn't find them). This is just me sharing information/benchmarks for anyone who needs it or finds it interesting.</p> <p>I tested the following models:</p> <ul> <li>Qwen3.5 from 0.8B to 122B-A10B</li> <li>Gemma 3 12B</li> </ul> <p>Here is my setup and the <code>llama-bench</code> results for zero context and at a depth of 32k to see how much performance degrades. I'm going for quality over speed, so of course there is room for improvements when using lower quants or even KV-cache quantization.</p> <p>I have a Raspberry Pi5 with:</p> <ul> <li>16GB RAM</li> <li>Active Cooler (stock)</li> <li>1TB SSD connec
Could not retrieve the full article text.
Read on Reddit r/LocalLLaMA →Reddit r/LocalLLaMA
https://www.reddit.com/r/LocalLLaMA/comments/1s8xuew/raspberry_pi5_llm_performance/Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
llamamodelbenchmark
How much faster is speaking, compared to typing on laptop vs phone vs writing?
So as I haven’t been able to speak the past short while , one thing I have noticed is that it is harder to communicate with others. I know what you are thinking: “Wow, who could have possibly guessed? It’s harder to converse when you can’t speak?”. Indeed, I didn’t expect it either. But how much harder is it to communicate? One proxy you can use is the classic typing metric, words per minute (wpm). So I spend some time looking at various forms of communication and how they differ between one another. For most tests, i used https://www.typingtom.com/english/typing-test/30s So I list below the forms of communication I have tried and how slow they are. Here are the rough tiers that I found: Ultra-slow-speed tier (~10-20wpm) Shaping out non-standard letters with my hands This is obviously the
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!