Alex Ziskind is not someone I mention often, but perhaps I should. I don’t have time to read the specs on every bit of hardware that comes out, so I watch his evaluations.
Here he’s managed to get his hands on a Tiiny.ai Pocket Sized AI Supercomputer.
The details, courtesy of Perplexity, are as follows:
CPU: 12‑core ARMv9.2 processor.
AI compute: Custom heterogeneous SoC + dedicated NPU, around 160–190 TOPS of AI throughput depending on source.
Memory/storage: 80 GB LPDDR5X (with ~48 GB mapped directly to the NPU) plus a 1 TB NVMe SSD.
Power envelope: About 30 W chip TDP and up to 65 W system power draw.
Form factor: Pocket mini‑PC, Guinness‑style “smallest mini PC that can run 100B+ LLM locally” marketing claim.
So this is a potent ARM based system with a slightly puzzling 32GB of shared memory and 48GB of ram dedicated to the NPU. It compares very favorably to the AMD 395+ AI Max systems that start around $2,300 for a 96GB unit.
Assessment:
I don’t know that anyone reading this Substack is going to buy a Kickstarter funded specialty device. This is appearing here because it’s redefining the envelope - instead of $2,300 to get GPT-OSS-120B running at home, now it’s $1,300.
If one of these things were dropped on my desk this morning, it would likely end up running an 8 bit quantized MedGemma-27B. I could put seven of these devices in a 1U rack tray for the price of ONE Nvidia RTX 6000 Pro. They aren’t very fast, but a broad, slow way to do a lot of medical inference is potentially of interest to me.
We ARE going to end up with inference in our homes and offices that we control, inference that’s going to be dramatically more capable than GPU based solutions, because it won’t BE a GPU, it’ll be dedicated to just one type of task.