Let’s be bear or bunny
The Machine Learning Compilation group (MLC) are my favourite team of AI researchers at the moment.
In less than two months they’ve released:
- Web Stable Diffusion—the full Stable Diffusion image generation model running entirely in a browser, after a 2GB download, powered by WebGPU. Try it here using Chrome Canary.
- Web LLM which pulls off the same trick but for vicuna-7b-v1, a genuinely useful LLaMA-derived ChatGPT-style Large Language Model! I wrote more about that in Web LLM runs the vicuna-7b Large Language Model entirely in your browser, and it’s very impressive.
And then a few days ago they released MLC LLM—“a universal solution that allows any language model to be deployed natively on a diverse set of hardware backends and native applications”.
As part of that release, they put up a TestFlight preview of an iPhone app that runs the LLM model locally on your phone. It needs 4GB of RAM just for the application, so a iPhone 12 Pro or later should work.
I’ve been running it on my phone. It totally works!
(This is actually the second LLM I’ve been able to run on my phone—I previously also tried Sheepy-T which runs the apparently less capable GPT-J.)
Since I’ve been writing online for more than twenty years I often try an ego search first, to see if the weird bundle of binary floating point numbers has picked up anything about me. Here’s what I got from the MLC LLM on my phone.
I could feel the back of my phone heating up as it slowly calculated all of those tokens!
Honestly, this is wildly better than I expected. I’m from the UK (but based in California since 2014). I wouldn’t call myself an “organizer” and while I care deeply about accessibility I’m not sure I’d credit myself as an outspoken advocate, but I’ve definitely advocated for web standards in the past.
But what on earth is that “Let’s be bear or bunny” pattern?
So I asked:
I think this is my favourite AI hallucination yet!
Here’s that definition, which I’m going to claim is co-written by myself and my hard working iPhone:
The “Let’s be bear or bunny” pattern is a way of looking at situations and problems in a certain way. It is a way of approaching a problem or situation with a sense of playfulness and lightheartedness, while still being able to think critically and analytically.
It is also a way of embracing the unexpected and being open to new possibilities. This pattern is often associated with the concept of “go with the flow” or “roll with the punches” in the face of challenges and difficulties. It encourages an attitude of relaxation and ease in the face of obstacles and difficulties
Honestly, I kind of love this. I’ve been trying out saying “Let’s be bear or bunny that!” at home. Natalie has hinted that this may have already played itself out, but I disagree.
More recent articles
- Weeknotes: Parquet in Datasette Lite, various talks, more LLM hacking - 4th June 2023
- It's infuriatingly hard to understand how closed models train on their input - 4th June 2023
- ChatGPT should include inline tips - 30th May 2023
- Lawyer cites fake cases invented by ChatGPT, judge is not amused - 27th May 2023
- llm, ttok and strip-tags - CLI tools for working with ChatGPT and other LLMs - 18th May 2023
- Delimiters won't save you from prompt injection - 11th May 2023
- Weeknotes: sqlite-utils 3.31, download-esm, Python in a sandbox - 10th May 2023
- Leaked Google document: "We Have No Moat, And Neither Does OpenAI" - 4th May 2023
- Midjourney 5.1 - 4th May 2023
- Prompt injection explained, with video, slides, and a transcript - 2nd May 2023