Simon Willison’s Weblog

Claude 3.5 Haiku

4th November 2024

Anthropic released Claude 3.5 Haiku today, a few days later than expected (they said it would be out by the end of October).

I was expecting this to be a complete replacement for their existing Claude 3 Haiku model, in the same way that Claude 3.5 Sonnet eclipsed the existing Claude 3 Sonnet while maintaining the same pricing.

Claude 3.5 Haiku is different. First, it doesn’t (yet) support image inputs—so Claude 3 Haiku remains the least expensive Anthropic model for handling those.

Secondly, it’s not priced the same as the previous Haiku. That was $0.25/million input and $1.25/million for output—the new 3.5 Haiku is 4x that at $1/million input and $5/million output.

Anthropic tweeted:

During final testing, Haiku surpassed Claude 3 Opus, our previous flagship model, on many benchmarks—at a fraction of the cost.

As a result, we’ve increased pricing for Claude 3.5 Haiku to reflect its increase in intelligence.

Given that Anthropic claim that their new Haiku out-performs their older Claude 3 Opus (still $15/m input and $75/m output!) this price isn’t disappointing, but it’s a small surprise nonetheless.

Accessing Claude 3.5 Haiku with LLM

I released a new version of my llm-claude-3 plugin with support for the new model. You can install (or upgrade) the plugin and run it like this:

llm install --upgrade llm-claude-3
llm keys set claude
# Paste API key here
llm -m claude-3.5-haiku 'describe memory management in Rust'

Here’s the output from that prompt.

Comparing prices

I added the new price to my LLM pricing calculator, which inspired me to extract this comparison table for the leading models from Gemini, Anthropic and OpenAI. Here they are sorted from least to most expensive:

Gemini 1.5 Flash-8B remains the model to beat on pricing: it’s 1/6th of the price of the new Haiku—far less capable, but still extremely useful for tasks such as audio transcription.

Also notable from Anthropic’s model comparison table: Claude 3.5 Haiku has a max output of 8,192 tokens (same as 3.5 Sonnet, but twice that of Claude 3 Opus and Claude 3 Haiku). 3.5 Haiku has a training cut-off date of July 2024, the most recent of any Anthropic model. 3.5 Sonnet is April 2024 and the Claude 3 family are all August 2023.