Simon Willison’s Weblog

Subscribe
Atom feed for jason-liu

2 posts tagged “jason-liu”

2025

I am once again shocked at how much better image retrieval performance you can get if you embed highly opinionated summaries of an image, a summary that came out of a visual language model, than using CLIP embeddings themselves. If you tell the LLM that the summary is going to be embedded and used to do search downstream. I had one system go from 28% recall at 5 using CLIP to 75% recall at 5 using an LLM summary.

Jason Liu

# 6th September 2025, 5:20 pm / ai, generative-ai, llms, embeddings, vision-llms, jason-liu

2024

What is prompt optimization? (via) Delightfully clear explanation of a simple automated prompt optimization strategy from Jason Liu. Gather a selection of examples and build an evaluation function to return a numeric score (the hard bit). Then try different shuffled subsets of those examples in your prompt and look for the example collection that provides the highest averaged score.

# 22nd May 2024, 4:02 pm / ai, prompt-engineering, generative-ai, llms, jason-liu