http://simonwillison.net/2025-09-06T17:20:27+00:00Simon Willison2025-09-06T17:20:27+00:002025-09-06T17:20:27+00:00https://simonwillison.net/2025/Sep/6/jason-liu/#atom-tag

<blockquote cite="https://twitter.com/jxnlco/status/1964050092312211636"><p>I am once again shocked at how much better image retrieval performance you can get if you embed highly opinionated summaries of an image, a summary that came out of a visual language model, than using CLIP embeddings themselves. If you tell the LLM that the summary is going to be embedded and used to do search downstream. I had one system go from 28% recall at 5 using CLIP to 75% recall at 5 using an LLM summary.</p></blockquote> <p class="cite">&mdash; <a href="https://twitter.com/jxnlco/status/1964050092312211636">Jason Liu</a></p> <p>Tags: <a href="https://simonwillison.net/tags/ai">ai</a>, <a href="https://simonwillison.net/tags/generative-ai">generative-ai</a>, <a href="https://simonwillison.net/tags/llms">llms</a>, <a href="https://simonwillison.net/tags/embeddings">embeddings</a>, <a href="https://simonwillison.net/tags/vision-llms">vision-llms</a>, <a href="https://simonwillison.net/tags/jason-liu">jason-liu</a></p>

2024-05-22T16:02:10+00:002024-05-22T16:02:10+00:00https://simonwillison.net/2024/May/22/what-is-prompt-optimization/#atom-tag

<p><strong><a href="https://jxnl.co/writing/2024/05/22/what-is-prompt-optimization/">What is prompt optimization?</a></strong></p> Delightfully clear explanation of a simple automated prompt optimization strategy from Jason Liu. Gather a selection of examples and build an evaluation function to return a numeric score (the hard bit). Then try different shuffled subsets of those examples in your prompt and look for the example collection that provides the highest averaged score. <p><small></small>Via <a href="https://twitter.com/jxnlco/status/1793279761696895047">@jxnlco</a></small></p> <p>Tags: <a href="https://simonwillison.net/tags/ai">ai</a>, <a href="https://simonwillison.net/tags/prompt-engineering">prompt-engineering</a>, <a href="https://simonwillison.net/tags/generative-ai">generative-ai</a>, <a href="https://simonwillison.net/tags/llms">llms</a>, <a href="https://simonwillison.net/tags/jason-liu">jason-liu</a></p>