Simon Willison’s Weblog

Although fine-tuning can feel like the more natural option—training on data is how GPT learned all of its other knowledge, after all—we generally do not recommend it as a way to teach the model knowledge. Fine-tuning is better suited to teaching specialized tasks or styles, and is less reliable for factual recall. [...] In contrast, message inputs are like short-term memory. When you insert knowledge into a message, it’s like taking an exam with open notes. With notes in hand, the model is more likely to arrive at correct answers.

— Ted Sanders, OpenAI # 15th April 2023, 1:44 pm

One way to avoid unspotted prediction errors is for the technology in its current state to have early and frequent contact with reality as it is iteratively developed, tested, deployed, and all the while improved. And there are creative ideas people don’t often discuss which can improve the safety landscape in surprising ways — for example, it’s easy to create a continuum of incrementally-better AIs (such as by deploying subsequent checkpoints of a given training run), which presents a safety opportunity very unlike our historical approach of infrequent major model upgrades.

— Greg Brockman # 14th April 2023, 6:08 pm

Before we scramble to deeply integrate LLMs everywhere in the economy, can we pause and think whether it is wise to do so?

This is quite immature technology and we don’t understand how it works.

If we’re not careful we’re setting ourselves up for a lot of correlated failures.

— Jan Leike, Alignment Team lead, OpenAI # 13th April 2023, 7:08 pm