Python generators for database result sets
17th June 2003
I’ve read several articles on Python generators now, and I had almost got my head around them, but then I read this: Iterators and Databases by Andy Todd, which demonstrates a simple but intuitive way of using generators to iterate through rows from a database query without having to load all of the rows in to a list in memory first. Brilliant.
Andy links to this article which includes a nice one line summary of why generators exist (generators return iterators):
The motivation behind iterators is twofold. Working with data as sequences is often the most straightforward approach, and a sequence that is processed in linear order often does not need to actually exist all at once.
More on generators:
- Charming Python: Iterators and simple generators is a good introductory tutorial
- PEP 255: Simple Generators is the PEP that formally describes the language feature
More recent articles
- Weeknotes: datasette-enrichments, datasette-comments, sqlite-chronicle - 8th December 2023
- Datasette Enrichments: a new plugin framework for augmenting your data - 1st December 2023
- llamafile is the new best way to run a LLM on your own computer - 29th November 2023
- Prompt injection explained, November 2023 edition - 27th November 2023
- I'm on the Newsroom Robots podcast, with thoughts on the OpenAI board - 25th November 2023
- Weeknotes: DevDay, GitHub Universe, OpenAI chaos - 22nd November 2023
- Deciphering clues in a news article to understand how it was reported - 22nd November 2023
- Exploring GPTs: ChatGPT in a trench coat? - 15th November 2023
- Financial sustainability for open source projects at GitHub Universe - 10th November 2023
- ospeak: a CLI tool for speaking text in the terminal via OpenAI - 7th November 2023