Weeknotes: Trapped in an eternal refactor
16th December 2021
I’m still working on refactoring Datasette’s table view. In doing so I spun out a new plugin, datasette-pretty-traces, which improves Datasette’s tooling for seeing the SQL that was executed to build a specific page.
You can try it out here.
I’ll probably merge this into Datasette core at some point in the future, but a neat thing about having plugin support is I can dash out initial versions of things like this without needing to polish them up and include them in a formal release of the parent project.
Progress on the eternal refactor
Datasette’s table view is the most important page in the application: it’s the interface that lets you browse a table, filter it, search it, run faceting against it and export it out as other formats.
It’s the nastiest code in the entire project, having grown to over a thousand lines of Python. While it has very thorough tests, the actual code itself is unwieldy enough that it’s slowing down progress on all kinds of things I want to get done before I ship Datasette 1.0.
So I’m picking away at it. I’ve broken the underlying tests up into two modules (test_table_api.py and test_table_html.py) and I’ve made some small improvements, but I’ve also spun up some not-yet-committed prototypes both against my experimental asyncinject library and a new experiment that involves something that, if you squint at it, looks a tiny bit like a new ORM. I do not want to build a new ORM!
I’m not happy with any of this yet, and it’s definitely blocking my progress on other things. I’ll just have to keep on chipping away and see if I can get to a breakthrough.
Releases this week
datasette-pretty-traces: 0.2.1—(3 releases total)—2021-12-13
Prettier formatting for ?_trace=1 traces
TIL this week
More recent articles
- Weeknotes: the Datasette Cloud API, a podcast appearance and more - 1st October 2023
- Things I've learned about building CLI tools in Python - 30th September 2023
- Talking Large Language Models with Rooftop Ruby - 29th September 2023
- Weeknotes: Embeddings, more embeddings and Datasette Cloud - 17th September 2023
- Build an image search engine with llm-clip, chat with models with llm chat - 12th September 2023
- LLM now provides tools for working with embeddings - 4th September 2023
- Datasette 1.0a4 and 1.0a5, plus weeknotes - 30th August 2023
- Making Large Language Models work for you - 27th August 2023
- Datasette Cloud, Datasette 1.0a3, llm-mlc and more - 16th August 2023
- How I make annotated presentations - 6th August 2023