Weeknotes: Trapped in an eternal refactor
I’m still working on refactoring Datasette’s table view. In doing so I spun out a new plugin, datasette-pretty-traces, which improves Datasette’s tooling for seeing the SQL that was executed to build a specific page.
I love tools like the Django Debug Toolbar which help show what’s going on under the hood of an application (see also the Tikibar, a run-in-production alternative we built at Eventbrite).
Datasette has long had a ?_trace=1 option for outputting debug information about SQL queries executed to build a page, but the output is a big block of JSON in the page footer, example here.
You can try it out here.
I’ll probably merge this into Datasette core at some point in the future, but a neat thing about having plugin support is I can dash out initial versions of things like this without needing to polish them up and include them in a formal release of the parent project.
Progress on the eternal refactor
Issue 1518, split from issue 878, is the all-consuming refactor.
Datasette’s table view is the most important page in the application: it’s the interface that lets you browse a table, filter it, search it, run faceting against it and export it out as other formats.
It’s the nastiest code in the entire project, having grown to over a thousand lines of Python. While it has very thorough tests, the actual code itself is unwieldy enough that it’s slowing down progress on all kinds of things I want to get done before I ship Datasette 1.0.
So I’m picking away at it. I’ve broken the underlying tests up into two modules (test_table_api.py and test_table_html.py) and I’ve made some small improvements, but I’ve also spun up some not-yet-committed prototypes both against my experimental asyncinject library and a new experiment that involves something that, if you squint at it, looks a tiny bit like a new ORM. I do not want to build a new ORM!
I’m not happy with any of this yet, and it’s definitely blocking my progress on other things. I’ll just have to keep on chipping away and see if I can get to a breakthrough.
Releases this week
datasette-pretty-traces: 0.2.1—(3 releases total)—2021-12-13
Prettier formatting for ?_trace=1 traces
TIL this week
More recent articles
- AI-enhanced development makes me more ambitious with my projects - 27th March 2023
- I built a ChatGPT plugin to answer questions about data hosted in Datasette - 24th March 2023
- Weeknotes: AI won't slow down, a new newsletter and a huge Datasette refactor - 22nd March 2023
- Don't trust AI to talk accurately about itself: Bard wasn't trained on Gmail - 22nd March 2023
- A conversation about prompt engineering with CBC Day 6 - 18th March 2023
- Could you train a ChatGPT-beating model for $85,000 and run it in a browser? - 17th March 2023