Simon Willison’s Weblog

628 items tagged “python”

hupper (via) Handy Python module for adding “live reload” development support to just about anything. I’m using it with Sanic—I run “hupper -m app” and it starts up my code in app.py and automatically reloads it any time any of the corresponding files changes on disk. # 23rd October 2017, 12:34 am

Parse shell one-liners with pyparsing. Neat introduction to the pyparsing library, both for parsing tokens into labeled sections and constructing an AST from them. # 22nd October 2017, 1:35 pm

Getting the Most out of Sqlite3 with Python. A couple of neat tricks I didn’t know: you can skip cursors entirely by calling .execute and .executemany directly on the connection object, and you can use the connection object as a context manager to execute transactions using a “with” block. # 22nd October 2017, 12:35 pm

Porting my blog to Python 3

This blog is now running on Python 3! Admittedly this is nearly nine years after the first release of Python 3.0, but it’s the first Python 3 project I’ve deployed myself so I’m pretty excited about it.

[... 883 words]

Deploying an asynchronous Python microservice with Sanic and Zeit Now

Back in 2008 Natalie Downe and I deployed what today we would call a microservice: json-head, a tiny Google App Engine app that allowed you to make an HTTP head request against a URL and get back the HTTP headers as JSON. One of our initial use-scase for this was Natalie’s addSizes.js, an unobtrusive jQuery script that could annotate links to PDFs and other large files with their corresponding file size pulled from the Content-Length header. Another potential use-case is detecting broken links, since the API can be used to spot 404 status codes (as in this example).

[... 1361 words]

Sanic. “Sanic is a Flask-like Python 3.5+ web server that’s written to go fast [...] On top of being Flask-like, Sanic supports async request handlers. This means you can use the new shiny async/await syntax from Python 3.5, making your code non-blocking and speedy”. # 7th October 2017, 6:39 pm

uvloop: Blazing fast Python networking. “uvloop makes asyncio fast. In fact, it is at least 2x faster than nodejs, gevent, as well as any other Python asynchronous framework. The performance of uvloop-based asyncio is close to that of Go programs.” # 7th October 2017, 5:53 pm

Building a statistical profiler in python. Generating flame graphs of production Python code is surprisingly straight-forward. Brian Pitts built a statistical flame graph profiler into our tikibar debugging tool at Eventbrite and it’s proved extremely useful. # 5th October 2017, 3:44 pm

Why is snapEDA.com slow? Is it because it uses Django?

No, it’s not slow because it uses Django.

[... 36 words]

What’s the best way to communicate with go libraries from within Python programs?

Go speaks HTTP extremely well, so one simple but powerful approach is to hook your Go libraries up as simple HTTP+JSON APIs and have Python call them over HTTP (the Requests: HTTP for Humans library is awesome for this).

[... 59 words]

What are some good resources to learn how to cleanse data using Python?

http://gnosis.cx/TPiP/ “Text Processing in Python” is a free online book that covers a bunch of useful topics related to data cleanup. It’s over 10 years old now but is still mostly relevant—the chapter on regular expressions is particularly good.

[... 61 words]

Is it better to create your own framework, or would it be best to just use Django or something like that?

You should absolutely use an existing framework such as Django rather than writing your own.

[... 176 words]

For a Django application, deployed on Heroku, what are my options for storing user-uploaded media files?

S3 is really a no-brainer for this, it’s extremely inexpensive, very easy to integrate with and unbelievably reliable. It’s so cheap that it will be practically free for testing purposes (expect to spend pennies a month on it).

[... 88 words]

Web app: what programming knowledge do I need to create a goings on app?

For this kind of application a much more important question than "how can I build it? is "where will I get the data from?"—If you don’t have a good answer for that building the app is a waste of your time. The world is littered with local events listings apps that no one uses because they don’t have good data.

[... 82 words]

How do I choose between asynchronous web frameworks? My tech group is fairly language agnostic and we’re trying to standardize on some technologies.

Since they are all pretty close to each other and it sounds like your tech group’s skills would support any of them, I would suggest having your tram build a simple prototype in all three so you can compare them for your own particular team and situation.

[... 76 words]

Which companies in London are using Python?

We use Python/Django for http://lanyrd.com/—we’re based in London.

[... 39 words]

What advice would Simon Willison give to a beginner Python/Django developer?

Build something and put it on the internet. Make sure you have an easy way to deploy new versions (Heroku is a good bet if you don’t want to figure out Fabric). Pick a project that’s useful to you—a simple blogging engine is often a good bet, or maybe something that aggregates together your posts from Twitter and Instagram and so on. Or come up with something a bit more creative!

[... 109 words]

Why does Python not have any data structures that store data in sorted order?

The bisect module provides functions for achieving this using a python list as the underlying data structure: http://docs.python.org/2/library...

[... 39 words]

What is the best way to deploy Django?

These days a popular and reliable method is to run gunicorn behind nginx. This tutorial includes notes on using upstart for process management which is handy if you are on Ubuntu: http://lincolnloop.com/django-be...

[... 47 words]

Do Flask and Django have a GUI interface like web2py?

No. The web2py GUI is something of an oddity in the Python world.

[... 65 words]

Which should I learn: Python or PHP?

Python will teach you more about programming than PHP—and you’ll be able to learn PHP easily if you learn Python first.

[... 36 words]

What are some hidden features of Python?

Generators and Iterators are pretty amazing. These two tutorials will really open your eyes as to how powerful they can be:

[... 63 words]

How do I learn Python fast?

Learn how to use the Python interactive prompt and start solving problems with it. The official Python tutorial is an excellent place to start: http://docs.python.org/2/tutorial/

[... 38 words]

What are some useful techniques and tricks using list comprehension?

List comprehensions are neat, but generators are even neater (and if you understand tricks with generators you’ll be able to use list comprehensions more effectively as well). Generator Tricks for Systems Programmers by David Beazley will blow you mind—it’s full of useful real-world examples as well. Read that, then check out his follow-up tutorial A Curious Course on Coroutines and Concurrency.

[... 81 words]

How can some really large services (like Dropbox) afford to use Python as a primary language, if it’s one to two orders of magnitude slower than other, compiled languages?

Because raw language speed often doesn’t matter that much. In the case if Dropbox the client software spends most of its time waiting for bits to load from the network or from disk. Most large websites spend their time waiting for the database. You can’t speed up network or disk performance by using a faster language.

[... 91 words]

Which social news websites for Python are out there?

There’s a Python subreddit: http://reddit.com/r/python

[... 21 words]

What are some apps, problems you would suggest to solve a new python developer?

The best way to learn python in my opinion is using the interactive prompt. Install ipython (a massive improvement on the standard python shell) and use it to interactively solve some simple tasks—things like downloading a CSV file from the web using the urllib library, parsing it with the csv module, then poking around in the data using python list comprehensions and saving some of the results out to a JSON file.

[... 95 words]

What is the scope, as a career, for a Python developer?

Don’t be an “X developer”. You’re selling yourself short if you define yourself by the technology you most frequently use.

[... 169 words]

Is it bad practice to have a variable that has the same name as a function?

Yes, it’s definitely not a good idea. In Python functions and variables share the same namespace, so if you create a variable with the same name as a function you won’t be able to call that function.

[... 93 words]

What are the best resources for learning regular expressions?

The O’Reilly book on Regular Expressions is absolutely superb. It will help you build a much deeper understanding if how they actually work than any online tutorial I’ve seen.

[... 62 words]