Items tagged python, testing
Filters: python × testing × Sorted by date
inline-snapshot. I'm a big fan of snapshot testing, where expected values are captured the first time a test suite runs and then asserted against in future runs. It's a very productive way to build a robust test suite.
inline-snapshot by Frank Hoffmann is a particularly neat implementation of the pattern. It defines a snapshot() function which you can use in your tests:
assert 1548 * 18489 == snapshot()
When you run that test using pytest --inline-snapshot=create the snapshot() function will be replaced in your code (using AST manipulation) with itself wrapping the repr() of the expected result:
assert 1548 * 18489 == snapshot(28620972)
If you modify the code and need to update the tests you can run pytest --inline-snapshot=fix to regenerate the recorded snapshot values. # 16th April 2024, 4:04 pm
time-machine example test for a segfault in Python (via) Here’s a really neat testing trick by Adam Johnson. Someone reported a segfault bug in his time-machine library. How you you write a unit test that exercises a segfault without crashing the entire test suite?
Adam’s solution is a test that does this:
subprocess.run([sys.executable, “-c”, code_that_crashes_python], check=True)
sys.executable is the path to the current Python executable—ensuring the code will run in the same virtual environment as the test suite itself. The -c option can be used to have it run a (multi-line) string of Python code, and check=True causes the subprocess.run() function to raise an error if the subprocess fails to execute cleanly and returns an error code.
I’m absolutely going to be borrowing this pattern next time I need to add tests to cover a crashing bug in one of my projects. # 23rd March 2024, 7:44 pm
pytest-icdiff (via) This is neat: “pip install pytest-icdiff” provides an instant usability upgrade to the output of failed tests in pytest, especially if the assertions involve comparing larger strings or nested JSON objects. # 3rd June 2023, 4:59 pm
pyfakefs usage (via) New to me pytest fixture library that provides a really easy way to mock Python’s filesystem functions—open(), os.path.listdir() and so on—so a test can run against a fake set of files. This looks incredibly useful. # 1st February 2023, 10:37 pm
When you have to mock a collaborator, avoid using the Mock object directly. Either use mock.create_autospec() or mock.patch(autospec=True) if at all possible. Autospeccing from the real collaborator means that if the collaborator’s interface changes, your tests will fail. Manually speccing or not speccing at all means that changes in the collaborator’s interface will not break your tests that use the collaborator: you could have 100% test coverage and your library would fall over when used!
— Thea Flowers # 17th March 2021, 4:44 pm
How to cheat at unit tests with pytest and Black
I’ve been making a lot of progress on Datasette Cloud this week. As an application that provides private hosted Datasette instances (initially targeted at data journalists and newsrooms) the majority of the code I’ve written deals with permissions: allowing people to form teams, invite team members, promote and demote team administrators and suchlike.
[... 933 words]parameterized. I love the @parametrize decorator in pytest, which lets you run the same test multiple times against multiple parameters. The only catch is that the decorator in pytest doesn’t work for old-style unittest TestCase tests, which means you can’t easily add it to test suites that were built using the older model. I just found out about parameterized which works with unittest tests whether or not you are running them using the pytest test runner. # 19th February 2019, 9:05 pm
Hynek Schlawack: Testing & Packaging (via) “How to ensure that your tests run code that you think they are running, and how to measure your coverage over multiple tox runs (in parallel!)”—Hynek makes a convincing argument for putting your packaged Python code in a src/ directory for ease of testing and coverage. # 22nd May 2018, 10:12 pm
Unit Testing Achievements. A plugin for Python’s nose test runner that adds achievements—“Night Shift: Make a failing suite pass between 12am and 5am.” # 28th February 2010, 3:56 pm
rlisagor’s freshen. A Python clone of Ruby’s innovative Cucumber testing framework. Tests are defined as a set of plain-text scenarios, which are then executed by being matched against test functions decorated with regular expressions. Has anyone used this or Cucumber? I’m intrigued but unconvinced—are the plain text scenarios really a useful way of defining tests? # 5th January 2010, 7:30 pm
Fabric factory. Promising looking continuous integration server written in Django, which uses Fabric scripts to define actions. # 21st September 2009, 6:35 pm
Testing Django Views for Concurrency Issues. Neat decorator for executing a Django view under high concurrency in your unit tests, to help spot errors caused by database race conditions that should be executed inside a transaction. # 27th May 2009, 10:01 am
Nose 0.11 released. My favourite Python testing tool just got some really neat new features, including the ability to parallelize tests across multiple processes (hence CPUs) using the multiprocess module, Xunit XML output for integration with continuous integration tools and a --failed switch to re-run only the last batch of failed tests. # 8th May 2009, 11:24 am
Changeset 9793: SMTP testing documentation. I didn’t know this trick: running “python -m smtpd -n -c DebuggingServer localhost:1025” will start up a simple SMTP server which dumps received e-mails to the terminal instead of forwarding them on. # 29th January 2009, 1:35 pm
Django now has fast tests. Changeset 9756 switched Django’s TestCase class to running tests inside a transaction and rolling back at the end (instead of doing a full dump and reload). “Ellington’s test suite, which was taking around 1.5-2 hours to run on Postgres, has been reduced to 10 minutes.” # 16th January 2009, 11:40 am
Localbuilder. Gareth Rushgrove’s neat little Python continuous integration tool—it watches a directory for changes, then runs a command when it spots any. # 14th January 2009, 10:57 pm
Django snippets: RequestFactory. I’ve been wanting this for ages; when I finally got around to writing it it turned out to only be a dozen or so lines of code. Makes it easy to create mock request objects in Django, which you can then use for testing view functions directly (bypassing the current test client mechanism which requires views to be assigned to a URLconf before they can be tested). # 12th August 2008, 12:40 pm
Python one-liner of the day. I love the idea of publishing one-liners accompanied by one-line test suites. # 26th April 2008, 10:24 am
xPyUnit: Uniting in Python with XML reporting. Should be just the ticket for integrating Django’s testing framework with Cruise Control. # 27th March 2008, 12:35 pm
Mock—Mocking and Test Utilities (via) New mocking library for Python based on the “action ... assertion” pattern (as opposed to the more common “record ... replay”). # 20th November 2007, 11:30 pm
nose 0.10.0 final! Nose is my favourite Python testing tool: it can auto-discover and execute tests in a directory hierarchy, which makes it easy to run just a sub-set of your test suite. # 11th October 2007, 1:52 pm
Test stubbing httplib2. Nice demonstration of monkey-patching as part of unit testing in Python. # 10th May 2007, 11:24 pm
The joy of pdb.set_trace(). I use nosetests --pdb-failures as my main entrypoint for Python debugging—it starts the debugger at the first failing test. # 25th April 2007, 11:37 pm
Dependency injection is the enterprisey name for trampling over namespaces with reckless abandon.
— Ted Dziuba # 16th December 2006, 1:05 am