Simon Willison’s Weblog

Subscribe

6 items tagged “memory”

2019

Using memory-profiler to debug excessive memory usage in healthkit-to-sqlite. This morning I figured out how to use the memory-profiler module (and mprof command line tool) to debug memory usage of Python processes. I added the details, including screenshots, to this GitHub issue. It helped me knock down RAM usage for my healthkit-to-sqlite from 2.5GB to just 80MB by making smarter usage of the ElementTree pull parser.

# 24th July 2019, 8:25 am / elementtree, memory, profiling, python, xml

This paper introduces Mesh, a plug-in replacement for malloc that, for the first time, eliminates fragmentation in unmodified C/C++ applications. Mesh combines novel randomized algorithms with widely-supported virtual memory operations to provably reduce fragmentation, breaking the classical Robson bounds with high probability. Mesh generally matches the runtime performance of state-of-the-art memory allocators while reducing memory consumption; in particular, it reduces the memory of consumption of Firefox by 16% and Redis by 39%.

Mesh: Compacting Memory Management for C/C++ Applications

# 18th February 2019, 3:26 pm / firefox, memory, redis

2009

Using Graphics Card Memory as Swap (via) Interesting idea: “Graphic cards contain a lot of very fast RAM, typically between 64 and 512 MB. With Linux, it’s possible to use it as swap space, or even as RAM disk.”

# 3rd November 2009, 11:01 am / graphicscards, linux, memory, ops, performance, ram, sysadmin

Finding and fixing memory leaks in Python. Using Dozer, a clever piece of WSGI middleware which displays sparklines of Python object counts and allows you to introspect them, using the gc module under the hood.

# 22nd April 2009, 12:16 pm / amirsalihefendic, debugging, memory, memoryleaks, middleware, python, wsgi

Heap Dump Analysis. Using jmap to dump the JVM’s memory to disk, then analysing it using the visualvm GUI tool.

# 2nd April 2009, 10:34 am / dominicmitchell, heapdump, java, jvm, memory, profiling, visualvm

2008

Troubleshooting Memory Usage. Useful for getting the most out of a VPS.

# 23rd January 2008, 6:52 pm / hosting, memory, vps