Hacker News new | past | comments | ask | show | jobs | submit login

The man page explains several of the differences (http://www.canonware.com/download/jemalloc/jemalloc-latest/d...). These include a different API (you can pass in some flags that explicitly direct it how to do alignment, initialization, etc), reduced use of sbrk in favor of mmap, multiple arenas for allocations of different sizes, per-cpu arenas to eliminate lock contention, etc.

Mozilla has been using jemalloc since the release of Firefox 3 in 2008, primarily because having multiple arenas for allocations of different sizes means that you end up with less fragmentation over time. Fragmentation happens when you deallocate ten 100-byte objects that are scattered around in memory, then need to allocate one thousand-byte object. You can't reuse those 10-byte empty spots in memory, because that 1000-byte allocation has to be contiguous. This means that your total memory usage has to grow. Jemalloc puts small allocations into buckets where they'll be closer together, making it more likely that new allocations will be able to fill the holes left by deallocations, and that freeing small objects will eventually empty a memory page, allowing you to return the page to the OS.

I remember that we were all pretty chuffed at the improvements in Firefox's memory usage; it was quite significant and would surely reduce the general perception that Firefox was a memory hog. Just using jemalloc reduced memory usage by 22% (http://blog.pavlov.net/2008/03/11/firefox-3-memory-usage/ has a really amazing graph comparing FF2 and FF3 to IE7).

Rust uses jemalloc for the same reasons; there's just no technical reason not to use it.




> Rust uses jemalloc for the same reasons; there's just no technical reason not to use it.

If so, I wonder if this is true for every language, then? For example, for C++, should gcc and clang emit binaries with jemalloc linked in?


no - its not the job of the compiler to choose your memory allocator. There is talk, however, of replacing the current glibc allocator with jemalloc[1], which is the more appropriate place for that decision to be made.

1 - https://sourceware.org/ml/libc-alpha/2014-10/msg00419.html


To me, it seems like the most sensible thing would be to make the choice of memory allocator a global OS configuration parameter—like enabling DEP on Windows. It'd still only be the program (or language runtime) deciding for itself whether to obey the allocator suggestion—but having one system-level place for this kind of policy information to be stored, where glibc et al could look for it, would make a lot more sense.


No thanks; that'd be worse. Then you'd have to test every release of your program with every allocator in combination with every other configuration change that could be made.

Besides, different applications work best with different allocators so it's definitely a choice the author of a program needs to make.


FreeBSD uses jemalloc by default in its libc. But it's not compiler's decision.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: