Memory Fragmentation – mvFragmentation
Many software engineers and their managers struggle to understand what memory fragmentation is and the effects memory fragmentation can have on the software they produce. The side effects will affect the performance of their software and result in a bad experience for the users of their software. That bad experience may be as subtle as running slowly, or more serious when the software fails to load an image file which everyone expects to load or catastrophic when a long-running service or web server fails a memory allocation and crashes.
Demonstration application
To help you understand memory fragmentation we’ve created a demonstration program, mvFragmentation.exe. Use this to generate different sequences of memory allocations, many of which will result in subtle or obvious memory fragmentation. Being able to reproduce memory fragmentation on demand in a reliable and predictable manner will allow you to repeatedly run the same test and inspect the memory allocation results in a log file and/or a virtual memory visualisation program. This activity will help you understand the nature of memory fragmentation and help you to produce higher-quality software that is more resilient against memory fragmentation.
mvFragmentation.exe is a test program used to demonstrate memory fragmentation.
mvFragmentation is intended to be used in conjunction with a virtual memory visualisation program such as:
- Memory Validator
- VM Validator
- VMMap
- Process Explorer
- Task Manager can also be used but does not provide any decent visualisation of virtual memory and is thus of limited use.
Many preset tests are provided which provide demonstrations of different memory allocation patterns. Some patterns create subtle memory fragmentation, some patterns create obvious memory fragmentation, and a few patterns do not create any noticeable memory fragmentation at all. If you would like to create a custom test you can modify the test parameters to create your own test.
Parameters that you can vary for each test are:
- Large block size.
- Small block size.
- Frag block size.
- The number of allocations in the test.
- Allocation lifetime for “frag blocks”.
- Allocator type. This is a choice of C runtime heap (malloc()), HeapAlloc() and VirtualAlloc().
Memory locations for each allocation can be logged to a file for study after the test is complete.
The test
The test is run in a loop that runs for a specific number of iterations. Each time through the loop three memory allocations are allocated and two memory allocations are deallocated. The third memory allocation is stored in an array for later deallocation. Each allocation and deallocation is allocated and deallocated using the allocator type specified. This allows you to understand how each allocator works. Every “lifetime” iterations the stored allocations are deallocated.
This simulates, very crudely, memory allocation and deallocation behaviour in many applications. Although this is a crude and simplistic simulation it does allow a simple and repeatable method to generate memory fragmentation. That the method is simple allows for much easier study and understanding of the consequences of each allocation pattern. The pattern is a combination of the sizes of each allocation, the lifetime of the “frag” block and the allocator type.
Here is an outline of the test
- For Num Allocations specified.
- Allocate a large block.
- Allocate a small block.
- Deallocate the large block.
- Allocate a frag block.
- Save the frag block for later deallocation.
- Deallocate the small block.
- Increase the lifetime count.
- If the lifetime count exceeds the frag block lifetime deallocate all frag blocks and reset the lifetime count.
- Deallocate any remaining frag blocks.
The loop as a whole does not leak memory. However, because frag blocks are stored for a specific number of iterations through the loop (or until the end of the loop) the frag blocks can end up blocking larger allocations from using the space surrounding them, resulting in memory fragmentation.
Any memory allocation that fails is an indication that a memory allocation could not be satisfied. When this happens in this test program it is an indicator that memory fragmentation may have occurred. The test program provides a warning dialog (disabled by default) which if enabled provides a warning when a memory allocation fails. When this dialog is displayed the test loop is halted until the dialog is dismissed. This is an excellent opportunity to view the virtual memory map of the test program via one of the software tools we have recommended.
Download the test application mvFragmentation.exe.