Frequently Asked Questions?
We will attempt to answer any of your questions in this section of the site. If there are any other questions you would like to put to us please contact us and we will get back to you with a prompt answer.
Review questions that have been asked about C++ Memory Validator
Do I need to use the CRT for Memory Validator to detect memory leaks?
Which data items have the greatest performance impact?
How does Memory Validator detect uninitialised memory?
How does Memory Validator track COM objects?
Does Memory Validator track everything at the same time?
Memory Validator isn't collecting any data, why?
The program I'm trying to use with Memory Validator keeps
crashing, why?
I specified a callstack depth for my traces, why
are some of the traces shorter?
The function I want is not hooked, why not?
I have an idea for a feature, can it be added to Memory
Validator?
What file extensions does Memory Validator use
for itself?
Memory Validator isn't showing the data I expect to
see, why?
Some symbols are displayed as Ordinalxxx, why?
I've defined some ordinal to symbol conversions, why
aren't they working?
Does Memory Validator hook delay loaded functions?
My program is running slowly with Memory Validator,
why?
Why do some double delete callstacks show no allocation
location?
Why can I see datatypes that give no search results?
Memory Validator is a tool designed to detect memory
errors.
Why does it include a garbage collection function?
Why do does the memory view often show leaked memory which never gets collected
by the garbage collector?
When I do a leak detect, why are so many objects
marked as potentially leaked?
When I do show referencing pointers, or show referenced pointers, sometimes
I get no pointers shown, even though I know the object is not leaking. Why
is this?
When use show referencing pointers or show referenced pointers I sometimes
get Unknown values. Why is this?
Does Memory Validator require me to recompile and/or relink my program?
Why would I want to write an extension DLL for Memory Validator?
Q: Do I need to use the
CRT for Memory Validator to detect memory leaks?
A: No. Memory Validator inspects all Win32 heaps (HeapCreate
etc), the GlobalAlloc() heap, LocalAlloc() heap, CoTaskMemAlloc() heap, IMallocSpy,
as well as the C runtime heap(CRT). If you are using any of these heaps Memory
Validator is capable of monitoring the memory allocations and deallocations.
If you are using a custom heap manager of your own design (or a 3rd party
design) you can provide Memory Validator with information via the API so that
Memory Validator can track the behaviour of your custom heap manager.
Q: Which data items have the greatest performance impact?
A: This question in not straightforward to answer because
the answer depends in part, upon the target program that Memory Validator
is examining. A general answer will be given followed by some qualifications
to explain the answers. These answers are based upon the beta testing on
programs varying in size between 10,000, and 2,000,000 lines of C++.
We have not provided % figures as we believe such figures are misleading. The
performance change will be always in relation to the data generated by the target
program (for example a 2,000,000 line CAD program we have tested starts up in
much less time than a 300,000 line web authoring program we tested. The reason
for this was not coding quality, but the nature of the work each of the programs
was doing during startup. The web authoring program was preparing a lot of data
during startup which the CAD program did not need to do (*)).
Least impact listed first.
- Handle tracking
- COM object tracking
- Memory Allocation tracking (CRT, Win32 heap etc)
- Uninitialised data detection
- Buffer overrun detection.
In general the worst hit is with the buffer overrun detection. However if you
program uses the C (and Win32 shell) string functions very little there will
be no noticeable performance change. We have only found one program where the
buffer overrun detection caused the program to run very slowly, and this was
only during program startup when a very large amount of string processing was
being done. After the program startup the program performed as normal. The
buffer overrun detection is probably the most infrequently used feature on
Memory Validator.
The handle tracking functions produce very little performance overhead even on
very large programs.
The memory allocation tracking has a low overhead unless there are large numbers
of allocations in very tight loops.
The uninitialised data detection, by its very nature can have a high overhead,
but this is dependent upon the data it is examining.
(*) On a 2,000,000 line CAD program with all options enabled, Memory Validator
launches and runs the target program in 90 seconds. With the uninitialised data
and buffer tracking disabled Memory Validator launches and runs in 30 seconds.
One of Memory Validator's competitors takes over 40 minutes to do this task and
often fails towards the end of the 40 minute period.
Note that for the web authoring program mentioned above, startup times is typically
21 seconds. If uninitialised data detection is enabled, startup time increases
to 47 seconds. If buffer overflow detection is enabled, startup time increases
to 15 minutes. This is quite unusual and is due to the large amount of string
manipulation the program does at startup whilst it sets up various data structures
which later on in the program will improve its performance greatly.
If in doubt about the performance impact of Memory Validator on your product
we advise you to try Memory Validator on your product to determine the performance
impact. We think you will be favourably impressed compared to our competitors.
Q: How does Memory Validator detect uninitialised
memory?
A: Memory Validator detects uninitialised memory in memory
allocated by the debug C runtime heap. The debug C runtime heap initialises
all allocated memory with a signature byte of 0xCD. It also initialises any
uninitialised stack variables with a signature byte of 0xCC. Memory Validator
hooks all constructors of C++ objects and at the end of the constructor examines
the object for any uninitialised data bytes. If any are found they are reported
to the user.
0xCD is a valid value for a data byte but is unusual to be found as a WORD as
0xCDCD or a DWORD as 0xCDCDCDCD.
0xCC is a valid value for a data byte but is unusual to be found as a WORD as
0xCCCC or a DWORD as 0xCCCCCCCC.
It is up to the user to determine if the report of uninitialised data is correct
or not.
The technology for hooking the object's constructors is the same as is used to
hook COM objects.
Q: How does Memory Validator track COM objects?
A: Memory Validator hooks COM objects by hooking the Win32 functions
that return COM objects. These objects are then queried for their QueryInterface,
AddRef and Release function pointers. These functions are then hooked by rewriting
the instruction stream using some proprietary code. The hooks then examine the
return values to detect the reference count or returned object. Note that because
rewriting the instruction stream is not a generally recommended practice this
can sometimes fail (typically when the compiler optimises two functions to share
some common code). Memory Validator tries to detect when hooking will fail and
refuses to hook any functions that it knows it cannot hook safely. However some
COM objects cannot be hooked safely - we have found that you can debug some COM
objects using Memory Validator, and not others because of the internal structure
of the COM objects.
Memory Validator provides some controls for the COM hooking procedure. See
the section entitled Instruction Stream Rewriting Hooks on the tab of the
dialog. The instruction stream rewriting concept has been around for years (early
video games in the 1980s for the Commodore VIC 20 and Commodore 64 often rewrote
their instructions on the fly to gain a speed advantage) and is used, in varying
forms, by some of the competing products and complimentary products to Memory
Validator, and by Microsoft®.
Q: Does Memory Validator track everything
at the same time?
A: Memory Validator can be configured to track as much or as little
as you wish. If you are only interested in the handles returned from GetDC()
and passed into ReleaseDC() you can configure Memory Validator to do that. If
on the other hand you want to know about every handle allocated and deallocated,
every memory allocation and deallocation and all COM objects, all memory errors
and so forth, Memory Validator can do that too. When Memory Validator is tracking
more data items it has more work to do and Memory Validator will run slower
when tracking all data items than when tracking only a few.
Memory Validator was beta tested on a large CAD program and performed very well
even when tracking all data items.
Q: Memory Validator isn't collecting any data, why?
A: Memory
Validator provides many configuration options that control the collection
of data and the display of the data. It is possible to configure Memory Validator
to ignore the data you want collected.
For example: You may have run a session
with Memory Validator and turned off all the handle collecting functions from
the Data Hooks tab on the Data Collection dialog. If you do this, even with
the hooks inserted into your target program the hooks are disabled. No information
on handles will be communicated to Memory Validator.
Possible reasons for the data you require not being collected:
Data collection. Data collection is turned off. Check that the green data collect icon is not displayed on
the session toolbar (it will be disabled if data is being collected).
Data collection hook groups not installed. The appropriate hook group for the data
you wish to view may not have been installed in the target program when the
target program was attached to. Enable the hook group and re-run your session.
The hook groups installed are configured from the Collect tab on the Data Collection
dialog.
The individual hooks for the function you are interested in may have
been disabled. Check the hooks using the Data Hooks tab on the Data Collection
dialog. If the data hooks were installed, then the hooks will start collecting
data once you enable the data hooks.
Data is being collected but not displayed.
Check that the controls for the view you are using is set so that the appropriate
data will be displayed. Some of the views have the data display controls as
part of the view. The Memory tab has its control on the Display tab of the
Display Settings dialog.
Check that there are no filters (in Global, Session
and Local filter groups) that could be suppressing the display of the data.
Check that the thread filter is not suppressing the display of the data.
Q: The program I'm trying to use with Memory Validator
keeps crashing, why?
A: Assuming the crash is not a crash you
would normally get when not using Memory Validator, there are five possible
causes:
If you have the COM Reference Counting hooks installed, this may
be a cause of the problem as these hooks rewrite the instruction stream.
Sometimes rewriting the instruction stream doesn't work because some object
code produced by the optimising compiler, or hand written assembly code shares
common routines. Try turning off the COM Reference Counting hooks and trying
your test again.
If you have the Uninitialised Data hooks installed, this may
be a cause of the problem as these hooks rewrite the instruction stream. Sometimes
rewriting the instruction stream doesn't work because some object code produced
by the optimising compiler, or hand written assembly code shares common routines.
Try turning off the Uninitialised Data hooks and trying your test again.
Try removing all Target DLL extensions and Validator DLL extensions and trying
your test again.
Some DLLs from third party vendors which use system wide hooks
do not interact with Memory Validator and the target program very well. If
you can identify such DLLs, add then DLL name to the Don't Hook tab on the
Advanced Settings dialog and try again. Memory Validator will not hook these
DLLs.
There may be a bug in Memory Validator. Whilst we have tried to make
Memory Validator as robust as possible there are scenarios that we may not
have thought of and thus possible errors we have not catered for. Try to ensure
that the crash never happens if you are not using Memory Validator.
If the crash still happens and you have tried all the suggestions on this page, please
send details of the error to Software Verify and we will try to reproduce
the crash with a view to fixing any bugs found in as timely a manner as possible.
Please contact us at the address provided on the contact page.
Q: I specified a callstack depth for my traces, why
are some of the traces shorter?
A: If you have specified a callstack depth for all data
items collected the collected callstack may be shorter for a variety of reasons.
The actual callstack may have had less entries in it that the specified depth.
The callstack had more items in it that the specified depth, but some of the items
in the callstack were omitted to provide a stack trace that was useful. An
example will shed some light on this rather bizarre statement. Consider for
example tracking a call to operator new. For Debug builds operator new calls
to malloc_dbg(), thus the call stack will be something like (various functions
omitted for clarity):
- malloc_dbg()
- operator new()
- myFunctionThatCallsNew()
- functionBlues()
- functionJazz()
Memory Validator's callstack in this instance would be:
- myFunctionThatCallsNew()
- functionBlues()
- functionJazz()
so that examining the source code to myFunctionThatCallsNew() would show
the call to operator new(). This is done so that program calls to operator
new() are not confused with program calls to malloc() or malloc_dbg().
In the example above Memory Validator has removed 2 items from the callstack. Since
Memory Validator only collected your specified depth, you can see why the
number of items in the callstack may be slightly less than you expect. Memory
Validator attempts to correct for this by estimating the number of functions
that should not be included in the trace prior to collecting the trace. Most
of the time Memory Validator estimates correctly. On the occasions Memory
Validator gets its estimates incorrect the stack trace will be shorter than
expected.
If for a particular bug you are investigating the stack trace is a bit shorter
than you need, modify the data collection settings so that you collect more
of the callstack, or collect the entire callstack.
Q: The function I want is not hooked, why not?
A: We have provided hooks for all the functions that we think are required to be hooked in order
to provide as comprehensive a tool to aid you in your software development
tasks. As times pass Microsoft® releases new APIs with new functions in
the APIs. We intend to modify Memory Validator as new APIs are released.
However if we have missed an API which you think is important for a bug you are working
on, please let us know, and if we think the API should be added to the list
of APIs that Memory Validator hooks, we will do so. Please contact us at the
address provided on the . contact page.
Q: I have an idea for a feature, can it be added to
Memory Validator?
A: We have tried to add as many features to Memory Validator
that we thought would be useful to the potential users of Memory Validator.
In fact every feature in Memory Validator has been used to solve problems
and bugs on consulting work carried out for our customers and on internal
projects at Software Verify Limited. We know the features we have put
in the product are useful.
However it is possible we have overlooked a feature
that you may find very useful and which you cannot work out how to add to
Memory Validator via an extension DLL. We will be pleased to consider all
ideas for new features to Memory Validator. Please contact us at the address
provided on the contact page.
A Quake III mode, however much fun, is not useful in solving bugs. Sorry guys!
Q: What file extensions does Memory Validator use for itself?
A: Memory Validator stores most of the configuration data it needs in the registry.
However some data, such as the data hooks, coverage data and filter data is
stored in files. This section describes the file extensions used by Memory
Validator so that you can recognise such files. The file data structure is
not described as it may change from version to version of the product.
Session Export and Session Save
html. HTML export files
xml. XML export files
mvm. Memory Validator session files
Settings, Filters, Coverage
mvs. Settings
mvf. Filter files
mvc. Coverage files
mvd. Coverage files
Ordinal To Symbol Conversion
def. Linker definition files
ord. Ordinal data files
Program Launch, Extensions
dll. Extension DLLs
exe. Program files
Source Code
cpp. C++
c. C
h. C and C++
cxx. C++
hxx. C++
hpp. C++
Q: Memory Validator isn't showing the data I expect
to see, why?
A: A common reason for not seeing data that you believe you should see is that
Memory Validator has been configured to disable the hooks required to collect
the data you want. Check the following criteria:
- Check the correct hooks to monitor the memory, handle and/or COM allocations
are installed. See the Collect tab of the data collection settings dialog.
-
Check that the hooks are enabled. See the hooks tab of the data collection settings
dialog.
-
Check that the collected data is displayed. See the display tab of
the data collection settings dialog.
-
Check that global and session filters are not filtering the data, thus preventing its display.
-
Check that local filters (for the view being used) are not filtering the data, thus preventing its
display.
-
Check that the target program really is executing the code you think it is executing.
Q: Some symbols are displayed as Ordinalxxx, why?
A: When Memory
Validator cannot locate debug information for DLLs that have their functions
exported as ordinal values, the functions are named 'Ordinal' with the decimal
number of the function following. The ordinal values can be displayed as function
names if Memory Validator knows which linker definition (.def) file refers
to each DLL having ordinal exports. You can define ordinal handling behaviour
using the Ordinal Handling features of Memory Validator.
Also, don't forget to select the Map Ordinals to function names check box on the Source tab on display settings dialog
Q: I've defined some ordinal to symbol conversions,
why aren't they working?
A: If you have used the features of Memory Validator and the
ordinal names have not been converted in symbol names, there are two possible
reasons:
-You have not defined the ordinal to symbol conversion for the DLL in which
the ordinals are defined.
-You have forgotten to enable ordinal handling. Using
the Source tab on Display Settings dialog, select the Map Ordinals to function
names check box.
Q: Does Memory Validator hook delay loaded functions?
A: Yes, Memory Validator can hook delay loaded functions. Various controls are provided
so that you can control:
If delay loaded DLLs are hooked. See the Don't Hook tab of the Advanced Settings dialog.
If delay loaded functions are hooked. See the Collect tab of the Data Collection Settings dialog.
Q: My program is running slowly with Memory Validator,
why?
A: There are many reasons why your program may run slowly when used with Memory Validator.
Memory Validator provides you with many options to enable you to turn off data collection
for data that is not important. Turning off unwanted options prevents Memory
Validator from spending time examining data you don't want collected.
If you are not trying to isolate memory corruptions, turn off the buffer checking option.
If you are not trying to isolate uninitialised data, turn off uninitialised data
detection.
If you not trying to detect handle leaks, turn off all handle relatedhooks.
If you are not trying to detect leaks in GlobalAlloc, LocalAlloc and HeapAlloc,
turn off these options.
If you are not trying to detect CRT leaks, turn CRT leaks off.
If you do not need to collect complete callstacks, collect only the
part of the callstack that is interesting to you. Depending how deep your programs
callstacks get, changing this option can have quite a dramatic impact on your
program's performance.
If your program is still running slowly, it may be because
the program is allocating many blocks of memory in a tight loop. When this happens,
Memory Validator gets swamped with the sheer volume of data it needs to track,
and symbols (for the callstack) to resolve. When the program exits the tight
loop, the program performance will return to more normal speeds. Often this
is a sign that the program being worked on could be improved by redesigning
its memory allocation strategy. Examining the statistics on the objects view
will give you an insight into the number of allocations your program is making
and the allocation size frequency of those allocations.
Q: Why do some double delete callstacks show no allocation location?
A: Some double delete callstacks do not show the allocation location. There are four possible causes for this. Memory Validator was not collecting data when the memory was allocated. This could be because the memory was allocated before Memory Validator was attached to the process, or because data collection was turned off when the memory was allocated. Memory Validator has discarded the information about the memory allocation. This can happen if the Discard stack traces for freed memory check box is selected on the Misc tab of the Data Collection Settings dialog box. A possible cause is that the memory was allocated and then deleted in a statically linked CRT used by one DLL in the program, and then a pointer to the memory was also deleted by operator delete in the MSVCRT.DLL or MSVCRTD.DLL. Memory Validator does not track allocations made in the statically linked CRT, Memory Validator only tracks allocations made using the dynamically linked CRT (MSVCRT.DLL/MSVCRTD.DLL). A possible cause is that the double delete is being recognised for an invalid memory location. Consider the following pseudo code:
char *ptr;
ptr = new char [3000];
/ .. do some work
ptr = ptr + 10; / pointer no longer points at the start of the memory block
delete [] ptr; / incorrect
/ .. do some work
delete [] ptr; / incorrect double delete
In the above code fragment, not only is the delete call incorrect because of the incorrect pointer value, there is a second call to delete with the same pointer value. When Memory Validator looks for the associated allocation location, it will never find it because there are no allocations at the address "ptr + 10". This is why although the double delete can be detected, the allocation location cannot be detected.
Q: Why can I see datatypes that give no search results?
A: When using the Find Memory Dialog or the Find Object Type dialogs it is possible
to see datatypes to use for a search that give no search results. The reason
for this is that the datatypes have been loaded from the file type cache.
To remove these datatypes from the cache you must flush the datatype cache.
Q: Memory Validator is a tool designed to detect memory
errors. Why does it include a garbage collection function?
A: The garbage collection function is provided so that you can reset the leaked memory level to zero
whilst doing a debugging test. For example, you may do a test to determine
the memory leak exists, and then form a theory as to why that happened. Based
on this you may want to remove these leaks prior to retesting with you knowledge
of the leak. The garbage collector is not provided to replace the malloc/new
allocators. If you require a garbage collection solution there are several
vendors of garbage collectors to choose from. We do not offer opinions as
to which garbage collection solution to use. We also do not recommend garbage
collection as a solution as it always has poor performance when your application
consumes as much memory as the machine your application is running on. At
this point each pass of the garbage collector will touch all pages and cause
very heavy virtual memory paging effects with the consequence performance
hit.
Q: Why do does the memory view often show leaked memory
which never gets collected by the garbage collector?
A: The garbage collector never
collects internal MFC objects, which are temporary and are managed by MFC.
These do not normally show up as leaks. The Microsoft® C runtime has various
leaks in it, or apparent leaks in it. Some of this memory is referenced by
thread local storage slots and will be cleaned up when the particular thread
is destroyed. Any memory which is not marked as _IGNORE_BLOCK by MFC or _CRT_BLOCK
by the C runtime can and will be garbage collected. Memory on Win32 heaps
will not be garbage collected.
Q: When I do a leak detect, why are so many objects
marked as potentially leaked?
A: A potentially leaked memory block is a memory block that has
pointers to parts of the memory block, but not to the start of the memory
block. This can happen because the memory is deliberately managed this way
(CString for example uses offsets to help it manage various aspects of CString).
This can also happen because a stray pointer elsewhere in memory that is managed
by the program is pointing into the block. The memory is marked as potentially
leaked because it may be a genuine leak or it may be a deliberate strategy
of the program. It is up to you, with your knowledge of your program to determine
if it is a genuine leak or not.
Q: When I do show referencing pointers, or show referenced
pointers, sometimes I get no pointers shown, even though I know the object
is not leaking. Why is this?
A: This usually means that the object is being pointed to
by a pointer held in a static object, and object on the stack or an object
in a Win32 heap created using HeapCreate().
Q: When use show referencing pointers or show referenced
pointers I sometimes get Unknown values. Why is this?
A: Unknown pointers are usually pointers found in static program data or Win32 heap allocation data. Such
memory maybe a static C++ object or a C struct, or just an array holding pointers.
The 'Leak' field shows 'No' because the pointer has not been allocated.
Q: Does Memory Validator require me to recompile and/or relink my program? A: No Memory Validator does not require you to recompile and/or relink your program. Memory Validator does also allow you to link it into your program if you wish to access the extensible features of Memory Validator.
Q: Why would I want to write an extension DLL for Memory Validator?
A:
For the target application:
To use the Memory Validator instrumentation API to allow user defined heap actions to be tracked.
To use the Memory Validator instrumentation API to allow user defined reference counted
objects to be tracked.
To use the Memory Validator instrumentation API to allow
user defined hooks to be called before and after C++ constructors and C++ destructors.
For the user interface:
To provide custom text data to represent data collected
by the extension DLLs in the target application. This allows you to replace
arbitrary integer values collected by the stub (which may represent bit masks
or enumerations) with human readable strings.