Please enable JavaScript to view this site.

Memory Validator Help

The Custom Hooks tab allows you to specify hooks for functions that Memory Validator does not initially know about.


For example, custom hooks might let you monitor APIs in third party products or in APIs that are released after Memory Validator was released.


noteThis a very advanced topic.


interactiveRead on, or click on a setting in the picture below to find out more





Using custom hooks


Before adding custom hooks, you need to know the parameter list and return type of each function to be hooked and their calling conventionsexternalLink.


Add... instructionStep displays the custom hook dialog (below)


For other options (edit,remove, and enable), see modifying existing custom hooks further below.


seeAlsoThis topic is also discussed in the Memory Validator tutorial, which can be found on the Help menu.



Custom hook dialog


The custom hook dialog allows you to set up or edit the definition of a custom hook, including its parameters.


warningnote Take care! Failure to specify the information correctly may crash your application. If in doubt do not attempt to use this feature of Memory Validator.


At the end of this topic are some examples of how this dialog might be filled out for a few different functions.






Custom hook definition


DLL Name instructionStep Browse to navigate to a DLL instructionStep Open to enter the DLL name into the dialog  


Alternatively you can type the full path, or a relative path to the DLL into the DLL Name field.

Function Name instructionStep choose the name of the exported function from the drop-down list
       The list is automatically populated with the exported functions after choosing a DLL name above


The function name must be exported from the DLL you specified in order to appear in the list. If the function is not exported it cannot be hooked.


Function Ordinal instructionStep type the ordinal (decimal integer) if the function is imported by ordinal, otherwise enter -1 to ignore it


For example MAPIAllocateBuffer@8 in MAPI32.DLL has the ordinal 13 as found in the Export Address Table of the DLL


hmtoggle_plus0 Finding the ordinal of an exported function in a DLL One way to find the ordinal of a function exported from a DLL is to use the dumpbin.exe utility provided with Microsoft Visual Studio. e.g. dumpbin filename.dll /EXPORTS

In Visual Studio 9.0, the dumpbin.exe utility is found in VC\bin and is dependent on link.exe in the same directory and mspdb80.dll in the  Common7\IDE directory.


Calling convention instructionStep select the option that corresponds to the calling conventionexternalLink of the function




Do not guess this, if you don't know! Using the wrong calling convention will crash your application


seeAlsoSee also, the history of calling conventionsexternalLink


Number of Parameters instructionStep enter the number of parameters the function takes

This is the number of parameters the function has, not the number of parameters that you want to monitor (below).


Again - do not guess this! Using the wrong number of function parameters will crash your application when used with the stdcall calling convention.


Function Purpose instructionStep set the function purpose - Alloc, AllocMore, Realloc, or Free


Enabled instructionStep tick to enable hooking of the function


Only takes effect each time your application starts, so changing this in the middle of a session has no effect.


Datatype instructionStep enter the type of all allocations via this hook, or leave empty and Memory Validator will parse the source code to try and obtain the datatype


Do not include parentheses in the datatype. [] and <> are acceptable though.



Custom hook function parameters


The parameters section of the dialog allows you to define which parameters and return value are monitored for the function.


noteYou should specify all the other settings in the custom hook definition (see section above) prior to defining the parameters. This will ensure that the right menu choices are made available while setting up the parameters.


You don't need to monitor all the input parameters, or the return value  - just monitor what you need.


Generally for each function purpose (that you will have defined above), different parameter values are usually required:


For example:


Alloc        an input size, and an output pointer or return value pointer

AllocMore  an input size and an input pointer

Realloc        an input size, an input pointer, and an output pointer or return value pointer

Free        an input pointer


In some cases, an input parameter may also serve as an output.


If you want to monitor additional parameters, we recommend that you set their type to Miscellaneous or Pointer to Miscellaneous as appropriate.


Add instructionStep adds a row to the parameters table, firstly a return value, then parameters in order 0,1,2...


Remove instructionStep removes selected parameter definitions from the list


Remove All instructionStep clears the list of parameters


Defining custom hook function parameters


After adding a parameter definition, values can be edited by double clicking on the value in the table. A drop-down list will be displayed with appropriate choices in each column:



Parameter instructionStep choose the parameter index in the range 0 to N-1 where N is the number of parameters you specified to the function


You can only choose an index which is not already used by another parameter


You can't change the entry for a "return value"


Usage instructionStep set how the parameter is used:




In is input only and Out is output only, while In/Out serves as both

Return value, this and any output parameters are usually pointers to something.


Type instructionStep choose what the parameter type is:




All Pointer to... options are parameters that need dereferencing to read the values.


Memory Validator x64 also supports the following values:


Size (QWORD)                                a size

Pointer to size (QWORD)                        a pointer to a size

Miscellaneous (QWORD)                        a miscellaneous value

Pointer to Miscellaneous (QWORD)        a pointer to a miscellaneous value


noteNote that when specifying parameters as Pointer to ..., it's OK if that parameter is occasionally NULL - this will be identified and the pointer will not be dereferenced. Many Microsoft APIs allow NULL values for optional data, for example.



Modifying existing custom hooks


Once you have some custom hooks set up you can edit, remove or en/disable them in the following ways:


Edit... or double click a hook in the list instructionStep opens the custom hook dialog to change the hook attributes


The usual controls apply for removing list items:


Remove instructionStep removes selected items from the list


Remove All instructionStep clears the list

Alternatively, press b_del to delete selected items, and b_ctrl + b_a to select all items in the list first.


Enable or disable a custom hook by ticking or unticking the yellow boxes or change them all at once


Enable All instructionStep ticks all the hooks, enabling them all

Disable All instructionStep unticks everything

Import... instructionStep Imports a custom hook XML file.

Export... instructionStep Exports the custom hooks as an XML file. This can be useful for moving a custom hook definitions from one computer to another without copying the whole settings file.



Resetting custom hooks


Normally, the custom hooks will not be reset when you press the reset button, but you can make that the case if you wish:


Allow the Reset button to reset the custom hook definitions instructionStep when ticked, custom hooks will be reset with all other global settings



Reset All - Resets most global settings including those on other pages, but not the settings on this page unless explicitly requested above.



Reset - Resets the settings on the current page.



These examples show how to use the custom hooks dialog


Custom hook dialog - example 1


A custom hook dialog for a function using the __cdecl calling convention is shown below.


The function prototype for customAlloc1 in the DLL testCustomDLL.dll (provided) is:


   extern "C" void *customAlloc1(DWORD   size);      // input param


It has been specified using the extern "C" specifier so that the function name has no C++ name mangling decoration.


It uses the __cdecl calling convention, takes one input parameter and is marked as an allocator by the Alloc definition.


For datatype purposes it can be specified as returning the BYTE datatype.


The custom hook will monitor:


the return parameter, marked as a pointer

the single input parameter, marked as a size specifier.





Custom hook dialog - example 2


A custom hook dialog for a function using the __stdcall calling convention is shown below.


The function prototype for ?customStdCallAlloc3@@YGXKPAPAX@Z is in the DLL testCustomDLL.dll is:


   extern void  __stdcall customStdCallAlloc3(DWORD        size,         // input param
                                                   void        **ptr);       // output param


It has been compiled as C++, having C++ name mangling decoration.


It uses the __stdcall calling convention, and takes two input parameters.


The function purpose as an allocator is defined by the Alloc setting.


For datatype purposes it can be specified as returning the BYTE <STDCALL> datatype.


The custom hook will monitor:


the first input parameter, marked as a size specifier

the second output parameter, marked as a pointer to pointer.






Custom hook dialog - example 3


Similar to example 2, except we have a different function name and its purpose is now as a Realloc function, with a pointer return type.


The function prototype for customStdCallReAlloc1 is:


   extern void*  __stdcall customStdCallReAlloc1(DWORD   size,         // input param
                                                       void    **ptr);       // output param


The custom hook will monitor:


the return parameter as a pointer

the first input parameter, marked as a size specifier

the second input parameter, marked as a pointer to pointer.