This article applies to any company where your main product, or a substantial part of your main product is software and the software is a key part of your product. For example if you made top end oscilloscopes with an embedded PC inside them, the software to run the oscilloscope on top of the embedded OS would be important. This is unlike say if you are AOL, where the dialup software is important but can be easily outsourced without impacting your business at all.

We receive approaches via email, via Linked-In, etc, from outsourcing firms on a regular basis. Their message extoll the virtues of their multi-talented legion of software developers that can be hired from as little as $5.00 / hour. On the face of it getting staff at $5.00 sounds great, if you assume the staff know how to do their job. The list of technologies, computer languages and computing platforms they support is vast. You almost wonder how they do it. And at $5.00/hour. Whats the catch?

Cost

$5/hour. 10$/hour. Either of these rates is really attractive. No surprise, that is pretty much their unique selling point.

In pure financial terms this looks like a win. But the true cost of using outsourced software development is not the expense of paying for it. The cost is elsewhere. I’ll come back to this later.

Location and Language

Typically these operations are setup in low cost parts of the world, usually developing countries, ex-eastern bloc countries, India, China. Often with a nice shiny headquarters in the US or the UK.

Communication

Everyone (well, almost) has access to the Internet these days. Distributed teams are not new. Lots of people do it. We do it. The most famous web product team on the planet does it. It is quite possible your outsourcing company will work this way. If that is important to you, you should find out.

You will need a core spoken/written language for communications. That will probably be English, just because that is the way the world works and software tends to be written that way.

You’ll want to make sure any source code comments, version control commits and design specs/documents/presentations are also in English. I’ve seen comments in French in source code bases that were substantially English. No problem with French except that practically no one in the building knew what the comments said, except for the mathematician that wrote them. Is that useful? I don’t think so. Neither did the management when they found out

You probably have to assume that most of the people that will end up developing software for you will have English as a second or third language. This generally isn’t a problem, but I’ve found a few exceptions. Different languages have different rules on sentence construction – where the verb goes, how adjectives and nouns are used and so on. I’ve found over the years, from corresponding with our customer base, the I struggle with the sentence verb/noun/adjective formation rules used by some parts of the world which are not the same as English rules. When translated from their native language to English you get a strange beast which looks like English but which reads very differently. Everyone is different, you may not have that problem, but I struggle with it.

Staff

Is it reasonable to assume these people have good staff? I think it is. I think its also reasonable to assume they probably have some useless staff too, most companies do, so I don’t see why these companies would be any different.

Sounds OK so far, so what is the problem?

The main problem with outsourced software development is that as the development proceeds the following happens:

• You are paying the outsourcing firm to learn the technologies required to build your product.
• You are building up a reservoir of knowledge in the brains of their employees, not your employees.
• At some point the outsourcing company will know more about the internals of your product that you do. At that point you become dependent upon the outsourcing company. A perfect time for them to raise their costs. I don’t know if this happens, but logic dictates that it should.
• Once your product is complete, you have a lot of software that works and has no bugs (well, you can live in hope), but you don’t know anything about it. Is that a good position to be in? Sure you’ve got documentation on it and the full version control history. But fundamentally the knowledge is in the brains of people that don’t work for your company. How are you going to handle bug reports? Are you going to let your staff lose on a codebase they don’t know or are you going to hire the outsourcing company again? At what point can you break free from the outsourcing company?
• The staff at the outsourcing company do know how your product works. If they want to go and write their own product in the same marketspace they can, and they will do it better as they will have all the knowledge about the pitfalls and mistakes made creating your product. Of course you can mitigate against this will a clause in your contract forbidding such competition.
• Fundamentally, the software development business is about developing intellectual property. You want as much of that IP in the heads of the people that work for you, not in the heads of people that do not work for you. Outsourcing this work puts this knowledge outside of your company and is ultimately commercial suicide. Of course, staff can leave your company and move on to forge their careers. But that happens over time, you can manage that. Not the same as all the knowledge in the brains of the a separate company’s employees.

All the above reasons put you at a substantial disadvantage compared to your competitors that do not outsource their software development.

Exceptions

I can think of one exception to the above, but it is not really outsourcing. The exception is when you have one small discrete component of your software that you would like implemented and all you want is a functioning piece of code that can live in a DLL and get called to do its job. Examples would be a GUI widget library, a data compression library, a disassembler for a specific microprocessor etc. Typically you can find companies or lone developers that do things like this. They often have working examples you can try with very reasonable terms for use.

This is outsourcing in that you didn’t do the work, but it falls into the “not core activity” classification I mentioned at the start of the article, so I don’t think this really counts.

Conclusion

I always reply to people that approach us asking us to outsource our work to them. I explain that I regard their offer as asking us to commit commercial suicide. Normally I don’t get a reply. But I did get a reply from one gentleman, who shall remain nameless. Here is the edited exchange. Edited just to get to the nitty gritty. I have not edited any words or punctuation in the sentences.

Software Verification:
"We are not interested in outsourcing. Software development is a core activity here – outsourcing it is equivalent to outsourcing the core intellectual property. For our business that is suicide. For other businesses it makes sense."

Outsourcing reply:
"Honestly, I totally agree with you. But such is the reality nowadays.
…"

He then explained that he thought there were valid reasons for us to use his services. Good salesman, I guess.

make -f client.mk clean

If you do that, then a subsequent build will fail when it comes to updater.exe.manifest.

Setup

Assuming you have downloaded and installed Mozilla build and have also installed a suitable version of Visual Studio and the latest Microsoft Web SDK, you can start the build environment by running c:\mozilla-build\start_msvc9.bat (change the number for the version of Visual Studio you are using).

Read these instructions first.

Download the appropriate source tree with a mercurial command like this:

hg clone http://hg.mozilla.org/releases/mozilla-1.9.2/ src192

Basic Build

I’m building on drive M: and have downloaded the source into folder src192.

cd M:/src192

Configure the build with:

echo '. $topsrcdir/browser/config/mozconfig' > mozconfig

Build the build with

make -f client.mk build

This gives you a firefox release build. Strangely the build is created without debug symbols. I don’t understand this as debug symbols are really useful even with release builds. The symbols don’t have to ship with the executable code, so there is no good reason for not creating them.

Given that you’ve downloaded the source to build yourself the image its fair to assume that you are probably curious about the internal works or that you may be interested in modifying the source and/or writing an extension for it. Either way, you are going to want the symbols so that you can see what is happening in a debugger. Thus I don’t understand the lack of symbols by default.

Creating symbols

You can create symbols in several ways, by adding the following lines to your mozconfig.

echo 'ac_add_options --enable-debug' >> mozconfig

echo 'export MOZ_DEBUG_SYMBOLS=1' >> mozconfig
echo 'ac_add_options --enable-debugger-info-modules=yes' >> mozconfig

Getting a working build

At this point you’ve already done a build and have created executables. Thus it seems the appropriate thing to do is a make -f client.mk clean followed by a make -f client.mk build. This won’t work. The problem is that the clean deletes the manifest files that are present in the original mercurial source download.

In the end I wiped the whole source tree, downloaded from the source control again, modified mozconfig with the debug symbol options and did a build. That works.

This took me several attempts to understand – I thought that my various attempts at configuring the debug symbol options were breaking something, so I would change the options, clean then build. Each time taking several hours. I had this working in a virtual machine while I worked on other tasks. I probably spent about a day, including the build times before I decided to try again by starting from scratch.

Solution

I don’t know if this is true for non-Windows firefox builds, but if you want your builds to succeed don’t do a make -f client.mk clean prior to make -f client.mk build as it will prevent you from successfully building firefox.

I hope this information helps prevent you from wasting your time with this particular build problem.

Last week was a rollercoaster of a week for software development at Software Verification.

Off by one, again?

First off we found a nasty off-by-one bug in our nifty memory mapped performance tools, specifically the Performance Validator. The off-by-one didn’t cause any crashes or errors or bad data or anything like that. But it did cause us to eat memory like nobodies business. But for various reasons it hadn’t been found as it didn’t trigger any of our tests.

Then along comes a customer with his huge monolithic executable which won’t profile properly. He had already thrown us a curve balled by supplying it as a mixed mode app – half native C++, half C#. That in itself causes problems with profiling – the native profiler has to identify and ignore any functions that are managed (.Net). He was pleased with that turnaround but then surprised we couldn’t handle his app, as we had handled previous (smaller) versions of his app. The main reason he was using our profiler is that he had tried others and they couldn’t handle his app – and now neither could we! Unacceptable – well that was my first thought – I was half resigned to the fact that maybe there wasn’t a bug and this was just a goliath of an app that couldn’t be profiled.

I spent a day adding logging to every place, no matter how insignificant, in our function tree mapping code. This code uses shared memory mapped space exclusively, so you can’t refer to other nodes by addresses as the address in one process won’t be valid in the other processes reading the data. We had previously reorganised this code to give us a significant improvement in handling large data volumes and thus were surprised at the failure presented to us. Then came a long series of tests, each which was very slow (the logging writes to files and its a large executable to process). The logging data was huge. Some of the log files were GBs in size. Its amazing what notepad can open if you give it a chance!

Finally about 10 hours in I found the first failure. Shortly after that I found the root cause. We were using one of our memory mapped APIs for double duty. And as such the second use was incorrect – it was multiplying our correctly specified size by a prefixed size offset by one. This behaviour is correct for a different usage. Main cause of the problem – in my opinion, incorrectly named methods. A quick edit later and we have two more sensibly named methods and a much improved memory performance. A few tests later and a lot of logging disabled and we are back to sensible performance with this huge customer application (and a happy customer).

So chalk up one “how the hell did that happen?” followed by feelings of elation and pleasure as we fixed it so quickly.
I’m always amazed by off-by-one bugs. It doesn’t seem to matter how experienced you are – it does seem that they do reappear from time to time. Maybe that is one of the persils of logic for you, or tiredness.

I guess there is a Ph.D. for someone in studying CVS commits, file modification timestamps and off-by-one bugs and trying to map them to time-of-day/tiredness attributes.

That did eat my Wednesday and Thursday evenings, but it was worth it.

Not to be outdone…

I had always thought .Net Coverage Validator was a bit slow. It was good in GUI interaction tests (which is part of what .Net Coverage Validator is about – realtime code coverage feedback to aid testing) but not good on long running loops (a qsort() for example). I wanted to fix that. So following on from the success with the C++ profiling I went exploring an idea that had been rattling around in my head for some time. The Expert .Net 2.0 IL Assembler book (Serge Lidin, Microsoft Press) was an invaluable aid in this.

What were we doing that was so slow?

The previous (pre V3.00) .Net Coverage Validator implementation calls a method for each line that is visited in a .Net assembly. That method is in a unique DLL and has a unique ID. We were tracing application execution and when we found our specific method we’d walk up the callstack one item and that would be the location of a coverage line visit. This technique works, but it has a high overhead:

1. ICorProfiler / ICorProfiler2 callback overhead.
2. Callstack walking overhead.

The result is that for GUI operations, code coverage is fast enough that you don’t notice any problems. But for long running functions, or loops code coverage is very slow.

This needed replacing.

What are we doing now that is so fast?

The new implementation doesn’t trace methods or call a method of our choosing. For each line we modify a counter. The location of the counter and modification of it are placed directly into the ilAsm code for each C#./VB.Net method. Our first implementation of .Net Coverage Validator could not do this because our shared memory mapped coverage data architecture did not allow it – the shared memory may have moved during the execution run and thus the embedded counter location would be invalidated. The new architecture allows the pointer to the counter to be fixed.

The implementation and testing for this only took a few hours. Amazing. I thought it was going to fraught with trouble, not having done much serious ilAsm for a year or so.

Result?

The new architecture is so lightweight that you barely notice the performance overhead. Less than 1%. Your code runs just about at full speed even with code coverage in place.

As you can imagine, getting that implemented, working and tested in less than a day is an incredible feeling. Especially compared to the previous performance level we had.

So why feel stupid?

Having acheived such good performance (and naturally feeling quite good about yourself for a while afterwards) its hard not to look back on the previous implementation and think “Why did we accept that?, We could have done so much better”. And that is where the feeling stupid comes in. You’ve got to be self critical to improve. Pat yourself on the back for the good times and reflect on the past to try to recognise where you could have done better so that you don’t make the same mistake in the future.

And now for our next trick…

The inspiration for our first .Net Coverage Validator implementation came from our Java Coverage Validator tool. Java opcodes don’t allow you to modify memory directly like .Net ilAsm does, so we had to use the method calling technique for Java. However given our success with .Net we’ve gone back to the JVMTI header files (which didn’t exist when we first wrote the Java tools) and have found there may be a way to improve things. We’ll be looking at that soon.

A customer problem

The inspiration for this article was a customer reporting that Memory Validator was not reporting any allocations in a particular DLL of his mixed mode .Net/native application. The application was interesting in that it was a combination of C#, C++ written with one version of Visual Studio and some other DLLs also written in C++ with another version of Visual Studio. Only the memory for one of the DLLs was not being reported by Memory Validator and the customer wanted to know why and could we please fix the problem?

After some investigation we found the problem was a not with Memory Validator but with the DLL in question making a call to _CrtSetDbgFlag(0); which turned off all memory tracking for that DLL. Memory Validator honours the memory tracking flags built into Visual Studio and thus did not report these memory allocations. Armed with this information the customer did some digging into their code base and found that someone had deliberately added this call into their code. Removing the call fixed the problem.

The rest of this article explains how Microsoft tags data to be ignored and what flags are used to control this process.

Why does Microsoft mark these allocation as ignore?

The reason for this is that these allocations are for internal housekeeping and sometimes also for one-off allocations that will exist until the end of the application lifetime. Such allocations could show up as memory leaks at the end of the application – that would be misleading as they were intended to persist. Better to mark them as “ignore” and not report them during a memory leak report.

Microsoft debug CRT header block

Microsoft’s debug CRT prefixes each allocation with a header block. That header block looks like this:

#define nNoMansLandSize 4

typedef struct _CrtMemBlockHeader
{
    struct _CrtMemBlockHeader * pBlockHeaderNext;
    struct _CrtMemBlockHeader * pBlockHeaderPrev;
    char *                      szFileName;
    int                         nLine;
#ifdef _WIN64
    /* These items are reversed on Win64 to eliminate gaps in the struct
     * and ensure that sizeof(struct)%16 == 0, so 16-byte alignment is
     * maintained in the debug heap.
     */
    int                         nBlockUse;
    size_t                      nDataSize;
#else  /* _WIN64 */
    size_t                      nDataSize;
    int                         nBlockUse;
#endif  /* _WIN64 */
    long                        lRequest;
    unsigned char               gap[nNoMansLandSize];
    /* followed by:
     *  unsigned char           data[nDataSize];
     *  unsigned char           anotherGap[nNoMansLandSize];
     */
} _CrtMemBlockHeader;

How does Microsoft tag an allocation as ignore?

When the CRT wishes an allocation to be ignored for memory tracking purposes, six values in the debug memory allocation header for each allocation are set to specific values.

The Microsoft code goes out of its way to ensure no useful information can be gained from the header block for these ignored items.

When we first created MV we noticed that items marked as ignored should be ignored, otherwise you can end up with FALSE positive noise reported at the end of a memory debugging session due to the internal housekeeping of MFC/CRT.

How can you use this information in your application?

Microsoft also provides some flags which you can control which allows you to influence if any memory is reported as leaked. This is in addition to the CRT marking its own allocations as “ignore”. You can set these flags using the _CrtSetDbgFlag(int); function.

The following flags can be passed to _CrtSetDbgFlag() in any combination.

How do I disable memory tracking for the CRT?

If you call _CrtSetDbgFlag(0); any memory allocated after that point will not be tracked.

With the above settings, all blocks are marked as ignore. You can see the code for this in the Microsoft C runtime.

The code that marks the block as “ignore” is at line 404 in dbgheap.c in the Microsoft C runtime (also used by MFC). When your code arrives here, nBLockUse == 1 and _crtDbgFlag == 0.

dbgheap.c line 404 (line number will vary with Visual Studio version)
                if (_BLOCK_TYPE(nBlockUse) != _CRT_BLOCK &&
                     !(_crtDbgFlag & _CRTDBG_ALLOC_MEM_DF))
                     fIgnore = TRUE;

This sets fIgnore to TRUE. From this point onwards the memory tracking code ignores the memory and sets the values mentioned above in the memory block header.

Default values

The default value for _crtDbgFlag is set elsewhere in the Microsoft code with this line:

extern "C"
int _crtDbgFlag = _CRTDBG_ALLOC_MEM_DF | _CRTDBG_CHECK_DEFAULT_DF;

From time to time we provide special offers to customers in the form of a unique coupon code that can be used at purchase to get a specific discount. These coupons are also used to provide discounts to customers upgrading from say Performance Validator to C++ Developer Suite so that they do not pay for Performance Validator twice.

When the coupon management system was written, we used srand(clock()) thinking that would be an acceptable random value for generating coupons. The thinking was the management system would be running all the time and thus clock() would return a value that was unlikely to be hit twice for the number of valid coupons at any one time. However, the way the system is used is that users close the coupon management system when not in use and thus clock() will return values close to the starting time (start the app, navigate to the appropriate place, generate a coupon).

Result: Sooner or later a duplicate coupon is created. And that is when we noticed this problem.

This resulted in a confused customer (“My coupon has already been used”), a confused member of customer support (“That shouldn’t be possible!”) followed by some checking of the coupon files and then the code to see how it happened. Easy to fix, but better selection of the seed in the first place would have prevented the problem.

So if you want better random numbers don’t use clock() to seed srand().

Better seeds

• Use time(NULL) to get the time of day and cast the result to seed srand().
  time(NULL) returns the number of seconds elapsed since midnight January 1st, 1970.
• Use rdtsc() to get the CPU timestamp and cast the result to seed srand().
  rdtsc() is unlikely to return duplicate values as it returns the number of instructions executed by the processor since startup.

But! There are serious consequences for getting it wrong. So perhaps you’d better read about it after all

The problem that wasted my time

Why am I writing about datatype misalignment? Because its just eaten two days of my time and if what I share with you helps save you from such trouble, all the better.

The problem I was chasing was that three calls to CreateThread() were failing. All calls were failing with ERROR_NOACCESS. They would only fail if called from functions in the Thread Validator x64 profiling DLL injected into the target x64 application. If the same functions were called later in the application (via a Win32 API hook or directly from the target application) the functions would work. That meant that the input parameters were correct.

Lots of head scratching and trying many, many variations of input parameters and asking questions on Stack Overflow and we were stuck. I could only think it was to do with the callstack but I had no idea why. So I started investigating the value of RSP during the various calls. The investigating what happened if I pushed more data onto the stack to affect the stack pointer. After some trial and error I found a combination that worked. Then I experimented with that combination to determine if it was the values being pushed that were important or the actual value of the stack pointer that was important.

At this point I was confused, as I didn’t know about any stack alignment requirements, I only knew about data alignment requirements. I then went searching for appropriate information about stack alignments and found this handy document from Microsoft clarifies that.

What is datatype misalignment?

Datatype alignment is when the data read by the CPU falls on the natural datatype boundary of the datatype. For example, when you read a DWORD and the DWORD is aligned on a 4 byte boundary.

In the following code examples, let us assume that data points to a location aligned on a four byte boundary.

void aligned(BYTE	*data)
{
	DWORD	dw;

	dw = *(DWORD *)&bp[4];
}

Datatype misalignment is when the data read by the CPU does not fall on the natural datatype boundary of the datatype. For example, when you read a DWORD and the DWORD is not aligned on a 4 byte boundary.

void misaligned(BYTE	*data)
{
	DWORD	dw;

	dw = *(DWORD *)&bp[5];
}

Why should I care about datatype misalignment?

Aligned data reads and data writes happen at the maximum speed the memory subsystem and processor can provide. For example to read an aligned DWORD, one 32 bit data read needs to be performed.

DWORD	BYTE	[ 0];
	BYTE	[ 1];
	BYTE	[ 2];
	BYTE	[ 3];
DWORD	BYTE	[ 4];	// ignored
	BYTE	[ 5];	// read
	BYTE	[ 6];	// read
	BYTE	[ 7];	// read
DWORD	BYTE	[ 8];	// read
	BYTE	[ 9];	// ignored
	BYTE	[10];	// ignored
	BYTE	[11];	// ignored
DWORD	BYTE	[12];
	BYTE	[13];
	BYTE	[14];
	BYTE	[15];

Misligned data reads and data writes do not happen at the maximum speed the memory subsystem and processor can provide. For example to read a misaligned DWORD the processor has to fetch data for the two 32 bit words that the misaligned data straddles.

In the misaligned example shown above, the read happens at offset 5 in the input array. I’ve shown the input array first 16 bytes, marking where each DWORD starts and showing which bytes are read and which are ignored. If we assume the input array is aligned then the DWORD being read has 3 byte2 in the first DWORD and 1 bytes in the second DWORD. The processor has to read both DWORDs, then shuffle the bytes around, discarding the first byte from the first DWORD and discarding the last 3 bytes from the last DWORD, then combining the remaining bytes to form the requested DWORD.

Performance tests on 32 bit x86 processors shown performance drops of between 2x and 3x. Thats quite a hit. On some other architectures, the performance hit can be much worse. This largely depends on if the processor does the rearrangement (as on x86 and x64 processors) or if an operating system exception handler handles it for you (much slower).

I’ve shown the example with DWORDs, because the they are short enough to be easily shown in a diagram whereas 8 byte or larger values would be unweildy.

The above comments also apply to 8 byte values such as doubles, __int64, DWORD_PTR (on x64), etc.

Clearly, getting your datatype alignments optimized can be very handy in performance terms. Niave porting from 32 bit to 64 bit will not necessarily get you there. You may need to reorganise the order of some data members in your structures. We’ve had to do that with Thread Validator x64.

Not just performance problems either!

In addition to the performance problems mentioned above there is another, more important consideration to be aware of: On x64 Windows operating systems you must have the stackframe correctly aligned. Correct stack alignment on x64 systems means that the stack frame must be aligned on a 16 byte boundary.

Failure to ensure that this is the case will mean that a few Windows API calls will fail with the cryptic error code ERROR_NOACCESS (hex: 0×3E6, decimal: 998). This means “Invalid access to memory location”.

The problem with the error code ERROR_NOACCESS in this case is that the real error code that gets converted into ERROR_NOACCESS is STATUS_DATATYPE_MISALIGNMENT, which tells you a lot more. I spent quite a bit of time digging around until I found the true error code for the bug I was chasing that lead me to write this article.

If you are writing code using a compiler, the compiler will sort the stack alignment details out for you. However if you are writing code in assembler, or writing hooks using dynamically created machine language, you need to be aware of the 16 byte stack alignment requirement.

x64 datatype alignment requirements

Stack:
Correct stack alignment on x64 systems means that the stack frame must be aligned on a 16 byte boundary.

Data:

Anything larger than 8 bytes is aligned on the next power of 2 boundary.

Conclusion

Correct stack frame alignment is essential to ensure calling functions works reliably.

Correct datatype alignment is essential for maximum speed when accessing data.

Failure to align stack frames correctly could lead to Win32 API calls failing and or program failure or lack of correct behaviour.

Failure to align data correctly will lead to slow speed when accessing data. This could be disasterous for your application, depending upon what it is doing.

References

Microsoft x86/x64/IA64 alignment document.

Interesting problems including

• DLL Hooking
• DLL Import structure walking
• Injecting
• PE 64 bit DLL inspection

The first attempt at launching a 64 bit application and injecting into it worked. Awesome! First time out and something that hard to do worked first time. We’ve got a few datastructure size mismatches and a stack walking problem to look at, but looks like our first 64 bit port is going well.

Check back on the blog for progress or follow us on twitter. We’ll be announcing the progress of our various 64 bit ports on both the blog and twitter.

The 64 bit compiler also introduces some intrinsics which are defined in intrinsic.h. These intrinsics allow you to add calls to low level functionality in your 64 bit code. Such functionality includes setting breakpoints, getting CPU and hardware information (cpuid instruction) and reading the hardware timestamp counter.

In this article I’ll show you how you can use the same code for both 32 bit and 64 bit builds to have access to these intrinsics on both platforms.

__debugbreak()

The 64 bit compiler provides a convenient way for you to hard code breakpoints into your code. Often very useful for putting breakpoints in your code during testing. The __debugbreak() intrinsic provides this functionality.

There is no 32 bit __debugbreak();

For 32 bit systems you have to know 80386 assembly. The breakpoint instruction as opcode 0xcc. The inline assembly for this is __asm int 3;

	#define __debugbreak()				__asm { int 3 }

cpuid – 32 bit

On 32 bit systems there is no cpuid assembly instruction so you have to use the emit directive.

	#define cpuid	__asm __emit 0fh __asm __emit 0a2h

and then you can use cpuid anywhere you need to.

	void doCpuid()
	{
		__asm pushad;		// save all the registers - cpuid trashes EAX, EBX, ECX, EDX
		__asm mov eax, 0; 	// get simplest cpuid data

		cpuid;			// call cpuid, results returned in EAX, EBX, ECX, EDX

		// read cpuid results here before restoring the registers...

		__asm popad;		// restore registers
	}

cpuid – 64 bit

On 64 bit systems you have to use the intrinsic __cpuid(registers, 0) provided in the intrinsic.h file.

	void doCpuid()
	{
	    int registers[4];

	    __cpuid(registers, 0);
	}

rdtsc – 32 bit

On 32 bit systems there is no rdtsc assembly instruction so you have to use the emit directive.

	#define rdtsc	__asm __emit 0fh __asm __emit 031h

and then you can use rdtsc anywhere you need to.

	__int64 getTimeStamp()
	{
	    LARGE_INTEGER li;

	    rdtsc;

	    __asm	mov	li.LowPart, eax;
	    __asm	mov	li.HighPart, edx;
	    return li.QuadPart;
	}

rdtsc – 64 bit

On 64 bit systems you have to use the intrinsic __rdtsc() provided in the intrinsic.h file.

	__int64 getTimeStamp()
	{
	    return __rdtsc();
	}

Thats not very portable is it?

The problem with the above approach is that you end up having two implementations for these functions – one for your 32 bit build and one for your 64 bit build. It would be much more elegant to have a drop in replacement that you can use in your 32 bit code that will compile in the same manner as the 64 bit code that uses the intrinsics defined in intrinsic.h

Here is how you do it. Put all of the code below into a header file and #include that header file wherever you need access to __debugbreak(), __cpuid() or rdtsc().

#ifdef _WIN64
#include 
#else	// _WIN64
	// x86 architecture

	// __debugbreak()

	#if     _MSC_VER >= 1300
		// Win32, __debugbreak defined for VC2005 onwards
	#else	//_MSC_VER >= 1300
		// define for before VC 2005

		#define __debugbreak()				__asm { int 3 }
	#endif	//_MSC_VER >= 1300

	// __cpuid(registers, type)
	//		registers is int[4],
	//		type = 0

	// DO NOT add ";" after each instruction - it screws up the code generation

	#define rdtsc	__asm __emit 0fh __asm __emit 031h
	#define cpuid	__asm __emit 0fh __asm __emit 0a2h

	inline void __cpuid(int	cpuInfo[4],
						int	cpuType)
	{
		__asm pushad;
		__asm mov	eax, cpuType;

		cpuid;

		if (cpuInfo != NULL)
		{
			__asm mov	cpuInfo[0], eax;
			__asm mov	cpuInfo[1], ebx;
			__asm mov	cpuInfo[2], ecx;
			__asm mov	cpuInfo[3], edx;
		}

		__asm popad;
	}

	// __rdtsc()

	inline unsigned __int64 __rdtsc()
	{
		LARGE_INTEGER	li;

		rdtsc;

		__asm	mov	li.LowPart, eax;
		__asm	mov	li.HighPart, edx;
		return li.QuadPart;
	}

#endif	// _WIN64

Now you can just the the 64 bit style intrinsics in your code and not have to worry about any messy condition code for doing the 32 bit inline assembly or the 64 bit intrinsics. Much neater, elegant, readable and more maintainable.

Additional Information

If you wish to know more about __cpuid(), the parameters it takes and the values returned in the registers array, Microsoft have a __cpuid() instrinsic description which explains everything in great detail.

Cupid instruction

When I wrote this article I kept typing cupid instead of cpuid. I’m sure my mind was on something else . How would the cupid instruction be implemented…

Something happened on Sunday morning that made me re-assess if I should be a qualified First-Aider.
This post isn’t about software, but all the same it most likely applies to your workplace and quite possibly your private life.

What happened on Sunday? A gentleman I’d be talking with the evening before had a heart attack. People rushed around the campsite and surrounding buildings shouting “Medical Emergency! If you are a doctor, nurse or first aider please go to the main hall immediately”. I was in the campsite at the time. Your first thought is “What has happened?”, followed by “I’m not trained, I can’t help”. It was at that point I knew that I could do nothing to help whoever was in need of help. Its a horrible feeling. You don’t know who is ill, you most likely know who they are and you can’t help.

As it happened there was a doctor (a general practitioner) on site as well as a first aider that had done his training 20 years earlier. An ambulance arrived in short order and after some work resuscitating him they took him to hospital where he underwent appropriate treatment for the heart attack. Hopefully all will be well with him.

Over the remainder of the weekend over the course of various conversations it became clear that just about everyone had come to the same conclusion: Everyone really disliked the powerlessness of being unable to help and quite a few were considering getting themselves trained as a first aider so that they’d be suitably skilled should something happen again.

In the past I’d always been reluctant to consider getting qualified as a first aider. Not because I didn’t want to be nominated but more because I didn’t want to do the training. I can’t quite explain it – I think I had some embarrassment about the training you have to do with the dummy body. That and combined with the “it won’t happen near me” delusion – I deluded myself I’d never have any use for the training. How wrong I was.

Anyway, none of that matters now. I know if I’d be trained I could have helped and may have been of some assistance on Sunday. That would have counted. So I’m going to get some first aider training and encourage my colleagues to do so. How about you?

Under the hood this function uses structured exception handling to catch any exceptions that are thrown when the memory location read. If an exception is thrown the function returns TRUE, otherwise FALSE. So far so good.

So what could be wrong with that? A simplistic or naive interpretation would be “nothing”. But that ignores the fact that the callstacks of your application threads grow “on-demand”. This is done to avoid committing the full (default) 1MB upfront for each thread. This places a lower demand on application virtual memory and provides slightly faster startup for each thread. To allow each thread to grow on demand the stack has guard pages, which if you try to access them cause a guard page exception to be thrown which the OS handles gracefully, extends your stack space an appropriate amount and then returns execution to your application.

The problem with IsBadReadPtr() is that the exception handler inside IsBadReadPtr() eats the exception and thus the OS will not see it. So for the case where you end up using IsBadReadPtr() on a guard page you break the on-demand stack extension mechanism.

Raymond Chen of Microsoft has written a passionate post on this topic.

Raymond (and a few other folks) say that you should never use IsBadReadPtr(). I think thats a bit strong.

There are a few occasions where you may know what the datastructure is but you also know that it may have various memory protections on it. Such a case is when inspecting a DLL. Various parts are readonly. We have found during the last 10 years of writing tools like Memory Validator that it is not uncommon for a DLL loaded by LoadLibrary() to have memory protections on parts of the DLL that you don’t expect. We can’t control what DLLs our customer’s applications choose to load, so we have to handle all eventualities. We can’t just allow a crash to happen because we read a data location (in a customer DLL) that should be valid but isn’t.

Its also worth noting that the members of the team that wrote Boundschecker also came to the same conclusion and also tested certain DLL headers this way. You can find such code examples in the BugSlayer column in issues of Microsoft Systems Journal (MSJ) before it morphed into MSDN magazine.

One argument would be “Put an exception handler around it. Its an exceptional condition, handle it that way”.

The problem with that is sometimes that breaks the flow of the code and causes all manner of problems with the inability to mix C++ objects and SEH in the same function. Sometimes its much easier and simpler just to test for readability and abandon the function if you encounter one of these unusually constructed DLLs.

We are not advocating that you routinely use IsBadReadPtr() to hide the fact that you don’t know which objects to free, so you just call free on anything that passes IsBadReadPtr(). If you do that you will end up with exactly the problems that Raymond Chen describes.

But for the case where you do want IsBadReadPtr() functionality but you don’t want to use IsBadReadPtr(), what do you do? Here are drop in replacements for IsBadReadPtr() and IsBadWritePtr() that will not affect guard pages etc.

int isNotOKToReadMemory(void    *ptr,
                        DWORD   size)
{
	SIZE_T                          dw;
	MEMORY_BASIC_INFORMATION	mbi;
	int                             ok;

	dw = VirtualQuery(ptr, &mbi, sizeof(mbi));
	ok = ((mbi.Protect & PAGE_READONLY) ||
		  (mbi.Protect & PAGE_READWRITE) ||
		  (mbi.Protect & PAGE_WRITECOPY) ||
		  (mbi.Protect & PAGE_EXECUTE_READ) ||
		  (mbi.Protect & PAGE_EXECUTE_READWRITE) ||
		  (mbi.Protect & PAGE_EXECUTE_WRITECOPY));

	// check the page is not a guard page

	if (mbi.Protect & PAGE_GUARD)
		ok = FALSE;
	if (mbi.Protect & PAGE_NOACCESS)
		ok = FALSE;

	return !ok;
}

int isNotOKToWriteMemory(void   *ptr,
                         DWORD  size)
{
	SIZE_T                          dw;
	MEMORY_BASIC_INFORMATION	mbi;
	int                             ok;

	dw = VirtualQuery(ptr, &mbi, sizeof(mbi));
	ok = ((mbi.Protect & PAGE_READWRITE) ||
		  (mbi.Protect & PAGE_WRITECOPY) ||
		  (mbi.Protect & PAGE_EXECUTE_READWRITE) ||
		  (mbi.Protect & PAGE_EXECUTE_WRITECOPY));

	// check the page is not a guard page

	if (mbi.Protect & PAGE_GUARD)
		ok = FALSE;
	if (mbi.Protect & PAGE_NOACCESS)
		ok = FALSE;

	return !ok;
}

Remember: Use with caution, use sparingly and only if you need to. If you are routinely using IsBadReadPtr() or an equivalent to avoid keeping track of which data you should or should not use, you should think again about your software design.