Mick Mick - 1 year ago 139
C++ Question

"xxx.exe is not a valid Win32 application" after VS just built it

I have been successfully developing a WinAPI application in visual studio 2015 (using the IDE) on my Windows-7-64 PC. I usually test the program in Release mode.

I then made some edits to my source. The program compiled and linked without error but the program did not behave quite as I expected, so I switched to Debug mode and tried to build and run. Again VS compiled and linked without error but then complained

"Unable to start program 'f:\dropbox\blah\x64\Debug\xxx.exe'.
'f:\dropbox\blah\x64\Debug\xxx.exe' is not a valid Win32 application".

I thought it was strange, so I flipped back to release mode and tried again - the program started fine. I made some edits and re-built a few times, but then later VS declared

"Unable to start program 'f:\dropbox\blah\x64\Release\xxx.exe'.
'f:\dropbox\blah\x64\Release\xxx.exe' is not a valid Win32 application".

I tried clean all, restarted VS, even restarted my PC.. but all to no avail, I still get exactly the same errors.

EDIT: After reading about similar reports, I tried pausing dropbox syncing. It then seemed to work but only once or twice and then the problem returned. I then tried switching off multi-processor compilation and this seems to have allowed the release version of my program to run again. I have since edited-rebuilt-run many (50+?) times without problem - but it still refuses to run the debug version.

EDIT: FYI my antivirus software is Microsoft Security Essentials

EDIT: calling dumpbin and passing my (non-running debug exe) produces the following output:



1000 .00cfg
77BB8000 .data
1000 .gfids
4000 .idata
4000 .pdata
31000 .rdata
4000 .reloc
1000 .rsrc
DD000 .text

EDIT: Just tried compile-build-run on a different machine altogether (windows-10-64) that was linked via dropbox and have exactly the same symptoms, i.e. runs in release mode but not in debug mode.

EDIT: On the advice of Michael Burr I ran dependancy walker on my (non-working) debug exe and it reported these errors:
enter image description here
then out of curiosity I thought I'd have a look at what dep-walker said about my (working) release exe and found that I got exactly the same list of errors!... upon more searching I found this SO question in which it was concluded: "The gist of it: As someone elsewhere stated, the tool is a bit dated by now and does not always work properly with newer OS. Thus keep an eye open and don't get mislead by missing 'API-MS-WIN-CORE-COM-L1-1-0.DLL', ... the problem probably lies entirely elsewhere."

EDIT: I switch between debug and release mode from the selection box on the left in the image below and I run the program by clicking the green triangle.
enter image description here

EDIT: I generated the map file for the debug exe. It's too big to show here, but it starts off with the following lines...

Timestamp is 5811bed3 (Thu Oct 27 09:46:11 2016)

Preferred load address is 0000000140000000

Start Length Name Class
0001:00000000 00002840H .text$di CODE
0001:00002840 000da860H .text$mn CODE
0001:000dd0a0 00001020H .text$mn$00 CODE
0001:000de0c0 00001eb0H .text$x CODE
0001:000dff70 0000104bH .text$yd CODE
0002:00000000 00000110H .CRT$XCA DATA
0002:00000110 00000110H .CRT$XCAA DATA
0002:00000220 00000110H .CRT$XCL DATA
0002:00000330 00000128H .CRT$XCU DATA
0002:00000458 00000110H .CRT$XCZ DATA
0002:00000568 00000110H .CRT$XIA DATA
0002:00000678 00000110H .CRT$XIAA DATA
0002:00000788 00000110H .CRT$XIAC DATA
0002:00000898 00000110H .CRT$XIZ DATA
0002:000009a8 00000110H .CRT$XPA DATA
0002:00000ab8 00000110H .CRT$XPZ DATA
0002:00000bc8 00000110H .CRT$XTA DATA
0002:00000cd8 00000118H .CRT$XTZ DATA
0002:00000df0 0002c960H .rdata DATA
0002:0002d750 00000998H .rdata$r DATA
0002:0002e0e8 00000178H .rdata$zzzdbg DATA
0002:0002e260 00000110H .rtc$IAA DATA
0002:0002e370 00000188H .rtc$IMZ DATA
0002:0002e4f8 00000110H .rtc$IZZ DATA
0002:0002e608 00000110H .rtc$TAA DATA
0002:0002e718 00000188H .rtc$TMZ DATA
0002:0002e8a0 00000110H .rtc$TZZ DATA
0002:0002e9b0 00003b68H .xdata DATA
0002:00032518 00000275H .xdata$x DATA
0002:0003278d 00000000H .edata DATA
0003:00000000 000023e0H .data DATA
0003:000023e0 00000580H .data$r DATA
0003:00002960 77376001H .bss DATA
0004:00000000 0000369cH .pdata DATA
0005:00000000 00000ed0H .idata$5 DATA
0005:00000ed0 000000c8H .idata$2 DATA
0005:00000f98 00000018H .idata$3 DATA
0005:00000fb0 00000ed0H .idata$4 DATA
0005:00001e80 00001fc6H .idata$6 DATA
0006:00000000 0000015eH .gfids$y DATA
0007:00000000 0000011bH .00cfg DATA
0008:00000000 00000170H .rsrc$01 DATA
0008:00000170 000002ccH .rsrc$02 DATA

Address Publics by Value Rva+Base Lib:Object

0000:00000000 __guard_iat_table 0000000000000000 <absolute>
0000:00000000 __guard_longjmp_count 0000000000000000 <absolute>
0000:00000000 __guard_longjmp_table 0000000000000000 <absolute>
0000:00000000 __guard_fids_count 0000000000000000 <absolute>
0000:00000000 ___safe_se_handler_table 0000000000000000 <absolute>
0000:00000000 ___safe_se_handler_count 0000000000000000 <absolute>
0000:00000000 __guard_iat_count 0000000000000000 <absolute>
0000:00000000 __guard_fids_table 0000000000000000 <absolute>
0000:00000000 __dynamic_value_reloc_table 0000000000000000 <absolute>
0000:00000100 __guard_flags 0000000000000100 <absolute>
0000:00000000 __ImageBase 0000000140000000 <linker-defined>
0001:00002aa0 ?readstring@@YAXPEAD0@Z 0000000140003aa0 f COMMAND.obj
0001:00002b70 ?make_phere@@YAXH@Z 0000000140003b70 f COMMAND.obj
0001:00002c50 ?load_snap@@YAXXZ 0000000140003c50 f COMMAND.obj
0001:00002d30 ?i_rand_0_n_inclusive@@YAHH@Z 0000000140003d30 f COMMAND.obj

Answer Source
  77BB8000 .data

That's almost certainly the problem, you have a very large data section. Its size is suspiciously close to what it is possible in a 32-bit process on Windows. You can get a more consistent repro from this sample C program:

unsigned char kaboom[0x7d000000];

int main()
    return 0;

Not a very good error message btw, Microsoft did not reserve an error code for this corner-case. And sure, it is not going to repeat that well when you are close to the edge with 0x77BB8000. The amount of space that is available for that data section varies from one run to the next. A 32-bit process has a 2 GB address space (0x80000000) but subtracted from that are the non-mappable regions at the start and the end of the address space and the space required for the operating system DLLs (at least ntdll.dll and kernel32.dll). And the space you lose due to ASLR (Address Space Layout Randomization), a number that changes. And DLLs that are injected, like those used by anti-malware and Dropbox.

It can't be guessed why your data section needs to be so large. Or why this needs to be a 32-bit process, just about nobody still has a 32-bit operating system. Target x64 instead for the quick fix, use the free store (malloc etc) to allocate large arrays.