Dia Dia - 8 months ago 61
C# Question

Exception when closing a Win32 dialog box in dll (from WPF application)

I have a C# application that is calling a C++ function from a DLL. The C++ function shows a dialog box and a exit button only.

The C++ function in DLL is looking like:

//the exported function for clients to call in DLL
HINSTANCE hInstance;
__declspec(dllexport) int __cdecl StartDialog(string inString)
hInstance = LoadLibrary(TEXT("My.dll"));
DialogBox(hInstance, MAKEINTRESOURCE(ID_DLL_Dialog), NULL, DialogProc);
return 1;

BOOL CALLBACK DialogProc(HWND hwnd, UINT message, WPARAM wParam, LPARAM lParam)
switch (message)
switch (LOWORD(wParam))
return TRUE;
return FALSE;

If I call my
function in a simple C++ program, it works. I can show the dialog, and can close it properly when I click exit in my dialog.

typedef int(__cdecl *StartDialogFunc)(string);
StartDialogFunc StartDialog = nullptr;
HINSTANCE hDll = LoadLibrary(TEXT("My.dll"));
StartDialog = (StartDialogFunc)GetProcAddress(hDll, "StartDialog");

int i = StartDialog("hello"); //this is working
cout << i;

If I call it in my C# application, after I click the exit button, the dialog closes, and give me an exception. The C# code looks like:

[DllImport("My.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern int StartDialog(string s);

int i = StartDialog("hello"); //this causes a exception

The error message is like:

Debug Assertion Failed!

Program: (some paths...) My.dll

File: d:\program files (x86)\microsoft visual studio 14.0\vc\include\xmemory0

Line: 100

Expression: "(_Ptr_user & (_BIG_ALLOCATION_ALIGNMENT - 1)) == 0" && 0

For information on how your program can cause an assertion failure, see the Visual C++ documentation on asserts.

How can I know what is exactly wrong in my DLL?

Answer Source

Try changing the C++ function signature to take a WCHAR*, as C++'s std::string is incompatible with C#. Also, to get a handle to the current DLL, use GetModuleHandle(NULL), not LoadLibrary.

HINSTANCE hInstance;

__declspec(dllexport) int __cdecl StartDialog(const WCHAR* inString)
    hInstance = GetModuleHandle(NULL);
    DialogBox(hInstance, MAKEINTRESOURCE(ID_DLL_Dialog), NULL, DialogProc);
    return 1;