I've built a simple, custom, PAM module and the module works as expected with pamtester and with some other pam testing code I found online.
I've also built a simple dotnet core application which uses pinvoke to authenticate against pam.
As mentioned, the custom module works with pamtester and the custom dotnet app works (up to the point of receiving the conv callback) against the default ubuntu 15.10 pam setup.
However, if I hook the custom app up to the custom PAM module, I get an error in auth.log to say
dotnet: PAM unable to dlopen(pam_permit_log_response.so):
/lib/security/pam_permit_log_response.so: cannot open shared object file: No
such file or directory
dotnet: PAM unable to dlopen(/lib/x86_64-linux-
gnu/security/pam_permit_log_response.so: undefined symbol: pam_syslog
I still don't know why it's an issue calling into my custom module from dotnet and not from pamtester but, to answer the most important question: "How I can build and/or configure my custom pam module to behave the same as the pre-existing system modules?"
The answer is that previously, I was linking the module using the following:
ld -x --shared -o pam_permit_log_response.so pam_permit_log_response.o
To fix the problem, I needed to indicate to the linker that this is a pam module using the -lpam flag so the call to link becomes:
ld -x --shared -o pam_permit_log_response.so pam_permit_log_response.o -lpam