Earlier I managed to port some C++ CryptoPP Rijndael_128 CBC code to MCrypt PHP, but now I'm having problems with CFB mode. The C++ and PHP results do not match (well the first byte matches but this could be coincidence, everything else doesn't). With some diagnostics, it looks like PHP's mcrypt is not setting the key length correctly?
Here's the C++ (diagnostics and sundries removed for simplicity):
CFB_Mode<AES>::Encryption encryptor(g_encrypt_key, AES::DEFAULT_KEYLENGTH, g_encrypt_iv);
StringSource ss( sInput.c_str(), true,
new StreamTransformationFilter( encryptor,
new HexEncoder( new StringSink( sEncryptedOut ) )
$cipher = mcrypt_module_open(MCRYPT_RIJNDAEL_128, '', MCRYPT_MODE_CFB, '')
mcrypt_generic_init($cipher, $g_encrypt_key, $g_encrypt_iv);
$sEncryptedOutput = mcrypt_generic( $cipher, $sInput);
It's been awhile, but I had some similar problems with mcrypt and openSSL using CFB a couple years ago. In the end, I discovered mcrypt used a different default feedback chain size than openssl in CFB mode. That is to say, I believe an openSSL AES128 in CFB used a block size and feedback size of 128 bits, while mcrypt used a block size of 128bits and a feedback size of 8 bits. I have no way to confirm this, it was just speculation at the time based on reading some old forum posts. Regardless of the truth of that theory, I was not the only person or first to have this particular issue.
The solution for me was to use nOFB as yourself. According to the PHP mcrypt library reference
MCRYPT_MODE_NOFB forces the feedback-chain to equal the algorithm's block size, in this case a 128bit block/feedback for AES128 (Rijndael), which matches with what the manpage for the mcrypt module states about nOFB. This is good as everything I found said nOFB feedback is synchronous to the block size. Thus, both mcrypt and OpenSSL in nOFB were now 128 bit key/iv/block/feedback sizes for AES128 and everything worked fine.
As far as PHP reporting 256bit keysizes (32 bytes), the function that returns the current cipher-algorithm key size actually returns the maximum key size, which isn't clearly stated in the documentation. I know this because my little class I use all the time now for various projects works perfectly fine with openSSL and any other AES libraries in CBC or nOFB . This wouldn't be the case if mcrypt was padding my 128bit(16 char) key with an additional 128bits of null string, or whatever, and wouldn't be technically correct anyhow.
Not really a good answer, but the best I got based on a very amateurish foray into cryptography several years ago.