juanitogan juanitogan - 7 months ago 63
Ruby Question

Trouble with OpenSSL on RHEL 6.3 and all Ruby installers

OpenSSL does not appear to be compiling correctly when installing any version of Ruby on our RHEL 6.3 system. I have been trying to leave user installs of RVM behind and replace them with root installs via ruby-install and chruby. OpenSSL works okay in our RVM user installs (with the prescribed RVM fix) as well as in the built-in system install of Ruby 1.8.7 in

/usr/bin
.

OpenSSL is broken in each Ruby version I have tried with ruby-install, ruby-build, and even RVM when using their latest suggested fix. 1.9.3-p392 (our prod version), 1.9 latest, and 2.1.0 current. I have tried every openssl fix/workaround I can find, such as the
--with-openssl-dir=/some/dir
config pointing to various openssl folders, but nothing works for me.

Here are some relevant messages from a few of my many attempts:


[root@dbatcit ~]# ruby-install ruby
>>> Installing ruby 2.1.0 into /opt/rubies/ruby-2.1.0 ...
>>> Installing dependencies for ruby 2.1.0 ...
Loaded plugins: product-id, rhnplugin, security, subscription-manager
Updating certificate-based repositories.
Unable to read consumer identity
Setting up Install Process
Package gcc-4.4.7-4.el6.x86_64 already installed and latest version
Package automake-1.11.1-4.el6.noarch already installed and latest version
Package zlib-devel-1.2.3-29.el6.x86_64 already installed and latest version
Package libyaml-devel-0.1.3-1.el6.x86_64 already installed and latest version
Package openssl-devel-1.0.1e-16.el6_5.4.x86_64 already installed and latest version
Package gdbm-devel-1.8.0-36.el6.x86_64 already installed and latest version
Package readline-devel-6.0-4.el6.x86_64 already installed and latest version
Package ncurses-devel-5.7-3.20090208.el6.x86_64 already installed and latest version
Package libffi-devel-3.0.5-3.2.el6.x86_64 already installed and latest version
Nothing to do
.
.
make[2]: Entering directory `/usr/local/src/ruby-2.1.0/ext/openssl'
compiling ossl_pkey.c
compiling ossl_ssl.c
ossl_ssl.c:121: error: âTLSv1_2_methodâ undeclared here (not in a function)
ossl_ssl.c:122: error: âTLSv1_2_server_methodâ undeclared here (not in a function)
ossl_ssl.c:123: error: âTLSv1_2_client_methodâ undeclared here (not in a function)
ossl_ssl.c:127: error: âTLSv1_1_methodâ undeclared here (not in a function)
ossl_ssl.c:128: error: âTLSv1_1_server_methodâ undeclared here (not in a function)
ossl_ssl.c:129: error: âTLSv1_1_client_methodâ undeclared here (not in a function)
make[2]: *** [ossl_ssl.o] Error 1
make[2]: Leaving directory `/usr/local/src/ruby-2.1.0/ext/openssl'
make[1]: *** [ext/openssl/all] Error 2
make[1]: Leaving directory `/usr/local/src/ruby-2.1.0'
make: *** [build-ext] Error 2
!!! Compiling ruby 2.1.0 failed!



[root@dbatcit ~]# ruby-install ruby 1.9
>>> Installing ruby 1.9.3-p484 into /opt/rubies/ruby-1.9.3-p484 ...
>>> Installing dependencies for ruby 1.9.3-p484 ...
Loaded plugins: product-id, rhnplugin, security, subscription-manager
Updating certificate-based repositories.
Unable to read consumer identity
Setting up Install Process
Package gcc-4.4.7-4.el6.x86_64 already installed and latest version
Package automake-1.11.1-4.el6.noarch already installed and latest version
Package zlib-devel-1.2.3-29.el6.x86_64 already installed and latest version
Package libyaml-devel-0.1.3-1.el6.x86_64 already installed and latest version
Package openssl-devel-1.0.1e-16.el6_5.4.x86_64 already installed and latest version
Package gdbm-devel-1.8.0-36.el6.x86_64 already installed and latest version
Package readline-devel-6.0-4.el6.x86_64 already installed and latest version
Package ncurses-devel-5.7-3.20090208.el6.x86_64 already installed and latest version
Package libffi-devel-3.0.5-3.2.el6.x86_64 already installed and latest version
Nothing to do
.
.
make[2]: Entering directory `/usr/local/src/ruby-1.9.3-p484/ext/openssl'
compiling ossl_pkey.c
compiling ossl_ssl.c
compiling ossl_pkcs12.c
compiling ossl_bn.c
compiling ossl_hmac.c
ossl_hmac.c: In function âossl_hmac_copyâ:
ossl_hmac.c:90: warning: implicit declaration of function âHMAC_CTX_copyâ
compiling ossl_asn1.c
compiling ossl.c
compiling ossl_bio.c
compiling ossl_pkey_rsa.c
compiling ossl_ocsp.c
ossl_ocsp.c: In function âossl_ocspreq_add_certidâ:
ossl_ocsp.c:180: warning: function called through a non-compatible type
ossl_ocsp.c:180: note: if this code is reached, the program will abort
ossl_ocsp.c: In function âossl_ocspreq_get_certidâ:
ossl_ocsp.c:200: warning: function called through a non-compatible type
ossl_ocsp.c:200: note: if this code is reached, the program will abort
ossl_ocsp.c: In function âossl_ocspbres_get_statusâ:
ossl_ocsp.c:541: warning: function called through a non-compatible type
ossl_ocsp.c:541: note: if this code is reached, the program will abort
compiling ossl_pkey_dh.c
ossl_pkey_dh.c: In function âossl_dh_initializeâ:
ossl_pkey_dh.c:184: warning: function called through a non-compatible type
ossl_pkey_dh.c:184: note: if this code is reached, the program will abort
ossl_pkey_dh.c: In function âossl_dh_to_public_keyâ:
ossl_pkey_dh.c:372: warning: function called through a non-compatible type
ossl_pkey_dh.c:372: note: if this code is reached, the program will abort
compiling ossl_ns_spki.c
compiling ossl_x509attr.c
compiling ossl_x509name.c
ossl_x509name.c: In function âossl_x509name_hash_oldâ:
ossl_x509name.c:342: warning: implicit declaration of function âX509_NAME_hash_oldâ
compiling ossl_pkcs7.c
compiling ossl_pkey_ec.c
ossl_pkey_ec.c: In function âossl_ec_group_initializeâ:
ossl_pkey_ec.c:784: warning: function called through a non-compatible type
ossl_pkey_ec.c:784: note: if this code is reached, the program will abort
ossl_pkey_ec.c: In function âossl_ec_group_to_stringâ:
ossl_pkey_ec.c:1154: warning: function called through a non-compatible type
ossl_pkey_ec.c:1154: note: if this code is reached, the program will abort
compiling ossl_ssl_session.c
ossl_ssl_session.c: In function âossl_ssl_session_initializeâ:
ossl_ssl_session.c:53: warning: function called through a non-compatible type
ossl_ssl_session.c:53: note: if this code is reached, the program will abort
ossl_ssl_session.c:57: warning: function called through a non-compatible type
ossl_ssl_session.c:57: note: if this code is reached, the program will abort
ossl_ssl_session.c: In function âossl_ssl_session_to_pemâ:
ossl_ssl_session.c:251: warning: function called through a non-compatible type
ossl_ssl_session.c:251: note: if this code is reached, the program will abort
compiling openssl_missing.c
compiling ossl_x509.c
compiling ossl_x509cert.c
compiling ossl_digest.c
compiling ossl_pkcs5.c
ossl_pkcs5.c: In function âossl_pkcs5_pbkdf2_hmacâ:
ossl_pkcs5.c:39: warning: implicit declaration of function âPKCS5_PBKDF2_HMACâ
compiling ossl_rand.c
compiling ossl_engine.c
compiling ossl_x509crl.c
compiling ossl_cipher.c
ossl_cipher.c: In function âossl_cipher_copyâ:
ossl_cipher.c:143: warning: implicit declaration of function âEVP_CIPHER_CTX_copyâ
compiling ossl_x509ext.c
compiling ossl_config.c
compiling ossl_x509store.c
compiling ossl_x509revoked.c
compiling ossl_pkey_dsa.c
compiling ossl_x509req.c
linking shared-object openssl.so
installing default openssl libraries
make[2]: Leaving directory `/usr/local/src/ruby-1.9.3-p484/ext/openssl'
.
.
>>> Successfully installed ruby 1.9.3-p484 into /opt/rubies/ruby-1.9.3-p484


Note all the ossl warnings above.

Restart session.
Test system Ruby 1.8.7 openssl: Works.
Test ruby-install Ruby 1.9.3 openssl: Fails.

[root@dbatcit ~]# chruby
ruby-1.9.3-p484
[root@dbatcit ~]# which ruby
/usr/bin/ruby
[root@dbatcit ~]# ruby -v
ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]
[root@dbatcit ~]# ruby -ropenssl -e "puts OpenSSL::VERSION"
1.0.0
[root@dbatcit ~]# chruby 1.9
[root@dbatcit ~]# chruby
* ruby-1.9.3-p484
[root@dbatcit ~]# which ruby
/opt/rubies/ruby-1.9.3-p484/bin/ruby
[root@dbatcit ~]# ruby -v
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
[root@dbatcit ~]# ruby -ropenssl -e "puts OpenSSL::VERSION"
/opt/rubies/ruby-1.9.3-p484/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': /opt/rubies/ruby-1.9.3-p484/lib/ruby/1.9.1/x86_64-linux/openssl.so: undefined symbol: EC_GROUP_new_curve_GF2m - /opt/rubies/ruby-1.9.3-p484/lib/ruby/1.9.1/x86_64-linux/openssl.so (LoadError)
from /opt/rubies/ruby-1.9.3-p484/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
from /opt/rubies/ruby-1.9.3-p484/lib/ruby/1.9.1/openssl.rb:17:in `<top (required)>'
from /opt/rubies/ruby-1.9.3-p484/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
from /opt/rubies/ruby-1.9.3-p484/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
[root@dbatcit ~]#


[root@dbatcit ~]# which -a openssl
/usr/bin/openssl
/usr/local/bin/openssl
[root@dbatcit ~]# openssl version
OpenSSL 1.0.1e-fips 11 Feb 2013
[root@dbatcit ~]# /usr/local/bin/openssl version
OpenSSL 0.9.8d 28 Sep 2006


Test RVM Ruby 1.9.3 openssl in user home: Works.

[userbob@dbatcit ~]$ ruby -v
ruby 1.9.3p392 (2013-02-22 revision 39386) [x86_64-linux]
[userbob@dbatcit ~]$ ruby -ropenssl -e "puts OpenSSL::VERSION"
1.1.0
[userbob@dbatcit ~]$ which openssl
/usr/local/bin/openssl
[userbob@dbatcit ~]$ openssl version
OpenSSL 0.9.8d 28 Sep 2006
[userbob@dbatcit ~]$ .rvm/usr/bin/openssl version
OpenSSL 1.0.1c 10 May 2012

Answer

After far too many hours of research, and learning far more about Linux than I ever cared to, I have narrowed the problem to basic peculiarities of RHEL and OpenSSL and an incorrect assumption made by Ruby (extconf.rb) during installation. The following sites gave me some good clues as to what to look at but I found nothing that put it all together like what I needed.

...and not much thanks to a whole bunch of red herrings involving Ruby patches and EC2M. Also, RVM needs to correct their optimism that they have accounted for this with autolibs and should reinstate their previous openssl page.

Basic solution rules

Rule 1

The install of OpenSSL (1.0.1e) created and maintained by yum in /usr/bin cannot be used to compile Ruby's OpenSSL extension correctly -- at least, not on my machine at this time with the latest versions of Ruby (1.9.3-p484, 2.0.0, 2.1.0). RHEL 6.3. I can only surmise that this is due to peculiarities in RedHat's compilation of OpenSSL as hinted at in the OpenSSL FAQ.

Rule 2

I found two old versions (0.9.8) of OpenSSL in /usr/local (in bin + openssl, and ssl/bin) and updating/replacing these got me a bit closer to a solution. For whatever reason, every manual install of OpenSSL 1.0.1f in /usr/local (regardless of bin,openssl,ssl directory arrangement) insisted on putting the libraries in /usr/local/lib64 instead of /usr/local/lib (unless I hacked the Makefile, of course). Ruby's expconf.rb script, however, assumes the OpenSSL libraries will always be in a lib directory. Chasing down this single annoyance (and clash with OpenSSL) was the hardest part of all this. Therefore, to make using an install of OpenSSL in /usr/local work, you must do two things: (1) install Ruby with the --with-openssl-dir switch, and (2) recompile Ruby's OpenSSL extension while also modifying the Makefile to point to lib64 instead of lib. Thus, run something like the following string of commands as root:

ruby-install ruby 1.9.3-p545 -- --with-openssl-dir=/usr/local
cd /usr/local/src/ruby-1.9.3-p545/ext/openssl
ruby extconf.rb

Edit openssl's Makefile to replace something like this:

  libpath = . $(libdir) /usr/local/lib
  LIBPATH =  -L. -L$(libdir) -Wl,-R$(libdir) -L/usr/local/lib -Wl,-R/usr/local/lib

with something like this:

  libpath = . $(libdir) /usr/local/lib64
  LIBPATH =  -L. -L$(libdir) -Wl,-R$(libdir) -L/usr/local/lib64 -Wl,-R/usr/local/lib64

Save, and back to the command line:

make
make install

The new Ruby install should now work with OpenSSL properly. As a quick check, I restart my sudo session and then (assuming using chruby):

chruby 1.9
ruby -ropenssl -e "puts OpenSSL::VERSION"

Rule 3

Installing OpenSSL anywhere besides /usr puts the libraries in the expected lib instead of lib64. (Don't ask me why... dunno.) This may be the more maintainable solution as it lets you avoid hacking up the Makefile. This is also the solution RVM uses when running rvm pkg install openssl. Thus, to install both OpenSSL and Ruby (in /opt), you may run commands something like these (I run as sudo bash):

Install OpenSSL:

cd /opt/local
wget http://www.openssl.org/source/openssl-1.0.1f.tar.gz
tar -xzf openssl-1.0.1f.tar.gz
cd openssl-1.0.1f
./config --prefix=/opt/local shared no-asm zlib > openssl_config.log
make > openssl_make.log
make install > openssl_install.log

(The shared switch is required for Ruby to install without error, the no-asm switch helps get rid of a Make warning but does not appear to be required, and zlib and other switches are optional.)

Optional, update openssl certs:

cd /opt/local/ssl
wget http://curl.haxx.se/ca/cacert.pem
mv cacert.pem cert.pem
cd /opt

Back to Ruby:

ruby-install ruby 1.9.3-p545 -- --with-openssl-dir=/opt/local

The new Ruby install should now work with OpenSSL properly. As a quick check, I restart my sudo session and then (assuming using chruby):

chruby 1.9
ruby -ropenssl -e "puts OpenSSL::VERSION"
Comments