morgan freeman morgan freeman - 2 years ago 85
Ruby Question

How to measure how much memory each gem requires at initialization?

I have a Rails 2.3.10 app with bundler. At startup the memory footprint is quite big (300MB in development mode).

I would like to see how much memory each gem takes on startup.

Answer Source

We had a problem in which our basic Rails app, with no traffic or requests, had a footprint of ~140MB on startup.

We used the following approach to trace the memory requirement of each gem specified in the Gemfile of our app, without having to try to patch bundler.

  1. using rails new myappname generate a new empty rails app
  2. Copy the Gemfile from the main project to this new rails project
  3. run bundle install and then rails server to ensure it is possible to boot up the rails server and that any basic configurations required are loaded
  4. Open up the Gemfile and with the exception of the specification for the rails gem, append require: false at the end of each line. Ensure that any other gems that are specified with one name but required using :require => 'othergemname' are using the older ruby Hash notation so that the pattern match below will catch it.
  5. Run bundle install again to regenerate the Gemfile.lock
  6. Create the following script which will use manually require each gem specified in the Gemfile and log the system memory consumed by the rails process before and after.

    # require_and_profile.rb
    def require_and_profile(gemname = nil)
      unless gemname
        puts "%-20s: %10s | %10s" % ['gem','increment','total']
      # This is how to get memory of calling process in OS X, check host OS for variants
      memory_usage = `ps -o rss= -p #{}`.to_i / 1024.0
      require gemname
      puts "%-20s: %10.2f | %10.2f" % [ gemname, (`ps -o rss= -p #{}`.to_i / 1024.0 - memory_usage), (`ps -o rss= -p #{}`.to_i / 1024.0)]
    pattern = /^[^#]*gem[ ]*['"]([^,'"]*)['"][ ,~>0-9\.'"]*(:require[ => ]*['"]([^'"]*)['"][, ])?/
    require_and_profile'Gemfile').each do |line|
      if line.match(pattern)
      if line.match(pattern)[3]
        require_and_profile line.match(pattern)[3]
          require_and_profile line.match(pattern)[1]
  7. Run rails c

  8. load 'require_and_profile.rb'
  9. The output shows how much (in MB) each gem adds to the base app footprint (increment) and what the total footprint is after inclusion of the gem (total).

This helped us identify for example, that we'd been requiring asset-sync in our boot when we only needed it in the :asset group. We do find that on different boot-ups the memory footprint of each gem is not exactly the same, but running it a few times does show you the patterns of which ones are the memory-hungry gems.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download