1. Jun
    14

    Benchmark ruby versus node.js on Heroku

    Posted in : ruby, node, heroku, sinatra, mongo_mapper, unicorn, express, mongoose, and cluster

    I’ve been playing a lot with node lately thanks to this PeepCode screencast and since Heroku released their new Celadon Cedar stack I’ve been wanting to benchmark ruby versus node.js for a restful API which I need to build.

    At first I tried to compare bare node.js against eventmachine_httpserver. It did quickly became obvious that this kind of micro-benchmark wasn’t going to be very helpful in deciding which one to choose.

    Building the API was going to be messy unless I did start using something like sinatra or express.

    Methodology

    I did setup 2 similar repositories on github:

    https://github.com/JosephHalter/heroku_ruby_bench
    

    This repository is using sinatra, mongo_mapper, yajl and unicorn.

    https://github.com/JosephHalter/heroku_node_bench
    

    This repository is using express, mongoose and cluster.

    I deployed both on Heroku, did setup a free mongohq account for both and run a few tests with various number of workers. All tests have been done from a datacenter in Paris using the following command:

    ab -k -t 10 -c 1000 http://evening-robot-961.herokuapp.com/restaurant/4df550c5c3aaaa0100000001
    ab -k -t 10 -c 1000 http://cold-mist-128.herokuapp.com/restaurant/4df55a0fab9e270007000001

    evening-robot-961 is running on node 0.4.7 and cold-mist-128 is running on ruby 1.9.2

    Checklist

    For those asking:

    • yes, I’ve waited enough time between each heroku scale
    • yes, I’ve retried each test a crazy number of times

    Heating

    Here are the raw results for a concurrency of 100, just to ensure everything works as expected before starting the real test:

                             | completed | failed |
    express/node             | 3358      | 0      |
    express/cluser/3 workers | 7473      | 0      |
    sinatra/thin             | 1649      | 0      |
    unicorn/4 workers        | 6080      | 0      |
    

    Results

    Here are the raw results for a concurrency of 1000:

                               | completed | failed |
    node/1 dyno                | 40524     | 27865  |
    node/5 dynos               | 20755     | 11575  |
    node/10 dynos              | 36953     | 9866   |
    node/20 dynos              | 34724     | 7486   |
    node/40 dynos              | 33919     | 8863   |
    node/60 dynos              | 32218     | 8984   |
    cluster/1 worker           | 21307     | 13193  |
    cluster/2 workers          | 41679     | 18904  |
    cluster/3 workers          | 40700     | 12864  |
    cluster/3 workers/5 dynos  | 37292     |  8360  |
    cluster/3 workers/10 dynos | 19787     | 10870  |
    cluster/3 workers/15 dynos | 35894     |  7119  |
    cluster/3 workers/20 dynos | 35871     |  9807  |
    cluster/3 workers/40 dynos | 32727     |  8371  |
    cluster/4 workers          | 22262     | 14738  |
    thin/1 dyno                | 38813     | 36769  |
    thin/5 dynos               | 40885     | 35178  |
    thin/10 dynos              | 42141     | 29082  |
    thin/40 dynos              | 33014     |  9732  |
    thin/60 dynos              | 31392     |  8747  |
    unicorn/1 worker           | 41032     | 39498  |
    unicorn/2 workers          | 24991     | 22152  |
    unicorn/3 workers          | 22601     | 16319  |
    unicorn/3 workers/5 dynos  | 20386     | 11012  |
    unicorn/4 workers          | 44127     | 37607  |
    unicorn/4 workers/5 dynos  | 39591     | 13426  |
    unicorn/4 workers/10 dynos | 35672     |  9511  |
    unicorn/4 workers/15 dynos | 35925     |  7997  |
    unicorn/4 workers/20 dynos | 34611     |  8131  |
    unicorn/4 workers/40 dynos | 18873     | 11125  |
    unicorn/8 workers          | 45904     | 39819  |
    

    A few charts to make it easier to read:

    express

    express with cluster

    express with cluster/3 workers

    sinatra with thin

    sinatra with unicorn

    sinatra with unicorn/4 workers

    Conclusions

    Using either ruby or node I could easily get more than 2000req/s. I think both are viable alternatives. With only 1 dyno however, you’ll start to have failed connection when faced with massive concurrency because the backlog is full. Increasing the number dynos doesn’t magically allow you to handle more requests per second, however it can decrease the number of failed connections. Heroku pricing is linear to the number of dynos you scale to, however your throughput does only only improve marginally. It could be that I’m benchmarking from only 1 server but I’ve seen almost no difference between having 40 dynos or 60. You’ll see one when you receive the bill so be cautious, especially if you use an auto-scale tool.

    Apart from that, we’re seeing a huge improvement when using unicorn with 4 workers instead of thin which was the only possibility before Celadon Cedar so a big thank you to Heroku who did make this possible. The same applies to node with cluster, you can do more with 15 dynos running cluster with 3 workers than with 60 dynos of node alone (for a quarter of the price!). Special thanks to TJ Holowaychuk who did help us to fixing a stupid issue which prevented us from using cluster on Heroku.

    Comments

  2. May
    23

    Link to static assets on Rails 3.1

    Posted in : rails, sass, assets, and sprockets

    Having just migrated an application to Rails 3.1, we discovered that the following line is not valid anymore:

    = javascript_include_tag "rightjs", "application", :cache => "all"

    The easiest solution is moving all your javascripts in app/assets/javascripts/ then create a file all.js

    // require "rightjs"
    // require "application"

    Or we can also just add the require in application.js itself. Soon we had to do the same for our stylesheets: move them to app/assets/stylesheets.

    Also, the default path where image_tag will look for our images is now app/assets/images and no more public/images. The public folder does now shrink to only a handful of files.

    The only remaining issue should now be with images (and fonts) referenced in the stylesheets as ../images doesn’t work anymore. The first thing we tried was changing:

    url(../images/myimage.png)

    To:

    url(/assets/myimage.png)

    Ok now it works, but is only suitable for dev because it does hit our application to serve the static files. Hopefully, Rails has a new rake task to precompile all assets to the public/assets folder:

    bundle exec rake assets:precompile RAILS_ENV=production

    Don’t forget to specify the environment otherwise the css won’t be minified properly. Also our files will be renamed from:

    application.css
    

    to:

    application-ab5f...(more letters and digits).css
    

    It combines the original filename with a hash, a MD5 digest of the file content after evalutation. This is to ensure proper cache reloading - the old way with query string didn’t work with some proxies. In order to avoid serving static assets in production, we have to reference these images directly, with the hash in filename. So intead of:

    url(/assets/myimage.png)

    What we want is:

    url(/assets/myimage-df02(more letters and digits).png)

    Of course, doing it manually like suggested here would be tedious. The solution is to rename our stylesheet to application.css.erb (or application.css.scss.erb) and type:

    url(<%= asset_path "myimage.png" %>)

    Don’t bother trying to write:

    url(<%= asset_path "images/myimage.png" %>)

    Even if the image is actually in app/assets/images because it won’t generate the right link. We did also moved our fonts to app/assets/fonts, it works no differently that for images. Reminder: if you forgot to specify the environment (RAILS_ENV=production) when running the assets:precompile tasks the links won’t include the necessary hash in filename.

    Comments

  3. Apr
    29

    Generate checksum of file with openssl

    Posted in : openssl, chef, and checksum

    Here’s how to generate checksums of file using OpenSSL:

    # SHA256, used in chef cookbooks
    openssl dgst -sha256 path/to/myfile
    
    # MD5
    openssl dgst -md5 path/to/myfile

    Comments

  4. Apr
    29

    Compile Psych support in Ruby 1.9.2p180

    Posted in : rvm, ruby, 1.9.2, homebrew, and yaml

    Ruby 1.9.2 comes with a new YAML parser: Psych from Aaron Patterson (aka Tenderlove). But it’s compiled only if you have installed libyaml in some of ruby’s mkmf default search location.

    Here’s how to make sure ruby will compile it:

    You have homebrew installed in /usr/local:

    brew install libyaml
    rvm install ruby-1.9.2-p180

    You have homebrew installed in ~/.homebrew:

    brew install libyaml
    rvm install ruby-1.9.2-p180 -C --with-libyaml-include=$HOME/.homebrew/include,--with-libyaml-lib=$HOME/.homebrew/lib

    I tried to use the --with-libyaml-dir=$HOME/.homebrew/ shortcut, but it don’t work. So don’t loose your time.

    Comments

  5. Sep
    26

    Install Mongrel2 on Mac

    Posted in : mac, mongrel2, server, homebrew, and zeromq

    Want to play with Zed Shaw’s Mongrel2 on your mac? Using Homebrew in your home and not in /usr/local ?

    Here’s how to get the ØMQ Homebrew’s install picked up by Mongrel2’s Makefile:

    # install zeromq via homebrew
    > brew install zeromq
    
    # download and install mongrel2
    wget http://mongrel2.org/static/downloads/mongrel2-1.2.tar.bz2
    tar xvjf mongrel2-1.2.tar.bz2
    cd mongrel2-1.2
    
    # to install in your home
    PREFIX=$HOME OPTFLAGS="-I`brew --prefix`/include -L`brew --prefix`/lib" make all install
    
    # to install in the default location (/usr/local)
    sudo OPTFLAGS="-I`brew --prefix`/include -L`brew --prefix`/lib" make all install
    

    Comments