Examples of crashes we observed:

/vendor/rails/actionpack/lib/action_controller/routing/route.rb:205):2:
   [BUG] Segmentation fault
/opt/ruby-enterprise-1.8.7-2011.03/lib/ruby/1.8/yaml.rb:133:
   [BUG] Segmentation fault
/vendor/rails/activesupport/lib/active_support/vendor/i18n-0.3.7/i18n/
   backend/base.rb:257: [BUG] Segmentation fault
/vendor/rails/actionpack/lib/action_view/template.rb:226: [BUG] Segmentation fault
/opt/ruby-enterprise-1.8.7-2011.03/lib/ruby/gems/1.8/gems/pauldix-sax-machine-0.0.14/
   lib/sax-machine/sax_document.rb:30: [BUG] Segmentation fault
/vendor/rails/activesupport/lib/active_support/memoizable.rb:32: [BUG] Segmentation fault

After banging my head against the wall for a week I found a solution (even two) and what might seem to be a likely reason for the segfaults. Two “suspects” – lack of available memory and incorrect version of libxml were ruled out. What seems to be the actual reason is the total size of the localisation files in config/locales loaded upon startup:

$ du -shb config/locales
1665858    config/locales

$ cd config/locales
$ find . -type f | wc -l
805

So ~1.6Mb in 805 files give occasional segfaults. Adding 200Kb of localisation files more started giving 100% segfaults on script/console startup.

Now I’ve found two workarounds for this problem.

1. Recompile REE with –no-tcmalloc flag

./ruby-enterprise-1.8.7-2011.03/installer --no-tcmalloc

Note that on 64-bit platforms tcmalloc is disabled by default.

2. Enable large pages feature in tcmalloc

This is described in tcmalloc documentation as: “Internally, tcmalloc divides its memory into “pages.”  The default page size is chosen to minimize memory use by reducing fragmentation. The cost is that keeping track of these pages can cost tcmalloc time. We’ve added a new, experimental flag to tcmalloc that enables a larger page size.  In general, this will increase the memory needs of applications using tcmalloc.  However, in many cases it will speed up the applications as well, particularly if they allocate and free a lot of memory.  We’ve seen average speedups of 3-5% on Google applications.”

There’s a warning – “this feature is still very experimental”, but it works to solve the problem with too many localisation files.

To compile REE with tcmalloc with large pages enables I just edited ruby-enterprise-1.8.7-2011.03/source/distro/google-perftools-1.7/src/common.h – replaced

#if defined(TCMALLOC_LARGE_PAGES)
static const size_t kPageShift  = 15;
static const size_t kNumClasses = 95;
static const size_t kMaxThreadCacheSize = 4 << 20;
#else
static const size_t kPageShift  = 12;
static const size_t kNumClasses = 61;
static const size_t kMaxThreadCacheSize = 2 << 20;
#endif

with

static const size_t kPageShift  = 15;
static const size_t kNumClasses = 95;
static const size_t kMaxThreadCacheSize = 4 << 20;

On production servers I opted for no tcmalloc for now – but I hope there’ll be a better way to deal with this issue soon.

Rails are providing developers with really excellent abstractions, but it’s always good to know what’s under the hood and how it all works.

There are a few things that might potentially cause bugs in your code, or waste your time (speaking from my own experience). So here goes:

1. Beware of globally keyed fragments

Let’s take example from Rails tutorial:

<% cache do %>
  All available products:
  <% Product.all.each do |p| %>
    <%= link_to p.name, product_url(p) %>
  <% end %>
<% end %>

Now if you need to deal with a multi-language site you might want to make cache fragment language dependent. What might seem a convenient solution:

<%- cache([user.locale.to_s]) do -%>

will turn into a source of very interesting problems. While calling the cache method without parameters will automatically create a controller/action specific cache key, calling it with a key will make this fragment a globally keyed fragment. Cache key in the first case is going to look like “views/localhost:3000/controller-name”, and in the other case “views/en” – this is not as unique identifier any more.

While automatic cache key naming provided by rails is very convenient, it is very easy to run into a problem with duplicate cache key names used in different places.

2. Another pitfall of automatic cache key naming is that you shall never assume that when creating a cache with global key you can later find it using e.g. telnet interface to memcache. Example – add

<%- cache('unique_cache_key') do -%>
<%- end -%>

in your view and then try to read directly from memcache:

$ telnet localhost 11211
GET unique_cache_key
END

At the same time

GET views/unique_cache_key

will work. It’s easy to make this mistake trying to check or delete cache keys directly from memcache when using Rails cache methods.

3. delete_matched is not supported by memcached (see rails/activesupport/lib/active_support/cache/mem_cache_store.rb)

In practice that means that if you’re using memcached as Rails cache engine and trying to delete or expire fragment cache using standard Rails methods and regexp – you’ll fail.

expire_fragment(/base\/xyz.*/)

will fail miserably. Ideal solution is not to use explicit cache expiration, but rather create cache keys in such a way that doesn’t require expiration. Alternatively it’s possible to use extensions implementing delete_matched for memcached (haven’t tried it myself though).

Tip: one very useful tool for checking memcached is peep by Evan Weaver – allows you to peek into the cache and see what’s really cached and how it is used.

I’d like to say big thanks to the organisers of the conference (especially CJ @cjkihlbom) – everything went really smooth, even though there’s been 150 people attending this year compared to 90 last year.
Some points that I’d really like to highlight are:

• a lot of time to meet people and discuss: 30 minutes talks followed by 30 minutes breaks, no q&a – those who had questions had an opportunity to talk to the speakers during the breaks
• venue was great (of course, the boat – there was enough space for everyone to move around, but at the same time it was compact enough not to get lost also everyone had an opportunity to have lunch and dinner together
• “job board” a huge white board where anyone can post information about open positions in their companies – it got filled withing firts few hours – job market is really hot
• lightning talks that any participant can give – 5 minute talks in the end of the day – it was really great
• real coffee espresso, latte, cappuccino, americano – you name it – professional baristas were at your service
• 5K Nordic Ruby run organised on the second day’s morning

Day 0

The conference started with a party on Göthenborg ship. It was a great way to get to know the speakers and participants in an informal setting.

Day 1

“GitHub Flavored Ruby” was by Tom Preston-Werner (@mojombo) from GitHub.

Tom had a hard task of waking up the auidence after yesterday’s pre-conference party, and he succeeded in this task. Tom explained what is a README driven development that he and some of his colleagues in GitHub are using – a golden mean between waterfall and cowboy coding.
Start a new project with writing a README file, where basic design principles and conventions will be described, and don’t forget to update it as the architecture changes.

Another interesting topic was TomDoc – code level documentation – a better alternative than RDoc/Yard, designed for hackers/humans, not for machines to read.

Modularisation was yet another subject – Tom explained how at GitHub they try to split the code into independent modules to reduce complexity of the system.
Grit is a good example of that – a module for Ruby-Git interaction was written even before the rails code for GitHub.

Other examples of modles created in GitHub (and open-sourced) are: resque, jekyll, proxymachine and chimney, albino, gollum, bart, camo, failbot, giraffe, nodeload (in node.js), ernie (in erlang), stratocaster (for event feed), ghromosome (which is apparently not used anymore, but worth mentioning just because of the fun name).

And naturally for versioning of the components Tom proposes to use Semantic versioning.

A few notes from the after-talk discussion with Tom and others:
- Github uses git powered deploys – instead of copying the new release from git and then changing symlink to current release, they use single directory and do direct git updates – rollbacks are also easy to do using git fucntionality
- it’s possible to deploy Github from Campfire by giving deploy command to a robot
- to avoid downtime – build_cache is executed on a local machine before deploy and pushed to the servers during deploy
- GitHub is still running on Rails 2.3, Tom’s estimate for migration to Rails 3 is ~2 weeks of “stop commits and migrate” (the estimate seems to be pretty consistent – I’ve been talking to several people about it)
- experience from a fairly large web-app developers: MySQL to Postgres migration took 3 weeks

“API Design Matters” was by Anthony Eden (@aeden) from DNSimple

Central points from Anthony’s presentation:
- APIs outlive their implementation (“Think about the children” when designing APIs
- Design APIs so that they are easy to extend
- 5 Cs: consistent, clear, convenient, concise and complete
- The best API is no API
- If you’re a software developer – you are and API designer
- And use semantic versioning that Tom was talking about

Reading recommendations:
- Jasmine Blanchette “The little manual of API design”
- Joshua Bloch “How to design a good API and why it matters”

“Bridging the gap – Using JavaScript in Rails to write DRY rich client applications” by Thorben Schröder (@walski), KOPFMaschine

An interesting talk about using Java Script from Ruby code and the other way around.
Implementing validations on the client side reduces amount of round-trips and speeds up user interaction, but since smart people can hack JavaScript and bypass client-side validations, also server side validations are necessary. To avoid writing validations twice use The Ruby Racer,  CommonJS and CommonJS models to DRY your code and write validations only once and use them from both Ruby and JS code. Pretty smart, huh?

“Limited Red Society” by Joseph Wilk (@josephwilk)

Slides: http://www.slideshare.net/josephwilk/the-limited-red-society-8367200

Kanban is limiting number of tasks the team has in progress at any given moment. Joseph described in his talk how to apply this methodology to testing.

It’s important to have metrics for your testing and it’s important to visualize them. How much time do you spend in “red” when your tests are failing, which modules in your code are changed most often, are they the same that usually fail tests, are there any oscillating test cases that pass then fail then pass again?

One thing to be noted – some metrics have limited lifespan – measure what you’re interested in and then throw them away.

Reading recommendations:
“Refactoring in Ruby”
“Refactoring. Ruby Edition” Kent Beck

“Must It Always Be About Sex?” by Joshua Wehner (@jaw6)

Links from the talk: http://pinboard.in/u:jaw6/t:nr2011/

Minorities and women are under-represented in our community. Black man is more likely to be hit by lightning than become a computer science professor.
75% of women engineers leave profession because they are not socially accepted in the working culture. In the beginning of computer era programming was mostly a job for women.
Do you think that your application is mobile friendly if you have iPhone optimised version? Think again – the most popular phone with a web browser is Nokia 1100 – more than 250 million phones sold to date,
and that kind of devices are the only way how most people in Africa or India access the Internet.

What do we need to do to change current situation?

Just today I came across an article on the same subject: Two 11-year-old entrepreneurs learned the hard way what it’s like to be a minority in tech during Startup Weekend

“Infinite data – finite solutions” by Randall Thomas (@daksis) from Engine Yard

Quick dive into Bayesian statistics. That is the technology that helps to filter out spam from your inbox: “Keeps spam at Bayes”

Bayes + DAG (Directed Acyclic Graph) = BBN (Bayesian Belief Network)

Probably the most scientific talk of the conference and two great demos: reconstruction of Bach’s first invention using BBN (amazing stuff) and analysing whiskey properties

Book to read: “Robust Nonparametric Statistical Methods”
Site to check: Eliezer Yudkowsky
Tool to check: Weka – analysis tool – live example how to analyze properites of whiskey.

A great collection of links on Bayesian Networks:  https://gist.github.com/1031597

“Taking Back Education” by Joe O’Brien (@objo)

Education. We are in the top 5 to 2% of the developers by just being here – attending the conference.
Why education system sucks? There’s a clear need for apprenticeship institute.

Harward Business Review research indicated that team IQ depends on diversity, not on IQ of individual members.
Adding a women to the team cnsistently improves team performance (good reference for Diversity problems talk by Joshua)

In the end of the first day there was a series of “lightning talks”.

* Anthony Eden (@aeden) proposed to use DNS protocol for resolving dependencies. Definitely idea worth studying.
* Twitter will be native on all iOS devices – think about it, and what do you want to do with it.
* Stephen Sykes – obfuscated code for fun and ruby quiz – really, really fun – see the slides here: http://www.stephensykes.com/obfu_lightening.pdf
* Emily Bach (backconsulting.com) shared her experiences in teaching TDD
* Theo from Burt talked about his experiences and lessons learned from scaling:
- build two of everything from the start – will help with horizontal scaling later
- fail fast (Göthenburg ship made it to China and back to Sweden and sank a hundred meters from the home harbour, Vaasa is a lot better example – it sank right after it was deployed
- queues
* Tom from GitHub talked about Hubot – a Campfire robot they wrote in node.js and keep developing just for fun – absolutely amazing stuff, every team should do that
* David from Streamio (one of the sponsors of Nordic Ruby) did a “shameless self-promotion” of his company
* Alex who has been doing a great job filming the conference talked about making a movie – what it takes
* Antony Sastre – talked about photography – really inspiring, Antony was a web developer, then left the field and was doing photography for living for two years, now came back to web development
* Nikolai (@nikolayb) Is this a bad idea? Deploying always to production on a site with millions of page views per day, one branch always – test code on a live service? Somehow it works…
* Paul Wilson (@paulanthonywils) talked about Agile Delivery Network – doing better government IT, and here’s a bunch of links from his talk: https://gist.github.com/1035392
* Paul Campbell (@paulk) talked about whenever.js and made it public on GitHub in the end of his talk http://whatever-js.heroku.com/#1, https://github.com/paulca/whenever.js

Day one ended with a BBQ dinner at the River Cafe.

Day 2

Paul Campbell who ended Day 1 with his lightning talk also started Day 2 with a autobiographical talk “In search of me fein” (myself in Irish).

7 days in my life, zombie movies and finding inspiration
Technologist, developers, live in a bubble – we have no problems finding jobs, we can travel, we can choose where we live and work. It’s not like that outsie of our industry.

“Actors on stage” by Elise Huard (@elise_huard)

Slides: http://www.slideshare.net/ehuard/ruby-hollywood-nordic

We have multiple cores in our processors now – how do we benefit from it?
MRI is the only Ruby where parallel threads are executed time-scliced in one real thread.

Cellucloid – a concurrent object framework for Ruby.

Languages that implement actors: Erlang, Scala (Actors are in standard library), Closure (Actors called Agents)

Mats hates threads
Should there still be a better cuncurrency primitives in MRI? shall we create a branch of Ruby to implement it?

“Beyond Ruby” by Jakob Mattson from Burt (@jakobmattsson)

Slides: http://jakobmattsson.tumblr.com/post/6664704401/beyond-ruby

Bold statement: Ruby is not a dynamic language. Not everyting in Ruby is an object, despite what many books claim.
E.g. dot (.) or “or” (||) operators cannot be redefined. Or operator doesn’t evaluate second argument if the first one is true.
Ruby is not as moldable as it could be.
It would be cool to implement calling by name – passing expression along with the context to the callee and letting it decide whether to evaluate it or not.
Io language is probably 80% there. A new language needs to be invented. And coding it would not be that difficult – because if it is
truly dynamic, then most of it can be implemented in itself.

“Fewer constraints” by Ryan Smith (@ryandotsmith) from Heroku

Slides: http://dl.dropbox.com/u/1579953/concurrency_deck_with_notes.pdf

Continuing the theme that Elise started this morning. How to improve performance of your application by parallelizing.
Amdahl’s law – adding processors gives only marginal speed-up. “Even though we have 10 workers
available to us, we are only seeing a speedup factor of 5.2, with 20 workers it get only to 6.9″.

Real life example – improving queue performance. Using Fuzzy FIFO instead of traditional queues FIFO implementation.
Example with using spinlock in PostgreSQL to implement it. See queue_classic on github.

“Mountain Dew and My Trail of Tears” by Aaron Patterson (@tenderlove) from AT&T Interactive

Aaron discussed in depth how to work with legacy code.

- use “ruby -w” to see the warnings in your code, warnings can be used by developers to inform about upcoming deprecations
- Liskov substitution principle (LSP)
- Single responsibility principple
- Method extraction
- Object#extend
- Code seams
- sleep sort: http://dis.4chan.org/read/prog/1295544154

Recommended reading: Working effectibely with legacy code

“Legacy” by Chad Fowler (@chadfowler)

Chad started with playing a recording of Beethoven – this is legacy. Miro is legacy. Gaudi is legacy too, and the only reason to visit Barcelona.

Legacy is good.

Nobody will remember your work when you die. There are still systems in use written 25 years ago – why don’t we write that kind of code anymore?
Average lifespan of a business software is 5 years (figure is probably made up)

Recommended reading:

Michael Feathers “Your Code.. it’s Alive..”
Richard P. Gabriel “Design Beyond Human Abilities”

What kind of software lasts – either small (cell-size) or systems consisting of cell-sized software (like UNIX).

Force heterogenity – languages, OSes, frameworks – will change over time. If it’s hard to force heterogenity – do it all the time.

“Must.Try.Harder” by Keavy McMinn (@keavy)

An absolutely fascinating and inspirational talk from a woman who decided to train for an IronMan from pretty much zero.
She did it. A rare inspirational example of how to go outside your comfort zone and achieve a goal that looked impossible.

Other conference notes:

Fredrik Rubensson: http://www.highlevelbits.com/2011/06/nordic-ruby.html
Andreas Ronge / Jayway:  http://blog.jayway.com/2011/06/19/notes-and-thoughts-from-a-great-conference/

1. Rails.vim by Tim Pope

To install just copy autoload/rails.vim, plugin/rails.vim, and doc/rails.txt to corresponding directories in ~/.vim

Vim scripts section has a full description of the functionality. Some highlights:

• Easy navigation between models, controllers and views with :Rmodel, :Rview, :Rcontroller commands
• Syntax highlighting
• CTRL-X CTRL-U for autocompletion
• :Rtree for project tree (see item 2)

2. NERD Tree by Marty Grenfell

Another must-have. Provides you with an easy way to navigate your project tree. Rails.vim nicely integrates with this one.

3. Colour schemas

If you want to save your eyes – use dark background when developing. Personally I prefer one of the standard vim themes – Desert , but Ocean Deep is also very good.

Copy theme file to ~/.vim/colors and then use

:colorscheme oceandeep

command to apply it.

But let’s start from the beginning. I should say right away that we at HeiaHeia are using Ruby Enterprise Edition, so I didn’t have to apply patches to Ruby source code that Evan is talking about in his post. Before starting to analyse GC current usage it will be useful to read a brilliant overview of Ruby garbage collection by Joe Damato. It’ll help to understand what’s going to happen next. It is also useful to read REE documentation on garbage collector performance tuning.

And as before any optimisation it is good to get the reference metrics, so that you know you actually improved something by changing settings, and didn’t make it worse. In this case I measured:

To measure number of garbage collector calls per one page I used Scrap – a nice tool by Chris Heald. Chris also describes the tuning process in great detail, so I’m not going to repeat it – just go and read his blog.

To measure local test execution time I just ran rake test 5 times and took the average of all runs.

To measure application server CPU load and average response time I used NewRelic tool (free version should be enough to do the measurement).

When I first loaded feed page with Scrap enabled I saw 36 GC cycles, and Ruby spend 1.12s in GC cycles (these figures are from the development server, so response time is big). After playing a bit with the settings and monitoring GC cycles number and unused heap after each allocation with Scrap, I ended up with the same settings as Twitter uses in production:

export RUBY_HEAP_MIN_SLOTS=500000
export RUBY_HEAP_SLOTS_INCREMENT=250000
export RUBY_HEAP_SLOTS_GROWTH_FACTOR=1
export RUBY_GC_MALLOC_LIMIT=50000000

These settings reduced GC cycles number down to 7 (from 36), and Ruby spent now only 0.62s in GC instead of 1.12s when loading feed page (again, load times are bigger on our development server).

After introducing same settings on my local machine project tests took only 148s – down from 274 seconds before optimisation – a whopping 46% improvement.

We have multiple identical application servers, so I introduced the new settings only on one of the application servers, to compare the results on a live system (during low traffic hours). Here’s the picture from NewRelic:

Server        Instances        Apdex          Resp.time    Throughput    CPU          Memory
hh-app1     3 Instances         0.86      	407 ms       30 rpm      14 %	      349 MB
hh-app2     3 Instances         0.91   	        311 ms	     30 rpm       8 %	      413 MB

hh-app2 had optimised garbage collector. With the same throughput CPU load  was only 8% vs 14% with non-optimised GC. However that improvement came at a cost of increased memory consumption (413M vs 349M). However response time and lower CPU load proved to be a lot more important than memory consumption, so I rolled out the new settings on all production servers.

Making nginx utilise GC settings when spawning Passenger instances is easy, and is well described by Chris. Here are instructions that work nice with server and Nginx setup on Ubuntu as I described in earlier posts.

Create /usr/local/bin/ruby-with-env file (as a root) that will set GC settings in the environment and then launch Ruby:

#!/bin/bash
export RUBY_HEAP_MIN_SLOTS=1500000
export RUBY_HEAP_SLOTS_INCREMENT=500000
export RUBY_HEAP_SLOTS_GROWTH_FACTOR=1
export RUBY_GC_MALLOC_LIMIT=50000000
exec "/usr/local/bin/ruby" "$@"

sudo chmod a+x /usr/local/bin/ruby-with-env

passenger_ruby /usr/local/bin/ruby;

passenger_ruby /usr/local/bin/ruby-with-env;

Even though there’s Ubuntu nginx package available (which works perfectly when you’re running PHP apps using FCGI), if you want to take into use Phusion Passenger, you’ll need to recompile Nginx from sources.

Instructions below were verified on Ubuntu 10.04 (Lucid Lynx) Server Edition.

If you already have Nginx ubuntu package installed, uninstall it (NOTE! purge will delete all configuration files – so if you changed them – make a backup for future reference):

sudo aptitude purge nginx

Assuming you already have Rails stack installed, install Passenger gem:

sudo gem install passenger

At the time of this writing the latest version of Passenger is 2.2.11

Let’s check dependencies that Ubuntu nginx package has and install them before compilation:

aptitude show nginx | grep Depends

Depends: libc6 (>= 2.4), libpcre3 (>= 7.7), libssl0.9.8 (>= 0.9.8k-1), zlib1g

sudo apt-get install libc6 libpcre3 libssl0.9.8 zlib1g
sudo /usr/local/bin/passenger-install-nginx-module

sudo /usr/local/bin/passenger-make-enterprisey

sudo vim /etc/init.d/nginx

#! /bin/sh

### BEGIN INIT INFO
# Provides:          nginx
# Required-Start:    $local_fs $remote_fs $network $syslog
# Required-Stop:     $local_fs $remote_fs $network $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: starts the nginx web server
# Description:       starts nginx using start-stop-daemon
### END INIT INFO

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/nginx/sbin
DAEMON=/opt/nginx/sbin/nginx
NAME=nginx
DESC=nginx

test -x $DAEMON || exit 0

# Include nginx defaults if available
if [ -f /etc/default/nginx ] ; then
        . /etc/default/nginx
fi

set -e

. /lib/lsb/init-functions

test_nginx_config() {
  if nginx -t $DAEMON_OPTS
  then
    return 0
  else
    return $?
  fi
}

case "$1" in
  start)
        echo -n "Starting $DESC: "
        test_nginx_config
        start-stop-daemon --start --quiet --pidfile /var/run/$NAME.pid \
                --exec $DAEMON -- $DAEMON_OPTS || true
        echo "$NAME."
        ;;
  stop)
        echo -n "Stopping $DESC: "
        start-stop-daemon --stop --quiet --pidfile /var/run/$NAME.pid \
                --exec $DAEMON || true
        echo "$NAME."
        ;;
  restart|force-reload)
        echo -n "Restarting $DESC: "
        start-stop-daemon --stop --quiet --pidfile \
                /var/run/$NAME.pid --exec $DAEMON || true
        sleep 1
        test_nginx_config
        start-stop-daemon --start --quiet --pidfile \
                /var/run/$NAME.pid --exec $DAEMON -- $DAEMON_OPTS || true
        echo "$NAME."
        ;;
  reload)
        echo -n "Reloading $DESC configuration: "
        test_nginx_config
        start-stop-daemon --stop --signal HUP --quiet --pidfile /var/run/$NAME.pid \
            --exec $DAEMON || true
        echo "$NAME."
        ;;
  configtest)
        echo -n "Testing $DESC configuration: "
        if test_nginx_config
        then
          echo "$NAME."
        else
          exit $?
        fi
        ;;
  status)
        status_of_proc -p /var/run/$NAME.pid "$DAEMON" nginx && exit 0 || exit $?
        ;;
  *)
        echo "Usage: $NAME {start|stop|restart|reload|force-reload|status|configtest}" >&2
        exit 1
        ;;
esac

exit 0

sudo chmod +x /etc/init.d/nginx
sudo /usr/sbin/update-rc.d -f nginx defaults

user www-data;
worker_processes  4;

error_log  /opt/nginx/logs/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  8192;
    use epoll;
}

http {
    passenger_root /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.11;
    passenger_ruby /usr/local/bin/ruby;

    include       /opt/nginx/conf/mime.types;

    # set a default type for the rare situation that
    # nothing matches from the mimie-type include
    default_type application/octet-stream;

    # This log format is compatible with any tool like awstats
    # that can parse standard apache logs.
    log_format main '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"' ;

    access_log  /opt/nginx/logs/access.log;

    sendfile        on;
    tcp_nopush     on;

    keepalive_timeout  0;
    tcp_nodelay        on;

    gzip  on;
    gzip_disable "MSIE [1-6]\.(?!.*SV1)";
    gzip_http_version 1.0;
    gzip_comp_level 2;
    gzip_proxied any;
    gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;

    include /opt/nginx/conf/sites-enabled/*;
}

sudo mkdir /opt/nginx/conf/sites-available
sudo mkdir /opt/nginx/conf/sites-enabled

sudo vim /opt/nginx/conf/sites-available/mysite.com

server {
        listen   80;
        server_name  www.mysite.com;

        access_log  /home/user/logs/www.mysite.com/access.log;
        root   /home/user/www.mysite.com/public;

        # serve static content directly
        location ~* \.(ico|jpg|gif|png|css|js|swf|html)$ {
          if (-f $request_filename) {
            expires max;
            break;
          }
        }

        passenger_enabled on;

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        location ~ /\.ht {
          deny  all;
        }
}

sudo ln -s /opt/nginx/conf/sites-available/mysite.com /opt/nginx/conf/sites-enabled/mysite.com
sudo /etc/init.d/nginx restart

Conference has drawn very interesting speakers and  international crowd – from Finland (naturally), Sweden, Poland, Germany, Russia and other countries. I made a few notes from selected talks on the conference.

Chris Wanstrath / GitHub (@defunkt)

• Slides: http://www.slideshare.net/err/inside-github
• Linus Trovalds Google tech talk about git – where Linus tell you that you’re stpid if you’re not using git
• rack-mobile-detect – is used by GitHub, super useful if you’re planning to create mobile optimized version
• GitHub uses Unicorn as an application server – personally I’m not sure if that’s better than Apache + Passenger. Chris tells that Unicorn is cool, because does fair load balancing on Linux kernel level, also Rails are loaded only once – and then required number of processes are forked – and this is very fast, a lot faster than loading rails separately for each Mongrel. And when one of the processes dies – there’s no need to re-load Rails, but just fork another process.
• GitHub users BERT to forward requests to one of their six servers – BERT to Erlang is the same as JSON to JavaScript
• GitHub doesn’t use delayed_job anymore since they needed several queues with different priorities – so far they use resque, but are considering developing a real queue management system

Jose Valim / Plataforma (@josevalim)

• Brazilian Rails consultants and developers: http://plataformatec.com.br/
• Developed several interesting components to make their life easier – extracted common components from several customer projects and open-sourced them
• User authorisation and roles: Devise – http://blog.plataformatec.com.br/2010/04/authentication-is-with-devise/
• Nice forms with labels, error handling, etc. : Simple Form – http://github.com/plataformatec/simple_form

Mike Dirolf / MongoDB (@mdirolf) & Jonathan Weiss / CouchDB (@jweiss)

• Two very interesting NoSQL presentations: http://www.slideshare.net/mdirolf/mongodb-at-frozenrails, http://www.slideshare.net/jweiss/couchdb-on-rails-frozenrails-2010
• Answering a question from the audience Mike said that fault tolerance is something that is not quite there yet in MongoDB – there is a possibility to do recovery, but there’s a chance data might be lost
• There is no such issue in CouchDB, it also is written in Erland and provides REST API
• Links to project homepages: http://couchdb.apache.org/ & http://www.mongodb.org/ – worth taking a look at both

Yehuda Katz / EngineYard (@wycats)

• Very interesting presentation about developing web application for mobile devices (in Rails 3)
• One of the central messages – HTML5 is coming – learn how to use it now, if you want to stay in business
• I really hope Yehuda will publish his slides – it was a presentation worth spreading around
• interesting thoughts:
  • same optimization methods as we traditionally use for Rails apps on desktop are not suitable for mobile browsers due to constraints – connection availability, memory, cpu, battery, etc.
  • don’t count on browser cach on mobile devices – separate data and presentation, cache presentation using localStorage feature of HTML5, load only data from the web after that
  • mobile browsers are updated a lot faster than desktop borwser – there are new browsers in the new phones, and nobody uses 2 year old phone
  • HTML5 will become mainstream on mobile devices a lot faster than on desktop (IE6 will not happen on mobile)
  • when developing apps for mobile take into account that in some places people still pay per kilobyte of traffic
  • incremental rendering on the mobile is evil – it consumes battery and can be very slow, unpleasant and unusable for the user
• battery – unoptimized sites can easily drain the battery, and even though users are most likely to blame the phone manufacturer it is good to think about it
• http://Railsdispatch.com – recommended reading about Rails 3

Carl Lerche / Engine Yard (@carllerche)

• Carl told about Rails 3 and migration to version 3 from earlier ones http://www.slideshare.net/carllerche/frozen-rails-slides
• Migration itself is not that difficult (provided that you know what you’re doing) – it can even be done in 15 minutes, but thus far stability and performance of Rails 3 / Ruby 1.9.1 are not quite there yet
• Rails 3 is not finished – the plan is to make RC for RailsConf, and the it will be released “when it’s ready”
• When asked if they have done performance tests on Rails 3, Carl answered that, yes, they did, and its sad.  Performance optimization is the next big step after stabilization.
• When asked which Ruby release is better to use with Rails 3, Carl answered – 1.8.7, since Ruby 1.9.1 still segfaults and is not ready for production sites.

JetBrains (@rubymine):

• new release of RubyMine is available with Rails3 support
• guys from JetBrains tell that  TextMate development/maintenance is lagging behind, and they observe migration of TextMate and even vim users to RubyMine (there’s now vim interface emultion)

overall impression:

• Apple is the king – virtually everybody was walking around with either MacBook, iPhone, iPad or all of the above
• first Rails conference in Finland was a huge success – I really hope there will be FrozenRails 2011

This proved to be a non-trivial task. There is a number of tutorials on the net, but none of them described the full solution. So after getting it all to work, I decided to share all the tips and tricks. Feel free to comment, if you will find problems with the following set of instructions.
SSH to redmine.mydomain.com as a user that will be running Redmine (in the following examples it will be ‘redmine_user’).
First, you need to compile openssl – it will be required for curl, git and redmine.

mkdir ~/tmp
cd ~/tmp
wget http://www.openssl.org/source/openssl-0.9.8k.tar.gz
tar xzvf openssl-0.9.8k.tar.gz
cd openssl-0.9.8k
./config shared zlib --prefix=$HOME/.packages
make
make install

Let’s tell the world that we keep binaries and libraries also in the local directory. Edit ~/.bashrc (it is used by all non-login shells):

export TZ='Europe/Helsinki'
export PATH="$HOME/.packages/bin:$PATH"
export LD_LIBRARY_PATH="$HOME/.packages/lib"
export GEM_HOME="$HOME/.gems"
export GEM_PATH="$GEM_HOME:/usr/lib/ruby/gems/1.8"
export PATH="$HOME/.packages/bin:$HOME/.gems/bin:$PATH"
export RUBYLIB="$HOME/.packages/lib:$RUBYLIB"
export LD_LIBRARY_PATH="$HOME/.packages/lib"

# this ensures our gem install processes don't get killed by the DreamHost police
alias gem="nice -n19 ~/.packages/bin/gem"

(You can skip TZ – it is just usefule to have correct time set for your environment. Use tzselect to find out correct TZ string for your region)

And edit ~/.bash_profile (it is used by login shells):

umask 002
PS1='[\h:$PWD]$ '
alias ll="ls -l"
EDITOR="/usr/bin/vim"
. .bashrc

Now let’s apply the changes in active shell:

cd ~ : . .bash_profile

Then you need to compile curl, to be able to compile git with curl and execute clone commands on your server.

cd ~/tmp
wget http://curl.haxx.se/download/curl-7.19.5.tar.gz
tar xzvf curl-7.19.5.tar.gz
cd curl-7.19.5
./configure --prefix=$HOME/.packages --with-ssl=$HOME/.packages
make
make install

Now get and compile Git. If you are not using Dreamhost PS, you might want to compile it with NO_MMPAP=1, to reduce
probability of git process getting killed by Dreamhost police robots due to extensive memory

cd ~/tmp
wget http://www.kernel.org/pub/software/scm/git/git-1.6.4.tar.gz
tar xzvf git-1.6.4.tar.gz
cd git-1.6.4
./configure --prefix=$HOME/.packages --with-curl=$HOME/.packages
make
make install

Let’s start with ruby and rails related stuff. First readline library is needed, for script/console to work.

cd ~/tmp
wget http://ftp.gnu.org/gnu/readline/readline-5.2.tar.gz
tar xzvf readline-5.2.tar.gz
cd readline-5.2
./configure --prefix=$HOME/.packages
make
make install

Install ruby:

cd ~/tmp
wget ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.7-p72.tar.gz
tar zxvf ruby-1.8.7-p72.tar.gz
cd ruby-1.8.7-p72
./configure --prefix=$HOME/.packages --with-openssl-dir=$HOME/.packages --with-readline-dir=$HOME/.packages
make
make install

Install rubygems:

cd ~/tmp
wget http://rubyforge.org/frs/download.php/60718/rubygems-1.3.5.tgz
tar zxvf rubygems-1.3.5.tgz
cd rubygems-1.3.5
ruby setup.rb config --prefix=$HOME/.packages
ruby setup.rb setup
ruby setup.rb install

Now you can install all required gems and freeze them if necessary.

Installing RedMine

Refer to http://wiki.dreamhost.com/Redmine for details.

But there’s one trick – if you want RedMine to use just compiled version of Git – edit in lib/redmine/scm/adapters/git_adapter.rb:

# Git executable name
GIT_BIN = "/home/username/.packages/bin/git"

If this is not done, default Dreamhost git will be used (which is too old at the moment 1.4.4.4) and Git repository browsing will not work in RedMine
(see http://groups.google.com/group/phusion-passenger/browse_thread/thread/5080d7c7cfbcf20e).

Setting up Git repository

Refer to http://wiki.dreamhost.com/Git for details.

Couple of tricks here. There are bugs in WebDAV functionality in Ubuntu – so if you’re using it – launch Nautilus, use “File->Connect to server…” menu from there (not from the system menu), don’t enter user name in the dialog – leave it empty and enter it when you’re requested username and password in the next dialog.

When setting up WebDAV access rights, give access to user “redmine”.

Setting Git repository copy for Redmine

Create ~/.netrc for your redmine user on Dreamhost

Insert the following line in that file

machine git.mydomain.com login redmine password [redmine_password]

where redmine_password is the password you gave to user redmine in the previous step.
.netrc will ensure that password is not asked when git is accessing the repository.
You don’t need to bother about creating .netrc if you have a public repository (or at least available for cloning without password).

Create a local copy of the repository

For RedMine to be able to display Git repository, it needs to have a local clone of the repository.

mkdir ~/git_project_clones
cd git_project_clones
git clone http://git.mydomain.com/repository_name

Now you should have a local copy that can be used from RedMine.
All you have left to do is to set up regular pulls from the master repository to this local copy. Use command

crontab -e

Add the following line to pull latest change into local copy every 5 minutes:

*/5 * * * * cd /home/redmine_user/git_project_clones/repository_name && /home/redmine_user/.packages/bin/git pull

Make sure that you specify full path to git – otherwise it will execute Dreamhost default git 1.4.4.4 and command will fail with “refusing to create funny ref ‘remotes/origin/*’ locally” error.

Save and exit cron editor.

(You might also want to check Redmine own wiki: http://www.redmine.org/wiki/redmine/HowTo_keep_in_sync_your_git_repository_for_redmine)

Set repository in the settings of your project in RedMine: /home/redmine_user/git_project_clones/repository_name/.git

Now you need to manually update Git repository changesets in RedMine:

cd ~/mydomain.com
script/runner "Repository.fetch_changesets" -e production

And set a hook to your repository to do this every time repository is updated – edit ~/git_project_clones/repository_name/.git/hooks/post-update file and add the following command ther:

cd /home/redmine_user/mydomain.com && script/runner "Repository.fetch_changesets" -e production

When writing this blogpost I found a lot of useful information on Dreamhost own wiki as well on these blog posts:
http://www.wavethenavel.com/2008/09/08/bootstrapping-a-dreamhost-account-for-rails-and-git/
http://juliobiason.net/2008/05/19/git-repositories-on-dreamhost-via-ssh/
http://www.simonecarletti.com/blog/2009/07/configuring-git-repository-with-redmine/

(To be fare I should say that Dreamhost provides excellent value for money. If you are looking for a good hosting – use IVANKUZNETSOV promocode and get a $50 discount when setting up an account on Dreamhost)

A thought of reinstalling all modules and reconfiguring Joomla from scratch was simply too depressive, so I decided to try another CMS. As a Ruby on Rails convert and a strong believer in open-source ideology I decided to go for Radiant – open-source CMS written in RoR. It is still in beta (latest release is 0.6.4), but it is surprisingly stable and powerful. Take a look at the footer of www.ruby-lang.org – official Ruby programming language web site – it is powered by Radiant

Installation of Radiant was rather easy – thanks to this guide and my prior experience with RoR applications deployment on Dreamhost. It took me a couple of hours to figure out how to actually create sites with Radiant – there are not that many tutorials available yet, so it is pretty much  “make by example”. Split into pages, snippets and layouts makes a lot of sense onse you get your head around it.

From my experience Joomla is an overkill for most of the small sites, and despite being WYSIWYG, it still requires a professional or at least a tech savvy to configure it. After Radiant is set up and configured it is no more difficult to add content there than to edit a wiki page because of its Textile support. But it is so much simpler and easier to use than Joomla.

I managed to restore the ruined site in a day’s time – fetched most of the lost content from Google cache, converted Joomla template into Radiant’s layouts and recreated the pages (well, it was a small site after all). First time I dealt with Joomla – I spent several days trying to figure out where are the settings that I actually need in the endless menus.

Radiant is clearly following “less is better” principle. If you want to try Radiant – there’s a live demo where you can do whatever you want with the content.