what_i_am.upto(2010)

After staying almost the whole last week in the hospital because all of a sudden my appendix decided to hurt and part ways with me, I finally managed to finish my article I announced earlier on installing a whole Rails stack with Apache2, MySQL, Ruby, Rubygems, Phusion Passenger from a single Debian package file using the DebGem service and the Brightbox Passenger Debian packages.

This article won’t cover many basics of using the DebGem service. If you need an introduction, you might want to check out my earlier postings on the topic.

Preparing the VM

Just like in my second post on DebGem, I used a vanilla Ubuntu JeOS 8.04.1 LTS install in Virtualbox. Apart from the openssh-server package for easier access to the machine and an initial sudo apt-get update && sudo apt-get upgrade, the machine was untouched.

To start out, I installed the most basic required cli tools with

sudo apt-get install bash-completion nano wget curl

since JeOS is so much “Just enough OS” it does not even supply these very common packages. After this, the machine is ready to install the stack.

Adding the DebGem and Brightbox repositories

The additional debian repositories needed to be added to the apt package sources now in order for the Gems and Passenger to be installed properly:

# Switch to root shell
sudo -s 

# Add debgem repository
sh -c 'echo "deb http://www.debgem.com/apt ubuntu-8.04 rubyforge" > /etc/apt/sources.list.d/debgem.list'
sh -c 'wget -q http://www.debgem.com/apt/debgem.gpg -O- | apt-key add -'
sh -c 'echo "APT::Cache-Limit 33554432;" > /etc/apt/apt.conf'

# Add brightbox passenger repository
sh -c 'echo "deb http://apt.brightbox.net hardy main" > /etc/apt/sources.list.d/brightbox.list'
sh -c 'wget -q -O - http://apt.brightbox.net/release.asc | apt-key add -'

# Update repository index
apt-get update

Installing the stack from the deb file

I uploaded the sample .deb file and it’s sources to github. The tarball of the project can be retrieved and extracted with:

wget -nv http://github.com/colszowka/railsstack.deb/tarball/master -O- | tar xzv -C ~/

Now just switch over to the extracted folder’s subdirectory ‘deb’:

  cd colszowka-railsstack.deb-8bfcf1812c76766c095eb5a1510f2e84742b703a/deb/

The installation of the whole stack can now be invoked with

dpkg -i --force-depends railsstack.deb

Unfortunately, the dpkg command is unable to resolve remote dependencies, so the output will print a multitude of unresolved ones. But worry not, these will get automatically resolved by apt-get when running the following command:

apt-get -f install

The whole install will kick off now, installing as much as ruby, ri, rdoc, irb, rubygems, apache2, mysql-server-5.0, mysql-client-5.0, rails2.2.2-ruby1.8, libfastthread-ruby1.8, libapache2-mod-passenger and libmysql-ruby completely automatically. You will get asked for your desired MySQL root user password once, but that’s about it. After a couple of minutes, everything should be ready for the big test. Do not forget though to exit the root shell when installing Typo!

Test run with Typo 5.1.98

To check if Apache2/Passenger, MySQL, Ruby, Rails and everything else worked fine, I gave it a spin with the Typo Rails open source blog app:

cd ~
wget http://rubyforge.org/frs/download.php/49562/typo-5.1.98.tgz
tar xfz typo-5.1.98.tgz
cd typo-5.1.98
cp config/database.yml.example config/database.yml
nano config/database.yml # Enter db credentials...

As a next step, the databases needed to be setup and the hideous .htaccess file from Typo’s public folder must be removed due to the problems it causes in combination with Apache.

rake db:create:all
rake db:migrate RAILS_ENV=production
rm public/.htaccess

To finish things, I just removed the default Apache2 host and defined my own for the Typo app using Passenger:

sudo rm /etc/apache2/sites-enabled/000-default
sudo nano /etc/apache2/sites-enabled/passenger

The config looks like this:

<VirtualHost *:80>
    ServerName 127.0.0.1
    DocumentRoot /home/colszowka/typo-5.1.98/public
</VirtualHost>

Just a restart of Apache missing, sudo /etc/init.d/apache2 restart, and everything’s done, done! Let’s verify everything works:

$ curl localhost
<html><body>You are being <a href="http://localhost/accounts/signup">redirected</a>.</body></html>

Sweet! To have a look at the site with my web browser, I created a port forwarding in VirtualBox (I wrote on how to accomplish that earlier) and pointed my browser to localhost:3123, the port I chose for the forwarding. The result in pictures rather than words:

The structure of the .deb package

The Debian (meta-)package used to setup the whole Apache/Passenger/MySQL/Ruby/Rails stack in merely ten minutes is pretty straight forward. It only consists of a control file sitting inside a DEBIAN folder. The file only describes the packages it depends upon, which then will be pulled in by apt-get. Take a look at it over at github

The control file looks like following:

Package: railsstack
Essential: no
Priority: extra
Section: metapackages
Maintainer: Christoph Olszowka 
Architecture: i386
Version: 1
Depends: ruby, ri, rdoc, irb, rubygems, apache2, mysql-server-5.0, mysql-client-5.0, rails2.2.2-ruby1.8, libfastthread-ruby1.8, libapache2-mod-passenger, libmysql-ruby
Description: Metapackage for installing a Ruby on Rails stack on Apache2 with MySQL using Passenger (built by Brightbox) and DebGem.com

Note: Unfortunately and as mentioned before, this will currently still pull in loads of Rails versions and it’s dependencies since obviously the DebGem gem dependency list is still not working properly (or it is intendend to work like this, which to me makes no sense when I pull in a specific version of Rails, like I did in the control file with depending upon rails2.2.2-ruby1.8)

Building the package is simple: dpkg -b package railsstack.deb. After that, just put the .deb file on your server and execute mentioned commands:

dpkg -i --force-depends railsstack.deb
apt-get -f install

Conclusion

Providing .deb metapackages allows for really simple, quick and straight-forward deployment of any Rails app on a vanilla Ubuntu JeOS Server. Rubygems even get updated automatically by DebGem to the latest version, so I did not have to stick with the old default Ubuntu package of Rubygems.

I did not try this yet with the actual Ubuntu Server distribution, which is much more widely used on servers, but since the server edition depends upon the same basic packages (apache2, mysql-server-5.0) as JeOS does, things should work the same way, maybe even quicker if you install the basic packages like Apache and MySQL from the CD.

Update, 2009-01-12: I actually tried this process on a new install of Ubuntu Server 8.04 with preinstalled LAMP, and everything worked just perfectly with the same steps as described above. The package manager recognized the already installed MySQL and Apache packages and did not try to reinstall them, just pulling in Passenger, Ruby and Rails.

Similar packages for Sinatra, Merb or anything you might want up and running quickly on your server should be no problem at well with a customized Railsstack.deb, so with a little help from DebGem and Brightbox, setting up a working Ruby environment for almost any Ruby server application becomes a no-brainer.

With the steps mentioned above, I was able to go from a freshly installed Ubuntu JeOS 8.04.1 to accessing the Typo site hosted on the machine in just 10 minutes, and plenty of this time was spent with downloading old Rails versions from DebGem, which hopefully will get sorted while the service is still in beta. When it becomes possible to pull in just the required versions of gems, it might be possible to have Typo running in just 5 minutes.

Like already announced in my previous first look into the new DebGem service by Phusion. I had a deeper look into the service today, giving it a spin under a freshly installed Ubuntu Server JeOS virtual machine.

JeOS is great since it has a kernel tweaked for virtual environments and the disc image has a footprint of only 100 megs, which ensures download and install times are low.

Setting up the vm for testing

Since DebGem currently only offers binary packages for Ubuntu 8.04 LTS, I picked up the JeOS cd image for that, which can be downloaded in the latest version 8.04.1 from here.

After setting up a new Ubuntu-based virtual machine VirtualBox and getting through the installation steps for Ubuntu JeOS (I won’t get into detail about that…), as a first step I updated the whole system with apt by issuing:

sudo apt-get update && sudo apt-get upgrade

This was followed by the install of the most basic tools:

sudo apt-get install bash-completion nano wget curl man

I also installed openssh-server so I could access the machine from my regular terminal, but that is optional. If you decided to follow along, please remember that you will have to set port forwarding for VirtualBox properly. On the machine you are running VirtualBox on, first find the name of your VM by typing:

VBoxManage list vms

Find your new virtual machine and remember it’s name. Now you will have to set the port forwarding for ssh with the following commands, putting your vm’s name in the obvious spot:

VBoxManage setextradata "YOUR VMS NAME" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/GuestPort" 22
VBoxManage setextradata "YOUR VMS NAME" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/HostPort" 2222
VBoxManage setextradata "YOUR VMS NAME" "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/Protocol" TCP

You can double-check if everything worked out by issuing,

VBoxManage getextradata "YOUR VMS NAME" enumerate

which should give you a list of your machines properties, including the newly set forwarding options. Now you will have to completely shut down and start up your vm so these options get applied. After that, you should be able to ssh to the machine with:

ssh localhost -p 2222

Setting up Ruby and MySQL

Just install everything from the packages by issuing

sudo apt-get install ruby ri rdoc irb rubygems mysql-server-5.0 mysql-client-5.0

MySQL will ask you for a new password for the root user, type it in and you’re ready.

Adding the DebGem apt repository

To add the Rubyforge gems to apt, issue the following commands:

sudo sh -c 'echo "deb http://www.debgem.com/apt ubuntu-8.04 rubyforge" > /etc/apt/sources.list.d/debgem.list'
sudo sh -c 'wget -q http://www.debgem.com/apt/debgem.gpg -O- | apt-key add -'
sudo sh -c 'echo "APT::Cache-Limit 33554432;" > /etc/apt/apt.conf'
sudo apt-get update

This will add the debgem repository, add their key so apt-get won’t complain and also raise the cache-limit, which is required due to the amount of packages served by DebGem. Finally, we update the whole cache so the Gems will appear in apt-get.

Please remember the following though, taken from the DebGem usage page:

Note: These URLs will likely change after DebGem has gone out of beta, so please check this web page again if apt-get update doesn’t work anymore.

Testing DebGem, part 1: Hpricot

I chose Hpricot because its basic functionality can be tested in a single line with irb, while it still is a native extension which normally would have required me to install at least the build-essential Ubuntu package, if not some required libraries as well which I would have to hunt down in the Ubuntu repositories. Installing Hpricot via DebGem on the other hand is pretty straight-forward:

apt-get install libhpricot-ruby1.8

After that and opposing to my findings in my previous post, when I was not able to install any gems from DebGem due to my manual install of RubyGems, the Hpricot gem now appeared in my gem list:

gem list --local
*** LOCAL GEMS ***
hpricot (0.6.164, 0.5, 0.4)

Now, let’s see if it works!

$ irb1.8
irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'hpricot'
=> true
irb(main):006:0> Hpricot('<div id="foo">bar</div>').at("#foo").inner_html
=> "bar"

Sweet! On a vanilla Ubuntu JeOS system, I was able to set up a working Ruby install including Rubygems in no time and without the need to search for build dependencies. DebGem even auto-updated my Rubygems when I downloaded Hpricot, which normally is the probably most ugly thing about the Ubuntu Rubygems package:

$ gem --version
1.3.1

Testing DebGem, part 2: DataMapper

DataMapper is a native extension as well, featuring plenty of dependencies and database access. On my initial review of DebGem, apt-get wanted me to pull in plenty of old versions as dependencies even when trying to install a specific version of DataMapper. Since then, the latest version of DataMapper, 0.9.9, has found it’s way into the repository. Also, Hongli Lai from the Phusion team posted a comment to mentioned article, saying that the DebGem team had removed most of the unneccessary dependencies.

When trying to install libdatamapper0.9.9-ruby1.8 today, DebGem still wanted me to pull in plenty of packages, but this time only the most recent versions, the reason for all the bulk print on my screen being the usage of metapackages. There still seems to be an issue with the projected package disc space usage though, which Hongli Lai also pointed out, since the announced 22.2 mb did not end up on my hard disc. The full install of Datamapper 0.9.9 weighs 10 mb on my hard disc.

Since I wanted to use MySQL, I also had to install the do-mysql gem:

sudo apt-get install libdo-mysql0.9.10.1-ruby1.8

As a next step, I set up a testing database through mysql console:

$ mysql -u root -p
mysql> create database datamapper_test;
Query OK, 1 row affected (0.00 sec)

I created a simple DataMapper example, which was the following:

require 'rubygems'
require 'dm-core'

DataMapper.setup(:default, 'mysql://root:pass@localhost/datamapper_test')

class Message
  include DataMapper::Resource
  property :id,         Serial
  property :body,       String
end

DataMapper.auto_migrate!

Message.create(:body => "I'm a Message")

puts Message.all.inspect

Executing the whole thing went perfectly right, with Datamapper obviously connecting to the database, migrating the table, creating the message and retrieving all messages afterwards. The console output was, just as expected:

[<Message id=1 body="I'm a Message">]

Again, that wasn’t hard at all.

Testing DebGem, part 3: Rails

As a final test, I wanted to see if I could get a Rails app to run with gems installed from DebGem. Currently, when installing a specific version of Rails, say rails2.2.2-ruby1.8, all the dependencies like ActiveRecord, ActionPack and so on will be pulled in all versions available via DebGem – exactly the same way as it was in my earlier review for DataMapper. The result was a vm crammed with:

actionmailer (2.2.2, 2.2.1, 2.2.0, 2.1.2, 2.1.1, 2.1.0, 2.0.5, 2.0.4, 2.0.2, 2.0.1, 2.0.0, 1.3.6, 1.2.5, 1.1.5, 1.0.1)
actionpack (2.2.2, 2.2.1, 2.2.0, 2.1.2, 2.1.1, 2.1.0, 2.0.5, 2.0.4, 2.0.2, 2.0.1, 2.0.0, 1.13.6, 1.12.5, 1.11.2, 1.10.2, 1.9.1, 1.8.1, 1.7.0, 1.6.0, 1.5.1, 1.4.0, 1.3.1, 1.2.0, 1.1.0, 1.0.1)
activerecord (2.2.2, 2.2.1, 2.2.0, 2.1.2, 2.1.1, 2.1.0, 2.0.5, 2.0.4, 2.0.2, 2.0.1, 2.0.0, 1.15.6, 1.14.4, 1.13.2, 1.12.2, 1.11.1, 1.10.1, 1.9.1, 1.8.0, 1.7.0, 1.6.0, 1.5.1, 1.4.0, 1.3.0, 1.2.0, 1.1.0, 1.0.0)
activeresource (2.2.2, 2.2.1, 2.2.0, 2.1.2, 2.1.1, 2.1.0, 2.0.5, 2.0.4, 2.0.2, 2.0.1)
activesupport (2.2.2, 2.2.1, 2.2.0, 2.1.2, 2.1.1, 2.1.0, 2.0.5, 2.0.4, 2.0.2, 2.0.1, 2.0.0, 1.4.4, 1.3.1, 1.2.5, 1.1.1, 1.0.4, 1.0.1)

This is only the result of issuing sudo apt-get install rails2.2.2-ruby1.8! Obviously the package dependencies need some fine-tuning, but I think that is absolutely ok for a service that just came out into beta.

Since I needed an app to test Rails with, I wanted to create a new one with the rails command. That did not work since the corresponding script could not be found on my operating system. Maybe this can be fixed with a modified path though. I didn’t bother and pulled a sample app from my main machine. After generating the controller “debgem”, which worked flawlessly, I added an index action to the newly created controller:

# app/controllers/debgem_controller.rb 
class DebgemController < ApplicationController
  def index
    render :text => "Hello from Debgem"
  end
end

A ruby script/server later, I was able to curl my action:

$ curl localhost:3000/debgem
Hello from Debgem

Great success!

Conclusion

To summarize all this, I still think that DebGem is a very promising service with some rough ends which will hopefully get sorted out while in beta. The folks at Phusion seem as they really want to make this work like a charm, taking care of the issues mentioned by users. I’m sure they will take care of the remaining issues and make this even more kick-ass. If you are having issues with DebGem, be sure to check out the DebGem Google Group.

The service makes it really easy to setup a whole Ruby environment on a Ubuntu / Debian machine quickly and without much fuzz. No need to build anything (on Ubuntu 8.04, for now), no build-dependencies to pull in before issuing gem install foo. That makes setting up a Rails environment a no-brainer.

In the upcoming follow-up to this article, I’ll focus on setting up a complete Rails stack with Apache2/Passenger from a single .deb Metapackage, so if you’re interested in that check back soon or subscribe to my RSS feed.