clog

What a day! There were certainly some great talks at Rails Edge on the first day. This all happened during a storm that must have been an act of God. I certainly had no idea what was in store for me once I left to drive home… hopefully others got home OK. I’m just glad my car didn’t float away on me at any point (honestly I’m surprised it didn’t). I was very close to just bunking in the car at some random parking lot overnight. Had I known how bad it was I would have just stayed at the hotel…

Anyway… the #1 highlight I took from day 1 is: “Be Zealous.” There are many other important highlights, but I think this one is the most important. Chad Fowler did a superb job of going over how to produce “quick and clean” (as opposed to quick and dirty) code. It took a strong force of will to get Rails where it is today. It wouldn’t have happened without the strong opinions and determination of the core team. If we’re going to continue to make the world a better place for software development, then we need to stay true to our values. These are mainly MVC, CRUD, constraint-driven development, metaprogramming and domain-specific constructs and more recently REST.

From my perspective, I hope Ruby/Rails developers continue to evangelize for the cause - but I also hope that we can all recognize when we have turned down the wrong path. Too often I’ve heard the excuse of “yeah it’s a bad way to do it, but there are already thousands of applications in production that rely on that.” The end result is that the problem just becomes worse and worse.

No matter how far down the wrong road you’ve gone, turn back.

Maintaining a zealous stance for the Ruby and Rails core values while also being open to questioning our current path will obviously require a delicate balance. The forces may pull in different directions, but developers will need to learn how to walk the line. To quote Dave Thomas’ speech today, “An object is made from a class and a class is a type of object.” Patterns often become cyclical.

Dave Thomas’ speech on metaprogramming really cut to the chase on how frameworks like Rails are built from Ruby and how you can leverage this power for yourself. If you find yourself coding a non-trivial application, then you really should think of domain-specific ways to describe your application. You can do much more than just build an API, your application can (and probably should) consist of it’s own language in many places. The language is built on Ruby (and possibly Rails as well), but is pieced together in sections that are self-describing. If done properly, this will empower you with productive development throughout the product’s life-cycle. It may be that your framework is so good that it will survive future generations.

A passive observer probably would have thought today’s conference was really “Macbook Pro/iPhone Edge.” A slightly more observant person may have thought it was really Java-loves-Rails Edge. A majority of the crowd seemed to consist of people that were either currently Java developers looking into Ruby, or Java developers that have already made the jump. Add a great talk on JRuby into the mix and there’s a lot of Java floating around. I find this amazing considering the disregarding attitude that seemed to come from the Java community towards Ruby and Rails just a couple of years ago. This change is definitely good for both camps though. The number of Java developers swarming towards Ruby is very reminiscent of how C++ developers swarmed to Java circa 1999.

Personally, I never drank the coffee. It’s not that I didn’t think Java was a great technology, I just felt that it only went half way and didn’t really see the point in that. In my mind, if you wanted efficiency - then just use C++. If you wanted productivity and multi-platform functionality, then Perl and PHP really got things done much faster (despite their crappy OO support) - at least as far as web application development goes. A majority of us that started on Rails back in the beta, 1.0/1.1 days were PHP developers that believed in dynamic languages but also believed that there must be a better way.

Whatever the path, we seem to be converging on the same road now and this can only be a good thing.

Even though I don’t really care too much about Java, I was blown away by Justin Gehtland’s presentation on JRuby. This technology really has come far and has the enormous advantage of being able to run all of the bajillion of Java libraries out there in conjunction with Ruby (and Ruby on Rails) code. This technology has developed amazingly fast and I really wonder if this may end up being the Rails platform of choice down the line. It certainly makes the migration from Java to Ruby quite painless.

Marcel Molina quickly went over his “Presenter” concepts and lamented over the state of Rails tempting language options. It is definitely clear that ERB is near the end of it’s life-span in this role.

Both Marcel and Chad reiterated many times about the evils of putting code in your views. I have recently felt a little over-zealous about all of the times I’ve gone batty about ActiveRecord calls that are in a view. I haven’t even gotten to where I try thump the no-code-in-view philosophy yet. I definitely feel encouraged that this seems to be the perspective of the core team. I’ll also remember that being zealous is generally good…

Ezra Zygmuntowicz (man that’s a tongue twister) proved that stable extreme-volume rails environments are indeed a reality. His discussion on Xen was excellent. In short, the platform of choice is: Linux (via Xen) + Nginx + mongrel + monit. The swiftiply mongrel patch to mongrel is now stable and can greatly increase your throughput. If you don’t know these technologies yet, you owe it to yourself to get to know them.

Last but not least, Mike Mangino gave another great talk on “RESTful” Rails development. Simply helpful was covered, along with changes that are going into Edge Rails - to be Released with Rails 2.0. After talking over REST concepts with the other developers and listening to Mike’s speech, I walked away convinced that REST is indeed the way to go.

I’m looking forward to Day 2, provided that I can swim my way down there!

I’m looking forward to heading down to the Rails Edge conference in Chicago tomorrow (8/23/07). Tribune Interactive was nice enough to pay my way. I will be sharing my notes and experiences here, so stop by (or check the feed) occasionally.

A “Vertical Component” is what I call a component that is strapped on to an existing system and is meant to have end-to-end functionality. It adds functionality X to product Y by tacking on a completely separate working system. This component can work independently but is usually integrated with an existing system.

The concept of a “Vertical Solution” is a bit more common. This is a product that meets the needs of a specific market. A car is an example of this. A car meets the needs of ground transportation. The term can be applied to either the product or the market, but usually means the same thing in both contexts.

A vertical solution is well integrated. All of its pieces come together (hopefully) in harmony to deliver a well-rounded and polished product. (Most) cars roll off of the assembly line looking like a single product. They’re built of many separate components, but they are all made to fit (or integrate with) the car.

What would we have if we attached a vertical component to a car? Well cars don’t (normally) fly, so let’s add that feature. Hot-air balloons already exist and can deliver this functionality, so all we need to do is find a hot-air balloon and attach it to the car. After a few days of hacking something up, there’s a huge metal harness around the car and a hot-air balloon attached to the top. Now we have a car that flies. Surely we can sell this car for a load of money and get rich right? After all, people have been waiting so long for an affordable flying car. It gets great mileage too!! Unfortunately no one wants a hot-air balloon car. It would be cumbersome, unpractical, and would look rediculous.

You don’t see this much in the physical world. Physical items typically need to look presentable and be useful in some way in order to be desired by the public.

Unfortunately this hasn’t remained the same in the online world. The web is overloaded with multi-million dollar hodge-podge applications that look like something out of a bad Mad Max movie. It’s both sad and amusing that so many internet companies try to strap together a bunch of canned (“vertical”) components into “mission-critical enterprise-ready” applications that have the quality you would expect from Fisher Price’s My-First-Website.

Why do so many companies do this? They don’t know any better. They don’t have the resources in-house (usually) and refuse to either hire those resources or listen to someone who knows better.

Of course, our crazy friends in the Mad Max world had come up with something givin the pieces they had. You could be excused for this type of behavior if you live in post-apocalypse times, but we’re not quite there yet.

The really sad thing is the amount of money companies pay for this crap. Some guy is going home every night thinking “I can’t believe I’m becoming a millionaire out of selling this crap.” I promise you he’s not driving his Corvette home with a-new-and-improved duct-taped-on wooden bumpers. No he’s driving home the real thing by selling you the internet equivalent.

Javascript is the duct tape of the web world. If you’re integrating a bunch of “vendor solutions” via javascript then you’re basically putting rail-road tie bumpers on your Corvette. Even worse, you’re probably paying out the (use your imagination) for it. That’s not to say Javascript is bad, it’s a great tool when used correctly. The heart of web applications live on the back-end though.

There are no silver bullets. If you want a great product, you have to create it. End of story.

Several months ago an exceptional former manager of mine planted a seed in my mind about how to determine project timelines. We were discussing the timeline of a particularly complex calendaring project and he suggested that it may be impossible to determine such a timeline. At the time, the idea of my manager telling me something like this is impossible struck me as odd. It flew in the face of everything I had learned in my software engineering classes. I had already discarded most of the things I learned in those classes as not being compatible with the real world, but the concept of mapping out timelines remained. I felt that an experienced manager should be able to determine an approximate completion date.

The more I’ve thought about this, the more I believe that my former manager was right in more ways than he realized. The only way to be able to determine an accurate timeline is to draw on previous experience so you can predict the future. The problem with software development is that there is almost never any previous experience to draw from. If you’re repeating an old project or something very close to it - than the work is really already done and you don’t need to bother with much development. Otherwise, you’re treading into the unknown. How long does it take to build the unknown? It’s a shot in the dark at best.

I still believe it’s a good idea to have target dates. They provide something to work toward and keep productivity high. Be prepared to meet those targets with reduced feature sets though as certain pieces of the technology don’t work out and/or prove to be more complicated than originally expected (which is often the case). Don’t get rid of good workers because they couldn’t predict the impossible. Definitely don’t make the mistake of promoting a hard launch before you have a product that works! It doesn’t take much research to realize that this approach fails more often than not.

Those of us that are iphone users and sysadmins or developers can rejoice! The handy dandy webshell program is fairly easy to set up and will fill your remote admin via iphone needs. It’s not perfect, but it gets the job done. It also allows an ssh login over https so you don’t have to worry about VPN. I may be able to finally leave that laptop at home when I go out… Google for webshell and give it a whirl.

I’ve actually had CachedModel/Memcached running for quite some time, but I didn’t notice that it stopped automatically working for simple calls when Rails 1.2 came out. At the time we were just ramping up with our new dev team and the other Rails sites I had done were nearing the end of their useful life-span. So database load was low in general, a perfect time for Rails 1.2 to sneak one in on me.

Thanks to a combination of successful product launches and ever more dynamic websites (allowing for an ever decreasing amount of cacheable material) our main database server began to become stressed. Finally I noticed that memcached was in fact not caching much at all. After looking into it, I found this ticket describing the problem and (fortunately) a fix.

Once I fixed the problem (and one of the other devs added some key fragment caching) the load went down almost 90%!! It’s ironic that it took CachedModel breaking for me to be reminded just how great it is.

If you want to try it, follow this tutorial for setting up CachedModel. It’s still fairly relevant.

There’s a fancy new Acts as Cached plugin available that also abstracts memcached and may be any better.

If you’re running high-capacity sites, some type of memcached interface is a must-have!!

A theme should allow the look and feel of an application to change. A theme has a one-to-one relationship with the application, meaning that an application should only be running one theme at a time. A theme per user would be a ‘skin’ and an application that runs multiple sites with a different “theme” for each one is a whole new can of worms. I’ve had a need to do the latter recently, but I’ll go over my (site_support) solution to that in the next post. Meanwhile, here’s my solution for adding themes to your application.

When to use it: A single application connects to a single database and has multiple theme options, but only one is running at a time.

Start by installing my hacked version of theme_support

script/plugin install http://www.chrispcritter.com/theme_support_mod/ 

ActionMailer currently doesn’t work with it, I’ll have to tackle a fix for it soon. I’ll post my findings.

At this point you should at least be able to get your poject running in mongrel.

To create the them, you’ll need to create a “themes” folder in your project and start your first theme. The theme must have the following structure:

  $app_root
    themes/
      [theme_name]
        cache/
        images/
        stylesheets/
        javascripts/
        views/           
          layouts/      

You may notice that this is a little different than the structure described in the theme_support documentation. I could not get layouts to work outside of the view path (so I just moved it into views). I really think this is better anyway - since it matches the standard view structure. The layout I describe also has a “cache” folder. The theme_support plugin supposedly supports cached files in a public/themes structure but I had some difficulty getting this to work and really wanted to keep all theme related content in it’s own folder. I ended up setting page caching to go into the themes/[theme_name] folder with some mod_rewrite magic. It may be better to add a “public” folder to each theme for this at some point.

Add a line similar to the following to config/environment.rb (where is your default theme):

THEME = ENV["THEME"] ? ENV["THEME"] : ''

And add the below line to pull out your site specific configuration options.

CONFIG = YAML.load(File.read("#{RAILS_ROOT}/config/#{THEME}.yml"))

If you’re using apache, your rewrite rules will look like the following:

RewriteRule ^([^.]+)$ themes//$1.html [QSA]
RewriteRule ^images/(.*)$ themes/THEME>/images/$1 [QSA]

You will need to modify the caching paths in environment.rb so they point to the right place. Add the following lines:

ActionController::Base.fragment_cache_store = :file_store, "#{RAILS_ROOT}/themes/#{THEME}/cache"
ActionController::Base.page_cache_directory = "#{RAILS_ROOT}/themes/#{THEME}"

Add a link in public that points to your themes directory “public/themes -> ../themes/” That will preserve the routes.

It’s not the most elegant solution but it works. It bypasses some routes functionality and the sym-link in particular is ugly. I’ll post more mods to theme_support as/if I make them along with simpler install instructions, but this was really just a stepping stone to my site_support setup - so it may be awhile before I get back to it.

Caching a highly dynamic application can be hard. Page caching doesn’t work well because some elements of the page need to change where others don’t. Action caching helps in the very few instances where you want all of your before filters to run before loading the cached view. In this instance the before filters are typically redundant and may be doing some heavy lifting that can really slow things down. Fragment caching will cache a certain section of the view or layout but seems somewhat useless at first glance because all of the heavy lifting was already done in the controller.

How do you get around this problem? There are really only three options:

1) Put the necessary code into a model and call it in the view inside a fragment cache. This essentially by-passes the controller and completely breaks MVC rules.

2) Put the necessary code into a helper and call it in the view inside a fragment cache. This works in situations where a helper is necessary, but usually this isn’t what we’re looking for. Calling heavy lifting that’s in helpers from a view is not quite as ugly as calling from a model, but is generally bad design. Helpers should only manipulate existing data, not pull new data.

3) Pass a Proc object from the controller to the view, and call it from inside the fragment cache.

The third option is best. It doesn’t break MVC because the code that gets executed is only executed inside the controller argument. It’s execution is simply delayed until it gets called from within the view. It does dirty your views somewhat and probably won’t work for liquid templates, but it (mostly) follows the rules and gets the job done in a way that is more elegant than the first two options.

This is how you do it.

In this instance, I want to retrieve data that is used for the ‘category_links’ action and view (normally called as a partial). Notice that the Event model contains the actual code to retrieve the data. Recent trends in how to separate model and controller functionality have convinced me this is the way to go. Also note that the controller still lies between the view in model in this setup. See: skinny-controller-fat-model

def category_links
  # Set up a Ruby Proc method to pass (usually to a view).  This 
  # method is defined here but cannot be called until @get_event_categories.call 
  # is used.  This allows the logic execution to be deferred until view 
  # time, but it is defined here (where it should be).  This doesn't 
  # break MVC rules while maintaining a clean syntax.  See Ruby Proc Class    
  @get_event_categories = lambda do
    categories = Event.categories.collect {|c| [c.category, c.event_count, {
      :controller => 'events',
      :action => 'list',
      'event[category]' => c.category,
      'event_date[start_date]' => date_range_start,
      'event_date[end_date]'   => date_range_end}]}

    sub_categories = Event.sub_categories.collect {|c| [c.sub_category, c.event_count, {
        :controller => 'events',
        :action => 'list',
        'event[sub_category]' => c.sub_category,
        'event_date[start_date]' => date_range_start,
        'event_date[end_date]'   => date_range_end}]}  

    return categories, sub_categories
  end
end

@get_event_categories is defined as an object proc variable that contains the code needed to retrieve the categories and sub_categories variables. None of the code was actually executed. Ruby has just stored the code and the environment needed to run it in @get_event_categories.

For completeness sake, here is what gets called above via the Event API:

def categories
  conditions = Event.theme_filters

  find_hash = {}
  find_hash[:select] = 'events.id,category,count(events.id) AS event_count'
  find_hash[:conditions] = conditions if conditions && conditions != ''
  find_hash[:group] = 'category'
  find(:all, find_hash)      
end

def sub_categories
  conditions = Event.theme_filters
  conditions += ' AND ' if conditions && conditions != ''

  find(:all,
    :select => 'id,sub_category, count(events.id) AS event_count',
    :conditions => "#{conditions} sub_category != ''",
    :group  => 'sub_category')      
end

Here is the view code for category_links:

<% categories, sub_categories = @get_event_categories.call%>
<% categories.each do |category, event_count, cat_params| %>
  <%= link_to_remote category, {
    :url => cat_params,
      :loading => "Element.show('status')"}, {
    :href => url_for(cat_params)} %>
  (<%= event_count %>)
    

<% end %>

<% sub_categories.each do |sub_category, event_count, sub_cat_params| %>
  <%= link_to_remote sub_category, {
    :url => sub_cat_params, 
      :loading => "Element.show('status')"}, {
    :href => url_for(sub_cat_params)}%>
  (<%= event_count %>)
    

<% end %>

The @get_event_categories proc method is called first and the result is copied into categories, and sub_catagories variables. These are then used to generate the actual links.

No caching you say? Since these are almost always called as partials, I don’t cache here. It is assumed that a direct call wants a non-cached version.

My event layout contains the following rail section:

<% cache(:action_suffix => "rail_links")  do %>    
  

    

        <%= render :partial => 'price_links', :layout => false %>
    
    

      Browse by date
    
            
    

      <%= render :partial => 'date_links', :layout => false %>
    
    

      Browse by category
    
  
    
                   
        <%= render :partial => 'category_links', :layout => false %>
    
    

      Browse by location
    
  
    
                      
      <%= render :partial => 'location_links', :layout => false %>
    
    

      Browse by venue
    
          
    
          
      <%= render :partial => 'venue_links', :layout => false %>
    
  

<% end %>

Notice that the entire rail gets cached, and the category links (generated with our proc call) are among them. All of the other links have similar proc calls. I have a before_filter in my controller that retrieves the proc objects before going to the view (for the appropriate actions). If the cache is hit, then the proc methods are never called and no time is wasted on the expensive data pulls and calculations. You may notice that the category link calls are fairly expensive (they group and count the results, which adds a lot of time to the query). The other links are just as expensive. Doing this literally saves seconds on the page load speed.

So there it is. Try it out and marvel at the newfound speed of your application!

Perhaps I tried this before and it didn’t work for me, but through some trial and error I figured out how to do nested includes in Rails. This means you can join a join of a joined table without ever having to mess with the join syntax. Of course you also get all of the other cool benefits of using Rails relationships.

My previous article on the subject touched on using single nested includes, but didn’t touch on multiple nesting. Here’s how you do it.

Take the following classes and relationships:

class A < Base
  has_many :B
  #relationship to C via the join table (B)
  has_many :C,  :through => :B
end

class B < Base
  belongs_to :A
  has_many :C
  has_many :D
end

class C < Base
  belongs_to :B
  belongs_to :D,
    :include => :A
end

class D < Base
  has_many :C
  has_many :B
    :through => :C,
    :include => :D
end

Say you want to search on A, and include B, C, D. All you need to do is:

A.find(:all, :include => {:B => {:C => 'D'}})

You simply keep embedding Hashes for each relationship. You can apparently do this indefinitely. So also joining an ‘E’ model that belonged to D would require:

A.find(:all, :include => {:B => {:C => {:D => 'E'}}})

Just another one of the really nifty and less advertised features of Rails.

My Rails 1.2 upgrade could have gone a lot better. At first everything seemed fine. Then I tried setting up a new test environment (which involves using ar_fixtures). The fixtures were created fine, but when I tried to test with them, I got the following.

TypeError: BigDecimal can't be coerced into BigDecimal

So after perusing I noticed that the Rails 1.2+ar_fixtures is outputting an empty BigDecimal object (I think it used to just output 0.0). Well.. that seems legit.. but it’s not working. After doing some extensive googling I don’t seem to uncover many other people having the problem, but I do find this bug reporting the same bug. Low and behold this is a Ruby core library! The ticket has been open for two months, so I think surely it doesn’t still exist… I try it out with irb and get the following:

irb -r 'yaml' -r 'bigdecimal'
irb(main):001:0> YAML::load( BigDecimal.new('1.1').to_yaml )
TypeError: BigDecimal can't be coerced into BigDecimal
        from /usr/local/lib/ruby/1.8/irb.rb:298:in `inspect'
        from /usr/local/lib/ruby/1.8/irb.rb:298:in `output_value'
        from /usr/local/lib/ruby/1.8/irb.rb:151:in `eval_input'
        from /usr/local/lib/ruby/1.8/irb.rb:259:in `signal_status'
        from /usr/local/lib/ruby/1.8/irb.rb:147:in `eval_input'
        from /usr/local/lib/ruby/1.8/irb/ruby-lex.rb:244:in `each_top_level_statement'
        from /usr/local/lib/ruby/1.8/irb/ruby-lex.rb:230:in `loop'
        from /usr/local/lib/ruby/1.8/irb/ruby-lex.rb:230:in `each_top_level_statement'
        from /usr/local/lib/ruby/1.8/irb/ruby-lex.rb:229:in `catch'
        from /usr/local/lib/ruby/1.8/irb/ruby-lex.rb:229:in `each_top_level_statement'
        from /usr/local/lib/ruby/1.8/irb.rb:146:in `eval_input'
        from /usr/local/lib/ruby/1.8/irb.rb:70:in `start'
        from /usr/local/lib/ruby/1.8/irb.rb:69:in `catch'
        from /usr/local/lib/ruby/1.8/irb.rb:69:in `start'
        from /usr/local/bin/irb:13
Maybe IRB bug!!

After going through the source I end up in C libraries and patching is certainly an extreme measure here. So it seems I have to live with the bug. On a hunch I search/replace all of the empty BigDecimal objects to 0.0 and it finally works. But… what’s up with this? This looks like a rather serious bug. I would think Matz himself would be in there fixing it after two months! Maybe I’ll take a shot at fixing it myself…

So… if you get bit by the “BigDecimal can’t be coerced into BigDecimal” bug you can get around it by converting it to string:

irb(main):002:0> YAML::load( BigDecimal.new('1.1').to_s.to_yaml )
=> "0.11E1"

In my case I have to search and replace these fixtures post export, with:

contents = ''
if Event.to_fixture  && 
   File.open('test/fixtures/events.yml', 'r') {|fin| contents << fin.read;} &&
   File.open('test/fixtures/events.yml', 'w') {|fout| fout << contents.gsub('!ruby/object:BigDecimal {}', '0.0'); fout.flush}  

  puts 'Event fixtures created'
else
  puts 'Failed to create Event fixtures'
end

You can probably figure out most workarounds from that.