Debugging the Rails Asset Pipeline with Heroku Buildpacks

When I was working on a Rails 3.0.3 application trying to speed up the user experience, tools like YSlow and PageSpeed kept tell me to do things that would have been so much easier in the asset pipeline – combine, minify, and so on.

In my current 3.1.1 application, I’ve found that assets can be really slow in development mode – probably why they optimized dev mode for 3.2. One solution involved tracking down some misconfigured static asset serving in my rails engines. Another step involved turning off a lot of autoloading during asset serving.

Debugging the Asset Pipeline on Heroku

There are a number of challenges to run the asset pipline on heroku, especially when you insist on running an unconventional rails layout, and requirejs-rails to boot. The biggest challenges are that the error messages are awful, and the uniqueness of the environment, which means that local testing of the asset compile step only gets you halfway there. You also have to get the precompile step to run at all.

Get it to compile locally

I skipped this step at first. There are number of simple things which could be wrong – such as having config.assets.compile or config.serve_static_assets off. While I have no reason to run precompiled assets locally, it’s still possible to run rake assets:precompile.

Start running the rake tasks from the rails directory to make sure the basic process is working. Once that runs, run rake assets:clean and try again.

Get it to compile from the repo root directory

I ran into a lot of trouble here because the asset precompile process actually shells out to new rake instances, which start from scratch, context wise. Since I had made loading rails conditional in my base rakefile to make running other tasks faster, this didn’t work out of the box. I set up the asset tasks to depend on a task which changed the directory. Then when rake shelled out, it ran in the rails directory and picked up the standard rails rakefile.

Compile with a bogus database address

If you have set up your project to use Heroku-style DATABASE_URL by default, this is just a matter of passing the right variable.

DATABASE_URL=postgres://foo/bar rake assets:precompile

Sometimes code will connect to the database during precompile, but Heroku doesn’t have a database available at that time. I ran into this problem with ActiveAdmin I had to add a hack to prevent loading ActiveAdmin routes in routes.rb (fixed upstream)

break if ARGV.join.include?('assets:precompile')

Get asset precompilation to TRY to run on Heroku

My application was coming up as a Ruby/Rack application. This was enough to get it to run, but asset precompilation wasn’t running, which broke some pages. The odd directory structure was throwing it off, so we’ll need a few concessions to standard rails directory structure. But which ones? Reading the Heroku Ruby Buildpack source tells us that it needs two files to exist:

config/environment.rb (to show up as Rails 2+)
config/application.rb (containing Rails::Application, to also be Rails 3+)

Debug asset compilation with a custom buildpack

At first the error I was getting looked like a Ruby syntax error. The only sense I could make sense of it, however, was as a Javascript syntax error. Unfortunately, the reference was to a line number in a combined file, before minification destroyed the line numbers. This can be worked around by temporarily setting config.assets.compress = false in production.rb. Now I could see that javascript was choking on Coffeescript which hadn’t been translated.

But what is actually going wrong? Some information can be gotten by running rake with --trace – but we don’t control the precompile command on Heroku. However, we can.

On the latest Cedar stack, Heroku has built a much more versatile platform, that can and does run systems other than Rails and Ruby. This process is orchestrated by buildpacks, which literally run the slug compile by calling a small number of programs, which then do pretty much anything they like. The Ruby buildpack is conveniently written in Ruby.

Since the buildpack is just code, we can modify the asset call to include the --trace flag. (You could use my ruby buildpack fork, but I might make more changes at any time, and heroku appears to use it ‘live’.

Heroku’s platform has multiple buildpacks. Each includes a detect script that tries to determine if a repo is appropriate for it. I don’t know whether a custom buildback becomes exclusive, or just get’s first crack at the project. In any case, it appears that the only way to use a buildpack is to set it at instance creation time, so I created a new one for asset debugging.

heroku create --stack cedar --buildpack

Use Your Node

The problem turned out to be a missing node binary. In order to enforce it’s total isolation, Heroku gives each slug compile it’s own local, relative bin/node Since I had proven locally that the easiest way to make the forking rake behave was to cd into the rails subdirectory, it no longer had node at the correct relative path. I created a railshost/bin/node script to call the appropriate path (make sure to pass the arguments!) Heroku Support later suggested a link, though I was leery of running a link through the version control system.


  • Check your config variables
  • Run it locally
  • Change directories as necessary
  • Add stubs to make project detect as Rails
  • Make sure you aren’t connecting to the database
  • Make sure you can use the project-relative path bin/node
  • Turn off asset compression to investigate JS errors (and don’t forget to turn it back on)
  • Use --trace to debug rake tasks
  • Use a custom buildpack to debug the process on Heroku.
Posted Saturday, March 3rd, 2012 under Essay.

Comments are closed.