Ruby on Rails gives us incredible power. MVC architecture, automatic reloading, easy model relations, cacheing, asset management. It also gives us incredible overhead, especially when it comes to testing.
I recently ran across this in Event Faster Websites, quoting Jakob Nielsen’s research:
- 0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result.
- 1.0 second is about the limit for the user’s flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data.
- 10 seconds is about the limit for keeping the user’s attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect.
Where do your tests lie on that scale?
Rail Overhead
Rails tests tend to be slow for two reasons:
- Loading the entire framework with all it’s nifty features on every test run.
- Using the entire framework with all it’s nifty features on every test.
Not using the full stack (especially hitting the database) basically comes down to mocking and stubbing, which is a topic unto itself. What I’m going t focus on here is not loading the entire framework. This doesn’t mean addressing loading won’t cause some mocking and stubbing – anywhere you butt up against the Rails API or ActiveSomething, you’re going to have to stub that bit out to avoid loading it, an exercises that should make a nice object lesson in modularity and coupling.
Fast Rails Tests
Getting separately testable pieces won’t be easy. You actually need something that doesn’t depend on Rails, such as accessing an API, or that can be easily stubbed out, such pulling a couple methods into a module. You can do some refactoring in the standard test environment, but then you’ve got to figure out how to actually test them without rails.
Corey Haines demonstrated /spec_no_rails. That was a little long for me. as well as a little negative, so I went with /spec_clean. I wanted, of course, to have /spec be the plain tests and /spec_rails the slow ones, but that runs afoul of the many and various tools, such as autotest, which make assumptions about your directory structure.
I also tried have a /spec/clean directory, but I ran into trouble with the default load paths and assumptions made about spec_helper.rb. In the end I had a separate directory for spec_clean. To avoid the ambiguity of spec_helper, I had spec_rails_helper and spec_clean_helper. Getting this to work with the default load path required creating/spec/spec_clean_helper.rb` and having it load the correct file, which could then adjust the load paths appropriately.
You have to be careful adjusting the load paths. Oh, and in your tests, don’t forget that won’t have Rails magic constant loading, so you’ll have to require everything explicitly. My first draft of spec_clean_helper looked like this:
$LOAD_PATH << File.join(dir)
$LOAD_PATH << File.join(File.join(dir), "../app")
Dir[File.join(dir, "support/**/*.rb")].each {|f| require f}
require File.join(dir, '../config/initializers/my_init')
RSpec.configure do |config| ...
If you want to go a step further and avoid Bundler’s startup overhead, you’ll also have to make provisions for anything outside of Rubgems – directly loading anything that falls under Bundler’s path or git options.
