Wednesday, June 19, 2013

Demystifying the Rails Asset Pipeline

One of the prominent features of Rails 3.1 was the asset pipeline. The asset pipeline provides some great functionality to rails applications and engines, including dynamic compilation of high-level assets, such as coffeescript and sass, concatenation and compression of asset files in production, and more. To be honest, the asset pipeline was just simply magic to me for quite some time. Recently, I've been slowly peeling back the layers and glancing under the hood, and in this post, I would like to share some of my discoveries.

Put simply, the asset pipeline does a few things:
  • Compiles assets such as Coffeescript and SASS down to Javascript and CSS
  • Concatenates all Javascript and CSS assets in order to serve less files
  • Compresses (minifies) files to be served
  • Generates a digest to be added to the file name, allowing the file to be cached indefinitely (if the file contents change, a new digest, and thus a new filename, is generated)
The asset pipeline has several configuration options, and the default configuration in development mode is very different than the default configuration in production mode. To make the functionality of the asset pipeline clear, let's start with what it does by default in development mode.

Development Mode:

The first piece of the puzzle to be understood is the concept of manifest files. A manifest file is simply a Javascript or CSS file which contains some special directives. These directives specify which other Javascript or CSS files should be included when this file is served up. When a rails app is first generated, some default manifest files are generated in app/assets/application.js and app/assets/stylesheets/application.css. For simplicity, let's just talk about the Javascript manifest (everything discussed will apply similarly to the CSS files as well). At the bottom of the file, there are three lines:
//= require jquery
//= require jquery_ujs
//= require_tree .
This means that when we compile this file, we must include the files jquery.js, jquery_ujs.js, and every javascript file in the current directory tree. The files specified here are found by searching the asset path, which by default includes app/assets/javascripts, vendor/assets/javascripts, and lib/assets/javascripts. (Similar paths exist for stylesheets). Now, if you run the rails app and visit the path /assets/application.js in the browser, you will see that the jquery files have been concatenated together, along with the manifest file itself, and any other Javascript files in the directory. This is the basic operation of the asset pipeline.

Another very important part of the asset pipeline is the generation of HTML tags which reference the assets. For example, we can use javascript_include_tag within a view or layout to include a Javascript file in the view. You'll notice that by default, if a view calls javascript_include_tag "application", the generated code looks something like this:
<script src="/assets/jquery.js?body=1" type="text/javascript"></script>
<script src="/assets/jquery_ujs.js?body=1" type="text/javascript"></script>
<script src="/assets/articles.js?body=1" type="text/javascript"></script>
<script src="/assets/application.js?body=1" type="text/javascript"></script>
Two things stand out here. First, there are four script tags where we may expect only one. Second, the URLs have a parameter "body=1". This is because the asset pipeline in development runs by default in debug mode. This simply means that instead of concatenating all the files referenced by the manifest file, all the files will by included separately as above. The "body=1" parameter simply tells the pipeline not to do any concatenation when including a given file, but simply to serve the body of the given file. To see this in action, visit the paths /assets/application.js and /assets/application.js?body=1 and compare the two outcomes.

Furthermore, in development mode, the asset pipeline does not compress the files which are concatenated, or generate digests when it generates the paths.

All of the functionality described above is configurable. To configure the asset pipeline for development mode, open config/environments/development.rb and set the following configuration variables:

  • config.assets.enabled - enable the asset pipeline. Defaults to true.
  • config.assets.paths - the paths which are searched to find assets referenced by the manifest file. You may add directory paths to this array in order to include them in the search.
  • config.assets.debug - turns on debug mode. If set to true, the javascript_include_tag (and stylesheet_link_tag for stylesheets) will include each file in the manifest separately and append "?body=1". Otherwise, they will be concatenated.
  • config.assets.digest - tells javascript_include_tag (and stylesheet_link_tag) to generate digests and append them to the filenames.
  • config.assets.compress - tells the asset pipeline to compress (minify) all assets which are served.

I would encourage starting up a rails app and changing these configuration parameters to see what they do. It will help a lot in understanding how the pipeline works. Remember to restart the server after changing the configuration.

Production Mode

So that's enough about development mode, how about production mode? Well, as you may guess, config.assets.debug is set to false, config.assets.digest is set to true, and config.assets.compress is set to true by default. However, there is one more trick which is used in production: precompiling!

In production, by default, assets must be precompiled in order to be served. This means that before the app is deployed to production, the rake task assets:precompile must be executed in order to run all assets through the pipeline, generate static Javascript and CSS files to be served, and save them in the public/assets directory. Then, when the app is run, the pipeline is disabled, and when a request comes in to a /assets URL, the precompiled static file is served by the web server. This means that all compilation, concatenation, compression, and digest generation is done by a rake task beforehand, rather than on the fly. This makes perfect sense in production since the assets shouldn't be changing often, and serving static files is a lot faster than compiling the assets for every request.

With this in mind, there are two more configuration parameters to consider:

  • config.assets.compile - whether or not the asset pipeline should compile assets on the fly. By default, true in development mode and false in production mode.
  • config.assets.precompile - an array of file paths to be precompiled. For example, application.js and application.css are included by default, but if you wish to precompile some other files in production mode (for example, if they need to be referenced separately by an HTML page in the app), then this file's path may be added to the array.

One more thing. Rails engines.

Rails engines are wonderful things. Mountable engines essentially give you the ability to have a rails app within another rails app. The inner rails app may also have its own set of assets to add to the pipeline. The good news is, the paths for the engine assets are added to the asset search path, and the engine assets are namespaced. So, if you have an engine called my_engine, it may have a file app/assets/my_engine/application.js. This file may be included into the main app's manifest file by adding the following line to the application's manifest:
//= require my_engine/application.js
In this way, engines no longer need to supply generators to copy their assets into the public folder of the rails app. Instead, the asset pipeline takes care of it for you.

I've learned a lot about the Rails Asset Pipeline in the last little while. I hope you've found this explanation helpful. For much more information and configuration parameters, check out

Any comments about the Rails Asset Pipeline? Is it still all black magic?

No comments:

Post a Comment