Introduction

When using the Chrome DevTools (F12) you may have noticed that the HTML, CSS or Javascript code can look quite awkward. Why is the code so awfully structured so you can hardly recognize anything? This is done by purpose, of course, but why?

There are two major reasons why frontend developers format their code like this. For one, if the code is stripped off from white spaces, tabs and newlines, doing so can significantly shrink the size of the files, thus speeding up transfer to client devices. This is particularly important for mobile devices when most data plans are still more expensive than DSL. Even better, if many files can be concatenated to just a few bigger ones, this save a lot of the request/response data transfers of the HTML protocol.Simply put, its much quicker to send one 100 KB file to the client browser than 10 files with 10 KB each.

The other reason is obfuscation. Program code, regardless of being HTML5, CSS or Javascript, is still intellectual property that their creators want to be safe from pirating. This is kind of difficult when code has to be delivered in its source form to the browser, just like it happens with HTML, CSS and Javascript. The method of choice is obfuscation, which makes it difficult to extract readable code from the files that are sent to the browser while maintaining syntactically correct source code.

So shrinking, concatenating and obfuscating have been performed before professional webapps were deployed to a web server. But there is more being done with this kind of source code before it is deployed. Most prominently, most assets are “revisioned” so they change their name when content has changed. This is important to circumvent many potential problems with browser or network caches. If a file’s content has changed, its name also changes to make sure no cache gets in the way when the new content needs to be delivered.

This article prototypes a typical build pipeline. If a webapp is ready to go online, the build process steps through several stages until the desired outcome is ready to be deployed.

Task Runner

In earlier times, build steps were manually performed. Manually starting and connecting build steps is error prone and time consuming. If you need to deploy your software with only small feature increments, e.g. for testing, this can really get messy. In order to automate this process, task runners have become very popular among frontend webapp developers. The two most reknown solutions are ‘Grunt’ and  ‘Gulp’. During this tutorial, we will use Gulp because of its easier handling of build tasks.

Task runners don’t run in a browser environment. Nevertheless, they need some kind of execution environment, just like Java. With the advent of “Nodes.JS”, arbitrary Javascript programs can be run without a browser. For webapp developers, this is quite convenient, because they can now employ highly configurable build utilities which are written in Javascript, just like they are used to when writing their apps, but without a DOM. Such build tools are mere utitlities which output their messages or log data on the console. If we think this further, we might be able to use these build tools in a CI environment like Jenkins. This article will cover Jenkins integration in the last chapter.

Prerequisites

Readers of this tutorial must be familiar with HTML5, CSS3, and Javascript. The build pipeline is based on NodeJS and Gulp, so if you want to follow the instructions, you must install Node.js. It is also recommended that you use Webstorm as your IDE because it provides a convenient Gulp integration.

NodeJS provides a Javascript runtime environment based on Google’s V8 engine. We don’t use a browser for running these tools, of course. Therefore, we can’t use the browser’s <script src=....> tag to load Javascript files. We make use of Node’s module architecture instead. Variables local to the modules will be private, because a module is wrapped in a function by NodeJS (see module wrapper).

For preparation, we define the required tools in the package.json file in the project’s root.

{
  "name": "buildpipeline",
  "version": "1.0.0",
  "description": "",
  "main": "Gulpfile.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [
    "Gulp"
  ],
  "author": "sgoemans",
}

The above file is required by NodeJS. You can create a basic boilerplate version of this file by entering npm init in your terminal. Preferably you created a project folder beforehand and switched to that, so package.json is already in it’s correct place:

c:> npm init

You are asked a few questions and then a basic package.json is written to the current directory where you can continue editing it in your IDE.

Please note that once you have a fully functional package.json file containing all your dependencies, you just commit this file and your source code to your version control system. Never commit the node_modules folder as this folder can easily be recreated by letting npm read package.json file and automatically downloading all missing dependencies. To do so you just need to enter npm install but right now thats not necessary because we already installed the modules manually.

If you’ve accidentally checked in node_modules before, that’s okay. You can remove it like this:

$ echo 'node_modules' >> .gitignore
$ git rm -r --cached node_modules
$ git commit -am 'ignore node_modules'