Gulp Organization & Structure



Gulp is amazing. The amount of time it’s saved me since my adoption is immeasurable. Not only does it speed things up, but it allows Developers to focus on future technologies that will ultimately make the web a more usable and beautiful place.

This article won’t go over how to use Gulp or the philosophy behind Task Runners. Rather, I’m simply going to share how I organize my Gulp build to hopefully help and inspire others. Another goal of mine is to draw feedback from the community to improve my own build, so please, any suggestions are welcomed!

The big picture

Before I show you how I structure my build,  I feel responsible to persuade you why you should restructure your build.

Below is a screenshot of how I like to keep my files organized. Imagine this being within the root of your web app, or wherever you prefer to keep your package.json. As you can see, I have an actual gulp folder that contains some additional files. The config.js file contains all of my paths and file names that will be used throughout the build. The “tasks” folder contains all of individual tasks separated into their own files.

The biggest benefit I’ve experienced so far by using this methodology has been clearer thinking. It allows me to more easily reason about the build, and not have to scroll through a 500 line gulpfile every time I want to change the CSS minification. Instead, each task is at most 25 lines long without any scrolling required.

Gulp build root folder   Gulp build tasks folder

Of course at the end of the day it’s mostly a stylistic preference. However if you like what you see, let’s get going.

Dependencies

The only dependency is the “require-dir” package. This module is great because it allows us to require whole directories which fits our use-case perfectly.

So before we can begin, run this command in your project root to add it to your package.json file

npm install require-dir --save-dev

gulpfile.js

As I mentioned earlier, I’ve partitioned each task into it’s own file to make things more modular. My gulpfile.js file simply requires all of my tasks and bootstraps the build.

// Module to require whole directories
var requireDir = require('require-dir');

// Pulling in all tasks from the tasks folder
requireDir('./gulp/tasks', { recurse: true });

config.js

Under the gulp folder, you’ll find a config.js file. I use this to set up all of the paths and filenames needed throughout the various tasks I use. I store everything inside an object and then load it from within each task when I need it.

module.exports = {
  paths: {
    project: './',
    css: {
      entry: './assets/css/local/app.scss',
      all: './assets/css/local/**/*.scss',
      dest: './assets/css'
    },
    js: {
      entry: './assets/js/local/app.js',
      vendor: './assets/js/vendor/*.js',
      dest: './assets/js',
      all: './assets/js/**/*.js'
    }
  },
  names: {
    css: 'app.min.css',
    js: {
      app: 'app.min.js',
      vendor: 'vendor.min.js'
    }
  }
};

An example task

Obviously every task will be different depending on what you want to do. However I think it’s helpful to show you one of mine so you can see how I tie everything together. Below is the “styles.js” from the above tasks folder:

var gulp = require('gulp'),
    config = require('../config'), // Relative to this file
    sass = require('gulp-sass'),
    rename = require('gulp-rename'),
    browserSync = require('browser-sync'),
    please = require('gulp-pleeease'),
    reload = browserSync.reload;

gulp.task('styles', function() {
  return gulp.src(config.paths.css.entry)
    .pipe(sass())
    .pipe(please({
      "autoprefixer": true,
      "filters": true,
      "rem": true,
      "opacity": true
    }))
    .pipe(rename(config.names.css))
    .pipe(gulp.dest(config.paths.css.dest))
    .pipe(reload({
      stream: true
    }));
});

As you can see I’m loading the config.js file that we mentioned above to the “config” variable. I’m then able to use all of our paths and file names inside the task.

default.js

Lastly, here’s what my default.js file looks like. The main purpose of this file is to load all of the previously created tasks like normal. Simple and neat.

var config = require('../config'),
    gulp = require('gulp'),
    browserSync = require('browser-sync');

gulp.task('default', ['styles', 'scripts', 'server'], function() {
  gulp.watch(config.paths.css.all, ['styles', browserSync.reload]);
  gulp.watch(config.paths.js.all, ['scripts', browserSync.reload]);
});

Update: 1-30-16

In the comments, rasenplanscher made a great contribution with his preferred method of structuring Gulp using Pipes and Helpers instead. I agree with his approach and recommend people take a look to see if you like this particular style instead: https://github.com/rasenplanscher/gulp-file-structure

I’ll be making an updated build next month, stay tuned.

  • David

    I really liked you tutorial, Andrew. Thanks a lot. Nice article!

  • Nev Stokes

    Love this organisational structure — thanks for writing it up Andrew!

    The only downside is that it breaks the standard gulp zsh plugin task name tab completion (which is awesome).

    It seems that a quick workaround is to add commented lines to the main gulpfile.js

    // gulp.task ‘default’
    // gulp.task ‘styles’

    // …

    • Interesting…thanks a lot for sharing Nev. Valuable info!

  • Great article! Thanks for publishing.

    You may not have encountered this (as it my be related to me using a pre-release of Gulp#4.0), but I’m currently encountering the following error when I run my `default` task: `AssertionError: Task never defined: html`.

    It appears as though it’s related to order in which `requireDir` loads the task files. For instance, if my `default` task depends upon the `html` task, the error is thrown as `requireDir` loaded the `default` file before `html`.

    I’ve gotten around this currently by renaming “default.js” to “x-default.js”. Any thoughts on why this may be happening? Guidance on a better solution?

  • Check this tool i have build. It’s called “Gulp the builder” https://github.com/kitze/gtb 🙂

  • SocialChooozy

    Thank you, just applied it in a new project 😉

  • Pingback: Leo Kanell()

  • Pingback: Free hosting()

  • Pingback: Domain and hosting reseller()

  • Pingback: rs3 runescape bot engine tribot.org()

  • rasenplanscher

    I have basically the same setup – and for the exact same reason.

    I do not rely on configuration, though. Instead, I use helpers and lazy pipes to streamline and DRY up everything. One advantage, in my view, of gulp over grunt is not needing all that configuration. Also, why make your paths configurable? You don’t change your file structure often, do you? I feel it’s more of a hassle to seperate those globs out of the tasks where you need them…

    • Interesting, thanks for sharing. Do you have your setup in a Gist by chance?

      • rasenplanscher

        You’re welcome! I’ll make a gist/repo within the next couple of days, in which I’ll have my basic setup.

      • rasenplanscher

        Here it is: https://github.com/rasenplanscher/gulp-file-structure

        It’s the bare-bones extract from my current project.

        • So after taking a look, I’m convinced your approach is better haha. I still prefer to keep my paths separated though. My best argument is that it just feels cleaner to me. However I agree with you that it’s not really configurable so why do it at all? Either way, your approach of using helpers and pipes is definitely something I’ll be adopting in the future. I think it’s a big improvement. Thanks again for sharing your insight!

          • rasenplanscher

            You’re very welcome 🙂

        • Clinton Campbell

          After digging through the code a few times, I’m still struggling to track the flow of execution and the advantages of your approach. Would you be willing to elaborate a bit more on what’s happening here and why you decided to take this approach? I’m sure it would be helpful to more than just me.

          • rasenplanscher

            Sure!
            Concerning the flow of execution: just like in Andrew’s setup gulp loads the gulpfile, requires, and builds the tasks. Those, in turn, require helpers and lazy pipes from the respective directories. There’s actually not much magic to it, besides the require-dir package that returns the files’ exports from those directories as properties on an otherwise plain object, which I use in conjunction with the index files through which I ensure uniform use of the package. Thus, `var helper = require(‘../helpers’); var plugin = helper.load;` is equivalent to `var plugin = require(‘require-dir’)(‘../helpers’).load`.

            I went that way in order to not write the same stuff twice. While building the app and styles tasks I realized that they really differed only in the pipes they applied to their source files. The same was partially true of the vet task. Therefore I separated the commonalities out into helpers. Additionally, I found that I’d do the same transformations for production that I do for development, plus something extra, but I don’t need the same environment, like watches. Hence the pipes directory.

            Hope this helps 🙂

          • Clinton Campbell

            Thanks. Looking at lazypipe and a couple of plugins helped out quite a bit too. I like the general strategy, but I still found it difficult to grasp the overall build process even when all of the individual pieces made sense. Given that the repo is pretty bare bones, I realize it might click easier if I was actually seeing the workflow in action. That said, I’m not sure you could consider it self-documenting in the same way as most other gulp code (including Andrew’s) — too many hops between directories and files when tracing the flow of execution for a particular task.

            Reading a few guides on splitting out gulp tasks, I’ve so far preferred the approach of exporting functions in a separate files within the gulp dir to be passed to gulp.task in the gulpfile.js. Though it’s not as aesthetically pleasing as the as the gulpfile that merely imports require-dir, it does make it easier to understand the toolchain in a glance.

            As I get a bit more comfortable with gulp, I’m going to try to make some use of lazypipe and see if it takes me in a similar direction to you. I’m certainly curious about how the code transformed at various stages of refactoring, and I’m thinking this exercise would help scratch that itch.

          • rasenplanscher

            You make a few good points! My build-chain is actually still evolving and I will try to get the current state up today or tomorrow. You’ll find that I’ve also switched to the recommended pattern of calling gulp.task only in the gulpfile while providing the actual code from separate files in gulp/tasks – so, on that point, I’m totally with you.

            Also, I’ve dropped lazypipe in favor of stream-combiner – actually due to lazypipe’s lead developer’s own recommendation to do so for complex setups (https://github.com/OverZealous/lazypipe/issues/4#issuecomment-36785601). However, I’ve also upgraded to gulp 4, which I heartily recommend for everybody! That, however broke my setup and I switched to pumpify, which does the same as stream-combiner but with streams2.

            I still think lazypipe is a great tool and it’ll solve the simpler cases it was built for more elegantly, I think – at least as it stands now, in my project; I might find a more elegant way of using pumpify, yet…

          • Clinton Campbell

            @rasenplanscher:disqus Thanks for the follow up. If you do have a chance to post, I’ll definitely take a look. Especially curious about gulp 4. The project seems to have been a bit quiet since announcing they were near release middle of last year, and there aren’t a lot of examples to see it in action.

          • Hey @clinton_campbell:disqus

            I’m in the process of writing an updated article using Gulp 4, Babel and ES6. Hoping to have it released in a week or so. I agree — I wish there were more posts on this. Looks like we have to do it ourselves =)

  • Pingback: life insurance lawyer attorney()

  • Pingback: official website()

  • Pingback: economics tuition()

  • Pingback: dknight speaker()

  • Pingback: onunbununcocugu()

  • rasenplanscher

    You’re welcome! I’ll make a repo within the next couple of days, in which I’ll have my basic setup.

  • Pingback: economics tuition()

  • Pingback: economics tuition()

  • Pingback: anal bleaching()

  • Pingback: mapa niemiec przed 2 wojna()

  • Pingback: Anti Milben Mittel()

  • Pingback: tsa precheck airports()

  • Pingback: woodworm treatment()

  • Pingback: www.plumbingquotesnow.com()

  • Pingback: diabetes destroyer()

  • Pingback: electricien saint maur des fosses tarif()

  • rasenplanscher

    I’ve just pushed up the improved version of my build chain to https://github.com/rasenplanscher/gulp-file-structure

    It’s still not as elegant as I’d like it to be and I will improve on that as it progresses. For the time being, however, I fell this is a major step in the right direction. I hope this helps and am looking forward to your feedback

    @clinton_campbell:disqus @simpleblend:disqus
    I’m with you on the gulp 4 announcement issue 🙂