Build Your Own Yeoman Generator
Yeoman’s generators are what give its platform flexibility, allowing you to reuse the same tool for any kind of project you may be working on, JavaScript or otherwise, and that’s before I mention the enormous library of over 400 community contributed generators. Sometimes though, you may have some specific setup that you like to employ in your own projects and for situations like this, it makes sense to abstract all the boilerplate into your own generator.
In this article, we will be building a Yeoman generator that will allow us to build a single page site, one row at a time, it’s a fairly simple example but it will allow us to cover a lot of the more interesting features Yeoman provides.
Getting Set Up
I am going to assume you have Node.js setup, if not, you can grab the installation from here. Besides that, we will need to have Yeoman installed as well as the generator for creating generators. You can accomplish this via the following command to npm:
npm install -g yo generator-generator
Finally, we need to create the actual project, so navigate to the folder of your choosing and run:
yo generator
This will start the generator and ask you some questions like the project name and your GitHub account, for this article, I am going to name my generator one page.
The File Structure
Don’t be alarmed by the large number of files generated by the command, it will all make sense in just a moment.
The first couple files are dotfiles for various things like Git and Travis CI, we have a package.json file for the generator itself, a readme file and a folder for tests. Besides that, each command in our generator gets a folder with an index.js file and a folder for templates.
The index.js file needs to export the actual generator object which will get run by Yeoman. I am going to clear everything inside the actual generator so we can start from scratch, here is what the file should look like after that:
'use strict';
var util = require('util');
var path = require('path');
var yeoman = require('yeoman-generator');
var chalk = require('chalk');
var OnepageGenerator = yeoman.generators.Base.extend({
});
module.exports = OnepageGenerator;
A Yeoman Generator can extend from two different pre-built options: the Base generator, which you can see this one inherits from, or the NamedBase generator, which is actually the same thing except it adds a single parameter that the user can set when they call your generator. It doesn’t matter too much which one you choose, you can always add parameters manually if you need more.
All that this code is doing is creating this generator object and exporting it out, Yeoman will actually retrieve the exported object and run it. The way it runs, is by first calling the constructor method to set the object up and then it will go through all the methods you create on this object (in the order you created them) and run them one at a time. This means it doesn’t really matter what you call the functions and it gives you the flexibility to name them things that make sense to you.
Now, the other thing you should know is that Yeoman has its own functions for dealing with the file system that really help you out with paths. All the functions you would normally use like mkdir, read, write, copy, etc, have been provided, but Yeoman will use the template directory in this command’s folder as the path for reading data and the folder the user is running your generator as the root path for outputting files. This means you don’t even need to think about the full paths when you work with files, you can merely run copy and yeoman will handle the two different locations for you.
Getting Input
Yeoman allows you to add questions to your generator so that you can receive input from the user and customize the behavior appropriately. We are going to have two questions in our generator. The first being the name of the project and the second being whether or not to include a dummy section as an example.
To accomplish this, we will add a function that will prompt the user for these two pieces of info and then store the results on our object itself:
var OnepageGenerator = yeoman.generators.Base.extend({
promptUser: function() {
var done = this.async();
// have Yeoman greet the user
console.log(this.yeoman);
var prompts = [{
name: 'appName',
message: 'What is your app's name ?'
},{
type: 'confirm',
name: 'addDemoSection',
message: 'Would you like to generate a demo section ?',
default: true
}];
this.prompt(prompts, function (props) {
this.appName = props.appName;
this.addDemoSection = props.addDemoSection;
done();
}.bind(this));
}
});
The first line inside the function sets a done variable from the object’s async method. Yeoman tries to run your methods in the order that they are defined, but if you run any async code, the function will exit before the actual work gets completed and Yeoman will start the next function early. To get around this, you can call the async method, which will return a callback and then Yeoman will wait to go on to the next function until that callback gets executed, which you can see it does at the end, after prompting the user.
The next line where we just log yeoman, that’s the Yeoman logo which you saw when we ran the Yeoman generator just before. Next, we defined a list of prompts, each prompt has a type, a name and a message. If no type was specified, it will default to ‘input’ which is for standard text entry. There are a lot of cool input types like lists, check boxes and passwords, you can view the full list here. In our application, we are using one text input and one ‘confirm’ (yes/no) type of input.
With the array of questions ready, we can pass it to the prompt method along with a callback function. The first parameter of the callback function is the list of answers received back from the user. We then attach those properties onto our main object and call the done method to go on to the next function.
Scaffolding Our Application
Our application has the outer skeleton, which will include the header, menu, footer and some CSS, and then we have the inner content which will go in its own directory along with a custom CSS file per section. This will allow you to set global properties in the main file and customize each row by itself.
Let’s start with the header file, just create a new file in the templates directory called _header.html with the following:
http://cdnjs.cloudflare.com/ajax/libs/semantic-ui/0.12.0/css/semantic.min.css" rel="stylesheet">
The file names don’t need to start with an underscore, but it’s a best practice to differentiate the name from the final output name so you can tell which one you are dealing with. Also, you can see that besides for including our main CSS file, I am also including the Semantic UI Library as a grid for rows and for the menu (not to mention the great styling).
Another thing you may have noticed, is I used an EJS-styled placeholder for the title and it will get filled in at runtime. Yeoman uses Underscore templates (well lO dash) and as such you can create your files like this and the data will be generated at runtime.
Next, let’s create the footer (put this in a file named _footer.html in the templates folder):
It just closes the HTML document for us, as we won’t be having any JavaScript in our application. The last HTML file required for the outer scaffold is the menu, we are going to generate the actual links at runtime, but we can write the outer container in a template file called _menu.html:
It’s a plain div with some classes provided by Semantic UI. We will pull in the actual links based on the generated sections later. Next, let’s make the _main.css file:
body{
margin: 0;
font-family: "Open Sans", "Helvetica Neue", "Helvetica", "Arial", sans-serif;
}
Just some code to help with the menu positioning and change the font, but this is where you can put any default styles you want on the entire document. Now we have to make templates for the individual sections and their accompanying CSS files, these are pretty basic files as we will leave them pretty blank for the user to customize, so inside of _section.html add the following:
<%= content %>
Just an outer div with a unique id which we will generate based on the section’s name and a class for the Semantic grid system. Then inside, I just added an H1 tag which again will just contain the section’s name. Next, let’s create the CSS file, so create a file called section.css with the following:
#<%= id %>{
background: #FFFFFF;
color: #000;
}
This is more of a placeholder with the same ID as the outer container, but it’s common to change the background of each row to separate the content, so I added that property in for convenience. Now, just for completion’s sake, let’s create a file called _gruntfile.js but we will populate it later and let’s fill in the _package.json file that was provided:
{
"name": "appname",
"version": "0.0.0",
"devDependencies": {
"grunt": "~0.4.2",
"grunt-contrib-connect": "~0.6.0",
"grunt-contrib-concat": "~0.3.0",
"grunt-contrib-cssmin": "~0.7.0"
}
}
It’s a little bit of a spoiler as to what we will be doing later, but it’s the dependencies we will need for our Grunt tasks.
With all these files ready, let’s go add the methods to scaffold a new project. So back in our index.js file, right after the promptUser method, let’s add a function to create all the folders we will need:
scaffoldFolders: function(){
this.mkdir("app");
this.mkdir("app/css");
this.mkdir("app/sections");
this.mkdir("build");
},
I added an app folder and inside, two more directories: css and sections, this is where the end-user would build there application. I also created a build folder which is where the different sections and CSS files will get compiled together and built into a single file.
Next, let’s add another method to copy over some of our templates:
copyMainFiles: function(){
this.copy("_footer.html", "app/footer.html");
this.copy("_gruntfile.js", "Gruntfile.js");
this.copy("_package.json", "package.json");
this.copy("_main.css", "app/css/main.css");
var context = {
site_name: this.appName
};
this.template("_header.html", "app/header.html", context);
},
Here we use two new methods, copy and template, which are pretty similar in function. copy will take the file from the templates directory and move it to the output folder, using the provided paths. template does the same thing, except before writing to the output folder it will run it through Underscore’s templating function along with the context in order to fill in the placeholders.
I didn’t copy the menu over yet because for that, we need to generate the links, but we can’t really generate the links until we add the demo section. So the next method we create, will add the demo section:
generateDemoSection: function(){
if (this.addDemoSection) {
var context = {
content: "Demo Section",
id: this._.classify("Demo Section")
}
var fileBase = Date.now() + "_" + this._.underscored("Demo Section");
var htmlFile = "app/sections/" + fileBase + ".html";
var cssFile = "app/css/" + fileBase + ".css";
this.template("_section.html", htmlFile, context);
this.template("_section.css", cssFile, context);
}
},
Another function that you may not be familiar with is the classify function, which is provided to you by Underscore Strings. What it does is it takes a string and it creates a “class” version of it, it will remove things like spaces and create a camel-cased version, suitable for things like HTML classes and IDs; underscored does the same thing except instead of camel-casing it snake-cases them. Besides that, it’s all stuff we have done in the previous function, the only other thing worth mentioning is that we are pre-pending a time-stamp, both to keep the files unique but also for ordering. When we load the files in, they are alphabetized so having the time as the prefix will keep them in order.
The last file that needs to be copied over is the menu.html file but to do that, we need to generate the links. The problem is we don’t really know at this stage which files were generated before or if the user added sections manually. So to build up the menu, we need to read from the output path and after reading the sections that exist there, we need to construct the menu links. There are a handful of new functions that we haven’t used yet, so we will go through it line by line:
generateMenu: function(){
var menu = this.read("_menu.html");
var t = '<%= name %>';
var files = this.expand("app/sections/*.html");
for (var i = 0; i < files.length; i++) {
var name = this._.chain(files[i]).strRight("_").strLeftBack(".html").humanize().value();
var context = {
name: name,
id: this._.classify(name)
};
var link = this.engine(t, context);
menu = this.append(menu, "div.menu", link);
}
this.write("app/menu.html", menu);
},
The first line reads in the menu’s outer HTML and then I created a template string that we can use for each link. After that, I used the expand function, which accepts a resource path, relative to the folder the generator was called in (the output path) and it returns an array of all the files that match the provided pattern, in our case this will return all the section HTML files.
Then we cycle though the list of files and for each one, we use Underscore Strings to remove the timestamp and file extension so that we are left with just the name of the file. The humanize method will take things that are camel-cased or characters like underscores and dashes and convert them to spaces, so you get a human readable string. It will also capitalize the first letter of the string which will work out great for our menu. With the name separated, we create a context object along with the ID we used before, so that the links will actually take us to the correct sections.
Now we have two new functions we haven’t seen before, engine and append. engine takes a template string as the first parameter and a context object as the second and it will run it through the templating engine and returns the results. append accepts an HTML string as the first parameter, a selector as the second, and something to insert as the third parameter. What it will do is insert the provided content at the end of all the elements that match the selector. Here we are adding the link to the end of the menu div.
Last, but not least, we have the write method which will take our computed HTML string and write it out to the file. Now, just to finish up here, let’s add a method to run npm:
runNpm: function(){
var done = this.async();
this.npmInstall("", function(){
console.log("nEverything Setup !!!n");
done();
});
}
The first parameter for npmInstall is the paths, but you can just leave this blank, besides that I am just printing out a message at the end, telling the user the app is ready.
So as a quick recap, when our generator runs, it will ask the user for the project name and whether or not to include a demo section. After that, it will go on to create the folder structure and copy in all the files needed for our project. Once done, it will run npm to install the dependencies we defined and it will display the message we just put in.
This is pretty much all the main generator needs to do, but it’s not that useful without the Grunt tasks. Currently, it’s just a bunch of separate files but in order to properly develop the sections, we need a way to preview them as a single file and we will also need to the ability to build the results. So open _gruntfile.js from the templates directory and let’s get started:
module.exports = function(grunt) {
grunt.initConfig({
//task config
});
grunt.loadNpmTasks('grunt-contrib-connect');
grunt.loadNpmTasks('grunt-contrib-cssmin');
grunt.loadNpmTasks('grunt-contrib-concat');
grunt.registerTask('serve', ['connect']);
grunt.registerTask('build', ['concat', 'cssmin']);
grunt.registerTask('default', ['build']);
};
So far, this is just the standard boilerplate, we need to export a function and inside we include the three dependencies we added to the package.json file, and then we register the tasks. We will have a serve task which will act like a server and let us view the separate files together while we develop our site and then we have the build command which will combine all the HTML and CSS in our app together for deployment. I also added the last line, so if you just run grunt by itself, it will assume you want to build.
Now, let’s start with the configuration for the build command as it’s a lot shorter:
concat: {
dist: {
src: ["app/header.html", "app/menu.html", "app/sections/*.html", "app/footer.html"],
dest: "build/index.html"
}
},
cssmin: {
css: {
files: {
"build/css/main.css": ["app/css/*.css"]
}
}
}
For the HTML, we are just going to concatenate all the HTML files together in the order specified into the build folder under the name index.html. With the CSS, we also want to minify it, so we are using the cssmin plugin and we are specifying that we want to output a file called main.css inside the build/css folder and we want it to include the minified versions of all the CSS files in our project.
Connect Server
Next, we need to add the configuration for the Connect server, connect provides a lot of great middleware for serving static files or directories, but here we need to combine all the files first, so we are going to have to create some custom handlers. To begin, let’s put the base configuration in:
connect: {
server: {
options: {
keepalive: true,
open: true,
middleware: function(){
//custom handlers here
}
}
}
},
The keepalive option tells Grunt to keep the server running, the open option will tell Grunt to open the URL in your browser automatically when you start the server (it’s more of a personal preference though) and the middleware function is meant to return an array of middleware handlers to process the requests.
In our application, there are basically two resources we need to handle, the root resource (our “index.html”) in which case we need to combine all our HTML files and return them and then the “main.css” resource, in which case we would want to return all the CSS files combined together. As for anything else, we can just return a 404.
So let’s start by creating an array for the middleware and let’s add the first one (this goes inside the middleware property’s function, where I placed the comment):
var middleware = [];
middleware.push(function(req, res, next) {
if (req.url !== "/") return next();
res.setHeader("Content-type", "text/html");
var html = grunt.file.read("app/header.html");
html += grunt.file.read("app/menu.html");
var files = grunt.file.expand("app/sections/*.html");
for (var i = 0; i < files.length; i++) {
html += grunt.file.read(files[i]);
}
html += grunt.file.read("app/footer.html");
res.end(html);
});
We have kind of shifted from Yeoman’s API to Grunt’s, but luckily the command names are almost identical, we are still using read to read files and expand to get the list of files. Besides that, all we are doing is setting the content-type and returning the concatenated version of all the HTML files. If you are unfamiliar with a middleware based server, basically it will run through the middleware stack until somewhere along the line, a function can handle the request.
In the first line, we check if the request is for the root URL, if it is we handle the request, otherwise we call next() which will pass the request down the line onto the next middleware.
Next, we need to hand the request to /css/main.css which, if you remember, is what we set up in the header:
middleware.push(function(req, res, next){
if (req.url !== "/css/main.css") return next();
res.setHeader("Content-type", "text/css");
var css = "";
var files = grunt.file.expand("app/css/*.css");
for (var i = 0; i < files.length; i++) {
css += grunt.file.read(files[i]);
}
res.end(css);
});
Basically, this is the same function as before except for the CSS files. Last, but not least, I will add a 404 message and return the middleware stack:
middleware.push(function(req, res){
res.statusCode = 404;
res.end("Not Found");
});
return middleware;
This means if any requests don’t get handled by the previous two functions, then we will send this 404 message. And that’s all there is to it, we have the generator which will create the project and the Grunt tasks which will allow us to preview and build our app. The last topic I want to cover briefly, is sub generators.
Sub Generators
We have already created the main generator that builds out our application but Yeoman allows you to create as many sub-generators as you want to use after the initial scaffolding. In our application, we need one to generate new sections. To get started, we can actually use a sub generator from the generator:generator to scaffold the file, to do this, just run the following command from inside our onepage folder:
yo generator:subgenerator section
This will create a new folder called section with an index.js file and a templates folder just like our main (app) generator. Let’s empty out the generator like we did last time and you should be left with the following:
'use strict';
var util = require('util');
var yeoman = require('yeoman-generator');
var SectionGenerator = yeoman.generators.NamedBase.extend({
});
module.exports = SectionGenerator;
You may also notice we are extending from the NamedBase not the regular Base, that means you have to supply a name parameter when you call the generator and you can then access that name using this.name. Now this code is essentially the two functions we already wrote in the previous generator: generateDemoSection and generateMenu, so we can just copy those two functions here. I will replace the name of the first one to something more appropriate:
generateSection: function(){
var context = {
content: this.name,
id: this._.classify(this.name)
}
var fileBase = Date.now() + "_" + this._.underscored(this.name);
var htmlFile = "app/sections/" + fileBase + ".html";
var cssFile = "app/css/" + fileBase + ".css";
this.template("_section.html", htmlFile, context);
this.template("_section.css", cssFile, context);
},
generateMenu: function(){
var menu = this.read("_menu.html");
var t = '<%= name %>';
var files = this.expand("app/sections/*.html");
for (var i = 0; i < files.length; i++) {
var name = this._.chain(files[i]).strRight("_").strLeftBack(".html").humanize().value();
var context = {
name: name,
id: this._.classify(name)
};
var link = this.engine(t, context);
menu = this.append(menu, "div.menu", link);
}
this.write("app/menu.html", menu);
}
The only difference is instead of entering the name directly, I am using the this.name property. The only other thing is we need to move the template files _sections.html, _section.css and _menu.html from app/templates to section/templates as that is where the template/read commands will look.
Code Duplication
The problem now is code duplication. So back in the app/index.js file, instead of it having the same code as the sub generator, we can just call the sub generator and pass it the name argument. You can remove generateMenu from app/index.js altogether and we will modify generateDemoSection to the following:
generateDemoSection: function() {
if (this.addDemoSection) {
var done = this.async();
this.invoke("onepage:section", {args: ["Demo Section"]}, function(){
done();
});
} else {
this.write( "app/menu.html", "");
}
},
So if the user wanted to create the demo section, then we invoke the sub generator passing in the first argument which is the name. If the user did not want the demo post, we still need to create something for the menu because our Grunt tasks will try and read it, so in the last section, we just write an empty string to the menu file.
Our generator is finally complete and we can now test it out.
Testing It Out
The first thing we need to do is install our generator on your system. Instead of installing the gem regularly, we can use the link command to just link the folder into the gem path, that way we can continue to make changes here without needing to re-install it every time. From the project directory, simply run nom link.
The last step is to actually run the generator. Just create a new folder and inside run yo onepage this will run our main generator and install the dependencies via npm. We can then use our Grunt tasks to view the page (grunt serve) or add some more sections with our generator using something like yo onpage:section "Another section" and the new files will be added.
Conclusion
In this article, we covered a lot of the common features but there are still more interesting options to check out. As you can probably tell, there is a bit of boilerplate required when building a generator, but that is kind of the point, you get it all done once and then you’re able to use it throughout all your applications.
I hope you enjoyed reading this article, if you have any questions or comments, feel free to leave them below.
