We need the InitPalette and GetColour functions from last time, though the InitPalette function will now store the 216 colours as JavaScript objects (with r, g, and b components) rather than Notch's encoded version:

// Before:
colours.push(r1 << 16 | g1 << 8 | b1);

// After:
colours.push({
  r: r1,
  g: g1,
  b: b1
});

This saves us from having to decode the value before we use it - which is much more practical in JavaScript, because the canvas image data is stored as separate rgba values by default.

On with the new stuff. First up is to make sure we can draw pixels to our canvas element. The process goes like this...

1. Get the canvas's image data with the createImageData function.
2. Manipulate the bits of the data as we desire
3. Replace the canvas data using the putImageData function
4. Repeat from step 2

All the JavaScript you see is "inline" for simplicity. All functions are global (and for some reason I uppercased the function names - I sometimes go a bit weird when writing tutorials). A lot of arrays are passed by reference... It's for demonstration purposes only! Namespace things, mix'em in, plaster some classical OOP on them, make them into endo-functing monads... REMEMBER: This isn't production code, it was just me dissecting and porting Notch's Minicraft effort!

Drawing pixels with canvas

Right, first we grab the canvas context and keep a reference to the image data. We'll also populate our palette, ready for using.

<canvas id="board" width="160" height="120"></canvas>
...
var ctx = document.getElementById("board").getContext("2d"),
    output = ctx.createImageData(ctx.canvas.width, ctx.canvas.height),
    colours = [];

// Set up our palette
InitPalette(colours);

Then we run a loop where we pick a random colour index (from the 216 we generated) and put it in each pixel. The canvas data is stored 4 bytes per pixel (rgba) and we put 256 for alpha (opaque).

(function run() {
  for (var y = 0; y < ctx.canvas.height; y++) {
    for (var x = 0; x < ctx.canvas.width; x++) {

      // Pick a random palette colour
      var cc = ~~(Math.random() * 216),
          xo = x * 4,
          yo = y * ctx.canvas.width * 4,
          col = colours[cc];

      // Replace the canvas image pixel
      output.data[xo + yo] = col.r;
      output.data[xo + yo + 1] = col.g;
      output.data[xo + yo + 2] = col.b;
      output.data[xo + yo + 3] = 256;
    }
  }

  // Draw it
  ctx.putImageData(output, 0, 0);
  setTimeout(run, 50);
})();

Alright! We have some palatte-ized noise on the screen.

Loading an image

Next, we need to load in the grey-scale graphic file (which is the spritesheet from Minicraft). To load an image as canvas data, we'll create a regular ol' JavaScript image then "paint" it onto a canvas element. Then extract the data from the canvas element and discard the JavaScript image and the canvas element. We just want the datas.

function LoadSpriteSheet(filename, callback) {
  var img = new Image(),
      pixels = [];
  img.onload = function() {
    var can = document.createElement("canvas"),
        ctx = can.getContext("2d");
    can.setAttribute("width", img.width);
    can.setAttribute("height", img.height);
    ctx.drawImage(img, 0, 0);

    var data = ctx.getImageData(0, 0, img.width, img.height).data;

    // The important bit!
    for(var i = 0; i < data.length; i += 4) {
      pixels.push(~~(data[i] / 64));
    };

    // Send it back
    callback && callback({
      width: img.width,
      height: img.height,
      pixels: pixels
    });
  }
  img.src = filename;
}

If you're playing along at home, remember you need to have a web server running to load the image (you can't run this sample from a file:/// url) otherwise you'll get a cross-domain error when trying to manipulate the canvas data.

There's one weird bit in there that is specifically for our colourising code: first, because we only care about greyscale for now - we only look at every 4th byte in the data (because if it's greyscale then r=g=b). And second, every byte that we keep we first divide by 64.

WTF?! Ok, it's a Notch-ism... When he was creating the graphics in his graphics editor, he carefully chose 4 rgb colour values that when divided by 64 (and floored) would return either 0, 1, 2, or 3! That's a handy index for us to work with in code, but at the same time - a decent scale to work in in photoshop.

What we end up with then, is an array sized width * height of integers from 0 to 3. To actually load an image with this function we pass the path and a callback function (because the image load is asynchronous).

var spritesheet = null;
LoadSpriteSheet("sheet.png", function(sheet) {
  spritesheet = sheet;
  run();
});

We then call our run function. For this demo we will blit the spritesheet directly to the output canvas image - this code is almost identical to the "random static" code - except instead of a random colour index, we read the integers from the spritesheet. Because this will only give us a colour from 0 to 3 (which would be from black to very very dark someother colour) we multiply it by, say, 10 - and get the colour from the palette.

function run() {
  for (var y = 0; y < ctx.canvas.height; y++) {
    for (var x = 0; x < ctx.canvas.width; x++) {
      var cc = spritesheet.pixels[x + (y * spritesheet.width)],
          xo = x * 4,
          yo = y * ctx.canvas.width * 4,
          col = colours[cc * 10]; // make it brighter!

      output.data[xo + yo] = col.r;
      output.data[xo + yo + 1] = col.g;
      output.data[xo + yo + 2] = col.b;
      output.data[xo + yo + 3] = 256;
    }
  }
  ctx.putImageData(output, 0, 0);
  setTimeout(run, 50);
};

What you see now is a strangely colourised version of part of the spritesheet. We're getting closer!

A kind of double-buffer

Right, in the next step we'll create an extra array, the same size as the screen, that will represent... the screen. It will hold screen size, the spritesheet, and the pixels to eventually render. All the sprite rendering we will do to this array, and then at the end copy it to the output canvas element.

buffer = {
  width: ctx.canvas.width,
  height: ctx.canvas.height,
  sheet: null,
  pixels: []
},

When the spritesheet is loaded, it is stored in the buffer too (instead of the global variable we made last time). To render the buffer to the output canvas we do exactly the same as before - but pass in the arrays to copy from and too.

function render(buffer, pixels) {
  for (var y = 0; y < buffer.height; y++) {
    for (var x = 0; x < buffer.width; x++) {
      var cc = buffer.pixels[x + (y * buffer.width)],
          xo = x * 4,
          yo = y * buffer.width * 4,
          col = colours[cc];

      if(!col) continue; // Make sure it's defined
      pixels.data[xo + yo] = col.r;
      pixels.data[xo + yo + 1] = col.g;
      pixels.data[xo + yo + 2] = col.b;
      pixels.data[xo + yo + 3] = 256;
    }
  }
}

Now we can re-create our original static - but utilising the buffer:

for (var y = 0; y < buffer.height; y++) {
  for (var x = 0; x < buffer.width; x++) {
    buffer.pixels[x + y * buffer.width] = ~~(Math.random() * 216);
  }
}

// Transfer screen to output buffer
render(buffer, output);
ctx.putImageData(output, 0, 0);

A small step backwards - but now we're rendering noise with our back buffer.

Plotting some sprites

With our buffer in hand, we can now do some real work! Anything we put into the buffer will end up on the screen - so now we can copy pieces of the spritesheet data straight into the buffer array. We'll make a renderTile function that copies an 8 by 8 pixel rectangle from spritesheet to a given x and y location. This code is ported pretty much directly from Notch's screen.render function.

function renderTile(buffer, xp, yp, tile, colors) {
  var xTile = tile % 32,
      yTile = ~~(tile / 32),
      toffs = xTile * 8 + yTile * 8 * buffer.sheet.width;

  for(var y = 0; y < 8; y++) {
    if (y + yp < 0 || y + yp >= buffer.height) continue; // make sure the pixel is on screen.
    for(var x = 0; x < 8; x++) {
      if (x + xp < 0 || x + xp >= buffer.width) continue; // make sure the pixel is on screen.
      var col = (colors >> (buffer.sheet.pixels[x + y * buffer.sheet.width + toffs] * 8)) & 255;
      if (col < 255) buffer.pixels[(x + xp) + (y + yp) * buffer.width] = col;
    }
  }
}

To render a tile, we pass in the buffer, the position to render, the tile index, and the colour we want to tint the tile with. Most of the code just figures out the correct offset from the spritesheet (notice the magic number "32"? that's because the sheet is 32 x 32 tiles. The magic number 8? Our tiles are 8 x 8 pixels).

But the important line is this whole procedure is this: var col = (colors >> (buffer.sheet.pixels[x + y * buffer.sheet.width + toffs] * 8)) & 255;

This is where the colourising magic happens. The pixel value from 0 to 3 is found from the spritesheet. This is then multiplied by 8 to give either the value: 0, 8, 16, or 24. The colour number is then right shifted by that amount and anded by 255.

Does this sound kind of familiar? It's the inverse of what the ol' GetColour function did - which encoded 4 colour values into 1 integer. This is just doing the opposite: extracting one colour out of the integer and applying it to the correct pixel. The next line then checks that the value is not 255 - which will be our "invisible" colour for transparent areas.

We can call this function for each 8x8 sprite/tile we want to render. The tile number is the integer index into the spritesheet: 0 is the first tile in the first row, 1 is the second tile in the first row, 32 (1 * 32 + 0) is the first tile in the second row, 33 (1 * 32 + 1) is the second tile in the second row...

renderTile(buffer, 20, 20, (5 * 32) + 0, GetColour(-1, 100, 320, 222));
renderTile(buffer, 30, 20, (5 * 32) + 2, GetColour(-1, 100, 320, 222));
renderTile(buffer, 40, 20, (5 * 32) + 3, GetColour(-1, 100, 320, 222));

Ah, that's more like it - Three colourised Minicraft tools!

Messing around with it

With the ability to plot tiles and sprites at a given location - we have all we need to make some games! In each of your entity's render functionality you can call the renderTile as many times as you need. For example, to render a background over the whole screen:

for(var y = 0; y < buffer.height / 8; y++) {
    for(var x = 0; x < buffer.width / 8; x++) {
        renderTile(buffer, x * 8, y * 8, 1, GetColour(0, 130, 120, 0));
    }
}

You can plot individual tiles to make a font plotter, or stitch a few tiles together for bigger sprites... whatever you want. Here's the code for the weird demo from the beginning of the article. Notice that the Minicraft players are actually 4 tiles in a square:

var rnd = function(min, max){ if(!max){ max=min; min=0; }; return ~~(Math.random()*(max-min))+min;},
    rndCol = function(){ return parseInt(rnd(5) + "" + rnd(5) + "" + rnd(5), 10); };

// Render peoples
var midX = ~~(Math.sin(new Date().getTime() / 400) * 20) + 70,
    midY = ~~(buffer.height / 2) - 8,
    col1 = GetColour(-1, 1, rndCol(), rndCol()),
    col2 = GetColour(-1, 1, 23, 555);

// Man 1
renderTile(buffer, midX + 16, midY, (14 * 32) + 0, col1);
renderTile(buffer, midX + 24, midY, (14 * 32) + 1, col1);
renderTile(buffer, midX + 16, midY + 8, (15 * 32) + 0, col1);
renderTile(buffer, midX + 24, midY + 8, (15 * 32) + 1, col1);

// Man 2
renderTile(buffer, midX, midY, (16 * 32) + 0, col2);
renderTile(buffer, midX + 8, midY, (16 * 32) + 1, col2);
renderTile(buffer, midX, midY + 8, (17 * 32) + 0, col2);
renderTile(buffer, midX + 8, midY + 8, (17 * 32) + 1, col2);

Check out the Minicraft source (or here on GitHub) - especially the render functions of each entity (eg player.java) to see how Notch implements animation and rendering based on the player's direction, if they are attacking, etc.

Anyhow... that's it. I'd highly recommend not using any of the code above, but re-implement it yourself from your brain... this code is already ported from Java which was written in a 48 hour game competition, by a mad man - so do yourself a favour and make your own (more idiomatic) version!

It's worth it I think... dynamically colourising sprites like this is a great way to prototype the look and feel of your game, add in old-school colour-cycling effects, and easily add variety to your entities.

Right, now, on to todays topic: "Colour like you're Notch" (Awright, punning on my own blog post titles!). While watching the livestream of Notch coding Minicraft, I was intrigued by his dynamic sprite colorising. After a little reverse engineering and experimentation I realised it was dang simple, and we could apply something similar to HTML5 games.

The idea is that you create your sprite sheet using only 4 colours. At runtime, you replace the 4 colours with any colours you like - so you can quickly modify the look and feel of the game, or have varied characters that differ only by colour - just like the ol' NES games.

While 95% of Notch's code is extremely straightforward and obvious, the colour system got a bit hairy due to the fact he faffed around with the implementation a few times to get the result he wanted. The end result seems more complex than it really is. The overview of the system goes like so:

1. Set up a limited (256 colour) indexed palette to be used in the game. The colour channels will be divided into 5 "shades".
2. Draw the sprites in 4 colours. When loading the image into the game, reduce the pixels to values from 0 to 3 to represent the colour.
3. When drawing a sprite to the screen, dynamically pass in the 4 colours to use.
4. For each pixel, find the index in the palette and replace the pixel with that colour.
5. Profit!

The trickiest part is realising that Notch just made up a custom colour selecting system. Throughout the Minicraft code you'll see lots of instances of a line like: Color.get(100, 232, -1, 555) - although it kind of looks like some kind of RGBA format, it's really defining 4 separate colours, each made up of 3 channels. The number 100 represents r = 1, g = 0, b = 0 and would be a very dark red. 555 represents r = 5, g = 5, b = 5 and would be white (full red, full green, and full blue). So let's see it in action...

In Game.java, the colour palette is set up. It uses 6 bits per channel to get our 256 colours in 5 shades. Here's the directly ported JavaScript version (well, except that I'm passing in the number of bits to use, so you can play with it):

function InitPalette(colours, bits) {
  // Set colours
  for (var r = 0; r < bits; r++) {
    for (var g = 0; g < bits; g++) {
      for (var b = 0; b < bits; b++) {
        var rr = r * 255 / (bits - 1),
            gg = g * 255 / (bits - 1),
	    bb = b * 255 / (bits - 1);

        // Set the mid tone
        var mid = (rr * 30 + gg * 59 + bb * 11) / 100;

        // Tint the palette
        var r1 = ~~(((rr + mid * 1) / 2) * 230 / 255 + 10),
            g1 = ~~(((gg + mid * 1) / 2) * 230 / 255 + 10),
            b1 = ~~(((bb + mid * 1) / 2) * 230 / 255 + 10);

        // Push RGB values into a single integer
        colours.push(r1 << 16 | g1 << 8 | b1);
      }
    }
  }
}

You can have a look at the palette that's generated by this function with this jsFiddle. One cool thing to notice is the tinting process. Try changing the final line to push the non-tinted values (rr instead of r1, gg instead of g1 etc). Without the tinting the game colours would look like this. Quite a nice way to give the game a "feel". Here they are side-by-side - the top one is tinted to be more "pastel-ly".

The final line colours.push(r1 << 16 | g1 << 8 | b1) converts the RGB values into a single integer (the shift-left operator moves the red bits up the top, the green bits in the middle, and leaves the blue bits at the end). If you want to use this in your HTML5 games, then it'd be better to skip this encoding and store each colour as a simple object - because we just need to decode it later (perhaps the shift version is faster in JavaScript too? Might need a bit of testing!).

Anyway - now we have an indexed palette of colours. To get the colours back at our will we need to use the accessor methods Notch created in Color.java. Here's a JS version:

function GetColour(a, b, c, d) {
  function getIndex(d) {
    if (d < 0) return 255;
    var r = ~~(d / 100) % 10; // First digit
    var g = ~~(d / 10) % 10; // Second digit
    var b = d % 10; // Third digit
    return r * 36 + g * 6 + b;
  }

  return (getIndex(d) << 24) + (getIndex(c) << 16) + (getIndex(b) << 8) + (getIndex(a));
}

Notch's version is the same code, but he uses an overloaded get method for fetching the indexes. The final line returns a single integer like in the palette generation - it pushes the first colour 24 bits up the integer, the second colour 16 bits up etc. The value that it pushes is the index in the palette. This is calculated by the getIndex function by turning the decimal number into 3 channels. The function will accept a number between 0 and 555 (5 x reds, 5 x greens, and 5 x blues). If you put a number higher than 555 into it, it'll die. So don't do that.

Now we can get a single number to represent our 4 colours. GetColour(500, 50, 5, 505) == -1190846796 (Be careful, it's tempting to put 500, 050, 005, 505 so it looks nice, but remember - a leading 0 mean the number is octal in JavaScript, so 050 will actually be 40 in decimal).

Colouring in

Now that we have a way to encode the colours, we need to be able to decode them. Which is just a matter of reversing the getColour method, really: pop out the values and find the palette index! To show it in action I'll just draw 4 rectangles in the four colours specified by GetColour:

function DrawColours(ctx, colourInt, palette) {
    for(var i = 0; i < 4; i++) {
        var colIndx = (colourInt >> i * 8) & 255,
            col = palette[colIndx],
            b1 = col & 255,
            g1 = (col >> 8) & 255,
            r1 = (col >> 16) & 255;
        ctx.fillStyle = "rgb(" + r1 + "," + g1 + "," + b1 + ")";
        ctx.fillRect(i * 10, 0, 10, 10);
    }
}

DrawColours(ctx, GetColour(510, 0, 555, 55), colours);

The result is a red-ish rectangle, a black, a "white", and an aqua blue rectangle. The decoding part can be found in the Minicraft source in Screen.java in the render function. The original code looked like this:
int col = (colors >> (sheet.pixels[xs + ys * sheet.width + toffs] * 8)) & 255;. First the pixel colour (for 0 to 3) is found in the sprite sheet that corresponds with the current tile. This is the "i" from our DrawColours function. The number is then multiplied by 8 to get the correct distance to shift by (0, 8, 16, or 24 bits). The result is the number we pushed in with the GetColour call - which was the index of the palette to use. The actual colour components can then be extracted and drawn.

Ok, that's it for this post - it's getting too long. As you can see, the system is very simple - it's just encoding and decoding palette indexes. In the next post I'll show you how to load in the sprite sheet, covert it to the correct 0 - 4 format we need, and plot the colourised tiles. Sounds exciting, hey?

First, open up Textmate and go to Bundle Editor then Edit Languages:

Next find JavaScript and find the key/value pair for variable.language.js. Here you'll see that super and this are lumped together. All we need to do is add our friend (and an additional pipe symbol) to the list:

That's all that's required to make _this a first-class, syntax-highlighted, citizen. Now your callbacks are nice and legible.

And there we go! You can also add in self if you're not a _this fan.

I am.

And in case you weren't convinced by my compelling arguments - Dave Sims leaves you with this piece of food-for-thought from The Deer Hunter...

In the last few years or so, JavaScript has come out of the closet and revealed itself as a powerful (though, like any good hero, slightly-flawed) functional language. Many people still haven't taken to it and choose to continue writing in an OOP style that they were told was good and enterprise-y. That's not a bad thing - that C/C#/Java coders can force JavaScript to behave mostly-but-not-exactly like their familiar ol' imperative friends is testament to it's flexibility: JavaScript can easily pretend to be imperative, because it can do anything.

So you can force JavaScript to have classical OOP inheritance (rather than the prototypical inheritance that is true to it's heart) and you can go on pretending you're coding Java and swearing when something doesn't work like your used to. And you can keep on swearing when you inherit a pile of spaghetti code written by the newbie who used to have your new job.

But heck, the fact that a designer somewhere can pick up enough JavaScript to actually achieve their goal is fantastic. The fact that most JavaScript code makes you want to cry is not so fantastic - but it's the natural consequence of having all but a tiny fraction of one percent of available JavaScript tutorials written by people who don't get JavaScript.

Perhaps what's needed it to really cast off the ugly baggage that's holding it back. Harmony proposes some great new features, and the less certain "Strawman" proposal puts shorter function syntax, and arrow function syntax on the table - but I think if we're really going achieve some real understanding we need to simply rename the entire language LispScript.

Once JavaScript gets prefix notation, the transformation will be complete.

It does this by establishing an abstraction: creating a language that can be used to describe our requirements. If we think about (or re-think) our problems in terms of this language then it becomes easier to decompose and "mix and match" component parts as needed. It turns out that you can restructure a lot of your code using this simple language.

In Structure and Interpretation of Computer Programs (Go and read or watch SICP now - it's free. I didn't realise I knew nothing about programming until I was about halfway through this book) this concept is referred to as creating "conventional interfaces".

As an example, I've dissected a real world problem below. The problem is broken into exercises that can be run and modified on the spot. Each exercise includes some "things to try" - some small tests - which will hopefully reinforce the ideas and make everything nice and clear.

The problem

The HN bookmarklet runs on the Hacker News main page and refreshes the (normally static) view every few minutes. Changes in article rank, and votes are highlighted as to provide an overview of changes at a glance.

For this case study we'll concentrate on the piece of code that removes the indicators for "runs of rising articles": after the new HN page is loaded and the rise and fall indicators have been added, I take all of the "rises" - the green circles that indicate a story has risen - and look for runs of same-valued rises. If a run (more than 2) articles are found, the indicators fade away. The reason is that a run is (probably) caused by some other story nose-diving, therefore every other article above it gets a bump - and the page has a long trails of green dots on the page that aren't very informative.

An answer

My first solution was to just loop over each rising element using nested loops. If the stories rank was 1 above the previous, and the change the same as the previous, then it was added to a temporary array. When the order was broken then temporary array was iterated and all the contained elements were removed from the page. Then the process was started again until all elements were checked.

It seemed to work, but I started to wonder if jQuery's find method always returned the elements in the same order as the DOM nodes on page. I couldn't find anything that confirmed or denied it, so I decided it would be wise to sort the articles by rank before I did the run checks. Throwing a sort into the mix really uglified my code (and I wasn't liking that local state variable anyway) - so a new strategy was formulated. Map/reduce!

The beauty of this approach is that once you've decomposed the problem in the correct terms, you can easily start mashing things together. If you suddenly need an extra sort, or and additional check, then it's just an extra black box in the middle of a chain of operations. Here's how we'll tackle our problem in map/reduce language: Read the following code as a paragraph, substituting any periods with the word "then":

$("getAllOfTheGreenCircles")
    .map( the DOM elements in to nice JavaScript objects )
    .sort( them in order of rank )
    .reduce( the elements into arrays of runs )
    .filter( out all of the long runs )
    .reduce( the long runs into a one dimensional array )
    .forEach( element in the array, remove it )

That's the plan - we just need to write some functions in place of the english descriptions.

Each step will be explained below with some runnable code. Once we've completed each step (for example, after we cover the first "map") then in subsequent examples the code will be replaced with a named function - so it's nice and short and we can concentrate on the proceeding step. The named functions are editable way down the bottom of this page, so once you've done the exercises you can modify and mix & match them.

Because our problem is based on "real" code, I'll use markup from the Hacker News bookmarklet. The bookmarklet injects elements in front of Hacker News articles that have changed. The indicators are green or pink - green is a rising article, pink is a sinking one. The number shows how many rank position it has moved. Next to the indicator is the actual rank of the article at the moment.

For this case study, we'll be concerned with the small green circles indicating that an article has risen, and it's corresponding rank.

Step 1: Enumerating with jQuery

Before we can start mashing up our list elements we need a list. All of the green "rises" have the class name hnu-up and we can fetch them with jQuery. Because jQuery returns a jQuery list, we have to get the real underlying array by calling .get() on the result.

Click the "run" button below to execute the code. You can modify the code to see your changes in action - but I haven't put too much error checking in place so it will be quite easy to smash this page, and you'll need to refresh.

$(".hnu-up").get()

Nothing very interesting about step 1, but we can see how the code runner works: the code is read and evaluated, and the results are displayed below. In the output, you can click on the output line numbers to see which DOM objects in the article list have been affected. For each exercise that follows, the article list will follow you down the page.

Things to try

• Remove the call to .get() and see the difference. The elements are the same, but on is wrapped in a jQuery object. If we don't have a real JavaScript array then we won't be using the native JavaScript functions.

Also, for those brave souls not using jQuery - if you try selecting the spans with document.querySelectorAll then you'll receive a "nodelist", not an array. To get a real array you need to slice the results: Array.prototype.slice.call(nodelist).

Step 2: Mapping with map

The Map function operates on an array - running each element of the array through a function that you define. The return value of the function becomes the new element in the list. We say that the function maps elements to their new value. For an example, if we wanted to map an array of integers to a new array where each element was tripled:

[1, 2, 3, 4].map(function(el){ return el * 3; });

gives a new array with values: [3, 6, 9, 12]. We only used the first parameter (el) above, but the callback function actually gets given 3 parameters for each element: the array element itself; the element's index in the array (zero indexed, of course), and the entire array (for some reason. Well, actually good reasons, but you won't need this too often to begin with).

Our problem description was to map( the DOM elements in to nice JavaScript objects). In step 1 we got the array of DOM elements, and now we want to extract the juicy data values it contains. For each element we'll return an object literal with the element's "rise", its rank, and a jQuery wrapped object so we can manipulate it easily later on.

$(".hnu-up")
.get()
.map(function(el) {
    var rank = el.nextSibling.textContent;
    return {
        rise: parseInt($(el).text(), 10),
        rank: parseInt(rank.replace(/[^0-9]/g,""), 10),
        $: $(el)
    };
});

This code is a bit weird looking - as we had to do some "DOM level" stuff with the nextSibling and textContent. This is because jQuery doesn't really do text nodes - but Hacker News' markup required it. Because I wanted to keep this case study real I didn't over-simplify it.

The el item our callback receives is the circle DOM node from jQuery - so to get the "rise" we just grab its text value and parse it to an integer. The rank is similar, but requires a lil' bit of regex-ing to get rid of phantom quotes that appear when you manipulate whitespace elements.

The result of map will always be a new array with the same length as the original. The items might be exactly the same as the input array, or like our example, might be completely different. Because the result is an array, it can then be further manipulated by array methods - such as another map!

jQuery contains a couple of it's own map functions that work similarly (though not identical) to the JavaScript version. If you're interested, read of my comparison of jQuery vs jQuery vs JavaScript.

Things to try

• Replace the ".hnu-up" elements with the down elements: $(".hnu-down").
• Remove the final semi-colon and add another map function that doubles each item's rank.
• Instead of our funky object, return a simple array of "rank" integers [1,2,3,5,6,7].

Step 3: Sorting with sort

Next up, we need to sort( them in order of rank ). Sorting has been in JavaScript forever so we won't spend too much time here. If you call sort() on an array it will try and sort the elements alphabetically or numerically but you can also specify your own comparer function.

The function compares each item "a" with each other item "b". If the return value is less than 0, then "a" comes before "b". If it's greater than 0 then "b" comes before "a". Otherwise, they are equal in sort importance. We want to sort our list in order of rank, so we subtract the "a" rank from the "b": a.rank - b.rank.

$(".hnu-up")
   .get()
   .map(extractValuesFromElement)
   .sort(function(a, b) {
       return a.rank - b.rank;
   });
       

Note that I've replaced the anonymous map function we wrote in the last example with a named function "extractValuesFromElement" - it's the same code, but naming it makes these un-syntax-highlighted textboxes much easier to decipher! The code is defined and editable below in the "named functions" section.

Things to try

• Sort the elements in descending rank order.
• Sort the elements by "rise".

Step 4: Reducing with reduce

Ok, the big one. Reduce can seem a bit weird at first - especially because it's a bit of a "swiss army knife" whose function can't be summed up as easily as map, or filter. Reduce is sometimes called "fold", or "compress" (or lots of other names) which suggest that it makes arrays smaller in length - and they usually do. The purpose is build up a single result based on the values of all the individual elements. The most common example is adding all of the elements of an integer array:

[1,2,3,4].reduce(function(accumulator, element) {
    return accumulator + element;
}, 0);

Returns 10. I find reduce most easy to use when I name the parameters acc (for the accumulator) and el (for the element). The accumulator is passed along as to each element so you can keep track of what's going on. In the example above we gave it the initial value of 0, and then each element got added to this running total.

But the accumulator does not have to be an integer. Reduce is often used for building up "lists of lists". We need this functionality to achieve our step 4 goal to reduce( the elements into arrays of runs ).

Our input is the sorted list of all elements, and our output will be a list of "list of runs". We'll use reduce to group sequential elements with the same rise value. This is going to involve a bit of array manipulation. We start with an empty list that we pass as the accumulator. For each element, we get the value of the last run, or create a new empty list.

If the element is "in order" then we append it to the existing list, otherwise we add it to a new list that contains just itself. The test for "in order" is a small function that checks that the new rank is one more than the previous element's, and the rise is the same.

$(".hnu-up")
.get()
.map(extractValuesFromElement)
.sort(sortByRank)
.reduce(function(acc, el) {
    var head = acc.length ? acc[acc.length - 1] : [],
        isSeq = function(prev, el){
            return el.rise === prev.rise &&
                el.rank === prev.rank + 1;
        };
    if(head.length && isSeq(head[head.length - 1], el)) {
        head.push(el);
    }
    else {
        acc.push([el]);
    }
    return acc;
}, []);
    

The output shows we now have reduced our array to 3 elements. Each element is an array of items that are in sequentially ranked with the same rise. If you click on the "0:" or "2:" you can see the grouping is correct.

Reducing is used to tally or group - so if you find yourself looking for totals, or trying to push elements together - reduce is your man. If remember the parameter names "accumulator and element" then things will be clearer when you look at examples that unhelpfully label them "a and b"!

Things to try

• Modify the internal isSeq function to only check that the rise is the same (remove the "rank" clause).
• Modify it to only check that the rank is sequential, and don't check for "rise".

Step 5: Filtering with filter

Now we're cooking. We have an array of arrays containing elements that we potentially want to remove. 2 out of the 3 arrays have more than one item in them - these are the ones we are interested in. To get them we need to filter( out all of the long runs ). Filter is very similar in structure and use to map, but instead of modifying elements, we return a boolean value that indicates if we wish to keep the element in the new array. Our output array will therefore either be the same length (if everything matches the test), or shorter.

Our definition of a "long run" is a sequence of 2 or more items. Therefore, we test the element and return true if it's length is greater than 1.

$(".hnu-up")
.get()
.map(extractValuesFromElement)
.sort(sortByRank)
.reduce(groupIntoRuns, [])
.filter(function(el) {
    return el.length > 1;
});
    

After applying our filter, we are left with arrays of long runs. Like map, we also are given the array index and complete array if we need it. Index is particularly useful for filtering, as you we do things like return index < 3 to get the first 3 elements, or return index % 2 === 1 to leave only every other element.

Like map, jQuery has it's own filter method for filtering lists of jQuery objects. It can work the same as the JavaScript version but more commonly it's used to filter based on a CSS selector string.

Things to try

• Return only elements that have runs of 2 or less.
• Go back to exercise 1 and select every other (or every 3rd) element.

Step 6: Flattening with... reduce?

We're still not quite there yet. Although we know all of the elements we want to remove, they are stored inside arrays. We could do a nested loop and delete them, but that's looking pretty ugly to us now. A cleaner way is to reduce( the long runs into a one dimensional array ). Ok, obviously I'm saying we need to use reduce, but in many functional languages "reducing an array of arrays to an one dimensional array" has a name: it's called flatten.

Thankfully flatten is very easy to write with reduce (and in most languages it actually is). Our accumulator starts as an empty list, and for each element (each array) we just concatenate it to the accumulator.

$(".hnu-up")
.get()
.map(extractValuesFromElement)
.sort(sortByRank)
.reduce(groupIntoRuns, [])
.filter(returnAnyLongRuns)
.reduce(function(acc, el) {
    return acc.concat(el);
},[]);
    

The result: a flat array containing the five elements that have gots to go. The flatten-via-reduce is quite idiomatic, so be sure to commit it to memory.

Step 7: Iterating with forEach

All that's left do now is eradicate our green circles. The approach is simple: forEach( element in the array, remove it ). The forEach method applies a given function to every element in the collection. It's like a more beautiful for loop.

$(".hnu-up")
.get()
.map(extractValuesFromElement)
.sort(sortByRank)
.reduce(groupIntoRuns, [])
.filter(returnAnyLongRuns)
.reduce(flattenRuns,[])
.forEach(function(el) {
    el.$
        .fadeOut("slow")
        .delay(2000)
        .fadeIn();
});
    

Luckily, way back in exercise 2 we had the forethought to keep a reference to the DOM element's jQuery selector just for this moment. We fade it out slowly so the user sees what we are doing. Then, because it's just an example, we bring it back so you can keep on playing.

Things to try

• Replace the anonymous function with the named removeIndicator function (listed below).
• Re-write all of your code you've ever written with map/reduce and friends.

Named Functions

Nothing to see here: these are the named functions we've been substituting above to keep code nice and short. If you make changes to the functions be sure to hit the "update functions" button to update them (they're just global functions).

// Functions for examples
extractValuesFromElement = function(el) {
    var rank = el.nextSibling.textContent;
    return {
        rise: parseInt($(el).text(), 10),
        rank: parseInt(rank.replace(/[^0-9]/g,""), 10),
        $: $(el)
    };
};

sortByRank = function(a, b) { return a.rank - b.rank; };

groupIntoRuns = function(acc, el) {
    var head = acc.length ? acc[acc.length - 1] : [],
        isSeq = function(prev, el){
            return el.rise === prev.rise &&
                el.rank === prev.rank + 1;
        };
    if(head.length && isSeq(head[head.length - 1], el)) {
        head.push(el);
    }
    else {
        acc.push([el]);
    }
    return acc;
};

returnAnyLongRuns = function (el) { return el.length > 1; };

flattenRuns = function (acc, el) { return acc.concat(el); };

removeIndicator = function (el) {
    el.$
        .fadeOut("slow")
        .delay(2000)
        .fadeIn();
};

In conclusion...

I hope this case study hints at the power of establishing conventional interfaces. Breaking up your tasks into black box components can be a fantastic way to reduce complexity - making it easier to spot issues or highlight possible improvements. But, be warned... these things can become addictive - and you are always left feeling that there might be an even cleaner, more concise way to do things: because there always is!

So here's a bookmarklet that aims to make things both better and worse. It assumes (naturally) that you leave Hacker News open in a tab, on the front page, all day. Naturally. It automatically updates the page whenever you return to the tab. All of the changes are (fairly subtly) highlighted, so it requires but a quick glance to see if you've missed something important.

Let me explain further, in Hacker News form...

...and, some boring details.

Like a true perfectionist, it sprang from a personal need to take Hacker News procrastination to the next level. Perhaps such a bookmarklet or extension already exists, but it would hardly be in the procrastinator spirit to not do it myself. If you want to have a play, the code is on GitHub.

It's pretty straightforward - consisting of the jQuery bookmarklet code, a timer object for watching window blurs and focusses and triggering refreshes, and the guts of it for pulling apart the DOM.

Originally the timer was tied to the requestAnimationFrame which I thought was a pretty sneaky hack: because that timer only ticks when the current tab is focussed. But it only works in Chrome (and FF nightlies, I think) at the moment, so I fell back to window.onfocus. The timer also includes a 1.5 second delay when you return - so refreshes aren't triggered accidentally as you tab past.

The DOM manipulation stuff is a bit hairy because Hacker News has markup from 1997 (another reason to love it): all tables, no classes or IDs. I have a regular-ol' HN account - so it might fail if you have some wiz-bang account that is laid out differently to mine. Lemme know!

PostScript: a couple of people asked me how to change the refresh time from 2 minutes... I haven't exposed anything to do this, but you can just fork the code, or even easier - open up your JavaScript console and type:

hnutimer.refreshTime = 30000;

The time is in milliseconds (30000 is 30 seconds times 1000 milliseconds!)

PPS: HackemUp on Hacker News.

Take for example, this DOMTimeStamp test case. In Firefox 4, I receive:

Type:number
Value:1301839830719 (13 digits)
Date:Sun Apr 03 2011 16:10:30 GMT+0200 (CEST)

My trusty-ish old iPhone 3GS concurs with Firefox.

Type:number
Value:1301839909995 (13 digits)
Date:Sun Apr 03 2011 16:11:49 GMT+0200 (CEST)

The iOS simulator for an iPhone4 however, thinks that it's half an hour ago.

Type:number
Value:1301838297961 (13 digits)
Date:Sun Apr 03 2011 15:44:57 GMT+0200 (CEST)

All fairly reasonable. But, Chrome 10 decides to take things to the next level:

Type:number
Value:1301837259664648 (16 digits)
Date:Mon Jul 31 43223 23:01:04 GMT+0200 (CEST)

That's why I use Chrome. It's like living waaay in the future. The one lesson I've learned from all this is that in the year 43223 we'll still be using JavaScript. Awesome!

It's an offline app, so don't be afraid to "add it to home screen" - and if you don't want to get dizzy, then lock the phone to portrait mode!

It just uses a dodgy implementation of some "sparklines" to plot the X, Y, and Z rotation that you can fetch by listening to the devicemotion event. It should work in Chrome - but doesn't seem to. I'll check it out, and update this post with some more details tonight. Or tomorrow night. Or.

onclick="(function(e){ alert(e); })(event)"

The event keyword gets magically passed into the context of the handler code - so you can write onclick="alert(event)". Though, I think you'd have to do alert( event || window.event ) for IE. tw;dt (too windows; did not test).

Next, if you want an anonymous function then you'll need to wrap it so it self-executes.

And finally, if you still want a handle to the this element, you'll need to pass that in too:

(function(e, obj){ alert(obj); })(event, this)

• about  Go
  Default Sparse Luna Lander
• 0/
  
  Passes
• Seed size
• Water chance
• "smoothing"
• speed

I stumbled over a thread on a forum about terrain generation using "cellular automata" type algorithms, which, owing to my past flirtings in the area, I found most interesting. Here's an implementation of it in Canvas. The options are:

• stop/go: make it go and not go!
• default: drop down with some presets. I like "luna lander".
• Passes: how many "refinements" the algorithm makes. Because each pass doubles the size of the array to process, don't make this too high!
• Seed size: the initial size of the array. Smaller arrays make it easier to see what's going on - and provde more "sparse" land shapes
• Water chance: the chance (from 0 to 1) that an initial cell will contain water
• "Smoothing": gets rid of some "jaggies" on the coasts by only allowing it to be water if it's surrounded by X other water's (something like that... I forget)
• Speed: how fast to do each iteration (in ms).

I can't for the life of me find that forum thread, but the basic idea went like this:

1. Create a small matrix and seed with either water or land
2. Double the size of the matrix so that each cell now takes up 4 cells
3. Refine the new big map, selecting land or water based on how many surrounding cells are land or water
4. Jump back to 2 for a few iterations.

The results are pretty interesting - I love how simple rules can create such intricate fractal-y outcomes. I've put the terrain generation code on GitHub if you want to take a look (beware: it's pretty "loopy")!