In this post, I will discuss a technique we use at SoundCloud to make the loading of image appear smoother and faster.
The situation: soundcloud.com is a single page application which, among other things, ends up displaying a lot of users’ avatars. The avatars are shown in many different situations from a tiny avatar on a waveform to a large profile image, so we have encoded the image into many different sizes. This same technique could definitely apply to Gravatar images on any site.
When displaying one of these images on screen, we want it to be shown to the user as fast as possible, so this technique makes use of the browser’s cache of previously loaded images. Quite simply, when displaying a large avatar image, we first display a smaller version of it, stretched out to the full size, and when the large one has loaded, fade it in over the top.
In essence, the code for this looks something like this:
In the end, not too complicated, and it gives a nice effect to the loading of your images. But there’s a problem here: we don’t want to make a request to get the small image just to show it for a couple of milliseconds. The overhead of making HTTP requests means that loading the larger image will usually not take significantly longer than the small one. So, therefore it only makes sense to use this technique if a smaller image has already been loaded in this session (and hence, will be served from the browser’s cache). But how do we know which images are probably in cache?
Each time an avatar is loaded, we need to keep track of that, but over time, there could be many thousands of avatars loaded in one session, so it needs to be memory efficient. Instead of tracking the full URLs of loaded images, we extract just the minimum amount of information to identify a image, and use a bitmask to store which sizes have been loaded.
// a simple map object, { identifier => loaded sizes }varloadedImages={},// Let's assume a basic url structure like this:// "http://somesite.com/{identifier}-{size}.jpg" imageRegex=/\/(\w+)-(\w+)\.jpg$/,// a list of the available sizes.// format is [pixel size, filename representation]sizes=[[20,'tiny'],[40,'small'],[100,'medium'],[200,'large']];// extract the identifier and size.functionstoreInfo(url){varparts=imageRegex.exec(url),id=parts[1]size=parts[2],index;// find the index which contains this sizesizes.some(function(info,index){if(info[1]===size){loadedImages[id]|=1<<index;returntrue;}});}// once the image has loaded, then store it into the map$('.fullImage').load(function(){storeInfo(this.src);});
So let’s take a look at what’s happening there. When the image loads, we extract the important parts from the url: namely the identifier and the size modifier. Each size is then mapped to a number — its index in the sizes array — and the appropriate bit is turned on in the loadedImages map. The code on line 27 does this conversion and bit manipulation. 1 << index is essentially the same as Math.pow(2, index). By storing only a single number in the object, we actually can save quite a bit of memory. A single number object could contain many different flags. For example, assume we have four different sizes and 10,000 images in the map:
The memory footprint of these two objects differ 30%: 1,372,432 bytes for the booleans, and 1,052,384 for the integers. The greatest amount of memory used in these is actually for the key names, so depending on how you identify the images, compressing that as much as possible will also help. Numeric keys are stored particularly efficiently by V8.
So anyway, back to the issue. We now have a map showing us which images have been loaded during this session, so it’s time to use that information for choosing a placeholder.
// find the largest image smaller than the requested onefunctiongetPlaceholder(fullUrl){varparts=imageRegex.exec(fullUrl),id=parts[1],targetSize=parts[2],targetIndex;sizes.some(function(info,index){if(info[1]<targetSize){targetIndex=index;returntrue;}});while(targetIndex>=0){if(loadedImages[id]&1<<targetIndex){returnfullUrl.replace(/\w+\.jpg$/,sizes[targetIndex][1]+'.jpg');}--targetIndex;}}// and in usage:varplaceholderUrl=getPlaceholder(fullSizeUrl);if(placeholderUrl){// there has been a smaller image loaded previously, so...addTheFadeInBehaviour();}else{// no smaller image has been loaded so...loadFullSizeAsNormal();}
This technique is not completely basic, and I’ve deliberately glossed over some of the finer details, but it can create a very nice effect on your site, and is especially effective for long-lived single-page apps. I hope you find it useful.
Modular code is great. For both development and ongoing maintainence, it brings you many benefits, however most discussion in this area is focussed on writing modular applications. Modular code belongs in all parts of your development, in particular I want to talk about using it when developing a standalone library. The most popular library for client-side modules is RequireJS, so I will focus on that.
Case study
A few months ago, I closed a long-standing issue on a unit test framework I built, called Tyrtle: to break the source into more manageable pieces. The code had grown to almost 1500 lines and it was getting very difficult to work on. After spending a day getting it all together, in the end I had a much simpler project which was a lot more maintainable, at the cost of a couple more kilobytes. These are some things I’d recommend from my experience: there’s definitely different ways to achieve the same effect, or even maybe some better practices out there, but this is what I’ve found so far.
Some important questions
Should I use modules?
It depends. If your code is of a certain size (upwards of ~600 lines), then you could probably get some benefit separating the code into modules in individual files. But below this, you have to consider that it does introduce some overhead.
Though RequireJS is quite a large library, the author, James Burke has thankfully produced a minimal library which provides the same interface but just a small subset of features, which will still be fine for our purposes. This is called almond and it’s only 2kb after minification. If 2kb is too significant a percentage of your library’s size, then you probably should stick to a single file.
Do I check in the compiled library?
This is up to you, and there are differing opinions here. My opinion on this is that you should.
In theory, you should be able to put your whole project on Github with a nice Makefile and README and let everyone build it for themselves, but in reality, that’s more effort than most people are willing to invest, so most libraries provide a simple ready-to-go version somewhere in the repo.
When exactly you run the build and check in the compiled version is up to you. Some libraries will do it for every commit, others only update the precompiled version when there’s an officially tagged release. If you go with updating on each commit, make sure your users know that file is not necessarily a stable version to use.
You should also decide now where the compiled version of your library will live. Sometimes it’s just in the root, other times it’s put into a subdirectory (named dist, for example), especially if there are a few different versions included in each build.
Having the built library in the repository itself will be useful if you want your library to be used with client-side package managers such as Bower.
The arguments against checking in the built library are all quite valid: in general, it is considered bad practice to check in build artefacts; and it increases confusion for other developers wanting to contribute. You will just need to weigh the ease-of-use of your library against these concerns. If you did want to avoid checking in build artefacts, yet still provide a simple version for your users, one suggestion would be to create another repository for stable releases, and then also provide configuration tailored to a particular package manager (eg: a component.json file for Bower).
Application structure
Put all your source code for the library itself into a subdirectory, called src or similar. All other code, including the build configuration and tests all live outside, in their own directories (/build, /tests) or in the root if there’s only one or two files.
Splitting up the source files into directories can follow the same guidelines you’d use for any other project, but unless you have a very large number of modules, it is unlikely you’ll need to go any deeper than just /src.
AMD? CommonJS?
I recommend CommonJS. For the code to run in the browser, you’ll need to use AMD modules, but converting to AMD is simple (covered below), and lets you avoid writing meaningless boilerplate. So, just like a nodejs module, just use require, module or exports as if they were available in scope.
Obviously in building a project with RequireJS should start with bringing in the library. Install it with npm but make sure it’s in your package.json file by using the --save flag:
1
$ npm install --save requirejs
When you install this, it will create a link to a binary of r.js as node_modules/.bin/r.js. r.js is RequireJS’s command line tool for converting and combining files.
We’ll also need almond, so grab the library and put it into /vendor. We’ll come back to this later.
Write modular!
Start developing your library (or converting your existing one), putting each logical module of code into its own file. When there’s an interdependency, you specifically import the modules you need. When it comes time to run or deploy your code, you’ll need to glue it back together, which brings us to the build step.
Automating the build
The overall goal here is to make your life easier, and as every good programmer knows, the key to an easy life is automation, so we’ll configure as we go. There’s a ton of systems you can choose for automation, including Makefiles, Cakefiles, Rakefiles, Jakefiles, GruntJS and more. For what we need, I recommend using a Makefile. It might not appear to be the most intuitive interface, but it’s very powerful and could win purely on ubiquity. It was created in 1977, and is widely known by developers of many different languages. This benefit is not to be underestimated: there is a ton of great documentation, community knowledge and best practices to help you along the way.
With a good Makefile (or any other tool you choose), your project’s build instructions should be to run one simple command, and never “Install the node modules, then the front end modules, then copy this, then…”.
Better living through Makefiles
A Makefile is a text file which defines a number of build ‘targets’, each of which could depend on different targets first. Each then lists the instructions for making that target; for example, copying files or compiling code. In general, the target is the path of a file – typically something which is created by that part of the build; otherwise any label can be given to a target. This is called a dummy target. If the target is a file and it exists on the file system and is up to date, then that step will be skipped.
To execute a target, you run make [target], eg: make build, make clean, or simply just make. As a convention, the first target you should declare in a Makefile is called ‘all’. This is what is executed when invoked without a target specified — not because it’s called “all” but because it’s first.
One little gotcha to watch out for: the indentation is important in Makefiles, and must be done with a tab character. If you indent with spaces, you’ll get a nasty and confusing error message, specifically: Makefile:2: *** missing separator. Stop.
Starting from the start, we want the build step to run when you type make. Create a file in your directory root and name it Makefile (with no extension) and enter this:
1
all: build
This declares a dummy target called all which depends upon another dummy target named build. Creating targets such as this which contain no actions of their own is fine, and can often help to split up a large build process into smaller logical steps. Now we need to define the build target. The build step can be considered done if the output library file exists and is up to date, so we can add that as a target itself. To avoid repetition, it makes things easier to use variables here. Variables can be used in the target names as well as in any commands a target defines.
1234567891011
# target is the final output file for your libraryTARGET=myLib.js
BUILD_OPTIONS=build/build.js
RJS=node_modules/.bin/r.js
all: build
build: $(TARGET)$(TARGET):
$(RJS) -o $(BUILD_OPTIONS)out=$(TARGET)
The $(BUILD_OPTIONS) are for the r.js tool, and we’ll come to that soon, but apart from that, you’ve actually pretty much got everything you need: the files in the source directory are converted to AMD, combined and saved. Except…
Declare the dependencies
Remember we said that the build process should be just make? Nowhere in here does it declare that a target depends on r.js, it just assumes it’ll be in that location. The steps to make sure this exists should be added as a target and set as a dependency where needed:
What this is saying is that to build myLib.js ($(TARGET)), it depends on a file existing in node_modules/.bin/r.js ($(RJS)). If it doesn’t exist, then run npm install.
Now, you’d be right in pointing out that npm is an undeclared dependency in this case, and that npm will in turn rely on Node. You will need to set some baseline expectation for the environment you’re expecting it to be run in. For a javascript library, I think you’d be safe in assuming NodeJS and NPM are available, but that’s up to you. For applications I’m working on at SoundCloud, we assume nothing (well, very very little) about the environment the code is executing in, and hence, the build script will actually download and compile Node and Nginx during a build. This is a good practice, since you can be guaranteed of the environment if you build it yourself, however I’d say it’s overkill for just a library.
Build only when needed
If you were to run this now, you’d see the build do its thing: the node modules downloaded and then your source files would be combined and written to the output file. Sweet! Run it again, and you’ll see a message “make: Nothing to be done for `all’”. This means that make has detected that everything is up to date and so it doesn’t need to do anything — this is a key goal of a build process as it means you won’t be wasting time downloading or rebuilding things which don’t need it — but hang on, how does it know it’s up to date? In this case, it is looking at the dependencies of the target and sees that you now have a file in myLib.js and therefore its job is done. That’s good, but if you change one of your source files and run make, it will happily tell you there’s nothing to be done again, leaving you with an out of date library! Bummer!
To fix this, we need to tell it which files will influence the build, potentially making the final file out of date, so by collecting a list of all the source files and listing them as dependencies, an update to any one of them will make the target rebuild. Rather than manually list them all (which could be tedious and error-prone), you can just use a shell command in the Makefile.
12345678910111213141516
TARGET=myLib.js
# find, in the 'src' directory, all the files with a name ending in .jsSRC_FILES=$(shell find src -type f -name "*.js")BUILD_OPTIONS=build/build.js
RJS=node_modules/.bin/r.js
all: build
build: $(TARGET)$(TARGET): $(RJS)$(SRC_FILES)$(RJS) -o $(BUILD_OPTIONS)out=$(TARGET)$(RJS):
npm install
Now run make and you’ll see the build happening. Run it again, nothing will happen. Edit any source file, and it will build again. Perfect.
Cleanliness…
One final thing to mention about Makefiles (and any other build automation) is they should offer a way to restore back to a “clean” state, as if make had never been executed. By convention, this is defined in a target called “clean”.
Now (once we get the r.js build options sorted), building is as simple (and as standard) as make.
r.js build options
In the Makefile, we referenced a file at build/build.js which contains the r.js build options and now we should fill them out.
These options are an extension to the requirejs configuration, for example, setting the base path of your source files, shimming non-AMD-ready third-party modules, or defining special paths for certain module names. There are several addition options for r.js
include defines a list of files to add into the build. The modules required by these files (using require('some/module')) will also be found by the r.js tool and automatically included in an optimal order. Here, you should include the base entry point for your library — that is, module which exports the library itself. Here is where, you can include almond, which will provide us the basic interface needed for AMD modules to work.
optimize defines which sort of minification you’d like to use.
cjsTranslate will convert CommonJS modules to AMD modules
wrap is the most important part of this configuration. If you were just to combine your files into one, all would be declared in global scope, quite possibly interfering with other application code (especially so if the application is using requirejs). The wrap configuration lets you define fragments of code to prepend and append to the combined file, and this will be how your library is exposed to the application. Using UMD (Universal module definition), and putting its boilerplate in these fragments, we can then cater our library to whatever environment it will be used in. More on this below.
Thorough documentation for r.js is strangely difficult to find, but the best source I’ve found so far is the source itself. In the r.js repo is an example build script with good documentation on the options.
As an example, here are the build options used for Tyrtle.
The UMD pattern can be implemented by using these fragments as the wrapping files in your build.
1234567891011
// wrap-start.frag.js(function(root,factory){if(typeofdefine==='function'){define(factory);}elseif(typeofexports==='object'){module.exports=factory();}else{// change "myLib" to whatever your library is calledroot.myLib=factory();}}(this,function(){
(Your code will be inserted here.)
12345
// wrap-end.frag.js// change "my-lib" to your 'entry-point' module name returnrequire('my-lib');}));
To briefly explain what is happening here: it defines an IIFE which is passed root (eg: window in the browser), and a factory function, which returns your library. When this code is executed at the run time of the application, if that application is using a module system with define (AMD), or simply assigning to module.exports (CommonJS), your library will be accessible as a module for the application. If neither of these are used, then your library is attached to the global scope using whatever name you decide (myLib in the above example).
This style gives you a lot of flexibility, while keeping your code a good citizen which fits in to whatever is the style of the including application.
Mo’ builds, mo’ options
By modifying the options to the r.js Makefile, you can obviously produce a range of different outputs. Here’s some common options and how they could fit into a Makefile.
Production and development builds
In the example r.js options above, as a unit test framework, Tyrtle had no need for minification, so the optimize option was set to “none”, since it makes the builds faster and aids debugging, but what if you wanted to provide your users with both a ‘development’ and ‘production’ version of your library, to save them minifying it themselves? Simple! Let’s add a new target to the Makefile.
If you want to provide different builds of your system with specialised code for a particular platform, this is made simple with requirejs’s path configuration. Though it has many uses, you can use it to dynamically alias a module name to a particular file, and r.js allows you to change this configuration on the fly.
A mobile-specific build of a library might allow you to leave out large sections of code (eg: if it were optionally interfacing with Flash), or use jQuery 2.0 instead of 1.9, since supporting old IE would no longer be necessary.
The paths option passed into r.js changes which file is retrieved when using require. The source code would contain something like $ = require('jquery'), and this option changes it so that jQuery 2.0 is used.
Custom lodash builds
If you want to use some of the functions provided by Underscore, but are wary of increasing the file size of your library, switching to lodash is a good idea, since it provides Underscore compatibility, but with a powerful customisable build process. I won’t go too much into the options of lodash, but here’s an example of how you might integrate it with your Makefile.
Lodash is shipped as an NPM module named lodash-cli, and it provides an executable for generating a build, so those will need to be added as dependencies.
12345678910111213
LODASH=node_modules/lodash/build.js
LODASH_MODULE=vendor/lodash
LODASH_LIB=$(LODASH_MODULE).js
# to access the builder, we just need the module$(LODASH):
npm install
$(LODASH_LIB): $(LODASH)$(LODASH)include=throttle,extend,bindAll exports=amd --output $(LODASH_LIB) --minify
build:
$(RJS) -o $(BUILD_OPTIONS)out=myLibrary.mobile.js paths._="$(LODASH_MODULE)"
Lodash also has a mobile mode which could be used in combination with a mobile-specific build to further reduce file size and increase performance.
12
$(LODASH_LIB_MOBILE): $(LODASH)$(LODASH) mobile include=throttle,extend,bindAll exports=amd --output $(LODASH_LIB_MOBILE) --minify
Wrapping up
If you think a modular approach to building a javascript library is a good idea — and it probably is — then using requirejs and a simple Makefile will let you do everything you need with minimum fuss. There are plenty more fun things you can do in a build step, but those will have to wait for another post.
This week, I’ve been at the Fluent Conference in San Francisco. It’s O’Reilly’s first javascript conference, and it brought in some really large names in the field including Steve Souders, Brendan Eich, Ward Cunningham, Paul Irish, Lea Verou… and me.
I was doing just a short presentation on the Share Your Stack line of talks, and so I was discussing what we’ve used and how we’ve built The Next SoundCloud. I was also quite honoured to be contacted by Dave Nugent and Andrew de Andrade from the San Francisco JS Users’ group who asked me to give a talk there too. I don’t know why, but I was kind of expecting a medium-ish sized crowd, and that I’d be on for 20-30 minutes or so. Turns out there were over 300 people and my slot was for an hour. Cool (gulp).
Anyway, I think it went really well and I got to talk to a lot of really interesting developers doing some really cool stuff. Due to popular demand, I’m uploading the slides I used for the talk. First of all, I hope they work on your computer (interoperability wasn’t a key concern at the time), and that they make some sense. For my actual talk, I had embedded soundcloud.com into the presentation using iframes, but this won’t work for you unless you’re already in the beta, so I’ve replaced them with static images. You’ll just have to imagine all the awesomeness of it.
Coming shortly, I’ll be writing a blog post for SoundCloud Backstage, and probably write another post here with more in-depth details about some of the techniques I spoke about in the talks.
PS: Due to all the people I spoke to mocking my luddite ways, I finally caved and created a Twitter account: @spadgos
This weekend was the AmsterdamMusic Hack Day, which I was lucky enough to go along to. Although the weather seemed way too nice to spend the entire weekend indoors staring at a keyboard, plenty of people were just as silly as me. Some of the standouts were the Theremin-controlled autotuned karaoke machine, the Bunny Boogie (where a robotic rabbit can upload recordings to SoundCloud and then will dance along to the songs you play there) and a mashup which finds popular animated gifs which match the BPM of songs in your Spotify library and synchronise the two.
I worked with Sasha on building an experiment we called “CitySonar”. The concept is simple, though perhaps hard to describe in words. Let’s try anyway:
Choose a location on a map (we used Google Maps)
Choose a geographical feature, such as banks, schools or cafes.
A marker is placed on the screen for each of these features near you.
When you press play, a ‘sonar’ sweeps around the map and plays a note for each of the markers.
Markers further away are lower notes, ones closer are higher. Each different type of feature can have different characteristics for their notes, including the octave range, and the attack and release of the notes.
Warning full frontal nerdity: This is a client-side app, built entirely in Javascript. The only server-side component was a small NodeJS proxy to get around cross-origin security restrictions. The data was pulled from the OpenStreetMap XAPI, custom map tiles provided by Stamen Maps, and client-side audio generation was done by Audiolet.
One great thing about the OpenStreetMap data is that it tracks heaps of geographic features, categorised quite granularly, so it’s not just “bars” and “cafes” nearby, but power poles, bicycle parking, post boxes, and even brothels. Mapping these features gave some really interesting insights and some interesting results for our experiment.
For example, in a large city like Amsterdam, searching for restaurants will end up with a cacophony, as these tend to be clustered quite tightly in neighbourhoods. Post boxes, however are evenly spaced thoughout the city and actually can create something resembling a melody. Using features which are clustered in a single place (for example, embassies or brothels) can be used for a loud hit in your soundscape.
The quality of the project is still firmly in the “hack” category — it did go from idea to completion in 24 hours, remember! — but it’s rather functional still, if you ignore some visual glitching… Also, OpenStreetMap is a community-built map and it’s definitely not complete yet: if you see features missing in your area, don’t forget you can go add them yourself!
You can check it out at http://citysonar.herokuapp.com, and grab the code on GitHub, but my god, the code is hacky. Please don’t look at it. Also, the API we use is very slow sometimes, so you’ll just need to be patient sometimes.
Bonus video
Here is the recording from the presentations. CitySonar starts at 33:20.
A small post to announce a small plugin for Sublime Text 2. It’s called CSSpecific and simply, it calculates the specificity of your CSS selectors.
Specificiwhat?
When two CSS selectors target the same element, one of them has to win, right? Deciding which one is the winner is a matter of detecting which is the most specific. You can read the full technical details of this in the CSS Spec, but basically it works like this:
A specificity score is 3 values. We’ll refer to these values as “a,b,c”.
A selector’s specificity is calculated as follows:
count the number of ID selectors in the selector (= a)
count the number of class selectors, attributes selectors, and pseudo-classes in the selector (= b)
count the number of type selectors and pseudo-elements in the selector (= c)
ignore the universal selector
At the end, whichever has the highest score wins, reading left to right, a > b > c. Note that any number of lower-valued selectors never outvalues a higher valued selector. So, even a selector which had 1000 classes in it would be less specific than one which had a single id. (0, 1000, 0) < (1, 0, 0)
The plugin
So the plugin, when activated, reads all the CSS selectors from your file (even in HTML files), and collates them in a display panel alongside their score. For simplicity the score is presented as a single number: a * 100 + b * 10 + c, which means that if you DO have a selector with 1000 classes, this plugin will tell you that it has a higher score, but you know what? If you have a selector with more than 10 classes or elements, you got bigger problems, pal.
A little while ago, I started working at SoundCloud — it’s a really fun place to work, and very exciting given the scale and growth of the platform there. They’re so nice, they even put up an interview with me!
I’m working across the HTML5 projects which SoundCloud are producing, including the mobile site and the widget. The widget is the player which users can embed into their websites or blogs, and just recently we’ve released the new version which is entirely based on HTML5. I should mention that very little of the code was produced by me, since it was done before I started, but now I’m working on bugfixes and further improvements.
Anyway, the key point here is that it’s designed to be embedded on blogs. It’s not very difficult to get the code to embed it normally — just go to any track or user or anything, and click on “share” — but since I’m now on a new blogging engine which is specifically marketed as a “hacker’s” blog, I thought I’d put together a little plugin to make it even easier for those people using Jekyll or Octopress.
It’s actually trivially simple in the end, but the source is on github. It’s currently buried within my blog’s repo, but maybe soon I’ll move it to its own repository.
Using it is very simple too. The only downside currenly is that it requires you to know the relevant id if you want to embed a group, playlist, track or app widget. This information can be copied from the HTML which the SoundCloud site gives you, but it’s not immediately obvious. Perhaps I’ll see what we can do about that… The good news is that if you want a widget for a user’s tracks, or their favorite tracks, then you can just use their username in lieu of an id.
Examples
Here’s the most basic usage: a resource type (users, favorites, groups, playlists, apps, or tracks), and an id.
So there you have it. I’ll write more soon about the sidebar plugin, and perhaps I may have organised my code better by then and you’ll be able to get these plugins for yourself. Who knows??
The last blog was, to be kind, crap. It was something I’d just thrown together to get it working. I chose a standard off-the-shelf wordpress theme, and that was about the limit of the effort I spent getting it set up. For someone who writes about the web, it was an embarassment, really.
Anyway, that was the past!
I’ve now set up with a new blogging engine, Octopress, which is built on top of Jekyll. It’s described as a “hacker’s” blog — you write the content in a language such as markdown or textile, and then Octopress parses and converts it into static HTML pages. From there, you publish via rsync or, what I’m doing now, pushing to a github repository. Deployment via version control seems so very right, you know?
Despite my comments about the stock-standardness of my previous blog, the current one is still using the default packaged theme, but I’ll get onto that shortly, I swear. I’ll also look into migrating (and perhaps vetting) the posts from the old blog too.
Recently, I’ve had one of those events in my professional life. They happen (for me) even less frequently than changing jobs. Yes, I have changed my preferred text editor. gasp!
I’ve moved from Komodo, which is a very competent and extensible (with Javascript!) editor, to the new kid on the block, Sublime Text 2. Sublime is a product still in beta, but I can recommend it as being definitely stable enough for every day use.
I’ll apologise up front if the following sounds too gushing or like an advertising pitch. I have no commercial interest in the product whatsoever, I actually just really like it and want others to try it out.
When starting with Sublime, the first thing you’ll notice is how amazingly fast it is. I am still amazed at how quickly it starts and continues to be, even on my little old netbook. The second thing is that you’ll see that Sublime is a text editor made by programmers for programmers. The default colour scheme is a very pleasing and eye-friendly pastels on dark grey, the interface is absolutely no-nonsense: no toolbars; no buttons for new file, save, cut, copy or paste; just one menu bar, an (albeit slightly gimmicky IMO) “minimap” of the file, and your code. All commands are available from a “command palette” which is opened via a key binding. Similarly, opening project files or searching for functions is done with another palette. These palettes, apart from being super-quick, also provide fuzzy text matching. For example, to make the selected text upper case, I just select the text, then press Ctrl+Shift+P, and type “chup”, which will find “CHange text to UPper case”, or if I want to open “lib/test/PolygonTest.js”, Ctrl+P and “l/t/p” or “PolTe” or whatever narrows it down enough with the minimum amount of typing. Everything is possible without even touching your mouse.
Probably the most surprising thing is the configuration settings. Every programmer worth his salt will fiddle and adjust settings to get the work environment Just Right, and usually this means poring through dialog after dialog of checkboxes and pull-downs. In Sublime, all the configuration (yes, ALL) is done via JSON text files. This is shocking at first, but amazingly liberating once you start to use it. As a programmer, every day you build complex data-structures capable of modelling anything by combining numbers and strings into nested arrays and maps — why couldn’t you configure a text editor in the same way? Settings are applied instantly upon save, meaning you can test out each of the settings very easily and get it just how you like it. Even key bindings are handled this way. Every interaction you have, from moving the cursor, to basic copy/paste/save actions, even to what happens when you click a mouse button is configurable. Even better though is that you can bind the same key combo to several actions and then affix conditions (referred to as “context” in the config) to that setting. For example, the “enter” key can add a new line in regular text, but you can also add another binding which makes it prepend commenting characters (//) if the current context is a comment. Triple-click might normally select a line, but if it’s on a function definition it could select the entire function body for you. After all this tweaking, you can commit your plain-text settings to version control for portability between your workstations, which is very handy!
One other feature I’ve not seen in any other editor is multiple selections. Komodo and other editors (VI, Emacs…) have “block editing” where you can make rectangular selections and edits across rows and columns. This is particularly useful for modifying blocks of repetitive code which happen to align vertically. Sublime takes that a step further by allowing for multiple selections. It is surprisingly useful for reformatting code, and as a replacement for Find-and-Replace. Just select (let’s say) the variable you want to rename, and hit Ctrl+D. The next occurrence of that word will get selected. Repeat as often as you like, the just start typing and you’ll be typing in multiple discrete locations. This feature alone has reduced the amount of macros and regexes I use to clean up text drastically. You’ll wonder how you lived without it, seriously.
The last thing I want to mention is the fledgling, but fast-growing developer community around Sublime. As a tool for programmers, its been designed to be extended with plugins, written in Python. What’s even better is that there’s a package repository plugin called Package Control, written by Will Bond, that completely streamlines the plugin installation and upgrade process integrated directly into the editor. Plugins are hosted on Github or Bitbucket and you can pull the upgrades as their published. After installing that package (which you can do just by pasting a command into the integrated python shell..!), all the available packages are shown in a command palette, and are installed without even requiring a restart. Since the code for the plugins is on Github, it’s really easy to fork it and send in pull requests if you find a bug, too. I highly recommend it!
And there’s just enough time for a quick little shout-out for one of my own plugins. If you’re writing Javascript or PHP with Sublime, you should totally install the JSDocs plugin. You can install it with the aforementioned Package Control plugin, or there’s instructions on the Github page. This plugin makes writing Javadoc-style comments a breeze by adding context-sensitive autocompletes, comment-aware linebreaking, and it even inspects your code to prefill @param and @return tags in a docblock. The readme file has much more information and many examples. If you run into any problems, leave me a note in the issue tracker and I’ll get right onto it!
I’ve been working on the previously-mentioned Myrtle project, as well as its unit-testing sister project, Tyrtle (which will get further plugs soon…), and I kind of stumbled across this interesting little thing.
In javascript, there’s a “magic” variable made available inside every function, called arguments. It appears to be an array at first glance, but it isn’t. Observe:
12345678910111213141516171819202122
functionf(){vararr=['x','y'];// a real array, for comparisontypeofarguments;// "object"typeofarr;// "object"arguments.length;// 2arr.length;// 2arguments[0];// "x"arr[0];// "x"typeofarguments.join;// "undefined" !!!typeofarr.join;// "function"Object.prototype.toString.call(arguments);// "[object Arguments]" !!!Object.prototype.toString.call(arr);// "[object Array]"}f('x','y');
It is actually a special type of object called Arguments, and it’s documented well on the Mozilla Developer Network. Essentially, it is just an enumerable object in that it has a .length property and other properties in the slots 0...length - 1, but it doesn’t have Array as its prototype, and hence, it doesn’t have any of the array functions on it.
Obviously, in most cases where you want to actually do something useful with the arguments object, you actually want to have an array, rather than this strange object. Converting it to an array is usually done like so (as is recommended in the MDN article linked above):
This works. If you really wanted to be pedantic about it, you could say that it isn’t actually safe, in case some code overwrites Array.prototype.slice, but I digress. In any case, it’s a lot of typing and can be very confusing to both newbies.
Now, in the interests of openness, I should say that I originally wrote this post to talk about this amazing new technique I discovered: [].concat(arguments). I wrote benchmarks and everything. It was shorter and performed about 4x better on Firefox. Then I actually used it in my code and discovered that it doesn’t even work. So, there you go. I thought I’d keep the bulk of this article and instead compare some other methods which actually do work…
I wrote some functions which convert their arguments to an array and then return it. The first two both use slice, but I wanted to see if there was a difference between creating an array literal or using the prototype (spoiler: there isn’t).
Yes, some of these numbers are bizarre, but they’re not typos. Push/unshift is 250% faster than slice on Firefox, and only about 10% faster on Chrome. Yes, splice is 94% slower than any other method on Chrome — even slower than Firefox in this test. Unshift out-performs Push on Chrome by about 8%, too.
Of course, the real benefit of [].slice.apply(arguments) is that it’s a one-liner. In real life usage, at best, the push/unshift technique requires 2 lines, but could be extracted to an external function. Of course, adding another function call is not free, so I did another benchmark.
123456789
functionconvertWithPush(args){varx=[];x.push.apply(x,args);returnx;}functionpushWithFnCall(){returnconvertWithPush(arguments);}// and the same for unshift
In Chrome, the extra function call led to a ~33% slow down. It did not seem to affect Firefox’s performance at all, however.
In summary:
If you want a simple one liner, go with: var args = [].slice.apply(arguments)
If you want better performance, use: var args = []; args.push.apply(args, arguments)
Never use splice
Please do run the tests yourself (and here for version 2 which has the additional benchmarks). Let me know if I’ve forgotten a different method, or if I’ve stuffed up my tests somehow.
I’m writing to introduce a small project I’ve just put together. I call it Myrtle(which I guess is some sort of play on words on the Mock Turtle), but in any case, it is a Javascript Mocking Framework!
The project code is available on Github, or you can just grab Myrtle.js if you like.
So anyway, let’s look at what it does. Let’s say you’re writing some tests for your code, and you want to check that a particular function is called. For example, you might have a Validator class which should be called when a form input is changed. Here’s the signature of a basic validation class:
12345678910
varValidator={/** * @param {DOMElement} a HTML Input element * @param {String} a value to validate * @return {Boolean} If the value was valid. */validate:function(input,inputValue){// code here}};
The first step is to spy on the validate method:
1
varinfo=Myrtle.spy(Validator,'validate');
Spying on a method modifies it so that metadata about that method is stored with each call to it, without changing the actual functionality. You can access this metadata through the API object returned by Myrtle.spy.
123456789101112
varinfo=Myrtle.spy(Validator,'validate');$(myInputElement).val('a changed value').trigger('change');// let's check that validate was calledinfo.callCount();// 1// and let's see what it returnedinfo.lastReturn();// true// and let's check what the parameters passed to it wereinfo.lastArgs();// [myInputElement, "a changed value"]
What’s important to remember here is that the validate method is still executed as if nothing ever happened. Myrtle wouldn’t be a good spy otherwise…
Other times you want to actually stop functions from running. For example, in a test environment, you might want to suppress error messages which you’re triggering on purpose, or to stop AJAX functions from executing. In these cases, you can stub out a function.
1
Myrtle.stub(Notifications,'displayError');
This replaces the displayError method with a function which does nothing. Care should be taken in these cases that other code isn’t expecting a return value from functions which you stub out completely.
You can also replace a function using stub. This is useful if you want to simulate a method, but not actually execute it – for example, when an AJAX method is called, you can put together a fake response.
You’ll see there that the first parameter passed to the stub function is the original method. This is useful in cases where you may want to only stub it out based on some criteria, or to modify arguments or return values of the original method.
The last feature currently in Myrtle is some basic speed profiling. If you want to find out how fast your function is executed, or if it runs slower given some input, you can turn on profiling and find out.
12345678
varinfo=Myrtle.profile(calc,'factorial');calc.factorial(3);calc.factorial(8);calc.factorial(12);info.getAverageTime();// a number, like 12 (ms)info.getQuickest();// { args : [3], ret : 6, time: 1 }info.getSlowest();// { args: [12], ret: 479001600, time: 20 }
Those are the three main features of Myrtle (so far). However, the nice thing about them is that they can all be combined in any way. You can spy on and stub out a method, or spy and profile. Stubbing and profiling probably isn’t so useful though, since you’d only be measuring the speed of performing your replacement method.
There are a few ways to combine the options, and it’s important to know that each of the Myrtle functions returns the exact same API object per function. To demonstrate:
123
varinfo1=Myrtle.spy(Obj,'foo');// turn on spyingvarinfo2=Myrtle.stub(Obj,'foo');// turn on stubbinginfo1===info2;// true
There’s also the Myrtle function itself which can be used:
1234567
varinfo=Myrtle(Obj,'foo',{spy:true,stub:function(){// my replacement},profile:true});
Of course, the last thing to cover is an important one: tidying up. Chances are that you don’t want to stub out your methods for all the tests, and you want to restore them to how they were before at some point. Myrtle makes this super easy.
12345678910111213
// if you have a reference to the API object:info.release();// and even if you don't, remember that// Myrtle will give you it easily.Myrtle(Obj,'foo').release();// and if you're really lazy, just clean them all up!Myrtle.releaseAll();// if you're not sure if you've left// some hanging about, just check!Myrtle.size();
So, like I said, the code is on Github. Go have look and let me know what you think!
