PayPerks Techblog

Adventures in agile development


Taming the JavaScript Spaghetti Monster

Generators and iterators have arrived in JavaScript and with them comes some powerful new functionality. I’d like to take a quick look at what generators are, their behavior in JavaScript and how to use them to manage asynchronous code.

To get started, you’ll need Node v0.11.2 or greater. You can easily install this by using the very awesome nvm node version manager. As these are not yet finalized in the ES6 spec their usage is experimental and subject to change and you will therefore need to invoke node with the --harmony flag. e.g. node --harmony app.js

Now, a few definitions from MDN:

“An iterator is an object that knows how to access items from a collection one at a time, while keeping track of its current position within that sequence. In JavaScript an iterator is an object that provides a next() method which returns the next item in the sequence.”

“A generator is a special type of function that works as a factory for iterators. A function becomes a generator if it contains one or more yield expressions.”

Consider the following generator function range – we know it’s a generator given the function* syntax. This will generate all integers from 0 to the specified maximum. Unlike a traditional function which would build the entire array in memory before returning it, our generator yields each value in turn, pausing execution after each yield (but maintaining state) and resuming execution upon successive next() calls to the generator.

1
2
3
4
5
function* range(n) {
  for (var i = 0; i < n; i++) {
      yield i
  }
}

Note that instead of a value, generator methods return an object with the properties value – the value returned by yield – and done – a boolean indicating if the generator is finished. Let’s see this in action.

1
2
3
4
5
6
7
var r = range(5)
// print the first 5 integers
var result = {}
while (!result.done) {
  result = r.next()
  console.log(r.value)
}

The for...of statement provides a handy shortcut to the value and will take care of terminating the loop once the generator is done.

1
2
3
for (num of range(5)) {
  console.log(num)
}

One last thing you should know is that as well as yielding values, generators are also able to receive values when next() is invoked with a parameter. This will be returned as the value of the last yield and will resume execution until the next yield is executed. Let’s modify our range function so that passing a flag of true will reset the generator.

1
2
3
4
5
6
7
8
function* range(n) {
  for (var i = 0; i < n; i++) {
      var result = yield i
      if (result) {
          i = -1
      }
  }
}
1
2
3
4
5
var r = range(3)
console.log(r.next().value) // 0
console.log(r.next().value) // 1
console.log(r.next(true).value) //0
console.log(r.next().value) // 1

Ok, enough theory already! How can I use this to write clean, maintainable and callback-free JavaScript?

For sake of simplicity, let’s assume we wanted the ability to “pause” execution at multiple points during function execution. Consider the following code:

1
2
3
4
5
sleep(function* () {
  console.log("Hello Word")
  yield 1000
  console.log("Goodnight!")
})

In the implementation below, sleep takes a generator, invokes it and passes it to the nap function. In effect nap acts as a controller using the next() method to execute code inside the generator, suspend execution, receives the sleep time from the yield and finally resume execution by calling next() again.

1
2
3
4
5
6
7
8
9
10
11
12
function sleep(generator) {
  nap(generator(), 0)
}

function nap(iter, time) {
  setTimeout(function() {
      var result = iter.next()
      if (!result.done) {
          nap(iter, result.value)
      }
  }, time)
}

Hopefully this example has provided a glimpse into how powerful generators can be when combined with the functional nature of JavaScript. At this point I would encourage you to start exploring libraries that take care of these mechanics for you. A good starting point is co brought to you by the prolific TJ Holowaychuk.

I’d also recommend the following posts, all of which provided invaluable information for getting me started:

Happy coroutines!

Mark (mark at payperks dot com) is the Director of Engineering at PayPerks.

Fintech Hackathon 2013

Last weekend our team had the pleasure of participating in a 24-hour Fintech Hackathon at the new MongoDB HQ. It turned into more of a 30-hour event, and we had an awesome time. We left exhausted and happy, satisfied with the fruits of our labor: a working hack in production! So what did we build…

Bookies For Good – Put your money where your heart is™

The day kicked off at 11am with a furious planning session, followed by a quick trip to Office Depot for last minute supplies (oversized Post-It Easel Pads, sharpies, etc). Then the hacking began. The goal of the event was to use APIs made available from the event sponsors, and we chose three:

We decided early on we wanted to build something fun and accessible so we made a site where people wager on financial outcomes — the twist being the loser doesn’t pay the winner, but instead the loser’s money goes to a charity org of the winner’s choice! Using public sentiment recommendations from PsychSignal, users can make wagers on the relative performance of publicly traded equities (pulled from DowJones), and based on the outcome, the less lucky user uses Dwolla to transfer money from their account to the selected charity’s account, without ever leaving the site.

At launch 24 hours after the first commit, the app supported live payments to charity:water, Angelwish, Action Against Hunger, Code 2040, Girls Who Code, and hackNY.

Not bad for 24 hours!

The experience for the team was phenomenal. We were lucky enough to have our talented designers Phil and Leslie cranking out the UI almost as fast as Claudio, Mark and Ali could tie it together with the APIs, while I kept very busy making sure all the coffee cups (and sparkling water glasses from this amazing machine in Mongo’s offices) stayed nice and full. But despite our initial roles, we quickly found an excellent self-organizing, cross-functional rhythm, with hourly stand ups to keep us on track.

We cut scope ruthlessly to be able to launch a successful product on time, and learned how much we can get done when everyone is working without interruption on a clearly defined, shared goal. There were a few touch and go moments around 4:30am (18 hours in) when we found a nasty bug, but thanks to some quick (or, as quick as can be expected at that hour) thinking we were back on track, and spent the last few hours testing and polishing.

We’ve already started implementing some of what we learned in our day-to-day work lives at PayPerks (no-meeting Wednesdays are back; as are ‘strongly suggested’ quiet times between 11 – 3 each day), but it also gave us a few ideas about hosting an event of our own sometime. That however is a post for another time. ;)

A big thanks to our friends at Betterment, Francesca Krihely and Nick Gavronsky for organizing a great event, and also to Gordon Zheng at Dwolla for a few helpful pointers!

Check out a list of the other great projects, and coverage from AlleyWatch.com and Dwolla.

All in all and awesome event. See everyone next time!

Tim (tim at payperks dot com) is a Director on the Product team at PayPerks

Introducing the One Day Sprint

Last week we tried an interesting experiment: a single day sprint. We wanted to condense our two week process into one day and see if it would yield any insights or highlight problem areas.

This was an all hands on deck activity with everyone in the company (technical and non-technical) involved in building a new standalone marketing tool for our business development team with loose guidance in the form of a short bulletpoint list of requirements from our founders.

Here’s what we learned:

  1. Requirements gathering takes time. Reaching consensus on what needs to be built and capturing those requirements in the form of high quality user stories took up almost half the day.

  2. Having everyone work on a single feature gave us an intense focus and we benefitted from the collaboration. A lot of us thought that merge conflicts would be an issue, but with frequent code updates these were generally trivial to resolve. No meetings increased efficiency too.

  3. The added pressure of a very tangible deadline instilled a sense of “hustle” in the team. There was a relentless drive to get things done and we saw an immediate spike in velocity.

  4. Cut scope to release on time. As the day progressed it became increasingly clear that we weren’t going to deliver all the requirements. Fortunately the upfront time we invested in proper story writing allowed us to easily de-scope lower priority features while still meeting the core business requirements.

What we’re going to be doing differently from now on:

  1. Two week sprints are out, one week sprints are in! We liked the extra sense of urgency the time pressure put on us. We’re going to try one week sprints for a while and see if it gives us the same buzz.

  2. Slice and layer requirements. This is the tricky part. In order to be able to cut scope and release on time we need to get better at writing our stories so that they’re not only narrow slices through the stack, but that they build on top of one another and deliver tangible, functional business value at each stopping point, or as early as possible.

At the end of the day, this was an extremely worthwhile exercise. Not only did we develop a useful product feature ready for production deployment, our business team gained a deeper insight into our software development process. They had a first-hand glimpse of “how the sausage is made”.

This will not only increase effectiveness of communication during product development, but also help the business better understand, simply, that designing and building good software is hard work!

Mark (mark at payperks dot com) is the Director of Engineering at PayPerks.

Why You Should Be Doing Code Katas (Hint: It’s Not Why You Think)

I’m sure most of us have heard about code katas. I’d like to argue that you should be doing them, but not because of the reasons you think. For those unfamiliar with the term, a code kata is a discrete coding exercise undertaken regularly to keep coding skills sharp. The idea is that developers should spend time practicing their skills so that when they need to code “on the job” they’ll be better at it.

This thinking is flawed. For most developers the majority of the challenges they face are not about writing better code. The challenges are more often understanding unclear business requirements, coming up with practical solutions from ill-defined problems or fixing and modifying code they didn’t write. No amount of practicing fizz buzz is going to make you better at this!

However, there is still intrinsic value in code katas for teams. If done properly, they can be an effective way to experiment and learn, sans consequences in the production codebase. Without a well defined problem, learning a new tool or technique can also be difficult – a specific coding exercise often provides the necessary friction to learn how to apply something correctly. Most importantly however, we found them to be a “safe place” to demo code (and solicit peer feedback) removed from professional ego.

Here’s the kata we most recently tried:

Given an input string of arbitrary length, create a function in any language of your choice that will make each line exactly 30 characters long whilst respecting word boundaries.

So for example, the following text:

1
"Sweet pepper ricebean salsify sierra leone bologi mung bean catsear cabbage. Kohlrabi desert raisin lettuce cucumber bok choy soko cress scallion groundnut tomatillo catsear."

Could be returned formatted as:

1
2
3
4
5
6
Sweet  pepper ricebean salsify
sierra  leone bologi mung bean
catsear    cabbage.   Kohlrabi 
desert raisin lettuce cucumber
bok  choy  soko cress scallion 
groundnut tomatillo catsear.

A fortnight later we scheduled a session for everyone to demo their solutions. And this, I think, is where the magic happened. Without the pressure of deliverables, the code reviews became enjoyable – we had everything from JavaScript through to Erlang with a crazy Scheme solution to boot! We touched on some interesting discussions about error handling, exceptions, code conventions and naming. All important things to reach consensus on when attempting to manage complexity in a large production codebase.

One of the signs of a high-performing team is that the code starts to look like it’s been written by one person. There is no ownership over different areas, no “experts” in particular modules, just a place where anyone can modify any piece of code at any time. Team code katas might just be a way to make this happen.

And yes, my solution went into a recursive death spiral if you asked it for a line length shorter than the longest word ;–)

Mark (mark at payperks dot com) is the Director of Engineering at PayPerks.

Clean Code

“Like everything metaphysical the harmony between thought and reality is to be found in the grammar of the language.” – Ludwig Wittgenstein

Today we discussed refactoring existing code to improve its design and make it “beautiful and clean”. I thought I’d share our findings here.

Before attempting to write clean code, we need to define what it is. For our purposes “clean” is code that solves the business problems in the most succinct and readable manner possible.

What’s interesting about this definition is that nowhere is there any mention of “optimized” or “clever”. Instead we’re focussing on code that’s as closely representative of the business domain and as maintainable as possible. This means we should always ensure our code artifacts mimic the real-world terminology of the problem we’re trying to solve. Names have an inherent power and magical quality about them – when conjuring the machine, choose your words wisely!

A useful concept for correctly modeling the business domain in object-oriented code is the single responsibility principle – the idea that every class should ideally have a single purpose and all functionality related to this should be encapsulated by the class.

As an example, consider the following hypothetical class that represents a user in the system:

1
2
3
4
class User(object):
    def __init__(self, first, last, card_provider):
        self.first = first
        self.last = last


And then, as a function that prints the user’s display name:

1
2
def display_name(user):
    return user.first + " " + user.last

Following the principle of single responsibility, display_name should belong to the User class because this class is responsible for all operations on a user.

After refactoring, our user class now looks like this:

1
2
3
4
5
6
7
class User(object):
    def __init__(self, first, last, card_provider):
        self.first = first
        self.last = last

    def display_name(self):
        return self.first + " " + self.last


There are a couple more modifications we can make to ensure our code is more “pythonic”:

1
2
3
4
5
6
7
8
class User(object):
    def __init__(self, first, last, card_provider):
        self.first = first
        self.last = last

    @property
    def display_name(self):
        return "%s %s" % (self.first, self.last)


Firstly, display_name is intrinsically a “property” of the object so it should be treated as such. Methods are actions that are performed on or by the object, properties are attributes that should be either set or read.

Secondly, don’t use + for string concatenation in python as it’s not type safe and because strings are immutable, inefficient too.

The last idea we explored was using exceptions to improve code readability and robustness. All too often we’ve seen code that relies on return values to indicate a failure state, for example:

1
2
3
4
5
6
def find_first_even_number(numbers):
    for num in numbers:
        if u % 2 == 0:
            return num
    # if no even number is found, return -1 to indicate an error
    return -1


And then for code that consumes this function:

1
2
3
4
5
6
7
8
9
import random
random_numbers = [random.randint(1, 100) for r in range(10)]

first_even = find_first_even_number(random_numbers)

if first_even == -1:
    print "Could not find an even number"
else:
    print "First even found is %s" % (first_even, )


There are a number of problems with this approach – what if we wanted to check for the first odd number allowing for negative numbers in the input? Then -1 would be a poor choice of error code! We could use another number, but then we’d have different functions in our codebase all returning different error codes. Not very beautiful at all!

Secondly, if we wanted to handle the exception at a different point in the code then we’d have to remember to pass the return value up the calling stack until it was needed – not particularly clean or maintainable. Instead, consider the approach below:

1
2
3
4
5
6
7
8
9
class NumberNotFoundError(Exception):
    pass

def find_first_even_number(numbers):
    for num in numbers:
        if u % 2 == 0:
            return num
    # if no even number is found, throw an exception
    raise NumberNotFoundError("Couldn't find an even number")

Now,

1
2
3
4
5
try:
    first_even = find_first_even_number(random_numbers)
    print "First even found is %s" % (first_even, )
except NumberNotFoundError, ex:
    print ex.message

This reads much more nicely than the first implementation. It’s immediately apparently what type of error we’re expecting to encounter and how we plan to handle it.

Software development is hard. Writing good code is even harder. There are guiding principles that we can use to structure our code and tools that make this easier, but ultimately it’s about craftsmanship – thinking deeply about a problem and creating a solution that other developers will appreciate, even if the end user never sees the workmanship!

Claudio Figueiredo is a senior engineer at PayPerks

Welcome to the PayPerks Tech Blog

We’ve been hard at work coding, testing, deploying, maintaining, and improving our platform (multiple apps, running on AWS) over the past couple of years. Having directly benefited from the open source community in big ways and small (“surely someone has already written a package for that!”), along with the thoughtful advice of others who have gone before us and taken the time to write about their experiences, we decided as a team it was time to spend some of our time writing about what we’ve done, what we’re doing, and what we recommend you may or may not want to do or not do.

We’ll be honest and open about things that have worked for us and things that did not, the learnings we’ve made from mistakes we’ve made, and what we’d do differently if we did it again.

You’ll hear from a growing cast of characters, who will write about all sorts of technical topics (python, django, mongo, mongoengine, javascript, html5, aws, jenkins, zabbix, puppet, mysql), our adventures in agile development (our kanban board, writing good stories & requirements, testing, retrospectives), and other topics including deploying, settings management, test data, security, collaboration, analytics, compliance, music, keeping sane, beer, comics, all sorts of ways to use keyboard shortcuts, and if you’re lucky, that time we deleted the production database.

Whatever the topic, we promise to keep it interesting and to the point.

On that note, welcome.

-Jake

Jake (@jake_net, jake at payperks dot com) is a co-founder of PayPerks and welcomes Ali, Tim, Rob, Phil, Mark, Claudio, Jordan, Leslie, Diana, Awais, Inam, Maryam, Muzaffar, Verdan, Afzal, and Sobia as founding members of this blog.