What we've been up to in Quarter 1 of 2016

It's been a busy first quarter here at Red Comet Labs and we'd like to share what we've been up to.

Welcome Ben!

First of all, Ben Janecke joined the team, bringing his excellent CSS skills and JavaScript experience. We're thrilled to have Ben's expertise on the team, even though Garren is now outnumbered by Capetonians!

DevConf 2016

At the beginning of March, all three members of the Red Comet Labs team had their talks accepted for DevConf 2016, so this also gave us a chance to get together and do a bit of Red Comet Labs planning and strategy. Our strat session had a bit of a delayed start after, ahem... Andrew managed to miss his flight from Cape Town. Then, at DevConf, we got to hang out with over 500 other members of the South African dev community and attend some interesting talks.

Here are the abstracts of the team's talks. Do let us know if you're interested in hearing more about our topics.

Garren: Building responsive offline web applications using CouchBD and PouchDB

People are using our web applications on mobile devices, so internet connectivity is no longer always guaranteed. We need to build applications that work offline. Enter PouchDB, a database that works in the browser, and CouchDB, a NoSql database that replicates. Combine both of these and you can build responsive applications that work, whether the user is online or offline.

In this talk I will introduce both of these great database technologies, show you how to get started with CouchDB and PouchDB, and demo the many ways in which you can combine them for an amazing web experience for your users.

Ben: Composition over inheritance in Javascript

The gang of four make two important statements in their book:

  • "Program to an interface and not an implementation" and
  • "Prefer composition over inheritance"

This talk is about more than just that. It's about what we call the two pillars of Javascript: Prototypical inheritance and Functional Programming, and how we can use the two pillars to enable composition.

  • Learn when and where to use composition
  • Get a better understanding about how prototypes and classes work in Javascript
  • Learn about functional tools closures, lambdas, functors, higher order functions, etc.

Andrew: An introduction to React.js

In this talk, I give a high-level overview of React.js, so that the audience will feel comfortable with the general concepts and know how to get started. Many people would have heard of React.js and will want to know what all the fuss is about. This talk will give a brief outline of the history of React.js, why it is incredibly useful, and the problems it solves.

I will give brief code examples (showing the outline of a React.js component). For people who'd like to take this topic further, I will provide links to websites that give in-depth tutorials and will also provide a boilerplate Github repository for interested listeners to pursue further.

Garren on ZADev, Web Platform and FLOSS Weekly Podcasts

Garren has spoken on three podcasts over the last few months:

Opensource work

Ben has been working on Polar Bear, which is an opinionated Node.js scaffold for web applications. It follows all the 12factor principles. We have been dogfooding it on some of our clients projects.

Garren's latest contributions to PouchDB-find involve getting it to feature compatibility with CouchDB Mango. He also continues to work on NMO, Fauxton and has even contributed his first Erlang code to CouchDB! Garren also built a webpack starter template for the excellent tachyons project.

A look under the covers of PouchDB-find

PouchDB-find is a new API and syntax that allows for a simpler way to query PouchDB. It is much more suited to ad-hoc querying and a fair amount easier to learn than PouchDB’s current way of querying documents via Map/Reduce. It is a MongoDB-inspired query language to query a PouchDB database. It works with PouchDB, Cloudant Query and CouchDB Mango (CouchDB 2.0 Release).

CouchDB Internals - The Prequel

I'm currently taking the Introduction to CouchDB Development. The first part, obviously, was focused on learning Erlang. This alone has made the course worth it. I still have a long way to go in understanding Erlang but I now at least have a basic knowledge of what it's about. We have now reached the part where we are digging into the internals of CouchDB and I will be documentating some of my discoveries, assumptions and bad ideas as I grapple with the source code.

Before I start diving into the internals of CouchDB I want to understand how to get CouchDB up and running, how to run its tests, the layout of the code and how to configure it.

Setup

To get up and running is pretty straight forward. The CouchDB wiki has all this information it's just a matter of searching through it to find it. The first thing we want to do is get Couchdb running in dev mode. The wiki has a list of dependancies that need to be installed before looking at CouchDB. Using Homebrew it was pretty simple.

We need to checkout the code from its git repository

git checkout git://git.apache.org/couchdb.git
cd couchdb
./bootstrap &&./configure
make dev
make check

./bootstrap && ./configure sets up the build process. make dev will create a compiled version of couchdb that can be run by./utils/run, finally make check will run the javascript tests located at /test/javasript

Project layout

couchdb_src.png

The layout of the code is pretty self-explanatory, with the src directory being the meat of the project.

This folder breaks CouchDB's code up quite nicely. Any directory prefixed with couchdb is couchdb specific code, the other directories are all the 3rd party libraries that CouchDB uses. I will be explaining all this code more in the next couple blogs as I dive into it.

Running the tests

CouchDB is tested in two ways. It has a selection of javascript tests that can be run from the browser by visiting your local Futon and clicking on Test Suite and Run All. They can also be run from the command line /test/javascript/run. There is also a collection of etap tests found under, not suprisingly, /test/etap. These can be run individually by ./test/etap/run and the file you want to run eg ./test/etap/run ./test/etap/001-load.t. I haven't found a way to run them all at once. Currently I'm not sure when you would write a javascript test or an etap test. My best guess would be to use etap to test some of the internal api's that you couldn't get at by writing a javascript test. That would possibly mean that the etap tests are more like unit tests while the javascript tests are more functional.

Configuration files

CouchDB has two main configuration files, etc/couchdb/default.ini is the main configuration file with all the standard defaults and configurations. I recommend reading through it as it gives you a good idea of how CouchDB is configured. It also lists all the default handlers.

[httpd]
 port = 5984
 bind_address = 127.0.0.1
 authentication_handlers = {couch_httpd_oauth, oauth_authentication_handler}, \
{couch_httpd_auth, cookie_authentication_handler}, \
{couch_httpd_auth, default_authentication_handler}
 default_handler = {couch_httpd_db, handle_request}
 secure_rewrites = true
 vhost_global_handlers = _utils, _uuids, _session, _oauth, _users
 allow_jsonp = false

The [httpd] section shown above can be found in default.ini. What is interesting for the default_handler andauthentication_handlers, if you wanted to, you could write your own handlers. You could then change the handler config to use your one instead.

The other important config file is local.ini, this is used for overriding any settings found in default.ini. From what I understand of it, you can add your own handlers there for specific requests, set a different port address and a multitude of other options.

Going forward

I found that before I dive into code I always need to understand how the project works, hence this post. Maybe I need to see the forest before looking at individual trees. Exploring the code and writing this post helped me quite a bit. Leave a comment if I have missed anything or have documented something wrong.