hal+json is the way to go for representing REST resources

If you are implementing REST APIs, and are thinking about using application/json as your content type, please consider application/hal+json. It allows you to represent link relations and embedded resources in a standardized manner.

The content type offers three things:

  1. Semantic link relations using the _link key. The self link is a good example.
  2. Embedded resources using the _embedded key, which are a subset of the link relations
  3. Properties with arbitrary keys

As an example, you might have a programmable power strip with three sockets. The strip itself can be modeled as a resource. It will have three optionally embedded resources with the link relation sockets. These resources themselves will have a link relation called toggle. Doing e.g. a POST on this resource will allow you to toggle it. The lovely thing here is that the API will give you the correct URL to turn on or off the socket. An example request and response:


GET http://foo.com/powerstrip/1
Accept: application/hal+json

{
"_links" : {
"self" : { "href" : "http://somedomain/powerstrip/1" },
"sockets" : [
{ "href" : "http://somedomain/powerstrip/1/sockets/1", "name" : "Socket 1" },
{ "href" : "http://somedomain/powerstrip/1/sockets/1", "name" : "Socket 2" },
{ "href" : "http://somedomain/powerstrip/1/sockets/1", "name" : "Socket 3" }
]
},
"_embedded" : {
"sockets" : [
{
"_links" : {
"self" : { "href" : "http://somedomain/powerstrip/1/sockets/1" },
"toggle" : { "href" : "http://somedomain/powerstrip/1/sockets/1/off" }
}
"state" : "on"
},
{
"_links" : {
"self" : { "href" : "http://somedomain/powerstrip/1/sockets/2" },
"toggle" : { "href" : "http://somedomain/powerstrip/1/sockets/2/on" }
}
"state" : "off"
},
{
"_links" : {
"self" : { "href" : "http://somedomain/powerstrip/1/sockets/3" },
"toggle" : { "href" : "http://somedomain/powerstrip/1/sockets/3/on" }
}
"state" : "off"
}
]
},
"numberOfSockets" : 3,
"voltage" : 230
}

The content type is described in an RFC draft, and is well on its way to become a standard. Make sure to also read the associated web linking RFC.

Amazon is already using this content type in its AppStream API.

The strength of using link relations and a content type such as HAL is that you can actually document your link relations, which are a fundamental part of your API. You should actively design the link relations and make them meaningful.

For resources, I recommend that you document the following aspects:

  1. Expected link relations and their embeddedness
  2. Attributes of the resource
  3. Example method calls for all allowed methods (GET, POST, …) with example responses 

For link relations you should document these aspects:

  1. Synopsis what the link relation means or represents, and which resource is to be expected
  2. Allowed methods with optional templated arguments
I place the example method calls with the resource documentation, since they might be redundant if specified with the link relations. But you should link to the resource documentation from the associated link relation documentation.

Extracting Names from Email Addresses

Given a CSV file with the following format:


;;firstname.lastname@somehost.com

The task is to extract the names from the email addresses. We assume that the names are seperated by periods (.) and that all the names are supposed to be capitalized and printed with strings:


#!/usr/bin/env python

import sys

if len( sys.argv ) < 2:
print "Usage: %s filename" % sys.argv[ 0 ]
sys.exit( 1 )

textFileName = sys.argv[ 1 ]
textFile = open( textFileName, "r" )

for line in textFile:
fields = line.strip().split( ';' )
email = fields[ 2 ].split( "@" )
emailName = email[ 0 ].split( '.' )
capitalizedName = [ x[:1].upper() + x[1:].lower() for x in emailName ]
print '%s;%s;%s' % ( capitalizedName[ 0 ], ' '.join( capitalizedName[ 1: ] ), fields[ 2 ] )

How to backup your Raspberry Pi SD-Card

In the following I will explain on how you can backup your Pi’s SD-card, so that whenever it breaks down, you can simply restore the backup image to a fresh SD-Card. This may also help with tinkering, when you totally screwed up your Pi’s Linux-installation. I will explain the first step specifically for Mac OS X, but you can do this similarly on other UNIXes and Linux, by using the mount and umount commands.

So, shut down your Pi using the command sudo shutdown -h now and remove the SD-card after the Pi has done so. Take the card and insert it into your Mac’s SD-card slot.

First of all, we need to find out which disk device has been assigned to the card. We can do this with diskutil list:

user@mymacintosh:~ $ diskutil list                                                                
/dev/disk0                                                                                            
   #:                       TYPE NAME                    SIZE       IDENTIFIER                        
   0:      GUID_partition_scheme                        *512.1 GB   disk0                             
   1:                        EFI EFI                     209.7 MB   disk0s1                           
   2:                  Apple_HFS M4                      511.1 GB   disk0s2                           
   3:                 Apple_Boot Recovery HD             784.2 MB   disk0s3                           
/dev/disk1                                                                                            
   #:                       TYPE NAME                    SIZE       IDENTIFIER                        
   0:     FDisk_partition_scheme                        *15.9 GB    disk1                             
   1:             Windows_FAT_32                         58.7 MB    disk1s1                         
   2:                      Linux                         15.9 GB    disk1s2                           

Then unmount the card using diskutil unmountDisk:

user@mymacintosh:~ $ diskutil umountDisk /dev/disk1                                              
Unmount of all volumes on disk1 was successful                                                     

Next we will use dd to produce an image, which we can compress using bzip2, or pbzip2 for more performance. You can use MacPorts to install pbzip2.

user@mymacintosh:~ $ sudo dd if=/dev/disk1 | pbzip2 > raspberry_pi_$(date “+%Y-%m-%d”).img.bz2 
31116288+0 records in                                                                                                           
31116288+0 records out                                                                                                        
15931539456 bytes transferred in 2347.692010 secs (6786043 bytes/sec)                                                           

Restoring the image is also easy:

user@mymacintosh:~ $ bunzip2 raspberry_pi_2014-03-22.img.bz2 | sudo dd of=/dev/disk1      

Getting Emacs for Mac OS X to honour the shell environment

If your are using the excellent Emacs for Mac OS X distribution, you may have noticed that it does not use the environment you may have defined in your ~/.bashrc. This can be a problem if you are using MacPorts, for example to install clojure and Leiningen. You will get an error message like this, when trying to run inferior-lisp in clojure-mode:

apply: Searching for program: No such file or directory, lein

This can be avoided by using the excellent exec-path-from-shell package for Emacs. It is available via (M)ELPA, just install it using list-packages, or download it from github. You can enable it, especially for OS X, by putting this in your init.el:
(when (memq window-system '(mac ns))
(exec-path-from-shell-initialize))

How to use find and rsync to copy a bunch of files to a local or remote destination

I use rsync a lot. And I use find a lot. You can use them both together, by using the magnificent xargs command. Two problems are there to solve: xargs usually just pastes the arguments from find at the end of the specified command. No way to specify the destination directory. Second, file names with spaces are a problem.

The solutions are:

  • The -print0 option of find uses 0-bytes to separate results, instead of spaces and newlines
  • The -0 option of xargs tells xargs to look for said 0-bytes
  • The -J option of xargs allows us to tell xargs where to put the file names that we feed to it.

So let’s put it together:

pi@raspberrypi ~ $ find . -name ‘your*pattern.jpg’ -print0 | xargs -J % -0 rsync -aP % user@host:some/dir/

Thanks to redeyeblind for the insightful blog post.

How to resize tmux windows using OS X terminal

A while back I pointed out a blog post by someone else, which described how to get C-left and C-right working in tmux and OS X. Now I noticed that I cannot resize windows in tmux. This is because tmux seems to expect xterm keys for C-arrow. The xterm keycodes for the arrow keys are:

  • left key: ^[[D
  • right key: ^[[C
  • up key: ^[[A
  • down key: ^[[B
  • C-left: ^[[1;5D
  • C-right: ^[[1;5C
  • C-up: ^[[1;5A
  • C-down: ^[[1;5B

I can now configure OS X terminal to send those key codes, and tmux works fine. However, other apps running in tmux will break, because they don’t expect to get xterm key codes. I found a workaround in the ArchWiki, which suggests to create your own terminfo entry. I will try that and report back here.

Update: The solution is adding two lines to the ~/.tmux.conf file:

set -g default-terminal “xterm-256color”
setw -g xterm-keys on                   
Update 2: And here is my OS X Terminal.app configuration:

Update 3: And a very last update… It seems that tmux does not support bce (background color erase), which xterm does. This is a problem for progams like htop, vim, or mc. You will see rendering errors, if you do not fix this.

So you need to make your own terminfo file and own terminal type, called xterm-256color-nobce. You can do this on the command line:

pi@raspberrypi ~ $ infocmp xterm-256color | sed ‘s/bce, //’ | sed ‘s/xterm-256color/xterm-256color-bce/g’ > xterm-256color-nobce

This will create a new terminfo source file, which does not advertise the bce feature. You can install this with the following command:

pi@raspberrypi ~ $ sudo tic ./xterm-256color-nobce

This makes the terminal type xterm-256color-nobce available. After this, change your ~/.tmux.conf once again, to use the new terminal type per default:

set -g default-terminal “xterm-256color-nobce”

Update 4: To make the Ctrl+Arrow keys also work in regular OS X Terminal, you need to edit or create ~/.inputrc to contain this:

# xterm keys for skipping a word
“e[1;5C”: forward-word
“e[1;5D”: backward-word
“e[5C”: forward-word
“e[5D”: backward-word

The “1;” variant is otherwise not recognized by GNU readline on OS X.

How to setup an OpenWRT router as a WiFi bridge to an Ethernet router

You can use an OpenWRT router or access point to connect WiFi enabled devices to a router, which only has wired Ethernet. For this to work, I am assuming you already have an access point or router running OpenWRT, in this case version 12.09, Attitude Adjustment.
Log into your router, using the LuCi frontend, and go to the Network/Interfaces tab:

There you should see your LAN device. Edit it to have an appropriate IP address from your local subnet. Most often your network will be 192.168.0.0 and your existing router will have the IP 192.168.1.1. But your mileage may vary…

Lets put in a static IP address, so we can find our router in case something goes wrong. Also make sure to set the netmask (in this case 255.255.255.0), gateway and DNS server (both probably should point to your router, 192.168.1.1).

Now go to the Physical Settings tab. Here, it’s important to check “Bridge interfaces” and to select both the ethernet adapter, most likely eth0, and the wireless network. One of the ethernet devices will say “wan”, if your are using a router instead of an access point for this. You don’t want that device.

Hit “Save & Apply” when you are ready. And be sure to have the de-bricking guide ready, if something goes wrong…

How to run tmux via ssh instantly

With my Raspberry Pi, what I do very, very often is this:

localhost$ ssh raspberrypi.local    # Here I already type the next command and wait a while
raspberrypi$ tmux attach

This is all well and good, but sometimes the Pi is down, and I will attach to one of my local tmux sessions. Very annoying. Instead you could try to do this:

localhost$ ssh raspberrypi.local tmux attach
not a terminal

Well, that did no good. So a look at the man-page of ssh or a quick search reveals this gem:

localhost$ ssh raspberrypi.local -t tmux attach

This allocates a pseudo terminal, which is needed by tmux to function correctly. This is also done by ssh, if no command is given, but a login shell is spawned.

Some tmux cheats

There’s a lot of stuff you can do with tmux. Here are some nice to know things:

  • C-b C-o: cycle contents of current windows (or swap if there are only two windows)
  • C-b C-SPC: switch between vertical and horizontal split
  • C-b n, p: next or previous screen
  • C-b [: copy mode
    • Use Emacs bindings to copy and paste:
    • C-SPC: begin selection
    • C-w or M-w to copy
  • C-b ] to yank (paste)