The search for the "perfect" Advent Calendar (involves Python and Processing)

I grew up with Advent Calendars, and they are very common in the UK. Shops across the country sell calendars with typically 24 doors on them, one for each day from December 1 to December 24. Behind each door is a small gift (usually chocolate or something similarly sweet and edible).

The numbers on the doors of the calendar are usually arranged somewhat haphazardly. Part of the fun each day is finding the next door to open. It's the search for the chocolate that makes the calendars enjoyable. Here's an example layout from an Advent Calendar that I bought in Paul in London:

This is an example of a very common 6x4 (and sometimes 4x6) layout for calendars. In this blog I'm going to develop code to find "pleasing" Advent Calendar layouts in the 6x4 shape. But first, here's a little animation of the Paul calendar in action. What makes it "pleasing" is that the numbers don't cluster together (mostly).

 

So, that calendar isn't bad but it would be nice to generate a calendar where the numbers are spread about such that from day to day you have to hunt for the next number. The number you are searching for won't be near yesterday's number and if you see a number close to it then you are likely in the wrong place. That latter restriction adds a lot of pleasure because we are so used to seeing consecutive numbers that our brains instantly start looking nearby.

To do that I broke out the Python (I'm no Python expert so I appreciate Pythonistas who write in with improvements to my code).

First I needed a way of measuring whether an Advent Calendar is pleasing. I settled upon the following score. If you're not into mathematics it might look formidable but the basic idea is that if two numbers are close together (e.g. 1 and 3) then they need to be far apart when put on the calendar.

 

And we want to find a calendar with the lowest score possible (the score gets bigger when numbers that are near each other (e.g. 2 and 5) are placed close to each other on the calendar). I assume the calendar is laid out with coordinates 0 through 5 for the horizontal and 0 through 3 for the vertical and (xi, yi) are the coordinates of the number i where it appears on the calendar. For example, in the Paul calendar the number 24 appears at (x24, y24) which is (4, 2).

With that implemented in Python I can get a score for the Paul calendar above: it's 419.923652806. Now let's try a really boring calendar like this. It's not fun at all:

 

And it has a score of 474.491619783. But if you really want to have no fun at all you make this Advent Calendar which has a score of 504.143329596. It's horribly regular and boring.

So, to find the "perfect" calendar I wrote some more Python code to generate random calendars and score them. After running 1 million calendars this is the one that's the most pleasing and has a low score of 386.229085028.

I used brute force and randomness to find a pleasing calendar but there are likely other interesting techniques for laying the calendar out. I would love to hear from readers with ideas.

In the mean time feel free to steal my "Perfect Advent Calendar Layout" for your own creations!

 

Animations

The animations were made with a small program written in Processing that outputs 25 frames as separate, numbers PNG files. I combinted these into a GIF using ffmpeg.

Generate the palette:

ffmpeg -i advent-1-7-13-19-2-8-14-20-3-9-15-21-4-10-16-22-5-11-17-23-6-12-18-24-4d.png -vf palettegen palette.png

Combine the frames into a GIF:

ffmpeg -v warning -i advent-1-7-13-19-2-8-14-20-3-9-15-21-4-10-16-22-5-11-17-23-6-12-18-24-4d.png -i palette.png -lavfi "paletteuse,setpts=20*PTS" -y advent-1-7-13-19-2-8-14-20-3-9-15-21-4-10-16-22-5-11-17-23-6-12-18-24.gif

 

Code

The code can be found in the advent repository on my GitHub. advent.py is the program for scoring and generating advent calendars. advent.py is for scoring and generating calendars. advent.pde is the Processing code to animate found calendars.

 

How did Haribo do?

The real Advent Calendar at the top of this blog is from Haribo and is in the 6x4 layout and so I can feed it to my code and score it: 411.18428542. Hmm. Not so great. Here's that calendar animated:

A lot of the early numbers are close together spoiling the score and the fun of the search.

Update

A reader suggested a genetic algorithm approach and so I've added a function called swap() that generates a random calendar and then swaps random elements trying to find a better layout. It works very nicely.

Here's a layout with score 378.277000309 found using that function:

Others have found even better layouts. See the comments below and this Hacker News thread.

Turning a cheap 'police light' into an IoT device

If you've read my blog in the past you'll know I like to make ambient devices: mixtures of electronics and physical objects that blend into a home and provide a useful service. I have, for example, a model bus that shows the live times of buses near my home, and a Totoro that shows the weather forecast, and an old candle mug turned into a breathing nightlight using sea glass gathered on a beach.

The Totoro uses an ESP8266 in the form of a NodeMCU for a useful combination of WiFi connectivity, HTTP and GPIO for controlling physical devices like the LEDs in its eyes.

One of the challenges with working with these devices is updating the software on the NodeMCU when new functionality is implemented. Every code change has to be uploaded via a USB cable. For a new project I decided to make use of Cloudflare's Workers product to provide a simple API that tells an ambient device what to do.

By creating an API that just controls the physical aspects of the device (in this case a motor and an LED) I've offloaded the logic of when the motor or LED should be on or off to the Cloudflare Worker. It can be modified at will and the device has minimal code just to poll the API for updates.

 

Better living through microcontrollers

This project came about because I happened upon a cheap, rotating 'police light' in Flying Tiger which I bought on a whim thinking I'd find a use for it. It's very simple: there's a single switch that turns it on and off, a motor to rotate the internal reflector and a single LED. There's also a battery compartment for 3 AA batteries; that compartment turns out to be a good size to hide the NodeMCU in.

And 3 AA batteries is 4.5V which made me think I could get away with powering the entire device on 5V from an old wall wart. Thus was born a project to microcontroller-ize the light. In doing so I separated the LED and motor so they can be controlled separetly.

Although this project is small and a little silly it demonstrates something powerful: simple, maintainable code in the IoT device talking to a much more complex API implemented on Cloudflare's Workers which can be updated quickly. Also, the code in the Worker is totally secure, unlike the code running in the device which is at the mercy of its owner.

The demonstration project here uses a simple API that returns a state for the bluelight's motor ("on" or "off") and for the LED ("off", "steady", "flashing"). Because the LED is hooked up to the microcontroller it's possible to make it do things that are more interesting than being just on or off, hence I added a flashing mode.

The API response looks like this:

{ "motor": "on", "led": "flashing" }

The code on the NodeMCU polls the API once every 10 seconds and checks the result and sets the appropriate motor and LED states. All the logic concerning whether the motor should run or the LED be illuminated is in the Cloudflare Worker making it easy to update.

If you want to build something like this yourself... here's a description of what I did. It's not a total tutorial on how to transform the Flying Tiger lamp, so if you aren't familiar with this type of hardware hacking you'll do well to read this and some Adafruit tutorials. This might make a good first real project to get your hands dirty with some microcontroller fun.

 

Hacking the hardware

The first step is to dismantle the bluelight and cut a few wires. First, you remove the top blue portion by removing three screws and popping it off. Then hold the central black spindle in one hand and pull off the reflector (if you just pull on the reflector you'll find the entire internal mechanism falls apart... if you do this don't worry, it's fixable, just a nuisance).

Then remove the bottom four screw and take the cover off to reveal the internals. Here you are going to find a motor (with two wires), two wires disappearing up to the LED (one of those two wires has a resistor on it: that's because you can't just connect an LED to power otherwise the current will cause it to burn out).

First step is to snip the wires where they connect to the switch and the battery pack so you end up with two wires to the motor and two wires (plus resistor) to the LED. You can pop the switch out since you won't use it again (this is where I put my power socket).

In that picture you can see a capacitor soldered across the terminals of the motor. That's there to cut down on any noise generated by the motor and should be left in place. We'll be adding a diode across the motor on the circuit board as well for a circuit that eventually looks like this:

There are only three components added to the existing parts of the lamp: one resistor (R2, 330 ohms), one NPN transistor (Q1, 2N3904) and one diode (D1, IN4001). R1 and L1 are the existing LED and its current limiting resistor, M1 and C1 are the existing motor and its noise-suppressing capacitor. There are two connections to the NodeMCU labelled PORT which I connected to D8 and D7 to control them from code.

Turning the port connected to R2/Q1 sends power into the transistor Q1's base. Q1 turns on and power flows through M1/C1 to turn the motor and reflector. When the port goes low the motor stops. The diode D1 protects the transistor from a spike in current in the reverse direction that can occur when the motor comes to a halt.

One question that occurs with transistors is "How much current needs to pass across the transistor?". This is critical when choosing a suitable transistor. I happened to have some 2N3904's lying around and so I measured the current through the motor when running (roughly 70mA) and when stalled by holding the spindle with my finger (roughly 140mA). The 2n3904 data sheet says the continuous collector current is rated at 200mA for continuous operation. Should be OK. But if the motor was more power hungry a different transistor would be needed.

I built also this onto a piece of strip board with a power connector I'd scavenged from somewhere else.

Only four pins from the NodeMCU at actually soldered to the board: Vin, GND, D8 and D7. The whole contraption fits inside the battery compartment of the lamp (I removed the metal battery clips and snipped out some plastic).

The power wires and cables to the LED and motor fit through an existing hole in the plastic.

Then it's just a simple matter of putting the cover back on and uploading code. 

 

The code

As I mentioned above I've implemented fairly simple code on the NodeMCU that polls an API every ten seconds. The API is implemented as a Cloudflare Worker and any logic about when the LED should be lit, or when the motor should turn can be implemented and updated quickly in the Worker. For test purposes there's a very simple Worker in the source code repository that turns the LED on at 0743, makes it start flashing at 0744 and turns the motor on at 0745, at 0746 everything turns off again. The idea is that this warns me to leave the house to get to the bus.

Of course, any amount of much more complex logic could be implemented in the Worker. And all without ever having to touch the code embedded in the NodeMCU.

To get started you'll need to build the NodeMCU's firmware. I always use this site because it makes it easy to get a custom firmware build. For this project I built with the following modules: file, gpio, http, net, node, pwm, sjson, tmr, uart, wifi, tls.

Once you have the firmware I use esptool to flash the hardware like this:

esptool.py --port /dev/cu.SLAB_USBtoUART write_flash -fm qio 0x00000 nodemcu-master-11-modules-2018-11-30-12-31-22-integer.bin

Note that I used the integer build since I don't need floating point arithmetic. On the firmware is flashed it's then possible to connect to the NodeMCU using a terminal program (I use miniterm.py) and upload Lua code. The Makefile in the repo has commands for uploading and connecting to the device.

If you reuse my code you'll need to edit bluelight-config.lua for your WiFi network and for the API endpoint you've set up. Note that I'm using a really simple authentication scheme for the API (a shared secret in a parameter), but you could implement something much more robust for a real device.

Once the code is uploaded and the device connects to WiFi it starts hitting the API end point every ten seconds. Just modified worker.js to implement whatever logic you want and start controlling a cheap blue 'police light' from the Internet.

The beauty of this is that once the code is running on the lamp any logic changes can be done in the Worker without having to go through the somewhat laborious code uploading process for the NodeMCU. It also means that once installed somewhere there's no need to do maintenance on the device itself, just change the code running the API to make the lamp do different things.

Since the Worker can itself make calls to other APIs it's just a SMOP (Simple Matter of Programming) to make the lamp turn on when it's about to rain, your sports team is winning, the bus is about to arrive, ...