Showing posts with label Arduino. Show all posts
Showing posts with label Arduino. Show all posts

Serializing To A Human Interface Device

If you've read my previous post you'll know that I've been looking at a cheap and simple way of adding serial communication to a breadboard Arduino clone (such as this one). To summarise the situation so far; adding true RS-232 serial communication is both expensive and difficult as the required part is only available as a surface mount component but I discovered V-USB which allows me to emulate low speed USB devices. The end result was that I managed to use V-USB to emulate a USB keyboard. Being able to pass data from the Arduino to the PC by simply emulating key presses is useful but a) it is rather slow and b) different keyboard mappings lead to different characters being and typed and more importantly c) it doesn't allow me to send data to the Arduino. So on we go...

Let's start with what I haven't managed to achieve; a USB CDC ACM device for RS-232 communication. Unfortunately CDC ACM devices require bulk endpoints (these allow for large sporadic transfers using all remaining available bandwidth, but with no guarantees on bandwidth or latency) and these are only officially supported for high speed USB devices. V-USB only allows me to emulate low speed USB devices, and while most operating systems used to allow low speed devices to create bulk endpoints, even though this is contrary to the spec, modern versions of Linux (and possibly Windows) do not. I did manage to get a device configured correctly but as soon as I plugged it in the bulk endpoints were detected and converted to interrupt endpoints which stopped the device from working. However, all is not lost as I do have a solution which I think is just as good; serializing data to and from a generic USB Human Interface Device.

The USB specification defines the USB Human Interface Device (HID) class to support, as the name suggests, devices with some form of human interface. This doesn't mean sticking a USB cable into your arm, but rather defines common devices such as keyboards, mice and game controllers as well as devices like exercise machines, audio controllers and medical instruments. While such devices may communicate data in a variety of forms it all passes to and from the device using the same protocol. This means that when you plug any such device into practically any computer with a USB port it will be recognised and basic drivers will be loaded.

Writing code to communicate with a USB HID device isn't that much more complex than interfacing with a classic serial port and given the standard driver support we can rely on the operating system taking care of most of the communication for us.

For what follows I'm assuming the same basic USB circuit that I described in the previous post as we know it works and it is cheap to build.


Now we have the circuit let's move on to the software we need to write. Unlike with the USBKeyboard library, that powered The Caffeine Button, we will need both firmware for the Arduino and host software that will run on the PC and interface with the basic HID drivers the operating system provides. Given that you can't test the host software until we have a working device we'll start by looking at the firmware.

The first thing you have to do when constructing a HID is to define its descriptor. The descriptor is how the device presents itself to the operating system and defines the type of device as well as the size and type of any communication messages. Now you will probably never need to edit this but I thought it was worth showing you the full descriptor we are using:
PROGMEM char usbHidReportDescriptor[USB_CFG_HID_REPORT_DESCRIPTOR_LENGTH] = {
    0x06, 0x00, 0xff,              // USAGE_PAGE (Generic Desktop)
    0x09, 0x01,                    // USAGE (Vendor Usage 1)
    0xa1, 0x01,                    // COLLECTION (Application)
    0x15, 0x00,                    //   LOGICAL_MINIMUM (0)
    0x26, 0xff, 0x00,              //   LOGICAL_MAXIMUM (255)
    0x75, 0x08,                    //   REPORT_SIZE (8)
    0x95, OUT_BUFFER_SIZE,         //   REPORT_COUNT (currently 8)
    0x09, 0x00,                    //   USAGE (Undefined)  
    0x82, 0x02, 0x01,              //   INPUT (Data,Var,Abs,Buf)
    0x95, IN_BUFFER_SIZE,          //   REPORT_COUNT (currently 32)
    0x09, 0x00,                    //   USAGE (Undefined)        
    0xb2, 0x02, 0x01,              //   FEATURE (Data,Var,Abs,Buf)
    0xc0                           // END_COLLECTION
};
In this descriptor we define two reports of different sizes which we will use for transferring data to and from the device. The first thing to point out is that the specification defines input and output with respect to the host PC and not the device. So an input message is actually used for writing out from the device rather than for receiving data. Given this, we can see that the descriptor defines two message types. Firstly (lines 7 to 10) we define an 8 byte (OUT_BUFFER_SIZE is defined as 8) input report (the size is defined in bits so we have 8 bits times the count to give 8 bytes) which means we can write 8 byte blocks of data back to the PC we are connected to. The second message type is defined as a FEATURE message of 32 bytes (because IN_BUFFER_SIZE is defined as 32 and the REPORT_SIZE hasn't been redefined so it still 8 bits) which we will use for passing data from the PC to the USB device. As I said you will probably never need to edit this structure especially as you can tweak the message sizes, if necessary, by adjusting the two constants instead. If you do decided to change the descriptor it is worth noting that some operating systems are more forgiving than others. For example, with Linux if you have a defined a message of 8 bytes but only have two to send then you can do that and everything will work. Under Windows, however, if you only send two bytes the device will simply stop functioning altogether so you will need to pad the message to be exactly 8 bytes. This also means that it is easy to check that your descriptor matches what you are actually doing by quickly testing under Windows (I've been doing this with a copy of Windows XP running under VirtualBox).

Now that we know the size of the messages we will send and receive we still need to decide upon their format, i.e. the protocol we will use for our data that we are sending on top of the USB protocol. If we were only interested in sending textual data then we could send null terminated data (i.e. put a zero value byte into the array after the last byte of data), but if we want to send arbitrary bytes then using 0 as an end of data marker seems an odd choice. For this reason I've opted to set the first byte of each message to the length of the valid data in the array. This is both simple to use and results in firmware code that is slightly simpler (and hence smaller) than checking for the null terminator. This does of course mean that in an 8 byte message we can only fit 7 bytes of actual data plus the length marker (this is no different than with null terminated data of course). If you know the messages you want to send will always be of a fixed length then tweaking the buffer sizes to suit might make for a more efficient transfer of data. In general, as you will see shortly, as a user of the library a lot of these details are dealt with for you.

From the very beginning my aim was to find a drop-in replacement for the standard Arduino Serial object and so I've made the USBSerial library implement the same Stream interface. This means you can use any of the methods defined in the Stream interface for reading and writing data and the details about buffer sizes etc. are hidden within the library.

To show how easy the library is to use, here is a simple example where the sketch simply echos back any bytes that it is sent.
#include <USBSerial.h>

char buffer[IN_BUFFER_SIZE];

void setup() {
    USBSerial.begin();
}

void loop() {
  USBSerial.update();
  if(USBSerial.available() > 0) {
    int size = USBSerial.readBytes(buffer, IN_BUFFER_SIZE);
    if (size > 0) {
      USBSerial.write((const uint8_t*)buffer, size);
    }
  }
}
Note that I've used the same IN_BUFFER_SIZE constant in this example as within the library itself, as there is no reason to define a buffer that is bigger than we can ever expect to fill. The only line that you wouldn't find in a similar example using the standard Serial object is line 10 where we make sure that the USB connection is up to date (you need to do this approximately once every 50ms to keep the connection alive). Before we move on to looking at the host software there are a few things you need to know before trying to use the library.

Unfortunately the V-USB part of the library needs customizing for each project, so you can't simply drop the library into the Arduino sketchbook folder, as the USB manufacturer and product identifiers have to be unique for different devices. These identifiers are set in the usbconfig.h file. V-USB doesn't actually provide a copy of usbconfig.h what is provided is a file called usbconfig-prototype.h which you can copy and rename as a starting point. I've already done a lot of the configuration for you by editing usbconfig-prototype.h leaving just four lines you need to edit for yourself. Firstly you need to set the vendor name property by editing lines 244 and 245:
#define USB_CFG_VENDOR_NAME     'o', 'b', 'd', 'e', 'v', '.', 'a', 't'
#define USB_CFG_VENDOR_NAME_LEN 8
and then the device name by editing lines 254 and 256:
#define USB_CFG_DEVICE_NAME     'T', 'e', 'm', 'p', 'l', 'a', 't', 'e'
#define USB_CFG_DEVICE_NAME_LEN 8
These values have to be changed and can't be set to any random value because as part of the V-USB license agreement you need to conform to the following rules (taken verbatim from the file USB-IDs-for-free.txt):

(2) The textual manufacturer identification MUST contain either an Internet domain name (e.g. "mycompany.com") registered and owned by you, or an e-mail address under your control (e.g. myname@gmx.net"). You can embed the domain name or e-mail address in any string you like, e.g. "Objective Development http://www.obdev.at/vusb/".

(3) You are responsible for retaining ownership of the domain or e-mail address for as long as any of your products are in use.

(4) You may choose any string for the textual product identification, as long as this string is unique within the scope of your textual manufacturer identification.
Once properly configured you should be able to compile (I recommend using arduino-mk instead of the Arduino IDE) and use the library without issue, and without understanding how it actually works internally (if you are interested in the details then both my code and the V-USB library contain vast amounts of code comments which should help you get a better understanding) so let's move on to looking at the host software.

As I've already mentioned connecting the device to a PC usually causes generic HID drivers to be loaded by the operating system. This means that you should be able to use any programming language you like to write the host software as long as it can talk to the generic USB drivers. I've included host software written in Java using javahidapi but, for instance, you could also use PyUSB if you prefer to program using Python. The important thing to remember is the protocol for passing data that we defined earlier: data to the USB device is sent as 32 byte feature requests with the first byte being the length of the valid data in the rest of the array, while data from the USB device is in 8 byte chunks again with the first byte being the length of the valid data.

As with the firmware code we have already discussed, I've written a simple Java library to hide most of the details behind standard interfaces, which allow you to read and write data using the standard Java InputStream and OutputStream interfaces. Full details of the available methods can be found in the Javadoc but a simple echo example shows most of the important details.
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.PrintStream;

import englishcoffeedrinker.arduino.USBSerial;

public class SimpleEcho {
  public static void main(String[] args) throws Exception {
    // get an instance of the USBSerial class for the specified device
    USBSerial serial =
        USBSerial.getInstance("englishcoffeedrinker.co.uk", "EchoChamber");

    // open the underlying USB connection to the device
    serial.connect();

    // create an output stream to write characters to the device
    PrintStream out = new PrintStream(serial.getOutputStream());

    // send a simple message
    out.println("hello world!");

    // ensure the message has been sent and not buffered internally somewhere
    out.flush();

    // create a reader for getting characters back from the device
    BufferedReader in =
        new BufferedReader(new InputStreamReader(serial.getInputStream()));

    String line;
    while((line = in.readLine()) == null) {
      // keep checking the device until a line of text is returned
    }

    // display the message sent from the device
    System.out.println(line);

    // we have finished so disconnect our connection to the device
    serial.disconnect();
  }
}
Essentially, lines 10 to 14 get an instance of the USBSerial class for a specific device, in this case found via the manufacturer and product identifiers although other methods are available, and then opens the connection. Lines 16 to 23 then use the OutputStream to write data to the device while lines 26 to 35 read it back, with line 38 cleaning up by closing the connection. For anyone who is happy programming in Java this should look no different than reading or writing to and from any other type of stream, which means it should be easy to integrate within any project where you want to communicate with an Arduino.

To make life a little easier I've also included a slightly more useful demo application that effectively reproduces the Serial Monitor from the Arduino IDE. You can see it running here connected to a device running the simple echo sketch from earlier in this post, but it should work with any device that uses the USBSerial library.

I've included another example with the USBSerial library that shows you can use this for more than just echo commands. This is the CmdMsg example, which is an example I've talked about before on this blog, but this version uses the USBSerial library, and hence can be controlled through this new USB Serial Monitor, rather than using the standard Serial library.

If you've read all the way to here then I'm guessing you might want to know where you can get the library from, well it is available under the GPL licence (a restriction imposed because I'm using the free USB vendor and product IDs from Objective Development) from my SVN repository. Do let me know if you find it useful or if you have any suggestions for improvement.

When I set out to try and add serial support to a breadboarded Arduino (specifically this circuit) I did have a device I wanted to build in mind, so I'm sure at some point I'll blog again about using this library in a real device rather than just the simple examples included with the library that do nothing more than prove everything works.

The Caffeine Button

In a couple of previous posts (here and here) I've shown how easy and cheap it is to go from a prototyped setup using an Arduino to a standalone circuit built from just a handful of components. While those posts were more of an academic exercise to prove it was possible I've also now built, and blogged about, such a circuit that I'm actually using in anger. The problem is that an actual Arduino isn't just an Atmel ATMega328P-PU it also has accompanying electronics which enable you to talk to a computer via the USB connection which is great both for debugging and for interfacing external hardware to a PC.

When you connect an Arduino to a PC what actually happens is that the supporting circuitry creates a USB CDC ACM device which emulates a good old fashioned RS-232 serial port. If I wanted to add similar functionality to my standalone circuits then the most common way of doing so would be to use an FT232RL, but the chip alone would almost double the cost of the circuit plus it is only available as a surface mount part making it difficult to experiment with on a breadboard and I'm not sure my soldering skills are good enough to deal with surface mount parts either.

After pondering this for a bit and doing a little research I came across a potential solution in the form of V-USB. V-USB is a software only implementation of low speed USB for Atmel microcontrollers, such as the ATMega328P-PU. Unfortunately the distribution doesn't directly support the Arduino (it supports the ATMega328P-PU but not through the Arduino IDE etc.), however, I did find a previous attempt to add Arduino support although this project seems to have been abandoned as it hasn't seen any updates in over three years. It did, however, give me a good point to start from.

So far I haven't managed to emulate a serial port, but I have managed to make the Arduino behave like a USB keyboard which means that I can use it for debugging by having it pretend to press lots of keys in sequence, which is better than nothing. Before we get to looking at how to make use of V-USB we need to wire a USB plug up to the Arduino.


USB Connection Parts List
PartUnit CostQuantity
USB Socket, Type B£0.501
3.6V, 0.5W Zener Diode£0.0712
68Ω Resistor£0.0082
2.2kΩ Resistor£0.0081
As you can see it isn't a particularly complex circuit to build. Essentially we have the two data lines D- and D+ linked to pins 2 and 4 of the Arduino and as the USB spec states that these run on 3.3V we restrict the voltage using two 3.6V zener diodes to step down from the 5V output of the Arduino. The Arduino itself is powered from the other two USB pins which provide 5V/GND (this also means that you could use USB to power a standalone ATMega328P-PU without needing a voltage regulator and smoothing capacitors). The final connection is between pin 5 and D- via a 2.2kΩ pull-up resistor which allows the connection to be connected/disconnected from within software (in theory you can link this to V+ instead if you don't need this flexibility but I found that this meant that the hardware wasn't always recognized correctly when it was connected to a PC). For the USB plug itself, I opted to use a Type B plug (the same as the Arduino) so that I could get away with a single cable snaking across my desk, but you should be able to use any USB plug with the same circuit. One thing to be careful with when constructing the circuit is that in most cases the metal case of the USB plug is connected to the ground pin, so be careful that you don't end up with any of the other connections catching the case otherwise you might end up with them pulled to ground which will cause weird things to happen; depending which pin is involved either your PC won't recognize you have anything plugged in or it might end up thinking you have a high speed or high power device connected and things won't work properly.

Now we have the hardware what we need is some working software we can upload to the Arduino. As I said above I'm using a previous attempt to get this working as my starting point. This code hasn't been updated for over three years and I'm guessing that a number of things have changed within the Arduino libraries since then as the code didn't just work. After a bit of trial and error I discovered that the problems were mostly related to timer code that had been added to get V-USB to work with the Arduino and which was now doing the exact opposite. Removing this code got me to a point where when I plugged in the cable the Arduino was now being recognized as a USB v1.01 HID Keyboard, which I know is my device because of the vendor and product strings being shown in this screenshot.


Having got the code running I then took the plunge to update the version of V-USB being used to the latest version (currently 20121206) which, after a few little tweaks, was a success. Unfortunately because of the way the library has to be configured a single copy can't be shared between multiple projects, but in the rest of this post I'll explain how to customize the library and show you a fully worked example: TheCaffeineButton.

Firstly I've had problems compiling the library through the Arduino IDE, so I'd recommend compiling any sketches that use the library via the arduino-mk project that I discussed in a previous post. This allows you to have libraries local to a sketch by putting them in a libs subfolder. A basic sketch to use the libary then looks like the following:
// pull in the USB Keyboard library
#include <USBKeyboard.h>

void setup() {
  // TODO: setup like stuff in here...
}

void loop() {
  // poll the USB connection
  USBKeyboard.update();

  // TODO: whatever you want in here...
}
You simply include the library (line 2) and then poll the connection every time through the main loop (line 6). As I mentioned though, the library needs customizing for each project and so if you try and compile the sketch at this point you'll end up with an error:
libs/USBKeyboard/usbdrv.h:12:23: fatal error: usbconfig.h: No such file or
directory
V-USB doesn't provide a copy of usbconfig.h as it is project specific, what they do provide is a file called usbconfig-prototype.h which you can copy and rename as a starting point. I've already done a lot of the configuration for you by editing usbconfig-prototype.h leaving just four lines you need to edit for yourself. Firstly you need to set the vendor name property by editing lines 244 and 245:
#define USB_CFG_VENDOR_NAME     'o', 'b', 'd', 'e', 'v', '.', 'a', 't'
#define USB_CFG_VENDOR_NAME_LEN 8
and then the device name by editing lines 254 and 256:
#define USB_CFG_DEVICE_NAME     'T', 'e', 'm', 'p', 'l', 'a', 't', 'e'
#define USB_CFG_DEVICE_NAME_LEN 8
These values have to be changed and can't be set to any random value because as part of the V-USB license agreement you need to conform to the following rules (taken verbatim from the file USB-IDs-for-free.txt):

(2) The textual manufacturer identification MUST contain either an Internet domain name (e.g. "mycompany.com") registered and owned by you, or an e-mail address under your control (e.g. myname@gmx.net"). You can embed the domain name or e-mail address in any string you like, e.g. "Objective Development http://www.obdev.at/vusb/".

(3) You are responsible for retaining ownership of the domain or e-mail address for as long as any of your products are in use.

(4) You may choose any string for the textual product identification, as long as this string is unique within the scope of your textual manufacturer identification.
If you glance back at the screenshot I showed earlier of my example device being recognized then you can see that I followed these rules by setting the vendor name to englishcoffeedrinker.co.uk and the device name to TheCaffeineButton.

So having created a valid usbconfig.h file you can now compile the basic sketch shown above. Of course this does nothing other than allow the Arduino to be recognized as a USB keyboard. If you want to actually do something interesting then you need a little more code. As an example, I'll finally introduce you to The Caffeine Button!

Caffeine (usually in the form of coffee) is my one true addiction and so I thought I'd build a device with a single button that when pressed types out the chemical formula for caffeine: C8H10N4O2. The full sketch is fairly simple and assumes the basic circuit above with a push button between pin 12 and ground (it uses the internal pull-up resistor to keep the part list down to the single button):
// pull in the USB Keyboard library
#include <USBKeyboard.h>

// using Bounce makes working with buttons nice and easy
// http://www.arduino.cc/playground/Code/Bounce
#include <Bounce.h>

// we have the button on pin 12
#define BUTTON_PIN 12

// create a Bounce instance to manage the button
Bounce button = Bounce(BUTTON_PIN, 5);

void setup() {
  // initialise the pin the button is connected to
  pinMode(BUTTON_PIN, INPUT);

  // enable the internal pull-up resistor
  digitalWrite(BUTTON_PIN, HIGH);
}

void loop() {
  // poll the USB connection
  USBKeyboard.update();

  // check the status of the button
  button.update();

  if (button.fallingEdge()) {
    // if the button has just been pressed then...

    // ...print out the formula for caffeine
    USBKeyboard.print("C8H10N4O2");
  }
}
There are actually four different methods you can use to emulate pressing keys:
void write(byte keycode);
void write(byte keycode, byte modifiers);
void print(const char* text);
void println(const char* text);
The first two method send a USB key usage code (with or without a modifier, such as the shift key) and then release the key, while the last two methods translate alphanumeric characters (and space) into a sequence of keystrokes (followed by the enter key in the println case) to make the library a little easier to use. Unfortunately there is no guarantee that my simple sketch will always result in C8H10N402 being displayed when you press the button.

When you press a key on a keyboard a keycode is sent to the computer which determines which letter has been pressed. This allows the same physical keyboard to be used for different languages simply by changing the printed labels on each key. Unfortunately this makes it impossible to translate a string into a sequence of keycodes which will always display the same on every computer. For example, if you send keycode 28 on a computer with an English keyboard mapping you'll get a 'y', but if the keyboard mapping is German you'll get a 'z'. I've defined a bunch of constants in USBKeyboard.h (i.e. KEY_A) which work for English, and I've used the same mapping in the print and println methods. If you can set the keyboard mapping for the device to English then these methods will work properly for you, if not you might need to tweak the mappings to get what you need. You can find the full list of mappings in chapter 10 of the Universal Serial Bus HID Usage Tables document should you need more details.

So here we have the final working item. As you can see I built the main USB circuit onto a prototyping shield so I can experiment with lots of different circuits without having to keep recreating the basics every time, and in this case have simply jammed the button between pins 12 and ground.

If you've read all the way to here then I'm guessing you might want to know where you can get the library from, well it is available under the GPL licence (a restriction imposed because I'm using the free USB vendor and product IDs from Objective Development) from my SVN repository.

Whilst I haven't yet been able to emulate a serial port I haven't given up and when/if I'm successful I'm sure there will be a post about it and another Arduino library for you to play with.

Arduino Without The IDE

Some of you may remember that I've previously blogged about shrinking down an Arduino by buying the ATmega328P-PU and associated components. This is a great way of using the Arduino in a more permanent setting as it is much cheaper than using a full Arduino. I did, however, have a problem in that the Arduino IDE seemed to use the wrong upload speed when trying to burn the bootloader or upload a sketch. This meant I had to mess around copying the command out of the IDE and into a terminal and then tweaking it so that it used the right upload speed. Honestly while the IDE is easy to use (especially for novices) it isn't really a very good IDE and suffers from a number of problems which really annoy me. So I set out to look for an alternative way of compiling and uploading code to the Arduino.

Given that I already knew that the Arduino IDE is just a front-end to a number of other tools which are responsible for the compiling and uploading (which is why I could copy the relevant commands into a terminal and tweak them) I knew that it was likely someone else had already figured out how to use the Arduino without the IDE. In fact a couple of people have published details on how to do this, but the most comprehensive approach I found was by Tim Marston.

Tim's arduino-mk project supports compiling Arduino code, uploading it to an Arduino, and has a serial monitor as well. As the project name suggests this is all implemented as a classic Makefile meaning that it works from the command line, leaving me free to choose whichever code editor I want to use. The really impressive part though was that it worked first time; not only to upload code to my Arduino, but also to upload code to my hand-built replacement. There were, however, a few missing features.

Firstly the build process didn't pull in any libraries in the users sketchbook folder (which the IDE does), meaning some code I had didn't initially compile. Secondly the project didn't have any support for burning the bootloaders to new chips, something the IDE should be able to do, although in my experience it doesn't work properly (the same upload speed problem). Fortunately arduino-mk is an open-source project and Tim is more than happy to accept patches for bug fixes or new features. Whilst I don't think I've ever written a Makefile from scratch before, I know enough about how they work to add to an existing file and so with a little effort I managed to figure out adding support for both libraries in the sketchbook folder and the burning of bootloaders. Both features have now been incorporated into the latest release which means everyone else now has access to them as well.

If you are happy working from the command line and aren't really a fan of the Arduino IDE then I would definitely recommend trying out arduino-mk.

Back To Front But Functional

While the circuit required for programming an ATmega328P-PU chip (i.e. a standalone Arduino) is fairly simply (regardless of whether you are running at 8MHz or 16MHz) it would still be a pain to have to set everything up on a breadboard each time I wanted to programme a chip. The solution was to build a shield for the Arduino which I could simply slot on whenever I wanted to programme a new ATmega328P-PU.

Now I could have gone all fancy and designed a custom PCB for this project, but for such a simple project (where I wouldn't ever need more than one shield) this seemed overkill. Instead I built the shield out of a budget prototyping shield from oomlout. Obviously I don't want to solder an ATmega328P-PU to the shield as that would defeat the purpose, so I've added a 28pin DIP socket in its place. It took me a while to figure out the best placement of the components and to cut appropriate sized pieces of wire (lots of the connections are actually made simply by soldering the leads together on the underside of the board), although actually soldering everything in place was fairly quick (the advantage of lots of planning time). Most of the components were held in place by bending their leads slightly. The DIP socket, however, kept falling out. Unfortunately one of those occasions was just as I was about to solder it into place, and I managed to put it in back to front :(

Fortunately, the only problem with having the DIP socket back to front is that the notched end of the chip doesn't match up with the notch on the socket, which is why I've added a white paint mark to the socket to remind me to put the chip in the wrong way around. Anyway I now have a functioning shield which I can use to programme an ATmega328P-PU to run at either 8MHz or 16MHz and to upload sketches to it. I'd call that a success.

Double The Speed For Just 35p

In my previous post I explained how I'd eventually managed to figure out the procedure for configuring and programming an ATmega329P-PU micro-controller. The configuration in that post was deliberately designed to be as minimal as possible, which was why I was limited to using the internal 8MHz RC oscillator. While 8MHz is possibly fast enough for a lot of projects, there are occasions where speed is important; a lightening trigger for a camera for example. In these cases running the chip at 16MHz would be preferable. Fortunately adding an external 16MHz crystal to double the speed is both easy and cheap!

Before we get to the hardware though, we need add a new entry to the boards.txt file defining the new setup (see the previous post for details on this and how to upload code etc.).
atmega328p16mhz.name=ATmega328P (16 MHz external crystal)

atmega328p16mhz.upload.protocol=stk500v1
atmega328p16mhz.upload.maximum_size=30720
atmega328p16mhz.upload.speed=9600

atmega328p16mhz.bootloader.low_fuses=0xD7
atmega328p16mhz.bootloader.high_fuses=0xDF
atmega328p16mhz.bootloader.extended_fuses=0x07
atmega328p16mhz.bootloader.unlock_bits=0x3F
atmega328p16mhz.bootloader.lock_bits=0x0F

atmega328p16mhz.build.mcu=atmega328p
atmega328p16mhz.build.f_cpu=16000000L
atmega328p16mhz.build.core=arduino:arduino
atmega328p16mhz.build.variant=standard
Most of this is identical to the definition for the 8MHz version. The two main changes are to the low fuse (to specify that we are using an external full-swing crystal) and to the processing speed (changed from 8000000 to 16000000).

To make this work we need to add three small components to the previous circuits: a 16MHz crystal and two 22pF capacitors. Fortunately these are very cheap components and add just 36p to the cost of the circuit (making a total cost of £5.54 or £3.87 if you ordered parts for ten circuits). You can see how these are wired up in these diagrams (for programming on the left and in use on the right).


Configuring the fuses and uploading sketches to this new circuit is exactly the same procedure as for the 8MHz version, other than ensuring you select the 16MHz version from the boards menu.

There is, however, one important thing to note. Once you have configured the chip to run at 16MHz, you can't then use it without a 16MHz crystal present. You can't even re-programme the fuses to set it back to run at 8MHz. So if you want to go back to using the internal RC oscillator re-programme the fuses with the crystal present and then simplify the circuit.

Honey, I Shrunk The Arduino

As you might be able to tell from recent posts, I've been doing quite a bit of work with an Arduino. I've now got at least one project that I'd like to make a little more permanent, rather than it just being a bunch of components on a breadboard.

The easiest option would probably be to solder the components onto a prototyping shield (for example, these budget models from oomlout). There are two problems with this approach though. Firstly not every component is ideally suited for the form factor of an Arduino shield (for example a 16x2 character LCD takes up most of the prototyping space), and secondly to be any use a shield still needs to be connected to an Arduino. While Arduino's aren't extortionately expensive at just over £20 (depending where you buy them from), that's quite a cost to add to a project.

There are cheaper versions of the Arduino available, such as the Arduino Mini, which would bring the cost down to around £10 and which might be ideal for some projects. If you want to move your project onto a custom PCB though, you probably don't want to have to solder on a second PCB. The solution is to build the Arduino components directly into your circuit rather than relying on a pre-built solution. This should be cheaper than buying an Arduino and allows you to lay out the components in the most efficient way on a custom PCB.

The problem of course is figuring out how to construct an Arduino from the component parts. I spent quite a while trawling the internet trying to find the answers, but couldn't find a single page that collected everything you needed to know in one place, so I'm sure you can guess what the rest of this post is going to cover :)

The first decision you have to make is which Arduino compatible microcontroller you are going to base your circuit around. This will depend on the number of input/output pins you need as well as the size of the compiled sketch you intend to use. Given that I'm prototyping my ideas using an Arduino UNO, I decided to stick with the ATmega328P-PU chip the UNO is based around. This is by far the most expensive component you will need to buy, but you can pick them up for around £3. You will find that a number of places sell them with the Arduino bootloader programmed in, for which they charge a premium. Unless you are simply wanting to replace the chip in an existing Arduino UNO you don't actually need to use the bootloader, so save yourself a few pounds and buy the plain un-programmed chips.

Most of the tutorials on using a ATmega328P-PU expect you to run the chip at 16MHz (the same speed as the UNO) but this requires you to provide an external clock signal using a 16MHz crystal. As components mean costs, I decided I'd be a cheapskate and run the chip just using the internal 8MHz clock. We don't need any other components in order to programme the chip, but we do need to tell the Arduino software how to work with the chip.

Extra boards/chips are made available through the Arduino software by adding them to the boards.txt file in the hardware/arduino/ folder of the Ardunio installation (for full details of the format of this file try this Arduino wiki page). After a fair amount of messing around (starting from this example) I eventually figured out the following definition:
atmega328p8mhz.name=ATmega328P (8 MHz internal clock)

atmega328p8mhz.upload.protocol=stk500v1
atmega328p8mhz.upload.maximum_size=30720
atmega328p8mhz.upload.speed=9600

atmega328p8mhz.bootloader.low_fuses=0xE2
atmega328p8mhz.bootloader.high_fuses=0xDF
atmega328p8mhz.bootloader.extended_fuses=0x07
atmega328p8mhz.bootloader.unlock_bits=0x3F
atmega328p8mhz.bootloader.lock_bits=0x0F

atmega328p8mhz.build.mcu=atmega328p
atmega328p8mhz.build.f_cpu=8000000L
atmega328p8mhz.build.core=arduino:arduino
atmega328p8mhz.build.variant=standard
I'm sure that to most people (me included) that isn't self explanatory, so let's step through it to see what it means.

Firstly, line 1 simply assigns a human readable name to the board definition (this is what will apeartr on the Boards menu). Lines 3-5 state that we will be uploading to the chip using the right protocol to use the Arduino as the ISP (In System Programmer), the maximum sketch size, and the speed at which we will upload to the chip. Unfortunately, at least in version 1.0 of the Arduino software the upload speed seems to be ignored which will come back to bite us later.

Lines 7 to 11 specify the fuse settings. These are essentially used to configure the chip. I used Engbedded's fuse calculator to figure out the correct settings. Note that I'm not using the default fuse settings that the calculator suggests as I've turned off the Divide clock by 8 internally option. I found that otherwise the chip seemed to be running at 1MHz rather than 8MHz (i.e. the blink example sketch while setup to blink at 1 second intervals appeared to be actually blinking at 8 second intervals). I also specified the smallest bootloader space option, given that we aren't going to use one. Lines 13 and 14 specify the chip type and speed and finally lines 15 and 16 specify which internal libraries to compile the sketch against (in this example the standard variant which is the UNO).

So now that the Arduino software knows how to programme the board, we need to turn our existing Arduino UNO into an ISP which we can use to programme the chip. Now there is an example sketch called ArduinoISP bundled with the Arduino software, but try as I might I couldn't get it to work. Instead I'm using a slightly improved version released by Adafruit. Simply download the new version into your sketchbook folder and then upload it to your Arduino UNO the same way you would any normal sketch. Now wire the ATmega382P-PU you want to programme to the Arduino as shown to the left (this is also documented in the comments at the top of the ArduinoISP sketch, which also talks about adding LEDs for feedback; feel free to add these if you wish). Now in the IDE, from the Tools menu set the Programmer to ArduinoISP and the board to ATmega328P (8 MHz internal clock).

Now before we can actually programme the chip we need to set the fuses to the correct values as discussed above. To do this simply choose Burn Bootloader from the Tools menu (note that this is slightly misleading in that we aren't burning a bootloader as we didn't specify one in boards.txt, but this menu item sets the fuses as well as burning the bootloader if specified). Hopefully this will "just work", but for some reason my version of the IDE uses the wrong upload speed and as such this fails. Fortunately it's fairly easy to do manually. The easiest way is to get the IDE to generate the command line for you. To do this open the Preferences dialog from the File menu, and turn on the verbose output during upload. Now try to burn the bootloader again. It will still fail, but the first message will be the command line it is trying to use. On my machine this is:

/usr/share/arduino/hardware/tools/avrdude -C/usr/share/arduino/hardware/tools/avrdude.conf -v -v -v -v -patmega328p -cstk500v1 -P/dev/ttyACM0 -b19200 -e -Ulock:w:0x3F:m -Uefuse:w:0xFF:m -Uhfuse:w:0xD9:m -Ulfuse:w:0x62:m

This command is setting the baud rate to 19200 which is an awful lot faster than both the upload speed we specified in boards.txt and the baud rate specified in the ArduinoISP sketch, both of which use 9600. So simply change the -b19200 to -b9600 and execute the command manually (from a terminal or command prompt etc.). Hopefully that should work correctly. Once you have the fuses set, assuming you don't want to change the way the chip is configured, you won't need to do this step in the future.

Now you are ready to actually upload a sketch to the new chip. I usually test that everything is working using the simple Blink example that comes with the Arduino IDE. As you have already connected the Aruino to pin 13 of the new chip (this doubles as the SCK pin) I usually change the sketch to use pin 8 which is the bottom right pin if you have yours in the same orientation as the above diagram, so that you can see the sketch working without having to change any of the wiring used for programming. Now to upload to the new chip you need to use the "Upload Using Programmer" option from the File menu (making sure that the programmer and board are still set correctly) and not the Upload button on the toolbar. If setting the fuses failed the first time and you had to manually change the baud rate then you will probably run into the same problem here as well, but the solution is the same (run the command from the IDE then copy it out, change the baud rate, and then run it manually). Once it has uploaded successfully you should find your LED is blinking on and off at one second intervals.

Now, as the main aim was to remove the need to dedicate a full Arduino UNO to each project we need to power the chip in some other way than by being connected to the UNO. Fortunately this is also fairly simple as you can see from the setup to the left. This uses just four components; a voltage regulator, two capacitors, and a 9V battery connector.

I've actually found that the supply from a fresh 9V battery is stable enough that you can do without the two capacitors, but as they are stipulated by the datasheet for the voltage regulator I've included them.

Minimal Arduino Part List
PartUnit Cost
ATMEL ATmega328P-PU£3.02
5V Voltage Regulator£0.40
100µF Capacitor£0.151
9V Battery Connector£0.60
These extra components are, as you can see from this table, much cheaper than the main ATmega328P-PU, and adding the two capacitors adds just 30p to the cost. In total the components add up to just £4.32 + VAT for a grand total of £5.19 -- or around half the cost of the Arduino Mini, or a quarter of the cost of an Arduino UNO. The circuit could be made even cheaper if you were willing to buy the components in bulk; for example if you buy 10 ATmega328P-PU then they cost just £1.63 each, so you could make 10 of these circuits for just £3.52 each.

The one thing I haven't tried yet is using any analog inputs with this setup. The datasheet for the ATmega328P-PU says that the AREF pin needs to be connected to power via a low-pass filter to remove noise if the analog to digital converter is being used. I've seen a number of websites that at least suggest this isn't necessary, especially if powering via a battery, rather than an alternating current based power source. I'll figure this out in due course but even if I do need to add extra components they will be priced in the pence so won't add much to the overall cost.

Levelling Out The Wear

I've been playing around with an Arduino since May, but up until now I haven't had any need to use the 1024 bytes of EEPROM embedded in the ATMEGA328 microcontroller at the heart of the Arduino UNO. EEPROM, or to give it its full name Electrically Erasable Programmable Read-Only Memory, is really useful for storing configuration data as any values saved to EEPROM are retained even when the power is removed.

The main problem with EEPROM is that it can quickly wear out. The datasheet for the ATMEGA328 gives an expected life of just 100,000 writes for each 32 byte page. If you are just using the EEPROM to save configuration data, which won't change very often, then this won't be a problem, but if you intend to frequently write to the EEPROM this could quickly become a real issue. To mitigate this problem what we need is a wear levelling algorithm.

In the sketch I'm working on I have just four bytes that I need to store as configuration data. The easiest option would be to simply store them at offsets 0, 1, 2 and 3, but of course this would mean that every time they were changed the same four bytes (and hence the same 32 byte page) would be re-written, while the rest of the EEPROM would remain unused. Clearly this won't result in even wear across the EEPROM.

My solution, which probably isn't optimal, is to simply shift the data along by one byte each time it needs to be written. This completely ignores the idea of 32 byte pages, but if data is being written often enough then hopefully the pages should get fairly evenly used.

Now the main problem of shifting the data along by one byte each time it changes is knowing where to read it back from when the Arduino starts up. The trick here is to employ some form of multi-byte marker that denotes the location of the data block. In this example I'm going to use the four characters "TS01" to stand for Test Sketch version 1. This allows me to increment the version number if I change the format of the stored data in the future, and tells me which sketch the data refers to. This second point is important if you are going to use the EEPROM across different sketches (less important if you dedicate an Arduino to a single project). So enough talking lets start putting this into practice.

Firstly, while the core Arduino software does include an EEPROM library it's fairly rudimentary. Fortunately another Arduino user has contributed the EEPROMex library which offers a whole bunch of useful methods (note that I did have a few problems getting this library to work; you have to make sure that the folder and filenames all share the same capitalization -- EEPROMex, and the header file doesn't need to include the original EEPROM library). This extended library is capable of writing arbitrary structures to EEPROM, so instead of having separate variables for the configuration data it's easier to store them in a custom structure giving us the following code:
#include <EEPROMex.h>

#define CONFIG_VERSION "TS01"

struct ConfigStructure {
    byte a, b, c, d;
    char version[5];
} config = {
    76, 74, true, 15,
    CONFIG_VERSION
}, stored;

int offset = 0;
As you can see this snippet imports the library, defines a constant to hold the multi-byte marker and then creates the custom structure. We also create two instances of the structure config and stored giving the first one some sensible default values for each variable, so that if we don't find any stored settings the sketch can still operate. You'll also notice that I've put the multi-byte marker at the end. This is so that it only gets written if we have successfully written the rest of the data -- I'll come back to this point later.

Now before we get into actually reading and writing the settings I'm going to introduce a utility function for comparing the current and stored settings.
boolean compareSettings() {
  if (config.a != stored.a) return false;
  if (config.b != stored.b) return false;
  if (config.c != stored.c) return false;
  if (config.d != stored.d) return false;
 
  return true; 
}
Ideally I would have liked to actually pass the two instances to the function so that it wasn't a hard coded comparison of the config and stored instances, but there is an issue with passing user defined structures to functions in the version of the Arduino IDE I'm using (it has been fixed in 1.0.2 but I'm still on 1.0.0). Anyway this function simply checks that the two instances hold identical values for each variable.

So now we can turn our attention to reading stored settings out of the EEPROM.
void setup() {
  //search through the EEPROM for a valid config structure
  for ( ; offset < EEPROMSizeUno-sizeof(stored) ; ++offset) {
      
      //read a struct sized block from the EEPROM
      EEPROM.readBlock(offset, stored);

      if (strcmp(stored.version, CONFIG_VERSION) == 0) {
        //if the block has the right version tag then we can
        //assume that the rest of the struct is valid

        //overwrite the default settings with those we have
        //just loaded from the EEPROM
        config = stored;

        //we don't need to look any further so break out of
        //the loop to speed things up a bit
        break;
     }
  }
  
  if (!compareSettings()) {
    //if they haven't changed the settings from the default then
    //there is no reason to waste an EEPROM write in storing them
    //so assume the stored version is the default
    stored = config;  

    //as there wasn't anything in the EEPROM set the offset to -1
    offset = -1;
  }
}
Hopefully the comments make this code easy to understand, but essentially all it does is read a block of data from the EEPROM, see if it is a valid settings object by checking if the version variable is the same as the CONFIG_VERSION constant. If it is then we ensure that the current config structure is the same as the stored structure, otherwise we move on 1 byte and try again (this all happens in lines 3 to 20, ). Once we have searched through the entire EEPROM, either we will have found some settings and config and stored will be equal, or we haven't found any and they will differ. Note that we control the loop using the EEPROMSizeUno constant defined by the EEPROMex library, if you have a different Arduino then you will need to change this. If, when we have searched through the EEPROM, config and stored aren't the same then lines 22 to 30 make them so (so we don't store the default values to EEPROM for no reason) and reset the offset so that when it is incremented to save the settings it moves to the beginning of the address range.

Now in the main part of your sketch when you decided the settings need to change you can simply update the config instance and call a saveSettings function to actually write them to the EEPROM.
void saveSettings() {
  if (!compareSettings()) {
    //we need to store the new settings in EEPROM

    //move on one position from the previous place we stored things
    ++offset;

    //if writing at offset would mean going outside the EEPROM limit then
    //reset the pointer to the beginning of the EEPROM
    if (offset > EEPROMSizeUno-sizeof(config)) offset = 0;

    //do the actual EEPROM writing
    EEPROM.updateBlock(offset, config);

    //stored should now be the same as config
    stored = config;
  }
}
Essentially if the config and stored instances differ then we need to write config to the EEPROM. We increment the offset (the wear levelling part), check we haven't gone outside the valid range, and then write the block (we actually use update so that we don't write bytes that won't actually change) and now we know that config and stored are the same again.

I mentioned earlier that the reason for putting the multi-byte marker at the end of the config structure was so that I knew once it had been written then the rest of the data must have been as well. The problem here is that if writing fails before the start of the marker we will have no way of knowing, other than we will see random configuration data. The easiest option would be to add a different marker at the start. If both markers are present and in the right place then the data between them must be valid. The problem here is that a four character marker takes up more space than the config data I'm saving so having two of them seems like a waste.

The approach I'm taking is to compute a hash of the four config values and store this as the first piece of data. This means that when I read the structure back I can check for the multi-byte end marker and that the data matches the hash. The problem is that the hash function is specific to the number and type of config variables you want to store and so isn't a generic solution but something you would have to adapt to your needs. As an example though here is how I'm currently hashing the four bytes of configuration data.
const int prime = 31;
int result = 1;
result = prime * result + config.a;
result = prime * result + config.b;
result = prime * result + config.c;
result = prime * result + config.d;
This code (which is based on what Eclipse generates for a four byte java structure) generates an integer (two bytes on the Arduino) based upon the four config values. Changing any of the config values changes the result of the hash function. There is also a second reason for using a hash function: data stored in EEPROM can degrade over time and the hash function allows us to check that the data hasn't changed since we stored it.

I know this hasn’t led to a single example sketch you can use, but hopefully the snippets I've presented will come in handy.

An 8-Bit Waterfall

So far my Arduino projects have all needed only a small number of output pins. In a previous post I talked about moving some of the logic off the Arduino and into other integrated circuits in order to free up pins on the Arduino. While this can help, I'm sure there will come a time when I'll need more output pins then the Arduino Uno has to offer. One solution would be to upgrade to an Arduino Mega as it has 54 digital output pins compared to the 14 digital pins on the Arduino Uno. The cost difference of around £15, however, doesn't make this an attractive option. Fortunately there is a much cheaper way of adding extra output pins; 8-bit shift registers.

Whilst you don't need to know the full details of how a shift register works an understanding of the basic principle will help. Essentially data is shifted into the chip one bit at a time using two pins; a data pin specifying the bit value and a clock pin that is triggered to show when the bit has been set. Once all the bits have been passed to the chip a third pin (known as the latch) is triggered at which point each bit is output on a separate pin of the chip. In the case of an 8-bit shift register (the 74HC595N for example) this allows us to control 8 pins (one for each bit) using just three pins on the Arduino (one each for data, clock and latch). In this way we can add five new digital output pins to an Arduino for just 70p (I got five chips for £3.50 from oomlout).

The 74HC595N is in fact such a common way to add extra pins to an Arduino that there is a simple tutorial already available, although I didn't find this before I'd already wired up the circuit you can see to the left. Due to a shortage of space on my breadboard I only bothered to wire LEDs to three of the output pins, but it should be clear how to add LEDs to the other five output pins should you wish to. Also it's worth noting that I haven't needed to add a capacitor to the latch pin (a capacitor would ensure a smooth flow of power, but the Arduino on it's own seems to work fine and I didn't have any spare capacitors lying around). It's important to make sure you pull the enable pin to ground otherwise you'll find the lights flicker uncontrolably, if they come on at all.

Using this circuit I can cycle through all combinations of the three LEDs by writing the numbers 0 to 7 to the data pin. This works because we only need three bits to encode numbers 0 to 7 which will cause the relevant pins to go LOW/HIGH as appropriate. For those of you who aren't so comfortable with binary the following table will help.

IntegerBinary
000000000
100000001
200000010
300000011
400000100
500000101
600000110
700000111

So to turn on the yellow LED we would write 2 to the chip or to turn on the green and orange LEDs we would write 5. The following simple sketch can be used to show that this does indeed work.
//the pins we are using
int latchPin = 2;
int clockPin = 3;
int dataPin = 4;

void setup() {
  //set all the pins used to talk to the chip
  //as output pins so we can write to them
  pinMode(latchPin, OUTPUT);
  pinMode(clockPin, OUTPUT);
  pinMode(dataPin, OUTPUT);
}

void loop() {
  for (int i = 0; i < 8; i++) {
    //for each of the numbers 0 to 7...

    //take the latchPin low so the LEDs don't
    //change while we are writing data
    digitalWrite(latchPin, LOW);

    //shift out the bits
    shiftOut(dataPin, clockPin, MSBFIRST, i);  

    //take the latch pin high so the pins reflect
    //the data we have sent
    digitalWrite(latchPin, HIGH);

    // pause before next value:
    delay(2000);
  }
}
Now while it is possible to control all 8 pins in the same way, sending a number from 0 to 256 to set the 8 pins isn't really an ideal approach (unless there are a small fixed number of output you need at which point you could define a constant for each of them and write those out as necessary). Ideally what we want to be able to do is to simply set each pin as LOW or HIGH without changing the state of any of the other pins. Fortunately this is easy to do using the Arduino bitSet and bitClear functions. For example, take this snippet of an Arduino sketch.
//assume all pins start out LOW
byte state = B00000000;

//set the third pin high, i.e.
//state = B00000100
bitSet(state, 2);

//take the latchPin low so the LEDs don't
//change while we are writing data
digitalWrite(latchPin, LOW);

//shift out the bits
shiftOut(dataPin, clockPin, MSBFIRST, state);  

//take the latch pin high so the pins reflect
//the data we have sent
digitalWrite(latchPin, HIGH);
Most of this code is from the previous sketch, the interesting bit is line 6. Here we set the 2nd bit (which forces it HIGH). Remember that, as in most computer situations, numbering starts from 0, so this sets the third pin HIGH, which given our example circuit would turn on just the orange LED. The bitClear function works in exactly the same way, but pulls the bit LOW rather than HIGH. It's reasonably straightforward to convert these ideas into a couple of functions which make working with a shift register easy. Partly as a learning exercise, but mostly as I think it will be useful, I've actually created a new Arduino library called ShiftRegister so that I can easily include the functionality in any sketch where I need it. You can grab the library from my subversion repository. The library includes the following example sketch which assumes the same circuit we've already looked at.
/**
 * ShiftRegister
 * Copyright (c) Mark A. Greenwood, 2012
 * This work is licensed under the Creative Commons
 * Attribution-NonCommercial-ShareAlike 3.0 Unported License.
 * To view a copy of this license, visit
 * http://creativecommons.org/licenses/by-nc-sa/3.0/.
 **/

#include <ShiftRegister.h>

ShiftRegister sr(4,3,2);

void setup() {
  //there is nothing to do
}

void loop() {
  sr.setPin(0, HIGH);      
  delay(2000);
       
  sr.setPin(1, HIGH);
  delay(2000);
    
  sr.setPin(2, HIGH);      
  delay(2000);
     
  sr.setPin(0, LOW);      
  delay(2000);
      
  sr.setPin(1, LOW);      
  delay(2000);

  sr.setPin(2, LOW);      
  delay(2000);
}
If you run this sketch on the Arduino you'll see that it simply lights up the three LEDs starting with the green one, and then turns them off again in the same order. I've tried to make it easy to configure the libary and as you can see in line 12 you can specify which pins you are using on the Arduino. By default calling setPin with two arguments automatically causes the changes to be sent to the shift register. If you want to send a number of changes at once then you can as this snippet shows.
//set pins 0 and 2 high
sr.setPin(0, HIGH, false);
sr.setPin(2, HIGH, false);

//push the new values to the shift register
sr.sync();
By this point you may be wondering why I called this post "An 8-Bit Waterfall", well the answer lies in one of the pins of the 74HC757N which we haven't looked at yet; the overflow pin.

When you have shifted in 8 bits the register is full. Shifting in a 9th bit causes the first bit you pushed in to be pushed out on the overflow pin. So just like water cascades down a waterfall so data can cascade down a series of shift registers. The circuit on the left shows how we can link two 8-bit shift registers together to give us 13 new digital output pins for just £1.40.

You can see the waterfall in action by running the example ShiftRegister sketch using this new circuit. What you'll notice is that the second set of LEDs is always one step behind the first set. To make actual use of the second register we need a new version of the library which can handle multiple registers. My imagination failed me and so the new library is called ShiftRegisters! You can also grab this version of the library from my subversion repository. This library includes the following example sketch.
/**
 * ShiftRegisters
 * Copyright (c) Mark A. Greenwood, 2012
 * This work is licensed under the Creative Commons
 * Attribution-NonCommercial-ShareAlike 3.0 Unported License.
 * To view a copy of this license, visit
 * http://creativecommons.org/licenses/by-nc-sa/3.0/.
 **/

#include <ShiftRegisters.h>

ShiftRegisters sr(4,3,2,2);

void setup() {
  //there is nothing to do
}

void loop() {
  sr.setPin(0, HIGH);      
  delay(2000);
       
  sr.setPin(1, HIGH);
  delay(2000);
    
  sr.setPin(2, HIGH);      
  delay(2000);

  sr.setPin(8, HIGH);      
  delay(2000);
       
  sr.setPin(9, HIGH);
  delay(2000);
    
  sr.setPin(10, HIGH);      
  delay(2000);
     
  sr.setPin(0, 0, LOW, true);      
  delay(2000);
      
  sr.setPin(0, 1, LOW, true);      
  delay(2000);

  sr.setPin(0, 2, LOW, true);      
  delay(2000);

  sr.setPin(1, 0, LOW, true);      
  delay(2000);
      
  sr.setPin(1, 1, LOW, true);      
  delay(2000);

  sr.setPin(1, 2, LOW, true);      
  delay(2000);
}
This example simply turns the six LEDs on and then off in sequence, but shows that there are two ways of addressing each pin. Firstly we can simply address each pin in turn; so the pins on the first chip are 0 to 7 and on the second 8 to 15, etc. Or we can specify pin 0 to 7 for each chip (numbered 0 and 1). Note that if you use the second approach you have to use the four argument version of the method and specify if you want sync called for you or not (this is because the arguments preclude me producing a three argument version as the compiler gets confused). The second method will be slightly quicker as there is a little less maths involved, although which you find most intuitive is up to you. Note also that line 12 now includes (as the last argument) the number of shift registers we have in sequence. If you don't specify the number then the library assumes there are 32 registers giving you 256 pins in total (or 233 new pins). Given the way the library is written (the pin variable is a byte) 32 registers is also the maximum it can support.

You could use the second library with just one shift register if you wanted to, but I've kept the two versions separate in case code size is an issue as the second version is larger and there is no point including extra code within a sketch if it isn't needed.

So for my initial investment of £3.50 I can add an additional 37 digital output pins to my Arduino Uno. Of course you can't use these pins for input as you could with the extra pins on an Arduino Mega, but you can also add extra input pins in a similar way.

Fritzing: Custom Components

So I've been playing around with a few more electronic circuits over the last week or so and found myself wanting to move some of the simple logic processing off the Arduino. As an example, I wanted to implement the following logic table

InputsOutputs
ABYZ
0000
0101
1111
1011

Now if you look closely at this table then you will see that Y is the same as A, while Z is simply A or B. Now this would be simple to implement using an Arduino, in fact you would just need the following sketch.
void setup() {                
  pinMode(11, OUTPUT); // output Y
  pinMode(12, OUTPUT); // output Z
  
  pinMode(8, INPUT); // input A
  pinMode(9, INPUT); // input B  
}

void loop() {
  digitalWrite(11, digitalRead(8));
  digitalWrite(12, digitalRead(8) | digitalread(9));
}
This could be simplified slightly by not using the Arduino to set Y and just using A directly, but it would still require three pins to read A and B and generate Z. Given the limited number of pins on an Arduino this seems wasteful. Now most logic gates can be built from a combination of transistors, but again they can quickly get complicated, especially if you need more than one logical operation. Fortunately there are a number of cheap integrated circuits that you can buy that implement the standard logical operations (and some not so common). After a quick trip to Maplin on Thursday, 99 pence got me a single chip with four 2-input OR gates: the 74HCT32N from NXP. It was simple to add it to the breadboard and wire it into my circuit instead of using the Arduino to compute the OR.

The only problem arose when I went to record my circuit using Fritzing. Clearly Fritzing can't support every single component in the world, and it didn't have a specific component to represent the 74HCT32N. It does, however, have a number of generic parts which are easy to customize. One of these allows you to represent any integrated circuit by specifying the number of pins etc. The problem is that this generates a breadboard view where the chip is simply labelled IC and a schematic view that just labels the pins on the chip and tells you nothing about the internal workings. Now for a complex chip this makes sense but for something as simple as OR gates it would be helpful if the schematic view included them.
The left hand chip is how Fritzing represented the 74HCT32N when I used the generic IC part, the chip on the right is how I wanted it to look. Now given this is a screenshot I obviously succeeded, and the rest of this post will explain how.

Now you can create parts from scratch, but after an hour or so of trying and getting nowhere, I hit upon a simpler solution.

The first step is to use the parts editor (ignoring the warning about it being bug ridden) to label the pins and properly name the component etc. Once you are happy with those values click to save it as a new part.

The new part will be listed in the "MINE" parts bin. Simply right-click on it and choose "Export Part". This will give you a file with a fzpz extension. This is just a zip file which you need to unpack. You should find that you now have five files: a fzp file which describes the part and four SVG image files for the icon, breadboard, schematic and PCB views.

You can now go ahead and edit the SVG files. Be careful not to edit or alter any of the connectors in the images but otherwise it's safe to make changes. If you are wanting to draw logic gates inside a chip then you can simply grab the appropriate images from Wikipedia which makes life really easy as all you need to add are the connections between the gates and the pins. As well as improving the schematic view I also edited the icon and breadboard views: the breadboard view now gives the full part number and the icon shows OR instead of IC. Note that to avoid any problems try saving to plain SVG rather than any application specific extension.

Once you are happy with the images go back to the part editor and replace the images with the new versions (even though I didn't edit the PCB image I replaced it anyway as a bug in the part editor means it seems to get lost if you don't). Once you are happy you can export the part again if you want to keep a backup copy or share it with other Fritzing users.

I'm going to make any custom components I produce available to anyone who wants them. For now there is just this single chip but they will all appear here.

An Arduino Powered (Scale) Speed Trap

After a break of around two decades I've recently started building a model railway. One of the issues I've faced is trying to work out how fast I should be running the trains so that their speed reflects reality given the scale at which they are modelled. I'm guessing the details won't interest everyone reading this post but if you are interested then I've blogged about this from the model railway side on one of my other blogs. Suffice it to say that what I needed was to be able to measure the time it took for a model locomotive to travel a certain distance. Now I could have used a ruler and a stop watch but this would never be very accurate. So being me I turned to a more computer oriented solution: an Arduino powered speed trap!

The brief I set myself was simple:
  • two switches to measure the time taken to travel a given distance
  • a green LED to signify the train was travelling below a given speed limit
  • a red LED to signify the speed limit was being broken
  • full speed details passed back to the PC for display
The main question was what form of switches should I use. Physical switches were out as I would never be able to accurately place them on the track in a way that any locomotive would be able to trigger them without incident. A reed switch would be easy to use and to hide on the layout, but would mean adding a magnet to each locomotive which seemed a bit daft. An infrared beam across the track would also work, but hiding it on the layout would be difficult. In the end the only sensible idea I could come up with was using a light dependent resistor (LDR) and watching for a sudden change in resistance as the light was blocked by the moving locomotive.

So on my way home on Thursday I called in at my local Maplin store and bought two of the smallest and flattest LDRs they had (specifically the 1.8k-4.5k version).

Now while I knew that the more light you shine on an LDR the less resistance it has I wasn't sure of the best way of making use of this information in conjunction with the Arduino. Fortunately the web is awash with information and tutorials and I quickly came across the solution.

So with my two LDRs, two LEDs and a bunch of resistors I knocked together the following (note that both views were generated at the same time using Fritzing, I really am impressed by this application).


As you can probably see this is a little more complicated than it needs to be as it uses two resistors for each LED, but this was the best I could manage with the resistors I had.

Of course the hardware is only half of the solution. Without appropriate microcode the Arduino isn't going to do anything useful. Fortunately I'm better at writing software than I am at designing circuits so this half was easier.

The code is (fairly) straightforward. Essentially it's a state machine that (ignoring invalid inputs) follows the following steps:
  1. wait until sensor 1 is triggered
  2. when sensor 1 is triggered record the time
  3. wait until sensor 2 is triggered
  4. determine the time difference between the two sensors being triggered and use this to calculate the speed of the locomotive
  5. wait until both sensors have returned to normal then return to step 1
This is easy to implement and the full Arduino sketch is as follows:

/**
 * ScaleSpeed
 * Copyright (c) Mark A. Greenwood, 2012
 * This work is licensed under the Creative Commons
 * Attribution-NonCommercial-ShareAlike 3.0 Unported License.
 * To view a copy of this license, visit
 * http://creativecommons.org/licenses/by-nc-sa/3.0/. 
 **/

//keep the sketch size down by only compiling debug code into the
//binary when debugging is actually turned on
#define DEBUG 0

//all possible states of the state machine
const byte TRACK_SECTION_CLEAR = 0;
const byte ENGINE_ENTERING_SECTION = 1;
const byte ENGINE_LEAVING_SECTION = 2;

//the current state machine state
byte state = TRACK_SECTION_CLEAR;

//the analog pins used for each sensor
const byte SENSOR_1 = 0;
const byte SENSOR_2 = 1;

//the digital pins for the signalling LEDs
const byte GREEN_LIGHT = 13;
const byte RED_LIGHT = 12;

//the threshold values for each sensor
int sensor1 = 1024;
int sensor2 = 1024;

//intermediate steps to calcualte the scale distance we are measuring
const float scale = 76; //this is OO gauge
const float distance = 74; //measured in mm
const float scaleKilometer = 1000000.0/scale;

//This is the only value we actually need to do the calculation
//We could do this calculation on the computer and pass it across
//or store it in EEPROM. it all depends if we expect to always be
//attached to a computer or if we want to run standalone etc.
const float scaleDistance = distance/scaleKilometer;

//the track speed limit in mph
const float speedLimit = 15;

//the time (in milliseconds from the Arduino starting up that the
//first sensor was last triggered
unsigned long time;

void setup(void) {
  //enable output on the digital pins
  pinMode(GREEN_LIGHT, OUTPUT);
  pinMode(RED_LIGHT, OUTPUT);

  //turn on both LEDs to show we are calibrating the sensors
  digitalWrite(GREEN_LIGHT, HIGH);
  digitalWrite(RED_LIGHT, HIGH);
  
  //configure serial communication
  Serial.begin(9600);
  
  //let the user know we are calibrating the sensors
  Serial.print("Callibrating...");
  
  while (millis() < 5000) {
    //for the first five seconds check and store the lowest light
    //level seen on each sensor
    sensor1 = min(sensor1, analogRead(SENSOR_1));
    sensor2 = min(sensor2, analogRead(SENSOR_2));
  }
  
  //the cut off level for triggering the state machine
  //is half the resistance seen during calibration
  sensor1 = sensor1/2;
  sensor2 = sensor2/2;  
  
  //we have now finished callibration so tell the user...
  Serial.println(" Done");
  
  //... and set the signalling to green (i.e. we haven't yet seen
  //anything break the speed limit!
  digitalWrite(GREEN_LIGHT, HIGH);
  digitalWrite(RED_LIGHT, LOW);
}

void loop(void) {
    
  if (state == TRACK_SECTION_CLEAR) {
    //last time we checked the track was clear
    
    if (analogRead(SENSOR_1) < sensor1) {
      //but now the first sensor has been triggered so...
      
      //store the time at which the sensor was triggered
      time = millis();

      //advance into the next state
      state = ENGINE_ENTERING_SECTION;
      
      #if (DEBUG)
        Serial.println("Train entering measured distance");
      #endif

    }
  }
  else if (state == ENGINE_ENTERING_SECTION) {
    //the last time we checked the first sensor had triggered but
    //the second was yet to trigger
    
    if (analogRead(SENSOR_2) < sensor2) {
      //but now the second sensor has triggered as well so...
      
      //get the difference in ms between the two sensors triggering
      unsigned long diff = (millis() - time);

      //calculate scale speed in kph
      //3600000 is number of milliseconds in an hour
      float kph = scaleDistance*(3600000.0/(float)diff);

      //convert kph to mph
      float mph = kph*0.621371;
      
      //report the time and speed to the user
      Serial.print("Speed Trap Record: ");
      Serial.print(diff);
      Serial.print("ms ");
      Serial.print(kph);
      Serial.print("kph ");
      Serial.print(mph);
      Serial.println("mph");
      
      if (mph > speedLimit) {
        //if the speed we calculated was above the speed limit
        //then turn off the green LED and turn on the red one
        digitalWrite(GREEN_LIGHT, LOW);
        digitalWrite(RED_LIGHT, HIGH);
      }
      else {
       //if the speed we calculated was not above the speed limit
       //then turn off the red LED and turn on the green one
       digitalWrite(GREEN_LIGHT, HIGH); 
       digitalWrite(RED_LIGHT, LOW);
      }
      
      //move into the next state
      state = ENGINE_LEAVING_SECTION;
    }
  }
  else if (state = ENGINE_LEAVING_SECTION) {
    //last time we checked both sensors had triggered but both
    //had yet to reset back to normal

    if (analogRead(SENSOR_1) > sensor1 && analogRead(SENSOR_2) > sensor2) {
      //both sensots are now clear so...
      
      //move back to the first state ready for next time
      state = TRACK_SECTION_CLEAR; 
      
      #if (DEBUG)
        Serial.println("Train is clear of measured distance");
      #endif
    }
  }
}

Now that might look like a lot of code but if you remove the code comments there isn't really that much going on. Everything up to line 50 just creates some constants and variables ready for the speed calculations. The setup loop in lines 52 to 86 calibrates the two LDRs: we look at the sensors for five seconds and record the lowest light level we see and set the threshold for triggering at half this value. The main loop method then simply implements the state machine we outlined above using a set of if...else statements.

So the main question is does it work? Testing by simply using my hands to cover the sensors suggested that everything worked well. The last thing to do was to actual time a locomotive. As you can see from this photo it was easy to attach to the track for testing and I can report that it works really well.

There are many ways in which the code and hardware could be improved. Configuring the distance, speed limit and scale without re-compiling would be useful, as would using a small LCD or multi-segment LED to display the speed, but for now, at least, I'll leave those as exercises for the interested reader.