Back To Front But Functional

While the circuit required for programming an ATmega328P-PU chip (i.e. a standalone Arduino) is fairly simply (regardless of whether you are running at 8MHz or 16MHz) it would still be a pain to have to set everything up on a breadboard each time I wanted to programme a chip. The solution was to build a shield for the Arduino which I could simply slot on whenever I wanted to programme a new ATmega328P-PU.

Now I could have gone all fancy and designed a custom PCB for this project, but for such a simple project (where I wouldn't ever need more than one shield) this seemed overkill. Instead I built the shield out of a budget prototyping shield from oomlout. Obviously I don't want to solder an ATmega328P-PU to the shield as that would defeat the purpose, so I've added a 28pin DIP socket in its place. It took me a while to figure out the best placement of the components and to cut appropriate sized pieces of wire (lots of the connections are actually made simply by soldering the leads together on the underside of the board), although actually soldering everything in place was fairly quick (the advantage of lots of planning time). Most of the components were held in place by bending their leads slightly. The DIP socket, however, kept falling out. Unfortunately one of those occasions was just as I was about to solder it into place, and I managed to put it in back to front :(

Fortunately, the only problem with having the DIP socket back to front is that the notched end of the chip doesn't match up with the notch on the socket, which is why I've added a white paint mark to the socket to remind me to put the chip in the wrong way around. Anyway I now have a functioning shield which I can use to programme an ATmega328P-PU to run at either 8MHz or 16MHz and to upload sketches to it. I'd call that a success.

Double The Speed For Just 35p

In my previous post I explained how I'd eventually managed to figure out the procedure for configuring and programming an ATmega329P-PU micro-controller. The configuration in that post was deliberately designed to be as minimal as possible, which was why I was limited to using the internal 8MHz RC oscillator. While 8MHz is possibly fast enough for a lot of projects, there are occasions where speed is important; a lightening trigger for a camera for example. In these cases running the chip at 16MHz would be preferable. Fortunately adding an external 16MHz crystal to double the speed is both easy and cheap!

Before we get to the hardware though, we need add a new entry to the boards.txt file defining the new setup (see the previous post for details on this and how to upload code etc.).
atmega328p16mhz.name=ATmega328P (16 MHz external crystal)

atmega328p16mhz.upload.protocol=stk500v1
atmega328p16mhz.upload.maximum_size=30720
atmega328p16mhz.upload.speed=9600

atmega328p16mhz.bootloader.low_fuses=0xD7
atmega328p16mhz.bootloader.high_fuses=0xDF
atmega328p16mhz.bootloader.extended_fuses=0x07
atmega328p16mhz.bootloader.unlock_bits=0x3F
atmega328p16mhz.bootloader.lock_bits=0x0F

atmega328p16mhz.build.mcu=atmega328p
atmega328p16mhz.build.f_cpu=16000000L
atmega328p16mhz.build.core=arduino:arduino
atmega328p16mhz.build.variant=standard
Most of this is identical to the definition for the 8MHz version. The two main changes are to the low fuse (to specify that we are using an external full-swing crystal) and to the processing speed (changed from 8000000 to 16000000).

To make this work we need to add three small components to the previous circuits: a 16MHz crystal and two 22pF capacitors. Fortunately these are very cheap components and add just 36p to the cost of the circuit (making a total cost of £5.54 or £3.87 if you ordered parts for ten circuits). You can see how these are wired up in these diagrams (for programming on the left and in use on the right).


Configuring the fuses and uploading sketches to this new circuit is exactly the same procedure as for the 8MHz version, other than ensuring you select the 16MHz version from the boards menu.

There is, however, one important thing to note. Once you have configured the chip to run at 16MHz, you can't then use it without a 16MHz crystal present. You can't even re-programme the fuses to set it back to run at 8MHz. So if you want to go back to using the internal RC oscillator re-programme the fuses with the crystal present and then simplify the circuit.

Honey, I Shrunk The Arduino

As you might be able to tell from recent posts, I've been doing quite a bit of work with an Arduino. I've now got at least one project that I'd like to make a little more permanent, rather than it just being a bunch of components on a breadboard.

The easiest option would probably be to solder the components onto a prototyping shield (for example, these budget models from oomlout). There are two problems with this approach though. Firstly not every component is ideally suited for the form factor of an Arduino shield (for example a 16x2 character LCD takes up most of the prototyping space), and secondly to be any use a shield still needs to be connected to an Arduino. While Arduino's aren't extortionately expensive at just over £20 (depending where you buy them from), that's quite a cost to add to a project.

There are cheaper versions of the Arduino available, such as the Arduino Mini, which would bring the cost down to around £10 and which might be ideal for some projects. If you want to move your project onto a custom PCB though, you probably don't want to have to solder on a second PCB. The solution is to build the Arduino components directly into your circuit rather than relying on a pre-built solution. This should be cheaper than buying an Arduino and allows you to lay out the components in the most efficient way on a custom PCB.

The problem of course is figuring out how to construct an Arduino from the component parts. I spent quite a while trawling the internet trying to find the answers, but couldn't find a single page that collected everything you needed to know in one place, so I'm sure you can guess what the rest of this post is going to cover :)

The first decision you have to make is which Arduino compatible microcontroller you are going to base your circuit around. This will depend on the number of input/output pins you need as well as the size of the compiled sketch you intend to use. Given that I'm prototyping my ideas using an Arduino UNO, I decided to stick with the ATmega328P-PU chip the UNO is based around. This is by far the most expensive component you will need to buy, but you can pick them up for around £3. You will find that a number of places sell them with the Arduino bootloader programmed in, for which they charge a premium. Unless you are simply wanting to replace the chip in an existing Arduino UNO you don't actually need to use the bootloader, so save yourself a few pounds and buy the plain un-programmed chips.

Most of the tutorials on using a ATmega328P-PU expect you to run the chip at 16MHz (the same speed as the UNO) but this requires you to provide an external clock signal using a 16MHz crystal. As components mean costs, I decided I'd be a cheapskate and run the chip just using the internal 8MHz clock. We don't need any other components in order to programme the chip, but we do need to tell the Arduino software how to work with the chip.

Extra boards/chips are made available through the Arduino software by adding them to the boards.txt file in the hardware/arduino/ folder of the Ardunio installation (for full details of the format of this file try this Arduino wiki page). After a fair amount of messing around (starting from this example) I eventually figured out the following definition:
atmega328p8mhz.name=ATmega328P (8 MHz internal clock)

atmega328p8mhz.upload.protocol=stk500v1
atmega328p8mhz.upload.maximum_size=30720
atmega328p8mhz.upload.speed=9600

atmega328p8mhz.bootloader.low_fuses=0xE2
atmega328p8mhz.bootloader.high_fuses=0xDF
atmega328p8mhz.bootloader.extended_fuses=0x07
atmega328p8mhz.bootloader.unlock_bits=0x3F
atmega328p8mhz.bootloader.lock_bits=0x0F

atmega328p8mhz.build.mcu=atmega328p
atmega328p8mhz.build.f_cpu=8000000L
atmega328p8mhz.build.core=arduino:arduino
atmega328p8mhz.build.variant=standard
I'm sure that to most people (me included) that isn't self explanatory, so let's step through it to see what it means.

Firstly, line 1 simply assigns a human readable name to the board definition (this is what will apeartr on the Boards menu). Lines 3-5 state that we will be uploading to the chip using the right protocol to use the Arduino as the ISP (In System Programmer), the maximum sketch size, and the speed at which we will upload to the chip. Unfortunately, at least in version 1.0 of the Arduino software the upload speed seems to be ignored which will come back to bite us later.

Lines 7 to 11 specify the fuse settings. These are essentially used to configure the chip. I used Engbedded's fuse calculator to figure out the correct settings. Note that I'm not using the default fuse settings that the calculator suggests as I've turned off the Divide clock by 8 internally option. I found that otherwise the chip seemed to be running at 1MHz rather than 8MHz (i.e. the blink example sketch while setup to blink at 1 second intervals appeared to be actually blinking at 8 second intervals). I also specified the smallest bootloader space option, given that we aren't going to use one. Lines 13 and 14 specify the chip type and speed and finally lines 15 and 16 specify which internal libraries to compile the sketch against (in this example the standard variant which is the UNO).

So now that the Arduino software knows how to programme the board, we need to turn our existing Arduino UNO into an ISP which we can use to programme the chip. Now there is an example sketch called ArduinoISP bundled with the Arduino software, but try as I might I couldn't get it to work. Instead I'm using a slightly improved version released by Adafruit. Simply download the new version into your sketchbook folder and then upload it to your Arduino UNO the same way you would any normal sketch. Now wire the ATmega382P-PU you want to programme to the Arduino as shown to the left (this is also documented in the comments at the top of the ArduinoISP sketch, which also talks about adding LEDs for feedback; feel free to add these if you wish). Now in the IDE, from the Tools menu set the Programmer to ArduinoISP and the board to ATmega328P (8 MHz internal clock).

Now before we can actually programme the chip we need to set the fuses to the correct values as discussed above. To do this simply choose Burn Bootloader from the Tools menu (note that this is slightly misleading in that we aren't burning a bootloader as we didn't specify one in boards.txt, but this menu item sets the fuses as well as burning the bootloader if specified). Hopefully this will "just work", but for some reason my version of the IDE uses the wrong upload speed and as such this fails. Fortunately it's fairly easy to do manually. The easiest way is to get the IDE to generate the command line for you. To do this open the Preferences dialog from the File menu, and turn on the verbose output during upload. Now try to burn the bootloader again. It will still fail, but the first message will be the command line it is trying to use. On my machine this is:

/usr/share/arduino/hardware/tools/avrdude -C/usr/share/arduino/hardware/tools/avrdude.conf -v -v -v -v -patmega328p -cstk500v1 -P/dev/ttyACM0 -b19200 -e -Ulock:w:0x3F:m -Uefuse:w:0xFF:m -Uhfuse:w:0xD9:m -Ulfuse:w:0x62:m

This command is setting the baud rate to 19200 which is an awful lot faster than both the upload speed we specified in boards.txt and the baud rate specified in the ArduinoISP sketch, both of which use 9600. So simply change the -b19200 to -b9600 and execute the command manually (from a terminal or command prompt etc.). Hopefully that should work correctly. Once you have the fuses set, assuming you don't want to change the way the chip is configured, you won't need to do this step in the future.

Now you are ready to actually upload a sketch to the new chip. I usually test that everything is working using the simple Blink example that comes with the Arduino IDE. As you have already connected the Aruino to pin 13 of the new chip (this doubles as the SCK pin) I usually change the sketch to use pin 8 which is the bottom right pin if you have yours in the same orientation as the above diagram, so that you can see the sketch working without having to change any of the wiring used for programming. Now to upload to the new chip you need to use the "Upload Using Programmer" option from the File menu (making sure that the programmer and board are still set correctly) and not the Upload button on the toolbar. If setting the fuses failed the first time and you had to manually change the baud rate then you will probably run into the same problem here as well, but the solution is the same (run the command from the IDE then copy it out, change the baud rate, and then run it manually). Once it has uploaded successfully you should find your LED is blinking on and off at one second intervals.

Now, as the main aim was to remove the need to dedicate a full Arduino UNO to each project we need to power the chip in some other way than by being connected to the UNO. Fortunately this is also fairly simple as you can see from the setup to the left. This uses just four components; a voltage regulator, two capacitors, and a 9V battery connector.

I've actually found that the supply from a fresh 9V battery is stable enough that you can do without the two capacitors, but as they are stipulated by the datasheet for the voltage regulator I've included them.

Minimal Arduino Part List
PartUnit Cost
ATMEL ATmega328P-PU£3.02
5V Voltage Regulator£0.40
100µF Capacitor£0.151
9V Battery Connector£0.60
These extra components are, as you can see from this table, much cheaper than the main ATmega328P-PU, and adding the two capacitors adds just 30p to the cost. In total the components add up to just £4.32 + VAT for a grand total of £5.19 -- or around half the cost of the Arduino Mini, or a quarter of the cost of an Arduino UNO. The circuit could be made even cheaper if you were willing to buy the components in bulk; for example if you buy 10 ATmega328P-PU then they cost just £1.63 each, so you could make 10 of these circuits for just £3.52 each.

The one thing I haven't tried yet is using any analog inputs with this setup. The datasheet for the ATmega328P-PU says that the AREF pin needs to be connected to power via a low-pass filter to remove noise if the analog to digital converter is being used. I've seen a number of websites that at least suggest this isn't necessary, especially if powering via a battery, rather than an alternating current based power source. I'll figure this out in due course but even if I do need to add extra components they will be priced in the pence so won't add much to the overall cost.

Levelling Out The Wear

I've been playing around with an Arduino since May, but up until now I haven't had any need to use the 1024 bytes of EEPROM embedded in the ATMEGA328 microcontroller at the heart of the Arduino UNO. EEPROM, or to give it its full name Electrically Erasable Programmable Read-Only Memory, is really useful for storing configuration data as any values saved to EEPROM are retained even when the power is removed.

The main problem with EEPROM is that it can quickly wear out. The datasheet for the ATMEGA328 gives an expected life of just 100,000 writes for each 32 byte page. If you are just using the EEPROM to save configuration data, which won't change very often, then this won't be a problem, but if you intend to frequently write to the EEPROM this could quickly become a real issue. To mitigate this problem what we need is a wear levelling algorithm.

In the sketch I'm working on I have just four bytes that I need to store as configuration data. The easiest option would be to simply store them at offsets 0, 1, 2 and 3, but of course this would mean that every time they were changed the same four bytes (and hence the same 32 byte page) would be re-written, while the rest of the EEPROM would remain unused. Clearly this won't result in even wear across the EEPROM.

My solution, which probably isn't optimal, is to simply shift the data along by one byte each time it needs to be written. This completely ignores the idea of 32 byte pages, but if data is being written often enough then hopefully the pages should get fairly evenly used.

Now the main problem of shifting the data along by one byte each time it changes is knowing where to read it back from when the Arduino starts up. The trick here is to employ some form of multi-byte marker that denotes the location of the data block. In this example I'm going to use the four characters "TS01" to stand for Test Sketch version 1. This allows me to increment the version number if I change the format of the stored data in the future, and tells me which sketch the data refers to. This second point is important if you are going to use the EEPROM across different sketches (less important if you dedicate an Arduino to a single project). So enough talking lets start putting this into practice.

Firstly, while the core Arduino software does include an EEPROM library it's fairly rudimentary. Fortunately another Arduino user has contributed the EEPROMex library which offers a whole bunch of useful methods (note that I did have a few problems getting this library to work; you have to make sure that the folder and filenames all share the same capitalization -- EEPROMex, and the header file doesn't need to include the original EEPROM library). This extended library is capable of writing arbitrary structures to EEPROM, so instead of having separate variables for the configuration data it's easier to store them in a custom structure giving us the following code:
#include <EEPROMex.h>

#define CONFIG_VERSION "TS01"

struct ConfigStructure {
    byte a, b, c, d;
    char version[5];
} config = {
    76, 74, true, 15,
    CONFIG_VERSION
}, stored;

int offset = 0;
As you can see this snippet imports the library, defines a constant to hold the multi-byte marker and then creates the custom structure. We also create two instances of the structure config and stored giving the first one some sensible default values for each variable, so that if we don't find any stored settings the sketch can still operate. You'll also notice that I've put the multi-byte marker at the end. This is so that it only gets written if we have successfully written the rest of the data -- I'll come back to this point later.

Now before we get into actually reading and writing the settings I'm going to introduce a utility function for comparing the current and stored settings.
boolean compareSettings() {
  if (config.a != stored.a) return false;
  if (config.b != stored.b) return false;
  if (config.c != stored.c) return false;
  if (config.d != stored.d) return false;
 
  return true; 
}
Ideally I would have liked to actually pass the two instances to the function so that it wasn't a hard coded comparison of the config and stored instances, but there is an issue with passing user defined structures to functions in the version of the Arduino IDE I'm using (it has been fixed in 1.0.2 but I'm still on 1.0.0). Anyway this function simply checks that the two instances hold identical values for each variable.

So now we can turn our attention to reading stored settings out of the EEPROM.
void setup() {
  //search through the EEPROM for a valid config structure
  for ( ; offset < EEPROMSizeUno-sizeof(stored) ; ++offset) {
      
      //read a struct sized block from the EEPROM
      EEPROM.readBlock(offset, stored);

      if (strcmp(stored.version, CONFIG_VERSION) == 0) {
        //if the block has the right version tag then we can
        //assume that the rest of the struct is valid

        //overwrite the default settings with those we have
        //just loaded from the EEPROM
        config = stored;

        //we don't need to look any further so break out of
        //the loop to speed things up a bit
        break;
     }
  }
  
  if (!compareSettings()) {
    //if they haven't changed the settings from the default then
    //there is no reason to waste an EEPROM write in storing them
    //so assume the stored version is the default
    stored = config;  

    //as there wasn't anything in the EEPROM set the offset to -1
    offset = -1;
  }
}
Hopefully the comments make this code easy to understand, but essentially all it does is read a block of data from the EEPROM, see if it is a valid settings object by checking if the version variable is the same as the CONFIG_VERSION constant. If it is then we ensure that the current config structure is the same as the stored structure, otherwise we move on 1 byte and try again (this all happens in lines 3 to 20, ). Once we have searched through the entire EEPROM, either we will have found some settings and config and stored will be equal, or we haven't found any and they will differ. Note that we control the loop using the EEPROMSizeUno constant defined by the EEPROMex library, if you have a different Arduino then you will need to change this. If, when we have searched through the EEPROM, config and stored aren't the same then lines 22 to 30 make them so (so we don't store the default values to EEPROM for no reason) and reset the offset so that when it is incremented to save the settings it moves to the beginning of the address range.

Now in the main part of your sketch when you decided the settings need to change you can simply update the config instance and call a saveSettings function to actually write them to the EEPROM.
void saveSettings() {
  if (!compareSettings()) {
    //we need to store the new settings in EEPROM

    //move on one position from the previous place we stored things
    ++offset;

    //if writing at offset would mean going outside the EEPROM limit then
    //reset the pointer to the beginning of the EEPROM
    if (offset > EEPROMSizeUno-sizeof(config)) offset = 0;

    //do the actual EEPROM writing
    EEPROM.updateBlock(offset, config);

    //stored should now be the same as config
    stored = config;
  }
}
Essentially if the config and stored instances differ then we need to write config to the EEPROM. We increment the offset (the wear levelling part), check we haven't gone outside the valid range, and then write the block (we actually use update so that we don't write bytes that won't actually change) and now we know that config and stored are the same again.

I mentioned earlier that the reason for putting the multi-byte marker at the end of the config structure was so that I knew once it had been written then the rest of the data must have been as well. The problem here is that if writing fails before the start of the marker we will have no way of knowing, other than we will see random configuration data. The easiest option would be to add a different marker at the start. If both markers are present and in the right place then the data between them must be valid. The problem here is that a four character marker takes up more space than the config data I'm saving so having two of them seems like a waste.

The approach I'm taking is to compute a hash of the four config values and store this as the first piece of data. This means that when I read the structure back I can check for the multi-byte end marker and that the data matches the hash. The problem is that the hash function is specific to the number and type of config variables you want to store and so isn't a generic solution but something you would have to adapt to your needs. As an example though here is how I'm currently hashing the four bytes of configuration data.
const int prime = 31;
int result = 1;
result = prime * result + config.a;
result = prime * result + config.b;
result = prime * result + config.c;
result = prime * result + config.d;
This code (which is based on what Eclipse generates for a four byte java structure) generates an integer (two bytes on the Arduino) based upon the four config values. Changing any of the config values changes the result of the hash function. There is also a second reason for using a hash function: data stored in EEPROM can degrade over time and the hash function allows us to check that the data hasn't changed since we stored it.

I know this hasn’t led to a single example sketch you can use, but hopefully the snippets I've presented will come in handy.