Fritzing

Over the last few days I've been playing around with my Arduino and have finally got as far as interfacing it with other electronic components rather than just writing software to run on it. Whilst it is easy to store the software side of multiple Arduino projects safely I wasn't entirely sure of the best way of recording the hardware setup -- basically I want to be able to take a snapshot of a project so that at some later date I can recreate it, either because I messed something up or because I'm playing with multiple projects at the same time and reusing components. It turns out that the solution is obvious: Fritzing.

If you look at any of the Arduino tutorial pages (e.g. the basic blink demo) you can see that it contains both a visual representation of the project and a traditional circuit diagram. I'd always assumed that these were produced separately but I was wrong.

These diagrams are all drawn using Fritzing. Fritzing supports three views of a project: breadboard, schematic and PCB. The breadboard view shows a drawing of your project that is almost a photograph of the real thing. The schematic view gives a traditional circuit diagram, and the PCB view allows you to convert a prototype into a PCB that could be manufactured. As far as possible, changes in one view are reflected in the other views.

This means it is easy to document a project simply be recreating it in the breadboard view and allowing Fritzing to generate the circuit diagram for you.

On top of all that the people behind Fritzing have also produced Fritzing Fab: a cheap way of printing custom PCBs. So you can easily prototype an idea with an Arduino, record the prototype in Fritzing, and then generate a permanent version by printing and populating the PCB. I haven't got as far as needing to print a PCB yet, but given how integrated the steps are it could turn out to be really useful. If nothing else expect any future Arduino related posts to include Fritzing generated images.

Message Passing

As I mentioned in a previous post, I've recently bought an Arduino to play with; both as a way of expanding into possibly doing some experimenting with electronics and also as I don't want to risk damaging my Raspberry Pi by playing with it's GPIO pins.

Now while it is fun to make lights blink on and off, and with the right electronics there are lots of things you can do which don't require the Arduino to be connected to a PC (other than to upload the code to it) I'm more interested (at least at the moment) on controlling the Arduino and any connected electronic components from within software running on my computer. This allows me to move some of the logic etc. off the Arduino, and means I can interface with it using any language I like; or at least any language that can talk to the USB port. The Arduino is programmed using C or C++ both of which are languages which I have only slight experience of (I now remember just how much I hate pointers) and so I can write the computer side of things in Java where I'm a lot more comfortable.

Given how I wanted to use the Arduino, one of the first main issues I've come across is finding a sensible way of passing messages backwards and forwards. I can easily read and write text to the USB port (actually a sequence of bytes but for this description saying text will suffice), the problem I had was figuring out a sensible way of converting the text into commands that I could easily interpret on the Arduino. There are a few libraries available for doing this (CmdMessenger for example) and a number of complex state machine based approaches but they all seemed either overly complex or over engineered for what I wanted. As I'm going to be in control of the code running on both the Arduino and the PC I can cut down on some of the error handling, and I will also know the full syntax of all possible commands. This allows me to reduce the code down to just the functional aspects I'll need. While it would certainly have been quicker and easier to use an existing library I a) wouldn't have (re)learnt as much as I did and b) it would have resulted in a larger code size. This second point is important as the ATmega328 PIC at the heart of the Arduino only has 32K of RAM. In comparison to modern computers this is almost nothing; although it is twice as much as the Acorn Electron on which I learnt to program and the same as the BBC Model B. My point is, that with such limited memory available (to hold both the program and the data) using a library with lots of functions I'll never need will reduce the amount of space I have available for no good reason. So let's dive in and I'll explain the approach I'm using.

The first part of any message passing system has to involve deciding on the format of the messages. For this example I'm assuming that all messages that we want to pass to the Arduino meet the following restrictions:
  • the message type is encoded as a single character
  • the command and any parameters are separated by the : character
  • each message is terminated by the new line character
  • each message can be at most 64 characters in length
So, as an example, we could have a command that takes two numbers adds them together and returns the result. Given that the command is doing addition we could use A to represent the message type. This would then give us the simple message A:3:5 to send (with a new line at the end) to the Arduino which should respond with 8. We could do something similar for subtraction, S:8:3 giving a response of 5.

So now that we know what a message will look like we need code to read one in:
//The maximum length (in characters) of a command
const int MAX_LEN = 64;

//the buffer that holds incoming data
char buffer[MAX_LEN+1];

//the offset in the buffer where the next byte should be placed
int offset = 0;

//true if there is a new command to be processed
boolean cmdReady = false;

/**
 * A SerialEvent occurs whenever a new data comes in the
 * hardware serial RX, and this method is called between
 * successive calls of loop()
 */
void serialEvent() {
  while (Serial.available() && !cmdReady) {
    //if there are bytes available and there isn't a completed
    //command waiting to be processed...
    
    //get the new byte:    
    char inChar = (char)Serial.read();

    //if the incoming character is a newline, set a flag
    //so the main loop can do something about it:
    if (inChar == '\n') {
      //a new line char signifies the end of the command so...
      
      //null terminate the string
      buffer[offset] = NULL;

      //reset the offset ready for next time
      offset = 0;

      //flag up that there is a command ready to process
      cmdReady = true;
    } else {
      //add the character to the end of the command
      buffer[offset] = inChar;

      //move the offset on ready for the next character
      ++offset;
    }
  }
}
Lines 1 to 11 define the maximum message size, create a buffer to hold a message, and set some flags for recording how much message we have read in so far and if we have read in a complete command. The method defined in the rest of the snippet is called frequently when running on the Arduino (between successive invocations of the main loop), and it is responsible for reading in the actual message. Firstly it checks to see if there is any data to read and even if there is it only continues if there isn't a completed command waiting to be processed (the loop condition on line 19). It then reads in the next available character. If the character is the new line character then we have read in a complete message, so we NULL terminate the message (line 32; this is a C thing so that we know where in the buffer the message finishes), reset the offset counter ready for the next message (line 35), and then flag that there is a command waiting to be processed (line 38). If it wasn't a new line character then we add it to the message we are building up (line 41) and then record where we should put the next character (line 44).

Now that we can read in a message we need to actually do something with it. Chances are that you will want to do this in the main loop method, probably with code that fits the following template:
void loop() {
  
  if (cmdReady) {
    //there is a command waiting to be processed...

    //so process it in here
  }

  //do other stuff that has to be done no matter what 
}
There are probably many ways you could set about processing each message but what follows is an example showing an implementation of the add message described above.
//The set of commands we understand
const char ADD = 'A';

//the set of token separators that split the cmd into sections
const char separator_tokens[] = { ':' };

void loop() {
  
  if (cmdReady) {
    //there is a command waiting to be processed...
     
    //get the actual command (i.e. the first token)
    char* current = strtok(buffer,separator_tokens);
    
    //switch based on the command
    switch (*current) {
        
      case ADD:
      {
        //get and convert to ints the two numbers to add
        int x = atoi(strtok(NULL,separator_tokens));
        int y = atoi(strtok(NULL,separator_tokens));
        
        //do the addition
        int z = x+y;
        
        //return the result
        Serial.println(z);
        
        break;
      }
        
      default:
        //this only happens if the command is unknown
        //which should never happen!
        Serial.print("Unknown Command!");
    }
           
    //we have processed the command
    cmdReady = false;
  }
  
  //do other stuff that has to be done no matter what 
}
So let's work through this to make it clear what's happening. Firstly, while the message types are encoded as single characters it's nicer to be able to refer to them by name, and so line 2 maps message type A to a constant called ADD. We also know that the sections of a command are separated by a colon, so we record this in line 5; note that we could have multiple separators if we wanted to, but the most important thing to remember is not to use a character you ever want to use in a message. Once we know we have a command to process the first thing we need to know is the type of the command. To do this, on line 16, we get the first section of the message (everything before the first colon) using the strtok method. Essentially this splits the message into sections, given the separator character, and returns the first section. Now as we are using single characters for the message type we can simply switch based upon the character (line 19). If we were using multiple characters for the message type then we would need to use a set of if-then-else statements instead to check the type. Once we know what the message type is we can then process the arguments (if it has any). In this example you can see that lines 21-34 deal with the ADD message. You don't really need to understand the example, other than to see that we can retrieve the arguments by calling strtok again (note that we pass in NULL instead of buffer to essentially tell the method to carry on from where it was last time). Once we have dealt with the current message we can get ready for the next one by resetting the cmdReady flag (line 43).

I've put together a slightly longer example which also includes messages for subtraction and echoing arguments back to the PC and which shows how the snippets I've presented go together to produce the full Arduino sketch. Hopefully someone else will find this useful, but please do let me know if I've missed something obvious or if there is a simpler way of achieving a similar result.

Show Comments

A while ago, when Blogger made one of their unannounced set of changes, the link at the bottom of each post that takes you to the comments started to jump to the comment form rather than the first comment. This meant that usually you ended up looking at the form without being able to see any of the comments. To see the comments you would then have to scroll back up the page to the find the end of the post and hence the first comment. I know that this really annoys me when I'm reading other peoples blogs, and so I thought I should at least attempt to fix my own blogs.

Unfortunately the layout tags don't include a link to the beginning of the comments, only date:posts.addCommentUrl which generates the link to the comment form. That link works by going to the #comment-form anchor. As the comments always appear before the form, we can insert another anchor with the same name, before the comments and then hey presto, the link will take us to the first comment instead of the form.

To make this change you need to edit the HTML version of your template (with the widgets expanded) and find the following HTML snippet
<a id='comments'/>
and replace it with
<a id='comments'/>
<a id='comment-form'/>
You should find that it appears twice in your template; once for threaded comments and once for unthreaded comments. Change it in both places to make sure you get the right one.

This works because the IDs of HTML elements are meant to be unique, and when they aren't the first one takes precedence. We could have made more changes to the template (change the comments to comment-form rather than adding the extra anchor, and remove the original comment-form anchor) but I decided it was safer to leave as much of the template alone as possible. Anyway you should now find that clicking the comment link takes you to the comments rather than the form which, as far as I'm concerned, makes more sense.

Hardware As Well As Software

One of the things that attracted me to the Raspberry Pi (apart from the price and philosophy behind the development of the device) was the fact that it can be used for hardware projects, not just software development. It has a set of GPIO pins along one edge which, in theory, can be connected to external sensors that can influence code running on the Raspberry Pi as well as motors, lights etc. which can in turn be controlled by the code. There are, however, a couple of things stopping me from experimenting with this though. Firstly I'm sure it would be easy to brick the Raspberry Pi if I got something wrong in the hardware and I'm not willing to risk that. Secondly because the Raspberry Pi is so new there aren't, as yet, lots of examples I could learn from. The second point is important. Last time I tried controlling hardware (an undergraduate course) I managed to accidentally switch around two pins which caused the model lift I was controlling to try and drive itself through the floor! I'm sure that in time plenty of examples (both hardware and software) will appear and I'll make some use of the GPIO pins on the Raspberry Pi, but until that time....

Probably the most common platform (at the moment at least) for playing around with hardware/software ideas is the Ardunio. What you can see to the left is an Arduino Uno board, which is the current reference board. Essentially an Arduino is a small board containing an 8 bit microcontroller (in the case of the Arduino Uno it's an ATmega328) which can be connected to a computer via a USB cable and which has the GPIO pins broken out to easily accessible headers. You program the microcontroller using C/C++, but once programmed an Arduino doesn't have to be connected to a computer to be used, which makes them ideal for embedding in all sorts of hardware projects. There is also a large community surrounding Arduino with plenty of example programs (they are called sketches for some reason) as well as numerous hardware extensions, known as shields, which can easily add new functionality.

So I've bought an Arduino Uno (for slightly less than the price of a Raspberry Pi) partly to play with the hardware side of things, but also as a good excuse for improving my C/C++ skills (I've never really used either language in anger for anything larger than a single function DLL or command line application). I don't have any firm plans yet, but I'm sure that when I do there will be a blog post or two.

Raspberry Flavoured Information Extraction

So in my post yesterday about the Raspberry Pi I mentioned that it could run GATE. Given that before the Raspberry Pi was released a lot of people were of the opinion that they wouldn't be powerful enough to run interpreted languages, such as Java, I decided that one of the first things I'd do with mine was show how wrong that is by running GATE. So having booted up the machine and logged in I installed Java with the single command:
sudo apt-get install openjdk-6-jdk
This took a while as it downloaded and installed all the required packages. I then grabbed the binary only copy of GATE's nightly build (while I wanted to run GATE I didn't want to try compiling it). Once downloaded I started up X to give me a graphical interface (the default distribution uses LXDE). From a terminal I then started GATE. The splash screen came up almost immediately, but then there was a bit of a wait while it built the developer interface. Once GATE had started I then loaded up a copy of ANNIE (the default information extraction pipeline). This was the point at which I then took the photo on the left. Now because Amazon seem to have mislaid a parcel I ordered, I'm stuck connecting the Raspberry Pi to my TV via a composite cable (I wanted to use a HDMI to DVI-D cable to attach to my nice monitor) and I can't get the screen to scale correctly. This means that the top and bottom are cut off slightly and it isn't filling the width. Anyway at least you can see GATE loaded and the Raspberry Pi resting on the box of chocolates on the floor.

I was intending to take some proper screenshots to show GATE working but unfortunately I couldn't figure out how (turns out that taking screenshots under LXDE is a slightly convoluted process), but here is a close up photo of the TV having done a small amount of information extraction.


I'm not sure what I'm going to try doing with the Raspberry Pi next so if anyone has anything they'd like to test let me know and I'll see what I can do.

Raspberry Pi: Small Computer, Big Ideas

When I was growing up, computers were only just starting to make inroads into schools and homes, mostly via affordable (not to everyone true, but to a large slice of the population than previously) machines such as the BBC Micro, and the ZX Spectrum. As I've mentioned before my first serious forays into computer programming where actually with an Acorn Electron (a cut down, and hence cheaper, version of the BBC Micro). While these machines were often used as glorified typewriters or for playing games they were also easy to programme. When you turned them on they didn't boot straight into some complex graphical operating system, but to a simple prompt. You could enter commands that would load programmes, or you could enter simple commands that would make the computer do something. If you messed up you just turned the machine off and then on again. There was no danger of you messing the machine up in any way that would stop it working perfectly next time you turned it on. Combined these two things led to a generation of kids who messed around programming computers and who have grown up and become software engineers.

Unfortunately as computers have become more complex, the ability to experiment with programming has reduced. Also it is much easier to mess up a modern computer so that it requires a complete re-install rather than just a quick on-off of the power switch. Together this things seem to have led to a severe drop in the number of kids who are learning to programme. Of course they aren't helped by the focus of IT lessons in schools towards knowing how to use office software rather than teaching computing skills.

Having been involved in university lab classes and marking assignments over the last decade, it is clear even to me that most students arriving for an undergraduate course in computing have very few existing skills. Unfortunately they tend to graduate without learning too many more. Yes they can knock up a programme to solve a particular assignment, but often they pay no attention to details such as maintainability or efficiency (time or space). Let's just say that if I had to build up a team of programmers I doubt I'd be willing to hire most current computer science graduates, given the level of programming ability I've seen. Now I'm not really in a position to affect more than a few students by giving constructive feedback on assignments etc. Fortunately there are plenty of other people in the UK who agree that the level of computing knowledge among today's kids has fallen so far that we are in danger of not producing enough qualified graduates. Their solution to the problem is the Raspberry Pi.

I've been following the Raspberry Pi project for a while, and when they went on sale at the end of February I got my order in at the first possible moment. Due to the overwhelming demand in the device the websites of the two distributors were almost brought down. Anyway I must have been one of the earliest through the ordering process because my Raspberry Pi was one of just 2,000 that have currently been sent out to customers. So what exactly is a Raspberry Pi?

Put simply a Raspberry Pi is a fully fledged computer. It has all the same fundamental components as any regular desktop computer but costs, wait for it, just $35 and is the size of a credit card! The idea being that it is as cheap to kit out an entire class with a Raspberry Pi each as it would be with a textbook each. The computer has 256MB of RAM (not all of it is available as some is used by the GPU), uses a 700MHz ARM CPU (similar to that which you'd find in a smartphone), and an SD card for storage. All you need to do is add a keyboard (and mouse if you are using a GUI), a display, and then power it using a mobile phone charger. For a display you can use either HDMI or composite which allows you to plug it into old and modern TV's as well as modern computer monitors (via a HDMI to DVI-D cable). The SD card contains the whole operating system and is easy to re-image (i.e. recreate) if it gets messed up (it took about 30 seconds for me to set up my card using the default Debian distribution).

The idea then is to get these into schools were kids can learn to programme and about how computers work without worrying about messing up a family PC, or a PC required for standard ICT lessons etc. Currently though most of the Raspberry Pi's will have been bought by people like me who are interested in technology and grew up programming simpler machines than we commonly use today. We will iron out any bugs in the hardware, and write software for the device, so that when, later in the year, a cased version is launched for schools there will be teaching material and applications available. I'll certainly try and help the community in anyway I can and for the first time in over a decade I have hope that maybe the level of computing skills I see in undergraduates might actually go up rather than down!

Oh and from a quick play this afternoon, it can run GATE! (For those who don't know GATE is how I earn my living).

A Non-Conformant Head

UPDATE 21st April: It would appear that the snippet of misformed HTML is no longer being included in our blogs! The information doesn't seem to be included in any other fashion so I'm assuming it will be back once Blogger have decided how they are going to do it properly.

I've been messing about with Blogger templates over the last few days and I've spotted that Blogger seem to have broken the HTML of every blog they host! They have added an invalid element within the head section of each page which causes the premature closing of the head section. Depending on what scripts etc. you use in your template this may cause a problem.

The new code that Blogger have added to our templates is aimed at adding extra metadata to each page, which in turn will enable Google to have more information about each page within a blog when they are included in a search result. They are using the schema.org metadata format to achieve this. Specifically each page of this blog now contains the following in the head section:
<itemscopetag itemscope='itemscope' itemtype='http://schema.org/Blog'>
   <meta content='Code from an English Coffee Drinker' itemprop='name'/>
</itemscopetag>
As you can probably gather, this snippet essentially tags the page as being from a blog whose title is "Code from an English Coffee Drinker". From the full Blog schema you can see that there is actually a whole set of properties that Google could set for each blog, and I'm guessing that at some point in the future they will add more information, which in turn will enrich their search result pages. Now I'm all for adding extra metadata (I've even written a GATE application that runs ANNIE over webpages and then embeds appropriate schema.org metadata), but unfortunately Blogger have messed up their implementation.

The problem is that they have used an itemscopetag tag, which isn't valid in any version of the HTML specification. Also the specification tells us that if, when parsing the head section of a page we encounter an unknown tag "act as if an end tag token with the tag name "head" had been seen, and reprocess the current token". This essentially causes the premature closing of the head section, with anything else now part of the body instead. Depending on what has been forced out of head and into body and which browser you are using you may see different results. For example, it looks as if links to the Chrome Web Store are broken by this.

What Blogger should have done was added the information to the body tag or one of the main content div tags instead. For example, they could have started the body as follows:
<body itemscope='itemscope' itemtype='http://schema.org/Blog'>
   <meta content='Code from an English Coffee Drinker' itemprop='name'/>
This would have embedded exactly the same metadata but in a format conformant with the HTML specification, and which follows the instructions given on the schema.org site.

Unfortunately there doesn't appear to be anyway to remove this code from our blogs. The best we can do is to move the piece of template code that generates the invalid tag (as well as lots of other code) as late in the head section as possible, so that it pushes the least possible code into the body. To do this you need to edit the HTML version of your template and move the line:
<b:include data='blog' name='all-head-content'/>
To just before the closing head tag so it looks like:
   <b:include data='blog' name='all-head-content'/>
</head>
Hopefully Blogger will fix the code the generate soon but until then we just have to minimize the damage they inflict on our blogs any way we can.