Disable Dynamic Views: An Update

Just over a month ago I blogged about a small GreaseMonkey script that you could install so that you didn't have to see any of Blogger's new dynamic view templates ever again. Well it turns out that there was a small bug in that version of the script.

If you remember, the script essentially works by tricking Blogger into thinking you are viewing the blog without JavaScript by adding v=0 to the query string of each link within the blog. Well it turns out that there was one case where this wasn't working properly; the link that jumps you to the comments within the post specific page. The problem was that these links point not only to a page but a specific anchor by adding #comments to the URL. Unfortunately my script was adding the query string after the anchor reference (i.e. post-page.html#comments?v=0) when it should add it to the page location (i.e. post-page.html?v=0#comments). I've updated the script to v1.1 which contains a fix for this. In theory if you have already installed the script your browser should eventually pick up the new version. But if you haven't yet installed the script or just want to make sure you have the latest version then you can install/upgrade by simply clicking this link.

One Byte At A Time

Whilst working on Postvorta one of the things I've tried to do is to make the code as efficient as possible in order that search results are returned as quickly as possible. Mostly this has involved caching data where possible as well as using efficient data structures and algorithms. Of course with Postvorta being a web application part of the time taken to show search results is dependent on the amount of data that is actually returned to a browser including; HTML pages, JavaScript files, style sheets, and images. I am already using JAWR to minify and compress JavaScript and CSS files which makes a real difference to the amount of data that you have to download each time you search but in this post I want to talk about a small issue I uncovered when trying to trim just a few bytes from the HTML pages.

I've recently been reading a book on Java Performance by Charlie Hunt and Binu John. While it covers quite a few aspects of performance that I was already aware of there is also quite a lot of information that is new to me. One chapter is devoted to performance tuning for web applications and as well as mentioning minifying and compressing static files (JavaScript, CSS etc.) it devotes a section to considering whitespace in the dynamically generated pages.

When you save a file of text, whitespace characters (spaces, tabs, new lines) all take up the same amount of disk space as any other character, i.e. 1 byte (I know this isn't entirely accurate but I don't want to get into a long discussion of line endings and encoding formats so this assumption will suffice for what follows). This is acceptable if you want to use whitespace for formatting but HTML specifically doesn't use whitespace in this way. Any sequence of whitespace in a HTML file is converted by the browser into a single space character, so it is wasteful to transmit extra whitespace than is needed for the page to be understood and rendered. Of course most people use whitespace not just for formatting but to make the HTML code easier to understand and debug. There are filters that I could add to Postvorta that would strip out all extraneous whitespace before transmitting the results back to the browser but a) this would make debugging the page tricky and b) each filter I add has it's own performance overhead. My plan, therefore, has been to try and re-work the code where possible to eliminate some whitespace while leaving the code readable and to not add an extra filter. In most cases this is easy, but there is one area where eliminating whitespace is more difficult.

When switching between HTML and Java in a JSP page whitespace is often inserted to ensure that the resulting page can be properly interpreted. Unfortunately in almost every case this whitespace is superfluous and can be removed. Fortunately there is an easy way of removing the blank lines from the output that these whitespace characters introduce. The easiest way is to added the following page directive to a JSP page:

<%@ page trimDirectiveWhitespaces="true" %>

While Postvorta currently only contains two pages (the results page and the advanced syntax page) this is easy to do, but in a more complex application there may be tens or hundreds of pages at which point this approach becomes less appealing. You can, however, enable the same feature for every page by editing the applications web.xml to add the following:

<jsp-config>
  <jsp-property-group>
    <url-pattern>*.jsp</url-pattern>
    <trim-directive-whitespaces>true</trim-directive-whitespaces>
  </jsp-property-group>
</jsp-config>

I tried both approaches and they do indeed produce the same output, which in my test case brought the page size down to 13,238 bytes from the original 13,300, saving me a total of 62 bytes! Now 62 bytes might not be very much but this is per page view and so can quickly mount up. Looking at the differences between the old and new pages I noticed that there were still quite a large number of blank lines in the head section of the HTML file that I thought should have been removed. It turns out that the problem is related to how I style the pages but is easy to solve.

I use SiteMesh (I'm using v2.4.1) to style all the pages within Postvorta. This allows me to define the main layout of the pages once and then use this to display all pages. For those of you who use Blogger, you can consider a SiteMesh layout to be equivalent to your blog template. The layout is applied via a SiteMesh specific filter and it appears, that when using the web.xml approach to enable the trimming of whitespace, the layout is applied after the whitespace has been trimmed. This means that whitespace within the main body of the page is removed but not within the head section. The trick is to use the page directive approach within the SiteMesh layout. This has the advantage of being applied to the entire page, only needs to be specified once, and in my test case saves another 15 bytes which brings the total page size down by 77 bytes to 13,223 bytes.

The total savings are small, but if Postvorta ever becomes really popular, shaving a few bytes here and there might well make a noticeable different to performance.

Postvorta Mk II: Faster And With More Features!

The Spitfire Mk II was essentially the same as the original model, just with an upgraded Merlin engine. Today I've done something similar to Postvorta my "intelligent blog search engine".

Those of you who read the initial blog posting I did on Postvorta may remember that underneath it all Postvorta relies on GATE Mímir for indexing and search. Yesterday we (i.e. the GATE group at the University of Sheffield) released new versions of most of the software we develop, including Mímir 4. While I'm not heavily involved with the development of Mímir I do use it quite substantially at work and I've been slowly updating all the systems I'm involved with, including Postvorta, to use this new version. Not only is this new version of Mímir faster it also takes a slightly different approach in the way it handles search results which is more suited to Postvorta than the old approach. I've also added some extra code to Postvorta to cache more information locally. Together these changes have resulted in Postvorta returning results an awful lot faster than before. You will also notice that switching between pages of results is significantly faster than it was before. Of course all these changes are "under the hood" so just like with the Spitfire Mk II, Postvorta should look roughly the same but work much faster. There is one new feature though that is worth talking about: result visualization.

When you search a blog using Postvorta it returns a list of relevant documents ordered from most recent to oldest. Combined with the different ways you can search a blog this ordering is usually the most useful. In some cases, however, especially when a search returns lots of results, it can be difficult to hunt through the posts to find the one you are interested in. To help with this I've started to think about different ways of visualizing the results. Whilst I've had a number of ideas the first to make it as far as a working, stable implementation is a date distribution graph.

A date distribution graph (in this context anyway) is simply a vertical bar chart showing how the results are distributed by month. The graph, just like the results, works backwards so the most recent month is on the left. The bars of the graph can be clicked on to go to the result page containing the first result for that month. Essentially it allows you to jump to posts from a given month directly without having to page through lots of irrelevant results. Currently, depending upon the number of search results, the graph can take a moment or two to be produced, but this is done asynchronously to the normal page loading to allow you to see the actual results as soon as possible.

As always I'd be interested in knowing what you think of this new feature. You can play with it either by searching this blog (the search box is just over on the right), or your own blog.

Putting A Spanner In The Works

I'm really not a luddite, I have no problem with new technical inventions that make life easier or cheaper. What I object to is using technology simply for the sake of it, when the result is inferior to what there was before. Unfortunately, in my view, a lot of recent "innovations" on blogger seem to fall into this second category. Today my object of hate is dynamic views.

Dynamic views are a nice way of showcasing the new web standards, HTML5 and CSS3, but personally I think they are horrible. In my experience they are slow to load (and believe me my machine ranks above average on CPU and memory), and they remove any sense of individuality from the blogs on which they are used. Whilst the presentation shouldn't be as important as the content, I think we all judge books by their covers and blogs by their layout. Unfortunately once a blog owner has decided to use dynamic layouts us as readers have no option to view the original template unless we turn off JavaScript. Unfortunately, if you turn off JavaScript you will be able to see the main page of the blog using the normal template but you won't be able to look at older posts and none of the widgets will work. Fortunately the rest of this post introduces a solution that means if you hate dynamic views as much as I do you can banish them for good!

When you turn JavaScript off Blogger doesn't gracefully show you the old template for a dynamic blog. What it does is give you a link to the front page of the blog with ?v=0 appended to the end. It turns out that adding v=0 to the query string of the URL for any page within a blog causes it to be displayed using the old template. Unfortunately Blogger doesn't re-write the links in a page when you add v=0 to the URL, so they all point back to the dynamic view. And even if you have JavaScript turned off they just send you back to the homepage not the actual page you wanted to look at. This is a horrid mess, but it does give us a way of fixing things.

My solution is a GreaseMonkey script that runs as the page loads, redirects you away from the dynamic view and back to the original template pages and re-writes all the out going links, where necessary, to include the v=0 parameter so that they all just work. If you use Firefox then you will need to install the GreaseMonkey addon before you can use the script, if you use Google Chrome then support for GreaseMonkey scripts is built in. Once you are ready you can install my script by clicking this link. Currently it will only be applied to blogspot hosted blogs so unfortunately you might still see the odd dynamic view blog (Blogger Buzz for example) but hopefully there shouldn't be too many.

Let me know what you think and if you have any suggestions for improvements, and certainly links to any blogspot hosted blogs where you still see the dynamic views.

Threaded Comments

It would appear that a few days ago Blogger introduced another new feature to our blogs: threaded comments.

Now while I actually quite like the idea (much like the lightbox fiasco) it has been introduced as the default if you use an embedded comment form and publish full feeds. The other problem (and the one that affected this blog) is that the HTML elements and CSS styles used to display the comments have changed, which (depending on your template) might mean that they look different or even wrong. So while I quite like the idea here are two little tips that might make life easier.

Firstly, let's assume that while they display correctly you just don't like threaded comments and want to turn them off. This is actually easily achievable with following piece of CSS.

.comment-actions {
   display: none;
}

You need to add this to your blog template, either by editing the HTML of your template and adding it anywhere inside a style element, or by going to advanced in the Template Designer where you can add arbitrary CSS. Essentially this just removes the reply link, so to leave a comment everyone is forced to use the main comment form at the bottom of the post. This has, however, the unfortunate side effect of also removing the default delete comment link. Personally I don't think that's too much of an issue as it is easy to moderate comments through the dashboard. If I have the time I'll try and figure out how to reinstate the delete link and then I'll publish another post explaining how.

On this blog I actually don't mind the comments being threaded, but I do mind them not displaying properly, which is exactly what was happening. When I customized the template I use for this blog I added some CSS styles to control how blockquotes appear. I wasn't intending to use them for structure within the template so I styled them to look like an actual quote. You can see the styles in action in this post where I quote part of an e-mail. The problem is that blogger are now using blockquote to layout comments. This meant that each comment was styled in the same way as that e-mail quote, and because the surrounding context was different the comments were actually overlapping one another slightly. All in all it looked pretty horrible. Fortunately there is an easy fix. Because I know that I only want to use my styles within the body of a post I can limit the CSS to only applying within that section of the page by prefixing the style with .post-body as the post is always contained within a div with the class post-body. So my three blockquote styles now look like.

.post-body blockquote {
   font-style: italic;
   width: 500px;
   margin: 0 auto;
   text-align: justify;
}

.post-body blockquote:before {
   display:block;
   content: open-quote;
   font-size: 300%;
   height: 0;
   margin-left: -0.8em;
}

.post-body blockquote:after {
   display: block;
   font-size: 300%;
   content: close-quote;
   margin-top: -0.8em;
   margin-left: 500px;
}

Of course depending on the template you use you may have different problems that show up because of the new comment system so you may need to experiment a little to find exactly what CSS tricks you need to apply to put things back the way they were before Blogger introduced the new feature/bug (delete as you see fit).

I Do Exist, Honest!

After a stressful day I appear to exist once again. If you tried to visit my blog yesterday you would have seen that it had been removed. Worse than that Google had decided to suspended my entire account. That meant I lost access not only to all my blogs, but photos in picasa, my calendars, my gmail account, and my Google+ profile. It was almost as if I no longer existed (online). I filled in the contact us form to request my account back and then almost a day later I got this e-mail from Google:

We apologize for any inconvenience you may have experienced. The issue you described should now be resolved.

I don't think I've ever been quite so relieved. While the whole experience was a bit of a nightmare it did get me thinking about how I could keep backups of all my Google data in case something similar happened again in the future. After a little bit of hunting around the web (via Google of course) I came across the Data Liberation Front.

The Data Liberation Front is a Google engineering team who are trying to make it easy to liberate your data from Google products. This is useful for either moving your data to a competing service or for simply backing it up locally.

They have step by step instructions on exporting your data from most of Google's products, but they are also developing Google Takeout. Google Takeout brings together a number of the export tools into a simple interface allowing you to select which data you want to download. It then collects the data together into a single zip file that you can easily download and archive. Currently it supports: +1’d sites, Buzz, Contacts and Circles, Picasa Web Albums, Profile, and Google+ Stream. As far as I can tell the aim is to add more products to this list and I'd certainly appreciate Blogger being included.

So if you don't have a recent backup of your Google hosted data maybe now is the time to do something about it.

Striped Clouds

I've spent quite a bit of my spare time over the last week or two doing some Java GUI programming, the reasons for which will become clear in a later post. Quite a lot of the GUI is table based and so I've spent quite a bit of time playing with custom rendering code for different data types to make things easier to visualize and edit. There are plenty of tutorials on how to do this spread over the web but one thing I found quite difficult was writing a renderer that worked reliably across different Java Look and Feels (L&F). The one renderer I wrote that highlights most of the problems I had was for displaying a checkbox in a table.

By default Java will display a checkbox for boolean data types, but unfortunately it doesn't disable the checkbox when it can't be edited. This leads to a situation where you can't change the state of the checkbox but there is no visual feedback to tell you this. So I wrote a simple renderer that would disable the checkbox if it wasn't editable. The first problem I found was that under the GTK+ L&F the background of the cell didn't change when the row was selected. It did under the default Metal L&F and after a little bit of debugging I discovered the problem. Every Swing component has an opaque property which determines if it's background is drawn or not. It turns out that the default value is dependent on the L&F. So under Metal checkbox's have an opaque background while under GTK+ they don't. Fortunately this is easy to fix simply by calling setOpaque(true). A similar problem occurs with the focus rectangle around the outside of each table cell, but again it's an easily fixed by calling setBorderPainted(true). These tweaks gave me a cell renderer which seemed to work well until, that is, I tried it under the Nimbus L&F.

The Nimbus L&F was introduced in Java SE 6 Update 10, and was meant to be the new cross platform L&F that would replace the aging Metal which has been the default since Swing was first developed. One of the nice things about Nimbus is that it is resolution independent and uses vector graphics rather than bitmaps. This should, in theory, lead to a crisper interface. Personally I'm not a fan, and as yet it hasn't replaced Metal as the default L&F. It seemed, however, sensible to make sure my rendering code worked correctly under Nimbus as well as Metal, GTK+ and CDE/Motif (these are the four L&Fs available by default when running Java under Ubuntu). Unfortunately it didn't.

Nimbus, in an attempt to be different, colours the background of table rows alternating colours -- by default white and a light gray. This is instead of drawing a border around the cells. The problem is that my renderer (and almost every example I've seen) gets the background colour for the cell from the table by calling either table.getBackground() or table.getSelectionBackground(). The selected background colour works correctly but the unselected cells get drawn with a dark gray background. There are three tricks to work around this while leaving the code working under the other L&Fs. The first is to get the alternative background colour from the UIManager class. The second is to recreate the unselected background colour to make it display correctly. Finally we use the modulus operator to determine which row colour we should be using. Adding these workarounds gives me the following cell renderer which seems to work under the four L&Fs available by default under Ubuntu as well as the Windows L&F.

import java.awt.Color;
import java.awt.Component;

import javax.swing.BorderFactory;
import javax.swing.JCheckBox;
import javax.swing.JTable;
import javax.swing.UIManager;
import javax.swing.border.Border;
import javax.swing.table.TableCellRenderer;

/**
 * A TableCellRenderer for JCheckBox that disables the checkbox when the
 * cell isn't editable to make it clear that you can't click on it
 * 
 * @author Mark A. Greenwood
 */
@SuppressWarnings("serial")
public class CheckBoxTableCellRenderer extends JCheckBox implements
                                                    TableCellRenderer {

  private static final Border NO_FOCUS =
    BorderFactory.createEmptyBorder(1, 1, 1, 1);;

  public CheckBoxTableCellRenderer() {
    super();
    setHorizontalAlignment(JCheckBox.CENTER);
    setBorderPainted(true);
    setOpaque(true);
  }

  public Component getTableCellRendererComponent(JTable table,
    Object value, boolean isSelected, boolean hasFocus,
    int row, int column) {

    // this is needed for Nimbus which has alternative rows in different
    // colors hopefully other L&Fs that also do this use the same key
    Color alternate = UIManager.getColor("Table.alternateRowColor");

    // strangely the background color from nimbus doesn't render properly
    // unless we convert it in this way. I'm guessing the problem is to do
    // with the DerivedColor class that Nimbus uses
    Color normal = new Color(table.getBackground().getRGB());

    if(isSelected) {
      setForeground(table.getSelectionForeground());
      setBackground(table.getSelectionBackground());
    } else {
      setForeground(table.getForeground());
      setBackground(alternate != null && row % 2 == 0 ? alternate : normal);
    }

    setEnabled(table.isCellEditable(row, column));
    setSelected(value != null && (Boolean)value);

    if(hasFocus) {
      setBorder(UIManager.getBorder("Table.focusCellHighlightBorder"));
    } else {
      setBorder(NO_FOCUS);
    }

    return this;
  }
}

One thing to note is that I've used the alternative colour for the even numbered rows (i.e. when row % 2 == 0). I've seen some web pages suggesting that you need to use the alternative colour on the odd rows. I'm not sure how Nimbus decides which colour to use for which rows so if you see them switched around for some reason you'll need to tweak the code slightly (i.e. use row % 2 == 1).