Developing new account types, Part 4: Displaying messages

This series of blog posts discusses the creation of a new account type implemented in JavaScript. Over the course of these blogs, I use the development of my Web Forums extension to explain the necessary actions in creating new account types.

In the previous blog post, I showed how to implement the folder update. Our next step is to display the messages themselves. As of this posting, I will refer less frequently to my JSExtended-based framework (Kent James's SkinkGlue is a less powerful variant; a final version will likely be a hybrid of the two technologies)—it will be slowly phased out over the rest of these series of blog posts.

URLs and pseudo-URLs

As mentioned earlier, messages have several representations. Earlier, we used the message key and the message header as our representations; now, we will be using two more forms: message URIs and necko URLs [1]. The message URI is more or less a serialization of the folder and key unique identifier. It does not have any further property of a "regular" URL (hence the title); most importantly, it is not (necessarily) something that can be run with necko. To convert them to necko URLs, you need to use the message service.

Because message URIs require an extra step to convert to necko URLs, most of the message service uses the message URI instead of the URL (anytime you see a raw string or a variable named messageURI, or (most of the time) URI, it is this pseudo-URL that is being referred to). Displaying messages involves a call to the aptly-named DisplayMessage. Unfortunately, it's also not quite so aptly-named in that it can also effectively mean "fetch the contents of this message to a stream," but I will discuss this later.

This is where the bad news starts. First off, mailnews is a bit lazy when it comes to out parameters. Technically, XPCOM requires that you pass in pointers to all outparams to receive the values; a lot of the calls to DisplayMessage don't pass this value because they ignore it anyways. Second, one of the key calls needed in DisplayMessage turns out to be a [noscript] method on an internal Gecko object. What this means is you can't actually implement the message service in JavaScript.

There is good news, however. Many of the methods in nsIMsgMessageService are actually variants of "fetch the contents of this message"; indeed, the standard implementations typically funnel the methods to a FetchMessage. My solution is to reduce all of this to a single method that you have to implement, and you get your choice of two ways to run it. Owing to implementation design artifacts, I've done it both ways and can show it to you.

Body channel

The first way is to stream the body as a channel. This is probably not the preferred method. Telling us that you did this is simple:

wfService.prototype = {
  getMessageContents: function (aMsgHdr, aMsgWindow, aCallback) {
    let task = new LoadMessageTask(aMsgHdr);
    aCallback.deliverMessageBodyAsChannel(task, "text/html");
  }
};

Seriously, that's the full code to say that you have a channel. The channel itself implements nsIChannel, but we only use very few methods: asyncOpen (we never synchronously open), isPending, cancel, suspend, and resume. The primary purpose of the channel is just to funnel the input stream of the body (not the message headers; those will be written based on the message header). The channel implementation is moderately simple:

function LoadMessageTask(hdr) {
  this._hdr = hdr;
  this._uri = hdr.folder.getUriForMsg(hdr);
  this._server = hdr.folder.server;
}
LoadMessageTask.prototype = {
  runTask: function (protocol) {
    this._listener.onStartRequest(this, this._channelCtxt);
    this._pipe = Cc["@mozilla.org/pipe;1"].createInstance(Ci.nsIPipe);
    this._pipe.init(false, false, 4096, 0, null);
    /* load url */
  },
  onUrlLoaded: function (document) {
    let body = /* body */;
    this._pipe.outputStream.write(body, body.length);
    this._listener.onDataAvailable(this, this._channelCtxt,
      this._pipe.inputStream, 0, this._pipe.inputStream.available());
  },
  onTaskCompleted: function (protocol) {
  this._listener.onStopRequest(this, this._channelCtxt, Cr.NS_OK);
  },
  QueryInterface: XPCOMUtils.generateQI([Ci.nsIChannel, Ci.nsIRequest]),
  asyncOpen: function (listener, context) {
    if (this._listener)
      throw Cr.NS_ERROR_ALREADY_OPENED;
    this._listener = listener;
    this._channelCtxt = context;

    // Fire off the task!
    this._server.wrappedJSObject.runTask(this);
  }
};

There are some things to note. First, this code can synchronously callback onStartRequest from runTask, which is a necko no-no. However, our magic glue channel gracefully handles this (by posting the call to asyncOpen in another event). Loading the input stream is done with a pipe here, and I'm doing a quick-and-easy implementation that does not take into account potential internationalization issues. I also haven't bothered to implement the other methods I should here, mostly because this code is primarily an artifact of an earlier approach, whose only purpose now is demonstrating channel-based loading.

Body input streams

The second method of implementation is just to give us the message body as an input stream:

getMessageContents: function (aMsgHdr, aMsgWindow, aCallback) {
  let pipe = Cc["@mozilla.org/pipe;1"].createInstance(Ci.nsIPipe);
  pipe.init(false, false, 4096, 0, null);
  aCallback.deliverMessageBodyAsStream(pipe.inputStream, "text/html");
  aMsgHdr.folder.server.wrappedJSObject.runTask(
    new LoadMessageTask(aMsgHdr, pipe.outputStream));
}

function LoadMessageTask(hdr, outstream) {
  this._hdr = hdr;
  this._outputStream = outstream;
}
LoadMessageTask.prototype = {
  runTask: function (protocol) {
    protocol.loadUrl(/* url */, protocol._oneShot);
  },
  onUrlLoaded: function (document) {
    let body = /* body */;
    this._outputStream.write(body, body.length);
  },
  onTaskCompleted: function (protocol) {
    this._outputStream.close();
  }
};

Here, the basic appraoch is still the same: we open up a pipe, stuff our body in one end and give the other end to the stream code. However, we don't need to do the other work that comes with loading the URI, which streamlines the code greatly. We can also pass in to the callback method an underlying request that will take care of network load stopping, etc., for us if we so choose, but the argument is optional.

More implementation

Naturally, you have to add some more contract implementations to get all of the services to work right. The following is a sample of my chrome.manifest as it stands:

component {207a7d55-ec83-4181-a8e7-c0b3128db70b} components/wfFolder.js
component {6387e3a1-72d4-464a-b6b0-8bc817d2bbbc} components/wfServer.js
component {74347a0c-6ccf-4b7a-a429-edd208288c55} components/wfService.js
contract @mozilla.org/nsMsgDatabase/msgDB-webforum {e8b6b6ca-cc12-46c7-9a2c-a0855c311e07}
contract @mozilla.org/rdf/resource-factory;1?name=webforum {207a7d55-ec83-4181-a8e7-c0b3128db70b}
contract @mozilla.org/messenger/server;1?type=webforum {6387e3a1-72d4-464a-b6b0-8bc817d2bbbc}
contract @mozilla.org/messenger/protocol/info;1?type=webforum {74347a0c-6ccf-4b7a-a429-edd208288c55}
contract @mozilla.org/messenger/backend;1?type=webforum {74347a0c-6ccf-4b7a-a429-edd208288c55}
contract @mozilla.org/messenger/messageservice;1?type=webforum-message {7e3d2918-d073-4c98-9ec7-f419a05c29de}

The first and last CIDs, as you'll notice, were not implemented by me (well, kind of). The first is the CID of nsMsgDatabase that I've exposed in one of my comm-central patches; the latter is the CID of my extension message service implementation. Also of importance is that I included a second contract-ID for my service implementation, this is for my new interface msgIAccountBackend, which is the source of the getMessageContents method I implemented earlier, and which you also need to implement to get it to work.

Finally, you need to generate the message URI properly. Fortunately, this just requires you to implement one method:

wfFolder.prototype = {
  get baseMessageURI() {
    if (!this._inner["#mBaseMessageURI"])
      this._inner["#mBaseMessageURI"] = "webforum-message" +
        this._inner["#mURI"].substring("webforum".length);
    return this._inner["#mBaseMessageURI"];
  }
};

Under the hood

For those who wish to know more about is actually going on, I am going to describe the full loading process, from the moment you click on the header to the time you see the output.

Clicking on the header (after some Gecko code that I'll elide) leads you to nsMsgDBView::SelectionChanged. This code is kicked back to the front-end via nsIMsgWindow::commandUpdater's summarizeSelection method. For Thunderbird, this is the method that handles clearing some updates and also decides whether or not to show the message summary (which is "yes if there is a collapsed thread, this pref is set, and this is not a news folder" [2]). Summarization is a topic I'll handle later.

In the case of a regular message, the result of the loading is to display the message. The message URI is constructed, and then passed to nsMessenger::OpenURL, which calls either nsIMsgMessageService::DisplayMessage or nsIWebNavigation::LoadURI, depending on whether or not it can find the message service. The message service converts its URI to the necko URL and then passes that—since it's passed in with the docshell as a consumer—to LoadURI with slightly different flags. And thus begins the real message loading.

Loading URLs by the docshell is somewhat complicated, but it boils down to creating the channel, opening it via AsyncOpen. When the channel is opened (OnStartRequest is called), it tries to find someone who can display it, based on the content type. It turns out that there is a display handler in the core mailnews code that can display message/rfc822 messages, which it does by converting the text into text/html (via libmime) and using the standard HTML display widget. I'm going to largely treat libmime as a black box; it processes text as OnDataAvailable is called and spits out HTML via a mixture of OnDataAvailable and callbacks via the channel's url's header sink, or the channel's url's message window's header sink.

The special extension message service implementation goes a few steps further. By managing the display and channel code itself, it allows new implementors to not worry so much about some of the particular requirements during the loading process. Its AsyncOpen method is guaranteed to not run OnStartRequest synchronously, and also properly manages the load groups and content type manipulation. Furthermore, the channel manually synthesizes the full RFC 822 envelope (the code inspired by some compose code), and ensures that the nsIStreamListener methods are called with the proper request parameter (the original loaded channel must be the request passed).

Alternative implementation

It is still possible to do this without using the helper implementation. In that case, there are alternatives. The first thing to do is to implement the network handler, for which you'll definitely need a protocol implementation, and probably a channel and url as well. A url that does not implement nsIMsgMailNewsUrl and nsIMsgMessageUrl is likely to run into problems with some parts of the code. You can possibly get by without a message service for now, but I suspect it is necessary for some other portions of the code. To get the message header display right, you need a message/rfc822 content-type (which gets changed to text/html, so it has to be settable!).

A possible alternate implementation would be to send a straight text/html channel for the body and then manually call the methods on the header sink, i.e., bypass libmime altogether. A word of caution about this approach is that libmime can output different things based on the query parameters in the URL, and I don't know which of those outputs are used or not.

Next steps

Now that we have message display working, we pretty much have a working implementation of the process of getting new messages and displaying them. There are several ways I can go from here, but for now, I'll make part 5 deal with the account manager. Other parts that I am planning to do soon include dealing with subscription, filters, and other such code.

Notes

1. If you are not aware, "necko" refers to the networking portion of the Mozilla codebase. The terms "URI" and "URL" also have standard meanings, but for the purposes of this guide, they mean different things. I will try to keep them distinct, but I have a tendency to naturally prefer "URI" most of the time, so I may slip up.
2. Unfortunately, a lot of the front-end code has taken it upon itself to hardcode checks for certain implementations to enable/disable features. Hopefully, as Kent James and I progress on this work, these barriers can be reduced.