an All Creative World site
Ivan's picture

Threading-id: faster and leaner emails

Gmail uses a smart threading technique for viewing email conversations. If there was a reply to a message you sent or received, Gmail will automatically display it in chronological order with the original. This conversation view continues to grow as you respond and new replies arrive, making it possible for you to follow the whole back and forth discussion in one place. Also, Gmail features a very important improvement compared to other email software. Quoted emails in replies are hidden, so they don't show up repeatedly with each reply. This makes following a conversation much easier. But what about the mail.app?

In Mac OS X mail.app you can have a similar threaded discussion view by selecting the View/ Organize by threads option, but unfortunately it doesn't hide the quoted emails and there is no way to show consequent emails in a one expanded thread. You can only look at them one by one, but at least they are all collected into one window to find them easier.

It's clear that you can perfectly well follow an email conversation without the need for quoted emails. Many times I receive emails that are a mile long because they include several nested quoted emails and the only new information is just 2 lines of text. That's a big waste.

Both mail.app and Gmail use the subject field to bind email conversations, so the quoted emails are not necessary for a perfectly functional threaded email discussion.

While the subject field works well to identify related emails, I think there may be a more precise and reliable way to bind messages together into one conversation. If you look at the expanded headers in an email message, you can see that each email has a unique ID that can be used to identify emails that belong to one conversation. (Highlighted on the screenshot.) I am sure Google engineers must have thought of this, so there is probably something wrong with this theory, but I don't have enough understanding of email protocols to verify the feasibility. If we were to use this ID to identify messages, we could even change the subject line when the discussion gets sidetracked, still the conversation stays bound together.

Regardless of what field is used to bind the emails together I propose to follow a practice of switching off the quotes. You can do this in mail.app if you go to Preferences /Composing and switching off Quote the text of the original message. I couldn't find an apparent way to do this in Gmail. Hopefully this option will be included soon.

This practice not only saves hard disk space, but also saves bandwidth. It may not seem much on a personal level, especially if you have broadband and a huge hard disk, but for a dial-up user or a user downloading messages through one of the mobile services it makes a big difference. Most importantly on a global level it means a lot less wasted resources.

But what if your email partner does not use an email application such as mail.app or Gmail that can bind messages together. He would rely on the good old quoted emails to look up the conversation. This improved new system of quoteless replies should downgrade well to old email applications. I suggest that we start including a new switch in email headers. This new switch would be a simple statement that the sender supports quoteless threading, such as Threading: on. The receiver application could check for this switch and if it exists it can safely return emails without quotes knowing that the original sender will be able to bind the sent and received emails together.

Since we are adding a new field in the email header we may as well add a unique ID that is solely used for email thread identification, such as Threading-id: xxx-xxx-xxx.... This way the quoteless email practice would work safely and independently from subject names or any other existing email fields. It would also downgrade well without any danger of annoyance of missing quoted history for users who are using email applications that does not yet support threading-id. Slowly, as more and more email applications adapt the threading-id technology the email traffic over the internet will get leaner and faster. Also, our email databases will get smaller and email applications work quicker.

If for instance if only Gmail would adopt the threading-id technique on it's own servers they would be able to save tens of thousands of megabytes of storage space instantly because all inbound traffic (Gmail account to another Gmail account) would benefit from the system. Also, from a user side, emails would download faster, which would result in a better user experience and more happy Gmail users.

Commenting on this Blog entry is closed.

synnestro's picture
29 pencils

the threading info you refer to is already included in the header info..
..in the example above, D72B64D8 would be part of the message id for the original email and is why it's part of the reference/in reply to fields.

Ok, it's not nice and clear cut, but i'm sure someone with a bit of applescript knowledge could read the header, pull the relevant original email message id and then group emails together that contain that message id in the relevant field.

hello, btw - this is my 1st post as i've just registered after reading this site for a few months !!

Peter

Ivan's picture

Hello Peter! Welcome to cb! I noticed that field, but it's there by default on every email message. Still, something has to identify email applications that support quoteless messages. Therefore maybe just a code such as Threading: on would be sufficient.

Damien's picture
14 pencils

I believe you're misunderstanding the reasons behind quoting and the message headers.

Quotes became the norm so that discussions could be continued within the email on specific points. The current "standard" of quoting the entire previous email and adding your piece on the top is a recent Microsoft thing, most likely because IMHO they couldn't work out how to do it any better. If you look at the shareware/freeware email client market, and indeed Fidonet readers before that, they all did quoting correctly, so are you saying you want to change the Internet standards & practices because of Outlook? Personally nothing annoys me more than to write a length email with many questions only to have the recipient reply back with an equally long reply at the top, completely breaking the flow of the email - again this is a recent habit brought on by Outlook's truly terrible quoting support.

As for email headers, I suggest taking at look at e.g. http://www.faqs.org/rfcs/rfc2822.html which describe the standard used today for formatting emails. You might like this part:

The "In-Reply-To:" and "References:" fields are used when creating a reply to a message. They hold the message identifier of the original message and the message identifiers of other messages (for example, in the case of a reply to a message which was itself a reply). The "In-Reply-To:" field may be used to identify the message (or messages) to which the new message is a reply, while the "References:" field may be used to identify a "thread" of conversation.

So we already have all we need, we just need some email clients to either fix or improve their support for these headers.

Damien

Ivan's picture

Thank you for the clarification Damien. I understand that in some cases you may want to write your answers in between lines of the original email and that works great for me too. But this way you're still storing the same information in your email database twice. Once the original and once the quoted version with replies. I think this is something we can certainly live on unless when you reply to an email in general and you don't insert your answer in between lines. In this case you may still want to drop quoting the original email.

Can you expand on how Outlook broke the standard? I'm not quite sure I understood. I thought that you can insert your reply in between the original message even with Outlook.

Damien's picture
14 pencils

To be able to properly handle the in-message-quoting as you describe you'd need some standard for referencing specific content from the previous email - simple enough in plaintext emails but gets much more complex with HTML emails.

As for Outlook breaking the standard, it has always by default put your signature and cursor at the top with the entire previous email quoted at the bottom, so when people are presented with that they just start trying away, breaking the conversation flow. And yes, it does allow you to throw comments into the middle of the quoted text but it does it *really* badly - you can't break out of the quote like you can in Thunderbird et al so when you hit (enter) to break into a paragraph to reply you're just adding a newline to the comment block; additionally if this mixed up email gets converted to plaintext to send out good luck trying to read it afterwards, its ends up a mess! In comparison Thunderbird handles quoting pretty darn near perfectly.

One other note to throw in, I really hate it when people reply to an old email to start a new discussion rather than creating a new message to that address - this really messes up viewing mailboxes by threads.

Threading is pretty cool when it works, but its nothing new - Fidonet readers have been doing it for nearly ten years ;-)

Damien

Benji's picture
3 pencils

Mail actually uses the Reference field, not Subject to put together emails in a thread. Easy way to check this is have someone reply to a string of emails you've been sending back and forward, but change the subject: you will find the new subject'ed email will be contained within the thread. Well, that's atleast how my Mail.app works! :)

Ivan's picture

Strange. It groups completetely unrelated messages for me if the subject is the same.

Creativebits is a blog about Creativity, Graphic Design, Adobe, Apple and other related subjects.

Featured Images

Do you need a great new logo?

If you need a logo for your company or product you can get it done with us.
In our logo store you can pick from over 28,000 pre-made logos that will be customized to your name for free or you can post a contest for us for just $250 and our designers from all over the world will submit dozens of logo design suggestions to your specific needs.

Marketplace