Whilst there's no official Gmail API, there is an unsupported but widely used standard, the functions used by mobile phones to access Google mail. Luckily for me, there's been a lot of work already done to figure out the format and protocol, probably the best documentation is the source of the libgmailer PHP project. The downside of it being unofficial is that it keeps getting broken by Google's changes, but there's an active community using it who seem to patch it up again very quickly.
Yahoo actually has an official mail API, but it suffers from a couple of serious flaws. First there's this language at the start of the documentation: "You may not use the Yahoo! Mail Web Service API to mine or scrape user data from the user's Yahoo! account." Umm, so I can access the data but can't 'mine' or 'scrape' it, whatever that means? Does that include creating a social graph from their mailbox? It certainly sounds like it.
Secondly, some basic functions like GetMessage to grab information about an individual email are only available to premium accounts. I'd imagine that would instantly cut down the potential audience by an order of magnitude.
MSN/Hotmail used to have a nice undocumented API through WebDAV/HttpMail. Unfortunately they shut down access to non-premium customers in apparent response to spammers. There are reports (bottom of article) that it's still possible to use it to download messages, just not send them, but I haven't tested that. It looks like the only alternative is screen-scraping.
This is a great example of the 'separate data silos with unusable content' problem that Doc Searls discussed in his Defrag talk. The user could gain a lot from allowing other services access to their mail, for example decent external mail integration onto Facebook, but it's not in the interest any of the companies that physically hold their data to allow that.