mailing list archives

Hi,

I'm implementing software which picks up and processes new email from
various IMAP servers. It logs in, does its thing, then logs out. The
accounts which are being checked are not being exclusively checked by my
software, so there are no guarantees about any flags being set on
messages, such as \Recent or \Seen. I need to avoid processing any
messages twice, and I'm not allowed to miss any new messages. (Nor am I
allowed to delete any messages.)

With POP3, determining whether new messages have arrived is quite
straightforward: Store a list of UIDs from past connections, then do a
UIDL on the POP3 server and compare the results.

With IMAP4, it seems to be more complicated, but I hope I'm missing
something here. I can't use message sequence numbers in any way to do
this, because they're unstable (since deleting a message decrements all
sequence numbers above that of the deleted message). And I can't use
UIDs if, like I'm experiencing when dealing with the Courier IMAP
server, I get a new UIDVALIDITY number every time I connect to the
server and open a mailbox.

So what's left? SEARCH SINCE? Is that going to find something new if the
date information in the message's header is wrong? And since I can only
narrow down to a day, I'm still going to have to look at headers to
determine whether I've seen the message before, right?

Any help would be appreciated.

Thanks!

Steve

On Sun, 6 Jan 2008, Steve Freitas wrote:
> With IMAP4, it seems to be more complicated, but I hope I'm missing
> something here.

IMAP should be easier, MUCH easier, than POP3; UIDs are numeric and 
strictly ascending.  That means that you never need to fetch the map to 
learn about new messages; you can simply fetch <lastuid+1:*>.

> I can't use message sequence numbers in any way to do
> this, because they're unstable (since deleting a message decrements all
> sequence numbers above that of the deleted message). And I can't use
> UIDs if, like I'm experiencing when dealing with the Courier IMAP
> server, I get a new UIDVALIDITY number every time I connect to the
> server and open a mailbox.

I suggest that you should use UIDs anyway and ignore the brokenness of 
that server.  Courier is a horrible server, but I suspect that this is a 
problem in that particular installation of Courier.

A server changing UIDVALIDITY has a specific meaning in IMAP: toss out 
everything you remember from before and start anew.  You should NOT 
attempt to overengineer and outsmart the protocol specification, even in 
the face of a broken server.  Otherwise, someone is going to attempt to 
overengineer and outsmart the protocol specification in order to get the 
desired results from your software.

Remember: for every action there is an equal and opposition retaliation!

If you absolutely insist upon not using UIDs,
 	tag FETCH 1:* BODY[HEADER.FIELDS (Message-ID)]
is a semi-reasonable approximation of POP3 UIDs.

Good luck!

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Sun, 2008-01-06 at 17:50 -1000, Steve Freitas wrote:
> With IMAP4, it seems to be more complicated, but I hope I'm missing
> something here. I can't use message sequence numbers in any way to do
> this, because they're unstable (since deleting a message decrements all
> sequence numbers above that of the deleted message). And I can't use
> UIDs if, like I'm experiencing when dealing with the Courier IMAP
> server, I get a new UIDVALIDITY number every time I connect to the
> server and open a mailbox.

Your Courier setup is then somehow broken. UIDVALIDITY doesn't normally
change, even with Courier. I think it's just as likely for POP3 UIDLs to
change as IMAP UIDVALIDITY.

Another possibility anyway could be using keywords. Mark them with some
"mysoftware-processed" keyword when you're finished with them. Although
not all servers support keywords.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20080107/7595c471/attachment.sig>