mailing list archives

If I had to rate Gmail's IMAP server by the standards of "ready for prime 
time" my rating would be quite damning; it is non-compliant with the IMAP 
specification and has demonstrable bugs in operation.

Rather than do so, I prefer to consider Gmail's IMAP server to be a first 
development test, and NOT something that is presented as being "ready for 
prime time".

I hope that Google feels the same way, and that Google will soon present 
a server that is "ready for prime time".

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

On 10/25/07, Mark Crispin <MRC@cac.washington.edu> wrote:
> If I had to rate Gmail's IMAP server by the standards of "ready for prime
> time" my rating would be quite damning; it is non-compliant with the IMAP
> specification and has demonstrable bugs in operation.
>
> Rather than do so, I prefer to consider Gmail's IMAP server to be a first
> development test, and NOT something that is presented as being "ready for
> prime time".
>
> I hope that Google feels the same way, and that Google will soon present
> a server that is "ready for prime time".

an other silly bug:

http://www.colino.net/wordpress-1.5/archives/2007/10/25/dear-gmail/

-- 
DINH Vi?t Ho?

On Sat, 27 Oct 2007, DINH Vi?t Ho? wrote:
> another silly bug:
> http://www.colino.net/wordpress-1.5/archives/2007/10/25/dear-gmail/

We definitely need to encourage them to Google these bugs.  Microsoft too; 
unfortunately it seems that Exchange server has regressed with protocol 
syntax bugs (take a look at BODYSTRUCTURE, especially of nested messages).

I exchanged a handful of messages with Google about their use of mailboxes 
(instead of keywords) to represent GMail labels and the problems that 
occur as a result of this.  They have two problems; first that labels can 
be UTF-8 and second that they worry about limited client support.

IMHO, we should give a hum of concensus for a commuity statement that 
keywords can be in modified UTF-7.  I don't see any syntax reason why this 
can not be done, although doubtless we'd all have to update our clients to 
handle MUTF7 in keywords.  That should not be a big deal for most of us.

MUTF7 sucks.  We all know that.  We should move forward on IMAP UTF-8 
ASAP.  I think that we should have a sweet, short, RFC in which a server 
advertises a UTF8 capability which the client has to switch on.  When on, 
the server accepts and outputs UTF-8 instead of MUTF7.  It is up to server 
how the strings are stored internally, but if it stored in UTF-8 then it 
must convert to/from MUTF7 when not in UTF-8 mode.

We do need better client support for keywords.  IMHO, we should do that 
and drag Outlook along, screaming and kicking.  That's better than the 
current situation where Outlook is dragging us back.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

On Sat Oct 27 17:36:06 2007, Mark Crispin wrote:
> I exchanged a handful of messages with Google about their use of  
> mailboxes (instead of keywords) to represent GMail labels and the  
> problems that occur as a result of this.  They have two problems;  
> first that labels can be UTF-8 and second that they worry about  
> limited client support.
> 
> IMHO, we should give a hum of concensus for a commuity statement  
> that keywords can be in modified UTF-7.  I don't see any syntax  
> reason why this can not be done, although doubtless we'd all have  
> to update our clients to handle MUTF7 in keywords.  That should not  
> be a big deal for most of us.
> 
> MUTF7 sucks.  We all know that.  We should move forward on IMAP  
> UTF-8 ASAP.  I think that we should have a sweet, short, RFC in  
> which a server advertises a UTF8 capability which the client has to  
> switch on.  When on, the server accepts and outputs UTF-8 instead  
> of MUTF7.  It is up to server how the strings are stored  
> internally, but if it stored in UTF-8 then it must convert to/from  
> MUTF7 when not in UTF-8 mode.

+1 - I was just having this conversation with - apparently - the same  
person you were.

But I'd like to turn off all the legacy encoding at the same time.  
Mailboxes, keywords, headers, bodies - the lot.

Dave.
-- 
Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@jabber.org
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

On Sat, 2007-10-27 at 23:21 +0100, Dave Cridland wrote:
> > MUTF7 sucks.  We all know that.  We should move forward on IMAP  
> > UTF-8 ASAP.  I think that we should have a sweet, short, RFC in  
> > which a server advertises a UTF8 capability which the client has to  
> > switch on.  When on, the server accepts and outputs UTF-8 instead  
> > of MUTF7.  It is up to server how the strings are stored  
> > internally, but if it stored in UTF-8 then it must convert to/from  
> > MUTF7 when not in UTF-8 mode.
> 
> +1 - I was just having this conversation with - apparently - the same  
> person you were.
> 
> But I'd like to turn off all the legacy encoding at the same time.  
> Mailboxes, keywords, headers, bodies - the lot.

What do you mean by header/body legacy encoding?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20071028/4ad00f02/attachment.sig>

On Sat, 27 Oct 2007, Dave Cridland wrote:
> But I'd like to turn off all the legacy encoding at the same time. Mailboxes, 
> keywords, headers, bodies - the lot.

You mean that you want the server to translate charsets to UTF-8?  I would 
support doing that with an alternative form of FETCH, e.g., UTF8.FETCH. 
But then there would have to be an error on a charset conversion failure, 
and it begs the question as to why the CONVERT extension in Lemonade 
doesn't do that.

IIRC, the only IMAP specific things that are 7-bit are keywords and 
mailbox names.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Sat, 2007-10-27 at 17:10 -0700, Mark Crispin wrote:
> On Sat, 27 Oct 2007, Dave Cridland wrote:
> > But I'd like to turn off all the legacy encoding at the same time. Mailboxes, 
> > keywords, headers, bodies - the lot.
> 
> You mean that you want the server to translate charsets to UTF-8?  I would 
> support doing that with an alternative form of FETCH, e.g., UTF8.FETCH. 
> But then there would have to be an error on a charset conversion failure, 
> and it begs the question as to why the CONVERT extension in Lemonade 
> doesn't do that.

Converting message bodies would break cryptographic signatures.
Converting headers automatically might also do bad things. I guess
ENVELOPE, BODY and BODYSTRUCTURE could be returned converted, but I
don't know if that helps much.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20071028/ffd8fe73/attachment.sig>

On Sun, 28 Oct 2007, Timo Sirainen wrote:
> Converting message bodies would break cryptographic signatures.
> Converting headers automatically might also do bad things. I guess
> ENVELOPE, BODY and BODYSTRUCTURE could be returned converted, but I
> don't know if that helps much.

Right, which is why FETCH should remain unchanged.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Mark Crispin writes:
> On Sun, 28 Oct 2007, Timo Sirainen wrote:
>> Converting message bodies would break cryptographic signatures. 
>> Converting headers automatically might also do bad things. I guess 
>> ENVELOPE, BODY and BODYSTRUCTURE could be returned converted, but I 
>> don't know if that helps much.
>
> Right, which is why FETCH should remain unchanged.

I haven't looked at CONVERT recently, but from what I remember, it can 
handle charset conversion. That leaves ENVELOPE, BODY, BODYSTRUCTURE, 
mailbox names and flags. So a new extension to say "give me those in 
UTF-8" makes sense to me.

But an extension to say only "use UTF-8 for mailboxes and flags (instead 
of mUTF-7)" makes even more sense. UTF-8 flags have real value. But I 
assume most clients need 8859-1 and 2047 support for other reason, 
andso changing the way ENVELOPE, BODY and BODYSTRUCTURE works doesn't 
really make life simpler for them.

Arnt

On Sun, 28 Oct 2007, Arnt Gulbrandsen wrote:
> I haven't looked at CONVERT recently, but from what I remember, it can handle 
> charset conversion. That leaves ENVELOPE, BODY, BODYSTRUCTURE, mailbox names 
> and flags. So a new extension to say "give me those in UTF-8" makes sense to 
> me.

IMHO, since CONVERT isn't done yet, it should be extended to do ENVELOPE, 
BODY[STRUCTURE] for those people who want that functionality.  The main 
benefit that I see is that it allows clients to have less knowledge of 
random charsets.  Otherwise, clients need to know how to generate MIME 
encoded words, and probably want to generate them in the same charset in 
replies that the original message used.

> But an extension to say only "use UTF-8 for mailboxes and flags (instead of 
> mUTF-7)" makes even more sense. UTF-8 flags have real value.

Well, there's several possible ways to go about doing all this.  Consider 
all of these to be strawmen.

1) An "ENABLE UTF-8" command that throws a big switch.  Rather unIMAPish.

2) UTF8 FETCH, UTF8 LIST, UTF8 LSUB commands.  IMAPish, but somewhat 
pessimistic about the future of certain extensions.

3) Make CONVERT do ENVELOPE/BODY[STRUCTURE] for those who want it, and add 
a UTF8 option to LISTEXT.  Somewhat optimistic about these extensions.

Personally, I vote for 2, simply because this is easy to apply upon the 
base specification.  Exchange's server, which is rapidly taking over while 
we're talking about ever-more esoteric extensions, only supports IDLE, 
NAMESPACE, and LITERAL+.  I don't see much future for a framework that 
requires complex extensions such as CONVERT and LISTEXT.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Uh, we've gone from "need utf-8 flags" to "should convert message data 
to utf-8 in the server" in a very short time, and personally I see very 
little justification for the latter.

Mark Crispin writes:
> On Sun, 28 Oct 2007, Arnt Gulbrandsen wrote:
>> But an extension to say only "use UTF-8 for mailboxes and flags 
>> (instead of mUTF-7)" makes even more sense. UTF-8 flags have real 
>> value.
>
> Well, there's several possible ways to go about doing all this.  
> Consider all of these to be strawmen.

I thought about the same options when I implemented mUTF-7, except that 
your strawman for 2 is slightly better than what I had in mind.

> 1) An "ENABLE UTF-8" command that throws a big switch.  Rather unIMAPish.

IFF a big switch is the right way to solve it, I like this, but... hm... 
how shall I say this? IMO, the value of ENABLE is O(1/n), where n is 
the number of times it is used, so I'd prefer to keep n small.

> 2) UTF8 FETCH, UTF8 LIST, UTF8 LSUB commands.  IMAPish, but somewhat 
> pessimistic about the future of certain extensions.

That also needs UTF8 SELECT (for the flag-related responses), UTF8 
LISTRIGHTS, UTF8 STATUS and perhaps UTF8 GENURLAUTH and UTF8 URLFETCH.

I didn't like RLIST+RLSUB, so I find it hard to be charmed by this crowd.

> 3) Make CONVERT do ENVELOPE/BODY[STRUCTURE] for those who want it, and 
> add a UTF8 option to LISTEXT.  Somewhat optimistic about these 
> extensions.

(This doesn't handle UTF-8 flag and mailbox names.)

I speculate that the audience for doing message data conversion in the 
server is the same as the audience for CONVERT. Most clients support 
local mailboxes or POP, and so they don't get any benefit.

So my tentative conclusion is to say that 1 is a lesser evil than 2, 
make the change affect only LIST, FLAGS and PERMANENTFLAGS responses, 
and leave message data conversion to the people who want convert.

Arnt

Arnt Gulbrandsen wrote:

> Uh, we've gone from "need utf-8 flags" to "should convert message data 
> to utf-8 in the server" in a very short time, and personally I see 
> very little justification for the latter.
>
> Mark Crispin writes:
>
>> On Sun, 28 Oct 2007, Arnt Gulbrandsen wrote:
>>
>>> But an extension to say only "use UTF-8 for mailboxes and flags 
>>> (instead of mUTF-7)" makes even more sense. UTF-8 flags have real 
>>> value.
>>
>> Well, there's several possible ways to go about doing all this.  
>> Consider all of these to be strawmen.
>
> I thought about the same options when I implemented mUTF-7, except 
> that your strawman for 2 is slightly better than what I had in mind.
>
>> 1) An "ENABLE UTF-8" command that throws a big switch.  Rather 
>> unIMAPish.
>
> IFF a big switch is the right way to solve it, I like this, but... 
> hm... how shall I say this? IMO, the value of ENABLE is O(1/n), where 
> n is the number of times it is used, so I'd prefer to keep n small.

IMHO, a big switch might be easier to implement and test (both on the 
server side and on the client side).

I've discussed this with Chris Newman in context of 
draft-ietf-eai-imap-utf8 and I think Chris also had the opinion that 
this is the way to go.

>> 2) UTF8 FETCH, UTF8 LIST, UTF8 LSUB commands.  IMAPish, but somewhat 
>> pessimistic about the future of certain extensions.
>
> That also needs UTF8 SELECT (for the flag-related responses), UTF8 
> LISTRIGHTS, UTF8 STATUS and perhaps UTF8 GENURLAUTH and UTF8 URLFETCH.

+1.

> I didn't like RLIST+RLSUB, so I find it hard to be charmed by this crowd.

Indeed. That is exactly why I started co-editing of the LISTEXT document.

>> 3) Make CONVERT do ENVELOPE/BODY[STRUCTURE] for those who want it, 
>> and add a UTF8 option to LISTEXT.
>
FYI, the latest [expired] version of draft-ietf-eai-imap-utf8 defines a 
LISTEXT option.

>> Somewhat optimistic about these extensions.
>
> (This doesn't handle UTF-8 flag and mailbox names.)
>
> I speculate that the audience for doing message data conversion in the 
> server is the same as the audience for CONVERT. Most clients support 
> local mailboxes or POP, and so they don't get any benefit.
>
> So my tentative conclusion is to say that 1 is a lesser evil than 2, 
> make the change affect only LIST, FLAGS and PERMANENTFLAGS responses, 
> and leave message data conversion to the people who want convert.

I prefer the choice # 1, followed by 3 and then followed by 2.

Arnt Gulbrandsen wrote:
>> 1) An "ENABLE UTF-8" command that throws a big switch.  Rather unIMAPish.
> 
> 
> IFF a big switch is the right way to solve it, I like this, but... hm... 
> how shall I say this? IMO, the value of ENABLE is O(1/n), where n is the 
> number of times it is used, so I'd prefer to keep n small.

Agreed.

On the off chance this is actually what's approved, "ENABLE" should be a 
slider.  '0' (default) is IMAP4, '1' should be IMAP4+UTF8, '2' should be 
the next feature we can't live without.

We should allow servers that require clients to support UTF8, although I 
would expect this rollout to take quite a lot of time.

I'm not endorsing this idea, but I do think there is an obvious way to 
mitigate the exponential number of feature toggles.

Tim

On Mon, 2007-10-29 at 12:23 -0700, Tim Showalter wrote:
> Arnt Gulbrandsen wrote:
> >> 1) An "ENABLE UTF-8" command that throws a big switch.  Rather unIMAPish.
> > 
> > 
> > IFF a big switch is the right way to solve it, I like this, but... hm... 
> > how shall I say this? IMO, the value of ENABLE is O(1/n), where n is the 
> > number of times it is used, so I'd prefer to keep n small.
> 
> Agreed.
> 
> On the off chance this is actually what's approved, "ENABLE" should be a 
> slider.  '0' (default) is IMAP4, '1' should be IMAP4+UTF8, '2' should be 
> the next feature we can't live without.

Client-side capabilities come to my mind. Has something like this been
discussed already?

1 ENABLE UTF-8 UNSOLICITED-RESPONSES-WITHOUT-IDLE ETC

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20071029/353ec552/attachment.sig>

Timo Sirainen wrote:
> Client-side capabilities come to my mind. Has something like this been
> discussed already?
> 
> 1 ENABLE UTF-8 UNSOLICITED-RESPONSES-WITHOUT-IDLE ETC

IMAP lore says this is what killed IMAP 3.

I'm not opposed to client-side capabilities, as I think we need a way to 
get rid of some of the cruft of the protocol eventually.  But I do think 
we ought to be able to specify a strict, increasing order for such 
extensions.

Tim

On Mon, 29 Oct 2007, Tim Showalter wrote:
>> 1 ENABLE UTF-8 UNSOLICITED-RESPONSES-WITHOUT-IDLE ETC
> IMAP lore says this is what killed IMAP 3.

There was quite a bit that killed IMAP3; a total misunderstanding the 
unsolicited data model of IMAP, an incorrect understanding of how to 
handle MIME, and some truly silly facilities like SET.EOL.

Then there are such obtuse statements as "this allows the client not to 
have to block on some complex predicate that involves watching to see an 
update in a cache cell" which make no sense until you you realize that it 
refers to one particular client written in Common Lisp.

> I'm not opposed to client-side capabilities, as I think we need a way to get 
> rid of some of the cruft of the protocol eventually.  But I do think we ought 
> to be able to specify a strict, increasing order for such extensions.

I am of two minds here.

I very much agree with getting rid of cruft.  On the other hand, a strict, 
increasing order of extensions opens a wide avenue for mischief in which a 
desirable facility can not be added without first adding other cruft that 
was slipped in the mandatory upgrade path.

If I knew a way to steer between this Scylla and Charybdis, I would offer 
it.

Perhaps we should just punt doing the right thing to IMAP5, and add 
UTF8.FLAGS as a FETCH command and UTF8.LIST/UTF8.LSUB as basic commands 
(with perhaps something for LISTEXT if you like...but I don't want a 
dependency on LISTEXT).

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

On Oct 29, 2007, at 3:05 PM, Mark Crispin wrote:

> Perhaps we should just punt doing the right thing to IMAP5, and add  
> UTF8.FLAGS as a FETCH command and UTF8.LIST/UTF8.LSUB as basic  
> commands (with perhaps something for LISTEXT if you like...but I  
> don't want a dependency on LISTEXT).

Is there a single place that summarizes goals of future IMAP proposals  
or extensions? I've seen several ideas come up on this list; are these  
captured publicly anywhere? I don't see anything on http:// 
www.imap.org/, and the ietf-imapext list looks a bit too specific.

Thanks,
--Sarah

Mark Crispin wrote:
>> IMAP lore says this is what killed IMAP 3.
> 
> There was quite a bit that killed IMAP3; a total misunderstanding the 
> unsolicited data model of IMAP, an incorrect understanding of how to 
> handle MIME, and some truly silly facilities like SET.EOL.
> 
> Then there are such obtuse statements as "this allows the client not to 
> have to block on some complex predicate that involves watching to see an 
> update in a cache cell" which make no sense until you you realize that 
> it refers to one particular client written in Common Lisp.

I withdraw the comment.  (I made it based on something I read on this 
list some number of years ago, and I've never read the IMAP3 document. 
Good thing, too. :-) )

>> I'm not opposed to client-side capabilities, as I think we need a way 
>> to get rid of some of the cruft of the protocol eventually.  But I do 
>> think we ought to be able to specify a strict, increasing order for 
>> such extensions.

> Perhaps we should just punt doing the right thing to IMAP5, and add 
> UTF8.FLAGS as a FETCH command and UTF8.LIST/UTF8.LSUB as basic commands 
> (with perhaps something for LISTEXT if you like...but I don't want a 
> dependency on LISTEXT).

What's the real difference between having strictly-increasing client 
capabilities, or revising the base spec?  In both cases, we're bundling 
some arbitrary set of protocol changes and requiring client/server 
support for all of them.


Thinking about this a little more, I think there's an argument to be 
made for limiting capabilities and trying to decrease the number of 
different available capability sets, but we probably also need some 
amount of forking near the cutting-edge of protocol.

But if UTF8 is the first one, and it's successful and widely adopted, 
whatever the next wave is, should require UTF8.

But I'm very concerned about the complexity here.  It's apparently hard 
to implement correct IMAP clients and servers now.

I'm not at all convinced of the wisdom of any of these approaches, but I 
am enjoying the discussion.

Tim

On Mon, 29 Oct 2007, Sarah Wilkin wrote:
> Is there a single place that summarizes goals of future IMAP proposals or 
> extensions? I've seen several ideas come up on this list; are these captured 
> publicly anywhere? I don't see anything on http://www.imap.org/, and the 
> ietf-imapext list looks a bit too specific.

IMHO, most of the people who've done IMAP over the years would rather 
curtail further IMAP extensions.  IMAP has too many extensions, and 
although every one has a passionate set of defenders the fact is that most 
clients and servers implement few (if any) of these.  Then there are the 
servers which misimplement extensions (such as Courier doing so for SORT 
and THREAD).

In case someone would like to put together such a list, here is a strawman 
(to throw to the wolves!) of how I would rate the currently published set 
of extensions.  Other people undoubtably have much different opinions and 
will vehemently disagree with this list.

(0) Needed.  Now.  Should drop everything else to address:

 		IMAP internationalization
 		UTF-8 mailbox names
 		UTF-8 flag names

(1) Should go into any major revision (e.g., IMAP5) of the base 
specification, either because it is essential or is "low-hanging fruit" 
that fixes an obvious deficiency:

rfc5051.txt	i;unicode-casemap comparator
rfc2177.txt	IDLE
rfc3502.txt	MULTIAPPEND
rfc4959.txt	SASL-IR
rfc4315.txt	UIDPLUS

(2) Nice to have, not strictly necessary, but for the most part 
low-hanging fruit:

rfc3516.txt	BINARY
rfc3348.txt	CHILDREN
rfc4731.txt	ESEARCH
rfc2088.txt	LITERAL+
rfc3691.txt	UNSELECT

(3) Nice to have, but sometimes misimplemented, and so may not make the 
cut:

rfc2342.txt	NAMESPACE
 		SORT/THREAD (in RFC Editor queue)

(4) Ideas that never really got off the ground for various reasons and 
are likely to fade into obscurity (sorry...):

rfc4314.txt	ACL
rfc4551.txt	CONDSTORE
rfc2971.txt	ID
rfc2221.txt	LOGIN-REFERRALS
rfc2193.txt	MAILBOX-REFERRALS
rfc2087.txt	QUOTA

(5) May become mandatory if IMAP on mobile devices takes off (as opposed 
to being killed), otherwise doomed to category (4):

rfc4469.txt	CATENATE
rfc4978.txt	COMPRESS
rfc4467.txt	URLAUTH
 		convert

(6) Much-discussed ideas that I don't see going anywhere (sorry...) but 
category (4) if they make it to publication:

 		annotations
 		LIST extensions

(7) Everything else bandied about on IMAPEXT, a group that should have 
dissolved years ago...

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

On Mon, 29 Oct 2007, Tim Showalter wrote:
> I withdraw the comment.  (I made it based on something I read on this list 
> some number of years ago, and I've never read the IMAP3 document. Good thing, 
> too. :-) )

It's instructive to read the IMAP3 document (RFC 1203), if only to see a 
dead-end branch in IMAP's evolution.

RFC 1176 was a very minor update from RFC 1064, basically adding the FIND 
command (a nascent form of LIST).  Much of RFC 1203 is a strong reaction 
against the IMAP architecture (complete with a diatribe to the effect that 
IMAP will die unless IMAP3 is adopted), especially the unsolicited data 
model (e.g., it tags FETCH responses).

As far as I know, no IMAP3 implementations were ever written.

> What's the real difference between having strictly-increasing client 
> capabilities, or revising the base spec?  In both cases, we're bundling some 
> arbitrary set of protocol changes and requiring client/server support for all 
> of them.

I think that there is much more opposition to adding to a base 
specification than to a service level.  A new base specification purported 
kills the previous one; whereas a service level might get ignored at a 
particular point.

Sometimes how you call things does matter!

> Thinking about this a little more, I think there's an argument to be made for 
> limiting capabilities and trying to decrease the number of different 
> available capability sets, but we probably also need some amount of forking 
> near the cutting-edge of protocol.

I agree completely!

> But if UTF8 is the first one, and it's successful and widely adopted, 
> whatever the next wave is, should require UTF8.

I think that upgrading IMAP for UTF-8 is essential, and was foreseen with 
the publication of RFC 2060.

> But I'm very concerned about the complexity here.  It's apparently hard to 
> implement correct IMAP clients and servers now.

I agree completely!

> I'm not at all convinced of the wisdom of any of these approaches, but I am 
> enjoying the discussion.

I think that we are all groping towards something that will address the 
problem without causing the Law of Unintended Consequences to come down 
hard upon us... ;-)

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

> IMHO, most of the people who've done IMAP over the years would  
> rather curtail further IMAP extensions.  IMAP has too many  
> extensions, and although every one has a passionate set of defenders  
> the fact is that most clients and servers implement few (if any) of  
> these.  Then there are the servers which misimplement extensions  
> (such as Courier doing so for SORT and THREAD).
>
> In case someone would like to put together such a list, here is a  
> strawman (to throw to the wolves!) of how I would rate the currently  
> published set of extensions.

This is great, thanks. Is there any timeline proposed for IMAP5, or  
something earlier for internationalization/UTF-8?

It would be great to have a base specification that covered many of  
the more popular extensions (especially IDLE, UIDPLUS, LITERAL+,  
UNSELECT), but I'm sure every client implementor has a different  
shortlist.

--Sarah

On 30.10.2007, at 1.16, Mark Crispin wrote:

> (4) Ideas that never really got off the ground for various reasons  
> and are likely to fade into obscurity (sorry...):
>
> rfc4314.txt	ACL
> rfc2087.txt	QUOTA

My list would look pretty much the same, except for these. A lot of  
people want shared mailboxes and besides administrator configuring  
them manually ACL is the only possibility. Clients don't currently  
have much support for it, but it doesn't really need support from  
normal clients. Just having some special "ACL (web)client" is enough.  
I hoped to have implemented ACL for my server this summer, but looks  
like it didn't happen.

I think quota is already pretty widely supported by clients, although  
in very limited form. If people have quota, they really want to know  
how much they have left (when they've reached the limit once..).  
Quota root is a pretty weird concept though and I don't think clients  
handle multiple ones well. Maybe QUOTA2 extension could do a better  
job with those.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 193 bytes
Desc: This is a digitally signed message part
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20071030/da193df1/attachment.sig>

On Mon Oct 29 23:16:23 2007, Mark Crispin wrote:
> On Mon, 29 Oct 2007, Sarah Wilkin wrote:
>> Is there a single place that summarizes goals of future IMAP  
>> proposals or extensions? I've seen several ideas come up on this  
>> list; are these captured publicly anywhere? I don't see anything  
>> on http://www.imap.org/, and the ietf-imapext list looks a bit too  
>> specific.
> 
> IMHO, most of the people who've done IMAP over the years would  
> rather curtail further IMAP extensions.  IMAP has too many  
> extensions, and although every one has a passionate set of  
> defenders the fact is that most clients and servers implement few  
> (if any) of these.  Then there are the servers which misimplement  
> extensions (such as Courier doing so for SORT and THREAD).
> 
> 
I disagree - but I do think that many people working on and with IMAP  
would rather see more profiles, so that fewer code-paths were  
involved. The problem with so many extensions is not the number of  
extensions per se, but the number of combinations of extensions.

> (0) Needed.  Now.  Should drop everything else to address:
> 
> 		IMAP internationalization
> 		UTF-8 mailbox names
> 		UTF-8 flag names
> 
> 
I can agree with this list. However, this is, oddly, the list were  
we've made least progress. IMHO, two of these are almost completely  
unaddressed.


> (1) Should go into any major revision (e.g., IMAP5) of the base  
> specification, either because it is essential or is "low-hanging  
> fruit" that fixes an obvious deficiency:
> 
> rfc5051.txt	i;unicode-casemap comparator
> rfc2177.txt	IDLE
> rfc3502.txt	MULTIAPPEND
> rfc4959.txt	SASL-IR
> rfc4315.txt	UIDPLUS
> 
> 
I would add NAMESPACE here, and remove MULTIAPPEND. I'm not convinced  
of the utility of MULTIAPPEND - my client supports it, but I've found  
I never use it. It's great if you're wanting to move large numbers of  
messages between servers on very high latency links, but why would  
you want to do this?


> (2) Nice to have, not strictly necessary, but for the most part  
> low-hanging fruit:
> 
> rfc3516.txt	BINARY
> rfc3348.txt	CHILDREN
> rfc4731.txt	ESEARCH
> rfc2088.txt	LITERAL+
> rfc3691.txt	UNSELECT
> 
> 
BINARY isn't particularly low-hanging - decoding on FETCH is easy,  
but handling BINARY APPEND is not.


> (3) Nice to have, but sometimes misimplemented, and so may not make  
> the cut:
> 
> rfc2342.txt	NAMESPACE
> 		SORT/THREAD (in RFC Editor queue)
> 
> 
I've found SORT and THREAD most often implemented in web-based  
clients (in particular THREAD), rather than desktop clients. The  
problem is that either the desktop client runs with a complete cache,  
so it can do it itself (and often in rather more complete ways than  
the extension provides), or else it's running online, typically in  
low bandwidth, and neither extension scales terribly well.


> (4) Ideas that never really got off the ground for various reasons  
> and are likely to fade into obscurity (sorry...):
> 
> rfc4314.txt	ACL

We (Isode) implemented a read-only ACL (ie, we never grant the "a"  
right), in order to provide several mainstream clients, including  
Thunderbird, with the "sharing" information which the clients like to  
display to users.

FWIW, I know of only two clients which allow read-write access to  
ACLs, and only one that does it properly.


> rfc4551.txt	CONDSTORE

I don't think this one is going away. This and QRESYNC are  
surprisingly well implemented, given their complexity. I already know  
of one wholly independent client implementation of QRESYNC, and it's  
seen by the author as critical.


> rfc2971.txt	ID

One of those things that nobody seems likely to implement, on the  
face of it. I'd agree that it'd never make the grade for inclusion  
into a base specification, although Id also note that it's  
implemented quite widely.


> rfc2221.txt	LOGIN-REFERRALS
> rfc2193.txt	MAILBOX-REFERRALS

Both these are broken in design, and painful to implement. I do the  
latter, and hate it with a passion - most servers which support it  
also support na?ve clients via a proxying mechanism, which is not  
only more efficient, but also far simpler to implement, thus  
penalizing people who've taken the time to do things right, in effect.


> rfc2087.txt	QUOTA
> 
> 
Weird. Alexey and I have repeatedly tried to design something  
equivalent but better, but it is not terribly easy.

On of the hardest thigs with QUOTA is the notion of QUOTAROOT - it's  
relatively obvious that it's needed, but it's terribly difficult for  
clients to use. In particular, it's unknown to a client whether -  
when faced with a mailbox that's threatening to be over quota -  
creating a new mailbox elsewhere will still be within the same quota  
or not.

Then there's the minor point that none of the quota counters are  
standardized at all.


> (5) May become mandatory if IMAP on mobile devices takes off (as  
> opposed to being killed), otherwise doomed to category (4):
> 
> rfc4469.txt	CATENATE
> rfc4978.txt	COMPRESS

Actually, this is a short-term thing. I doubt this will take off, in  
the long term.


> rfc4467.txt	URLAUTH
> 		convert
> 
> 
FWIW, none of these have much interest from the client developers I  
talk to. They're much more interested in IDLE/NOTIFY,  
CONDSTORE/QRESYNC, and CONTEXT.


> (6) Much-discussed ideas that I don't see going anywhere (sorry...)  
> but category (4) if they make it to publication:
> 
> 		annotations
> 		LIST extensions
> 
> 
Annotations would be great, but I can't see interest from server  
developers, currently. A mechanism for providing highly stable  
addressing of messages would serve almost as well, since then  
annotations could be held within an external server, as well as being  
able to glue things like calendaring in as well.

LIST extensions are a mere cleanup for LIST/LSUB, so I'd move them up  
- in fact, I'd say we should use these as the basis for UTF-8  
conversion, and deprecate the unextended LIST/LSUB.


> (7) Everything else bandied about on IMAPEXT, a group that should  
> have dissolved years ago...
> 
> 
You're getting perilously close to the suggestion that we've now  
invented everything that can possibly be useful. I think there's lots  
more to be done, and I'm not even sure of the sphere of that work.

I think some of this work may involve message organization (which is  
a big enough sphere in and of itself), some will involve  
reintegration of the plurality of communications methods and support  
we have (BURL/URLAUTH is a step in this direction, as would the  
annotation-like replacement I mention above). There's plenty more to  
be getting on with, I'm sure.

Dave.
-- 
Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@jabber.org
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Mark Crispin wrote:
> I think that there is much more opposition to adding to a base 
> specification than to a service level.  A new base specification 
> purported kills the previous one; whereas a service level might get 
> ignored at a particular point.
> 
> Sometimes how you call things does matter!

Well, I think that's true, but it's clear that base specifications get 
ignored, too.

I don't have a feel for which is the easier path, but I am fearful of 
the work involved in IMAP5, because I think it will bring out the trolls.

If we were to add a single client-side capability, we could provide a 
general mechanism.  I think it's desirable if we encourage client 
capabilities to be bundled, but I think requiring it is a mistake.

I don't believe IMAP5 would provide this flexibility.

Tim

On Mon Oct 29 23:34:29 2007, Mark Crispin wrote:
> On Mon, 29 Oct 2007, Tim Showalter wrote:
>> I withdraw the comment.  (I made it based on something I read on  
>> this list some number of years ago, and I've never read the IMAP3  
>> document. Good thing, too. :-) )
> 
> It's instructive to read the IMAP3 document (RFC 1203), if only to  
> see a dead-end branch in IMAP's evolution.
> 
> RFC 1176 was a very minor update from RFC 1064, basically adding  
> the FIND command (a nascent form of LIST).  Much of RFC 1203 is a  
> strong reaction against the IMAP architecture (complete with a  
> diatribe to the effect that IMAP will die unless IMAP3 is adopted),  
> especially the unsolicited data model (e.g., it tags FETCH  
> responses).
> 
> 
I think that tagged intermediate responses can be useful, as it  
happens, but not for FETCH - this does, as you say, break the data  
model. They'd have been useful for SEARCH, SORT and THREAD, though.

>> What's the real difference between having strictly-increasing  
>> client capabilities, or revising the base spec?  In both cases,  
>> we're bundling some arbitrary set of protocol changes and  
>> requiring client/server support for all of them.
> 
> I think that there is much more opposition to adding to a base  
> specification than to a service level.  A new base specification  
> purported kills the previous one; whereas a service level might get  
> ignored at a particular point.
> 
> Sometimes how you call things does matter!
> 
> 
Calling things a Profile can work too.


>> Thinking about this a little more, I think there's an argument to  
>> be made for limiting capabilities and trying to decrease the  
>> number of different available capability sets, but we probably  
>> also need some amount of forking near the cutting-edge of protocol.
> 
> I agree completely!
> 
> 
As do I, but I'm wary of discarding extensions as well. Extensions  
can be developed incrementally, whereas profiles, service levels, etc  
take a lot more effort.


>> But if UTF8 is the first one, and it's successful and widely  
>> adopted, whatever the next wave is, should require UTF8.
> 
> I think that upgrading IMAP for UTF-8 is essential, and was  
> foreseen with the publication of RFC 2060.
> 
> 
Mostly foreseen - certainly the lack of any encoding for keywords has  
bitten us. Otherwise, we have a relatively clear upgrade path, and  
EAI has taken advantage of some of this.


>> But I'm very concerned about the complexity here.  It's apparently  
>> hard to implement correct IMAP clients and servers now.
> 
> I agree completely!
> 
> 
I don't think it's hard to implement a client correctly; although I  
do think it'd hard to take full advantage of the protocol. But this  
is largely a result of the lack of design documentation in the later  
IMAP specifications. Reading the earlier ones, whilst they're not as  
polished as the later specifications, was very useful to me. (In  
particular, the unsolicited data model is not adequately explained in  
RFC3501, but is better explained in RFC1176 and RFC1064 - the fourth  
paragraph of RFC1064's "The Protocol" section ought to be required  
reading.).


>> I'm not at all convinced of the wisdom of any of these approaches,  
>> but I am enjoying the discussion.
> 
> I think that we are all groping towards something that will address  
> the problem without causing the Law of Unintended Consequences to  
> come down hard upon us... ;-)

Certainly anything we do requires a significant amount of thought.

Dave.
-- 
Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@jabber.org
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

On Mon, 29 Oct 2007, Sarah Wilkin wrote:
> This is great, thanks. Is there any timeline proposed for IMAP5, or something 
> earlier for internationalization/UTF-8?

IMAP5 is "just talk".  I do not know if it will progress beyond "just 
talk", or will end up as the dumping ground for all the "we should do this 
if we ever redesign from the ground up" stuff.

I'm sure that Apple has something similar, called "the new operating 
system that will replace Mac OS X."  Similarly Microsoft has "the new 
operating system that will replace NT."  etc. etc.  :-)

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

--On 30. Oktober 2007 02:11:50 +0200 Timo Sirainen <tss@iki.fi> wrote:

> On 30.10.2007, at 1.16, Mark Crispin wrote:
>
>> (4) Ideas that never really got off the ground for various reasons
>> and are likely to fade into obscurity (sorry...):
>>
>> rfc4314.txt	ACL
>> rfc2087.txt	QUOTA
>
> My list would look pretty much the same, except for these. A lot of
> people want shared mailboxes and besides administrator configuring them
> manually ACL is the only possibility. Clients don't currently have much
> support for it, but it doesn't really need support from normal clients.
> Just having some special "ACL (web)client" is enough.

I agree. Both extensions are essential for us. I use Mulberry, which of 
course supports both extensions, but the majority of our users uses IMP as 
webmail. We have added a quota display to IMP and provide Shawn Sivy's IMAP 
ACL Manager as a web frontend for ACL management.
-- 
     .:.Sebastian Hagedorn - RZKR-R1 (Geb?ude 52), Zimmer 18.:.
Zentrum f?r angewandte Informatik - Universit?tsweiter Service RRZK
.:.Universit?t zu K?ln / Cologne University - ? +49-221-478-5587.:.
                   .:.:.:.Skype: shagedorn.:.:.:.

On Tue, 30 Oct 2007, Timo Sirainen wrote:
> A lot of people 
> want shared mailboxes and besides administrator configuring them manually ACL 
> is the only possibility. Clients don't currently have much support for it, 
> but it doesn't really need support from normal clients. Just having some 
> special "ACL (web)client" is enough. I hoped to have implemented ACL for my 
> server this summer, but looks like it didn't happen.

I'm not convinced that this needs to be part of IMAP as opposed to an 
external facility (such as you note is already done).

> I think quota is already pretty widely supported by clients, although in very 
> limited form. If people have quota, they really want to know how much they 
> have left (when they've reached the limit once..). Quota root is a pretty 
> weird concept though and I don't think clients handle multiple ones well. 
> Maybe QUOTA2 extension could do a better job with those.

I don't think that many IMAP clients (as opposed to webmails) report 
quota.  Again, this seems to be an external facility.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Timo Sirainen wrote:

> On 30.10.2007, at 1.16, Mark Crispin wrote:
>
>> (4) Ideas that never really got off the ground for various reasons  
>> and are likely to fade into obscurity (sorry...):
>>
>> rfc4314.txt    ACL
>> rfc2087.txt    QUOTA
>
> My list would look pretty much the same, except for these. A lot of  
> people want shared mailboxes and besides administrator configuring  
> them manually ACL is the only possibility. Clients don't currently  
> have much support for it, but it doesn't really need support from  
> normal clients. Just having some special "ACL (web)client" is enough.  
> I hoped to have implemented ACL for my server this summer, but looks  
> like it didn't happen.
>
> I think quota is already pretty widely supported by clients, although  
> in very limited form. If people have quota, they really want to know  
> how much they have left (when they've reached the limit once..).  
> Quota root is a pretty weird concept though and I don't think clients  
> handle multiple ones well. Maybe QUOTA2 extension could do a better  
> job with those.

Indeed. Many clients actually require both of one of these extensions. 
Thunderbird would use both even if they are not advertised (yes, 
definitely a bug).

On Tue, 30 Oct 2007, Dave Cridland wrote:
>> 		IMAP internationalization
>> 		UTF-8 mailbox names
>> 		UTF-8 flag names
> I can agree with this list. However, this is, oddly, the list were we've made 
> least progress. IMHO, two of these are almost completely unaddressed.

Yes; and part of the reason why I feel that IMAPEXT has been wasting its 
time.

> I would add NAMESPACE here,

I believe that namespaces were a mistake.  They are invariably 
misimplemented; most clients treat them as a level of filesystem 
hierarchy.

The role of identifying other-user and/or shared mailboxes should be 
performed by a different mechanism.

> and remove MULTIAPPEND. I'm not convinced of the 
> utility of MULTIAPPEND - my client supports it, but I've found I never use 
> it. It's great if you're wanting to move large numbers of messages between 
> servers on very high latency links, but why would you want to do this?

MULTIAPPEND is crucial when transferring mailboxes from one server to 
another.  Otherwise you have to use a non-IMAP mechanism to do it 
properly.

Multiple appends using LITERAL+ have the same number of RTTs, but do not 
guarantee atomicity or full-success/full-failure.  And, for certain bulk 
applications, MULTIAPPEND plus LITERAL+ work together.

MULTIAPPEND should be part of the base definition of APPEND, especially if 
CATENATE becomes part of the base specification.

> BINARY isn't particularly low-hanging - decoding on FETCH is easy, but 
> handling BINARY APPEND is not.

Agreed.  These should have been two separate mechanisms.

> I've found SORT and THREAD most often implemented in web-based clients (in 
> particular THREAD), rather than desktop clients. The problem is that either 
> the desktop client runs with a complete cache, so it can do it itself (and 
> often in rather more complete ways than the extension provides), or else it's 
> running online, typically in low bandwidth, and neither extension scales 
> terribly well.

Nonsense.  Pine/Alpine is an online client that uses SORT/THREAD with 5 
and even 6 digit mailboxes with HUGE success.  I use Alpine all the time 
over EDGE and wouldn't think of threading without it.

>> rfc4551.txt	CONDSTORE
> I don't think this one is going away. This and QRESYNC are surprisingly well 
> implemented, given their complexity. I already know of one wholly independent 
> client implementation of QRESYNC, and it's seen by the author as critical.

I know that Isode considers this to be important.  I'm less certain about 
the wider market.

> LIST extensions are a mere cleanup for LIST/LSUB, so I'd move them up - in 
> fact, I'd say we should use these as the basis for UTF-8 conversion, and 
> deprecate the unextended LIST/LSUB.

I'm afraid that I disagree here.  I would deprecate LSUB (and the entire 
subscription facility) and stay with the simpler LIST.  LIST extensions 
added too many "nice to have" but not strictly necessary facilities, and 
thus is doomed IMHO.

> You're getting perilously close to the suggestion that we've now invented 
> everything that can possibly be useful. I think there's lots more to be done, 
> and I'm not even sure of the sphere of that work.

No, I think that IMAP is "mature".  The future useful work will be in 
whatever replaces IMAP.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Mon, 29 Oct 2007, Tim Showalter wrote:
>> I think that there is much more opposition to adding to a base 
>> specification than to a service level.  A new base specification purported 
>> kills the previous one; whereas a service level might get ignored at a 
>> particular point.
>> Sometimes how you call things does matter!
> Well, I think that's true, but it's clear that base specifications get 
> ignored, too.

Sigh, only too true!

> I don't have a feel for which is the easier path, but I am fearful of the 
> work involved in IMAP5, because I think it will bring out the trolls.

That is my fear too; but trolls are possible with service levels too. 
From the very earliest days of IMAP, the "IMAP will die unless IMAP adopts 
my favorite functionality" argument has been made.

> If we were to add a single client-side capability, we could provide a general 
> mechanism.  I think it's desirable if we encourage client capabilities to be 
> bundled, but I think requiring it is a mistake.
> I don't believe IMAP5 would provide this flexibility.

The idea with IMAP5 was NOT to have any client-side capabilities, but 
allow breaking assumptions of the past.

I think that the path of client-side capabilities leads to madness.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

On Tue, 30 Oct 2007, Dave Cridland wrote:
> I think that tagged intermediate responses can be useful, as it happens, but 
> not for FETCH - this does, as you say, break the data model. They'd have been 
> useful for SEARCH, SORT and THREAD, though.

Yes; but only with a server that allows multiple of these in progress at a 
time, and not as a tagged response (ESEARCH has the right idea).

>> I think that upgrading IMAP for UTF-8 is essential, and was foreseen with 
>> the publication of RFC 2060.
> Mostly foreseen - certainly the lack of any encoding for keywords has bitten 
> us. Otherwise, we have a relatively clear upgrade path, and EAI has taken 
> advantage of some of this.

It is true that I never considered encoding for keywords; and of course 
allowing does raise all the ambiguity issues brought about by Unicode. 
However, done properly it shouldn't raise security issues.

> I don't think it's hard to implement a client correctly; although I do think 
> it'd hard to take full advantage of the protocol. But this is largely a 
> result of the lack of design documentation in the later IMAP specifications. 
> Reading the earlier ones, whilst they're not as polished as the later 
> specifications, was very useful to me. (In particular, the unsolicited data 
> model is not adequately explained in RFC3501, but is better explained in 
> RFC1176 and RFC1064 - the fourth paragraph of RFC1064's "The Protocol" 
> section ought to be required reading.).

You can thank RFC 1203 for that.  I took incredible abuse at that time 
for explaining the unsolicited data mode.  Deleting that text was the best 
way to move forward.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

--On 29. Oktober 2007 17:59:42 -0700 Mark Crispin <MRC@cac.washington.edu> 
wrote:

> On Tue, 30 Oct 2007, Timo Sirainen wrote:
>> A lot of people
>> want shared mailboxes and besides administrator configuring them
>> manually ACL  is the only possibility. Clients don't currently have much
>> support for it,  but it doesn't really need support from normal clients.
>> Just having some  special "ACL (web)client" is enough. I hoped to have
>> implemented ACL for my  server this summer, but looks like it didn't
>> happen.
>
> I'm not convinced that this needs to be part of IMAP as opposed to an
> external facility (such as you note is already done).
>
>> I think quota is already pretty widely supported by clients, although in
>> very  limited form. If people have quota, they really want to know how
>> much they  have left (when they've reached the limit once..). Quota root
>> is a pretty  weird concept though and I don't think clients handle
>> multiple ones well.  Maybe QUOTA2 extension could do a better job with
>> those.
>
> I don't think that many IMAP clients (as opposed to webmails) report
> quota.  Again, this seems to be an external facility.

If those extensions didn't already exist, you would have a point. They 
could've been created as external facilities. But I don't see what taking 
them out of IMAP now would achieve. The claimed dichotomy between IMAP 
clients and webmail doesn't really exist, because those webmailers are IMAP 
clients too. They may not be exemplary ones, but they're what many of our 
users prefer.
-- 
     .:.Sebastian Hagedorn - RZKR-R1 (Geb?ude 52), Zimmer 18.:.
Zentrum f?r angewandte Informatik - Universit?tsweiter Service RRZK
.:.Universit?t zu K?ln / Cologne University - ? +49-221-478-5587.:.
                   .:.:.:.Skype: shagedorn.:.:.:.

> > LIST extensions are a mere cleanup for LIST/LSUB, so I'd move them up - in 
> > fact, I'd say we should use these as the basis for UTF-8 conversion, and 
> > deprecate the unextended LIST/LSUB.
> 
> I'm afraid that I disagree here.  I would deprecate LSUB (and the entire 
> subscription facility) and stay with the simpler LIST.  LIST extensions 
> added too many "nice to have" but not strictly necessary facilities, and 
> thus is doomed IMHO.

How would you handle mailboxes in public "areas" (namespaces, points in
a hierarchy) without something like the subscription facility? I thought
the whole idea was that a user might not want to have listed all the
mailboxes from such an area. Is the expectation now that users setup
their desired subset on the client, potentially repeating this setup on
different clients?

Cheers,

	- Joel

Mark Crispin writes:
> MULTIAPPEND is crucial when transferring mailboxes from one server to 
> another.  Otherwise you have to use a non-IMAP mechanism to do it 
> properly.
>
> Multiple appends using LITERAL+ have the same number of RTTs, but do 
> not guarantee atomicity or full-success/full-failure.

if I understand you correctly, you're saying that one should use 
multiappend to transfer an entire mailbox with one command.

I was afraid that someone might do that, so I didn't implement 
multiappend. I know people whose average mailbox size is 200MB (no idea 
what the top decile looks like ;) and transactions that large could be 
a problem. Single IMAP commands that large could be a problem, too. My 
code would have real problems with it.

Arnt

On Wed, 31 Oct 2007, Joel Reicher wrote:
> How would you handle mailboxes in public "areas" (namespaces, points in
> a hierarchy) without something like the subscription facility? I thought
> the whole idea was that a user might not want to have listed all the
> mailboxes from such an area. Is the expectation now that users setup
> their desired subset on the client, potentially repeating this setup on
> different clients?

Unfortunately, this is not how subscriptions have been used by most 
clients.

Most clients "work" by doing a one-time LIST of *, SUBSCRIBE all the 
returned names, and from then on do LSUB instead of LIST.  Any attempt to 
use subscriptions in the intended way with one of the few correct clients 
will be broken as soon as you use one of the majority clients.

This came about because the cretins who wrote [censored] over 10 years ago 
decided that since "you're not supposed to use * in LIST, that must be a 
bug in UW imapd, so this is what we do instead."  The majority of clients 
have mindlessly copied this behavior ever since.

I won't name [censored] client, and it is thankfully long extinct.

IMHO, subscriptions are hopelessly broken and are not reliably 
interoperable.  This qualifies subscriptions as cruft that should be 
deleted from IMAP.  I don't like saying this, but at some point it is wise 
to cut losses.  The benefit of having ones .newsrc exported via IMAP pales 
compared to the costs of all these clients that do LIST "" * followed by 
thousands of SUBSCRIBE commands.

If it is desirable to have some way of recording a subset of public 
mailboxes for use by multiple clients, it needs to be done by some other 
mechanism.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Wed, 31 Oct 2007, Arnt Gulbrandsen wrote:
> if I understand you correctly, you're saying that one should use multiappend 
> to transfer an entire mailbox with one command.

Yes.  We use this functionality.  A lot.

> I was afraid that someone might do that, so I didn't implement multiappend. I 
> know people whose average mailbox size is 200MB (no idea what the top decile 
> looks like ;) and transactions that large could be a problem. Single IMAP 
> commands that large could be a problem, too. My code would have real problems 
> with it.

I can understand the problem your implementation.  In my implementation, 
lots of little transfers are far more harmful for the server than one big 
one.  The RTTs, the vastly increased number of error points, and having to 
do the overhead of APPEND per message (check out, lock, append, unlock, 
check in) is devastating.

I will insist that multiappend be in the base of any IMAP5.  It was an 
oversight that it was not there in the first place.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

> IMHO, subscriptions are hopelessly broken and are not reliably 
> interoperable.  This qualifies subscriptions as cruft that should be 
> deleted from IMAP.  I don't like saying this, but at some point it is wise 
> to cut losses.  The benefit of having ones .newsrc exported via IMAP pales 
> compared to the costs of all these clients that do LIST "" * followed by 
> thousands of SUBSCRIBE commands.
> 
> If it is desirable to have some way of recording a subset of public 
> mailboxes for use by multiple clients, it needs to be done by some other 
> mechanism.

Thinking about this more, I'm tending toward the idea that subscriptions
are a type of client configuration data. Doing them with ACAP might be
the way to go.

Cheers,

	- Joel

On Wed Oct 31 05:11:27 2007, Joel Reicher wrote:
> Thinking about this more, I'm tending toward the idea that  
> subscriptions
> are a type of client configuration data. Doing them with ACAP might  
> be
> the way to go.

Well, you can do that - I can bookmark arbitrary IMAP URLs in ACAP,  
and that includes mailbox URLs. This ties in closely with Mark's  
"suscriptions are bookmarks" concept that I recall him saying. I do  
use this - for instance, I have a bookmark pointing to the archive of  
this list on the Cyrus public server.

But I don't use this facility often - personally, I don't find I need  
subscriptions (and hence never implemented them), but that might well  
be a result of the kinds of servers I use, and how I use them. So I  
freely admit I'm not the right person to ponder whether they're  
needed or not, and how we could replace them with something else.

Dave.
-- 
Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@jabber.org
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade