mailing list archives

Section 6.4.8 the UID command says:

      The number after the "*" in an untagged FETCH response is always a
      message sequence number, not a unique identifier, even for a UID
      command response.  However, server implementations MUST implicitly
      include the UID message data item as part of any FETCH response
      caused by a UID command, regardless of whether a UID was specified
      as a message data item to the FETCH.


My question is simply, why?  Why is the sequence number needed when the 
command
specified a UID, and the UID must be included in the response.  Seems to 
me the UID
command should have a UID response. 

snow.






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20110526/632fcbd5/attachment.html>

On Thu, 26 May 2011, John Snow wrote:
> My question is simply, why?  Why is the sequence number needed when the 
> command specified a UID, and the UID must be included in the response.  
> Seems to me the UID command should have a UID response. snow.

Right now, a client can parse a FETCH response at any time, inserting it 
into its cache without having to consider any context of pending commands.  
Indeed, the server can send FETCH responses at any time (modulus TCP 
flow-control/deadlock concerns), and it's *required* to do so for flag 
changes.  Sometimes using the UID and sometimes using msgno would make 
such unsolicited FETCH responses ambiguous.


Philip Guenther

On Thu, 26 May 2011, John Snow wrote:
> My question is simply, why?  Why is the sequence number needed when the
> command specified a UID, and the UID must be included in the response.
> Seems to me the UID command should have a UID response.

You have already asked this "question" many times.

Clients that use only UIDs and disregard message sequence numbers are
doomed to suck.

IMAP can not prevent the creation of clients that suck; but it will
nonetheless continue to enable the creation of clients that do not suck.

Nor will IMAP repeat the UID SEARCH command's mistake of command modality
in response interpretation.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On 5/26/2011 3:03 PM, John Snow wrote:
>
> Section 6.4.8 the UID command says:
>
>        The number after the "*" in an untagged FETCH response is always a
>        message sequence number, not a unique identifier, even for a UID
>        command response.  However, server implementations MUST implicitly
>        include the UID message data item as part of any FETCH response
>        caused by a UID command, regardless of whether a UID was specified
>        as a message data item to the FETCH.
>
> My question is simply, why?  Why is the sequence number needed when 
> the command
> specified a UID, and the UID must be included in the response.  Seems 
> to me the UID
> command should have a UID response.

Or my related question (feel free to re-subject if too far OT)...  Why do untagged responses in general (e.g. FETCH or EXPUNGE) always use the message sequence number, and not the UID?  Seriously its a royal PITA to have to even care about a consistent map of message sequence numbers in the first place, when the protocol supports message UIDs.

-- 
----------------------------
  Derek Konigsberg
  dkonigsberg@logicprobe.org
----------------------------

On Thu, 26 May 2011, Philip Guenther wrote:
> Right now, a client can parse a FETCH response at any time, inserting it
> into its cache without having to consider any context of pending commands.
> Indeed, the server can send FETCH responses at any time (modulus TCP
> flow-control/deadlock concerns), and it's *required* to do so for flag
> changes.  Sometimes using the UID and sometimes using msgno would make
> such unsolicited FETCH responses ambiguous.

This is the command modality in response interpretation mistake that was
made in UID SEARCH.  That mistake was allowed, against my better
judgement, on the assumption that clients that do any form of search are
command-modal at that point anyway.

That assumption turned out to be horribly wrong.  It blocked a desirable
extension for mobile devices.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Philip Guenther wrote:
> On Thu, 26 May 2011, John Snow wrote:
>   
>> My question is simply, why?  Why is the sequence number needed when the 
>> command specified a UID, and the UID must be included in the response.  
>> Seems to me the UID command should have a UID response. snow.
>>     
>
> Right now, a client can parse a FETCH response at any time, inserting it 
> into its cache without having to consider any context of pending commands.  
> Indeed, the server can send FETCH responses at any time (modulus TCP 
> flow-control/deadlock concerns), and it's *required* to do so for flag 
> changes.  Sometimes using the UID and sometimes using msgno would make 
> such unsolicited FETCH responses ambiguous.
>
> Philip Guenther
>   

A UID fetch response could just as easily be handled at any time.

 From the example:

   Example:    C: A999 UID FETCH 4827313:4828442 FLAGS
               S: * 23 FETCH (FLAGS (\Seen) UID 4827313)
               S: * 24 FETCH (FLAGS (\Seen) UID 4827943)
               S: * 25 FETCH (FLAGS (\Seen) UID 4828442)
               S: A999 OK UID FETCH completed

       why not this instead?

   Example:    C: A999 UID FETCH 4827313:4828442 FLAGS
               S: * UID 4827313 FETCH (FLAGS (\Seen))
               S: * UID 4827943 FETCH (FLAGS (\Seen) )
               S: * UID 4828442 FETCH (FLAGS (\Seen) )
               S: A999 OK UID FETCH completed


snow.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20110526/2f767f16/attachment.html>

Mark Crispin wrote:
> On Thu, 26 May 2011, John Snow wrote:
>> My question is simply, why?  Why is the sequence number needed when the
>> command specified a UID, and the UID must be included in the response.
>> Seems to me the UID command should have a UID response.
>
> You have already asked this "question" many times.
I may have, I don't remember.  I do remember asking how other servers 
deal with MSNs on a large mailbox.  I  remember saying I'd be happy for 
MSNs to go away.  I don't remember ever asking how it was decided.

I don't expect that my question is going to change the protocol.  I was 
just wondering why the UID command doesn't have a matching UID 
response.  I assume there was some big discussion when this was hammered 
out and wondered what the reasoning might have been.
>
> Clients that use only UIDs and disregard message sequence numbers are
> doomed to suck.
>
maybe, but I don't think it guaranteed. 
> IMAP can not prevent the creation of clients that suck; but it will
> nonetheless continue to enable the creation of clients that do not suck.
>
> Nor will IMAP repeat the UID SEARCH command's mistake of command modality
> in response interpretation.
>
> -- Mark --
>
> http://panda.com/mrc
> Democracy is two wolves and a sheep deciding what to eat for lunch.
> Liberty is a well-armed sheep contesting the vote.
> _______________________________________________
> Imap-protocol mailing list
> Imap-protocol@u.washington.edu
> http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20110526/c47cc391/attachment.html>

On Thu, 26 May 2011, Derek Konigsberg wrote:
> Or my related question (feel free to re-subject if too far OT)...  Why 
> do untagged responses in general (e.g. FETCH or EXPUNGE) always use the 
> message sequence number, and not the UID?  Seriously its a royal PITA to 
> have to even care about a consistent map of message sequence numbers in 
> the first place, when the protocol supports message UIDs.

This question was already answered for the FETCH case.  Did you not see 
how it applies to all message-data/metadata responses?

(The teaching of history is completely overrated.  Why would anyone want 
to study how things came to be the way they are and the engineering 
process that did so when they can just complain that things aren't how 
they wish they were.  We obviously understand everything better now and 
*this time* we'll get it perfectly right, even without reflection on the 
past.  We should delete the imap-protocol mail archives.


Philip Guenther

On Thu, 26 May 2011, John Snow wrote:
> S: * UID 4827313 FETCH (FLAGS (\Seen))

The only benefit of this is to force clients to suck.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Thu, 26 May 2011, John Snow wrote:
> Philip Guenther wrote:
...
> > Right now, a client can parse a FETCH response at any time, inserting 
> > it into its cache without having to consider any context of pending 
> > commands. Indeed, the server can send FETCH responses at any time 
> > (modulus TCP flow-control/deadlock concerns), and it's *required* to 
> > do so for flag changes.  Sometimes using the UID and sometimes using 
> > msgno would make such unsolicited FETCH responses ambiguous.
> 
> A UID fetch response could just as easily be handled at any time.
...
>       why not this instead?
> 
>   Example:    C: A999 UID FETCH 4827313:4828442 FLAGS
>               S: * UID 4827313 FETCH (FLAGS (\Seen))
...

And unsolicited FETCH responses would have used which format?  How would 
the server have decided that?


IMAP message data/metadata processing is basically a cache-fill/coherency 
algorithm: the client says "yo, server, please fill in my cache with at 
least the following bits" and the server does so, while concurrently 
keeping the cache coherent when metadata (flags) are changed by other 
clients...and those are handled consistently, with a single response 
format.

I take it you don't like that design.


Philip

On Thu, May 26, 2011 at 04:28:27PM -0400, John Snow wrote:
> Philip Guenther wrote:
> >On Thu, 26 May 2011, John Snow wrote:
> >>My question is simply, why?  Why is the sequence number needed
> >>when the command specified a UID, and the UID must be included
> >>in the response.  Seems to me the UID command should have a UID
> >>response. snow.
> >
> >Right now, a client can parse a FETCH response at any time,
> >inserting it into its cache without having to consider any context
> >of pending commands.  Indeed, the server can send FETCH responses
> >at any time (modulus TCP flow-control/deadlock concerns), and it's
> >*required* to do so for flag changes.  Sometimes using the UID and
> >sometimes using msgno would make such unsolicited FETCH responses
> >ambiguous.
> >
> >Philip Guenther
> 
> A UID fetch response could just as easily be handled at any time.
> 
> From the example:
> 
>   Example:    C: A999 UID FETCH 4827313:4828442 FLAGS
>               S: * 23 FETCH (FLAGS (\Seen) UID 4827313)
>               S: * 24 FETCH (FLAGS (\Seen) UID 4827943)
>               S: * 25 FETCH (FLAGS (\Seen) UID 4828442)
>               S: A999 OK UID FETCH completed
> 
>       why not this instead?
> 
>   Example:    C: A999 UID FETCH 4827313:4828442 FLAGS
>               S: * UID 4827313 FETCH (FLAGS (\Seen))
>               S: * UID 4827943 FETCH (FLAGS (\Seen) )
>               S: * UID 4828442 FETCH (FLAGS (\Seen) )
>               S: A999 OK UID FETCH completed

Apart from the more complex parser.  Parsing IMAP is
hairy enough as it is.

What I would like to see (if I was writing "IMAP" from
scratch) is putting the "UID" part as a generic search
expression.  So you would either do:

A999 FETCH 1:5 FLAGS

or

A999 FETCH (UID 4827313:4828442) FLAGS

or indeed:

A999 FETCH (OR UNSEEN FLAGGED) (WINDOW 1:20) FLAGS

The ability to request the first "N" messages that match
a particular filter (even more so if you could apply a sort
upfront as well) would be very handy.

And if you were doing something expensive like a full body
search, then being able to sort the result by something
trivial (like REVERSE SENTDATE) and then apply the search
only until you had filled your request window could
potentially save a lot of IO.

But none of that is a good reason to have a conditionally
mutating response format.

Bron ( though I'd love for it to be easier to require IDLE to
       spit out UIDs on every response )

On 27.5.2011, at 1.34, Philip Guenther wrote:

> (The teaching of history is completely overrated.  Why would anyone want 
> to study how things came to be the way they are and the engineering 
> process that did so when they can just complain that things aren't how 
> they wish they were.  We obviously understand everything better now and 
> *this time* we'll get it perfectly right, even without reflection on the 
> past.  We should delete the imap-protocol mail archives.

I've noticed there are many people who want web 2.0 kind of interface to emails (xml/ajax/dunno), and I've been thinking maybe I might just as well do something like that. I'm sure the protocol will suck, but if it gets us better email clients maybe it's a good thing overall. (And why aren't they just using IMAP? Because they're scared of even trying to use it. Hmm. IMAP over XML maybe?..)

On Thu, 26 May 2011, Philip Guenther wrote:
> IMAP message data/metadata processing is basically a cache-fill/coherency
> algorithm

Yes.  That is the entire point of IMAP.

> I take it you don't like that design.

The IMAP3 debacle was because a certain person did not.  The years pass,
and some things never change.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Philip Guenther wrote:
> On Thu, 26 May 2011, John Snow wrote:
>   
>> Philip Guenther wrote:
>>     
> ...
>   
>>> Right now, a client can parse a FETCH response at any time, inserting 
>>> it into its cache without having to consider any context of pending 
>>> commands. Indeed, the server can send FETCH responses at any time 
>>> (modulus TCP flow-control/deadlock concerns), and it's *required* to 
>>> do so for flag changes.  Sometimes using the UID and sometimes using 
>>> msgno would make such unsolicited FETCH responses ambiguous.
>>>       
>> A UID fetch response could just as easily be handled at any time.
>>     
> ...
>   
>>       why not this instead?
>>
>>   Example:    C: A999 UID FETCH 4827313:4828442 FLAGS
>>               S: * UID 4827313 FETCH (FLAGS (\Seen))
>>     
> ...
>
> And unsolicited FETCH responses would have used which format?  How would 
> the server have decided that?
>
>
> IMAP message data/metadata processing is basically a cache-fill/coherency 
> algorithm: the client says "yo, server, please fill in my cache with at 
> least the following bits" and the server does so, while concurrently 
> keeping the cache coherent when metadata (flags) are changed by other 
> clients...and those are handled consistently, with a single response 
> format.
>
> I take it you don't like that design.
>
>
> Philip
>   
Whether I like it or not doesn't really matter.  My problem was 
understanding the need.

I think I got it now.  The problem is that's there's 2 ways to identify 
a message.  The absolute
UID and the relative MSN.  I was thinking only the UID was really 
needed.  But, that doesn't
allow for a client to fetch only predetermined number of messages.  
Sure, you could fetch a
range of UIDs, but there's no way to know how many and what end of the 
mailbox they
might represent.  So you need the MSN to fetch the user desired 
messages, otherwise you'd
have fetch all the messages just to find the ones you want.  That would 
indeed suck.  Since
you need both, and the client and server must be in sync, it must be 
returned in the untagged
response.  To be consistent, it seems that the UID should be returned on 
all fetch responses.
Oh well...

Thanks for the help.




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20110526/90bbcad9/attachment.html>

On Thu May 26 23:36:37 2011, Bron Gondwana wrote:
> A999 FETCH (UID 4827313:4828442) FLAGS
> 
> 
SEARCHRES

> A999 FETCH (OR UNSEEN FLAGGED) (WINDOW 1:20) FLAGS

+ CONTEXT

> And if you were doing something expensive like a full body
> search, then being able to sort the result by something
> trivial (like REVERSE SENTDATE) and then apply the search
> only until you had filled your request window could
> potentially save a lot of IO.

+ ESORT

Dave.
-- 
Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@dave.cridland.net
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

On Fri, 27 May 2011, Timo Sirainen wrote:
> I've noticed there are many people who want web 2.0 kind of interface to
> emails (xml/ajax/dunno)

Only now?

> And why
> aren't they just using IMAP? Because they're scared of even trying to
> use it.

To those who only know Web 2.0, solutions that don't look like Web 2.0 are
hostile and problematic.

I think that it's the poor CS education that most students get these days:
Java as a first programming language, with cookbook software development
using large libraries and predefined templates.  I've met graduates of CS
programs who don't comprehend pointers and recursion!

There's a middle ground between the Scylla of being so mathematical that
real-world engineering is lost, and the Charybdis of making everything be
the assembly of Lego blocks.

> Hmm. IMAP over XML maybe?..

That wasn't an earthquake you just felt.  That was me shuddering in
horror.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Fri, 27 May 2011, Timo Sirainen wrote:

> I've noticed there are many people who want web 2.0 kind of interface to 
> emails (xml/ajax/dunno), and I've been thinking maybe I might just as 
> well do something like that. I'm sure the protocol will suck, but if it 
> gets us better email clients maybe it's a good thing overall. (And why 
> aren't they just using IMAP? Because they're scared of even trying to 
> use it. Hmm. IMAP over XML maybe?..)
>

Communigate has done something similar, see:
http://www.communigate.com/cgatepro/XMLAPI.html#Mailbox


Regards,
John

Timo Sirainen <tss@iki.fi> wrote:

> IMAP over XML maybe?..)

I'm sure it had been a long week when you wrote this, Timo.

A:  IMAP over HTTP, I suppose, though, why?  But nothing is "over" XML --
it's just a syntax.  Microsoft SOAP, for instance, is XML-encrypted DCE
RPC over HTTP, designed because circa 1997 people were willing to poke
holes in their firewalls for port 80.  HTTP is the key, not XML.

B:  The problem with the "runs in a browser" solution is that Javascript
can only make calls to other processes (or servers) by doing HTTP
requests back to the server it was downloaded from.  So it can't speak
any ordinary IETF ASCII protocol.  That's the key weakness of "web"
approaches.

Bill

On Thu, May 26, 2011 at 11:56:07PM +0100, Dave Cridland wrote:
> On Thu May 26 23:36:37 2011, Bron Gondwana wrote:
> >A999 FETCH (UID 4827313:4828442) FLAGS
> >
> >
> SEARCHRES
> 
> >A999 FETCH (OR UNSEEN FLAGGED) (WINDOW 1:20) FLAGS
> 
> + CONTEXT
> 
> >And if you were doing something expensive like a full body
> >search, then being able to sort the result by something
> >trivial (like REVERSE SENTDATE) and then apply the search
> >only until you had filled your request window could
> >potentially save a lot of IO.
> 
> + ESORT

Yeah, we got there eventually.

Bron ( still have to implement these in Cyrus )

On 05/26/2011 09:03 PM, Mark Crispin wrote:
> On Fri, 27 May 2011, Timo Sirainen wrote:
>> I've noticed there are many people who want web 2.0 kind of interface to
>> emails (xml/ajax/dunno)
>
> Only now?
>
>> And why
>> aren't they just using IMAP? Because they're scared of even trying to
>> use it.
>
> To those who only know Web 2.0, solutions that don't look like Web 2.0 are
> hostile and problematic.

There's a saying I've seen somewhere that I love to mention in 
discussions like this:
"Those who fail to understand network protocols are doomed to 
reimplement them over port 80"

> I think that it's the poor CS education that most students get these days:
> Java as a first programming language, with cookbook software development
> using large libraries and predefined templates. I've met graduates of CS
> programs who don't comprehend pointers and recursion!
I'm glad that I went to college before CS programs shifted to Java. 
Sure, the majority of my post-college development work has been in Java 
(or similar languages), but I think Java makes it too easy to gloss over 
the fundamentals that you really do need to understand.

> There's a middle ground between the Scylla of being so mathematical that
> real-world engineering is lost, and the Charybdis of making everything be
> the assembly of Lego blocks.
>
>> Hmm. IMAP over XML maybe?..
>
> That wasn't an earthquake you just felt. That was me shuddering in
> horror.
I think the most painful part of implementing an E-Mail client is 
dealing with all the legacy charsets and encodings, and the different 
ways they're encoded in what is essentially 7-bit US-ASCII.  If I could 
make one change, I'd declare that all human-readable protocol text was 
UTF-8 (and preferably that anything not intended to be human readable 
was just raw bytes inside a literal block).

It would also be nice if the IMAP grammar made more complete use of the 
parenthesized list format used by the majority of it.

To me, the biggest advantage of XML is that it allows for a standardized 
parser/generator library that's maintained independently from the actual 
data format.  (Of course the disadvantages of XML itself are probably 
obvious to everyone here.)

-- 
----------------------------
  Derek Konigsberg
  dkonigsberg@logicprobe.org
----------------------------

On Fri, 27 May 2011 11:53:40 +0300 (EEST)
John Kougoulos <koug@intracom.gr> wrote:

> On Fri, 27 May 2011, Timo Sirainen wrote:
> 
> > I've noticed there are many people who want web 2.0 kind of interface to 
> > emails (xml/ajax/dunno), and I've been thinking maybe I might just as 
> > well do something like that. I'm sure the protocol will suck, but if it 
> > gets us better email clients maybe it's a good thing overall. (And why 
> > aren't they just using IMAP? Because they're scared of even trying to 
> > use it. Hmm. IMAP over XML maybe?..)
> >
> 
> Communigate has done something similar, see:
> http://www.communigate.com/cgatepro/XMLAPI.html#Mailbox

We have a JSON API that may get polished up into something releasable
at some point.  I tried to keep it mirroring QRESYNC as closely as
possible, but unfortunately message sequence numbers are pretty useless
when your basic unit isn't a message, it's a cross-folder conversation.

Bron.

On 31.5.2011, at 18.45, Bill Janssen wrote:

> Timo Sirainen <tss@iki.fi> wrote:
> 
>> IMAP over XML maybe?..)
> 
> I'm sure it had been a long week when you wrote this, Timo.
> 
> A:  IMAP over HTTP, I suppose, though, why?  But nothing is "over" XML --
> it's just a syntax.

I meant changing the IMAP syntax to XML. For example instead of:

a FETCH 2:5 (RFC822.SIZE BODY.PEEK[HEADER.FIELDS (From To)])

it could be:

<fetch tag="a" seq="2" seq2="5">
  <size/>
  <header>From</header>
  <header>To</header>
</fetch>

But that probably still wouldn't be big enough of a change.

> Microsoft SOAP, for instance, is XML-encrypted DCE
> RPC over HTTP, designed because circa 1997 people were willing to poke
> holes in their firewalls for port 80.  HTTP is the key, not XML.


Sure, running it over HTTP would probably be required too, but I think that's a smaller problem.

On 05/26/2011 10:04 PM, Derek Konigsberg wrote:
> I think the most painful part of implementing an E-Mail client is 
> dealing with all the legacy charsets and encodings, and the different 
> ways they're encoded in what is essentially 7-bit US-ASCII.  If I 
> could make one change, I'd declare that all human-readable protocol 
> text was UTF-8 (and preferably that anything not intended to be human 
> readable was just raw bytes inside a literal block).

I have been told that there are some devices capable of reading email 
that do not support UTF-8. Not as in "they can't display it correctly" 
but as in "they can't decode it properly even when explicitly told this 
text is UTF-8." I would much rather be able to go back and declare all 
text to be UTF-8 and never have to worry about charsets ever again.

-- 
Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald E. Knuth

On Thu, 26 May 2011, Derek Konigsberg wrote:
> "Those who fail to understand network protocols are doomed to
> reimplement them over port 80"

Yeah, that's been the nightmare for the past decade or so.

> I think Java makes it too easy to gloss over
> the fundamentals that you really do need to understand.

My point exactly.

Then there are individuals who believe that a kid fresh out of college
with nothing but Java is a superior Java programmer to an experienced
individual that spends a week reading the Java book.  Thus, we have mobile
phones (are you paying attention, RIM?) that issue the message "Uncaught
Java language exception" prior to turning the phone into a pocket warmer.

> I think the most painful part of implementing an E-Mail client is
> dealing with all the legacy charsets and encodings, and the different
> ways they're encoded in what is essentially 7-bit US-ASCII.  If I could
> make one change, I'd declare that all human-readable protocol text was
> UTF-8 (and preferably that anything not intended to be human readable
> was just raw bytes inside a literal block).

Indeed.  The problem is that email predates UTF-8 - and for that matter
Unicode - by about two decades.  Email had largely been cast in stone in
the mid 1970s; and that casting was complete by the early 1980s.

The first non-ASCII email was Japanese; and they played ball with the
ASCII world by using ISO 2022 to shift between ASCII and JIS.  ISO 8859-1
(Latin 1) wasn't until 1987.

I was one of a group of individuals who tried to prevent the charset
disaster in the early 1990s.  We failed utterly.  Our plan was to status
quo of ASCII, ISO 8859-1, and Japan's ISO 2022 encoding; and then move on
with ISO 10646 (Unicode) as the One True Character Set For The Internet.

Certain individuals did not share that vision...

> It would also be nice if the IMAP grammar made more complete use of the
> parenthesized list format used by the majority of it.

Yes.  There should be a single syntax rule for all commands and responses,
and fewer types: atom, number, string, and list.  Atoms should be used
exclusively for protocol tokens (thus abolishing the astring/nstring mess)

I would also reform IMAP's Hollerith strings (literals) to send the entire
command first (thus one complete line) and then the string payloads.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Tue, 31 May 2011, Timo Sirainen wrote:
> I meant changing the IMAP syntax to XML.

JSON.  Please.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Thu, 26 May 2011, Joshua Cranmer wrote:
> I have been told that there are some devices capable of reading email
> that do not support UTF-8. Not as in "they can't display it correctly"
> but as in "they can't decode it properly even when explicitly told this
> text is UTF-8."

There is no excuse for a lack of UTF-8 support in any email device
produced after 1996, and especially after the publication of RFC 2130 in
1997.

Sadly, there is no enforcement.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On Fri May 27 05:11:43 2011, Mark Crispin wrote:
>> It would also be nice if the IMAP grammar made more complete use  
>> of the
>> parenthesized list format used by the majority of it.
> 
> Yes.  There should be a single syntax rule for all commands and  
> responses,
> and fewer types: atom, number, string, and list.  Atoms should be  
> used
> exclusively for protocol tokens (thus abolishing the  
> astring/nstring mess)
> 
> 
I'd note that ACAP essentially did this. You're free to slate ACAP  
for all sorts of reasons, but it had a much simpler and more  
consistent syntax, and the tagged intermediates meant that the UID  
SEARCH/SEARCH issue could be engineered around cleanly in some cases.  
(That said, had this existed in IMAP people would have tried to  
misuse it, of course, but SEARCH is one case where association with  
the command is pretty much essential).

> I would also reform IMAP's Hollerith strings (literals) to send the  
> entire
> command first (thus one complete line) and then the string payloads.

I'm not following the motivation for this. Parse a token at a time  
off the wire until you find a CRLF token - seems simple enough to me.  
It's not as if anyone uses fgets to parse anyway. What am I missing?

Dave.
-- 
Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@dave.cridland.net
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

On Fri May 27 05:19:45 2011, Mark Crispin wrote:
> On Thu, 26 May 2011, Joshua Cranmer wrote:
>> I have been told that there are some devices capable of reading  
>> email
>> that do not support UTF-8. Not as in "they can't display it  
>> correctly"
>> but as in "they can't decode it properly even when explicitly told  
>> this
>> text is UTF-8."
> 
> There is no excuse for a lack of UTF-8 support in any email device
> produced after 1996, and especially after the publication of RFC  
> 2130 in
> 1997.
> 
> Sadly, there is no enforcement.

But who cares?

If a device only understands ASCII, send them UTF-8 anyway and the  
worst that can happen is that a fully non-ASCII message is  
unreadable, which it would be anyway.

Of course, if these devices only understand one of the non-ASCII  
charsets (ie, those which are not ASCII subset compatible), then  
you're doomed anyway.

Dave.
-- 
Dave Cridland - mailto:dave@cridland.net - xmpp:dwd@dave.cridland.net
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

On Fri, 27 May 2011, Dave Cridland wrote:
>> I would also reform IMAP's Hollerith strings (literals) to send the
>> entire command first (thus one complete line) and then the string
>> payloads.
> I'm not following the motivation for this. Parse a token at a time
> off the wire until you find a CRLF token - seems simple enough to me.
> It's not as if anyone uses fgets to parse anyway. What am I missing?

The switch in modality between line and sized buffer in commands turns out
to be a problem in some server frameworks, particularly if multiple
literals appear in the same command.  A related issue is that you need to
read the literals before you know that the command is syntactically valid
and can be done.

Consider a framework in which you only get a thread at I/O complete
events, the thread is given up when it goes into I/O wait, and you get a
different thread each time.  You have to save complete, increasingly
complex, context at each mode switch for a state machine.  You can't rely
upon implied code state in the parser, which is the obvious way to handle
literals and what UW/Panda imapd do.

MULTIAPPEND and CATENATE are particularly extreme examples.

Literals done as I suggest wouldn't be particularly hard for the lexical
token-at-a-time parser; you just build a list of literals to fill in, then
fill them in.  It would be a lifesaver in cases where it's really painful
to break the parse across multiple threads.  [Guess who had to implement
an IMAP server in such an environment.]

If I could take a time machine and change one, but only one, thing in the
basic design of IMAP, this would be it.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

On 05/27/2011 03:51 AM, Dave Cridland wrote:
> If a device only understands ASCII, send them UTF-8 anyway and the 
> worst that can happen is that a fully non-ASCII message is unreadable, 
> which it would be anyway.

In this case, the device was a Japanese mobile phone, so ASCII-only 
support is a non-starter.

-- 
Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald E. Knuth