mailing list archives

meli community discussions

⚠️ if something does not work as intended when interracting with the mailing lists,
reach out Github mirror Gitea repo @epilys:matrix.org

E-mail headers
From: Liam Clarke <ml.cyresse@gmail.com>
To: imap-protocol@u.washington.edu
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: b6f3249e0801091938m12cbccd6j4fff9f406cf0fee1@mail.gmail.com permalink / raw / eml / mbox
Hi,

I'm very new to IMAP, looking to implement a client library and
somewhat worried by this line in the RFC:

> A client MUST be prepared to accept any server response at all times. This includes server data that was not requested.

For the most part handling of unrequested untagged responses seems to
be straightforward pattern matching, as the context doesn't really
matter - but what has me confused and a little worried is one of the
examples of two FETCH responses in the last section of the RFC
(apologies for the long c&p):

C:   a003 fetch 12 full
S:   * 12 FETCH (FLAGS (\Seen) INTERNALDATE "17-Jul-1996 02:44:25 -0700"
      RFC822.SIZE 4286 ENVELOPE ("Wed, 17 Jul 1996 02:23:25 -0700 (PDT)"
      "IMAP4rev1 WG mtg summary and minutes"
      (("Terry Gray" NIL "gray" "cac.washington.edu"))
      (("Terry Gray" NIL "gray" "cac.washington.edu"))
      (("Terry Gray" NIL "gray" "cac.washington.edu"))
      ((NIL NIL "imap" "cac.washington.edu"))
      ((NIL NIL "minutes" "CNRI.Reston.VA.US")
      ("John Klensin" NIL "KLENSIN" "MIT.EDU")) NIL NIL
      "<B27397-0100000@cac.washington.edu>")
       BODY ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 3028
       92))
S:    a003 OK FETCH completed
C:    a004 fetch 12 body[header]
S:    * 12 FETCH (BODY[HEADER] {342}
S:    Date: Wed, 17 Jul 1996 02:23:25 -0700 (PDT)
S:    From: Terry Gray <gray@cac.washington.edu>
S:    Subject: IMAP4rev1 WG mtg summary and minutes
S:    To: imap@cac.washington.edu
S:    cc: minutes@CNRI.Reston.VA.US, John Klensin <KLENSIN@MIT.EDU>
S:    Message-Id: <B27397-0100000@cac.washington.edu>
S:    MIME-Version: 1.0
S:    Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
S:
S:    )
S:    a004 OK FETCH completed

From the layout and note that " A long line in this sample is broken
for editorial clarity." I'm deducing that the server response to a003
is all on a single \r\n delimited line, but is the second actually
split over multiple \r\n delimited lines?

If so, would it be possible for lines from separate fetch responses to
arrive together? Borrowing trouble from the future, but just got
worried when I saw that.

Regards,

Liam Clarke-Hutchinson
Reply
E-mail headers
From: MRC@CAC.Washington.EDU
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: alpine.WNT.1.00.0801091945320.3080@Tomobiki-Cho.CAC.Washignton.EDU permalink / raw / eml / mbox
In the example that you quoted, you will notice that lines from the client 
are labelled with "C:" and lines from the server are labelled with "S:". 
But, in this example, there are some physical lines which are not 
labelled.  Those are all part of the same line that came from the server, 
and the line break that was in the example is not actually in the 
protocol.

I'm not sure what you mean by "the second".  Do you mean the response to 
the a004 command?  That is, indeed, a multiline response, but you will 
notice the use of a literal (the "{342}").

An IMAP client must read an arbitrarily long line from the server.  And by 
"arbitrarily long" I mean exactly that; there is no limit.  If the line 
does not end with a literal size specifier such as {342} that is the end 
of the response.  If, however, the line ends with such a specifier, then 
you must read exactly that number of octets in substitution for the 
literal size specifier, THEN more of the line afterwards.

For example, the two responses

* 12 FETCH FOO {3}
BAR ZAP {5}
ZOWIE

and

* 12 FETCH FOO BAR ZAP ZOWIE

are exactly equivalent.

Does this answer your question?

I would also like to suggest that, instead of implementing a client 
library, that you consider using an existing client library for your 
application.  One such library is my c-client library, part of the UW IMAP 
Tookit on
 	ftp://ftp.cac.washington.edu/mail/imap.tar.Z

On Thu, 10 Jan 2008, Liam Clarke wrote:
> I'm very new to IMAP, looking to implement a client library and
> somewhat worried by this line in the RFC:
>
>> A client MUST be prepared to accept any server response at all times. This includes server data that was not requested.
>
> For the most part handling of unrequested untagged responses seems to
> be straightforward pattern matching, as the context doesn't really
> matter - but what has me confused and a little worried is one of the
> examples of two FETCH responses in the last section of the RFC
> (apologies for the long c&p):
>
> C:   a003 fetch 12 full
> S:   * 12 FETCH (FLAGS (\Seen) INTERNALDATE "17-Jul-1996 02:44:25 -0700"
>      RFC822.SIZE 4286 ENVELOPE ("Wed, 17 Jul 1996 02:23:25 -0700 (PDT)"
>      "IMAP4rev1 WG mtg summary and minutes"
>      (("Terry Gray" NIL "gray" "cac.washington.edu"))
>      (("Terry Gray" NIL "gray" "cac.washington.edu"))
>      (("Terry Gray" NIL "gray" "cac.washington.edu"))
>      ((NIL NIL "imap" "cac.washington.edu"))
>      ((NIL NIL "minutes" "CNRI.Reston.VA.US")
>      ("John Klensin" NIL "KLENSIN" "MIT.EDU")) NIL NIL
>      "<B27397-0100000@cac.washington.edu>")
>       BODY ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 3028
>       92))
> S:    a003 OK FETCH completed
> C:    a004 fetch 12 body[header]
> S:    * 12 FETCH (BODY[HEADER] {342}
> S:    Date: Wed, 17 Jul 1996 02:23:25 -0700 (PDT)
> S:    From: Terry Gray <gray@cac.washington.edu>
> S:    Subject: IMAP4rev1 WG mtg summary and minutes
> S:    To: imap@cac.washington.edu
> S:    cc: minutes@CNRI.Reston.VA.US, John Klensin <KLENSIN@MIT.EDU>
> S:    Message-Id: <B27397-0100000@cac.washington.edu>
> S:    MIME-Version: 1.0
> S:    Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
> S:
> S:    )
> S:    a004 OK FETCH completed
>
>> From the layout and note that " A long line in this sample is broken
> for editorial clarity." I'm deducing that the server response to a003
> is all on a single \r\n delimited line, but is the second actually
> split over multiple \r\n delimited lines?
>
> If so, would it be possible for lines from separate fetch responses to
> arrive together? Borrowing trouble from the future, but just got
> worried when I saw that.
>
> Regards,
>
> Liam Clarke-Hutchinson
> _______________________________________________
> Imap-protocol mailing list
> Imap-protocol@u.washington.edu
> https://mailman1.u.washington.edu/mailman/listinfo/imap-protocol
>

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
Reply
E-mail headers
From: ams@oryx.com
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: 20080110040209.GA25060@toroid.org permalink / raw / eml / mbox
Hi Liam.

At 2008-01-10 16:38:00 +1300, ml.cyresse@gmail.com wrote:
>
> C:    a004 fetch 12 body[header]
> S:    * 12 FETCH (BODY[HEADER] {342}
> S:    Date: Wed, 17 Jul 1996 02:23:25 -0700 (PDT)
> S:    From: Terry Gray <gray@cac.washington.edu>
> S:    Subject: IMAP4rev1 WG mtg summary and minutes
> S:    To: imap@cac.washington.edu
> S:    cc: minutes@CNRI.Reston.VA.US, John Klensin <KLENSIN@MIT.EDU>
> S:    Message-Id: <B27397-0100000@cac.washington.edu>
> S:    MIME-Version: 1.0
> S:    Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
> S:
> S:    )
> S:    a004 OK FETCH completed

> [...] but is the second actually split over multiple \r\n delimited
> lines?

Not exactly. It contains a literal that contains multiple CRLFs.

> If so, would it be possible for lines from separate fetch responses to
> arrive together?

No. This response:

> S:    * 12 FETCH (BODY[HEADER] {342}

promises to send you 342 bytes of literal data immediately following the
CRLF (see RFC 3501 ?4.3 for an explanation of literals).

-- ams
Reply
E-mail headers
From: ml.cyresse@gmail.com
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: b6f3249e0801092004x584dac51t2156511e43b7a7d6@mail.gmail.com permalink / raw / eml / mbox
Hi Mark,

Thanks for that - my vague fear is that a poorly behaved server could
mix responses to one request with responses for another, which
probably betrays my lack of experience with IMAP servers. And I've
downloaded your library, just looking to implement some partial
functionality as a method of learning the protocol. I'm a learn by
doing kinda guy. Back to the RFC :)

Regards,

Liam Clarke-Hutchinson

On Jan 10, 2008 4:53 PM, Mark Crispin <MRC@cac.washington.edu> wrote:
> In the example that you quoted, you will notice that lines from the client
> are labelled with "C:" and lines from the server are labelled with "S:".
> But, in this example, there are some physical lines which are not
> labelled.  Those are all part of the same line that came from the server,
> and the line break that was in the example is not actually in the
> protocol.
>
> I'm not sure what you mean by "the second".  Do you mean the response to
> the a004 command?  That is, indeed, a multiline response, but you will
> notice the use of a literal (the "{342}").
>
> An IMAP client must read an arbitrarily long line from the server.  And by
> "arbitrarily long" I mean exactly that; there is no limit.  If the line
> does not end with a literal size specifier such as {342} that is the end
> of the response.  If, however, the line ends with such a specifier, then
> you must read exactly that number of octets in substitution for the
> literal size specifier, THEN more of the line afterwards.
>
> For example, the two responses
>
> * 12 FETCH FOO {3}
> BAR ZAP {5}
> ZOWIE
>
> and
>
> * 12 FETCH FOO BAR ZAP ZOWIE
>
> are exactly equivalent.
>
> Does this answer your question?
>
> I would also like to suggest that, instead of implementing a client
> library, that you consider using an existing client library for your
> application.  One such library is my c-client library, part of the UW IMAP
> Tookit on
>         ftp://ftp.cac.washington.edu/mail/imap.tar.Z
>
>
> On Thu, 10 Jan 2008, Liam Clarke wrote:
> > I'm very new to IMAP, looking to implement a client library and
> > somewhat worried by this line in the RFC:
> >
> >> A client MUST be prepared to accept any server response at all times. This includes server data that was not requested.
> >
> > For the most part handling of unrequested untagged responses seems to
> > be straightforward pattern matching, as the context doesn't really
> > matter - but what has me confused and a little worried is one of the
> > examples of two FETCH responses in the last section of the RFC
> > (apologies for the long c&p):
> >
> > C:   a003 fetch 12 full
> > S:   * 12 FETCH (FLAGS (\Seen) INTERNALDATE "17-Jul-1996 02:44:25 -0700"
> >      RFC822.SIZE 4286 ENVELOPE ("Wed, 17 Jul 1996 02:23:25 -0700 (PDT)"
> >      "IMAP4rev1 WG mtg summary and minutes"
> >      (("Terry Gray" NIL "gray" "cac.washington.edu"))
> >      (("Terry Gray" NIL "gray" "cac.washington.edu"))
> >      (("Terry Gray" NIL "gray" "cac.washington.edu"))
> >      ((NIL NIL "imap" "cac.washington.edu"))
> >      ((NIL NIL "minutes" "CNRI.Reston.VA.US")
> >      ("John Klensin" NIL "KLENSIN" "MIT.EDU")) NIL NIL
> >      "<B27397-0100000@cac.washington.edu>")
> >       BODY ("TEXT" "PLAIN" ("CHARSET" "US-ASCII") NIL NIL "7BIT" 3028
> >       92))
> > S:    a003 OK FETCH completed
> > C:    a004 fetch 12 body[header]
> > S:    * 12 FETCH (BODY[HEADER] {342}
> > S:    Date: Wed, 17 Jul 1996 02:23:25 -0700 (PDT)
> > S:    From: Terry Gray <gray@cac.washington.edu>
> > S:    Subject: IMAP4rev1 WG mtg summary and minutes
> > S:    To: imap@cac.washington.edu
> > S:    cc: minutes@CNRI.Reston.VA.US, John Klensin <KLENSIN@MIT.EDU>
> > S:    Message-Id: <B27397-0100000@cac.washington.edu>
> > S:    MIME-Version: 1.0
> > S:    Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
> > S:
> > S:    )
> > S:    a004 OK FETCH completed
> >
> >> From the layout and note that " A long line in this sample is broken
> > for editorial clarity." I'm deducing that the server response to a003
> > is all on a single \r\n delimited line, but is the second actually
> > split over multiple \r\n delimited lines?
> >
> > If so, would it be possible for lines from separate fetch responses to
> > arrive together? Borrowing trouble from the future, but just got
> > worried when I saw that.
> >
> > Regards,
> >
> > Liam Clarke-Hutchinson
> > _______________________________________________
> > Imap-protocol mailing list
> > Imap-protocol@u.washington.edu
> > https://mailman1.u.washington.edu/mailman/listinfo/imap-protocol
> >
>
> -- Mark --
>
> http://staff.washington.edu/mrc
> Science does not emerge from voting, party politics, or public debate.
> Si vis pacem, para bellum.
>
Reply
E-mail headers
From: ml.cyresse@gmail.com
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: b6f3249e0801092005j465aa77y998bda9894dd4248@mail.gmail.com permalink / raw / eml / mbox
Oh dear,

Thanjks Abhijit. I need to read more carefully.

On Jan 10, 2008 5:02 PM, Abhijit Menon-Sen <ams@oryx.com> wrote:
> Hi Liam.
>
> At 2008-01-10 16:38:00 +1300, ml.cyresse@gmail.com wrote:
> >
> > C:    a004 fetch 12 body[header]
> > S:    * 12 FETCH (BODY[HEADER] {342}
> > S:    Date: Wed, 17 Jul 1996 02:23:25 -0700 (PDT)
> > S:    From: Terry Gray <gray@cac.washington.edu>
> > S:    Subject: IMAP4rev1 WG mtg summary and minutes
> > S:    To: imap@cac.washington.edu
> > S:    cc: minutes@CNRI.Reston.VA.US, John Klensin <KLENSIN@MIT.EDU>
> > S:    Message-Id: <B27397-0100000@cac.washington.edu>
> > S:    MIME-Version: 1.0
> > S:    Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
> > S:
> > S:    )
> > S:    a004 OK FETCH completed
>
> > [...] but is the second actually split over multiple \r\n delimited
> > lines?
>
> Not exactly. It contains a literal that contains multiple CRLFs.
>
> > If so, would it be possible for lines from separate fetch responses to
> > arrive together?
>
> No. This response:
>
> > S:    * 12 FETCH (BODY[HEADER] {342}
>
> promises to send you 342 bytes of literal data immediately following the
> CRLF (see RFC 3501 ?4.3 for an explanation of literals).
>
> -- ams
>
Reply
E-mail headers
From: MRC@CAC.Washington.EDU
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: alpine.WNT.1.00.0801092005220.3080@Tomobiki-Cho.CAC.Washignton.EDU permalink / raw / eml / mbox
On Thu, 10 Jan 2008, Liam Clarke wrote:
> Thanks for that - my vague fear is that a poorly behaved server could
> mix responses to one request with responses for another

It all depends upon what you mean by this.

A server can not intertwine one response with another; that is, given 
responses

* 12 FOO BAR ZAP
* 13 GOLLY GEE WOW

A server can not send

* 12 FOO * 13 GOLLY GEE WOW
BAR ZAP

However, a server most definitely MAY send ANY untagged response to ANY 
command.  A client MUST NOT assume that (for example) if it is sending a 
SEARCH command that it will not receive a FETCH command.

More to the point: a client MUST NOT be modal in its untagged response 
processing based upon what command it is sending.

For example, the following is the WRONG way to implement SEARCH:
 	send a SEARCH command
 	read an untagged SEARCH response and use it
 	read the tagged OK/NO/BAD response

The following is the RIGHT way to implement SEARCH:
 	erase "list of searched messages"
 	send a SEARCH command
 	read and process some number of untagged responses
 	read the tagged OK/NO/BAD response
 	use the "list of searched messages"
with the following untagged response processor:
 	if response is ... then ...
 	else if response is SEARCH then
 		add value to the "list of searched messages"
 	else if response is ... then ...

Now...under normal circumstances in today's protocol, you will not get a 
SEARCH response except during a SEARCH command.  But you MUST NOT assume 
that, since maybe, just maybe, a protocol extension will cause SEARCH 
responses to happen at other times.

Most untagged responses can, and do, happen at any time in IMAP in real 
life: OK, NO, BAD, BYE, FLAGS, EXISTS, RECENT, EXPUNGE (some exceptions 
here), FETCH, CAPABILITY.

Only a few responses seem to be limited to specific commands in the base 
protocol: LIST, LSUB, SEARCH, STATUS.  But this can change at any time, so 
don't assume that it is so limited.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
Reply
E-mail headers
From: guenther+imap@sendmail.com
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: Pine.BSO.4.64.0801092110230.31840@vanye.mho.net permalink / raw / eml / mbox
On Thu, 10 Jan 2008, Liam Clarke wrote:
> Thanks for that - my vague fear is that a poorly behaved server could
> mix responses to one request with responses for another, which
> probably betrays my lack of experience with IMAP servers.

It's not 100% clear to me what you mean by "mix responses to one requests 
with responses for another".  Can a server send back FETCH responses for 
messages that you never asked about?  Yes!  Can it send back FETCH data 
items that you never asked about?  Yes!  Indeed, those both commonly occur 
for flags.  For example, let's say the flags on messages 2 and 3 have 
changed since the client last sent a command, and then it sends this:

 	a042 FETCH 2 RFC822.SIZE

The server may send back the following:
 	* 2 FETCH (RFC822.SIZE 123985 FLAGS (\seen \deleted))
 	* 3 FETCH (FLAGS (\seen \recent \deleted))
 	a042 Ok


IMAP FETCH is effectively a cache-filling protocol: the client asks for 
whatever data that it wants but doesn't already have and is told when all 
that data has been sent.  However, the server may send additional, 
unrequested data along the way in order to keep the client's cache up to 
date.  c.f. RFC 3501, section 5.2 for what servers are required and 
expected to send without being asked.


Philip Guenther
Reply
E-mail headers
From: ml.cyresse@gmail.com
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: b6f3249e0801101213v6ffa60c4hcc9e064a4a2a19e7@mail.gmail.com permalink / raw / eml / mbox
Thanks all for greatly clarifying my understanding (or lack thereof) of this.

> A server can not intertwine one response with another; that is, given responses

That clears my main misconception.

Regards,

Liam Clarke
Reply
E-mail headers
From: ams@oryx.com
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: 20080110043556.GA25217@toroid.org permalink / raw / eml / mbox
At 2008-01-09 21:23:53 -0700, guenther+imap@sendmail.com wrote:
>
> It's not 100% clear to me what you mean by "mix responses to one
> requests with responses for another".

I think he just meant this:

S: * 1 FETCH (BODY[HEADER.FIELDS] {666}
S: From: foo bar <foo@bar.example.org>
S: * 2 FETCH (UID 1 ...
S: To: bar foo <...

But of course, that can never happen, because each response is
conceptually a single long line, though it may have embedded
literals.

-- ams
Reply
E-mail headers
From: tss@iki.fi
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: 1199939792.8850.111.camel@hurina permalink / raw / eml / mbox
On Wed, 2008-01-09 at 21:23 -0700, Philip Guenther wrote:
> On Thu, 10 Jan 2008, Liam Clarke wrote:
> > Thanks for that - my vague fear is that a poorly behaved server could
> > mix responses to one request with responses for another, which
> > probably betrays my lack of experience with IMAP servers.
> 
> It's not 100% clear to me what you mean by "mix responses to one requests 
> with responses for another".  Can a server send back FETCH responses for 
> messages that you never asked about?  Yes!  Can it send back FETCH data 
> items that you never asked about?  Yes!  Indeed, those both commonly occur 
> for flags.

Also if client sends two commands, the server may mix them:

1 fetch 1:2 flags
2 fetch 1:2 uid
* 1 fetch (flags (\seen))
* 1 fetch (uid 100)
* 2 fetch (flags ())
* 2 fetch (uid 101)
2 ok
1 ok

Although I'm not sure if this is allowed:

1 fetch 1:2 flags
2 fetch 1:2 uid
* 1 fetch (flags (\seen) uid 100)
* 2 fetch (flags () uid 101)
2 ok
1 ok

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20080110/ec9d4a37/attachment.sig>
Reply
E-mail headers
From: MRC@CAC.Washington.EDU
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: alpine.WNT.1.00.0801092037120.3080@Tomobiki-Cho.CAC.Washignton.EDU permalink / raw / eml / mbox
On Thu, 10 Jan 2008, Timo Sirainen wrote:
> Although I'm not sure if this is allowed:
> 1 fetch 1:2 flags
> 2 fetch 1:2 uid
> * 1 fetch (flags (\seen) uid 100)
> * 2 fetch (flags () uid 101)
> 2 ok
> 1 ok

This is allowed, although I don't know of any server that does it.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
Reply
E-mail headers
From: pruiter@ca.ibm.com
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:41 -0000
Message-ID: OF18A22368.8F402169-ON882573CC.001B46DD-882573CC.001BE898@ca.ibm.com permalink / raw / eml / mbox
>Also if client sends two commands, the server may mix them:
>
>1 fetch 1:2 flags
>2 fetch 1:2 uid
>* 1 fetch (flags (\seen))
>* 1 fetch (uid 100)
>* 2 fetch (flags ())
>* 2 fetch (uid 101)
>2 ok
>1 ok

For someone new to the protocol this could be a confusing example since 
the tags are the same as the message numbers and they may somehow think 
the message numbers in the untagged responses tie back to the command with 
that tag.  Probably better as:
a fetch 1:2 flags
b fetch 1:2 uid
* 1 fetch (flags (\seen))
* 1 fetch (uid 100)
* 2 fetch (flags ())
* 2 fetch (uid 101)
b ok
a ok

Perry Ruiter
250-658-6517
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20080109/bd02fc42/attachment.html>
Reply