mailing list archives

meli community discussions

⚠️ if something does not work as intended when interracting with the mailing lists,
reach out Github mirror Gitea repo @epilys:matrix.org

E-mail headers
From: Rick Sanders <rfs9999@earthlink.net>
To: imap-protocol@u.washington.edu
Date: Fri, 08 Jun 2018 12:34:53 -0000
Message-ID: 54CC0B1A.5040701@earthlink.net permalink / raw / eml / mbox
Hi,

This may be due to my incomplete understanding of the IMAP RFC. My 
apologies if that's the case.

The most recent release of Zimbra has changed to the way it closes its 
response to FETCH.  Previously when FETCHING various items the response 
for each message would be terminated by ')' on a new line.  That's what 
I have used for years and it works for all other servers I have worked 
with (Gmail, Dovecot, CommuniGate, Openwave, etc).  But not Zimbra now.

 >> 1 FETCH 1:* (uid flags internaldate RFC822.SIZE 
body.peek[header.fields (From Date Message-Id Subject)])

For example here is what Gmail sends:

<< * 1 FETCH (UID 23910 RFC822.SIZE 17599 INTERNALDATE "16-Jan-2015 
19:28:27 +0000" FLAGS (NonJunk) BODY[HEADER.FIELDS (From Date Message-Id 
Subject)] {211}
<< From: comp.mail.pine@googlegroups.com
<< Subject: Digest for comp.mail.pine@googlegroups.com - 2 updates
<< Message-ID: <e89a8f92403a1c5886050cc9fa6d@google.com>
<< Date: Fri, 16 Jan 2015 19:28:27 +0000
<<
<< )
<< * 2 FETCH (UID 23911 RFC822.SIZE 14881 INTERNALDATE "16-Jan-2015 
20:30:20 +0000" FLAGS (NonJunk) BODY[HEADER.FIELDS (From Date Message-Id 
Subject)] {210}
<< From: comp.mail.imap@googlegroups.com
<< Subject: Digest for comp.mail.imap@googlegroups.com - 1 update
<< Message-ID: <485b397dd00762fb3c050ccad70b@google.com>
<< Date: Fri, 16 Jan 2015 20:30:20 +0000
<<
<< )

In contrast Zimbra sends:

<< * 1 FETCH (UID 10191 INTERNALDATE "03-Dec-2014 10:10:53 -0200" 
RFC822.SIZE 9219 BODY[HEADER.FIELDS (FROM DATE MESSAGE-ID SUBJECT)] {238}
<< From: joe <joe@abc.com>>
<< Subject: test message
<< Date: Wed, 3 Dec 2014 10:10:07 -0200
<< Message-ID: <201412031009340.006629001417608574@pmta05>
01-30-2015.19:38:32 <<
01-30-2015.19:38:32 <<  FLAGS (\Recent))
01-30-2015.19:38:32 << 1 OK FETCH completed

The last line of the response is FLAGS not ')'.  I've had to add special 
logic when Zimbra is involved to consider FLAGS as the end.

One of my customers recently upgraded to Zimbra and immediately the 
application had used for years stopped working.  :-)  And other Zimbra 
users have reported the problem to me as well.

What am I missing or doing wrong?

Thanks,
Rick

-- 
Rick Sanders
rfs9999@earthlink.net
IMAP Tools  http://www.athensfbc.com/imap-tools
Reply
E-mail headers
From: blong@google.com
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:53 -0000
Message-ID: CABa8R6uyNCm=fDhVk8-H5+-73kF_bjFu_Vijz7wVy1fSYaxRmw@mail.gmail.com permalink / raw / eml / mbox
There's nothing wrong with the order in which Zimbra is returning things.

That said, Gmail originally returned fetch attributes in the order they
were requested, but we ran into issues with some clients, so now we return
them in reverse sorted alphabetical order.

ahh, here's the bug we were working around:
https://bugzilla.mozilla.org/show_bug.cgi?id=272988
we needed to return RFC822.SIZE before BODY or Thunderbird had a cow.
Anyways, sorting also makes testing more consistent.

Anyhoo, you should be matching the parenthesis.  IMAP responses, especially
FETCH ones, kind of require you to actually parse the results and not use
something simplistic.

Brandon

On Fri, Jan 30, 2015 at 2:52 PM, Rick Sanders <rfs9999@earthlink.net> wrote:

> Hi,
>
> This may be due to my incomplete understanding of the IMAP RFC. My
> apologies if that's the case.
>
> The most recent release of Zimbra has changed to the way it closes its
> response to FETCH.  Previously when FETCHING various items the response for
> each message would be terminated by ')' on a new line.  That's what I have
> used for years and it works for all other servers I have worked with
> (Gmail, Dovecot, CommuniGate, Openwave, etc).  But not Zimbra now.
>
> >> 1 FETCH 1:* (uid flags internaldate RFC822.SIZE body.peek[header.fields
> (From Date Message-Id Subject)])
>
> For example here is what Gmail sends:
>
> << * 1 FETCH (UID 23910 RFC822.SIZE 17599 INTERNALDATE "16-Jan-2015
> 19:28:27 +0000" FLAGS (NonJunk) BODY[HEADER.FIELDS (From Date Message-Id
> Subject)] {211}
> << From: comp.mail.pine@googlegroups.com
> << Subject: Digest for comp.mail.pine@googlegroups.com - 2 updates
> << Message-ID: <e89a8f92403a1c5886050cc9fa6d@google.com>
> << Date: Fri, 16 Jan 2015 19:28:27 +0000
> <<
> << )
> << * 2 FETCH (UID 23911 RFC822.SIZE 14881 INTERNALDATE "16-Jan-2015
> 20:30:20 +0000" FLAGS (NonJunk) BODY[HEADER.FIELDS (From Date Message-Id
> Subject)] {210}
> << From: comp.mail.imap@googlegroups.com
> << Subject: Digest for comp.mail.imap@googlegroups.com - 1 update
> << Message-ID: <485b397dd00762fb3c050ccad70b@google.com>
> << Date: Fri, 16 Jan 2015 20:30:20 +0000
> <<
> << )
>
> In contrast Zimbra sends:
>
> << * 1 FETCH (UID 10191 INTERNALDATE "03-Dec-2014 10:10:53 -0200"
> RFC822.SIZE 9219 BODY[HEADER.FIELDS (FROM DATE MESSAGE-ID SUBJECT)] {238}
> << From: joe <joe@abc.com>>
> << Subject: test message
> << Date: Wed, 3 Dec 2014 10:10:07 -0200
> << Message-ID: <201412031009340.006629001417608574@pmta05>
> 01-30-2015.19:38:32 <<
> 01-30-2015.19:38:32 <<  FLAGS (\Recent))
> 01-30-2015.19:38:32 << 1 OK FETCH completed
>
> The last line of the response is FLAGS not ')'.  I've had to add special
> logic when Zimbra is involved to consider FLAGS as the end.
>
> One of my customers recently upgraded to Zimbra and immediately the
> application had used for years stopped working.  :-)  And other Zimbra
> users have reported the problem to me as well.
>
> What am I missing or doing wrong?
>
> Thanks,
> Rick
>
> --
> Rick Sanders
> rfs9999@earthlink.net
> IMAP Tools  http://www.athensfbc.com/imap-tools
> _______________________________________________
> Imap-protocol mailing list
> Imap-protocol@u.washington.edu
> http://mailman13.u.washington.edu/mailman/listinfo/imap-protocol
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20150130/9fdec190/attachment.html>
Reply
E-mail headers
From: Pidgeot18@verizon.net
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:53 -0000
Message-ID: 54CC1302.80003@verizon.net permalink / raw / eml / mbox
On 1/30/2015 4:52 PM, Rick Sanders wrote:
> What am I missing or doing wrong?

Your IMAP parser is buggy. The result of a FLAGS is, in the ABNF 
terminology:

msg-att         = "(" (msg-att-dynamic / msg-att-static)
                    *(SP (msg-att-dynamic / msg-att-static)) ")"


which is to say, that it is a list of terms in parentheses where each 
term is separated by *exactly* one ASCII space (=0x20) character. Where 
you're likely screwing up is you're misreading the literal string for 
the BODY data. This is a number of octets surrounded by curly braces, 
followed by a single CRLF pair, and then the exact number of octets.  If 
you're searching for a line that consists of particular data... you're 
doing something REALLY wrong. Here, let me screw up that match for you 
right now:
  )

(IMAP isn't like POP, SMTP, or NNTP--multiline data is counted in terms 
of exact octets, not until a CRLF.CRLF is found).

-- 
Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald E. Knuth
Reply
E-mail headers
From: jguthrie@brokersys.com
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:53 -0000
Message-ID: 54CC13C0.4010408@brokersys.com permalink / raw / eml / mbox
The closing parenthesis comes immediately after the last character of 
the last message attribute.  The message attribute before the 
parenthesis in the gmail responses is the message header, which ends in 
two newlines.  The last message attribute in the Zimbra response is the 
flags which ends in a parenthesis.

If Zimbra chose to respond with the message attributes in the same order 
as gmail, then it would look exactly the same as the gmail response.

So, assuming that the literal character counts in the gmail responses 
are accurate (I'm not going to count them to make sure) your response 
processing is broken.  When you get to the end of a message attribute, 
you should expect to either see a space, indicating that you haven't run 
out of message attributes, or a closing parenthesis.

On 1/30/2015 4:52 PM, Rick Sanders wrote:
> Hi,
>
> This may be due to my incomplete understanding of the IMAP RFC. My 
> apologies if that's the case.
>
> The most recent release of Zimbra has changed to the way it closes its 
> response to FETCH.  Previously when FETCHING various items the 
> response for each message would be terminated by ')' on a new line.  
> That's what I have used for years and it works for all other servers I 
> have worked with (Gmail, Dovecot, CommuniGate, Openwave, etc).  But 
> not Zimbra now.
>
> >> 1 FETCH 1:* (uid flags internaldate RFC822.SIZE 
> body.peek[header.fields (From Date Message-Id Subject)])
>
> For example here is what Gmail sends:
>
> << * 1 FETCH (UID 23910 RFC822.SIZE 17599 INTERNALDATE "16-Jan-2015 
> 19:28:27 +0000" FLAGS (NonJunk) BODY[HEADER.FIELDS (From Date 
> Message-Id Subject)] {211}
> << From: comp.mail.pine@googlegroups.com
> << Subject: Digest for comp.mail.pine@googlegroups.com - 2 updates
> << Message-ID: <e89a8f92403a1c5886050cc9fa6d@google.com>
> << Date: Fri, 16 Jan 2015 19:28:27 +0000
> <<
> << )
> << * 2 FETCH (UID 23911 RFC822.SIZE 14881 INTERNALDATE "16-Jan-2015 
> 20:30:20 +0000" FLAGS (NonJunk) BODY[HEADER.FIELDS (From Date 
> Message-Id Subject)] {210}
> << From: comp.mail.imap@googlegroups.com
> << Subject: Digest for comp.mail.imap@googlegroups.com - 1 update
> << Message-ID: <485b397dd00762fb3c050ccad70b@google.com>
> << Date: Fri, 16 Jan 2015 20:30:20 +0000
> <<
> << )
>
> In contrast Zimbra sends:
>
> << * 1 FETCH (UID 10191 INTERNALDATE "03-Dec-2014 10:10:53 -0200" 
> RFC822.SIZE 9219 BODY[HEADER.FIELDS (FROM DATE MESSAGE-ID SUBJECT)] {238}
> << From: joe <joe@abc.com>>
> << Subject: test message
> << Date: Wed, 3 Dec 2014 10:10:07 -0200
> << Message-ID: <201412031009340.006629001417608574@pmta05>
> 01-30-2015.19:38:32 <<
> 01-30-2015.19:38:32 <<  FLAGS (\Recent))
> 01-30-2015.19:38:32 << 1 OK FETCH completed
>
> The last line of the response is FLAGS not ')'.  I've had to add 
> special logic when Zimbra is involved to consider FLAGS as the end.
>
> One of my customers recently upgraded to Zimbra and immediately the 
> application had used for years stopped working.  :-)  And other Zimbra 
> users have reported the problem to me as well.
>
> What am I missing or doing wrong?
>
> Thanks,
> Rick
>
Reply
E-mail headers
From: rfs9999@earthlink.net
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:53 -0000
Message-ID: 54CC1830.9060808@earthlink.net permalink / raw / eml / mbox
> Your IMAP parser is buggy.

 >> you should be matching the parenthesis.  IMAP responses, especially 
FETCH ones, kind of require you to actually parse the results and not 
use something simplistic.

Thanks, guys.  I'll fix my parser.  SET SIMPLISTIC = OFF.

-Rick


-- 
Rick Sanders
rfs9999@earthlink.net
IMAP Tools  http://www.athensfbc.com/imap-tools
Reply
E-mail headers
From: arnt@gulbrandsen.priv.no
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:53 -0000
Message-ID: 1e8ecb2f-a952-4c28-a4ec-3cd85f3682a8@gulbrandsen.priv.no permalink / raw / eml / mbox
On Saturday, January 31, 2015 12:48:00 AM CEST, Rick Sanders wrote:
> Thanks, guys.  I'll fix my parser.  SET SIMPLISTIC = OFF.

Shocking that it worked for so many years without that fix.

Arnt
Reply