mailing list archives

meli community discussions

⚠️ if something does not work as intended when interracting with the mailing lists,
reach out Github mirror Gitea repo @epilys:matrix.org

E-mail headers
From: Paul J Stevens <paul@nfg.nl>
To: imap-protocol@u.washington.edu
Date: Fri, 08 Jun 2018 12:34:47 -0000
Message-ID: 4F53524C.3000308@nfg.nl permalink / raw / eml / mbox
Hi all,

I'm working on improving IMAP compliance in DBMail, using Timo's test-suite.

One thing has me stumped in the 'append' test

Comment in the test reads

# Two connections have mailbox SELECTed, one doesn't.
# The \recent flags can be given to either one of the SELECTed
connections,

# but never for the 3rd. We rely on mailbox state tracking to catch
duplicate
# \recent flags (which is why there are two FETCH FLAGS commands).

I don't understand why a status command on a 3rd session should be able
to see the updated number of recent messages. The 3rd session selecting
the mailbox and fetching the flags *before* one of the selected sessions
fetch them seems perfectly valid to me.

To me the test reads as if the recent status of all mailboxes should be
frozen at authentication, until a mailbox is selected or examined - but
that would defeat the purpose of the status command as I understand it.

If someone could shed some light on this I would appreciate it.

thanks.


-- 
________________________________________________________________
Paul J Stevens        pjstevns @ gmail, twitter, skype, linkedin

  * Premium Hosting Services and Web Application Consultancy *

           www.nfg.nl/info@nfg.nl/+31.85.877.99.97
________________________________________________________________
Reply
E-mail headers
From: tss@iki.fi
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:47 -0000
Message-ID: ACDE4DFC-35B3-4CD9-80F9-DD2AB711BBC4@iki.fi permalink / raw / eml / mbox
On 4.3.2012, at 13.30, Paul J Stevens wrote:

> One thing has me stumped in the 'append' test
> 
> # Two connections have mailbox SELECTed, one doesn't.
> # The \recent flags can be given to either one of the SELECTed
> connections,
> 
> # but never for the 3rd. We rely on mailbox state tracking to catch
> duplicate
> # \recent flags (which is why there are two FETCH FLAGS commands).
> 
> I don't understand why a status command on a 3rd session should be able
> to see the updated number of recent messages. The 3rd session selecting
> the mailbox and fetching the flags *before* one of the selected sessions
> fetch them seems perfectly valid to me.

I'm not entirely sure what you mean. Anyway, what the test is supposed to do is:

 - 1 and 2 connections have mailbox selected
 - 1 connection APPENDs a new message
 - most servers assign \recent flag to connection 1 before APPEND returns tagged OK
 - just in case some server might have assigned it to 2nd connection, the 2nd connection does a NOOP
 - now imaptest knows that the \recent must be in either connection 1 or connection 2 (both connections have informed the client with EXISTS that there are new messages, so the \recent flag must have been set)
 - (if there had been yet another EXAMINEd mailbox, I guess it could also show the message as \recent, but imaptest doesn't create such a connection)
 - any further connections that SELECT/EXAMINE/STATUS the mailbox won't see the message as \recent (by definition of how \recent works)
 - so the 3rd connection's STATUS should see updated message/unseen count, but recent count must be 0
Reply
E-mail headers
From: paul@nfg.nl
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:48 -0000
Message-ID: 4F536EBE.7080509@nfg.nl permalink / raw / eml / mbox
Timo,

Thanks for your reply,

On 03/04/2012 12:49 PM, Timo Sirainen wrote:
>> I don't understand why a status command on a 3rd session should be able
>> to see the updated number of recent messages.

Ai, I meant: I don't understand why the 3rd session *can't* see the
updated number of recent messages.

>> The 3rd session selecting
>> the mailbox and fetching the flags *before* one of the selected sessions
>> fetch them seems perfectly valid to me.
> 
> I'm not entirely sure what you mean. Anyway, what the test is supposed to do is:
> 
>  - 1 and 2 connections have mailbox selected
>  - 1 connection APPENDs a new message
>  - most servers assign \recent flag to connection 1 before APPEND returns tagged OK

That smells to me like an implementation dependent choice, not a
protocol dictated requirement.

>  - just in case some server might have assigned it to 2nd connection, the 2nd connection does a NOOP
>  - now imaptest knows that the \recent must be in either connection 1 or connection 2 (both connections have informed the client with EXISTS that there are new messages, so the \recent flag must have been set)

I'm here to learn, but I don't see this in the specs.

Returning an updated EXISTS response in a session doesn't imply that
later on the client will actually request the message flags.

I understand a \recent flag must never be returned more than once or in
more than one session. But as I (re)read rfc3501 a \recent flag is not
assigned to a session until a FETCH FLAGS response was returned.

>  - (if there had been yet another EXAMINEd mailbox, I guess it could also show the message as \recent, but imaptest doesn't create such a connection)
>  - any further connections that SELECT/EXAMINE/STATUS the mailbox won't see the message as \recent (by definition of how \recent works)
>  - so the 3rd connection's STATUS should see updated message/unseen count, but recent count must be 0

I get the feeling there are undocumented semantics in play here. Doing a
STATUS doesn't affect the mailbox, fine. So even when a STATUS response
returns a RECENT > 0, it seems perfectly valid to me if subsequence
SELECT/FETCH on the same mailbox with the same session don't return any
messages with the \recent flag.

What the test seems to imply is that following transcript with two
sessions A and B is incorrect.

A status mailbox (messages unseen recent)
* status mailbox (messages 1 unseen 1 recent 1)
A select mailbox
* 1 exists
* 1 recent
...
A ok [read-write] select completed

B status mailbox (messages unseen recent)
* status mailbox (messages 1 unseen 1 recent 1)
B select mailbox
* 1 exists
* 1 recent
...
B ok [read-write] select completed
B fetch 1 flags
* 1 fetch (flags (\recent))
B ok fetch completed

A fetch 1 flags
* 1 fetch (flags ())
A ok fetch completed


-- 
________________________________________________________________
Paul J Stevens        pjstevns @ gmail, twitter, skype, linkedin

  * Premium Hosting Services and Web Application Consultancy *

           www.nfg.nl/info@nfg.nl/+31.85.877.99.97
________________________________________________________________
Reply
E-mail headers
From: tss@iki.fi
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:48 -0000
Message-ID: 5F81467D-EBB4-49AE-9F0A-2FB96A26580C@iki.fi permalink / raw / eml / mbox
On 4.3.2012, at 15.31, Paul J Stevens wrote:

>>> The 3rd session selecting
>>> the mailbox and fetching the flags *before* one of the selected sessions
>>> fetch them seems perfectly valid to me.
>> 
>> I'm not entirely sure what you mean. Anyway, what the test is supposed to do is:
>> 
>> - 1 and 2 connections have mailbox selected
>> - 1 connection APPENDs a new message
>> - most servers assign \recent flag to connection 1 before APPEND returns tagged OK
> 
> That smells to me like an implementation dependent choice, not a
> protocol dictated requirement.

That is, yes, that's but:

>> - just in case some server might have assigned it to 2nd connection, the 2nd connection does a NOOP

These two combined is a requirement.

>> - now imaptest knows that the \recent must be in either connection 1 or connection 2 (both connections have informed the client with EXISTS that there are new messages, so the \recent flag must have been set)
> 
> I'm here to learn, but I don't see this in the specs.
> 
> Returning an updated EXISTS response in a session doesn't imply that
> later on the client will actually request the message flags.
> 
> I understand a \recent flag must never be returned more than once or in
> more than one session. But as I (re)read rfc3501 a \recent flag is not
> assigned to a session until a FETCH FLAGS response was returned.

I don't understand how you can read it that way:

           Message is "recently" arrived in this mailbox.  This session
           is the first session to have been notified about this
           message; if the session is read-write, subsequent sessions
           will not see \Recent set for this message. 

The \recent flag is assigned immediately when any IMAP read-write (=SELECTed) session is notified about a new message. The untagged RECENT reply should also be sent after the EXISTS reply when the message is \recent for that session.

> I get the feeling there are undocumented semantics in play here. Doing a
> STATUS doesn't affect the mailbox, fine. So even when a STATUS response
> returns a RECENT > 0, it seems perfectly valid to me if subsequence
> SELECT/FETCH on the same mailbox with the same session don't return any
> messages with the \recent flag.

If this happens, then between the STATUS RECENT>0 and SELECT there was another session that had SELECTed the mailbox.

> What the test seems to imply is that following transcript with two
> sessions A and B is incorrect.
> 
> A status mailbox (messages unseen recent)
> * status mailbox (messages 1 unseen 1 recent 1)
> A select mailbox
> * 1 exists
> * 1 recent
> ...
> A ok [read-write] select completed
> 
> B status mailbox (messages unseen recent)
> * status mailbox (messages 1 unseen 1 recent 1)

If B STATUS was sent after A SELECT then yes, it's incorrect.

> B select mailbox
> * 1 exists
> * 1 recent
> ...
> B ok [read-write] select completed
> B fetch 1 flags
> * 1 fetch (flags (\recent))
> B ok fetch completed
> 
> A fetch 1 flags
> * 1 fetch (flags ())
> A ok fetch completed

And this is definitely incorrect. The \recent flag exists only in session A. It's not visible in any other sessions.

Dovecot implements \recent handling mostly in memory. The only thing that needs to be stored about recent messages is first_recent_uid, which gets updated to UID+1 whenever UID's \recent flag is removed in some session.
Reply
E-mail headers
From: paul@nfg.nl
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:48 -0000
Message-ID: 4F53907D.2080404@nfg.nl permalink / raw / eml / mbox
On 03/04/2012 03:05 PM, Timo Sirainen wrote:

>> I understand a \recent flag must never be returned more than once or in
>> more than one session. But as I (re)read rfc3501 a \recent flag is not
>> assigned to a session until a FETCH FLAGS response was returned.
> 
> I don't understand how you can read it that way:
> 
>            Message is "recently" arrived in this mailbox.  This session
>            is the first session to have been notified about this
>            message; if the session is read-write, subsequent sessions
>            will not see \Recent set for this message. 

My bad. I (mis)understood above quote to apply to explicit flag
responses, i.e. the FETCH response.

After studying dovecot's behaviour, I conclude that *any* response for a
read-write opened mailbox must 'flush' the \recent flag on all messages
involved in the response.

I can live with that. At least it's consistent :-)

thanks again.



-- 
________________________________________________________________
Paul J Stevens        pjstevns @ gmail, twitter, skype, linkedin

  * Premium Hosting Services and Web Application Consultancy *

           www.nfg.nl/info@nfg.nl/+31.85.877.99.97
________________________________________________________________
Reply