mailing list archives

What's the correct syntax to do a SEARCH for an OR of three terms?
I've been using:

SEARCH OR OR HEADER Message-ID id1 HEADER Message-ID id2 HEADER Message-ID id3

RFC 3501 says:

       When multiple keys are specified, the result is the intersection
       (AND function) of all the messages that match those keys.  For
       example, the criteria DELETED FROM "SMITH" SINCE 1-Feb-1994 refers
       to all deleted messages from Smith that were placed in the mailbox
       since February 1, 1994.  A search key can also be a parenthesized
       list of one or more search keys (e.g., for use with the OR and NOT
       keys).

How would this work with OR?

SEARCH OR (HEADER Message-ID id1 HEADER Message-ID id2 HEADER Message-ID id3)

That doesn't match the BNF for OR.

SEARCH OR HEADER Message-ID id1 (HEADER Message-ID id2 HEADER Message-ID id3)

Are the multiple terms inside the parens combined using OR because they're
part of an OR key?  Or are they combined with AND because they're multiple
keys (first rule above)?  I assume the latter because there's no AND key
and otherwise there would be no way to express that case.


Assuming the first syntax at the top is correct, it appears that Exchange
2003 fails with more than two terms in OR.  Anyone know if that's a known
bug?  I can't find anything in the Microsoft knowledge base.

(A JavaMail customer reported this problem, I don't use Exchange myself.)

The Exchange 2003 IMAP connector's OR handling seems fine.

>  fetch 1:* rfc822.size
> * 1 FETCH (RFC822.SIZE 3940)
> * 2 FETCH (RFC822.SIZE 12133)
> * 3 FETCH (RFC822.SIZE 11243)
> * 4 FETCH (RFC822.SIZE 12210)
> * 5 FETCH (RFC822.SIZE 13601)
> * 6 FETCH (RFC822.SIZE 25480)
> * 7 FETCH (RFC822.SIZE 4631)
> * 8 FETCH (RFC822.SIZE 5712)
> * 9 FETCH (RFC822.SIZE 22364)
> * 10 FETCH (RFC822.SIZE 8505)
> * 11 FETCH (RFC822.SIZE 264403)
> * 12 FETCH (RFC822.SIZE 23832)
> * 13 FETCH (RFC822.SIZE 23422)
> * 14 FETCH (RFC822.SIZE 29670)
> * 15 FETCH (RFC822.SIZE 27351)
> c OK FETCH completed.
> d search smaller 4000
> * SEARCH 1
> d OK SEARCH completed.
> e search or smaller 4000 smaller 5000
> * SEARCH 1 7
> e OK SEARCH completed.
> f search or or smaller 4000 smaller 5000 smaller 9000
> * SEARCH 1 7 8 10
> f OK SEARCH completed.


Same test worked fine on Cyrus. I am typing in 3 OR terms according to  
the ABNF.

David
Synchronica Software



On Jan 29, 2009, at 10:56 AM, Bill Shannon wrote:

> What's the correct syntax to do a SEARCH for an OR of three terms?
> I've been using:
>
> SEARCH OR OR HEADER Message-ID id1 HEADER Message-ID id2 HEADER  
> Message-ID id3
>
> RFC 3501 says:
>
>      When multiple keys are specified, the result is the intersection
>      (AND function) of all the messages that match those keys.  For
>      example, the criteria DELETED FROM "SMITH" SINCE 1-Feb-1994  
> refers
>      to all deleted messages from Smith that were placed in the  
> mailbox
>      since February 1, 1994.  A search key can also be a parenthesized
>      list of one or more search keys (e.g., for use with the OR and  
> NOT
>      keys).
>
> How would this work with OR?
>
> SEARCH OR (HEADER Message-ID id1 HEADER Message-ID id2 HEADER  
> Message-ID id3)
>
> That doesn't match the BNF for OR.
>
> SEARCH OR HEADER Message-ID id1 (HEADER Message-ID id2 HEADER  
> Message-ID id3)
>
> Are the multiple terms inside the parens combined using OR because  
> they're
> part of an OR key?  Or are they combined with AND because they're  
> multiple
> keys (first rule above)?  I assume the latter because there's no AND  
> key
> and otherwise there would be no way to express that case.
>
>
> Assuming the first syntax at the top is correct, it appears that  
> Exchange
> 2003 fails with more than two terms in OR.  Anyone know if that's a  
> known
> bug?  I can't find anything in the Microsoft knowledge base.
>
> (A JavaMail customer reported this problem, I don't use Exchange  
> myself.)
>
>
> _______________________________________________
> Imap-protocol mailing list
> Imap-protocol@u.washington.edu
> http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20090129/861b300d/attachment.html>

On Thu, 29 Jan 2009, Bill Shannon wrote:
> What's the correct syntax to do a SEARCH for an OR of three terms?
> I've been using:
>
> SEARCH OR OR HEADER Message-ID id1 HEADER Message-ID id2 HEADER Message-ID id3

This results in a positive for any message with header "Message-ID" that 
contains either "id1", "id2", or "id3".  The order of evaluation isn't 
important, although I many servers would look for id1 first, then id2, 
then id3 (evaluating left-to-right and innermost first).

> SEARCH OR (HEADER Message-ID id1 HEADER Message-ID id2 HEADER Message-ID id3)

No.  That is a syntax error, since only one expression (the AND of the 
three HEADER searches) is passed to OR.

> SEARCH OR HEADER Message-ID id1 (HEADER Message-ID id2 HEADER Message-ID id3)

That will OR the id1 match with the AND of the id2/id3 searches.  That 
differs from

  SEARCH OR HEADER Message-ID id1 HEADER Message-ID id2 HEADER Message-ID id3

which will OR the id1 and ID2 search, then AND that with the id3 search.

This sounds like there is an evaluation order dependency, but there isn't. 
Criteria in IMAP are ANDed.  OR is a single criterion that takes two 
criteria as arguments.  Thus,
 	OR HEADER Message-ID id1 HEADER Message-ID id2
is a single criterion.

However, the human mind quickly boggles at the complexities, hence the 
provision of parentheses in the syntax.  Parentheses convert multiple 
(ANDed) criteria into a single criteria.  Consider:
 	SEARCH OR (FROM Bill TO Jane) (FROM Jane TO Bill)
which will match all messages that are between Bill and Jane directly, but 
not those from other individuals or those in which the other is merely a 
cc.

> Assuming the first syntax at the top is correct, it appears that Exchange
> 2003 fails with more than two terms in OR.  Anyone know if that's a known
> bug?  I can't find anything in the Microsoft knowledge base.

Are you sure that it's Exchange 2003?  AFAIK, 2003 was the last working 
version of Exchange and I recall it as working well.

Newer versions of Exchange certainly do suck in their implementation of 
SEARCH (and other aspects of IMAP).  Not coincidentally, the departure 
from the Exchange team of the IMAP-clueful guys at Microsoft was at about 
the same time...  :-(

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Bill Shannon writes:
> What's the correct syntax to do a SEARCH for an OR of three terms?
> I've been using:
>
> SEARCH OR OR HEADER Message-ID id1 HEADER Message-ID id2 HEADER 
> Message-ID id3

That's the right way.

> RFC 3501 says:
>
>       When multiple keys are specified, the result is the intersection
>       (AND function) of all the messages that match those keys.

It's trying to say "if you want OR, you have to say OR. If you want AND, 
you're in luck because that's default".

> How would this work with OR?
>
> SEARCH OR (HEADER Message-ID id1 HEADER Message-ID id2 HEADER Message-ID id3)

That one isn't syntactically valid since there's only one search term after OR.

(a and b) or c would be "OR (a b) c" or "OR c (a b)".

> Are the multiple terms inside the parens combined using OR because they're
> part of an OR key?

No.

> Or are they combined with AND because they're multiple
> keys (first rule above)?

Yes.

> Assuming the first syntax at the top is correct, it appears that Exchange
> 2003 fails with more than two terms in OR.

Can't have more than two terms in an OR. IF you want three, you have to 
have two ORs.

This command ORs together 14 search terms and ANDs the result of that 
with a single search term:

7pq uid search return () or or or from @spielzeughausen header 
"Resent-From" @spielzeughausen or header Sender @spielzeughausen header 
"Resent-Sender" @spielzeughausen or or or header "Return-Path" 
@spielzeughausen header "Reply-To" @spielzeughausen or To 
@spielzeughausen Cc @spielzeughausen or or header Bcc @spielzeughausen 
header "Resent-To" @spielzeughausen or header "Resent-Cc" 
@spielzeughausen header "Resent-Bcc" @spielzeughausen since 01-dec-2008

Arnt

Mark Crispin wrote:
> Are you sure that it's Exchange 2003?  AFAIK, 2003 was the last working 
> version of Exchange and I recall it as working well.

Yes, based on the trace from the customer, it's:

* OK Microsoft Exchange Server 2003 IMAP4rev1 server version 6.5.7638.1 
(myServer.org) ready.

Perhaps the bug is due to the particular terms or keys the customer is using.
He's used the same terms with success on other servers, so the problem seems
to be specific to Exchange.

Arnt Gulbrandsen wrote:
> Can't have more than two terms in an OR. IF you want three, you have to 
> have two ORs.

Right, that's what I meant.

On Thu, 29 Jan 2009, Bill Shannon wrote:
> Yes, based on the trace from the customer, it's:
> * OK Microsoft Exchange Server 2003 IMAP4rev1 server version 6.5.7638.1 (myServer.org) ready.

Bummer.

> Perhaps the bug is due to the particular terms or keys the customer is using.
> He's used the same terms with success on other servers, so the problem seems
> to be specific to Exchange.

Sadly, newer versions of Exchange are worse, and aren't the only offenders 
among broken IMAP server implementations.

The overall direction of the market seems to be not to use these 
fancy-schmancy features that "standard clients" (read: Outlook) doesn't 
use.  It gets quite depressing receiving bug reports for your client that 
feature the phrase "Outlook doesn't have this problem".

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Bill Shannon writes:
> Arnt Gulbrandsen wrote:
>> Can't have more than two terms in an OR. IF you want three, you have 
>> to have two ORs.
>
> Right, that's what I meant.

Sorry about the speechifying. Difficult to hit the right level of detail 
at the end the day when I'm tired.

Arnt