mailing list archives

The blog post 
http://googleonlinesecurity.blogspot.de/2014/04/new-security-measures-will-affect-older.html 
seems to have come to fruition as the "Access for less secure apps" 
setting as documented at 
https://support.google.com/accounts/answer/6010255?hl=en (but not 
particularly hyperlinked to/from many of the other GMail docs.)  It 
sounds like this started happening around July 15th, noting that 
2-factor accounts are not affected.

For the Firefox OS Gaia Email app we're trying to figure out exactly 
what the impact of this is and who is affected.  It does not seem 
straightforward because it seems like there are a number of heuristics 
in play.  Specifically, I have observed:

- My existing non-2-factor account seems to have been grandfathered so 
that the setting is enabled.

- I just created a brand new non-2-factor gmail account.  The Gmail 
Settings UI indicated IMAP was disabled and the "Access for less secure 
apps" account security setting was also disabled.  I then added the 
brand-new account in the app and things just magically worked.  IMAP got 
enabled in the gmail UI and "access for less secure apps" also got enabled.

I applaud both the effort to protect users and the use of whatever 
heuristics these are to avoid needlessly inflicting pain on users. 
However, it does leave me confused what users will be impacted.  Is it 
just GMail users over a certain account age who haven't leveraged PLAIN 
logins in some number of months?  Is it dependent on the suspicious 
login heuristics?  I do know that some testers have run into this 
problem recently, so it's not imagined.

So my questions are these:

1) Is it possible to get a better understanding of what's going on with 
when the setting will be enforced?

2) Is there some other venue for staying up-to-date with information 
like this for Gmail?  That blog post was somewhat nebulous, didn't get 
any coverage on a blog I was subscribed to at the time where I would 
have expected a mention (http://gmailblog.blogspot.com/), and I don't 
believe it or its contents were directly posted to any of these IMAP 
standardsy lists.  The July 15th thing seemed to be something people 
just inferred after it happened.

3) Is there some way I can help update documentation/hyperlinks on pages 
like https://developers.google.com/gmail/xoauth2_protocol (to link to 
the less secure apps docs)?  On 
https://support.google.com/accounts/answer/6010255?hl=en there is an 
affordance to say the article is not helpful and provide feedback, but I 
don't see anything on the developers site.


I do want to make it clear that I really appreciate Google/Brandon 
Long's active participation on this list and I understand how busy 
everyone involved likely is.  I'm also on board with the idea that, like 
web browsers, email apps/user agents should keep up with the state of 
the art standards for the benefit/safety/privacy of their users and the 
health of the net.  It's just that having more of an explicit heads up 
would help us make sure that we prioritize our engineering resources 
appropriately ahead of time rather than having to do things reactively.

Thanks!
Andrew

PS: The Gaia email app has also been deficient in notifying servers via 
"ID", if that's the venue I've been missing.  Although if so, I'd still 
argue this list or its friends would also be an appropriate place to post.

On Fri, Aug 29, 2014 at 8:50 AM, Andrew Sutherland <asuth@mozilla.com>
wrote:

>
> I applaud both the effort to protect users and the use of whatever
> heuristics these are to avoid needlessly inflicting pain on users. However,
> it does leave me confused what users will be impacted.  Is it just GMail
> users over a certain account age who haven't leveraged PLAIN logins in some
> number of months?  Is it dependent on the suspicious login heuristics?  I
> do know that some testers have run into this problem recently, so it's not
> imagined.
>

So, you have figured it out.  The state is actually tri-state, 'default',
'enabled', 'blocked'.  Any account which had used PLAIN logins in the last
N days prior to the launch was set to 'enabled'.  Any new account starts in
'default', which will moved to 'enabled' if a successful login occurs in
the first month of the account.   All other accounts were set to disabled.

We may also automatically disable it for any account which stops using
PLAIN logins for several months.

We didn't add a new error message since we just send the user to the
support page which now includes the "allow less secure apps" information.

Brandon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20140902/11347175/attachment.html>

I certainly don't want to treat this list like a Gmail specific list, so it
hasn't occurred to me to broadcast any of our changes to this list
specifically.

And we did reach out to our larger clients, assuming we knew who they were
and how to reach out to them.... sending us the ID command is certainly
helpful in that instance to know who are larger clients are.  We also had
some discussion with Thunderbird, but assumed our fallback was sufficient
for clients who couldn't really switch to OAUTH.  OAUTH2 is not ready for
Thunderbird at this point, really, since you have to pre-register to get
your client-id, and then also hard-code the various URLs for usage, as
there is no auto-discovery yet either.  Unfortunately, the bad guys aren't
waiting for the standards to be written.

I assume they didn't post that to the Gmail blog because its more important
to developers than to Gmail users at large.

Our abuse team is handling this, and so I'm not directly involved and don't
know the specifics to your questions.  I'll forward this to them to see
what we can say.

I'm not really sure why the developer docs would link to the "how to avoid
using this" support page... I guess I can suggest it.  I'm also not
thrilled with the wording on that page, "latest security standards".

Also, is the Gaia email app open source as well, ie is it something you can
reasonably use OAUTH2 with today?

Brandon


On Fri, Aug 29, 2014 at 8:50 AM, Andrew Sutherland <asuth@mozilla.com>
wrote:

> The blog post http://googleonlinesecurity.blogspot.de/2014/04/new-
> security-measures-will-affect-older.html seems to have come to fruition
> as the "Access for less secure apps" setting as documented at
> https://support.google.com/accounts/answer/6010255?hl=en (but not
> particularly hyperlinked to/from many of the other GMail docs.)  It sounds
> like this started happening around July 15th, noting that 2-factor accounts
> are not affected.
>
> For the Firefox OS Gaia Email app we're trying to figure out exactly what
> the impact of this is and who is affected.  It does not seem
> straightforward because it seems like there are a number of heuristics in
> play.  Specifically, I have observed:
>
> - My existing non-2-factor account seems to have been grandfathered so
> that the setting is enabled.
>
> - I just created a brand new non-2-factor gmail account.  The Gmail
> Settings UI indicated IMAP was disabled and the "Access for less secure
> apps" account security setting was also disabled.  I then added the
> brand-new account in the app and things just magically worked.  IMAP got
> enabled in the gmail UI and "access for less secure apps" also got enabled.
>
> I applaud both the effort to protect users and the use of whatever
> heuristics these are to avoid needlessly inflicting pain on users. However,
> it does leave me confused what users will be impacted.  Is it just GMail
> users over a certain account age who haven't leveraged PLAIN logins in some
> number of months?  Is it dependent on the suspicious login heuristics?  I
> do know that some testers have run into this problem recently, so it's not
> imagined.
>
> So my questions are these:
>
> 1) Is it possible to get a better understanding of what's going on with
> when the setting will be enforced?
>
> 2) Is there some other venue for staying up-to-date with information like
> this for Gmail?  That blog post was somewhat nebulous, didn't get any
> coverage on a blog I was subscribed to at the time where I would have
> expected a mention (http://gmailblog.blogspot.com/), and I don't believe
> it or its contents were directly posted to any of these IMAP standardsy
> lists.  The July 15th thing seemed to be something people just inferred
> after it happened.
>
> 3) Is there some way I can help update documentation/hyperlinks on pages
> like https://developers.google.com/gmail/xoauth2_protocol (to link to the
> less secure apps docs)?  On https://support.google.com/
> accounts/answer/6010255?hl=en there is an affordance to say the article
> is not helpful and provide feedback, but I don't see anything on the
> developers site.
>
>
> I do want to make it clear that I really appreciate Google/Brandon Long's
> active participation on this list and I understand how busy everyone
> involved likely is.  I'm also on board with the idea that, like web
> browsers, email apps/user agents should keep up with the state of the art
> standards for the benefit/safety/privacy of their users and the health of
> the net.  It's just that having more of an explicit heads up would help us
> make sure that we prioritize our engineering resources appropriately ahead
> of time rather than having to do things reactively.
>
> Thanks!
> Andrew
>
> PS: The Gaia email app has also been deficient in notifying servers via
> "ID", if that's the venue I've been missing.  Although if so, I'd still
> argue this list or its friends would also be an appropriate place to post.
> _______________________________________________
> Imap-protocol mailing list
> Imap-protocol@u.washington.edu
> http://mailman13.u.washington.edu/mailman/listinfo/imap-protocol
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20140829/44573899/attachment.html>

On 8/29/2014 3:53 PM, Brandon Long wrote:
> And we did reach out to our larger clients, assuming we knew who they 
> were and how to reach out to them.... sending us the ID command is 
> certainly helpful in that instance to know who are larger clients are. 
>  We also had some discussion with Thunderbird, but assumed our 
> fallback was sufficient for clients who couldn't really switch to 
> OAUTH.  OAUTH2 is not ready for Thunderbird at this point, really, 
> since you have to pre-register to get your client-id, and then also 
> hard-code the various URLs for usage, as there is no auto-discovery 
> yet either.  Unfortunately, the bad guys aren't waiting for the 
> standards to be written.

Since this is definitely of interest to others on the list--are there 
any updates on the dynamic client registration and autodiscovery drafts 
yet? I've seen progress on the Dynamic client registration draft 
(http://tools.ietf.org/html/draft-ietf-oauth-dyn-reg-20), although I 
admit I haven't read the spec closely yet to provide feedback; but I've 
been unable to find indication of any autodiscovery work.

-- 
Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald E. Knuth

Brandon Long wrote:
> I'm not really sure why the developer docs would link to the "how to
> avoid using this" support page... I guess I can suggest it.  I'm also
> not thrilled with the wording on that page, "latest security
> standards".

That's definitely confusing. Just talking about "less secure apps"
without specifying why do you consider them "less secure" is useless.
When I first read that page I thought it was refering to using a weak
ciphersuite in  TLS (for some definition of "weak").

(I did report it on the feedback form, but the page keeps the same
-other that now there is the ?Allow less secure apps? option- )

You need to relate [1] with [2] in order to understand what it is
talking about
1-https://support.google.com/accounts/answer/6010255?hl=en
2-http://googleonlinesecurity.blogspot.de/2014/04/new-security-measures-will-affect-older.html



> 
> OAUTH2 is not ready for Thunderbird at this point, really, since you
> have to pre-register to get your client-id, and then also hard-code
> the various URLs for usage, as there is no auto-discovery yet either.
> Unfortunately, the bad guys aren't waiting for the standards to be
> written. Our abuse team is handling this, and so I'm not directly
> involved and don't know the specifics (...)
> 
I wonder what/how they are trying to fix. An application-specific
password can provide the same benefits as OAUTH2* -except for the
automatic refresh when it expires after 6 months- yet they are only
available with two-factor authentication.
* Although with a little more work from the user generating them. For
feature parity, instead of the program handing an identifying token, the
user would need to follow a Wizard and choose the MUA they are using
(with 'other' being unrestricted).

On 08/29/2014 04:53 PM, Brandon Long wrote:
> I certainly don't want to treat this list like a Gmail specific list, 
> so it hasn't occurred to me to broadcast any of our changes to this 
> list specifically.
>
> And we did reach out to our larger clients, assuming we knew who they 
> were and how to reach out to them.... sending us the ID command is 
> certainly helpful in that instance to know who are larger clients are.

It doesn't need to be this list specifically.  I'm fine with joining a 
Gmail IMAP specific list or subscribing to a specific blog feed. It's 
even fine if it's a read-only announcement list or blog with comments 
turned off.  Gmail is a significant player in the mail server space and 
I don't think you'll hear a lot of complaints from people in the IMAP 
space being kept too up-to-date.  (Unless the 'status page' feed for 
Gmail gets forwarded!)


> Unfortunately, the bad guys aren't waiting for the standards to be 
> written.

I want to reiterate that I understand this and strongly agree with 
this.  Compromise of a user's email account is an extremely serious 
problem since not only is it used for private communication, it's also 
the gateway to compromise of the user's other accounts and a means of 
performing subsequent social engineering attacks on everyone they know 
or could potentially know.

My request is not that Google/Gmail slow down their protection of users, 
just more provide more updates with ideally a little more detail.  If 
the announcement has to come after the change or the details need to be 
a little nebulous, that's also okay.  Just knowing that the automated 
abuse-detection features are involved is useful, and hopefully doesn't 
provide would-be attackers with information they hadn't already figured out.

For example, we already explicitly detect the Gmail specific "NO 
[ALERT]" cases of things like the following:
- Application-specific password required
- Your account is not enabled for IMAP use.
- IMAP access is disabled for your domain.
and want to better handle:
- Please log in via your web browser: 
http://support.google.com/mail/accounts/bin/answer.py?answer=78754 (Failure)

I believe this new failure adds another specialized error code (and 
regrettably, we screw up here, mea culpa.)

Knowing what the string would be, whether it's localized, etc. would be 
handy.  We do these detections to try and provide appropriately 
localized error messages that might provide better context to the user 
and differentiate between persistent failures and transient failures 
(ex: NO [UNAVAILABLE]), although I should be clear these are 
insufficiently researched hacks.  I don't know if these strings are 
actually localized based on the user's locale or if they're always 
English and the URL is expected to do the localization fix-ups.

It's definitely clear we did the wrong thing for gmail, and maybe in 
general for ALERT, because even if there are other servers that might 
use ALERT in ways that are annoying to users if done naively, it's 
clearly not the case for gmail.  At the bare minimum we will make sure 
to detect and expose URLs in the errors in the future.


> Our abuse team is handling this, and so I'm not directly involved and 
> don't know the specifics to your questions.  I'll forward this to them 
> to see what we can say.

Thank you very much!


> I'm not really sure why the developer docs would link to the "how to 
> avoid using this" support page... I guess I can suggest it.

Apologies, I think I meant to link the page 
https://developers.google.com/gmail/oauth_overview instead of its 
child.  (Its heading in the sidebar is "IMAP and SMTP" but the page is 
"oauth_overview, which is how I got confused amongst my too many tabs.)

I don't mean this in a "how to avoid using this" context.  Mainly, I 
think it would be handy if it conveyed something like:

- If you don't use OAuth2, we will, to protect our users who are also 
your users, require users not using 2-factor authentication to 
explicitly enable support for plaintext logins via the Google Accounts 
Settings page.  Authentication will fail with a NO [ALERT] error that 
links to the support page 
https://support.google.com/accounts/answer/6010255?hl=en.

The goal is that if a client developer has a user who runs up against 
this problem, they can find some documentation on what's going on.  And 
if they're a new developer, they can be aware of the ramifications of 
cutting corners with security.  Importantly, they can do that during 
their planning stage too, not just after they implement it.

(And likewise on the support page side, linking to the XOAuth2 docs / 
elaborating on what makes an app "less secure" and how they can address 
it is also handy, just in that on the web it's not that hard to support 
statements that might otherwise be interpreted as FUD. I've already used 
the feedback mechanism to relay that, however.)


> Also, is the Gaia email app open source as well, ie is it something 
> you can reasonably use OAUTH2 with today?

Yes, the Gaia email app is open source[1].  And we are absolutely going 
to implement this on trunk/master over the next few weeks.  It was on 
our to-do list before the blog post, its priority went way up after the 
blog post, but unfortunately I made the wrong inferences from reading 
the blog post and so its relative priority still ended up beneath 
addressing some crasher blockers and other technical debt fires.

My concern and interest in how this affects our already shipped apps is 
because we are in the embarassing position where we are not currently 
able to to easily update the core Firefox OS apps that ship on the 
devices until a full Firefox OS upgrade version is released to phones 
via an over-the-air update.  Having more information lets us know how 
much of a problem this will pose to users of shipped devices and how 
much support can help affected users versus how hard we need to push on 
getting mitigating patches and/or smaller over-the-air updates rolled out.

As always, thanks for the quick response!
Andrew


1: And as of the last day or two our IMAP and SMTP protocols are now 
built on whiteout-io's open source mail library underpinnings as 
documented at http://emailjs.org/.  Woo!  (AKA a bunch of technical debt 
has now been addressed! :)

Hi,

With Gmail is XOAUTH2 the only login method that is not considered "less 
secure"?

For some reason I got the impression that AUTHENTICATE PLAIN was not 
considered "less secure".

Thanks
-Rick


-- 
Rick Sanders
rfs9999@earthlink.net
IMAP Tools  http://www.athensfbc.com/imap-tools

This makes me sad.

On Saturday, August 30, 2014 2:43:16 AM CEST, Andrew Sutherland wrote:
> For example, we already explicitly detect the Gmail specific 
> "NO [ALERT]" cases of things like the following:
> - Application-specific password required
> - Your account is not enabled for IMAP use.
> - IMAP access is disabled for your domain.

As if 5530 didn't exist.

> and want to better handle:
> - Please log in via your web browser: 
> http://support.google.com/mail/accounts/bin/answer.py?answer=78754 
> (Failure)

There is a response code for that, WEBALERT. Not standard, but at least 
it's better than trying to have a client parsing human-readable text.

> I believe this new failure adds another specialized error code 
> (and regrettably, we screw up here, mea culpa.)
>
> Knowing what the string would be, whether it's localized, etc. 
> would be handy.  We do these detections to try and provide 
> appropriately localized error messages that might provide better 
> context to the user and differentiate between persistent 
> failures and transient failures (ex: NO [UNAVAILABLE]), although 
> I should be clear these are insufficiently researched hacks.

The point of 5530 and that registry was that clients should not need to do 
such things. IMO Google screws up by not defininig response codes, not you.

Arnt

Anything that uses the user's password is generally considered 'less
secure'.

Basically, with the high prevalence of password reuse and
compromise/exfiltration/phishing/malware/etc, passwords are no longer a
sufficient method of proving account ownership.  On the web, with a Turing
machine available to us and a number of signals and the fact that the user
is actually sitting physical in front of a computer, we can mostly ensure
auth, but for IMAP which may be from a service or proxy and the prevalence
of smart phones which both travel and are often NAT'd across the country,
things are much more complicated.

So, yes, please use xoauth2 or the oauth-bearer when its available (we're
just waiting on the rfc to be published at this point).

And as good a time as any to remind folks that xoauth has been deprecated
for a while now and will cease to work next year.  Migrate your users now.

XOAuth2 should be supported as long as oauth-bearer since its has only
minor differences being based on an earlier draft, the tokens are all the
same.

Brandon
On Sep 26, 2014 5:36 AM, "Rick Sanders" <rfs9999@earthlink.net> wrote:

> Hi,
>
> With Gmail is XOAUTH2 the only login method that is not considered "less
> secure"?
>
> For some reason I got the impression that AUTHENTICATE PLAIN was not
> considered "less secure".
>
> Thanks
> -Rick
>
>
> --
> Rick Sanders
> rfs9999@earthlink.net
> IMAP Tools  http://www.athensfbc.com/imap-tools
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20140926/d887bfe2/attachment.html>

On 08/30/2014 05:02 PM, Arnt Gulbrandsen wrote:
>> and want to better handle:
>> - Please log in via your web browser: 
>> http://support.google.com/mail/accounts/bin/answer.py?answer=78754 
>> (Failure)
>
> There is a response code for that, WEBALERT. Not standard, but at 
> least it's better than trying to have a client parsing human-readable 
> text.

It appears Gmail may actually be doing this, which is great news! In a 
tangential bug of ours, a reporter using openssl s_client -connect gave 
us the following sanitized excerpt:

* NO [WEBALERT ***] Web login required.
a001 NO [ALERT] Please log in via your web browser: 
http://support.google.com/mail/accounts/bin/answer.py?answer=78754 (Failure)

A similar full example with URL seems to be provided at 
https://productforums.google.com/forum/#!topic/gmail/BYxYSdThpiw but I'm 
having trouble finding a lot of other examples/details.


It's very likely we missed this since our error handling / logging was 
only reporting the tagged response.  We try and avoid seeking full 
protocol traces/maximum debug logging from users because of privacy 
concerns.  Unfortunately, it does make it harder to learn about 
exceptional situations like this that aren't particularly documented and 
require state that is outside our control to reproduce.


Arnt, thank you very much for putting us on the right track, here! Do 
you know if we should expect WEBALERT to always be untagged, and/or how 
other servers (and what other servers) might use it?  A quick skim of 
open source clients (coremail2, android email) and servers (dovecot, 
cyrus) isn't enlightening me.


I'd like to archive this information in a more publicly available 
fashion for people.  I think http://imapwiki.org/ is probably the best 
resource I know of (other than actual standards :), but I'm open to 
other suggestions!

Thanks!
Andrew

On Sat, Aug 30, 2014 at 2:02 PM, Arnt Gulbrandsen <arnt@gulbrandsen.priv.no>
wrote:

> This makes me sad.
>
> On Saturday, August 30, 2014 2:43:16 AM CEST, Andrew Sutherland wrote:
>
>> For example, we already explicitly detect the Gmail specific "NO [ALERT]"
>> cases of things like the following:
>> - Application-specific password required
>> - Your account is not enabled for IMAP use.
>> - IMAP access is disabled for your domain.
>>
>
> As if 5530 didn't exist.


It didn't when we wrote the server.  Even now that it does, and I believe
I've mentioned this before, the 5530 codes are inherently non-backward
compatible.  We have no way of knowing if a client handles a given code,
and we originally went with ALERT so that people could actually see the
error.

Now, it turns out most clients ignore ALERT and don't show it to the user,
regardless of the spec.  We did go through and update most of the login
errors (and others) to correspond to the 5530 codes where there was a
match.  We could also use separate tagged/untagged responses to give both
an ALERT and a specific code, but that's obviously something of a hack and
its not clear most clients handle that either.


>  and want to better handle:
>> - Please log in via your web browser: http://support.google.com/
>> mail/accounts/bin/answer.py?answer=78754 (Failure)
>>
>
> There is a response code for that, WEBALERT. Not standard, but at least
> it's better than trying to have a client parsing human-readable text.


Yes, if we have a URL to send, we issue a WEBALERT.  There was some debate
internally about whether or not to re-use the 5530 syntax for our own
specific codes.


>  I believe this new failure adds another specialized error code (and
>> regrettably, we screw up here, mea culpa.)
>>
>> Knowing what the string would be, whether it's localized, etc. would be
>> handy.  We do these detections to try and provide appropriately localized
>> error messages that might provide better context to the user and
>> differentiate between persistent failures and transient failures (ex: NO
>> [UNAVAILABLE]), although I should be clear these are insufficiently
>> researched hacks.
>>
>
> The point of 5530 and that registry was that clients should not need to do
> such things. IMO Google screws up by not defininig response codes, not you.


Extending the available codes via RFC seems heavy weight, especially if its
for things that are more specific to our server.  Maybe a list of "generic
but more specific" ones could be made, but would we really have one for
"Your domain administrator has disabled IMAP for your domain from this IP"?

More examples that we send at this point (most of these also have further
information, this is a shortened version):
- [AUTHENTICATIONFAILED] Invalid credentials
- [ALERT] Your account is not ready for IMAP use. (this was historical and
unlikely to be sent to anyone anymore)
- [ALERT] Your account is not enabled for IMAP use.
- [ALERT] IMAP access is disabled for your domain.
- [ALERT] IMAP access to your domain is not allowed from this IP
- [ALERT] Account exceeded command or bandwidth limits.
- [ALERT] Too many simultaneous connections.
- [ALERT] Application-specific password required:
- [ALERT] Please log in via your web browser:
- [ALERT] Web login required:

As for the WEBALERT case, its expected that the correct thing to do is to
send the user to the url specified, but it has to be the user, they're
likely to have to authenticate the session.

Half of the above would be UNAVAILABLE or maybe CONTACTADMIN, some would be
LIMIT/OVERQUOTA... but its unclear how much "clearer" these would be if
they're translated to the user's language via a "generic" category.

We have debated having a per-message unique identifier, also useful when
people contact us about which error they're seeing, but haven't exposed it.
 I'd say we could translate these with our upcoming support for UTF8, but
these are almost assuredly seen prior to any ENABLE that would help.

Brandon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20140902/525d3e20/attachment.html>

>please use xoauth2 or the oauth-bearer when its available

Thanks, Brandon.

-Rick


-- 
Rick Sanders
rfs9999@earthlink.net
IMAP Tools  http://www.athensfbc.com/imap-tools

On Tuesday, September 2, 2014 10:19:45 PM CEST, Andrew Sutherland wrote:
> It appears Gmail may actually be doing this, which is great news!

WEBALERT is something gmail, and only gmail, does. I don't know what it 
means, but it does mean something other than the completely general ALERT. 
IMO WEBALERT is more usable from a client's point of view.

What an unpleasant situation.

Arnt

On Tuesday, September 2, 2014 11:09:47 PM CEST, Brandon Long wrote:
> It didn't when we wrote the server.  Even now that it does, and 
> I believe I've mentioned this before, the 5530 codes are 
> inherently non-backward compatible.

That's not a compatbility break, for two reasons.

One is that you know very, very little about what the client acts on, so 
5530 is business as usual. For example, I can't at the moment think of any 
GUI client that obeys the RECENT-related MUST rule in 3501, so sending the 
RECENT response is a bit hit and miss.

The other is that there was nothing there to really be compatible with. 
5530 is based on client and server surveys. Take NOPERM for example: 
Various servers tested for that, and all reported failure as "NO some 
human-readable text here", and I could not find any client that used the 
human-readable text as basis for any behaviour.

Or take OVERQUOTA. IIRC I found a client that did "if the response is NO 
and the human-readable text contains the string quot". I have a problem 
with talking about something is compatible with such fuzzily unclear fuzz.

> We have no way of knowing 
> if a client handles a given code, and we originally went with 
> ALERT so that people could actually see the error.
> 
> Now, it turns out most clients ignore ALERT and don't show it 
> to the user, regardless of the spec.

Exactly. I knew that and thought most people did. I understand now that I 
should have added a note to 5530 pointing it out. Sorry. I didn't, because 
5530 is about server to client communications, and ALERT is intended for 
when the server wants to bypass the client's state machine and speak to the 
user.

> We did go through and 
> update most of the login errors (and others) to correspond to 
> the 5530 codes where there was a match.

I've noticed. Good work.

> We could also use 
> separate tagged/untagged responses to give both an ALERT and a 
> specific code, but that's obviously something of a hack and its 
> not clear most clients handle that either.

My two cents: ALERT is a bit pointless, but if you want to send ALERT, send 
that in an untagged response of its own. ALERT is intended for the user, 
and the user has no use for the command tag. Put client-directed response 
codes in the tagged response, because most clients know their tags and tie 
the tagged response tightly to the command.

> Extending the available codes via RFC seems heavy weight, 
> especially if its for things that are more specific to our 
> server.  Maybe a list of "generic but more specific" ones could 
> be made, but would we really have one for "Your domain 
> administrator has disabled IMAP for your domain from this IP"?

Not quite that detailed, but something along those lines, yes, I think so. 
The servers I based 5530 on didn't have a difference between the gmail god 
and the customer's domain administrator and no ability to create problem 
tickets, otherwise I expect CONTACTADMIN would have been a bit richer. 
Perhaps: [PROBLEM issue-id contact-url], with the URL indicating at least 
whom to contact, and the issue-id being something that enables the 
contacted party to look up the problem.

> We have debated having a per-message unique identifier, also 
> useful when people contact us about which error they're seeing, 
> but haven't exposed it.  I'd say we could translate these with 
> our upcoming support for UTF8, but these are almost assuredly 
> seen prior to any ENABLE that would help.

I disagree...

You're talking about a server that issues UTF8=ACCEPT in CAPABILITY when 
asked, and how it behaves in some exceptional problem. If there's a 
problem, then I venture to suggest that there's a very slim chance that a) 
the client parses the hr-text at all and b) would be able to recover from 
the problem but c) is not able to recover because the hr-text contains 
UTF8.

Sending UTF8 if there isn't a problem sounds riskier. That's a common 
cazse, not exceptional, and there's more to lose.

Arnt

El mar, 02-09-2014 a las 14:09 -0700, Brandon Long escribi?:
> 
> 
> 
On Sat, Aug 30, 2014 at 2:02 PM, Arnt Gulbrandsen:
>         The point of 5530 and that registry was that clients should
>         not need to do such things. IMO Google screws up by not
>         defininig response codes, not you.
>
> 
> Extending the available codes via RFC seems heavy weight, especially
> if its for things that are more specific to our server.  Maybe a list
> of "generic but more specific" ones could be made, but would we really
> have one for "Your domain administrator has disabled IMAP for your
> domain from this IP"?

Instead of a RFC listing each error, that's more suited for creating a
registry of error codes. Although the idea of using a wiki seems better
than placing it at IANA.



> 
> As for the WEBALERT case, its expected that the correct thing to do is
> to send the user to the url specified, but it has to be the user,
> they're likely to have to authenticate the session.
> 
> 
> Half of the above would be UNAVAILABLE or maybe CONTACTADMIN, some
> would be LIMIT/OVERQUOTA... but its unclear how much "clearer" these
> would be if they're translated to the user's language via a "generic"
> category.
> 
> 
> We have debated having a per-message unique identifier, also useful
> when people contact us about which error they're seeing, but haven't
> exposed it.  I'd say we could translate these with our upcoming
> support for UTF8, 

I'm not too keen about providing translated messages unrequested.


> but these are almost assuredly seen prior to any ENABLE that would
> help.

Specially given that you can only use ENABLE in the authenticated state.

<random ideas>
It may be worth taking into account SMAP 'LANG' capability announcing if
creating an extension for handling that. A extension for supporting
translated messages might be:

The server exposes the capability by announcing a series of
LANG=<langtag> tokens on its CAPABILITY. <langtag> is a language tag as
defined by RFC 1766. Providing just LANG means the server supports some
translations but is not providing the full list.

The client requests internationalization of the messages with <tag> LANG
<langcode>. An untagged response is provided with the actual language
chosen by the server (eg. the user requested EN-COCKNEY but the server
provided EN-GB)

'<tag> LANG ?' could be used for requesting a list of languages
supported by the server.

And '<tag> LANG' for declaring support for the extension but stating no
preference. The server may choose a language based on an account
setting, a default for its domain, ip geolocation, etc. Note that the
server may decide to send an untagged LANG after login.


1- http://www.courier-mta.org/cone/smap1.html

On Wed, Sep 3, 2014 at 1:55 AM, Arnt Gulbrandsen <arnt@gulbrandsen.priv.no>
wrote:

> On Tuesday, September 2, 2014 10:19:45 PM CEST, Andrew Sutherland wrote:
>
>> It appears Gmail may actually be doing this, which is great news!
>>
>
> WEBALERT is something gmail, and only gmail, does. I don't know what it
> means, but it does mean something other than the completely general ALERT.
> IMO WEBALERT is more usable from a client's point of view.
>
> What an unpleasant situation.


Its basically an ALERT which directs the client to a web page.  Although
we've had free-form URLs in the error messages for years, this mechanism
makes it easier to actually parse the URL out.  Its currently only used for
login errors related to our permission system, and was suggested by one of
our client partners.

Its assumed the client would redirect the user to that webpage, ie HTTP 302
style.  One can imagine it being similar to implementing OAUTHBEARER where
the cilent would need an embedded webview to perform the oauth
authentication.  It doesn't require an embedded webview like OAUTH, since
its not expected that the client needs to scrape information out of the
webview.

Brandon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20140903/f1784a9a/attachment.html>

On Wed, Sep 3, 2014 at 4:17 AM, Arnt Gulbrandsen <arnt@gulbrandsen.priv.no>
wrote:

> On Tuesday, September 2, 2014 11:09:47 PM CEST, Brandon Long wrote:
>
>> It didn't when we wrote the server.  Even now that it does, and I believe
>> I've mentioned this before, the 5530 codes are inherently non-backward
>> compatible.
>>
>
> That's not a compatbility break, for two reasons.
>
> One is that you know very, very little about what the client acts on, so
> 5530 is business as usual. For example, I can't at the moment think of any
> GUI client that obeys the RECENT-related MUST rule in 3501, so sending the
> RECENT response is a bit hit and miss.
>
> The other is that there was nothing there to really be compatible with.
> 5530 is based on client and server surveys. Take NOPERM for example:
> Various servers tested for that, and all reported failure as "NO some
> human-readable text here", and I could not find any client that used the
> human-readable text as basis for any behaviour.
>
> Or take OVERQUOTA. IIRC I found a client that did "if the response is NO
> and the human-readable text contains the string quot". I have a problem
> with talking about something is compatible with such fuzzily unclear fuzz.


Sure, I was being specific to [ALERT] meaning "show the user", not any
fuzzy logic that people were using on the strings themselves.

[snip]

 We could also use separate tagged/untagged responses to give both an ALERT
>> and a specific code, but that's obviously something of a hack and its not
>> clear most clients handle that either.
>>
>
> My two cents: ALERT is a bit pointless, but if you want to send ALERT,
> send that in an untagged response of its own. ALERT is intended for the
> user, and the user has no use for the command tag. Put client-directed
> response codes in the tagged response, because most clients know their tags
> and tie the tagged response tightly to the command.


At this point, I'm willing to concede that point.

[snip]


> You're talking about a server that issues UTF8=ACCEPT in CAPABILITY when
> asked, and how it behaves in some exceptional problem. If there's a
> problem, then I venture to suggest that there's a very slim chance that a)
> the client parses the hr-text at all and b) would be able to recover from
> the problem but c) is not able to recover because the hr-text contains UTF8.
>
> Sending UTF8 if there isn't a problem sounds riskier. That's a common
> cazse, not exceptional, and there's more to lose.


Funny, we just got an external bug report today due to us sending UTF8 in
the OK response to logins for user's with UTF8 names (which we'll be
fixing).

It is interesting to violate in case of errors.  I assume that will make
things worse for people who are actually inspecting our strings, though.

Brandon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20140903/eca45780/attachment.html>

FWIW, there already is a language extension for IMAP. My name is on the 
RFC, even. I think Ned Freed has an implementation, but I haven't exactly 
been swamped by comments from would-be implementers.

I didn't implement it myself. To be frank I only finished the RFC because 
it was the easiest way out. Sometimes "ship and discard" is easier than any 
other alternative.

Arnt

Arnt Gulbrandsen wrote:
> FWIW, there already is a language extension for IMAP. My name is on the 
> RFC, even. I think Ned Freed has an implementation, but I haven't exactly 
> been swamped by comments from would-be implementers.
> 
> I didn't implement it myself. To be frank I only finished the RFC because 
> it was the easiest way out. Sometimes "ship and discard" is easier than any 
> other alternative.
> 
> Arnt

My bad. The proper thing would then be to use rfc 5255, of course.

Except that 5255 doesn't solve the problem at hand, it just changes it from 
"what to do before ENABLE" into "what to do before LANGUAGE".

I did that a few weeks ago: Spent several days writing code only to 
discover that I had solved one problem and now had another, exactly like 
the first ;)

Arnt

Arnt Gulbrandsen wrote:
> Except that 5255 doesn't solve the problem at hand, it just changes it from 
> "what to do before ENABLE" into "what to do before LANGUAGE".
> 
> I did that a few weeks ago: Spent several days writing code only to 
> discover that I had solved one problem and now had another, exactly like 
> the first ;)
> 
> Arnt

What do you mean? The client should send LANGUAGE as the first command
(that is, if they want translated messages) so the only problem would be
if the server greeting is a BYE (IP banned or exhausted resources).
The best way seems to attempt mapping them to rfc5530 codes. The second
is a clear UNAVAILABLE, while for the former there are several not
exactly matching ones: NOPERM, AUTHENTICATIONFAILED, PRIVACYREQUIRED.

Regards

The problem is: What it do if there's a problem, the description includes 
UTF8, and the client hasn't indicated via ENABLE that it can handle UTF in 
human-readable text.

The revised problem is: What it do if there's a problem, the description 
includes UTF8, and the client hasn't indicated via LANGUAGE that it can 
handle UTF in human-readable text.

Not much difference.

Arnt

Arnt Gulbrandsen wrote:
> The problem is: What it do if there's a problem, the description includes 
> UTF8, and the client hasn't indicated via ENABLE that it can handle UTF in 
> human-readable text.
> 
> The revised problem is: What it do if there's a problem, the description 
> includes UTF8, and the client hasn't indicated via LANGUAGE that it can 
> handle UTF in human-readable text.
> 
> Not much difference.
> 
> Arnt

You would have several sets of messages for the different languages.
Amongst them you include one that is used as default (if no language is
sent) and doesn't contain utf-8 characters. Just like LANG=C is limited
to ascii.