mailing list archives

meli community discussions

⚠️ if something does not work as intended when interracting with the mailing lists,
reach out Github mirror Gitea repo @epilys:matrix.org

E-mail headers
From: Timo Sirainen <tss@iki.fi>
To: imap-protocol@u.washington.edu
Date: Fri, 08 Jun 2018 12:34:51 -0000
Message-ID: FF6E4621-3C20-4C20-AE79-9E62CC9322CC@iki.fi permalink / raw / eml / mbox
I started wondering today if it would be a good idea to bring back the login referrals, only a little bit changed from their original behavior. I guess originally they didn't really get used because few clients supported them and proxying works for everyone. But nowadays many larger installations have IMAP proxies and backends in multiple data centers and even though everything works, doing unnecessary proxying between data centers increases latency and often also costs money. Using DNS to give the correct data center IP based on the source IP might work for some users, but that's somewhat difficult to set up and likely won't work very well for the increasing number of mobile users. So what if we got the most popular clients to implement support for login referrals and use them?

Problem 1: LOGIN tagged OK [REFERRAL] reply conflicts between Lemonade-required OK [CAPABILITY] reply.

So either one of them needs to be moved to an untagged OK reply. RFC 2221 only talks about tagged OK reply, while RFC 5550 makes it possible to use untagged OK reply. But I'm a bit worried about changing this now, there may well be clients that handle only the tagged OK [CAPABILITY] while there are almost no clients that handle [REFERRAL] anywhere. So my preferred server reply would be:

* OK [REFERRAL imap://user;AUTH=*;us1.imap.example.com/] Use this server for decreased latency
a OK [CAPABILITY ...] Logged in.

Problem 2: RFC 2221 specified OK [REFERRAL] to mean that user's mailboxes aren't on this server, but some shared mailboxes are.

Instead now the OK [REFERRAL] should simply be a suggestion to the client to connect to the specified server to get better latency, but it would work just as well to just ignore it and keep using the current server. In fact the client likely shouldn't reconnect at all, but just create all future connections using the referral's server. If the referral server ever fails, the client should connect back to the original server.

Does anyone else think this would be worth maybe writing a new updated RFC2221bis and trying to convince client developers to implement it?
Reply
E-mail headers
From: brong@fastmail.fm
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:51 -0000
Message-ID: 1389735787.25987.70781289.30514553@webmail.messagingengine.com permalink / raw / eml / mbox
On Wed, Jan 15, 2014, at 08:32 AM, Timo Sirainen wrote:
> I started wondering today if it would be a good idea to bring back the login referrals, only a little bit changed from their original behavior. I guess originally they didn't really get used because few clients supported them and proxying works for everyone. But nowadays many larger installations have IMAP proxies and backends in multiple data centers and even though everything works, doing unnecessary proxying between data centers increases latency and often also costs money. Using DNS to give the correct data center IP based on the source IP might work for some users, but that's somewhat difficult to set up and likely won't work very well for the increasing number of mobile users. So what if we got the most popular clients to implement support for login referrals and use them?
> 
> Problem 1: LOGIN tagged OK [REFERRAL] reply conflicts between Lemonade-required OK [CAPABILITY] reply.
> 
> So either one of them needs to be moved to an untagged OK reply. RFC 2221 only talks about tagged OK reply, while RFC 5550 makes it possible to use untagged OK reply. But I'm a bit worried about changing this now, there may well be clients that handle only the tagged OK [CAPABILITY] while there are almost no clients that handle [REFERRAL] anywhere. So my preferred server reply would be:
> 
> * OK [REFERRAL imap://user;AUTH=*;us1.imap.example.com/] Use this server for decreased latency
> a OK [CAPABILITY ...] Logged in.
> 
> Problem 2: RFC 2221 specified OK [REFERRAL] to mean that user's mailboxes aren't on this server, but some shared mailboxes are.
> 
> Instead now the OK [REFERRAL] should simply be a suggestion to the client to connect to the specified server to get better latency, but it would work just as well to just ignore it and keep using the current server. In fact the client likely shouldn't reconnect at all, but just create all future connections using the referral's server. If the referral server ever fails, the client should connect back to the original server.
> 
> Does anyone else think this would be worth maybe writing a new updated RFC2221bis and trying to convince client developers to implement it?

Any particular reason to reuse the same name with an altered meaning rather than coming up with a new name for this?  e.g.

* OK [CANONICAL imaps://user;AUTH=*;us1.imap.example.com/] home server is us1

Bron ( I can't believe you're sending the user to a non-ssl URL in this day and age! )

-- 
  Bron Gondwana
  brong@fastmail.fm
Reply
E-mail headers
From: blong@google.com
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:51 -0000
Message-ID: CABa8R6s366RnvJ632Fzh0JDUE9rjaYbmWxA6CAo-R8LVi040KA@mail.gmail.com permalink / raw / eml / mbox
I assume this would also mean that all of the "more specific" locations
would need to be alt names in the ssl certificate.

If you were blind load balancing (ie, just multiple A records), then this
would clearly be a win, but I'm not sure how many people with multiple DCs
are doing that.  Even then, knowing which proxy point is closer to the
user's backend may be easy to know, but knowing which proxy point is the
better network mid-point from the user to the backend is harder.  Granted,
that's more likely a problem once you go beyond two DCs, where the user may
be closest to a DC which doesn't have their data.  And I guess you can
always go with "give them a proxy closest to their data and let the network
figure out the best path to get there".

There's also the question of how do you handle having the user's data in
multiple DCs and the user traveling, though presumably you'd just evaluate
every time and it improves the next connection.  With mobile devices, a
user could have many very different network paths through-out the day.

Is it common for people with multiple DCs to have blind DNS load balancing?

We would be unlikely to implement this because our load balancing is not
blind and our network topology is designed to get people onto our network
as their closest point.  Also, load balancing for us is not strictly geo
based, otherwise we'd need a lot more capacity to handle daily peaks from
different parts of the world.

Brandon


On Tue, Jan 14, 2014 at 1:32 PM, Timo Sirainen <tss@iki.fi> wrote:

> I started wondering today if it would be a good idea to bring back the
> login referrals, only a little bit changed from their original behavior. I
> guess originally they didn't really get used because few clients supported
> them and proxying works for everyone. But nowadays many larger
> installations have IMAP proxies and backends in multiple data centers and
> even though everything works, doing unnecessary proxying between data
> centers increases latency and often also costs money. Using DNS to give the
> correct data center IP based on the source IP might work for some users,
> but that's somewhat difficult to set up and likely won't work very well for
> the increasing number of mobile users. So what if we got the most popular
> clients to implement support for login referrals and use them?
>
> Problem 1: LOGIN tagged OK [REFERRAL] reply conflicts between
> Lemonade-required OK [CAPABILITY] reply.
>
> So either one of them needs to be moved to an untagged OK reply. RFC 2221
> only talks about tagged OK reply, while RFC 5550 makes it possible to use
> untagged OK reply. But I'm a bit worried about changing this now, there may
> well be clients that handle only the tagged OK [CAPABILITY] while there are
> almost no clients that handle [REFERRAL] anywhere. So my preferred server
> reply would be:
>
> * OK [REFERRAL imap://user;AUTH=*;us1.imap.example.com/] Use this server
> for decreased latency
> a OK [CAPABILITY ...] Logged in.
>
> Problem 2: RFC 2221 specified OK [REFERRAL] to mean that user's mailboxes
> aren't on this server, but some shared mailboxes are.
>
> Instead now the OK [REFERRAL] should simply be a suggestion to the client
> to connect to the specified server to get better latency, but it would work
> just as well to just ignore it and keep using the current server. In fact
> the client likely shouldn't reconnect at all, but just create all future
> connections using the referral's server. If the referral server ever fails,
> the client should connect back to the original server.
>
> Does anyone else think this would be worth maybe writing a new updated
> RFC2221bis and trying to convince client developers to implement it?
>
> _______________________________________________
> Imap-protocol mailing list
> Imap-protocol@u.washington.edu
> http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20140114/87fc4d6b/attachment.html>
Reply
E-mail headers
From: stephan@rename-it.nl
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:51 -0000
Message-ID: 52D64637.5000203@rename-it.nl permalink / raw / eml / mbox
Hi,

On 1/14/2014 10:32 PM, Timo Sirainen wrote:
> Does anyone else think this would be worth maybe writing a new updated RFC2221bis and trying to convince client developers to implement it?

Oh, excellent opportunity to post the following problem with RFC2221
that emerged with the advent of RFC 5092 and RFC 3986 (I keep forgetting
about this). These RFCs updated the IMAP and generic URL scheme
respectively and added the IP-literal syntax element to accommodate for
IP6 addresses:

IP-literal    = "[" ( IPv6address / IPvFuture  ) "]"


The syntax of REFERRAL is specified in RFC2221, Section 5 and looks as
follows:

resp_text_code =/ "REFERRAL" SPACE <imapurl>

RFC 3501 shows the following for resp_text_code:

resp-text-code  = "ALERT" /
                  "BADCHARSET" [SP "(" astring *(SP astring) ")" ] /
                  capability-data / "PARSE" /
                  "PERMANENTFLAGS" SP "("
                  [flag-perm *(SP flag-perm)] ")" /
                  "READ-ONLY" / "READ-WRITE" / "TRYCREATE" /
                  "UIDNEXT" SP nz-number / "UIDVALIDITY" SP nz-number /
                  "UNSEEN" SP nz-number /
                  atom [SP 1*<any TEXT-CHAR except "]">]

Obviously, response code arguments cannot contain "]". However, with the
new IP-literal syntax element in the IMAP URL, the REFERRAL argument
could actually contain a "]" now. Clients that know about REFERRAL and
the most recent IMAP URL scheme version could still parse this with some
extra care, but clients that don't will fail to parse this properly.

This isn't easy to fix, since I don't see any means to escape the "]" in
the response code arguments.

Regards,

Stephan.
Reply
E-mail headers
From: blong@google.com
To: imap-protocol@localhost
Date: Fri, 08 Jun 2018 12:34:51 -0000
Message-ID: CABa8R6tRLa5K=t9dKzWJyS0r0-+SZhoFYSJJAFXpLDLsb25r9w@mail.gmail.com permalink / raw / eml / mbox
On Tue, Jan 14, 2014 at 1:43 PM, Bron Gondwana <brong@fastmail.fm> wrote:

> On Wed, Jan 15, 2014, at 08:32 AM, Timo Sirainen wrote:
> > I started wondering today if it would be a good idea to bring back the
> login referrals, only a little bit changed from their original behavior. I
> guess originally they didn't really get used because few clients supported
> them and proxying works for everyone. But nowadays many larger
> installations have IMAP proxies and backends in multiple data centers and
> even though everything works, doing unnecessary proxying between data
> centers increases latency and often also costs money. Using DNS to give the
> correct data center IP based on the source IP might work for some users,
> but that's somewhat difficult to set up and likely won't work very well for
> the increasing number of mobile users. So what if we got the most popular
> clients to implement support for login referrals and use them?
> >
> > Problem 1: LOGIN tagged OK [REFERRAL] reply conflicts between
> Lemonade-required OK [CAPABILITY] reply.
> >
> > So either one of them needs to be moved to an untagged OK reply. RFC
> 2221 only talks about tagged OK reply, while RFC 5550 makes it possible to
> use untagged OK reply. But I'm a bit worried about changing this now, there
> may well be clients that handle only the tagged OK [CAPABILITY] while there
> are almost no clients that handle [REFERRAL] anywhere. So my preferred
> server reply would be:
> >
> > * OK [REFERRAL imap://user;AUTH=*;us1.imap.example.com/] Use this
> server for decreased latency
> > a OK [CAPABILITY ...] Logged in.
> >
> > Problem 2: RFC 2221 specified OK [REFERRAL] to mean that user's
> mailboxes aren't on this server, but some shared mailboxes are.
> >
> > Instead now the OK [REFERRAL] should simply be a suggestion to the
> client to connect to the specified server to get better latency, but it
> would work just as well to just ignore it and keep using the current
> server. In fact the client likely shouldn't reconnect at all, but just
> create all future connections using the referral's server. If the referral
> server ever fails, the client should connect back to the original server.
> >
> > Does anyone else think this would be worth maybe writing a new updated
> RFC2221bis and trying to convince client developers to implement it?
>
> Any particular reason to reuse the same name with an altered meaning
> rather than coming up with a new name for this?  e.g.
>
> * OK [CANONICAL imaps://user;AUTH=*;us1.imap.example.com/] home server is
> us1
>
> Bron ( I can't believe you're sending the user to a non-ssl URL in this
> day and age! )
>

I'm sure he implied STARTTLS, that is, after all, the preferred way for
IETF protocols to work.  Or is that the difference between simap and imaps?

Brandon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20140114/3a6e9728/attachment.html>
Reply