Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Helge Kreutzmann]

Package: project
Severity: important

If I try to mail e.g. Marcos Fouces <***@debian.org>, this no
longer works. I get the following error message:

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

***@gmail.com
host gmail-smtp-in.l.google.com [173.194.79.26]
SMTP error from remote mail server after pipelined end of data:
550-5.7.26 This mail is unauthenticated, which poses a security risk to the
550-5.7.26 sender and Gmail users, and has been blocked. The sender must
550-5.7.26 authenticate with at least one of SPF or DKIM. For this message,
550-5.7.26 DKIM checks did not pass and SPF check for [helgefjell.de] did not
550-5.7.26 pass with ip: [82.195.75.114]. The sender should visit
550-5.7.26 https://support.google.com/mail/answer/81126#authentication for
550 5.7.26 instructions on setting up authentication. v26-20020aa7d65a000000b005231f55294dsi4996663edr.385 - gsmtp

The IP 82.195.75.114 resolves to
114.75.195.82.in-addr.arpa is an alias for 114.64-26.75.195.82.in-addr.arpa.
114.64-26.75.195.82.in-addr.arpa domain name pointer mailly.debian.org.

And of course, SPF/DKIM checks for my domain (helgefjell.de) fail for
this IP, which is @debian.org.

I don't know how it worked so far, and the error could be on my side, as I
recently switched my e-mail setup; however, I don't see anything I can
do to make DKIM/SPF point to @debian.org instead of @helgefjell.de,
when transferring e-mail to gmail.

Greetings

Helge

[Adam D. Barratt]

Hi,
Contacting DSA is generally a better way to ask about infrastructure
things than filing bugs on high-level pseudo-packages.

Post by Helge Kreutzmann
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
host gmail-smtp-in.l.google.com [173.194.79.26]
550-5.7.26 This mail is unauthenticated, which poses a security risk to the
550-5.7.26 sender and Gmail users, and has been blocked. The sender must
550-5.7.26 authenticate with at least one of SPF or DKIM. For this message,
550-5.7.26 DKIM checks did not pass and SPF check for
[helgefjell.de] did not
550-5.7.26 pass with ip: [82.195.75.114]. The sender should visit
550-5.7.26
https://support.google.com/mail/answer/81126#authentication for
550 5.7.26 instructions on setting up authentication. v26-
20020aa7d65a000000b005231f55294dsi4996663edr.385 - gsmtp
The IP 82.195.75.114 resolves to
114.75.195.82.in-addr.arpa is an alias for 114.64-26.75.195.82.in-
addr.arpa.
114.64-26.75.195.82.in-addr.arpa domain name pointer
mailly.debian.org.
And of course, SPF/DKIM checks for my domain (helgefjell.de) fail for

The DKIM signature warning has nothing to do with the forwarding, or
the involvement of debian.org at all. The reason that check fails is
that your mail has no DKIM signature, so obviously can't have a valid
one. Signing your mail would probably make gmail a lot happier with it
in general. (As a side note, the BTS breaks many common DKIM signature
strategies, but that's a different issue.)

The general issue is being worked on, as time and resources allow.

Regards,

Adam
(part of, but not on behalf of, DSA)

[Helge Kreutzmann]

Hello Adam,

Post by Adam D. Barratt
Contacting DSA is generally a better way to ask about infrastructure
things than filing bugs on high-level pseudo-packages.

Thanks, then I know this for the future.

Post by Adam D. Barratt

Post by Helge Kreutzmann
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
host gmail-smtp-in.l.google.com [173.194.79.26]
550-5.7.26 This mail is unauthenticated, which poses a security risk to the
550-5.7.26 sender and Gmail users, and has been blocked. The sender must
550-5.7.26 authenticate with at least one of SPF or DKIM. For this message,
550-5.7.26 DKIM checks did not pass and SPF check for
[helgefjell.de] did not
550-5.7.26 pass with ip: [82.195.75.114]. The sender should visit
550-5.7.26
https://support.google.com/mail/answer/81126#authentication for
550 5.7.26 instructions on setting up authentication. v26-
20020aa7d65a000000b005231f55294dsi4996663edr.385 - gsmtp
The IP 82.195.75.114 resolves to
114.75.195.82.in-addr.arpa is an alias for 114.64-26.75.195.82.in-
addr.arpa.
114.64-26.75.195.82.in-addr.arpa domain name pointer
mailly.debian.org.
And of course, SPF/DKIM checks for my domain (helgefjell.de) fail for

The DKIM signature warning has nothing to do with the forwarding, or
the involvement of debian.org at all. The reason that check fails is
that your mail has no DKIM signature, so obviously can't have a valid
one. Signing your mail would probably make gmail a lot happier with it
in general. (As a side note, the BTS breaks many common DKIM signature
strategies, but that's a different issue.)

Sigh.

Directly gmail accepts it.

Post by Adam D. Barratt
The general issue is being worked on, as time and resources allow.

Thanks a lot!

Greetings

Helge

[Adam D. Barratt]

Hello Adam,

[...]

Post by Adam D. Barratt

Post by Helge Kreutzmann
550-5.7.26 This mail is unauthenticated, which poses a
security
risk to the
550-5.7.26 sender and Gmail users, and has been blocked. The sender must
550-5.7.26 authenticate with at least one of SPF or DKIM. For this message,
550-5.7.26 DKIM checks did not pass and SPF check for
[helgefjell.de] did not

[...]

Post by Adam D. Barratt

Post by Helge Kreutzmann
550-5.7.26
https://support.google.com/mail/answer/81126#authentication for
550 5.7.26 instructions on setting up authentication. v26-
20020aa7d65a000000b005231f55294dsi4996663edr.385 - gsmtp
The IP 82.195.75.114 resolves to
114.75.195.82.in-addr.arpa is an alias for 114.64-
26.75.195.82.in-
addr.arpa.
114.64-26.75.195.82.in-addr.arpa domain name pointer
mailly.debian.org.
And of course, SPF/DKIM checks for my domain (helgefjell.de) fail

The DKIM signature warning has nothing to do with the forwarding,
or the involvement of debian.org at all. The reason that check
fails is that your mail has no DKIM signature, so obviously can't
have a valid one. Signing your mail would probably make gmail a lot
happier with it in general. (As a side note, the BTS breaks many
common DKIM signature strategies, but that's a different issue.)

Sigh.
Directly gmail accepts it.

I'm not sure why the sigh, but in any case your direct mail presumably
succeeds because it passes the SPF check. I was simply clarifying that
the DKIM check would fail in both cases.

Regards,

Adam

[Helge Kreutzmann]

Hello Adam,

Post by Adam D. Barratt

Directly gmail accepts it.

I'm not sure why the sigh, but in any case your direct mail presumably
succeeds because it passes the SPF check. I was simply clarifying that
the DKIM check would fail in both cases.

Well, I did have trouble sending directly to gmail accounts, which now
seems to work. Now the next e-mail problem arises, which I need to see
how much I can configure it to work. That's the sigh.

It's just that I never had this problem with mails to people with
@debian.org addresses, so either my new configuration or some other
change, I don't know.

I hope this explains it a little.

Greetings

Helge

[Russ Allbery]

Post by Helge Kreutzmann
It's just that I never had this problem with mails to people with
@debian.org addresses, so either my new configuration or some other
change, I don't know.

The problem I suspect is with email forwarding, and specifically email
forwarding to Gmail, which has recently ramped up the amount of
verification it does on messages. Because of email forwarding, Gmail sees
a message purportedly from helgefjell.de but actually delivered by
debian.org mail servers, and has now decided to be suspicious of that.

If that's correct, you'll only have this problem with Debian developers
who forward their @debian.org addresses to Gmail. Gmail handles some
large percentage of all email on the Internet, so this probably isn't
rare, but Debian developers are less likely to use it than random Internet
users for obvious reasons, so it doesn't surprise me you've not run into
the problem before. (In other words, I doubt this is a problem with your
local configuration.)

[Mattia Rizzolo]

Post by Russ Allbery
The problem I suspect is with email forwarding, and specifically email
forwarding to Gmail, which has recently ramped up the amount of
verification it does on messages. Because of email forwarding, Gmail sees
a message purportedly from helgefjell.de but actually delivered by
debian.org mail servers, and has now decided to be suspicious of that.

This is the exact use case that SRS was developer for, however gmail's
documentation does not recommend that (but the situation, as you noted,
worsened, so I tried it in some other similar setups and everything is
great, so...).
My understanding is that several DSA members were opposed to using SRS
for @debian.org forwarding, but maybe it's now time?

Alternatively, I wonder if ARC nowadays is respected enough (and if
Google cares about it)... I personally don't have any system with ARC
under my care.

[Adam D. Barratt]

Post by Mattia Rizzolo

Post by Russ Allbery
The problem I suspect is with email forwarding, and specifically email
forwarding to Gmail, which has recently ramped up the amount of
verification it does on messages. Because of email forwarding, Gmail sees
a message purportedly from helgefjell.de but actually delivered by
debian.org mail servers, and has now decided to be suspicious of that.

This is the exact use case that SRS was developer for, however
gmail's documentation does not recommend that (but the situation, as
you noted, worsened, so I tried it in some other similar setups and
everything is great, so...).

They sort of recommend it now. But also not. It's complicated. [tm]

Post by Mattia Rizzolo
My understanding is that several DSA members were opposed to using

That's essentially what's being worked on. But life, and free time, and
other priorities, keep getting in the way.

Regards,

Adam

[Adam D. Barratt]

Post by Adam D. Barratt

Post by Mattia Rizzolo

Post by Russ Allbery
The problem I suspect is with email forwarding, and specifically email
forwarding to Gmail, which has recently ramped up the amount of
verification it does on messages. Because of email forwarding, Gmail sees
a message purportedly from helgefjell.de but actually delivered by
debian.org mail servers, and has now decided to be suspicious of that.

This is the exact use case that SRS was developer for, however
gmail's documentation does not recommend that (but the situation, as
you noted, worsened, so I tried it in some other similar setups and
everything is great, so...).

They sort of recommend it now. But also not. It's complicated. [tm]

Post by Mattia Rizzolo
My understanding is that several DSA members were opposed to using

That's essentially what's being worked on. But life, and free time, and
other priorities, keep getting in the way.

An initial version, rewriting mails to Google-hosted domains from
"external" e-mail addresses (those for which debian.org's mail relays
don't consider themselves authoritative, so mostly not *.debian.org and
*.debconf.org) is now live.

Please let DSA know if you encounter any issues.

Regards,

Adam

[Helge Kreutzmann]

Hello Adam,

Post by Adam D. Barratt
An initial version, rewriting mails to Google-hosted domains from
"external" e-mail addresses (those for which debian.org's mail relays
don't consider themselves authoritative, so mostly not *.debian.org and
*.debconf.org) is now live.
Please let DSA know if you encounter any issues.

Thanks a lot for the speedy fixing.

I'll report any issues (if any).

Greetings

Helge

[Stephen Frost]

Greetings,

Post by Mattia Rizzolo
Alternatively, I wonder if ARC nowadays is respected enough (and if
Google cares about it)... I personally don't have any system with ARC
under my care.

Sadly, no, they don't seem to care one bit about ARC, except possibly if
it's their own ARC sigs.

If someone has some idea how to get them to care about ARC, I'd love to
hear about it, as I have folks on the one hand who view DKIM/DMARC as
too painful to set up but then they end up with bounces from gmail due
to my forwarding of messages through my server (which are being
ARC-signed by it and pass on that the SPF check was successful when they
arrived to my server)...

I'd encourage everyone running their own email servers to please get
DKIM/DMARC/ARC/SPF set up. Yeah, it's annoying, but it's not actually
all *that* bad to do.

Thanks,

Stephen

[Marco d'Itri]

Post by Stephen Frost
If someone has some idea how to get them to care about ARC, I'd love to
hear about it, as I have folks on the one hand who view DKIM/DMARC as
too painful to set up but then they end up with bounces from gmail due
to my forwarding of messages through my server (which are being
ARC-signed by it and pass on that the SPF check was successful when they
arrived to my server)...

I do not know of any situation in which DMARC adoption would improve
deliverability, and most people that configure it are just engaging in
cargo cult sysadmining.
DMARC with p=reject is useful when the sender domain is a phishing
victim, e.g. a financial organization, but most users do not need it.

In other words: if these people want to support use cases like
forwarding and participating to mailing lists then they should adopt
DKIM and ignore DMARC.

[Cord Beermann]

Post by Russ Allbery
The problem I suspect is with email forwarding, and specifically email
forwarding to Gmail, which has recently ramped up the amount of
verification it does on messages. Because of email forwarding, Gmail sees
a message purportedly from helgefjell.de but actually delivered by
debian.org mail servers, and has now decided to be suspicious of that.
If that's correct, you'll only have this problem with Debian developers
large percentage of all email on the Internet, so this probably isn't
rare, but Debian developers are less likely to use it than random Internet
users for obvious reasons, so it doesn't surprise me you've not run into
the problem before. (In other words, I doubt this is a problem with your
local configuration.)

As listmaster i can confirm that it is a big problem to deliver Mails to
gmail/outlook/yahoo. Yahoo Subscribers are mostly gone by now because they
bounced a lot, for gmail it is so much that we just ignore bounces because of
those rules.

If you decide to handle your mails to be curated by someone else you have to
live with an incomplete mailbox.

| helgefjell.de descriptive text "v=spf1 ip4:142.132.201.35 mx ~all"

so you flagged your mail has to come from that IP (or the MX) and from other
sources it should be considered suspicious.

Thats the result.

SRS/ARC and so on are just dirty patches that try to fix things that were
broken before, but they will break even more things like Mail signing.

As long as we have this Oligopol that doesn't care about what they send out
(i.e. Spamfloods through Outlook) things will only get worse.

Cord

[Russ Allbery]

Post by Cord Beermann
As listmaster i can confirm that it is a big problem to deliver Mails to
gmail/outlook/yahoo. Yahoo Subscribers are mostly gone by now because
they bounced a lot, for gmail it is so much that we just ignore bounces
because of those rules.

Yes, I gave up for the mailing lists I run and just rewrite the From
address to be the address of the list and move the actual sender to
Reply-To, and I see other technical mailing lists like the glibc lists
have started doing this as well (using the built-in Mailman feature, which
can optionally do this only if the sender domain has SPF/DMARC records).

[Stephen Frost]

Greetings,

Post by Russ Allbery

Post by Cord Beermann
As listmaster i can confirm that it is a big problem to deliver Mails to
gmail/outlook/yahoo. Yahoo Subscribers are mostly gone by now because
they bounced a lot, for gmail it is so much that we just ignore bounces
because of those rules.

Yes, I gave up for the mailing lists I run and just rewrite the From
address to be the address of the list and move the actual sender to
Reply-To, and I see other technical mailing lists like the glibc lists
have started doing this as well (using the built-in Mailman feature, which
can optionally do this only if the sender domain has SPF/DMARC records).

The answer that we (PostgreSQL folks, at least) went with was to stop
breaking DKIM because that's just a bad approach to take these days with
mailing lists. If you're curious about what PostgreSQL and now SPI are
using for our lists, it's called pgLister and is here:

https://gitlab.com/pglister/pglister

Others have hacked up mailman to make it stop breaking DKIM too (though
it's pretty grotty how they did it, I'll admit).

Yes, yes, I know that means a bunch of mailman features aren't
available. We've managed to survive even without them.

Thanks,

Stephen

[Stephen Frost]

Greetings,

Post by Cord Beermann
As listmaster i can confirm that it is a big problem to deliver Mails to
gmail/outlook/yahoo. Yahoo Subscribers are mostly gone by now because they
bounced a lot, for gmail it is so much that we just ignore bounces because of
those rules.

As a maintainer or some pretty big lists ... we don't have *that* much
trouble delivering to gmail, or others for that matter.

Post by Cord Beermann
| helgefjell.de descriptive text "v=spf1 ip4:142.132.201.35 mx ~all"
so you flagged your mail has to come from that IP (or the MX) and from other
sources it should be considered suspicious.

... but if it's DKIM signed, then it'll generally get delivered
properly.

Post by Cord Beermann
SRS/ARC and so on are just dirty patches that try to fix things that were
broken before, but they will break even more things like Mail signing.

ARC doesn't break DKIM signatures (unless someone's got a very broken
DKIM setup which over-signs ARC headers ... but if so, then that's on
them).

Thanks,

Stephen

[Russ Allbery]

Post by Helge Kreutzmann
I don't know how it worked so far, and the error could be on my side, as
I recently switched my e-mail setup; however, I don't see anything I can
transferring e-mail to gmail.

The mail to which I'm resonding also comes from your @helgefjell.de
domain, so I'm suspecting some DKIM/SPF issues there if you're using that
same address in your original mail message. But just in case you were
trying to send from your @debian.org address, one option is to send all of
your outgoing mail that is from your debian.org address through the
debian.org mail servers. See:

https://dsa.debian.org/user/mail-submit/

I don't think this is the direct answer to your original question, but I
suspect it would work around the problem.

[Helge Kreutzmann]

Hello Russ,

Post by Russ Allbery

Post by Helge Kreutzmann
I don't know how it worked so far, and the error could be on my side, as
I recently switched my e-mail setup; however, I don't see anything I can
transferring e-mail to gmail.

domain, so I'm suspecting some DKIM/SPF issues there if you're using that
same address in your original mail message. But just in case you were

Yes, this is my primary e-mail address

Post by Russ Allbery
your outgoing mail that is from your debian.org address through the
https://dsa.debian.org/user/mail-submit/
I don't think this is the direct answer to your original question, but I
suspect it would work around the problem.

Thanks for taking care, but I don't have an @debian.org address.

Greetings

Helge

[Byung-Hee HWANG]

Hellow all,

Sorry for late feedback. By chance, i discovered this story
#1043539. Also i'm using Gmail via forwarding (Postfix+OpenDKIM). I love
Postfix and DKIM stuff.

Though i am not debian member, i would like to share this experience
with you. If your dkim signature is OK, then Gmail does accept all
mails. So never use SRS. DKIM is enough.

<quote: postfix log with debian-bugs-dist mailing>
Nov 25 09:51:14 yw-1204 postfix/smtpd[94851]: connect from yw-0919.doraji.xyz[2600:1900:4000:af49:0:3::]
Nov 25 09:51:14 yw-1204 postfix/smtpd[94851]: Trusted TLS connection established from yw-0919.doraji.xyz[2600:1900:4000:af49:0:3::]: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256
Nov 25 09:51:14 yw-1204 postfix/smtpd[94851]: D6A8A6D3: client=yw-0919.doraji.xyz[2600:1900:4000:af49:0:3::]
Nov 25 09:51:14 yw-1204 postfix/cleanup[94856]: D6A8A6D3: resent-message-id=<***@bugs.debian.org>
Nov 25 09:51:15 yw-1204 postfix/cleanup[94856]: D6A8A6D3: message-id=<e98510e4-738c-1ea8-8741-***@debian.org>
Nov 25 09:51:15 yw-1204 opendkim[631]: RFC2822-From: Andreas Beckmann <***@debian.org>
Nov 25 09:51:15 yw-1204 opendkim[631]: RFC2821-From: bounce-debian-bugs-dist=soyeomul=***@lists.debian.org
Nov 25 09:51:15 yw-1204 opendkim[631]: RFC2821-To: soyeomul+***@gmail.com
Nov 25 09:51:15 yw-1204 opendkim[631]: D6A8A6D3: DKIM-Signature field added (s=yw-1204-doraji-xyz, d=doraji.xyz)
Nov 25 09:51:15 yw-1204 opendkim[631]: D6A8A6D3: DKIM-Signature field added (s=YW, d=doraji.xyz)
Nov 25 09:51:15 yw-1204 postfix/qmgr[91844]: D6A8A6D3: from=<bounce-debian-bugs-dist=soyeomul=***@lists.debian.org>, size=7421, nrcpt=1 (queue active)
Nov 25 09:51:15 yw-1204 postfix/smtpd[94851]: disconnect from yw-0919.doraji.xyz[2600:1900:4000:af49:0:3::] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
Nov 25 09:51:17 yw-1204 postfix/smtp[94857]: Verified TLS connection established to gmail-smtp-in.l.google.com[2607:f8b0:4023:1006::1b]:25: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256
Nov 25 09:51:18 yw-1204 postfix/smtp[94857]: D6A8A6D3: to=<soyeomul+***@gmail.com>, relay=gmail-smtp-in.l.google.com[2607:f8b0:4023:1006::1b]:25, delay=3.4, delays=0.25/0.02/2.2/0.98, dsn=2.0.0, status=sent (250 2.0.0 OK 1700905878 ep1-20020a056808444100b003b85c5d06d8si386914oib.242 - gsmtp)
Nov 25 09:51:18 yw-1204 postfix/qmgr[91844]: D6A8A6D3: removed
</quote>

Sincerely, Byung-Hee (Debian user in South Korea)

[Amanda Kipkalya Kones]

*HiHow are you doing I hope all is well. My name is Miss Amanda Kipkalya
Kones, I am 24 years old, I am a girl. Today I decided to extend my regards
to you. But I have the Mind you could be a good person, I think, and there
are good people who can appreciate the value of friendship. And i would
like to be your friend even more than that but as time goes by we will know
better well I will say that my mind convinced me that you may be the true
person to help me out in this my present condition i am now in a refugee
camp in Burkina Faso. I will tell you more about me as soon as I hear from
you.please reply me for more confidential discussions may be like that we
can become best friends in future sorry for my pictures i will enclose it
in my next mail and more about me when i hear from you okay i am living now
in a refugee camp, here is my WhatsApp number [phone] and you can
also call me direct call on the phone number as well, i will be waiting to
hear from you,Kisses and warmest regardsYours,Miss Amanda Kipkalya Kones.*