Skip to content

Self-hosted on Kubernetes: SMTP mailer sends EHLO [127.0.0.1], rejected by strict SMTP relays (all outbound email fails) #6778

Description

@MatteoCarrabba

Summary

On self-hosted Kubernetes deployments, the SMTP email provider sends EHLO [127.0.0.1] on every connection. Strict SMTP servers — notably Google Workspace's SMTP relay (smtp-relay.gmail.com) — reject this with a 421 tarpit, so every outbound email fails (invitations, verification, etc.) even when the relay is configured correctly.

Root cause

apps/sim/lib/messaging/email/providers/smtp.ts creates the nodemailer transport without a name option:

const transporter = nodemailer.createTransport({
  host: env.SMTP_HOST,
  port,
  secure,
  auth: ...,
})

When name is not provided, nodemailer's SMTPConnection._getHostname() falls back to os.hostname()and if that hostname contains no dot, it replaces it with [127.0.0.1]:

_getHostname() {
  let hostname;
  try { hostname = os.hostname() || ''; } catch (err) { hostname = 'localhost'; }
  if (!hostname || hostname.indexOf('.') < 0) {
    hostname = '[127.0.0.1]';   // <-- always taken on Kubernetes
  }
  ...
}

Kubernetes pod hostnames (e.g. sim-app-5b8f7d9c4-x2k9q) never contain a dot, so on k8s this fallback is taken 100% of the time and the mailer introduces itself as loopback.

Symptom

Against Google Workspace SMTP relay, every send fails at the EHLO stage:

Error: Server terminates connection. response=421-4.7.0 Try again later, closing connection. (EHLO)
421-4.7.0  For more information, go to
421 4.7.0  https://support.google.com/a/answer/3221692 ... - gsmtp

surfaced in the UI as All email providers failed: ... (e.g. when inviting workspace members). Note: Sim still creates the invitation DB row before the email attempt, so the UI can show a member as invited even though no email was ever delivered.

This is easy to misdiagnose as a relay/allowlist misconfiguration — the relay setup was correct in our case, and raw SMTP sessions from the same pod / same egress IP succeeded whenever a dotted EHLO name was used.

Reproduction

From any Kubernetes pod, open an SMTP session to smtp-relay.gmail.com:587:

  • EHLO [127.0.0.1]421-4.7.0 Try again later, closing connection. (EHLO)
  • EHLO any.dotted.name250 (and the full send succeeds)

Verified on v0.7.50; smtp.ts on current main still passes no name, so the bug should still be present.

Suggested fix

Plumb an optional env var through to the transport, e.g.:

name: env.SMTP_EHLO_NAME || undefined,

(plus the env schema entry / helm chart docs). Defaulting to the current behavior keeps it non-breaking; a fancier default could use os.hostname() only when it's a FQDN and otherwise fall back to the configured public app domain.

Workaround for other k8s users

Force the pod's kernel hostname to be a FQDN so nodemailer uses it:

kubectl -n <ns> patch deploy <app-deploy> --type=strategic \
  -p '{"spec":{"template":{"spec":{"hostname":"sim-app","subdomain":"sim-app","setHostnameAsFQDN":true}}}}'

After the rollout, os.hostname() returns sim-app.sim-app.<ns>.svc.cluster.local (contains dots), nodemailer sends it as the EHLO name, and Google's relay accepts it. Confirmed working.

Environment

  • Sim v0.7.50 (helm chart 1.4.0), self-hosted on GKE
  • Email provider: SMTP (smtp-relay.gmail.com:587, STARTTLS, IP-allowlist relay)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions