Chapter 09 Mail Transport & Defense Admin: #pSmtpQueue

Recommendation: Proxmox Mail Gateway as an additional layer in front of nextxch

A mail gateway in front accepts all mail from the internet, checks it with a second, independent anti-spam and antivirus engine and forwards only clean mail to nextxch. This page shows the architecture, a step-by-step setup of Proxmox Mail Gateway (open source, AGPLv3) in front of nextxch and the nextxch settings that matter.

Operational Tip & Architecture Note

The gateway is optional: nextxch has its own spam and virus protection. Important if you use it: nextxch always evaluates SPF, greylisting and DNSBL against the IP of the delivering connection, which behind the gateway is the gateway IP. There is currently no setting for trusted upstream relays. So run these checks in the gateway, where the real sender IP is visible, and turn them off in nextxch (step 5).

Why a gateway in front can make sense

Defense in depth: Two independent systems have to fail before a malicious message reaches a mailbox. A weakness in one product is not enough on its own.
Second anti-spam and antivirus engine: Proxmox Mail Gateway uses its own rules, Bayesian filter, SPF, DNSBL and ClamAV. What one filter misses, the other often catches.
Quarantine: Suspicious messages are held in the gateway and can be reviewed and released by administrators or users through the spam report.
Decoupling from the internet: Only the gateway accepts connections from the internet on port 25. nextxch sits behind it, with port 25 open to the gateway only.
Buffering during maintenance: If nextxch is briefly unavailable for an update or maintenance, the gateway keeps inbound messages in its queue and delivers them afterwards.

An honest assessment: nextxch ships with its own spam protection (SpamGuard with adaptive filter, SPF/DKIM/DMARC/ARC, greylisting, DNSBL, phishing heuristics) and integrates ClamAV. For most installations that is enough. A gateway in front is optional and pays off for higher requirements: policies demanding multi-stage filtering, a central quarantine, several mail servers behind one MX, or strict network separation.

Architecture: internet -> Proxmox Mail Gateway -> nextxch

The MX record points to the gateway. It accepts mail on port 25, filters it and forwards it via SMTP to nextxch. Optionally, nextxch sends outbound mail through the gateway's internal port 26.

Solid: inbound path (required). Dashed: outbound path through the gateway as smart host (optional). Without a smart host, nextxch keeps delivering outbound mail directly via DNS MX.

Proxmox® is a registered trademark of Proxmox Server Solutions GmbH. The name is used descriptively only; nextxch is not affiliated with Proxmox Server Solutions GmbH and is neither endorsed nor certified by it.

Setup at a glance

The order avoids losing mail: set up and test the gateway completely first, then switch the MX record, and close port 25 on nextxch last.

Reference / Configuration

1. Install Proxmox Mail Gateway (ISO, on Debian or as an LXC container)
2. Add relay domains (every domain nextxch accepts)
3. Default relay = nextxch host, port 25 (Mail Proxy -> Relaying)
4. Enable receiver verification, TLS and a certificate in the gateway
5. nextxch: turn off SPF/DMARC checks, greylisting and DNSBL (the gateway takes over)
6. Send a test message through the gateway, check Message Trace in nextxch
7. Switch the MX record to the gateway (remove the old MX only after the TTL expires)
8. Firewall: allow port 25 on nextxch from the gateway only
9. Optional outbound: gateway as smart host in nextxch (port 26, STARTTLS)

Example values on this page:
  Gateway:  pmg.company.com   203.0.113.25   (public, MX)
  nextxch:  mail.company.com  10.0.0.20      (internal, mailboxes)

1. Install Proxmox Mail Gateway

Proxmox Mail Gateway runs as a standalone appliance (ISO installer on hardware or as a VM), as packages on an existing Debian system or as an LXC container. If you use Proxmox VE, a ready-made container template is available in the appliance manager. Afterwards, you manage it through the web interface on port 8006.

Reference / Configuration

Option A: ISO installer (hardware or VM)
  Download the ISO from proxmox.com/downloads -> boot -> run the installer
  Host name: pmg.company.com   (FQDN, matching the future MX and PTR)
  IP:        static IP, e.g. 203.0.113.25 or a DMZ address behind NAT

Option B: on an existing Debian system
  Add the Proxmox package repository and install the packages as described in
  "Install Proxmox Mail Gateway on Debian" in the PMG admin guide

Option C: LXC container in Proxmox VE
  Proxmox VE -> Storage -> CT Templates -> Templates -> section "mail"
  -> download the Proxmox Mail Gateway template -> create the container

Web interface:  https://pmg.company.com:8006   (port 8006 from the admin network only)

Repositories: without a subscription, add the "pmg-no-subscription" repository.
For production, Proxmox recommends the enterprise repository (with subscription).

2. Relay domains, default relay and receiver verification in the gateway

All settings are in the PMG web interface under Configuration -> Mail Proxy. The gateway only accepts mail for listed relay domains and passes it to the default relay after filtering.

Reference / Configuration

Configuration -> Mail Proxy -> Relay Domains
  company.com
  company-group.com        (every domain configured in nextxch, including alias domains)

Configuration -> Mail Proxy -> Relaying
  Default Relay:      mail.company.com  (or 10.0.0.20)
  Relay Port:         25
  Relay Protocol:     SMTP
  Disable MX lookup:  yes               (address the nextxch host directly)

Configuration -> Mail Proxy -> Ports
  External SMTP Port: 25   (from the internet, untrusted)
  Internal SMTP Port: 26   (from nextxch, trusted, for outbound mail)

Configuration -> Mail Proxy -> Options
  Verify Receivers:   550   (reject unknown recipients right away)
  SPF:                yes
  DNSBL Sites:        e.g. zen.spamhaus.org   (observe the list's terms of use)

Several internal servers: Configuration -> Mail Proxy -> Transports
(domain or single address -> its own target server)

3. Switch the MX record to the gateway

Switch only once a test message through the gateway has arrived in nextxch (Message Trace in /admin/). Lower the TTL of the MX record to a few minutes beforehand. Do not change the records for nextxch itself (A/AAAA, PTR, Autodiscover): clients keep connecting directly to nextxch.

Reference / Configuration

; Gateway
pmg.company.com.      3600  IN  A     203.0.113.25
; PTR for 203.0.113.25 at your provider: pmg.company.com

; MX points to the gateway only
company.com.          3600  IN  MX 10 pmg.company.com.

; nextxch stays reachable for clients (IMAP, SMTP submission, webmail)
mail.company.com.     3600  IN  A     198.51.100.20

Test beforehand, without a DNS change:
  swaks --to anna.weber@company.com --server pmg.company.com --port 25 --tls

4. Optional outbound: gateway as smart host in nextxch

Outbound mail then also passes the gateway's filters (e.g. virus scanning, statistics). In the admin cockpit under SMTP, Queue Controls & Live MX Tester (#pSmtpQueue), enter the gateway as the global smart host, or use SMTP routes to send only certain senders through the gateway. Important: the gateway's public IP must then be in the SPF record of your domains, and its PTR must match the gateway's HELO name.

Reference / Configuration

# A) Global smart host: all outbound mail through the gateway
nextxch-cli update SmtpQueue --set outboundSmartHost=pmg.company.com \
  --set outboundSmartPort=26 --set outboundSmartHostTlsMode=StartTlsRequired

# B) Exception: one partner domain still directly via DNS MX (route without target host)
nextxch-cli create Route --set domain=partner.com --set bypassSmartHost=true \
  --set description="direct via MX, not through the gateway"

# C) Only certain senders through the gateway (#pSmtpQueue -> domain routes)
#    Destination domain:   *
#    Only for senders:     company.com, *.company.com   (SenderDomains)
#    Target host:          pmg.company.com:26           (several, comma-separated = failover)
#    TLS mode:             StartTlsRequired
#    Priority:             50                           (smaller = first; default 100)

# Add the gateway IP to the SPF record of the sender domain:
company.com.  IN  TXT  "v=spf1 mx ip4:203.0.113.25 -all"

# In the gateway: add the nextxch IP under Configuration -> Mail Proxy -> Networks
# if nextxch is not in the same subnet as the gateway.

5. Gateway IP in nextxch: what exists and what does not

Current code: nextxch has no setting that treats an upstream relay as trusted. There is no XCLIENT, no XFORWARD and no evaluation of the Received chain. SPF, greylisting and DNSBL use the IP of the delivering connection, which behind the gateway is the gateway IP. The existing TrustedProxyCidrs (Cluster:EnableProxyProtocol) apply only to TCP proxies such as HAProxy using the PROXY protocol. A mail gateway accepts the message itself and sends it again, so that does not fit. IP rules with relay permission do not help either; they only concern relaying to external recipients. Without changes, SPF would return softfail or fail for almost every sender, and messages without a valid DKIM signature from domains with DMARC p=reject would be rejected. Recommendation: perform these checks in the gateway and turn them off in nextxch.

Reference / Configuration

# nextxch: turn off SPF/DKIM/DMARC/ARC evaluation and greylisting, clear DNSBL
# (admin cockpit: SpamGuard & ClamAV (#pSpam) -> "Check SPF, DKIM, DMARC & ARC" and greylisting off)
nextxch-cli update SpamGuard --set enableEmailAuthChecks=false --set enableGreylisting=false
nextxch-cli update SpamGuard --data '{"dnsblZones":[]}'

# Stays active in nextxch:
#   adaptive content filter, phishing heuristics, allow/block lists, ClamAV (if enabled)

# Taken over by the gateway (sees the real sender IP):
#   Mail Proxy -> Options: SPF, DNSBL Sites, Verify Receivers
#   Mail Proxy -> Greylisting
#   Spam filter: DKIM verification (default)

# Connection limit: all inbound mail comes from one IP.
# Check MaxConnectionsPerIp (#pSmtpQueue, default 48) for high volumes.

6. Firewall and TLS on both legs

Close port 25 on nextxch in the host or perimeter firewall (Windows Firewall, nftables, pfSense/OPNsense). nextxch IP rules are not suited for this because they apply to all SMTP ports together and would therefore also hit the clients' 465 and 587. Encrypt both legs: internet <-> gateway and gateway <-> nextxch.

Reference / Configuration

Firewall nextxch (inbound)
  TCP 25         only from 203.0.113.25 / 10.0.0.25 (gateway)
  TCP 465, 587   clients (SMTP submission)  -> stay open
  TCP 993, 443   clients (IMAP, webmail, DAV, push) -> stay open

Firewall gateway (inbound)
  TCP 25         from the internet
  TCP 26         from nextxch only (smart host, if used)
  TCP 8006       from the admin network only

TLS internet <-> gateway
  Configuration -> Mail Proxy -> TLS: "Enable TLS" on
  Certificate: Certificates -> ACME (Let's Encrypt) for pmg.company.com

TLS gateway -> nextxch
  nextxch offers STARTTLS on port 25 with the certificate of mail.company.com
  Gateway: TLS destination policy for mail.company.com = encrypt
           (or verify if the default relay is entered as a host name)

TLS nextxch -> gateway (smart host)
  outboundSmartHostTlsMode = StartTlsRequired

7. DKIM/DMARC, greylisting and backscatter

Three points that are often overlooked in practice. The gateway must not change outbound mail, greylisting should run in one place only, and unknown recipients must be rejected during the SMTP dialog.

Reference / Configuration

DKIM / DMARC
  - nextxch signs outbound mail with DKIM. The gateway forwards it unchanged.
  - No disclaimer rule for outbound mail and no content changes in the gateway
    (otherwise the DKIM signature breaks and DMARC fails at the recipient).
  - Leave DKIM signing off in the gateway; maintain disclaimers in nextxch if needed.

Greylisting
  - Slows down spam bots but delays first messages from new senders (usually < 30 minutes per PMG docs).
  - Run it in the gateway only; behind the gateway it would be ineffective in nextxch.
  - For time-critical senders (e.g. login codes, order confirmations) use the welcomelist,
    or skip greylisting entirely if the other filters are sufficient.

Avoid backscatter
  - Verify Receivers = 550: the gateway asks nextxch via RCPT TO whether the address exists.
  - nextxch rejects unknown recipients at RCPT TO, so the check takes effect immediately.
  - Catch-all addresses in nextxch defeat the check and should be avoided.

8. Licence and cost (as of 1 Nov 2026)

Proxmox Mail Gateway is open source under the GNU AGPLv3 and fully usable without a subscription; updates then come from the pmg-no-subscription repository, which Proxmox intends for test and non-production systems. A subscription applies per host and year, according to Proxmox includes unlimited users and domains, and gives access to the enterprise repository. The prices below come from the Proxmox pricing page and may change; proxmox.com is always authoritative. nextxch does not sell Proxmox subscriptions.

Reference / Configuration

Licence:  GNU Affero General Public License v3 (AGPLv3), since PMG 5.0

Optional subscriptions per host and year (net, per proxmox.com, as of 1 Nov 2026):
  Community    EUR 190      no ticket support (forum)
  Basic        EUR 540      5 tickets/year, response 1 business day
  Standard     EUR 1,260    15 tickets/year, response 4 hours
  Premium      EUR 1,900    unlimited tickets, response 2 hours
  (response = first reply to critical requests within business hours)

Sources:
  https://www.proxmox.com/en/products/proxmox-mail-gateway/pricing
  https://pmg.proxmox.com/pmg-docs/pmg-admin-guide.html   (admin guide 9.1.1: licence, repositories, ports)

Settings, Default Values & Best Practices

Setting / Location	Description	Default Value	Production Best Practice
`PMG: Relay Domains (Mail Proxy)`	List of domains for which the gateway accepts and forwards mail. Mail to other domains is rejected.	`leer`	Add every domain configured in nextxch, including alias domains. When you add a domain in nextxch, add it to the gateway as well.
`PMG: Default Relay + Port (Mail Proxy -> Relaying)`	Destination for inbound mail after filtering (Postfix transport). Additional internal servers per domain via Transports.	`Port 25 · SMTP`	mail.company.com or 10.0.0.20, port 25, protocol SMTP. Enable "Disable MX lookup" if you enter a host name that has an MX record of its own.
`PMG: Verify Receivers (Mail Proxy -> Options)`	Receiver verification: before accepting, the gateway asks nextxch via RCPT TO whether the address exists and rejects unknown recipients during the SMTP dialog.	`aus`	Set to 550. Prevents backscatter because the gateway never accepts mail for made-up addresses that it would later have to bounce.
`PMG: TLS + Zertifikat`	STARTTLS for inbound and outbound connections of the gateway. Without a certificate of its own, PMG generates a self-signed one.	`TLS aus`	Turn TLS on, obtain a trusted certificate via ACME (Let's Encrypt) for pmg.company.com and set a TLS policy of "encrypt" for the nextxch host.
`PMG: Disclaimer & DKIM-Signatur`	The Disclaimer rule action appends text to the body; DKIM Signing signs mail received on the internal port.	`aus`	Leave both off. nextxch signs with DKIM; any change to the content in the gateway breaks the signature and therefore DMARC. Maintain disclaimers in nextxch only (added before signing).
`nextxch: Smart-Host (#pSmtpQueue · outboundSmartHost)`	Global smart host: all outbound mail goes to this relay instead of directly to the recipient MX.	`leer (direkt per MX) · Port 25`	pmg.company.com, relay port 26 (the gateway's internal port), TLS mode StartTlsRequired, no SMTP AUTH. In the gateway, the nextxch IP must be listed under Networks (the same subnet is allowed automatically).
`nextxch: Route · SenderDomains`	Sender condition of an SMTP route: domains (company.com, .company.com) or addresses (billing@company.com, @company.com). Empty = every sender.	`leer`	Send only certain tenants through the gateway: a route with destination domain * and SenderDomains = company.com, target host pmg.company.com:26.
`nextxch: Route · Priority`	Order among routes of the same rank; the smaller number wins, then the more specific pattern.	`100`	Rank comes before priority: sender + destination domain, then destination domain, then sender only (destination ), then , then global smart host, then DNS MX.
`nextxch: Route · BypassSmartHost`	Only for routes without a target host: mail to this domain goes directly via DNS MX even when a global smart host is set.	`false`	For individual partner domains that should not go through the gateway. If the route has a target host, the switch is ignored.
`nextxch: SPF, DKIM, DMARC & ARC prüfen (#pSpam · enableEmailAuthChecks)`	SPF is checked against the IP of the delivering connection. Behind the gateway that is the gateway IP, so SPF mostly returns softfail or fail for other domains.	`an`	Turn off behind the gateway and check SPF in the gateway (Mail Proxy -> Options -> SPF). This also drops the DKIM/DMARC evaluation in nextxch; PMG checks DKIM signatures in its spam filter.
`nextxch: Greylisting & DNSBL-Zonen (#pSpam)`	Both work with the IP of the delivering connection, which behind the gateway is always the same gateway IP.	`abhängig von der Konfiguration`	Turn off or clear in nextxch and use them in the gateway. Content filter, phishing heuristics and ClamAV in nextxch stay active.
`Firewall: Port 25 auf nextxch`	After the MX switch, only the gateway may reach nextxch on port 25.	`offen für alle`	In the host or perimeter firewall, allow port 25 only from 203.0.113.25 or the internal gateway IP. Ports 465, 587, 993 and 443 stay open for clients.