Chapter 17b Compliance, Storage & Ops Admin: #pArchiveSources

nextxch Archive: Microsoft 365, Exchange, Google Workspace and IMAP into the GoBD archive

nextxch Archive brings mail from systems that do not run on nextxch into the GoBD archive. Four source kinds are available in the admin under Compliance > Archive sources: SMTP journal (journal reports from Microsoft 365 and Exchange, copy rule from Google Workspace, BCC copies from any MTA), IMAP (any server, read-only), Microsoft 365 via Microsoft Graph and Google Workspace via the Gmail API. Every mailbox of the source becomes an archived mailbox without sign-in and without mail traffic; search, legal hold, evidence export and restore work on it as on your own mailboxes.

Operational Tip & Architecture Note

Recommended combination: the SMTP journal for complete ongoing capture from a cut-off date (users cannot delete anything before it is captured), plus the API source once or at a long interval for existing mail and the folder view. Both write into the same archive; a message captured twice is stored only once thanks to SHA-256. Licence: archived mailboxes of other systems are a separate licence line, your own nextxch mailboxes do not count there. 10 archived mailboxes are free in every tier, more come with a Business or Enterprise licence.

1. Quick start: create a journal source for Microsoft 365

New sources start disabled. Test the connection, then enable the source. The server generates the journal address with a secret token when only the domain is given; its MX must point to nextxch.

Reference / Configuration

# 1. Admin > Compliance > Archive sources > New source, kind "SMTP journal"
#    Domains of the old environment: company.com · journal domain: archive.company.com
#    -> journal address e.g. journal-k3x9q2m7v1c8b4n6z5ta@archive.company.com (copy it)
# 2. Protection: IP ranges of the Exchange Online servers, TLS required, allowed envelope senders
# 3. Microsoft Purview: Solutions > Data lifecycle management > Exchange (legacy) > Journal rules
#    or Exchange Online PowerShell:
Set-TransportConfig -JournalingReportNdrTo journal-ndr@archive.company.com
New-JournalRule -Name "nextxch Archive" -JournalEmailAddress journal-k3x9q2m7v1c8b4n6z5ta@archive.company.com -Scope Global -Enabled $true
Get-JournalRule | Format-Table Name,Enabled,JournalEmailAddress,Scope
# 4. Send a test message: the source shows the original with all envelope recipients (including Bcc)

2. Journal from Exchange on-premises and Google Workspace

Exchange on-premises: journal rule with the same journal address (premium journaling, needs the Exchange Enterprise CAL) or standard journaling per mailbox database to that address; in a hybrid environment also create the rule on the on-premises Exchange server. Google Workspace: a routing rule adds the journal address as an extra recipient; Google Vault is not needed. With "Comprehensive mail storage" mail from SMTP relay and Google services also lands in the mailboxes. Any other MTA sends BCC copies to the journal address.

Reference / Configuration

# Exchange Server (Exchange Management Shell)
New-JournalRule -Name "nextxch Archive" -JournalEmailAddress journal-k3x9q2m7v1c8b4n6z5ta@archive.company.com -Scope Global -Enabled $true

# Google Admin console: Menu > Apps > Google Workspace > Gmail > Routing > Configure
#   Messages to affect: inbound, outbound, internal inbound, internal outbound
#   Action: Modify message > Add more recipients > Advanced > journal address
#   Optional: do not deliver spam to this recipient, suppress bounces from this recipient
# Journal source in nextxch: format bcc or auto, IP ranges of Google's outbound servers, TLS required

3. IMAP: any server, read-only

Folders are opened with EXAMINE and content is fetched with BODY.PEEK[], flags at the source never change. A cursor per folder (UIDVALIDITY, last UID, HIGHESTMODSEQ with CONDSTORE) is updated after every message, an interruption resumes exactly there. TLS is mandatory (implicit or STARTTLS), optionally with a pinned certificate (SHA-256); sign-in with password or XOAUTH2, one account or a list of accounts. Trash and spam are skipped by default.

Reference / Configuration

{
  "host": "imap.oldhoster.com",
  "tls": "implicit",
  "auth": "password",
  "mode": "single",
  "username": "accounting@company.com",
  "address": "accounting@company.com",
  "excludeFolders": ["Drafts"],
  "since": "2016-01-01",
  "maxConnections": 2
}
# Secret: password (or oauthToken with xoauth2), in list mode accounts/<id>/password

4. Microsoft 365 via Microsoft Graph: app registration in Entra ID

The source reads existing and new mail with delta queries per folder, without EWS (Microsoft retires EWS in Exchange Online). Application permissions in Microsoft Graph with admin consent: Mail.Read and User.Read.All, GroupMember.Read.All only with the user filter "group". Sign-in with a certificate (recommended) or a client secret, clouds global, usgov, usgovdod and china. Graph throttling (429 with Retry-After) is honoured, a run resumes at the last fully processed page. Messages deleted at the source stay in the archive. Optionally, Exchange Online RBAC for Applications limits access to a group.

Reference / Configuration

# Entra admin center > App registrations > New registration "nextxch Archive" (single tenant)
# API permissions > Microsoft Graph > Application permissions:
#   Mail.Read, User.Read.All (+ GroupMember.Read.All with group filter) > Grant admin consent
# Certificates & secrets: upload a certificate (.cer) or create a client secret
{
  "tenantId": "company.onmicrosoft.com",
  "clientId": "11111111-2222-3333-4444-555555555555",
  "credentialType": "certificate",
  "cloud": "global",
  "userFilter": { "mode": "group", "groupId": "99999999-0000-0000-0000-000000000009" },
  "sinceUtc": "2019-01-01T00:00:00Z"
}
# Secrets: certificatePfx (Base64) + certificatePassword or clientSecret (the value, not the ID)

5. Google Workspace via the Gmail API: service account and domain-wide delegation

A service account in the customer's own Google Cloud project (Gmail quotas count per project) with the Gmail API and Admin SDK API enabled. In the Admin console its numeric client ID is authorized for domain-wide delegation with the scopes gmail.readonly and admin.directory.user.readonly, plus admin.directory.group.member.readonly for the "group" filter. Initial import via messages.list and messages.get (raw), then history sync; if the history has expired, the import restarts and already archived messages count as duplicates. The quota budget per user and project is adjustable.

Reference / Configuration

# Admin console: Security > Access and data control > API controls > Domain-wide delegation
#   client ID (numeric) + scopes:
#   https://www.googleapis.com/auth/gmail.readonly,
#   https://www.googleapis.com/auth/admin.directory.user.readonly
{
  "adminSubject": "admin@company.com",
  "domain": "company.com",
  "userFilter": "orgUnit",
  "orgUnitPath": "/Sales",
  "sinceUtc": "2018-01-01T00:00:00Z",
  "quotaUnitsPerUserMinute": 4800
}
# Secret: serviceAccountKey (JSON key file of the service account)

6. Storage, WORM retention and legal hold

Every archived mailbox has its own archive-only store in the GoBD archive. Per message: SHA-256 content address with deduplication across all mailboxes, duplicate check per mailbox by SHA-256 and Message-ID, WORM retention and legal hold of the source on blob and message, a hash chain as tamper evidence and a full-text index. The history per run counts stored, duplicate and rejected messages.

Reference / Configuration

Retention      RetentionDays 0..36500 (0 = no WORM retention)
Mode           Normal | Governance | Compliance (default)
Legal hold     per source, also applies to already archived messages
Storage        local or S3 with Object Lock (like the rest of the GoBD archive)
Status         per mailbox: last run, next run, stored, duplicates, rejected, errors

7. Search, export, restore

Archived mailboxes can be chosen as a separate scope in the compliance search (German search with compound splitting, OCR in attachments, ZUGFeRD fields) and each has buttons for search, evidence export and restore in the list. The evidence export delivers a ZIP with original .eml files and an Ed25519-signed manifest (see chapter 17a). The restore copies into a nextxch mailbox, the archive stays unchanged. Restoring back into Microsoft 365, Google or the IMAP server is not available yet; the export covers that. Search in every tier, evidence export and restore jobs from Business.

Reference / Configuration

# Search in an archived mailbox
curl -s "https://mail.company.com/api/compliance/search?archivedMailboxId=<id>&q=invoice&limit=100" -H "X-Api-Key: $KEY"

# Evidence export of an archived mailbox (202, status under GET /api/compliance/exports/{id})
curl -X POST https://mail.company.com/api/compliance/exports -H "X-Api-Key: $KEY" -H "Content-Type: application/json" \
  -d '{"scope":"mailbox","archivedMailboxId":"<id>","caseReference":"case 4711"}'

# Restore into a nextxch mailbox
curl -X POST https://mail.company.com/api/compliance/restore/jobs -H "X-Api-Key: $KEY" -H "Content-Type: application/json" \
  -d '{"source":"archive","account":"archived:<id>","target":"account","targetAccount":"successor@company.com"}'

8. Admin API: /api/admin/archive-sources

TenantAdmin within the own tenant, ServerAdmin everywhere; the ComplianceAuditor reads (to choose archived mailboxes for search and export) but never changes anything. Kind is Journal, Imap, Microsoft365 or Google. Enabling a mailbox beyond the licence quota is refused with a message including the current licence line. Journal sources receive by SMTP, there is no run for them (409 JOURNAL_IS_PUSHED).

Reference / Configuration

GET    /api/admin/archive-sources                          sources, licence line, available kinds with fields
GET    /api/admin/archive-sources/license                  used, allowed, free, remaining
POST   /api/admin/archive-sources                          create a source
GET    /api/admin/archive-sources/{id}                     source, mailboxes, names of the secrets
PUT    /api/admin/archive-sources/{id}                     change (settings, retention, legal hold, schedule)
DELETE /api/admin/archive-sources/{id}                     delete
POST   /api/admin/archive-sources/{id}/test                connection test
POST   /api/admin/archive-sources/{id}/discover            discover mailboxes
GET    /api/admin/archive-sources/{id}/mailboxes           archived mailboxes
PUT    /api/admin/archive-sources/{id}/mailboxes/{mbxId}   {"enabled": true} within the licence line
POST   /api/admin/archive-sources/{id}/run                 fetch now, optionally {"mailboxId": "..."}
GET    /api/admin/archive-sources/{id}/runs                history (mailboxId, limit)

curl -X POST https://mail.company.com/api/admin/archive-sources -H "X-Api-Key: $KEY" -H "Content-Type: application/json" -d '{
  "kind": "Journal", "name": "Microsoft 365 journal", "journalDomain": "archive.company.com",
  "settings": { "domains": ["company.com"], "allowedSenderNetworks": ["203.0.113.0/24"], "requireTls": true },
  "retentionDays": 3650, "lockMode": "Compliance" }'

Settings, Default Values & Best Practices

Setting / Location	Description	Default Value	Production Best Practice
`Journal: Domains (domains)`	Only envelope addresses of these domains and their subdomains become archived mailboxes (on first sight while auto-discovery is on). Messages without a matching address go to the source's "journal without assignment" store, which does not count towards the licence.	`leer · automatische Erkennung an`	Enter every domain of the old environment, otherwise mailboxes are missing from the assignment.
`Journal: Schutz (allowedSenderNetworks, requireTls, allowedEnvelopeSenders)`	Allowed IP addresses or CIDR ranges of the sending servers, acceptance over TLS only (STARTTLS or implicit) and allowed envelope senders (address, *@domain or domain). A journal address needs either an IP list or a secret token in its local part (at least 16 characters, generated by the server).	`keine IP-Liste · TLS aus · alle Absender`	Set the provider's IP ranges and require TLS; keep the token as well.
`Journal: Format (format)`	auto detects a journal report by X-MS-Journal-Report; envelope forces parsing as an Exchange or Microsoft 365 report, bcc treats every message as a copy (Google copy rule, other MTAs).	`auto`	Keep auto unless an MTA sends BCC copies with its own headers.
`Aufbewahrung (RetentionDays, LockMode)`	WORM retention in days per source (0 to 36,500, 0 = no WORM retention) with mode Normal, Governance or Compliance. Applies to the blob and the message of every archived mail.	`Compliance`	Choose the statutory period for business letters and accounting records; Compliance cannot be lifted before it expires.
`Legal Hold (LegalHold)`	Locks all messages of the source against deletion, also beyond the retention. When it is switched on, the hold is applied to already archived messages as well; every change is recorded in the audit log.	`aus`	Switch on for litigation or an audit and lift it only after legal has cleared it.
`Zeitplan und Erkennung (ScheduleMinutes, AutoDiscover)`	Fetch interval of the API sources (IMAP, Microsoft 365, Google) from 5 minutes to 7 days; in a cluster only the leader fetches. Journal sources receive by SMTP and have no schedule.	`60 min`	For the initial import of large tenants start the run manually and watch the history per mailbox.
`Geheimnisse (Secrets)`	Passwords, client secrets, certificates and service account keys are stored encrypted in the server keyring. They are write-only: API responses carry only their names, never the values, and they never appear in logs.	`verschlüsselt`	Note the expiry of the client secret and certificate; the connection test reports an expired secret.