Set of flow restrictions is represented as a "limits" module instance
that can be either created inline via "limits" directive in some modules
(including "remote" target and "smtp" endpoint) or defined globally and
referenced in configuration of modules mentioned above.
This permits a variety of use cases, including shared and separate
counters for various endpoints and also "modules group" style sharing
described in #195.
Allows pipeline routing to be used in places where a regular target is
required. Also allows to share parts of pipeline configurations on the
semantical level (instead of lexical level as it goes with config
snippets), see #195.
Use of inotify and possibly other mechanisms poses portability risks.
Notably, "cross-platform" abstractions such as fsnotify library remove
access to certain features that are important to use it correctly in
some cases e.g. it is preferable to listen only for IN_CLOSE_WRITE on
Linux instead of IN_MODIFY to prevent races and unexpected failures.
Pooling approach avoids such problems by either running reload code at a
different time than actual renewal or retrying later if parse fails.
With certificates being renewed before expiry (e.g. 1 week before) delay
is not a signficiant problem.
Closes#160.
This allows modules (and any other code) to implement arbitrary actions
on server shutdown and "reload". Later will be used to address issues
like #160. There are no plans to implement complete configuration
reloading, however, since it adds a lot of problems to the modules
framework and it is believed to be "not worth it" at the moment.
Closes#130.
Similar to global scoring system that was removed before, but works only
for DNSBL. It is simplier and only actually useful for BLs, since
rejecting/quarantining messages based on a single listing is usually bad
idea and leads to a high false positives rate.
Previous approach consisted of multiple independent options with unknown
interaction between each other and not offering enough flexibility for
local policy configuration.
Additionally, it was not possible to implement downgrade protection
mentioned in #178 because it was not clear what is "downgrade" since
options were not related in any linear order, this commit makes it
explicit via the "security levels" system:
MX: DNSSEC > MTA-STS > Nothing
TLS: Authenticated+Encrypted > Encrypted > Plaintext
Note DNSSEC and MTA-STS being different levels, they provide different
security guarantees. Keeping them together under "authenticated" level
would not provide enough granularity for levels-based downgrade
protection and local policies.
'common_domain' MX authentication option is removed. It was offering no
real protection and therefore is was problematic to use together with
planned downgrade protection.
All security level errors are marked as temporary to force requeueing
and allow local admin to troubleshoot them without losing messages.
'remote' tests are changed to use testTarget function to initialize
tested module instance, since security levels mapping requires some
pre-initialization.
Support for IP literals in address domain-part is disabled because it
is incompatible with the new verification logic and was broken anyway
(#176).
TLS without authentication is still better than no TLS at all.
To save latency in transactions with a misconfigured recipient server
that cannot use TLS at all but still advertises STARTTLS support,
downgrade to non-authenticated TLS is attempted only on verification
errors (x509.UnknownAuthorityError or x509.HostnameError) and malformed
certificate errors (x509.ConstraintViolationError and
x509.CertificateInvalidError). In all other cases 'remote' module
fallbacks to plaintext directly.
While rearranging code to support this, some additional changes were
made to allow simplier implementation of security levels idea from #178.
See https://tools.ietf.org/html/rfc7435.
See #178.
As revealed by latency tracing using runtime/trace, MTA-STS cache miss
essentially doubles the connection time for outbound delivery. This is
mostly because MTA-STS lookup have to estabilish a TCP+TLS connection to
obtain the policy text (shame on Google for pushing that terribly
misdesigned protocol, but, well, it is better than nothing so we adopt
it).
Additionally, there is a number of additional DNS lookups needed (e.g.
TLSA record for DANE). This commit rearranges connection code so it is
possible to run all "additional" queries in parallel with the connection
estabilishment. However, this changes the behavior of TLS requirement
checks (including MTA-STS). The connection to the candidate MX is
already estabilished and STARTTLS is always attempted if it is
available. Only after that the policy check is done, using the result of
TLS handshake attempt (if any). If for whatever reason, the candidate MX
cannot be used, the connection is then closed. This might bring
additional overhead in case of configuration errors on the recipient
side, but it is believed to not be a major problem since this should not
happen often.
runtime/trace together with 'go tool trace' provides extremely powerful
tooling for performance (latency) analysis. Since maddy prides itself on
being "optimized for concurrency", it is a good idea to actually live up
to this promise.
Closes#144. No need to reinvent the wheel. The original issue
proposed a solution to use in production to detect "performance
anomalies", it is possible to use runtime/trace in production too, but
the corresponding flag to enable profiler endpoint is hidden behind the
'debugflags' build tag at the moment.
For SMTP code, the basic latency information can be obtained from
regular logs since they include timestamps with millisecond granularity.
After the issue is apparent, it is possible to deploy the server
executable compiled with tracing support and obtain more information
... Also add missing context.Context arguments to smtpconn.C.
It is useful to define background tasks lifetimes more precisely,
especially involving timeouts and other cancellation methods.
On top of that, several tracing facilities are context-based (e.g.
runtime/trace), so it is possible to use them now.
Allow to override DNS resolver address via the -debug.dnsoverride flag
and SMTP port via -debug.smtpport.
All flags are not available unless maddy is built using the 'debugflags'
tag.
There is abstraction 'updates pipe' defined for future use with
configuration involving IMAP data replication (e.g. multiple nodes with
maddy instances + PostgreSQL replicas + S3 bucket for messages).
However, for the case of local SQLite3 DB, limited UDS-based
implementation is provided. It solves the problem of maddyctl not being
able to tell the server about modifications it makes. Alternative to
this approach would be to have server actually perform operations and
maddyctl being a dumb API client, but this requires a lot more complex
IPC interface and will not work when the server is down.
The intention is to keep to repo root clean while the list of packages
is slowly growing.
Additionally, a bunch of small (~30 LoC) files in the repo root is
merged into a single maddy.go file, for the same reason.
Most of the internal code is moved into the internal/ directory. Go
toolchain will make it impossible to import these packages from external
applications.
Some packages are renamed and moved into the pkg/ directory in the root.
According to https://github.com/golang-standards/project-layout this is
the de-facto standard to place "library code that's ok to use by
external applications" in.
To clearly define the purpose of top-level directories, README.md files
are added to each.
