Comment from Matthew:
This fixes a potential issue where the Prosody process gets blocked on sockets
waiting for them to close. Unlike non-TLS sockets, closing a TLS socket sends
layer 7 data, and this can cause problems for sockets which are in the process
of being cleaned up.
This depends on LuaSec changes which are not yet upstream.
From Martijn's original email:
So first my analysis of luasec. in ssl.c the socket is put into blocking
mode right before calling SSL_shutdown() inside meth_destroy(). My best
guess to why this is is because meth_destroy is linked to the __close
and __gc methods, which can't exactly be called multiple times and
luasec does want to make sure that a tls session is shutdown as clean
as possible.
I can't say I disagree with this reasoning and don't want to change this
behaviour. My solution to this without changing the current behaviour is
to introduce a shutdown() method. I am aware that this overlaps in a
conflicting way with tcp's shutdown method, but it stays close to the
OpenSSL name. This method calls SSL_shutdown() in the current
(non)blocking mode of the underlying socket and returns a boolean
whether or not the shutdown is completed (matching SSL_shutdown()'s 0
or 1 return values), and returns the familiar ssl_ioerror() strings on
error with a false for completion. This error can then be used to
determine if we have wantread/wantwrite to finalize things. Once
meth_shutdown() has been called once a shutdown flag will be set, which
indicates to meth_destroy() that the SSL_shutdown() has been handled
by the application and it shouldn't be needed to set the socket to
blocking mode. I've left the SSL_shutdown() call in the
LSEC_STATE_CONNECTED to prevent TOCTOU if the application reaches a
timeout for the shutdown code, which might allow SSL_shutdown() to
clean up anyway at the last possible moment.
Another thing I've changed to luasec is the call to socket_setblocking()
right before calling close(2) in socket_destroy() in usocket.c.
According to the latest POSIX[0]:
Note that the requirement for close() on a socket to block for up to
the current linger interval is not conditional on the O_NONBLOCK
setting.
Which I read to mean that removing O_NONBLOCK on the socket before close
doesn't impact the behaviour and only causes noise in system call
tracers. I didn't touch the windows bits of this, since I don't do
windows.
For the prosody side of things I've made the TLS shutdown bits resemble
interface:onwritable(), and put it under a combined guard of self._tls
and self.conn.shutdown. The self._tls bit is there to prevent getting
stuck on this condition, and self.conn.shutdown is there to prevent the
code being called by instances where the patched luasec isn't deployed.
The destroy() method can be called from various places and is read by
me as the "we give up" error path. To accommodate for these unexpected
entrypoints I've added a single call to self.conn:shutdown() to prevent
the socket being put into blocking mode. I have no expectations that
there is any other use here. Same as previous, the self.conn.shutdown
check is there to make sure it's not called on unpatched luasec
deployments and self._tls is there to make sure we don't call shutdown()
on tcp sockets.
I wouldn't recommend logging of the conn:shutdown() error inside
close(), since a lot of clients simply close the connection before
SSL_shutdown() is done.
Despite the warning we introduced, many people continue to try using
prosodyctl to manage Prosody in the presence of systemctl (e.g. #1688).
Also, despite the warning, prosodyctl proceeded with the operation. This means
the commands could be invoked by accident, and cause a situation that is hard
to recover from (needing to manually track down stray processes).
This commit disables all the problematic commands by default, but this can
still be overridden using --force or via a config option.
We only perform this check when we believe Prosody has been "installed" for
system-wide use (i.e. running it from a source directory is still supported).
The array:pluck() method mutates the args, replacing the table items
with the resulting strings. On later runs I assume it tries to index the
string, which returns nil, emptying the array.
Because the code was using `< now` in a lot of places, things expiring at the current second
wouldn't be marked as expired. It isn't noticeable in real-world scenarios but I wanted to
create OAuth 2.0 tokens valid for 0 second in integration tests and it wasn't possible.
By using `<=` instead of `<`, we make sure tokens don't live a single millisecond more than
what they are supposed to.
SQLCipher v3.4.1 (the version in Debian 12) is based on SQLite3 v3.15.2,
while UPSERT support was introduced in SQLite3 v3.24.0
This check was not needed before because we v3.24.0 has not been in a
version of Debian we support for a long, long time.
Note however that SQLCipher databases are not compatible across major
versions, upgrading from v3.x to v4.x requires executing a migration.
Attempts at making `prosodyctl mod_storage_sql upgrade` perform such a
migration has not been successful.
Executing the following in the `sqlcipher` tool should do the migration:
PRAGMA key = '<key material>';
PRAGMA cipher_migrate;
This enables use of encrypted databases if LuaDBI or LuaSQLite3 has been
linked against SQLCipher. Using `LD_PRELOAD` may work as well.
Requires SQLCipher >= 4.0.0 due to the use of UPSERT
Idea is to enable easily retrieving of secret values from files outside
of the config, e.g. via the method used by systemd credentials.
CREDENTIALS_DIRECTORY is expected to be set by the process manager
invoking Prosody, so being unset and unavailable from prosodyctl is
going to be normal and a warning is reported in that case. Care will
have to be taken to make it clear that prosodyctl check will not work
with such values. An error is thrown if the directory is unavailable
when running under Prosody.
Inspired by something MattJ said
Allows retrieving config values from files which are expected to be
relative to the config directory, extending on the ENV_ method of
retrieving config values from outside the config file.
- FileLine retrieves the first line, stripping any trailing newline
- FileContents reads the whole file
- FileLines reads lines into an array
This bestows the role specified by the 'host_user_role' setting onto
users of that host. For simplicity, only a single host can be specified.
Making it configurable allows for setups where VirtualHost and related
Components may be siblings instead of having a subdomain relationship.
For setups with many VirtualHosts sharing a single Component, the
'server_user_role' setting is more appropriate. Even more complicated
setups would have to resort to mod_firewall or similar.
The Lua hook based signal handling does not work correctly if signal
handling is setup in a coroutine. signalfd solves that in a nice way,
but is Linux-only.
Many thanks to Thilo Molitor and Kim Alvefur for their work on this module
while it was in the community repository. It has been stable for some time, is
widely used, and provides a feature that is important to most deployments.
This allows Prosody to easily provide friendly invitation links, even without
setting up mod_invites_page (which is a community module). Admins can
configure it to use a third-party deployment such as https://xmpp.link or they
can deploy their own based on
https://github.com/modernxmpp/easy-xmpp-invitation
Alternatively they can just install mod_invites_page and this will all be
handled automatically by that.
All commands are called with a '-h' argument, but this one doesn't have
that. Since it's meant to be machine readable, hiding it seems
marginally more sensible than implementing '-h'.
The word 'error' anywhere, especially in harmless debug messages, are
too often interpreted as fatal errors my some users, so best avoid that
word. These look too scary as it is, being tracebacks.
This improves consistency. Previously the 'source' field was only
provided in the original event when an item was added. It is used to
report the name of the module providing the item in a few places.
Also considered adding a new API to modulemanager returning a mapping
of items per module and then using that here.
The field `_provided_by` comes from module:provides(), but these items
comes from moduel:add_item(), which include the originating module as a
'source' field of the event. However, this is absent when items are
retrieved at a later time than the initial event.
