Commit graph

12491 commits

Author SHA1 Message Date
Matthew Wild
4dcabd36ce mod_websocket: Fire pre-session-close event (fixes #1800)
This event was added in a7c183bb4e64 and is required to make mod_smacks know
that a session was intentionally closed and shouldn't be hibernated (see
fcea4d9e7502).

Because this was missing from mod_websocket's session.close(), mod_smacks
would always attempt to hibernate websocket sessions even if they closed
cleanly.

That mod_websocket has its own copy of session.close() is something to fix
another day (probably not in the stable branch). So for now this commit makes
the minimal change to get things working again.

Thanks to Damian and the Jitsi team for reporting.
2023-02-20 18:10:15 +00:00
Matthew Wild
87ee5a2300 net.http.parser: Fix off-by-one error in chunk parser 2023-02-17 17:01:19 +00:00
Kim Alvefur
67ea0ee50e mod_admin_socket: Return error on unhandled input to prevent apparent freeze
When mod_admin_socket is loaded without mod_admin_shell, attempt to use
`prosodyctl shell` will appear to freeze after any input, since no
response is returned.
2023-02-16 17:20:09 +01:00
Matthew Wild
bc1e51eb83 net.http.server: Add new API to get HTTP request from a connection
This information is sometimes necessary in the context where we have a
connection that we know (or believe to be) associated with an incoming HTTP
request.

For example, it can be used to retrieve the IP address of a request (which may
differ from the IP address of the connection, due to X-Forwarded-For and co).

Thanks to the Jitsi team for highlighting this gap in the API.
2023-02-16 15:59:26 +00:00
Matthew Wild
0fd88a07a4 net.http.parser: Improve handling of responses without content-length
This ensures that we support responses without a content-length header, and
allow streaming them through the streaming handler interface. An example of
such a response would be Server-Sent Events streams.
2023-02-09 22:57:32 +00:00
Matthew Wild
28e3b19ca1 net.http: Add missing log parameter 2023-02-09 22:37:54 +00:00
Matthew Wild
a4556fc67a sessionmanager: Improve logging around session destruction 2023-02-09 15:09:03 +00:00
Kim Alvefur
d616d70f3c core.sessionmanager: Mark session as destroyed to prevent reentry
If it reaches this point, then the session will be most definitely be
destroyed, so try to prevent destroy_session() from being called again.
2023-01-19 21:40:09 +01:00
Kim Alvefur
b89b219406 core.sessionmanager: Skip log when (not) destroying destroyed sessions
On regular disconnects, </stream> is sent, then
sessionmanager.destroy_session() is called, then
sessionmanager.destroy_session() is called again when the TCP connection
is closed, from ondisconnect in mod_c2s.

It is a bit annoying and doesn't really tell you much.
2023-01-19 21:38:14 +01:00
Kim Alvefur
435e008568 util.startup: Close state on exit to ensure GC finalizers are called
Ensures a last round of garbage collection and that finalizers are
called. Fixes things like proper closing of SQLite3 state.

There are more calls to os.exit() but most of them exit with an error or
in a case where a final GC sweep might not matter as much.

It would be nice if this was the default.

Calling util.statup.exit() everywhere may be sensible, but would be more
involved, requiring imports everywhere.
2023-01-22 14:45:47 +01:00
Kim Alvefur
7c302e9c4c util.prosodyctl.shell: Close state on exit to fix saving shell history
This ensures a last round of garbage collection and finalizers, which
should include flushing the readline history file.

Test procedure:
```
$ ./prosodyctl shell
prosody> s2s:show() -- any command that is not the last in history
... output
prosody> bye
$ ./prosodyctl shell
prosody> ^P
```

After this, the shell prompt should contain the last command from before
the "bye". Before this patch, recent history is gone most of the time.
2023-01-22 14:42:07 +01:00
Kim Alvefur
81fd03356f mod_muc_mam: Copy "include total" behavior from mod_mam
Not sure why this was missing from MUC MAM, it already had some of the
code for dealing with it.
2023-01-21 17:14:55 +01:00
Kim Alvefur
d0321442c9 mod_muc_mam: Add mam#extended form fields #1796 (Thanks Rain)
Oversight in cabb022f31c0
2023-01-21 17:09:22 +01:00
Kim Alvefur
4830568435 mod_mam,mod_muc_mam: Minimize differences (reorder, copy some comments)
Should have no functional difference, but makes it easier keeping
mod_mam and mod_muc_mam in sync.
2023-01-21 16:54:43 +01:00
Kim Alvefur
67b6440d9b mod_smacks: Log something when hibernation starts
Will hopefully save future confusion about sessions being destroyed when
they are in fact not.
2023-01-19 20:59:28 +01:00
Kim Alvefur
c9fb0c2cab prosodyctl check dns: Check for Direct TLS SRV records even if not configured (fix #1793)
Existing such records may cause timeouts or errors in clients and
servers trying to connect, despite prosodyctl check saying all is well
2023-01-14 05:47:47 +01:00
Kim Alvefur
b18280d35f mod_invites: Prefer landing page over xmpp URI in shell command
To mirror behavior of prosodyctl invocation
2023-01-10 21:10:58 +01:00
Kim Alvefur
fe97c773a6 net.http.server: Fix #1789
Unregistering the response before sending the trailer of the chunked
transfer encoding prevents opportunistic writes from being invoked and
running this code again when, which may cause an error when closing the
file handle a second time.

Normally the file size is known, so no chuck headers are sent.
2023-01-08 13:35:04 +01:00
Kim Alvefur
91a064bd49 mod_storage_sql: Fix #1639
Patch by Peter Kieser
2023-01-01 16:20:58 +01:00
Matthew Wild
e853a1652b configure: Fix quoting of $LUA_SUFFIX (thanks shellcheck/Zash)
Transplanted 6b43bf85032b from trunk
2022-12-29 16:29:03 +00:00
Kim Alvefur
2317f6a09f mod_storage_sql: Don't avoid initialization under prosodyctl (fix #1787)
Fixes `prosodyctl adduser` etc.

Prior to d580e6a57cbb the line did nothing.

Sometimes storage in the prosodyctl context does cause weirdness, as it
is not in a host context, but rather a variant of global.
2022-12-29 18:06:35 +01:00
Kim Alvefur
baff85a52c Tag 0.12.2 2022-12-12 20:40:23 +01:00
Kim Alvefur
63fde85044 Merge 0.11->0.12 2022-12-12 07:07:13 +01:00
Kim Alvefur
caf2e27362 Tag 0.11.14 2022-12-12 07:03:31 +01:00
Kim Alvefur
4fa3808e8d util.stanza: Allow U+7F
Allowed by XML despite arguably being a control character.

Drops the part of the range meant to rule out octets invalid in UTF-8
(\247 starts a 4-byte sequence), since UTF-8 correctness is validated by
util.encodings.utf8.valid().
2022-11-22 23:56:01 +01:00
Kim Alvefur
d158455de6 mod_smacks: Disable resumption behavior on s2s
Since resumption is not supported on s2s currently, there is no point in
allocating resumption tokens.  The code that removes entries from
session_registry is only invoked for c2s sessions, thus enabling
resumable smacks on s2s adds an entry that never goes away.
2022-11-13 18:16:59 +01:00
Matthew Wild
d4daf40caa util.json: Accept empty arrays with whitespace (fixes #1782) 2022-11-04 12:26:10 +00:00
Matthew Wild
246d797f81 mod_http: Allow disabling CORS in the http_cors_override option and by default
Fixes #1779.

Due to an oversight in the logic, if the user set 'enabled' to false in an
override, it would disable the item's requested CORS settings, but still apply
Prosody's default CORS policy.

This change ensures that 'enabled = false' will now disable CORS entirely for
the requested item.

Due to the new structure of the code, it was necessary to have a flag to say
whether CORS is to be applied at all. Rather than hard-coding 'true' here, I
chose to add a new option: 'http_default_cors_enabled'. This is a boolean that
allows the operator to disable Prosody's default CORS policy entirely (the one
that is used when a module or config does not override it). This makes it
easier to disable CORS and then selectively enable it only on services you
want it on.
2022-10-31 14:32:02 +00:00
Kim Alvefur
20532df525 mod_admin_shell: Ensure available connection for column 'secure'
Similar to #1777
2022-10-23 16:22:12 +02:00
Kim Alvefur
c51152432a mod_admin_shell: Ensure connection exists to get port from (fixes #1777) 2022-10-23 16:19:39 +02:00
Kim Alvefur
bab0635179 util.startup: Ensure import() is available in prosodyctl (thanks keyzer)
Fixes error in mod_authz_internal due to import() being unavailable as
it was only loaded in Prosody proper
2022-10-20 16:31:18 +02:00
Kim Alvefur
bca11bfc73 util.jsonschema: Use same integer/float logic on Lua 5.2 and 5.3
Fixes test case type.json:0:1 covering treatment of 1.0 as an integer
according to the JSON definition
2022-10-09 15:42:25 +02:00
Kim Alvefur
060dc12013 util.jsonschema: Fix Lua 5.2 integer compat
math.type() is unavailable before Lua 5.3 so this should use the compat
function added at the top
2022-10-09 15:38:36 +02:00
Kim Alvefur
c617f27a54 util.jsonschema: Ignore test case for JavaScript specific detail
Also touching on how arrays are indistinguishable from tables in Lua
2022-10-09 15:24:30 +02:00
Kim Alvefur
8f850047e3 util.jsonschema: Ignore some further test cases for URI references
Full-URI references are not implemented
2022-10-09 15:24:26 +02:00
Kim Alvefur
1e74554e20 util.jsonschema: Sort test cases to skip
Piped trough `sort -g`
2022-10-09 14:58:07 +02:00
Kim Alvefur
f6727510da util.stanza: Return nil instead of nothing (fix test with luassert >=1.9)
Due to a change in luassert, a dependency luassert of the Busted test
framework, returning nothing is no longer treated as not falsy.
2022-09-15 11:05:21 +02:00
Kim Alvefur
e38e915ef5 mod_s2s: Fix firing buffer drain events
Fixes the same kind of issue as in 65563530375b but once and for all,
while improving similarity between incoming and outgoing connections.
2022-08-26 18:53:00 +02:00
Kim Alvefur
ee21d9480a mod_admin_shell: Switch names for user role management commands
user:roles() does not convey that this is the mutating command, it
should have been called setroles from the start but wasn't due to lack
of foresight. This has to accidentally removing roles when wanting to
show them.
2022-08-15 18:56:22 +02:00
Kim Alvefur
2dbbce2382 mod_storage_sql: Fix summary API with Postgres (fixes #1766)
The ORDER BY and LIMIT clauses are not needed and don't even make much
sense. This part was most likely a leftover from the :find method.

Tested with sqlite and postgres 14
2022-07-22 19:09:50 +02:00
Kim Alvefur
3ee6df83b0 storage tests: Add test for the archive:summary API
Passes with memory, internal, sqlite
Fails with postgres as in #1766
2022-07-22 18:32:45 +02:00
Kim Alvefur
78e7707791 mod_http_files: Log warning about legacy modules using mod_http_files
It is time. Most community modules should have been adjusted to work
with the new (net.http.files) way.

At some point this usage should be prevented.

Related to #1765
2022-07-17 17:05:28 +02:00
Kim Alvefur
96301b2229 mod_bookmarks: Reduce error about not having bookmarks to debug (thanks tom)
This is happens if the account is new and doesn't have any bookmarks
yet, which is not a problem.

Rarely seen since most clients currently use the older version of
XEP-0084 stored in XEP-0049 rather than in PEP, but at least one
(Converse.js )does.

One scenario in which this would show up often is with Converse.js as a
guest chat using anonymous authentication, where all "accounts" would
always be new and not have any bookmarks. This scenario probably does
not need to have mod_bookmarks at all, but if enabled globally it would
likely become loaded onto the VirtualHost unless explicitly disabled.
2022-07-26 23:44:33 +02:00
Kim Alvefur
9f51add3a8 mod_storage_sql: Fix bypass of load procedure under prosodyctl
There's no 'prosody.prosodyctl' property other than this one, introduced
in 6216743c188c in 2015.

Guessing that the intent was to skip this when running as a prosodyctl
command. The module.command code does its own version of this
initialization, so this seems likely.

Thanks raja for noticing
2022-07-26 00:39:16 +02:00
Kim Alvefur
814817ebf2 core.s2smanager: Don't remove unrelated session on close of bidi session
Normally with bidi, any outgoing connection should be the same as the
incoming, hence when closing a bidi connection it should be removed as a
route to the remote server. However it is not guaranteed, a remote bidi-capable server
might have decided to open a new connection for some reason. This can
lead to a situation where there are two bidi connections, and the s2sout
route is a locally initiated s2sout connection. In this case, such a
s2sout connection should be kept.

Noticed in a rare case where bidi has just been enabled on a running
server, and something establishes new connections immediately when a
connection is closed.
2022-06-27 01:22:36 +02:00
Kim Alvefur
1261dfba9f luacheck: Shut up (backports 3caff1f93520, ignores module deleted in trunk) 2022-05-30 17:34:58 +02:00
Kim Alvefur
7f2a660158 Backport 875f73ead4e8 8e4033213c62 to deal with luacheck 0.26 2022-07-08 19:42:48 +02:00
Kim Alvefur
89359b70dc util.datamapper: Improve handling of schemas with non-obvious "type"
The JSON Schema specification says that schemas are objects or booleans,
and that the 'type' property is optional and can be an array.

This module previously allowed bare type names as schemas and did not
really handle booleans.

It now handles missing 'type' properties and boolean 'true' as a schema.
Objects and arrays are guessed based on the presence of 'properties' or
'items' field.
2022-07-08 17:32:48 +02:00
Kim Alvefur
e700edc50f util.jsonschema: Fix validation to not assume presence of "type" field
MattJ reported a curious issue where validation did not work as
expected. Primarily that the "type" field was expected to be mandatory,
and thus leaving it out would result in no checks being performed.
This was likely caused by misreading during initial development.

Spent some time testing against
https://github.com/json-schema-org/JSON-Schema-Test-Suite.git and
discovered a multitude of issues, far too many to bother splitting into
separate commits.

More than half of them fail. Many because of features not implemented,
which have been marked NYI. For example, some require deep comparisons
e.g. when objects or arrays are present in enums fields.

Some because of quirks with how Lua differs from JavaScript, e.g. no
distinct array or object types. Tests involving fractional floating
point numbers. We're definitely not going to follow references to remote
resources. Or deal with UTF-16 sillyness. One test asserted that 1.0 is
an integer, where Lua 5.3+ will disagree.
2022-07-08 14:38:23 +02:00
Kim Alvefur
4f3ba05b4d net.unbound: Merge luaunbound and prosody defaults in absence of user config (fixes #1763) (thanks rgd)
add_defaults() is supposed to merge 3 tables, the defaults in
luaunbound, the defaults from prosody and any config from the prosody
config file.  In the case where no `unbound={}` has been in the config,
it skips over the merge and returns only the prosody built-in defaults.

This results in libunbound skipping reading resolv.conf and uses its
default behavior of full recursive resolution.

Prior to #1737 there were only two tables, the luaunbound defaults and
the prosody config, where bypassing the merge and returning the former
did the right thing.
2022-06-19 19:49:32 +02:00