mirrors/prosody
1
0
Fork 0
mirror of https://github.com/bjc/prosody.git synced 2025-04-05 22:27:38 +03:00
Code Issues Projects Releases Packages Wiki Activity Actions
11858 commits 12 branches 101 tags 40 MiB
Commit graph

64 commits

Author SHA1 Message Date
Kim Alvefur
aed2ada774 mod_http_file_share: Allow 'Authorization' header via CORS (thanks kawaii) 
Can't find anything saying anything on whether this is needed or not.
kawaii reported that both Chrome and Firefox complained unless the
header was added to the list of allowed headers.
 2021-10-20 23:06:07 +02:00
Jonas Schäfer
3e55057a85 mod_http_file_share: return 401 instead of 403 if authentication failed 
This is as per the HTTP standards [1]. Thankfully, the REQUIRED
www-authenticate header is already generated by the code.

   [1]: https://datatracker.ietf.org/doc/html/rfc7235#section-3.1
 2021-10-19 16:37:32 +02:00
Kim Alvefur
72e0ff88ba mod_http_file_share: Fix measuring how long periodic task take 2021-09-14 15:33:39 +02:00
Kim Alvefur
9af7bb723b mod_http_file_share: Measure current total usage 
In order to allow monitoring. Especially as there's not much in the way
of hard numbers on how much space gets used.
 2021-09-12 17:03:02 +02:00
Kim Alvefur
5392ef1dd6 mod_http_file_share: Fix traceback in global quota debug logging (thanks Martin) 
Error in util.human.units.format because of B(nil) when the global quota
is unset.
 2021-09-12 11:47:22 +02:00
Kim Alvefur
ae7f6c34f0 mod_http_file_share: Add optional global quota on total storage usage 
Before, maximum storage usage (assuming all users upload as much as they
could) would depend on the quota, retention period and number of users.
Since number of users can vary, this makes it hard to know how much
storage will be needed.

Adding a limit to the total overall storage use solves this, making it
simple to set it to some number based on what storage is actually
available.

Summary job run less often than the prune job since it touches the
entire archive; and started before the prune job since it's needed
before the first upload.
 2021-09-12 01:38:33 +02:00
Kim Alvefur
1904899421 mod_http_file_share: Update comment about x-frame-options 
X-Frame-Options was replaced by the Content-Security-Policy
'frame-ancestors' directive, but Internet Explorer does not support that
part of CSP.

Since it's just one line it doesn't hurt to keep until some future
spring cleaning event :)
 2021-06-20 16:30:24 +02:00
Kim Alvefur
2f7da2c6ea mod_http_file_share: Build list of measuring buckets for configured size limit 
Creates buckets up to the configured size limit or 1TB, whichever is
smaller, e.g. {1K, 4K, 16K, ... 4M, 16M}
 2021-06-08 13:33:40 +02:00
Kim Alvefur
a854ec1e4c mod_http_file_share: Handle out of bounds Range request 
Turns out you can seek past the end of the file without getting an
error.

Also rejects empty range instead of sending the whole file.
 2021-05-17 14:14:25 +02:00
Kim Alvefur
f5757cae7f mod_http_file_share: Support download resumption via Range requests 
Only a starting point is supported due to the way response:send_file()
sends everything it gets from the provided file handle but does not have
any way to specify how much to read.

This matches what Conversations appears to be doing.
 2021-05-16 16:52:59 +02:00
Kim Alvefur
ecd6b31eea mod_http_file_share: Include expiry time of the upload itself in token 
Lets an external upload service know this so it can do expiry itself.
Could possibly have been calculated based on the token expiry or
issuance time, explicit > implicit.
 2021-04-05 17:21:18 +02:00
Kim Alvefur
e590c9c92e mod_http_file_share: Include time of issuance in auth token 
In case an external upload service wants to have the original creation
time, or calculate the token expiry itself.
 2021-04-05 17:16:18 +02:00
Kim Alvefur
bb06c42b07 mod_http_file_share: Group related properties for readability 2021-04-05 17:15:12 +02:00
Kim Alvefur
f7131f81f2 mod_http_file_share: Log error opening file for writing 
util.error.coerce() doesn't work well with iolib
 2021-04-05 16:24:39 +02:00
Kim Alvefur
de641f6e24 mod_http_file_share: Fix logging of error opening file 
It's annoying that Lua interpolates the filename into the error message.
 2021-04-05 16:23:59 +02:00
Kim Alvefur
d8613bb7b9 mod_http_file_share: Include storage in prune time measurement 
Maybe the original idea was that you would measure storage separately?
 2021-03-31 09:43:05 +02:00
Kim Alvefur
29a56e7747 mod_http_file_share: Add internal command to check files consistency 
Background: Found a few files in my store that did not match the size
recorded in the slot, so I needed a way to check which which those were.
As it was a bit too much to type into the shell I added it here instead.
 2021-03-28 15:57:15 +02:00
Kim Alvefur
1be4a5c297 mod_http_file_share: Fix reporting of missing files 
This just gave an unhelpful 500 error.

It would be nice to have some wrapper code that could untangle the
embedded filename in the io libs errors.
 2021-03-28 13:51:06 +02:00
Kim Alvefur
9631d45fcc mod_http_file_share: Report number of items in caches to statsmanager 
This is neat, O(1) reporting, why don't we do this everywhere?
Gives you an idea of how much stuff is in the caches, which may help
inform decisions on whether the size is appropriate.
 2021-03-28 13:15:11 +02:00
Kim Alvefur
dfaf43cc3f mod_http_file_share: Skip removal of nothing 
In case none of the expired files could be deleted then it's a waste of
an API call to try to remove any of the metadata at all.
 2021-02-26 00:20:24 +01:00
Kim Alvefur
a2648b9d64 mod_http_file_share: Remove correct entries when not all expired files were deleted 
If any of the expired files could not be deleted then we should not
forget about that, we should complain loudly and try again.

The code got this backwards and would have removed only the entries
referring to still existing files.

Test procedure:
1.  Upload a file
2.  chown root:root http_file_share/
3.  In uploads.list, decrease 'when' enough to ensure expiry
4.  Reload mod_http_file_share
5.  Should see an error in the logs about failure to delete the file
6.  Should see that the metadata in uploads.list is still there
7.  chown http_file_share/ back to the previous owner
8.  Reload mod_http_file_share
9.  Should see logs about successful removal of expired file
10. Should see that the metadata in uploads.list is gone
11. Should see that the file was deleted
 2021-02-25 23:58:08 +01:00
Kim Alvefur
0a5ac437b9 mod_http_file_share: Fix traceback on missing file-type 
attempt to index a nil value (local 'filetype') casued by the :gsub call
 2021-02-24 16:35:56 +01:00
Kim Alvefur
b01915e81c mod_http_file_share: Allow credentials via CORS (needed for auth token) 2021-02-23 02:56:49 +01:00
Kim Alvefur
900fe6a1cb mod_http_file_share: Return a message from the base URL 2021-02-21 06:49:11 +01:00
Kim Alvefur
3151984042 mod_http_file_share: Prevent attempt to upload again after completion 2021-02-13 14:14:12 +01:00
Kim Alvefur
9501c7b976 mod_http_file_share: Support sending 100 Continue 
E.g. curl will ask for this when sending large uploads. Removes a delay
while it waits for an error or go-agead.
 2021-02-13 14:06:46 +01:00
Kim Alvefur
cdbb77ccbf mod_http_file_share: Collect cache hit/miss statistics for downloads 2021-02-02 22:16:20 +01:00
Kim Alvefur
e45dee37bd mod_http_file_share: Measure how long it takes to prune expired files 2021-02-02 22:11:53 +01:00
Kim Alvefur
da5cd9c963 mod_http_file_share: Collect statistics of files uploaded 2021-02-02 22:08:51 +01:00
Kim Alvefur
5d9802727b mod_http_file_share: Reorder arguments 
'filetype' is optional, so having it last seems sensible.
'slot' is pretty important, so moving it earlier seems sensible.
 2021-01-31 20:38:40 +01:00
Kim Alvefur
f5baa83714 mod_http_file_share: Update cached value while it is reasonably fresh 
This should ensure that cache entries until the oldest file that counted
to the last 24h becomes older than 24h.
 2021-01-31 17:44:19 +01:00
Kim Alvefur
864708efb7 mod_http_file_share: Cache quotas to avoid hitting storage 2021-01-31 17:56:49 +01:00
Kim Alvefur
855542e2ca mod_http_file_share: Split out some variables for later reuse 2021-01-31 17:56:35 +01:00
Kim Alvefur
4d3ffbf0f7 mod_http_file_share: Add support for daily upload quotas. 
Daily instead of total quotas, should be more efficient to calculate.
Still O(n), but a smaller n. Less affected by total retention period.
 2021-01-31 14:43:42 +01:00
Kim Alvefur
b1b3f09a5c mod_http_file_share: Add missing semicolon 
Last comma or semicolon isn't required but makes the diffs nicer once
you add another item after it.
 2021-01-31 14:41:56 +01:00
Kim Alvefur
1998e74fd4 mod_http_file_share: Clarify log message 
No expired ... what? Could be inferred from the module logging it, but
better to be explicit.
 2021-01-30 17:35:13 +01:00
Kim Alvefur
88ce2f1c94 mod_http_file_share: Indicate missing token via WWW-Authenticate header 2021-01-29 15:34:36 +01:00
Kim Alvefur
610278afa0 mod_http_file_share: Clarify message about missing Authorization header 2021-01-29 15:34:15 +01:00
Kim Alvefur
0289a561e6 mod_http_file_share: Allow started uploads to complete after token expired 
Otherwise uploads taking longer than 5 minutes would be rejected on
completion, and that's probably annoying.

Thanks jonas’
 2021-01-28 17:24:37 +01:00
Kim Alvefur
fab6ebb1e6 mod_http_file_share: Serve configurable set of safe mime types inline (thanks jonas’) 
Otherwise people complain about browser 'Save as' dialog.
 2021-01-28 17:13:49 +01:00
Kim Alvefur
22e4d92ce5 mod_http_file_share: Cache file metadata 
For faster access by avoiding archive ID.

No benchmarks were harmed in the making of this commit.

... no benchmarks were performed at all.
 2021-01-28 16:34:13 +01:00
Kim Alvefur
318d357557 mod_http_file_share: Extract all file properties into variables earlier 
A step towards adding caching, which will unpack into the same
variables.
 2021-01-28 16:23:38 +01:00
Kim Alvefur
217450a727 mod_http_file_share: Fix the obligatory misplaced closing bracket (thanks scansion) 2021-01-27 23:38:14 +01:00
Kim Alvefur
2cff8727c7 mod_http_file_share: Insert pauses to avoid blocknig for long periods 
Similar to the mod_mam cleanup job
 2021-01-27 22:10:11 +01:00
Kim Alvefur
395de7cde5 mod_http_file_share: Add support for removing old files (default 2 weeks) 2021-01-27 22:09:26 +01:00
Kim Alvefur
da882b75ab mod_http_file_share: More security headers 2021-01-27 19:51:36 +01:00
Kim Alvefur
6ce4972bf4 mod_http_file_share: Strip authorization type prefix a bit earlier 2021-01-27 18:26:24 +01:00
Kim Alvefur
3abab1efc3 mod_http_file_share: Factor out function for generating full filename 2021-01-27 18:13:15 +01:00
Kim Alvefur
17a85e6d3b mod_http_file_share: Use '.bin' file extension 
Distinct from '.dat' used by datamanager / internal stortage for Lua
object storage so that they can't easily be loaded by accident that way.
 2021-01-27 17:48:12 +01:00
Kim Alvefur
e4a97b8e7a mod_http_file_share: Validate file size early in HTTP PUT request 2021-01-27 17:47:04 +01:00