mirrors/prosody
1
0
Fork 0
mirror of https://github.com/bjc/prosody.git synced 2025-04-04 13:47:41 +03:00
Code Issues Projects Releases Packages Wiki Activity Actions
13750 commits 12 branches 101 tags 40 MiB
Commit graph

100 commits

Author SHA1 Message Date
Kim Alvefur
de641f6e24 mod_http_file_share: Fix logging of error opening file 
It's annoying that Lua interpolates the filename into the error message.
 2021-04-05 16:23:59 +02:00
Kim Alvefur
d8613bb7b9 mod_http_file_share: Include storage in prune time measurement 
Maybe the original idea was that you would measure storage separately?
 2021-03-31 09:43:05 +02:00
Kim Alvefur
29a56e7747 mod_http_file_share: Add internal command to check files consistency 
Background: Found a few files in my store that did not match the size
recorded in the slot, so I needed a way to check which which those were.
As it was a bit too much to type into the shell I added it here instead.
 2021-03-28 15:57:15 +02:00
Kim Alvefur
1be4a5c297 mod_http_file_share: Fix reporting of missing files 
This just gave an unhelpful 500 error.

It would be nice to have some wrapper code that could untangle the
embedded filename in the io libs errors.
 2021-03-28 13:51:06 +02:00
Kim Alvefur
9631d45fcc mod_http_file_share: Report number of items in caches to statsmanager 
This is neat, O(1) reporting, why don't we do this everywhere?
Gives you an idea of how much stuff is in the caches, which may help
inform decisions on whether the size is appropriate.
 2021-03-28 13:15:11 +02:00
Kim Alvefur
dfaf43cc3f mod_http_file_share: Skip removal of nothing 
In case none of the expired files could be deleted then it's a waste of
an API call to try to remove any of the metadata at all.
 2021-02-26 00:20:24 +01:00
Kim Alvefur
a2648b9d64 mod_http_file_share: Remove correct entries when not all expired files were deleted 
If any of the expired files could not be deleted then we should not
forget about that, we should complain loudly and try again.

The code got this backwards and would have removed only the entries
referring to still existing files.

Test procedure:
1.  Upload a file
2.  chown root:root http_file_share/
3.  In uploads.list, decrease 'when' enough to ensure expiry
4.  Reload mod_http_file_share
5.  Should see an error in the logs about failure to delete the file
6.  Should see that the metadata in uploads.list is still there
7.  chown http_file_share/ back to the previous owner
8.  Reload mod_http_file_share
9.  Should see logs about successful removal of expired file
10. Should see that the metadata in uploads.list is gone
11. Should see that the file was deleted
 2021-02-25 23:58:08 +01:00
Kim Alvefur
0a5ac437b9 mod_http_file_share: Fix traceback on missing file-type 
attempt to index a nil value (local 'filetype') casued by the :gsub call
 2021-02-24 16:35:56 +01:00
Kim Alvefur
b01915e81c mod_http_file_share: Allow credentials via CORS (needed for auth token) 2021-02-23 02:56:49 +01:00
Kim Alvefur
900fe6a1cb mod_http_file_share: Return a message from the base URL 2021-02-21 06:49:11 +01:00
Kim Alvefur
3151984042 mod_http_file_share: Prevent attempt to upload again after completion 2021-02-13 14:14:12 +01:00
Kim Alvefur
9501c7b976 mod_http_file_share: Support sending 100 Continue 
E.g. curl will ask for this when sending large uploads. Removes a delay
while it waits for an error or go-agead.
 2021-02-13 14:06:46 +01:00
Kim Alvefur
cdbb77ccbf mod_http_file_share: Collect cache hit/miss statistics for downloads 2021-02-02 22:16:20 +01:00
Kim Alvefur
e45dee37bd mod_http_file_share: Measure how long it takes to prune expired files 2021-02-02 22:11:53 +01:00
Kim Alvefur
da5cd9c963 mod_http_file_share: Collect statistics of files uploaded 2021-02-02 22:08:51 +01:00
Kim Alvefur
5d9802727b mod_http_file_share: Reorder arguments 
'filetype' is optional, so having it last seems sensible.
'slot' is pretty important, so moving it earlier seems sensible.
 2021-01-31 20:38:40 +01:00
Kim Alvefur
f5baa83714 mod_http_file_share: Update cached value while it is reasonably fresh 
This should ensure that cache entries until the oldest file that counted
to the last 24h becomes older than 24h.
 2021-01-31 17:44:19 +01:00
Kim Alvefur
864708efb7 mod_http_file_share: Cache quotas to avoid hitting storage 2021-01-31 17:56:49 +01:00
Kim Alvefur
855542e2ca mod_http_file_share: Split out some variables for later reuse 2021-01-31 17:56:35 +01:00
Kim Alvefur
4d3ffbf0f7 mod_http_file_share: Add support for daily upload quotas. 
Daily instead of total quotas, should be more efficient to calculate.
Still O(n), but a smaller n. Less affected by total retention period.
 2021-01-31 14:43:42 +01:00
Kim Alvefur
b1b3f09a5c mod_http_file_share: Add missing semicolon 
Last comma or semicolon isn't required but makes the diffs nicer once
you add another item after it.
 2021-01-31 14:41:56 +01:00
Kim Alvefur
1998e74fd4 mod_http_file_share: Clarify log message 
No expired ... what? Could be inferred from the module logging it, but
better to be explicit.
 2021-01-30 17:35:13 +01:00
Kim Alvefur
88ce2f1c94 mod_http_file_share: Indicate missing token via WWW-Authenticate header 2021-01-29 15:34:36 +01:00
Kim Alvefur
610278afa0 mod_http_file_share: Clarify message about missing Authorization header 2021-01-29 15:34:15 +01:00
Kim Alvefur
0289a561e6 mod_http_file_share: Allow started uploads to complete after token expired 
Otherwise uploads taking longer than 5 minutes would be rejected on
completion, and that's probably annoying.

Thanks jonas’
 2021-01-28 17:24:37 +01:00
Kim Alvefur
fab6ebb1e6 mod_http_file_share: Serve configurable set of safe mime types inline (thanks jonas’) 
Otherwise people complain about browser 'Save as' dialog.
 2021-01-28 17:13:49 +01:00
Kim Alvefur
22e4d92ce5 mod_http_file_share: Cache file metadata 
For faster access by avoiding archive ID.

No benchmarks were harmed in the making of this commit.

... no benchmarks were performed at all.
 2021-01-28 16:34:13 +01:00
Kim Alvefur
318d357557 mod_http_file_share: Extract all file properties into variables earlier 
A step towards adding caching, which will unpack into the same
variables.
 2021-01-28 16:23:38 +01:00
Kim Alvefur
217450a727 mod_http_file_share: Fix the obligatory misplaced closing bracket (thanks scansion) 2021-01-27 23:38:14 +01:00
Kim Alvefur
2cff8727c7 mod_http_file_share: Insert pauses to avoid blocknig for long periods 
Similar to the mod_mam cleanup job
 2021-01-27 22:10:11 +01:00
Kim Alvefur
395de7cde5 mod_http_file_share: Add support for removing old files (default 2 weeks) 2021-01-27 22:09:26 +01:00
Kim Alvefur
da882b75ab mod_http_file_share: More security headers 2021-01-27 19:51:36 +01:00
Kim Alvefur
6ce4972bf4 mod_http_file_share: Strip authorization type prefix a bit earlier 2021-01-27 18:26:24 +01:00
Kim Alvefur
3abab1efc3 mod_http_file_share: Factor out function for generating full filename 2021-01-27 18:13:15 +01:00
Kim Alvefur
17a85e6d3b mod_http_file_share: Use '.bin' file extension 
Distinct from '.dat' used by datamanager / internal stortage for Lua
object storage so that they can't easily be loaded by accident that way.
 2021-01-27 17:48:12 +01:00
Kim Alvefur
e4a97b8e7a mod_http_file_share: Validate file size early in HTTP PUT request 2021-01-27 17:47:04 +01:00
Kim Alvefur
93494930ea mod_http_file_share: Move Authorization type string 
It belongs with the header more than the token itself
 2021-01-27 17:34:48 +01:00
Kim Alvefur
943dc18486 mod_http_file_share: Add some logging 2021-01-27 17:29:26 +01:00
Kim Alvefur
5a99373c07 mod_http_file_share: Handle content-type being optional 2021-01-27 09:47:21 +01:00
Kim Alvefur
3ab51f6864 mod_http_file_share: Reject invalid file sizes 2021-01-27 00:36:49 +01:00
Kim Alvefur
debea1badc mod_http_file_share: Fix name of max-file-size tag 2021-01-27 00:29:12 +01:00
Kim Alvefur
78b0c08aac mod_http_file_share: Expand registry to fix extra tag 
Error registry compact format doesn't support extra.tag so needs to be
the more verbose format
 2021-01-27 00:28:42 +01:00
Kim Alvefur
987976f20f mod_http_file_share: Verify final file size on completion of upload 2021-01-27 00:09:49 +01:00
Kim Alvefur
456ca909e2 mod_http_file_share: Add file type filter 
Unlike mod_http_upload, this can't be bypassed by uploading with a
different file extension.
 2021-01-26 14:53:43 +01:00
Kim Alvefur
a98f3a3e64 mod_http_file_share: Add file size limit (default 10M) 2021-01-26 14:53:24 +01:00
Kim Alvefur
ea3b09dea8 mod_http_file_share: Validate that filename does not contain '/' 2021-01-26 14:52:37 +01:00
Kim Alvefur
62f77b9ac9 mod_http_file_share: Return proper error if unauthorized 2021-01-26 14:51:55 +01:00
Kim Alvefur
112b174e2d mod_http_file_share: Add basic access control 
Partly lifted from mod_external_services
 2021-01-26 14:39:11 +01:00
Kim Alvefur
c26e8b4898 mod_http_file_share: Add support for external file upload service 
PUT /upload/:slot/:filename
Authorization: Bearer JWT{ uploader, filename, filesize, filetype, exp }
 2021-01-26 14:27:51 +01:00
Kim Alvefur
4be9b33741 mod_http_file_share: Let's write another XEP-0363 implementation 
This variant is meant to improve upon mod_http_upload in some ways:

* Handle files much of arbitrary size efficiently
* Allow GET and PUT URLs to be different
* Remember Content-Type sent by client
* Avoid dependency on mod_http_files
* Built-in way to delegate storage to another httpd
 2021-01-26 03:19:17 +01:00