write "format" section

This commit is contained in:
DarkCat09 2024-07-05 10:59:18 +04:00
parent 1d3b25e374
commit ea00650e42
Signed by: DarkCat09
GPG key ID: 0A26CD5B3345D6E3

View file

@ -4,7 +4,7 @@ Mitm-Archive consists of two parts:
- [Addon for mitmproxy](https://git.dc09.ru/mitm-archive/addon) intercepting and saving (archiving) all HTTP responses, written in Python
- [Server](https://git.dc09.ru/mitm-archive/server) giving exactly the same responses as in an archive for corresponding method+domain+port+path+query, written in Go
"Archive" is an SQLite3 database and a directory storing headers and body for each archived response. See Format section for details.
"Archive" is an SQLite3 database and a directory storing headers and body for each archived response. See [Format](#format) section for details.
# User guide: addon
@ -69,6 +69,9 @@ $ export STORAGE=storage
$ mitmproxy -s addon.py
```
# User guide: server
// TODO
# What's not implemented
- Filter host instead of archiving everything (literally 2 lines of code, could be added soon after I figure out the best way to configure this)
- Addon is configured with env vars, Server uses command-line options; should be unified?
@ -89,3 +92,31 @@ Harder to implement and definitely will overcomplicate the project while neither
For these usage screnarios, especially with cookies, it's simplier and overall better
to self-host the web site server you are trying to archive
or re-implement it in your favourite programming language and self-host.
# Format
SQLite3 database contains `data` table with the following columns:
- `id` - integer primary key for each archived response
- `method` - string, specifies the request method, default `"GET"`
- `url` - string, URL formatted as `$scheme://$host:$port$path$query` (e.g. `https://dc09.ru:443/path?key=val`), required
- `code` - integer, HTTP response status code, default `200`
INSERT query is executed with `RETURNING id` clause.
In file system storage, the addon creates a directory (if not exists) with the numeric ID returned by SQLite as its name,
writes raw binary body data without any modifications to `{id}/body` file,
writes headers in HTTP/1 format (`name: value\r\n`) to `{id}/headers` file.
The FS storage structure can be represented graphically this way:
```
storage/
|- 1/
| |- headers
| |- body
|
|- 2/
| |- headers
| |- body
|
|- {id}/
...
```