mitm-archive/docs
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

write "format" section

Browse source
This commit is contained in:
DarkCat09 2024-07-05 10:59:18 +04:00
parent 1d3b25e374
commit ea00650e42
Signed by: DarkCat09
GPG key ID: 0A26CD5B3345D6E3
1 changed files with 32 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

33
README.md
View file

@ -4,7 +4,7 @@ Mitm-Archive consists of two parts:
- [Addon for mitmproxy](https://git.dc09.ru/mitm-archive/addon) intercepting and saving (archiving) all HTTP responses, written in Python - [Addon for mitmproxy](https://git.dc09.ru/mitm-archive/addon) intercepting and saving (archiving) all HTTP responses, written in Python
- [Server](https://git.dc09.ru/mitm-archive/server) giving exactly the same responses as in an archive for corresponding method+domain+port+path+query, written in Go - [Server](https://git.dc09.ru/mitm-archive/server) giving exactly the same responses as in an archive for corresponding method+domain+port+path+query, written in Go
"Archive" is an SQLite3 database and a directory storing headers and body for each archived response. See Format section for details. "Archive" is an SQLite3 database and a directory storing headers and body for each archived response. See [Format](#format) section for details.
# User guide: addon # User guide: addon
@ -69,6 +69,9 @@ $ export STORAGE=storage
$ mitmproxy -s addon.py $ mitmproxy -s addon.py
``` ```
# User guide: server
// TODO
# What's not implemented # What's not implemented
- Filter host instead of archiving everything (literally 2 lines of code, could be added soon after I figure out the best way to configure this) - Filter host instead of archiving everything (literally 2 lines of code, could be added soon after I figure out the best way to configure this)
- Addon is configured with env vars, Server uses command-line options; should be unified? - Addon is configured with env vars, Server uses command-line options; should be unified?
@ -89,3 +92,31 @@ Harder to implement and definitely will overcomplicate the project while neither
For these usage screnarios, especially with cookies, it's simplier and overall better For these usage screnarios, especially with cookies, it's simplier and overall better
to self-host the web site server you are trying to archive to self-host the web site server you are trying to archive
or re-implement it in your favourite programming language and self-host. or re-implement it in your favourite programming language and self-host.
# Format
SQLite3 database contains `data` table with the following columns:
- `id` - integer primary key for each archived response
- `method` - string, specifies the request method, default `"GET"`
- `url` - string, URL formatted as `$scheme://$host:$port$path$query` (e.g. `https://dc09.ru:443/path?key=val`), required
- `code` - integer, HTTP response status code, default `200`
INSERT query is executed with `RETURNING id` clause.
In file system storage, the addon creates a directory (if not exists) with the numeric ID returned by SQLite as its name,
writes raw binary body data without any modifications to `{id}/body` file,
writes headers in HTTP/1 format (`name: value\r\n`) to `{id}/headers` file.
The FS storage structure can be represented graphically this way:
```
storage/
|- 1/
| |- headers
| |- body
|
|- 2/
| |- headers
| |- body
|
|- {id}/
...
```