docs: manual to reproduce, clarifications
This commit is contained in:
parent
1aecbeab87
commit
b743d47f09
1 changed files with 63 additions and 15 deletions
78
README.md
78
README.md
|
@ -1,25 +1,54 @@
|
||||||
# sing-box vs xray
|
# sing-box vs xray
|
||||||
|
|
||||||
An attempt on benchmarking these two multi-protocol proxying frameworks.
|
An attempt on benchmarking these two multi-protocol proxying frameworks.
|
||||||
It's reproducible, configs are not redacted, I even uploaded self-signed TLS cert+key,
|
|
||||||
client commands are shown in `bench_*.txt` files.
|
|
||||||
All you need is iperf3 on client and server, hyperfine on client,
|
|
||||||
xray and sing-box binaries (take from "Releases" or compile by yourself) placed on client and server.
|
|
||||||
|
|
||||||
## In a nutshell
|
## In a nutshell
|
||||||
|
|
||||||
Results are inaccurate, they look like a statistical error, made by,
|
Results are inaccurate, they look like a statistical error, made by,
|
||||||
for example, different ISP's network load, that's why I said "an attempt on benchmarking".
|
for example, different ISP's network load, that's why I said "an attempt on benchmarking".
|
||||||
|
It would be great if someone with a powerful CPU and 1 Gbit at home could do the benchmarks.
|
||||||
|
|
||||||
Both proxies are almost the same speed. *Xray was sometimes a bit faster.*
|
Both proxies are almost the same speed. *Xray was sometimes a bit faster.*
|
||||||
But in iperf3 benchmark it often behaved weird: from 2nd packet speed was dropping to zero.
|
But in the iperf3 benchmark it very often behaved weird: from 2nd packet speed was dropping to zero
|
||||||
|
or I even got `iperf3: error - control socket has closed unexpectedly`.
|
||||||
|
|
||||||
So, use what you want / to what you already get used / what works better in your case.
|
So, use what you want / to what you already get used / what works better in your case.
|
||||||
As for me, I found sing-box' JSON config more convenient than Xray's.
|
As for me, I found sing-box' JSON config more convenient than Xray's.
|
||||||
On the other hand, Xray provides more features to hide a proxy,
|
SB supports many protocols and platforms, even can setup a TUN interface (like a VPN app).
|
||||||
|
On the other hand, Xray provides more "stealth" features to hide proxy traffic,
|
||||||
that is important, I guess (?), in China and Iran.
|
that is important, I guess (?), in China and Iran.
|
||||||
|
|
||||||
## Version
|
## To reproduce
|
||||||
|
|
||||||
|
- Get [sing-box](https://github.com/SagerNet/sing-box/releases/latest)
|
||||||
|
and [xray](github.com/XTLS/Xray-core/releases/latest) binaries
|
||||||
|
by downloading from "Releases" or compiling by yourself
|
||||||
|
- Compile [iperf3 fork with socks5 support](https://github.com/davidBar-On/iperf/tree/issue-1095-socks5-support)
|
||||||
|
by cloning git repo (don't forget that you need branch `issue-1095-socks5-support`, not master!)
|
||||||
|
and running `./configure && make` -- you'll get a built iperf in `./src/iperf3`
|
||||||
|
- Generate your own TLS cert (`cert.pem` and `key.pem` included in the repo are for `dc09.ru` domain name)
|
||||||
|
with `sing-box generate tls-keypair <insert domain here>` OR `xray tls cert --domain=<domain>`
|
||||||
|
- Replace `dc09.ru` in all configs to match your domain name instead of mine
|
||||||
|
- Upload sing-box and xray to your server, install iperf3 from a package manager
|
||||||
|
or upload compiled previously (on a server, you won't need socks support),
|
||||||
|
upload `server.json`, `server_xray.json`, `cert.pem` and `key.pem`
|
||||||
|
- Run `./sing-box run --config server.json &` on your server, then launch `iperf3 -s`
|
||||||
|
- In `config.json` edit the line `"final": "vless-out"` to default to `socks-out`,
|
||||||
|
the same for `config_xray.json`: edit `"outboundTag": "vless-out"` in the 2nd routing rule.
|
||||||
|
- Run `./sing-box run --config config.json` and `./xray run -c config_xray.json` on client,
|
||||||
|
sing-box will open port 2080 for a SOCKSv5 inbound, xray will open port 2081 for its inbound.
|
||||||
|
- Make tests with iperf3:
|
||||||
|
`repo_with_iperf_fork/src/iperf3 -c <address of your server or domain> --bidir --socks5 127.0.0.1:2080`
|
||||||
|
for sing-box client and `... --socks5 127.0.0.1:2081` for xray client.
|
||||||
|
- Change `"final": "socks-out"` and `"outboundTag": "socks-out"` to `trojan-out` to test with Trojan,
|
||||||
|
restart sing-box and xray on client, peform iperf3 tests,
|
||||||
|
then change default outbound back to `vless-out`, restart proxy clients again, peform tests
|
||||||
|
- Stop iperf3 server by hitting Ctrl-C, stop sing-box server proxy by bringing the task to foreground with `fg` command and hitting Ctrl-C
|
||||||
|
- Run `./xray run -c server_xray.json &` on the server, then launch `iperf3 -s`
|
||||||
|
- Repeat the tests
|
||||||
|
- Stop iperf3 with Ctrl-C, stop xray with `fg` and Ctrl-C
|
||||||
|
|
||||||
|
## Software versions used
|
||||||
|
|
||||||
sing-box built from dev-next branch, [26f092d](https://github.com/SagerNet/sing-box/commit/26f092da6fb0801b11c91fd5c8468e9949312e02)
|
sing-box built from dev-next branch, [26f092d](https://github.com/SagerNet/sing-box/commit/26f092da6fb0801b11c91fd5c8468e9949312e02)
|
||||||
```
|
```
|
||||||
|
@ -37,16 +66,32 @@ Xray 24.11.11 (Xray, Penetrates Everything.) 5a96ef6 (go1.23.2 linux/amd64)
|
||||||
A unified platform for anti-censorship.
|
A unified platform for anti-censorship.
|
||||||
```
|
```
|
||||||
|
|
||||||
## hyperfine, curl, direct outbound
|
Fork of iperf3 with socks support, run on client:
|
||||||
|
```
|
||||||
|
iperf 3.16+ (cJSON 1.7.15)
|
||||||
|
Linux dc09void 6.6.60_1 #1 SMP PREEMPT_DYNAMIC Mon Nov 11 21:45:58 UTC 2024 x86_64
|
||||||
|
Optional features available: CPU affinity setting, IPv6 flow label, TCP congestion algorithm setting, sendfile / zerocopy, socket pacing, authentication, bind to device, support IPv4 don't fragment, POSIX threads
|
||||||
|
```
|
||||||
|
|
||||||
|
Regular iperf3 from Alpine repo, run on server:
|
||||||
|
```
|
||||||
|
iperf 3.17.1 (cJSON 1.7.15)
|
||||||
|
Linux mx1.dc09.ru 6.6.61-0-virt #1-Alpine SMP PREEMPT_DYNAMIC 2024-11-14 20:10:08 x86_64
|
||||||
|
Optional features available: CPU affinity setting, IPv6 flow label, TCP congestion algorithm setting, sendfile / zerocopy, socket pacing, authentication, bind to device, support IPv4 don't fragment, POSIX threads
|
||||||
|
```
|
||||||
|
|
||||||
|
## Benchmark 1: hyperfine, curl, direct outbound
|
||||||
|
|
||||||
File: `bench_curl_direct.txt`
|
File: `bench_curl_direct.txt`
|
||||||
|
|
||||||
Measurement of execution time of curl.
|
Measurement of execution time of curl.
|
||||||
Shows overhead of a proxying software.
|
Shows overhead of a proxying software.
|
||||||
|
|
||||||
Proxies are set up to accept requests by SOCKSv5 and forward directly to net.
|
Proxies are set up to accept requests by SOCKSv5,
|
||||||
|
sing-box on port 2080, xray on 2081,
|
||||||
|
and forward directly to net.
|
||||||
|
|
||||||
units: ms (less is better)
|
Units: ms (less is better)
|
||||||
|
|
||||||
### ~1.5M binary file over https from dc09.ru
|
### ~1.5M binary file over https from dc09.ru
|
||||||
|
|
||||||
|
@ -72,16 +117,19 @@ units: ms (less is better)
|
||||||
|sing-box|338.6|390.5|445.5|
|
|sing-box|338.6|390.5|445.5|
|
||||||
|xray|342.3|390.8|426.0|
|
|xray|342.3|390.8|426.0|
|
||||||
|
|
||||||
## iperf3, proxied outbound
|
## Benchmark 2: iperf3, proxied outbound
|
||||||
|
|
||||||
Measurement of bandwidth with iperf3.
|
Measurement of bandwidth with iperf3.
|
||||||
Shows processing speed of a proxying software.
|
Shows processing speed of a proxying software.
|
||||||
|
|
||||||
Client proxy (sing-box or xray, specified by a table column) is connected to SOCKSv5, Trojan or VLESS inbound on dc09.ru;
|
Client proxy (sing-box or xray, specified in a table column) is set up
|
||||||
server proxy on dc09.ru is either sing-box or xray (heading before a table), accepts requests on all 3 inbounds;
|
to accept requests by SOCKSv5 on 2080 or 2081 and to connect to
|
||||||
iperf3 server is on the same host as a server proxy.
|
a SOCKSv5, Trojan over uTLS or VLESS over Reality inbound on dc09.ru;
|
||||||
|
server proxy on dc09.ru is either sing-box or xray (specified in a caption before a table),
|
||||||
|
accepts requests on all 3 inbounds on ports 2220, 2221 and 2222;
|
||||||
|
iperf3 server is running on the same host as a server proxy.
|
||||||
|
|
||||||
units: Mbit/s (more is better)
|
Units: Mbit/s (more is better)
|
||||||
|
|
||||||
### no proxy
|
### no proxy
|
||||||
|
|
||||||
|
|
Loading…
Add table
Reference in a new issue