diff --git a/results.txt b/results.txt index 222d49f..00016b6 100644 --- a/results.txt +++ b/results.txt @@ -2,116 +2,242 @@ /usr/sbin/unbound /usr/sbin/hyperfine dig: using custom build at bind9/bin/dig/dig +kdig: using custom build at knot-dns/src/kdig /usr/sbin/drill resolve: using custom build at hickory-dns/target/release/resolve +dog: using custom build at dog/target/release/dog +doggo: using custom build at doggo/doggo +q: using custom build at q/q ==> Starting Unbound ==> Benchmarks for A domain.tld. Benchmark 1: bind9/bin/dig/dig -p 2253 +short A domain.tld @127.0.0.1 - Time (mean ± σ): 10.7 ms ± 2.2 ms [User: 8.0 ms, System: 3.3 ms] - Range (min … max): 8.6 ms … 16.7 ms 322 runs + Time (mean ± σ): 12.2 ms ± 3.5 ms [User: 8.8 ms, System: 4.0 ms] + Range (min … max): 8.5 ms … 22.5 ms 211 runs -Benchmark 2: /usr/sbin/drill -p 2253 -Q A domain.tld @127.0.0.1 - Time (mean ± σ): 3.8 ms ± 0.5 ms [User: 2.6 ms, System: 0.9 ms] - Range (min … max): 3.0 ms … 4.9 ms 813 runs +Benchmark 2: knot-dns/src/kdig -p 2253 +short A domain.tld @127.0.0.1 + Time (mean ± σ): 3.7 ms ± 0.1 ms [User: 2.8 ms, System: 1.3 ms] + Range (min … max): 3.5 ms … 4.3 ms 807 runs -Benchmark 3: hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t A domain.tld +Benchmark 3: /usr/sbin/drill -p 2253 -Q A domain.tld @127.0.0.1 + Time (mean ± σ): 2.7 ms ± 0.4 ms [User: 1.9 ms, System: 0.7 ms] + Range (min … max): 2.2 ms … 5.2 ms 620 runs + +Benchmark 4: hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t A domain.tld Time (mean ± σ): 1.9 ms ± 0.1 ms [User: 1.2 ms, System: 1.3 ms] - Range (min … max): 1.8 ms … 3.4 ms 1594 runs + Range (min … max): 1.8 ms … 6.2 ms 1599 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 5: dog/target/release/dog -n 127.0.0.1:2253 -t A domain.tld + Time (mean ± σ): 285.3 µs ± 11.8 µs [User: 187.1 µs, System: 40.6 µs] + Range (min … max): 260.2 µs … 444.5 µs 10208 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 6: doggo/doggo -n 127.0.0.1:2253 --short -t A domain.tld + Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.7 ms, System: 1.0 ms] + Range (min … max): 1.5 ms … 2.0 ms 1771 runs + +Benchmark 7: q/q -s 127.0.0.1:2253 -r -t A domain.tld + Time (mean ± σ): 1.7 ms ± 0.1 ms [User: 0.8 ms, System: 1.0 ms] + Range (min … max): 1.6 ms … 2.2 ms 1741 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. Summary - hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t A domain.tld ran - 1.97 ± 0.28 times faster than /usr/sbin/drill -p 2253 -Q A domain.tld @127.0.0.1 - 5.63 ± 1.16 times faster than bind9/bin/dig/dig -p 2253 +short A domain.tld @127.0.0.1 + dog/target/release/dog -n 127.0.0.1:2253 -t A domain.tld ran + 5.77 ± 0.33 times faster than doggo/doggo -n 127.0.0.1:2253 --short -t A domain.tld + 5.87 ± 0.32 times faster than q/q -s 127.0.0.1:2253 -r -t A domain.tld + 6.61 ± 0.50 times faster than hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t A domain.tld + 9.47 ± 1.57 times faster than /usr/sbin/drill -p 2253 -Q A domain.tld @127.0.0.1 + 12.96 ± 0.62 times faster than knot-dns/src/kdig -p 2253 +short A domain.tld @127.0.0.1 + 42.90 ± 12.40 times faster than bind9/bin/dig/dig -p 2253 +short A domain.tld @127.0.0.1 ==> Benchmarks for AAAA domain.tld. Benchmark 1: bind9/bin/dig/dig -p 2253 +short AAAA domain.tld @127.0.0.1 - Time (mean ± σ): 9.8 ms ± 1.3 ms [User: 7.5 ms, System: 2.7 ms] - Range (min … max): 8.6 ms … 16.5 ms 227 runs + Time (mean ± σ): 12.6 ms ± 3.9 ms [User: 9.2 ms, System: 4.0 ms] + Range (min … max): 8.7 ms … 22.8 ms 196 runs + +Benchmark 2: knot-dns/src/kdig -p 2253 +short AAAA domain.tld @127.0.0.1 + Time (mean ± σ): 3.7 ms ± 0.1 ms [User: 2.8 ms, System: 1.4 ms] + Range (min … max): 3.5 ms … 4.1 ms 793 runs + +Benchmark 3: /usr/sbin/drill -p 2253 -Q AAAA domain.tld @127.0.0.1 + Time (mean ± σ): 4.1 ms ± 1.2 ms [User: 2.9 ms, System: 1.1 ms] + Range (min … max): 2.2 ms … 6.2 ms 911 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. -Benchmark 2: /usr/sbin/drill -p 2253 -Q AAAA domain.tld @127.0.0.1 - Time (mean ± σ): 3.5 ms ± 0.5 ms [User: 2.4 ms, System: 0.9 ms] - Range (min … max): 2.2 ms … 6.0 ms 823 runs +Benchmark 4: hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t AAAA domain.tld + Time (mean ± σ): 1.9 ms ± 0.4 ms [User: 1.3 ms, System: 1.2 ms] + Range (min … max): 1.8 ms … 14.7 ms 1616 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. -Benchmark 3: hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t AAAA domain.tld - Time (mean ± σ): 1.9 ms ± 0.8 ms [User: 1.3 ms, System: 1.3 ms] - Range (min … max): 1.8 ms … 33.9 ms 1591 runs +Benchmark 5: dog/target/release/dog -n 127.0.0.1:2253 -t AAAA domain.tld + Time (mean ± σ): 285.1 µs ± 14.1 µs [User: 187.1 µs, System: 40.7 µs] + Range (min … max): 261.6 µs … 592.2 µs 10402 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 6: doggo/doggo -n 127.0.0.1:2253 --short -t AAAA domain.tld + Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.7 ms, System: 1.0 ms] + Range (min … max): 1.5 ms … 2.2 ms 1721 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 7: q/q -s 127.0.0.1:2253 -r -t AAAA domain.tld + Time (mean ± σ): 1.7 ms ± 0.1 ms [User: 0.7 ms, System: 1.0 ms] + Range (min … max): 1.6 ms … 2.2 ms 1764 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. Summary - hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t AAAA domain.tld ran - 1.80 ± 0.83 times faster than /usr/sbin/drill -p 2253 -Q AAAA domain.tld @127.0.0.1 - 5.07 ± 2.32 times faster than bind9/bin/dig/dig -p 2253 +short AAAA domain.tld @127.0.0.1 + dog/target/release/dog -n 127.0.0.1:2253 -t AAAA domain.tld ran + 5.76 ± 0.36 times faster than doggo/doggo -n 127.0.0.1:2253 --short -t AAAA domain.tld + 5.86 ± 0.35 times faster than q/q -s 127.0.0.1:2253 -r -t AAAA domain.tld + 6.65 ± 1.56 times faster than hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t AAAA domain.tld + 12.96 ± 0.71 times faster than knot-dns/src/kdig -p 2253 +short AAAA domain.tld @127.0.0.1 + 14.53 ± 4.40 times faster than /usr/sbin/drill -p 2253 -Q AAAA domain.tld @127.0.0.1 + 44.17 ± 13.84 times faster than bind9/bin/dig/dig -p 2253 +short AAAA domain.tld @127.0.0.1 ==> Benchmarks for MX domain.tld. Benchmark 1: bind9/bin/dig/dig -p 2253 +short MX domain.tld @127.0.0.1 - Time (mean ± σ): 12.9 ms ± 4.0 ms [User: 9.6 ms, System: 3.9 ms] - Range (min … max): 8.7 ms … 23.1 ms 130 runs + Time (mean ± σ): 11.9 ms ± 2.6 ms [User: 8.7 ms, System: 3.8 ms] + Range (min … max): 8.7 ms … 18.2 ms 271 runs -Benchmark 2: /usr/sbin/drill -p 2253 -Q MX domain.tld @127.0.0.1 - Time (mean ± σ): 3.1 ms ± 0.2 ms [User: 2.2 ms, System: 0.8 ms] - Range (min … max): 2.7 ms … 3.6 ms 1030 runs +Benchmark 2: knot-dns/src/kdig -p 2253 +short MX domain.tld @127.0.0.1 + Time (mean ± σ): 3.7 ms ± 0.1 ms [User: 2.8 ms, System: 1.3 ms] + Range (min … max): 3.5 ms … 4.0 ms 803 runs -Benchmark 3: hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t MX domain.tld - Time (mean ± σ): 2.0 ms ± 2.0 ms [User: 1.3 ms, System: 1.3 ms] - Range (min … max): 1.8 ms … 81.8 ms 1633 runs +Benchmark 3: /usr/sbin/drill -p 2253 -Q MX domain.tld @127.0.0.1 + Time (mean ± σ): 4.0 ms ± 0.5 ms [User: 2.8 ms, System: 1.1 ms] + Range (min … max): 2.4 ms … 4.5 ms 1224 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 4: hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t MX domain.tld + Time (mean ± σ): 1.9 ms ± 0.1 ms [User: 1.2 ms, System: 1.3 ms] + Range (min … max): 1.8 ms … 3.1 ms 1580 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 5: dog/target/release/dog -n 127.0.0.1:2253 -t MX domain.tld + Time (mean ± σ): 287.4 µs ± 13.4 µs [User: 187.5 µs, System: 41.8 µs] + Range (min … max): 264.4 µs … 549.2 µs 10046 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 6: doggo/doggo -n 127.0.0.1:2253 --short -t MX domain.tld + Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.7 ms, System: 1.0 ms] + Range (min … max): 1.5 ms … 2.1 ms 1768 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 7: q/q -s 127.0.0.1:2253 -r -t MX domain.tld + Time (mean ± σ): 1.7 ms ± 0.1 ms [User: 0.8 ms, System: 1.0 ms] + Range (min … max): 1.6 ms … 2.2 ms 1788 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. Summary - hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t MX domain.tld ran - 1.57 ± 1.60 times faster than /usr/sbin/drill -p 2253 -Q MX domain.tld @127.0.0.1 - 6.54 ± 6.97 times faster than bind9/bin/dig/dig -p 2253 +short MX domain.tld @127.0.0.1 + dog/target/release/dog -n 127.0.0.1:2253 -t MX domain.tld ran + 5.72 ± 0.33 times faster than doggo/doggo -n 127.0.0.1:2253 --short -t MX domain.tld + 5.83 ± 0.33 times faster than q/q -s 127.0.0.1:2253 -r -t MX domain.tld + 6.57 ± 0.38 times faster than hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t MX domain.tld + 12.86 ± 0.65 times faster than knot-dns/src/kdig -p 2253 +short MX domain.tld @127.0.0.1 + 14.09 ± 1.77 times faster than /usr/sbin/drill -p 2253 -Q MX domain.tld @127.0.0.1 + 41.44 ± 9.30 times faster than bind9/bin/dig/dig -p 2253 +short MX domain.tld @127.0.0.1 ==> Benchmarks for CNAME mail.domain.tld. Benchmark 1: bind9/bin/dig/dig -p 2253 +short A mail.domain.tld @127.0.0.1 - Time (mean ± σ): 9.8 ms ± 1.4 ms [User: 7.5 ms, System: 2.8 ms] - Range (min … max): 8.7 ms … 15.5 ms 194 runs + Time (mean ± σ): 13.8 ms ± 4.4 ms [User: 9.9 ms, System: 4.6 ms] + Range (min … max): 8.8 ms … 23.1 ms 159 runs - Warning: The first benchmarking run for this command was significantly slower than the rest (15.4 ms). This could be caused by (filesystem) caches that were not filled until after the first run. You are already using the '--warmup' option which helps to fill these caches before the actual benchmark. You can either try to increase the warmup count further or re-run this benchmark on a quiet system in case it was a random outlier. Alternatively, consider using the '--prepare' option to clear the caches before each timing run. +Benchmark 2: knot-dns/src/kdig -p 2253 +short A mail.domain.tld @127.0.0.1 + Time (mean ± σ): 3.7 ms ± 0.1 ms [User: 2.8 ms, System: 1.3 ms] + Range (min … max): 3.5 ms … 4.2 ms 809 runs -Benchmark 2: /usr/sbin/drill -p 2253 -Q A mail.domain.tld @127.0.0.1 - Time (mean ± σ): 3.6 ms ± 0.1 ms [User: 2.5 ms, System: 0.9 ms] - Range (min … max): 3.1 ms … 4.0 ms 909 runs +Benchmark 3: /usr/sbin/drill -p 2253 -Q A mail.domain.tld @127.0.0.1 + Time (mean ± σ): 3.2 ms ± 1.0 ms [User: 2.2 ms, System: 0.9 ms] + Range (min … max): 2.2 ms … 6.5 ms 1328 runs -Benchmark 3: hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t A mail.domain.tld +Benchmark 4: hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t A mail.domain.tld Time (mean ± σ): 2.0 ms ± 0.9 ms [User: 1.3 ms, System: 1.3 ms] - Range (min … max): 1.9 ms … 30.8 ms 1523 runs + Range (min … max): 1.8 ms … 35.1 ms 1589 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. +Benchmark 5: dog/target/release/dog -n 127.0.0.1:2253 -t A mail.domain.tld + Time (mean ± σ): 283.5 µs ± 16.7 µs [User: 186.4 µs, System: 39.6 µs] + Range (min … max): 257.3 µs … 1241.4 µs 10893 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 6: doggo/doggo -n 127.0.0.1:2253 --short -t A mail.domain.tld + Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.7 ms, System: 1.1 ms] + Range (min … max): 1.5 ms … 2.0 ms 1738 runs + +Benchmark 7: q/q -s 127.0.0.1:2253 -r -t A mail.domain.tld + Time (mean ± σ): 1.7 ms ± 0.1 ms [User: 0.8 ms, System: 1.0 ms] + Range (min … max): 1.6 ms … 2.0 ms 1750 runs + Summary - hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t A mail.domain.tld ran - 1.80 ± 0.83 times faster than /usr/sbin/drill -p 2253 -Q A mail.domain.tld @127.0.0.1 - 4.89 ± 2.36 times faster than bind9/bin/dig/dig -p 2253 +short A mail.domain.tld @127.0.0.1 + dog/target/release/dog -n 127.0.0.1:2253 -t A mail.domain.tld ran + 5.80 ± 0.40 times faster than doggo/doggo -n 127.0.0.1:2253 --short -t A mail.domain.tld + 5.89 ± 0.39 times faster than q/q -s 127.0.0.1:2253 -r -t A mail.domain.tld + 6.98 ± 3.16 times faster than hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t A mail.domain.tld + 11.27 ± 3.53 times faster than /usr/sbin/drill -p 2253 -Q A mail.domain.tld @127.0.0.1 + 13.01 ± 0.82 times faster than knot-dns/src/kdig -p 2253 +short A mail.domain.tld @127.0.0.1 + 48.80 ± 15.68 times faster than bind9/bin/dig/dig -p 2253 +short A mail.domain.tld @127.0.0.1 ==> Benchmarks for TXT txt.domain.tld. Benchmark 1: bind9/bin/dig/dig -p 2253 +short TXT txt.domain.tld @127.0.0.1 - Time (mean ± σ): 11.6 ms ± 2.3 ms [User: 8.6 ms, System: 3.5 ms] - Range (min … max): 9.1 ms … 17.6 ms 229 runs - -Benchmark 2: /usr/sbin/drill -p 2253 -Q TXT txt.domain.tld @127.0.0.1 - Time (mean ± σ): 3.5 ms ± 1.5 ms [User: 2.4 ms, System: 0.9 ms] - Range (min … max): 2.2 ms … 11.0 ms 725 runs - -Benchmark 3: hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t TXT txt.domain.tld - Time (mean ± σ): 2.1 ms ± 0.9 ms [User: 1.2 ms, System: 1.4 ms] - Range (min … max): 1.9 ms … 36.7 ms 1461 runs + Time (mean ± σ): 10.9 ms ± 3.1 ms [User: 7.8 ms, System: 3.5 ms] + Range (min … max): 9.2 ms … 23.6 ms 217 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. +Benchmark 2: knot-dns/src/kdig -p 2253 +short TXT txt.domain.tld @127.0.0.1 + Time (mean ± σ): 3.8 ms ± 0.1 ms [User: 2.8 ms, System: 1.4 ms] + Range (min … max): 3.6 ms … 4.5 ms 798 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 3: /usr/sbin/drill -p 2253 -Q TXT txt.domain.tld @127.0.0.1 + Time (mean ± σ): 2.6 ms ± 0.1 ms [User: 1.9 ms, System: 0.6 ms] + Range (min … max): 2.5 ms … 3.1 ms 1159 runs + +Benchmark 4: hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t TXT txt.domain.tld + Time (mean ± σ): 2.0 ms ± 0.2 ms [User: 1.3 ms, System: 1.3 ms] + Range (min … max): 1.9 ms … 7.0 ms 1523 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 5: dog/target/release/dog -n 127.0.0.1:2253 -t TXT txt.domain.tld + Time (mean ± σ): 383.8 µs ± 14.6 µs [User: 231.2 µs, System: 70.2 µs] + Range (min … max): 356.5 µs … 640.9 µs 7669 runs + + Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. + +Benchmark 6: doggo/doggo -n 127.0.0.1:2253 --short -t TXT txt.domain.tld + Time (mean ± σ): 1.8 ms ± 0.1 ms [User: 0.7 ms, System: 1.1 ms] + Range (min … max): 1.6 ms … 2.3 ms 1735 runs + +Benchmark 7: q/q -s 127.0.0.1:2253 -r -t TXT txt.domain.tld + Time (mean ± σ): 1.8 ms ± 0.1 ms [User: 0.8 ms, System: 1.1 ms] + Range (min … max): 1.7 ms … 2.2 ms 1683 runs + Summary - hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t TXT txt.domain.tld ran - 1.70 ± 1.09 times faster than /usr/sbin/drill -p 2253 -Q TXT txt.domain.tld @127.0.0.1 - 5.67 ± 2.85 times faster than bind9/bin/dig/dig -p 2253 +short TXT txt.domain.tld @127.0.0.1 + dog/target/release/dog -n 127.0.0.1:2253 -t TXT txt.domain.tld ran + 4.63 ± 0.27 times faster than doggo/doggo -n 127.0.0.1:2253 --short -t TXT txt.domain.tld + 4.70 ± 0.24 times faster than q/q -s 127.0.0.1:2253 -r -t TXT txt.domain.tld + 5.23 ± 0.45 times faster than hickory-dns/target/release/resolve -n 127.0.0.1:2253 -t TXT txt.domain.tld + 6.78 ± 0.31 times faster than /usr/sbin/drill -p 2253 -Q TXT txt.domain.tld @127.0.0.1 + 9.85 ± 0.45 times faster than knot-dns/src/kdig -p 2253 +short TXT txt.domain.tld @127.0.0.1 + 28.39 ± 8.20 times faster than bind9/bin/dig/dig -p 2253 +short TXT txt.domain.tld @127.0.0.1 ==> All done