mirrors/utls
1
0
Fork 0
mirror of https://github.com/refraction-networking/utls.git synced 2025-04-02 03:27:35 +03:00
Code Issues Projects Releases Packages Wiki Activity
731 commits 7 branches 42 tags 12 MiB
Commit graph

81 commits

Author SHA1 Message Date
Mingye Chen
a99feacec2 sync: merge changes from go 1.24.0 2025-03-01 00:16:54 -07:00
Mingye Chen
cefe226467 sync: merge changes from go 1.23.4 2025-02-10 12:30:47 -07:00
Filippo Valsorda
548fd92733 crypto/ecdsa: implement deterministic and hedged signatures 
For the future, some test vectors we should generate and then share
through Wycheproof or CCTV:
 - A private key with a leading zero byte.
 - A hash longer than the modulus.
 - A hash longer than the P-521 modulus by a few bits.
 - Reductions happening in hashToNat and bits2octets.

Fixes #64802

Change-Id: Ia0f89781b2c78eedd5103cf0e9720630711c37ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/628681
TryBot-Bypass: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
 2024-11-19 23:02:26 +00:00
Roland Shoemaker
5578206479 crypto/tls: reorder client_hello extensions 
This sets us up for ECH, which need compressible extensions to be
contiguous. Put them at the end to make things easier for everyone.

Change-Id: I2a51f76d5fcd1b6d82325f5a4a8cde6d75f1be0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/585437
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
 2024-05-22 22:33:38 +00:00
Filippo Valsorda
245de0a13b crypto/tls: clarify group selection logic 
I initially thought the logic was broken, but writing the test I
realized it was actually very clever (derogative). It was relying on the
outer loop continuing after a supported match without a key share,
allowing a later key share to override it (but not a later supported
match because of the "if selectedGroup != 0 { continue }").

Replaced the clever loop with two hopefully more understandable loops,
and added a test (which was already passing).

We were however not checking that the selected group is in the supported
list if we found it in key shares first. (This was only a MAY.) Fixed.

Fixes #65686

Change-Id: I09ea44f90167ffa36809deb78255ed039a217b6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/586655
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
 2024-05-22 14:19:11 +00:00
Gaukas Wang
86e9b69fdd
sync: Go 1.21 with QUIC support (#208) 
* sync: Go 1.21rc3, QUIC support added (#207)

* sync: merge with upstream tag/go-1.21rc3 (#11)

* fix: all tests pass

* impl: UQUIC Transport

* deps: bump up min Go version

* new: uquic

* fix: add QUICTransportParameter

* deprecated: Go 1.19 no longer supported

Go 1.19 will fail to build or pass the test once we bump up to the new version.

* sync: crypto/tls: restrict RSA keys in certificates to <= 8192 bits (#209)

* [release-branch.go1.21] crypto/tls: restrict RSA keys in certificates to <= 8192 bits

Extremely large RSA keys in certificate chains can cause a client/server
to expend significant CPU time verifying signatures. Limit this by
restricting the size of RSA keys transmitted during handshakes to <=
8192 bits.

Based on a survey of publicly trusted RSA keys, there are currently only
three certificates in circulation with keys larger than this, and all
three appear to be test certificates that are not actively deployed. It
is possible there are larger keys in use in private PKIs, but we target
the web PKI, so causing breakage here in the interests of increasing the
default safety of users of crypto/tls seems reasonable.

Thanks to Mateusz Poliwczak for reporting this issue.

Fixes CVE-2023-29409

* build: [ci skip] boring not included

* fix: typo [ci skip]

* docs: replenish readme [ci skip]

replace old build status badge with new ones, bump up required version noted in docs, update developer contact to reflect current status.
 2023-08-03 23:22:53 -06:00
Filippo Valsorda
5ca720fc5e crypto/tls: don't reverify but check certificate expiration on resumption 
We used to inconsistently run certificate verification on the server on
resumption, but not on the client. This made TLS 1.3 resumption pretty
much useless, as it didn't save bytes, CPU, or round-trips.

This requires serializing the verified chains into the session ticket,
so it's a tradeoff making the ticket bigger to save computation (and for
consistency).

The previous behavior also had a "stickyness" issue: if a ticket
contained invalid certificates, they would be used even if the client
had in the meantime configured valid certificates for a full handshake.

We also didn't check expiration on the client side on resumption if
InsecureSkipVerify was set. Again for consistency, we do that now.

Also, we used to run VerifyPeerCertificates on resumption even if
NoClientCerts was set.

Fixes #31641

Change-Id: Icc88269ea4adb544fa81158114aae76f3c91a15f
Reviewed-on: https://go-review.googlesource.com/c/go/+/497895
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
 2023-05-25 00:14:50 +00:00
Filippo Valsorda
d154b73cf1 crypto/tls: implement Extended Master Secret 
All OpenSSL tests now test operation with EMS. To test a handshake
*without* EMS we need to pass -Options=-ExtendedMasterSecret which is
only available in OpenSSL 3.1, which breaks a number of other tests.

Updates #43922

Change-Id: Ib9ac79a1d03fab6bfba5fe9cd66689cff661cda7
Reviewed-on: https://go-review.googlesource.com/c/go/+/497376
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
 2023-05-25 00:13:30 +00:00
Filippo Valsorda
1143de0f03 crypto/tls: add QUIC 0-RTT APIs 
Fixes #60107

Change-Id: I158b1c2d80d8ebb5ed7a8e6f313f69060754e220
Reviewed-on: https://go-review.googlesource.com/c/go/+/496995
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
 2023-05-25 00:13:28 +00:00
Filippo Valsorda
866e886415 crypto/tls: add SessionState.Extra 
This can be used by applications to store additional data in a session.

Fixes #57753
For #60105

Change-Id: Ib42387ad64750fa8dbbdf51de5e9c86378bef0ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/496822
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Marten Seemann <martenseemann@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
 2023-05-24 23:57:01 +00:00
Filippo Valsorda
b838c1c320 crypto/tls: add SessionState and use it on the server side 
This change by itself is useless, because the application has no way to
access or provide SessionStates to crypto/tls, but they will be provided
in following CLs.

For #60105

Change-Id: I8d5de79b1eda0a778420134cf6f346246a1bb296
Reviewed-on: https://go-review.googlesource.com/c/go/+/496818
Reviewed-by: Marten Seemann <martenseemann@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
 2023-05-24 23:56:32 +00:00
Filippo Valsorda
65b9e15fc2 crypto/tls: reduce session ticket linkability 
Ever since session ticket key rotation was introduced in CL 9072, we've
been including a prefix in every ticket to identify what key it's
encrypted with. It's a small privacy gain, but the cost of trial
decryptions is also small, especially since the first key is probably
the most frequently used.

Also reissue tickets on every resumption so that the next connection
can't be linked to all the previous ones. Again the privacy gain is
small but the performance cost is small and it comes with a reduction in
complexity.

For #60105

Change-Id: I852f297162d2b79a3d9bf61f6171e8ce94b2537a
Reviewed-on: https://go-review.googlesource.com/c/go/+/496817
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
 2023-05-24 23:56:24 +00:00
Gaukas Wang
3721531ea9
Implement ClientHelloSpec JSON Unmarshaler (#176) 
* wip: staging work

* wip: staging work

* feat: ClientHello JSON Unmarshaler

Allowing unmarshalling a JSON object into a ClientHelloSpec.

* feat: ClientHello JSON Unmarshaler rev

- Revised JSON ClientHello format
- Implemented `TLSExtensionJSON` interface for some more extensions
 2023-03-30 09:13:47 -06:00
Filippo Valsorda
5661b3ddeb crypto/ecdsa: use bigmod and nistec instead of math/big and crypto/elliptic 
Ignoring custom curves, this makes the whole package constant-time.
There is a slight loss in performance for P-384 and P-521 because bigmod
is slower than math/big (but P-256 has an assembly scalar field
inversion, so doesn't use bigmod for anything big).

name                old time/op    new time/op    delta
Sign/P256-8           19.2µs ± 2%    19.1µs ± 2%     ~     (p=0.268 n=9+10)
Sign/P384-8            166µs ± 3%     188µs ± 2%  +13.52%  (p=0.000 n=10+10)
Sign/P521-8            337µs ± 2%     359µs ± 2%   +6.46%  (p=0.000 n=10+10)
Verify/P256-8         58.1µs ± 2%    58.1µs ± 2%     ~     (p=0.971 n=10+10)
Verify/P384-8          484µs ± 2%     569µs ±12%  +17.65%  (p=0.000 n=10+10)
Verify/P521-8         1.03ms ± 4%    1.14ms ± 2%  +11.02%  (p=0.000 n=10+10)
GenerateKey/P256-8    12.4µs ±12%    12.0µs ± 2%     ~     (p=0.063 n=10+10)
GenerateKey/P384-8     129µs ±18%     119µs ± 2%     ~     (p=0.190 n=10+10)
GenerateKey/P521-8     241µs ± 2%     240µs ± 2%     ~     (p=0.436 n=10+10)

name                old alloc/op   new alloc/op   delta
Sign/P256-8           3.08kB ± 0%    2.47kB ± 0%  -19.77%  (p=0.000 n=10+10)
Sign/P384-8           6.16kB ± 0%    2.64kB ± 0%  -57.16%  (p=0.000 n=10+10)
Sign/P521-8           7.87kB ± 0%    3.01kB ± 0%  -61.80%  (p=0.000 n=10+10)
Verify/P256-8         1.29kB ± 1%    0.48kB ± 0%  -62.69%  (p=0.000 n=10+10)
Verify/P384-8         2.49kB ± 1%    0.64kB ± 0%  -74.25%  (p=0.000 n=10+10)
Verify/P521-8         3.31kB ± 0%    0.96kB ± 0%  -71.02%  (p=0.000 n=7+10)
GenerateKey/P256-8      720B ± 0%      920B ± 0%  +27.78%  (p=0.000 n=10+10)
GenerateKey/P384-8      921B ± 0%     1120B ± 0%  +21.61%  (p=0.000 n=9+10)
GenerateKey/P521-8    1.30kB ± 0%    1.44kB ± 0%  +10.45%  (p=0.000 n=10+10)

name                old allocs/op  new allocs/op  delta
Sign/P256-8             45.0 ± 0%      33.0 ± 0%  -26.67%  (p=0.000 n=10+10)
Sign/P384-8             69.0 ± 0%      34.0 ± 0%  -50.72%  (p=0.000 n=10+10)
Sign/P521-8             71.0 ± 0%      35.0 ± 0%  -50.70%  (p=0.000 n=10+10)
Verify/P256-8           23.0 ± 0%      10.0 ± 0%  -56.52%  (p=0.000 n=10+10)
Verify/P384-8           43.0 ± 0%      14.0 ± 0%  -67.44%  (p=0.000 n=10+10)
Verify/P521-8           45.0 ± 0%      14.0 ± 0%  -68.89%  (p=0.000 n=7+10)
GenerateKey/P256-8      13.0 ± 0%      14.0 ± 0%   +7.69%  (p=0.000 n=10+10)
GenerateKey/P384-8      16.0 ± 0%      17.0 ± 0%   +6.25%  (p=0.000 n=10+10)
GenerateKey/P521-8      16.5 ± 3%      17.0 ± 0%   +3.03%  (p=0.033 n=10+10)

Change-Id: I4e074ef039b0f7ffbc436a4cdbe4ef90c647018d
Reviewed-on: https://go-review.googlesource.com/c/go/+/353849
Auto-Submit: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
 2022-11-21 16:19:34 +00:00
Gaukas Wang
54f1f4e2f9
Sync upstream crypto/tls (#120) 
* Merge Upstream (#4)

* crypto/rand, internal/syscall/unix: add support for getrandom syscall on solaris

The getrandom syscall is available on Solaris and Illumos, see
https://docs.oracle.com/cd/E88353_01/html/E37841/getrandom-2.html and
https://illumos.org/man/2/getrandom

Change-Id: Id1c65d6a5b2fbc80d20b43d8b32dab137ca950ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/299134
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* docs: clarify when APIs use context.Background.

The Go standard library retrofitted context support onto existing APIs
using context.Background and later offered variants that directly
supported user-defined context value specification. This commit makes
that behavior clear in documentation and suggests context-aware
alternatives if the user is looking for one.

An example motivation is supporting code for use in systems that expect
APIs to be cancelable for lifecycle correctness or load
shedding/management reasons, as alluded to in
https://blog.golang.org/context-and-structs.

Updates #44143

Change-Id: I2d7f954ddf9b48264d5ebc8d0007058ff9bddf14
Reviewed-on: https://go-review.googlesource.com/c/go/+/296152
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Jean de Klerk <deklerk@google.com>
Trust: Jean de Klerk <deklerk@google.com>
Run-TryBot: Jean de Klerk <deklerk@google.com>
TryBot-Result: Go Bot <gobot@golang.org>

* cmd: move GOEXPERIMENT knob from make.bash to cmd/go

This CL changes GOEXPERIMENT to act like other GO[CONFIG] environment
variables. Namely, that it can be set at make.bash time to provide a
default value used by the toolchain, but then can be manually set when
running either cmd/go or the individual tools (compiler, assembler,
linker).

For example, it's now possible to test rsc.io/tmp/fieldtrack by simply
running:

GOEXPERIMENT=fieldtrack go test -gcflags=-l rsc.io/tmp/fieldtrack \
  -ldflags=-k=rsc.io/tmp/fieldtrack.tracked

without needing to re-run make.bash. (-gcflags=-l is needed because
the compiler's inlining abilities have improved, so calling a function
with a for loop is no longer sufficient to suppress inlining.)

Fixes #42681.

Change-Id: I2cf8995d5d0d05f6785a2ee1d3b54b2cfb3331ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/300991
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>

* internal/poll: fix some grammar errors

Change-Id: I25a6424bce9d372fa46e8bdd856095845d3397bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/300889
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>

* all: remove duplicate words

Change-Id: Ib0469232a2b69a869e58d5d24990ad74ac96ea56
GitHub-Last-Rev: eb38e049ee1e773392ff3747e1eb2af20dd50dcd
GitHub-Pull-Request: golang/go#44805
Reviewed-on: https://go-review.googlesource.com/c/go/+/299109
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>

* all: add internal/itoa package

This replaces five implementations scattered across low level packages.
(And I plan to use it in a sixth soon.)
Three of the five were byte-for-byte identical.

Change-Id: I3bbbeeac63723a487986c912b604e10ad1e042f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/301549
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>

* internal/poll: eliminate the redundant type conversions of FD.Sysfd

Change-Id: Ib75662f717320510319c696520e645f54eec97f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/301569
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* crypto/tls: add HandshakeContext method to Conn

Adds the (*tls.Conn).HandshakeContext method. This allows
us to pass the context provided down the call stack to
eventually reach the tls.ClientHelloInfo and
tls.CertificateRequestInfo structs.
These contexts are exposed to the user as read-only via Context()
methods.

This allows users of (*tls.Config).GetCertificate and
(*tls.Config).GetClientCertificate to use the context for
request scoped parameters and cancellation.

Replace uses of (*tls.Conn).Handshake with (*tls.Conn).HandshakeContext
where appropriate, to propagate existing contexts.

Fixes #32406

Change-Id: I259939c744bdc9b805bf51a845a8bc462c042483
Reviewed-on: https://go-review.googlesource.com/c/go/+/295370
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>

* crypto/tls: remove flaky cancellation test

This will be reintroduced again once the source of the
flakiness has been determined and fixed.

Fixes #45084

Change-Id: I6677b27fcd71e8c9bb8edbe8e3be70e5a271ebd3
Reviewed-on: https://go-review.googlesource.com/c/go/+/302569
Trust: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Run-TryBot: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>

* crypto/rand, internal/syscall/unix: add support for getentropy syscall on darwin

The getentropy syscall is available on macOS since version 10.12, which
is the minimum required version since Go 1.15.

Change-Id: I294259af0b11df9669e4dc5fa891d2f2f039d91a
Reviewed-on: https://go-review.googlesource.com/c/go/+/302489
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>

* [dev.fuzz] internal/fuzz: add minimization of []byte

This works by minimizing for a maximum of one minute. We may consider
making this customizable in the future.

This only minimizes []byte inputs which caused a recoverable error. In
the future, it should support minimizing other appopriate types, and
minimizing types which caused non-recoverable errors (though this is
much more expensive).

The code in internal/fuzz/worker.go is copied from, or heavily inspired
by, code originally authored by Dmitry Vyukov and Josh Bleecher Snyder
as part of the go-fuzz project. Thanks to them for their contributions.
See https://github.com/dvyukov/go-fuzz.

Change-Id: I93dbac7ff874d6d0c1b9b9dda23930ae9921480c
Reviewed-on: https://go-review.googlesource.com/c/go/+/298909
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* all: explode GOEXPERIMENT=regabi into 5 sub-experiments

This separates GOEXPERIMENT=regabi into five sub-experiments:
regabiwrappers, regabig, regabireflect, regabidefer, and regabiargs.
Setting GOEXPERIMENT=regabi now implies the working subset of these
(currently, regabiwrappers, regabig, and regabireflect).

This simplifies testing, helps derisk the register ABI project,
and will also help with performance comparisons.

This replaces the -abiwrap flag to the compiler and linker with
the regabiwrappers experiment.

As part of this, regabiargs now enables registers for all calls
in the compiler. Previously, this was statically disabled in
regabiEnabledForAllCompilation, but now that we can control it
independently, this isn't necessary.

For #40724.

Change-Id: I5171e60cda6789031f2ef034cc2e7c5d62459122
Reviewed-on: https://go-review.googlesource.com/c/go/+/302070
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>

* [dev.fuzz] testing: print logs and error messages when fuzzing

Also improve the error messages for the use of
testing.F functions inside the Fuzz function.

Change-Id: I5fa48f8c7e0460a1da89a49a73e5af83c544e549
Reviewed-on: https://go-review.googlesource.com/c/go/+/298849
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* [dev.fuzz] internal/fuzz: reduce allocation in the mutator

When mutating a byte slice, mutate in place, and only allocate once if
the slice's capacity is less than the maximum size.

mutateBytes already should not allocate; we check a post-condition
that the slice's data pointer does not change.

This speeds up the mutator from 4 ms per value to 200-600 ns. For
example:

    goos: darwin
    goarch: amd64
    pkg: internal/fuzz
    cpu: Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
    BenchmarkMutatorBytes/1-8                5908735               275.3 ns/op
    BenchmarkMutatorBytes/10-8               5198473               282.0 ns/op
    BenchmarkMutatorBytes/100-8              4304750               233.9 ns/op
    BenchmarkMutatorBytes/1000-8             4623988               295.2 ns/op
    BenchmarkMutatorBytes/10000-8            4252104               458.5 ns/op
    BenchmarkMutatorBytes/100000-8           1236751               950.8 ns/op
    PASS
    ok      internal/fuzz   12.993s

Change-Id: I4bf2a04be6c648ef440af2c62bf0ffa3d310172c
Reviewed-on: https://go-review.googlesource.com/c/go/+/306675
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>

* all: update references to symbols moved from io/ioutil to io

Update references missed in CL 263142.

For #41190

Change-Id: I778760a6a69bd0440fec0848bdef539c9ccb4ee1
GitHub-Last-Rev: dda42b09fff36dc08ec1cdec50cc19e3da5058e5
GitHub-Pull-Request: golang/go#42874
Reviewed-on: https://go-review.googlesource.com/c/go/+/273946
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Cherry Zhang <cherryyz@google.com>

* net: make ErrClosed and ParseError implement net.Error

Fixes #45357

Change-Id: Iafd41fff232a89be4c88d4b1d66bc3c04d888bcc
Reviewed-on: https://go-review.googlesource.com/c/go/+/307030
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

* [dev.fuzz] internal/fuzz: small bug fixes and refactors to minimization

This fixes a few issues that were being masked since
log statements weren't being printed to stdout. Now
that they are, fix the bugs, and update the tests.

Also includes a few small refactors which will make
minimizing non-recoverable errors easier.

Change-Id: Ie2fd2e5534b3980317e1e1f3fd8e04750988c17f
Reviewed-on: https://go-review.googlesource.com/c/go/+/307810
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* internal/goexperiment,cmd: consolidate GOEXPERIMENTs into a new package

Currently there's knowledge about the list of GOEXPERIMENTs in a few
different places. This CL introduces a new package and consolidates
the list into one place: the internal/goexperiment.Flags struct type.

This package gives us a central place to document the experiments as
well as the GOEXPERIMENT environment variable itself. It will also
give us a place to put built-time constants derived from the enabled
experiments.

Now the objabi package constructs experiment names by reflecting over
this struct type rather than having a separate list of these names
(this is similar to how the compiler handles command-line flags and
debug options). We also expose a better-typed API to the toolchain for
propagating enabled experiments.

Change-Id: I06e026712b59fe2bd7cd11a869aedb48ffe5a4b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/307817
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

* internal/goexperiment: consolidate experiment-enabled constants

Currently, we have boolean and integral constants for GOEXPERIMENTs in
various places. Consolidate these into automatically generated
constants in the internal/goexperiment package.

Change-Id: I42a49aba2a3b4c722fedea23a613162cd8a67bee
Reviewed-on: https://go-review.googlesource.com/c/go/+/307818
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

* cmd/internal/objabi: make GOEXPERIMENT be a diff from default experiments

Right now the rules around handling default-on experiments are
complicated and a bit inconsistent. Notably, objabi.GOEXPERIMENT is
set to a comma-separated list of enabled experiments, but this may not
be the string a user should set the GOEXPERIMENT environment variable
to get that list of experiments: if an experiment is enabled by
default but gets turned off by GOEXPERIMENT, then the string we report
needs to include "no"+experiment to capture that default override.

This complication also seeps into the version string we print for "go
tool compile -V", etc. This logic is further complicated by the fact
that it only wants to include an experiment string if the set of
experiments varies from the default.

This CL rethinks how we handle default-on experiments. Now that
experiment state is all captured in a struct, we can simplify a lot of
this logic. objabi.GOEXPERIMENT will be set based on the delta from
the default set of experiments, which reflects what a user would
actually need to pass on the command line. Likewise, we include this
delta in the "-V" output, which simplifies this logic because if
there's nothing to show in the version string, the delta will be
empty.

Change-Id: I7ed307329541fc2c9f90edd463fbaf8e0cc9e8ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/307819
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

* runtime,cmd/link: include GOEXPERIMENTs in runtime.Version(), "go version X"

This adds the set of GOEXPERIMENTs to the build version if it differs
from the default set of experiments. This exposes the experiment
settings via runtime.Version() and "go version <binary>".

Change-Id: I143dbbc50f66a4cf175469199974e18848075af6
Reviewed-on: https://go-review.googlesource.com/c/go/+/307820
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

* [dev.fuzz] internal/fuzz: add stub for coverage

This change only includes a stub for the function
which will hook into the runtime to expose
coverage instrumentation while we're fuzzing.

Previously, we discussed an exported API named
FuzzCoverage, but since this is within the
internal/fuzz package, simply naming it coverage
seems appropriate.

Change-Id: Iba3240e53e0c4c434e937aa9bb1711a44fec9975
Reviewed-on: https://go-review.googlesource.com/c/go/+/308191
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* [dev.fuzz] internal/fuzz: add mutator for int types

Assuming this works, will follow up with another CL
that mutates other types.

Change-Id: Id61acaacd56ca41e3be52e400f8f768672313bbb
Reviewed-on: https://go-review.googlesource.com/c/go/+/308169
Trust: Katie Hockman <katie@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* [dev.fuzz] internal/fuzz: implement coverage and trace instrumentation

This CL makes two main changes to allow internal/fuzz to support
-d=libfuzzer instrumentation:

1. It extends cmd/link to define _counters and _ecounters symbols so
internal/fuzz can find the coverage counters.

2. It adds "trace" stub functions that implement the ABI expected by
cmd/compile for comparison instrumentation.

N.B., that -tags=libfuzzer should *not* be set, so that
internal/fuzz's trace routines will be used instead of runtime's
libfuzzer trampolines.

Also, the current implementation doesn't support multi-module builds
(i.e., compiling a Go program that spans multiple .so/.dll files).
Presumably this isn't an issue, since "go test -fuzz" will need to
recompile the binary with instrumentation anyway so it can make sure
to always use a single-module build. But we can revisit this if
necessary.

Change-Id: I9b1619119ab7477bebcfd5988b4b60499a7ab0d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/308289
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>

* [dev.fuzz] internal/fuzz: don't count time spent loading corpus

The -fuzztime flag tells us how much time to spend fuzzing, not
counting time spent running the seed corpus. We shouldn't count time
spent loading the cache either. If the cache is large, the time limit
may be exceeded before the coordinator starts the workers.

Change-Id: If00435faa5d24aabdb9003ebb9337fa2e47f22b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/307310
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>

* [dev.fuzz] internal/fuzz: improve cancellation in worker event loops

worker.runFuzzing now accepts a Context, used for cancellation instead
of doneC (which is removed). This is passed down through workerClient
RPC methods (ping, fuzz).

workerClient RPC methods now wrap the call method, which handles
marshaling and cancellation.

Both workerClient.call and workerServer.serve should return quickly
when their contexts are cancelled. Turns out, closing the pipe won't
actually unblock a read on all platforms. Instead, we were falling
back to SIGKILL in worker.stop, which works but takes longer than
necessary.

Also fixed missing newline in log message.

Change-Id: I7b5ae54d6eb9afd6361a07759f049f048952e0cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/303429
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>

* [dev.fuzz] testing: let -fuzztime specify a number of executions

-fuzztime now works similarly to -benchtime: if it's given a string
with an "x" suffix (as opposed to "s" or some other unit of
duration), the fuzzing system will generate and run a maximum number
of values.

This CL also implements tracking and printing counts, since most of
the work was already done.

Change-Id: I013007984b5adfc1a751c379dc98c8d46b4a97e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/306909
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>

* [dev.fuzz] testing: support T.Parallel in fuzz functions

While running the seed corpus, T.Parallel acts like it does in
subtests started with T.Run: it blocks until all other non-parallel
subtests have finished, then unblocks when the barrier chan is
closed. A semaphore (t.context.waitParallel) limits the number of
tests that run concurrently (determined by -test.parallel).

While fuzzing, T.Parallel has no effect, other than asserting that it
can't be called multiple times. We already run different inputs in
concurrent processes, but we can't run inputs concurrently in the same
process if we want to attribute crashes to specific inputs.

Change-Id: I2bac08e647e1d92ea410c83c3f3558a033fe3dd1
Reviewed-on: https://go-review.googlesource.com/c/go/+/300449
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>

* [dev.fuzz] internal/fuzz: move CoordinateFuzzing args into struct type

This improves readability a bit, and it should help with compatibility
for future clients when arguments are added or reordered.

Unfortunately, testing still can't import internal/fuzz, so the
interface there can't use this type.

Change-Id: I4cda2347884defcbbfc2bd01ab5b4a901d91549c
Reviewed-on: https://go-review.googlesource.com/c/go/+/308192
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>

* all: fix spellings

This follows the spelling choices that the Go project has made for English words.
https://github.com/golang/go/wiki/Spelling

Change-Id: Ie7c586d2cf23020cb492cfff58c0831d2d8d3a78
GitHub-Last-Rev: e16a32cd225a275f73d236bcb33703986d110ded
GitHub-Pull-Request: golang/go#45442
Reviewed-on: https://go-review.googlesource.com/c/go/+/308291
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>

* internal/poll: fix the intermittent build failures with pipe pool

Correlative CL 308089

Fixes #45059

Change-Id: I1ff9fbf64e6620d651f287ba2a28d40f964d78a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/308329
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>

* internal/poll: ensure that newPoolPipe doesn't return a nil pointer

The function could occasionally return a nil pointer as a non-nil
interface, confusing the calling code.

Fixes #45520

Change-Id: Ifd35613728efa2cee9903177e85d369155074804
Reviewed-on: https://go-review.googlesource.com/c/go/+/309429
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Andy Pan <panjf2000@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>

* [dev.fuzz] internal/fuzz: allow float types to be integer literals

Previously, something like `float64(0)` would fail to decode
since the 0 value is considered an integer literal, and the
float64 parsing code required a float literal. Be more flexible
here since an integer can always be converted to a float.

Change-Id: Id1c53ef2e8a9748a4f71176b00b453a329af4ade
Reviewed-on: https://go-review.googlesource.com/c/go/+/309032
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* [dev.fuzz] internal/fuzz: mutate other types

Change-Id: I8042c17268aca0a9bb2f692317207bb864b18680
Reviewed-on: https://go-review.googlesource.com/c/go/+/309033
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* internal/execabs: replace ioutil.WriteFile with os.WriteFile

Fixes #45532.

Change-Id: I844acd50d6fa1ce918969bbb52f79dd7412d289f
Reviewed-on: https://go-review.googlesource.com/c/go/+/309350
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Tobias Klauser <tobias.klauser@gmail.com>

* runtime: move next_gc and last_next_gc into gcControllerState

This change moves next_gc and last_next_gc into gcControllerState under
the names heapGoal and lastHeapGoal respectively. These are
fundamentally GC pacer related values, and so it makes sense for them to
live here.

Partially generated by

rf '
    ex . {
	memstats.next_gc -> gcController.heapGoal
	memstats.last_next_gc -> gcController.lastHeapGoal
    }
'

except for updates to comments and gcControllerState methods, where
they're accessed through the receiver, and trace-related renames of
NextGC -> HeapGoal, while we're here.

For #44167.

Change-Id: I1e871ad78a57b01be8d9f71bd662530c84853bed
Reviewed-on: https://go-review.googlesource.com/c/go/+/306603
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>

* internal/bytealg: port more performance-critical functions to ABIInternal

CL 308931 ported several runtime assembly functions to ABIInternal so
that compiler-generated ABIInternal calls don't go through ABI
wrappers, but it missed the runtime assembly functions that are
actually defined in internal/bytealg.

This eliminates the cost of wrappers for the BleveQuery and
GopherLuaKNucleotide benchmarks, but there's still more to do for
Tile38.

                                      0-base                1-wrappers
                                     sec/op        sec/op            vs base
BleveQuery                          6.507 ± 0%    6.477 ± 0%  -0.46% (p=0.004 n=20)
GopherLuaKNucleotide                30.39 ± 1%    30.34 ± 0%       ~ (p=0.301 n=20)
Tile38IntersectsCircle100kmRequest 1.038m ± 1%   1.080m ± 2%  +4.03% (p=0.000 n=20)

For #40724.

Change-Id: I0b722443f684fcb997b1d70802c5ed4b8d8f9829
Reviewed-on: https://go-review.googlesource.com/c/go/+/310184
Trust: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

* bytes: add asm implementation for index on ppc64x

This adds an asm implementation of index on ppc64le and
ppc64. It results in a significant improvement in some
of the benchmarks that use bytes.Index.

The implementation is based on a port of the s390x asm
implementation. Comments on the design are found
with the code.

The following improvements occurred on power8:

Index/10       70.7ns ± 0%    18.8ns ± 0%   -73.4
Index/32        165ns ± 0%      95ns ± 0%   -42.6
Index/4K       9.23µs ± 0%    4.91µs ± 0%   -46
Index/4M       9.52ms ± 0%    5.10ms ± 0%   -46.4
Index/64M       155ms ± 0%      85ms ± 0%   -45.1

Count/10       83.0ns ± 0%    32.1ns ± 0%   -61.3
Count/32        178ns ± 0%     109ns ± 0%   -38.8
Count/4K       9.24µs ± 0%    4.93µs ± 0%   -46
Count/4M       9.52ms ± 0%    5.10ms ± 0%   -46.4
Count/64M       155ms ± 0%      85ms ± 0%   -45.1

IndexHard1     2.36ms ± 0%    0.13ms ± 0%   -94.4
IndexHard2     2.36ms ± 0%    1.28ms ± 0%   -45.8
IndexHard3     2.36ms ± 0%    1.19ms ± 0%   -49.4
IndexHard4     2.36ms ± 0%    2.35ms ± 0%    -0.1

CountHard1     2.36ms ± 0%    0.13ms ± 0%   -94.4
CountHard2     2.36ms ± 0%    1.28ms ± 0%   -45.8
CountHard3     2.36ms ± 0%    1.19ms ± 0%   -49.4

IndexPeriodic/IndexPeriodic2  146µs ± 0%       8µs ± 0%   -94
IndexPeriodic/IndexPeriodic4  146µs ± 0%       8µs ± 0%   -94

Change-Id: I7dd2bb7e278726e27f51825ca8b2f8317d460e60
Reviewed-on: https://go-review.googlesource.com/c/go/+/309730
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Paul Murphy <murp@ibm.com>
Reviewed-by: Carlos Eduardo Seo <carlos.seo@linaro.org>
Trust: Carlos Eduardo Seo <carlos.seo@linaro.org>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>

* internal/goexperiment: move baseline configuration to objabi

We need to adjust baseline experiment configuration based on the
configured GOOS and GOARCH, so it can't live in goexperiment. Move it
to objabi.

Change-Id: I65f4ce56902c6c1a82735050773c58f2d1320cc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/310169
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

* internal/buildcfg: move build configuration out of cmd/internal/objabi

The go/build package needs access to this configuration,
so move it into a new package available to the standard library.

Change-Id: I868a94148b52350c76116451f4ad9191246adcff
Reviewed-on: https://go-review.googlesource.com/c/go/+/310731
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* crypto/tls: fix flaky handshake cancellation tests

Simplified both tests significantly by removing logic for writing
the client/server side messages. The flake was likely because of a
race between the closing of the local pipe from inside the test
and closing of the pipe from within the handshakeContext goroutine.
Wait to close the local pipe in the test until after the test
has finished running.

Fixes #45106
Fixes #45299

Change-Id: If7ca75aeff7df70cda03c934fa9d8513276d465d
Reviewed-on: https://go-review.googlesource.com/c/go/+/305250
Trust: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>

* [dev.fuzz] internal/fuzz: minimize non-recoverable errors

Assuming that this works for non-recoverable errors, there
will likely be a follow-up CL which refactors the minimization
for recoverable errors to use the same RPC flow (since that
more easily allows the worker to tell the coordinator that
it's minimizing and shouldn't send more inputs to other workers
to fuzz).

Change-Id: I32ac7cec4abe2d4c345c0ee77315233047efb1fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/309509
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* internal/buildcfg: make regabi an alias for regabi sub-experiments

Currently, specifying GOEXPERIMENT=regabi will turn on all regabi
sub-experiments, but GOEXPERIMENT=noregabi won't turn anything off.
Regabi also isn't a "real" experiment in the sense that nothing in the
code base should depend on it as an experiment flag (it should depend
on the appropriate sub-experiments).

Hence, drop Regabi from goexperiment.Flags and make "regabi" in
GOEXPERIMENT be a real alias for all of the sub-flags, so regabi will
turn on all of the sub-flags and noregabi will turn off all of the
sub-flags.

This way, once we enable the sub-experiments in the baseline
configuration, it will be easy to turn off with "noregabi".

For #40724.

Change-Id: I0fb95be42f756d412e729a396be607d629ae2bab
Reviewed-on: https://go-review.googlesource.com/c/go/+/310609
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

* internal/buildcfg: make regabi enable regabiargs

For #40724.

Change-Id: I6e9d7eb91883857479699972a974a39ce3d9d2cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/310849
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

* internal/buildcfg: enable regabiwrappers by default

For #40724.

Change-Id: I75d6ba2d3e4e2d858eea8053efd0f3fd4439dab7
Reviewed-on: https://go-review.googlesource.com/c/go/+/310172
Trust: Austin Clements <austin@google.com>
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

* internal/buildcfg: enable regabig by default

For #40724.

Change-Id: Ibf4ff8b24b501813839657ac195b909682ac7d0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/310173
Trust: Austin Clements <austin@google.com>
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

* net: pass MSG_CMSG_CLOEXEC flag in ReadMsgUnix

As mentioned in #42765, calling "recvmsg" syscall on Linux should come
with "MSG_CMSG_CLOEXEC" flag.

For other systems which not supports "MSG_CMSG_CLOEXEC". ReadMsgUnix()
would check the header. If the header type is "syscall.SCM_RIGHTS",
then ReadMsgUnix() would parse the SocketControlMessage and call each
fd with "syscall.CloseOnExec"

Fixes #42765

Change-Id: I74347db72b465685d7684bf0f32415d285845ebb
GitHub-Last-Rev: ca59e2c9e0e8de1ae590e9b6dc165cb768a574f5
GitHub-Pull-Request: golang/go#42768
Reviewed-on: https://go-review.googlesource.com/c/go/+/272226
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>

* internal/buildcfg: enable regabireflect by default

For #40724.

Change-Id: Ib3e4a67c3826176f0d51619754270022344ee194
Reviewed-on: https://go-review.googlesource.com/c/go/+/310174
Trust: Austin Clements <austin@google.com>
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

* internal/buildcfg: enable regabidefer by default

For #40724.

Change-Id: If3a66c0e29cb20dd29ac13c8d00aa46ee279ab97
Reviewed-on: https://go-review.googlesource.com/c/go/+/310175
Trust: Austin Clements <austin@google.com>
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

* internal/buildcfg: enable regabiargs by default

For #40724.

Change-Id: I7509668478d20dd625f210e5a33f5d896a76d6b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/310176
Trust: Austin Clements <austin@google.com>
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

* internal/poll, net: fix comments regarding accept and sysSocket

The implementation of accept was moved from package net to internal/poll
in CL 36799.

Change-Id: I6e5964e0ee22e9c84bc444860cdd497817451fec
Reviewed-on: https://go-review.googlesource.com/c/go/+/311571
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* internal/bytealg: add power9 version of bytes index

This adds a power9 version of the bytes.Index function
for little endian.

Here is the improvement on power9 for some of the Index
benchmarks:

Index/10           -0.14%
Index/32           -3.19%
Index/4K          -12.66%
Index/4M          -13.34%
Index/64M         -13.17%
Count/10           -0.59%
Count/32           -2.88%
Count/4K          -12.63%
Count/4M          -13.35%
Count/64M         -13.17%
IndexHard1        -23.03%
IndexHard2        -13.01%
IndexHard3        -22.12%
IndexHard4         +0.16%
CountHard1        -23.02%
CountHard2        -13.01%
CountHard3        -22.12%
IndexPeriodic/IndexPeriodic2  -22.85%
IndexPeriodic/IndexPeriodic4  -23.15%

Change-Id: Id72353e2771eba2efbb1544d5f0be65f8a9f0433
Reviewed-on: https://go-review.googlesource.com/c/go/+/311380
Run-TryBot: Carlos Eduardo Seo <carlos.seo@linaro.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Eduardo Seo <carlos.seo@linaro.org>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>

* cmd/compile, internal/abi: add FuncPCABIxxx intrinsics

When ABI wrappers are used, there are cases where in Go code we
need the PC of the defined function instead of the ABI wrapper.
Currently we work around this by define such functions as
ABIInternal, even if they do not actually follow the internal ABI.

This CL introduces internal/abi.FuncPCABIxxx functions as compiler
intrinsics, which return the underlying defined function's entry
PC if the argument is a direct reference of a function of the
expected ABI, and reject it if it is of a different ABI.

As a proof of concept, change runtime.goexit back to ABI0 and use
internal/abi.FuncPCABI0 to retrieve its PC.

Updates #44065.

Change-Id: I02286f0f9d99e6a3090f9e8169dbafc6804a2da6
Reviewed-on: https://go-review.googlesource.com/c/go/+/304232
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>

* [dev.fuzz] internal/fuzz: allow setting pcg seed via GODEBUG

Format is "fuzzseed=123".

Change-Id: Idb314270c8fd4307149c8503e13424b653ec4b0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/313651
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>

* [dev.fuzz] internal/fuzz: add extra []byte mutators

Adds four []byte mutators which:
  * insert a chunk of constant bytes
  * overwirtes a chunk with constant bytes
  * shuffle a range of bytes
  * swaps two chunks

Also updates the 'set byte to random value' mutator to use XOR in
order to avoid a no-op.

Additionally updates the rng call which chooses the []byte mutators
so all the available mutators are used.

Change-Id: I0703518922952f4b1c81b19b196ee91c73b0d5f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/313270
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>

* internal/buildcfg: enable regabi for Android

This will permit us to write ABIInternal assembler code for linux-amd64.

For #40724

Change-Id: I681866651554eda4229d6faa7f0c1ba42d07e57d
Reviewed-on: https://go-review.googlesource.com/c/go/+/315390
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

* internal/syscall/unix: use internal/abi.FuncPC for syscall wrapper

Following CL 313230, this is for internal/syscall/unix package.

Updates #45702.

Change-Id: Ie6d8c1923dfeae56896212393c5c2a6e257648d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/316649
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>

* internal/buildcfg: set Error instead of panicking

All build environment validation sets Error except for the
GOEXPERIMENT parser, which panics. Change it to also set Error so that
a bad GOEXPERIMENT doesn't cause everything that imports
internal/buildcfg to panic on init.

Change-Id: Ie9a506ef0978ecb410f2dcd784638f2167354175
Reviewed-on: https://go-review.googlesource.com/c/go/+/310970
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* [dev.fuzz] internal/fuzz: don't panic if types change

There was a bug where if the types to fuzz were
different from the types in a file in the on-disk
corpus, then the code would panic. We thought
this case was handled, but the final `continue`
in the nested loop still allowed the invalid
entry to be added to the corpus. Pulling the
validation into a helper function makes this
less brittle.

Change-Id: I401346f890ea30ab7cff9640cb555da2e3ff8cc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/313810
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* crypto/tls: enforce ALPN overlap when negotiated on both sides

During the TLS handshake if the server doesn't support any of the
application protocols requested by the client, send the
no_application_protocol alert and abort the handshake on the server
side. This enforces the requirements of RFC 7301.

Change-Id: Iced2bb5c6efc607497de1c40ee3de9c2b393fa5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/289209
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>

* internal/poll: cast off the last reference of SplicePipe in test

Updates #45059

Change-Id: I9f377abcc7b77136ae6cf4896b968f73c758b559
Reviewed-on: https://go-review.googlesource.com/c/go/+/317510
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>

* [dev.fuzz] testing,internal/fuzz: prevent unbounded memory growth

Usage of f.testContext.match.fullName to generate the test name causes
unbounded memory growth, eventually causing the fuzzer to slow down
as memory pressure increases.

Each time fuzzFn is invoked it generates a unique string and stores it
in a map. With the fuzzer running at around 100k executions per second
this consumed around ~30GB of memory in a handful of minutes.

Instead just use the base name of the test for mutated inputs, a special
name for seeded inputs, and the filename for inputs from the input
corpus.

Change-Id: I083f47df7e82f0c6b0bda244f158233784a13029
Reviewed-on: https://go-review.googlesource.com/c/go/+/316030
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>

* crypto/tls: make cipher suite preference ordering automatic

We now have a (well, two, depending on AES hardware support) universal
cipher suite preference order, based on their security and performance.
Peer and application lists are now treated as filters (and AES hardware
support hints) that are applied to this universal order.

This removes a complex and nuanced decision from the application's
responsibilities, one which we are better equipped to make and which
applications usually don't need to have an opinion about. It also lets
us worry less about what suites we support or enable, because we can be
confident that bad ones won't be selected over good ones.

This also moves 3DES suites to InsecureCipherSuites(), even if they are
not disabled by default. Just because we can keep them as a last resort
it doesn't mean they are secure. Thankfully we had not promised that
Insecure means disabled by default.

Notable test changes:

  - TestCipherSuiteCertPreferenceECDSA was testing that we'd pick the
    right certificate regardless of CipherSuite ordering, which is now
    completely ignored, as tested by TestCipherSuitePreference. Removed.

  - The openssl command of TestHandshakeServerExportKeyingMaterial was
    broken for TLS 1.0 in CL 262857, but its golden file was not
    regenerated, so the test kept passing. It now broke because the
    selected suite from the ones in the golden file changed.

  - In TestAESCipherReordering, "server strongly prefers AES-GCM" is
    removed because there is no way for a server to express a strong
    preference anymore; "client prefers AES-GCM and AES-CBC over ChaCha"
    switched to ChaCha20 when the server lacks AES hardware; and finally
    "client supports multiple AES-GCM" changed to always prefer AES-128
    per the universal preference list.

    * this is going back on an explicit decision from CL 262857, and
      while that client order is weird and does suggest a strong dislike
      for ChaCha20, we have a strong dislike for software AES, so it
      didn't feel worth making the logic more complex

  - All Client-* golden files had to be regenerated because the
    ClientHello cipher suites have changed.
    (Even when Config.CipherSuites was limited to one suite, the TLS 1.3
    default order changed.)

Fixes #45430
Fixes #41476 (as 3DES is now always the last resort)

Change-Id: If5f5d356c0f8d1f1c7542fb06644a478d6bad1e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/314609
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Trust: Filippo Valsorda <filippo@golang.org>

* cmd/compile,reflect: allow longer type names

Encode the length of type names and tags in a varint encoding
instead of a fixed 2-byte encoding. This allows lengths longer
than 65535 (which can happen for large unnamed structs).

Removed the alignment check for #14962, it isn't relevant any more
since we're no longer reading pointers directly out of this data
(it is encoded as an offset which is copied out bytewise).

Fixes #44155
Update #14962

Change-Id: I6084f6027e5955dc16777c87b0dd5ea2baa49629
Reviewed-on: https://go-review.googlesource.com/c/go/+/318249
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* [dev.fuzz] internal/fuzz: use coverage instrumentation while fuzzing

This change updates the go command behavior when
fuzzing to instrument the binary for code coverage,
and uses this coverage in the fuzzing engine to
determine if an input is interesting.

Unfortunately, we can't store and use the coverage
data for a given run of `go test` and re-use it
the next time we fuzz, since the edges could have
changed between builds. Instead, every entry in
the seed corpus and the on-disk corpus is run
by the workers before fuzzing begins, so that the
coordinator can get the baseline coverage for what
the fuzzing engine has already found (or what
the developers have already provided).

Users should run `go clean -fuzzcache` before
using this change, to clear out any existing
"interesting" values that were in the cache.
Previously, every single non-crashing input was
written to the on-disk corpus. Now, only inputs
that actually expand coverage are written.

This change includes a small hack in
cmd/go/internal/load/pkg.go which ensures that the Gcflags
that were explicitly set in cmd/go/internal/test/test.go
don't get cleared out.

Tests will be added in a follow-up change, since
they will be a bit more involved.

Change-Id: Ie659222d44475c6d68fa4a35d37c37cab3619d71
Reviewed-on: https://go-review.googlesource.com/c/go/+/312009
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* all: add //go:build lines to assembly files

Don't add them to files in vendor and cmd/vendor though. These will be
pulled in by updating the respective dependencies.

For #41184

Change-Id: Icc57458c9b3033c347124323f33084c85b224c70
Reviewed-on: https://go-review.googlesource.com/c/go/+/319389
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

* [dev.fuzz] internal/fuzz: include coverage in logged stats

Change-Id: I51ec70b69e802fd0d962ba9544e96e29b1627fef
Reviewed-on: https://go-review.googlesource.com/c/go/+/319590
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>

* [dev.fuzz] internal/fuzz: move coverage capture closer to function

When instrumented packages intersect with the packages used by the
testing or internal/fuzz packages the coverage counters become noisier,
as counters will be triggered by non-fuzzed harness code.

Ideally counters would be deterministic, as there are many advanced
fuzzing strategies that require mutating the input while maintaining
static coverage.

The simplest way to mitigate this noise is to capture the coverage
counters as closely as possible to the invocation of the fuzz target
in the testing package. In order to do this add a new function which
captures the current values of the counters, SnapshotCoverage. This
function copies the current counters into a static buffer,
coverageSnapshot, which workerServer.fuzz can then inspect when it
comes time to check if new coverage has been found.

This method is not foolproof. As the fuzz target is called in a
goroutine, harness code can still cause counters to be incremented
while the target is being executed. Despite this we do see
significant reduction in churn via this approach. For example,
running a  basic target that causes strconv to be instrumented for
500,000 iterations causes ~800 unique sets of coverage counters,
whereas by capturing the counters closer to the target we get ~40
unique sets.

It may be possible to make counters completely deterministic, but
likely this would require rewriting testing/F.Fuzz to not use tRunner
in a goroutine, and instead use it in a blocking manner (which I
couldn't figure out an obvious way to do), or by doing something even
more complex.

Change-Id: I95c2f3b1d7089c3e6885fc7628a0d3a8ac1a99cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/320329
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>

* [dev.fuzz] internal/fuzz: make minimization tests more reliable

* Introduced -fuzzminimizetime flag to control the number of time or
  the number of calls to spend minimizing. Defaults to 60s. Only works
  for unrecoverable crashes for now.
* Moved the count (used by -fuzztime=1000x) into shared
  memory. Calling workerClient.fuzz resets it, but it will remain
  after the worker processes crashes. workerClient.minimize resets it
  once before restarting the worker the first time, but the total
  number of runs should still be limited during minimization, even
  after multiple terminations and restarts.
* Renamed fuzzArgs.Count to Limit to avoid confusion.
* Several other small fixes and refactorings.

Change-Id: I03faa4c94405041f6dfe48568e5ead502f8dbbd2
Reviewed-on: https://go-review.googlesource.com/c/go/+/320171
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>

* [dev.typeparams] internal/buildcfg: turn on register ABI on all AMD64 platforms

Register ABI is already enabled by default on AMD64 on Linux
(including Android), macOS, and Windows. This CL enables it on the
rest, specifically, on FreeBSD, OpenBSD, NetBSD, DragonflyBSD,
Solaris (including Illumos), iOS (simulator), and Plan 9.

Change-Id: I80fa20c8bbc8d67b16a19f71b65422e890210ab5
Reviewed-on: https://go-review.googlesource.com/c/go/+/321332
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

* [dev.fuzz] internal/fuzz: remove old TODO

Change-Id: I997934ebcde0dee9017c85a0572597855d73cf64
Reviewed-on: https://go-review.googlesource.com/c/go/+/321569
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* [dev.typeparams] internal/buildcfg: enable defer/go wrapping everywhere

For register ABI, we wrap deferred/go'd function with arguments
or results in an argumentless closure, so the runtime can call the
function without knowing how to marshal the arguments, or
reserving frame for arguments and results.

The wrapping mechanism works everywhere, regardless of whether the
register ABI is used. And wrapping will simplify the compiler and
runtime's implementation for defer and go calls. For example, the
compiler will not need to marshal arguments for defer/go calls,
the opendefer metadata will not need to contain argument
information, and _defer record will be fixed-sized.

Enable wrapping everywhere.

Change-Id: I2032ba87249ceb686310dc640fb00696669ae912
Reviewed-on: https://go-review.googlesource.com/c/go/+/321958
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>

* Revert "[dev.fuzz] internal/fuzz: include coverage in logged stats"

This reverts commit 54f067812dd870c305daabd22ca190b0f48e672e.

Reason for revert: While this is helpful for the engineering team when we're debugging, it might lead to users feeling like the fuzzer is stuck and that there are a lot of edges that are still yet to be reached. In reality, it's very likely that the compiler will instrument more lines of code than are actually reachable by the fuzz target, so showing the ratio between number of edges hit vs. all edges can be misleading. In the future, we may want to consider making this information viewable by a debug flag or something similar.

Change-Id: Ied696f8bf644445bad22c872b64daa7add605ac6
Reviewed-on: https://go-review.googlesource.com/c/go/+/322632
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* [dev.fuzz] internal/fuzz,testing: treat panics as recoverable

And only log the last panic, not all of them, during minimization.
This change makes the worker processes quiet, so now the only
process that logs anything is the coordinator. This hides all of
the panics caused during minimization of an input which causes
a panic.

This change also alters the usage of tRunner such that we now
recover from recoverable panics instead of terminating the
process. This results in larger stack traces, since we include
a bit more of the trace within testing. There is a TODO to see
if it's possible to slice the stack up so that it is somewhat
more informative.

Change-Id: Ic85eabd2e70b078412fbb88adf424a8da25af876
Reviewed-on: https://go-review.googlesource.com/c/go/+/321230
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>

* [dev.fuzz] internal/fuzz: support minimization of strings, integers, and floats

Adds support for minimizing strings using the same logic as byte slices
as well as minimizing both signed and unsigned integers and floats using
extremely basic logic. A more complex approach is probably warranted in
the future, but for now this should be _good enough_.

Change-Id: Ibc6c3d6ae82685998f571aa2c1ecea2f85c2708b
Reviewed-on: https://go-review.googlesource.com/c/go/+/320669
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

* [dev.typeparams] internal/abi: define ARM64 register ABI constants

Change-Id: I9cdf0f2b6c1739f13a859a8e37351f8ecd77804a
Reviewed-on: https://go-review.googlesource.com/c/go/+/323932
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

* [dev.typeparams] internal/buildcfg: allow regabi GOEXPERIMENTs on ARM64

It is not working yet, but allow enabling the experiments so we
can develop.

Change-Id: I957eb05acb4d80b2858ff1f8c16bbfb24e0f6e56
Reviewed-on: https://go-review.googlesource.com/c/go/+/323933
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>

* [dev.fuzz] internal/fuzz: notify coordinator for minimization

When a worker process finds a crasher, it now sends that result
directly to the coordinator without attempting to minimize it
first. The coordinator stops sending new inputs and sends the
unminimized crasher back to a worker (any worker) for minimization.

This prevents wasted work during minimization and will help us
implement -keepfuzzing later on. We may also be able to minimize
interesting inputs with this approach later.

Since panics are recoverable errors (they don't terminate worker
processes), we no longer attempt to minimize non-recoverable errors.
This didn't work too well before: we lost too much state.

Change-Id: Id142c7e91a33f64584170b0d42d22cb1f22a92d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/321835
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>

* [dev.typeparams] internal/bytealg: call memeqbody directly in memequal_varlen on ARM64

Currently, memequal_varlen opens up a frame and call memequal,
which then tail-calls memeqbody. This CL changes memequal_varlen
tail-calls memeqbody directly.

This makes it simpler to switch to the register ABI in the next
CL.

Change-Id: Ia1367c0abb7f4755fe736c404411793fb9e5c04f
Reviewed-on: https://go-review.googlesource.com/c/go/+/324399
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

* [dev.typeparams] runtime, internal/bytealg: port performance-critical functions to register ABI on ARM64

This CL ports a few performance-critical assembly functions to use
register arguments directly. This is similar to CL 308931 and
CL 310184.

Change-Id: I6e30dfff17f76b8578ce8cfd51de21b66610fdb0
Reviewed-on: https://go-review.googlesource.com/c/go/+/324400
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>

* [dev.typeparams] internal/buildcfg: turn on regabiwrappers by default on ARM64

Change-Id: I8db0a797a745630ec35af3e56406fcb250ea59fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/324768
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>

* [dev.typeparams] internal/buildcfg: turn on regabireflect by default on ARM64

Change-Id: I4a0a093b07a287cc3a3e0ee939e7ee82d8e9b1aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/324889
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>

* [dev.typeparams] internal/buildcfg: turn on register ABI by default on ARM64

This CL enables all regabi experiments on ARM64 by default.
regabiwrappers and regabireflect are enabled in the previous CLs.
regabidefer is already enabled everywhere. regabig is no-op on
ARM64 as it already has a G register. regabiargs is enabled in
this CL.

Go1 benchmarks results (GOEXPERIMENT=regabi vs. none, on macOS/ARM64):

name                     old time/op    new time/op     delta
BinaryTree17-8              1.20s ± 1%      1.02s ± 0%  -15.08%  (p=0.000 n=9+9)
Fannkuch11-8                1.55s ± 0%      1.57s ± 0%   +1.53%  (p=0.000 n=9+8)
FmtFprintfEmpty-8          22.5ns ± 3%     14.7ns ± 1%  -34.47%  (p=0.000 n=10+8)
FmtFprintfString-8         38.4ns ± 0%     28.8ns ± 0%  -24.99%  (p=0.000 n=9+9)
FmtFprintfInt-8            38.7ns ± 2%     34.5ns ± 0%  -10.79%  (p=0.000 n=10+7)
FmtFprintfIntInt-8         61.1ns ± 1%     57.9ns ± 0%   -5.23%  (p=0.000 n=10+8)
FmtFprintfPrefixedInt-8    69.9ns ± 0%     64.4ns ± 0%   -7.78%  (p=0.000 n=8+8)
FmtFprintfFloat-8           106ns ± 0%       76ns ± 0%  -28.12%  (p=0.000 n=7+10)
FmtManyArgs-8               273ns ± 0%      236ns ± 1%  -13.57%  (p=0.000 n=9+10)
GobDecode-8                3.09ms ± 1%     2.02ms ± 0%  -34.70%  (p=0.000 n=9+10)
GobEncode-8                2.45ms ± 1%     1.44ms ± 1%  -41.26%  (p=0.000 n=10+10)
Gzip-8                      128ms ± 0%      124ms ± 0%   -2.89%  (p=0.000 n=7+8)
Gunzip-8                   23.6ms ± 1%     19.8ms ± 0%  -16.15%  (p=0.000 n=10+9)
HTTPClientServer-8         27.4µs ± 1%     26.3µs ± 0%   -4.05%  (p=0.000 n=10+10)
JSONEncode-8               4.47ms ± 1%     3.45ms ± 1%  -22.73%  (p=0.000 n=10+9)
JSONDecode-8               21.5ms ± 0%     17.2ms ± 0%  -19.78%  (p=0.000 n=9+9)
Mandelbrot200-8            2.33ms ± 1%     2.33ms ± 1%     ~     (p=0.842 n=9+10)
GoParse-8                  1.62ms ± 1%     1.32ms ± 1%  -18.67%  (p=0.000 n=10+10)
RegexpMatchEasy0_32-8      33.1ns ± 0%     26.3ns ± 0%  -20.50%  (p=0.000 n=8+10)
RegexpMatchEasy0_1K-8       121ns ± 6%      121ns ± 8%     ~     (p=0.926 n=10+10)
RegexpMatchEasy1_32-8      31.4ns ± 0%     24.7ns ± 0%  -21.50%  (p=0.000 n=9+10)
RegexpMatchEasy1_1K-8       177ns ± 0%      140ns ± 0%  -20.70%  (p=0.000 n=10+9)
RegexpMatchMedium_32-8     3.02ns ± 3%     2.12ns ± 0%  -29.73%  (p=0.000 n=10+10)
RegexpMatchMedium_1K-8     19.8µs ± 2%     17.1µs ± 0%  -13.50%  (p=0.000 n=9+9)
RegexpMatchHard_32-8        940ns ± 0%      872ns ± 0%   -7.20%  (p=0.000 n=9+8)
RegexpMatchHard_1K-8       28.5µs ± 1%     26.5µs ± 0%   -7.06%  (p=0.000 n=10+10)
Revcomp-8                   186ms ± 1%      179ms ± 1%   -3.66%  (p=0.000 n=10+10)
Template-8                 30.3ms ± 0%     22.3ms ± 0%  -26.58%  (p=0.000 n=8+9)
TimeParse-8                 133ns ± 0%      117ns ± 0%  -12.40%  (p=0.000 n=10+10)
TimeFormat-8                176ns ± 0%      141ns ± 0%  -19.92%  (p=0.000 n=8+9)
[Geo mean]                 21.4µs          17.8µs       -16.81%

name                     old speed      new speed       delta
GobDecode-8               249MB/s ± 1%    381MB/s ± 0%  +53.13%  (p=0.000 n=9+10)
GobEncode-8               314MB/s ± 1%    534MB/s ± 1%  +70.25%  (p=0.000 n=10+10)
Gzip-8                    152MB/s ± 0%    156MB/s ± 0%   +2.97%  (p=0.000 n=7+8)
Gunzip-8                  822MB/s ± 1%    981MB/s ± 0%  +19.26%  (p=0.000 n=10+9)
JSONEncode-8              434MB/s ± 1%    562MB/s ± 1%  +29.41%  (p=0.000 n=10+9)
JSONDecode-8             90.3MB/s ± 0%  112.5MB/s ± 0%  +24.66%  (p=0.000 n=9+9)
GoParse-8                35.7MB/s ± 1%   43.9MB/s ± 1%  +22.96%  (p=0.000 n=10+10)
RegexpMatchEasy0_32-8     967MB/s ± 0%   1216MB/s ± 0%  +25.78%  (p=0.000 n=8+10)
RegexpMatchEasy0_1K-8    8.46GB/s ± 6%   8.45GB/s ± 7%     ~     (p=0.912 n=10+10)
RegexpMatchEasy1_32-8    1.02GB/s ± 0%   1.30GB/s ± 0%  +27.40%  (p=0.000 n=9+10)
RegexpMatchEasy1_1K-8    5.78GB/s ± 0%   7.29GB/s ± 0%  +26.10%  (p=0.000 n=10+9)
RegexpMatchMedium_32-8    331MB/s ± 2%    471MB/s ± 0%  +42.29%  (p=0.000 n=10+10)
RegexpMatchMedium_1K-8   51.7MB/s ± 2%   59.8MB/s ± 0%…

* Fix conflicts

- Resolve compilation errors caused by undefined type or conflicting type
- Remove unsupported keyword used
- Replace internal package with local package
- Support Go 1.16 with fixed build flags on cpu and testenv
- Disable broken tests
- Remove unsupported suites

* uncomment broken tests

- uncomment broken tests
- skipped for now, a patch may require extensive work to come

Co-authored-by: Tobias Klauser <tklauser@distanz.ch>
Co-authored-by: Matt T. Proud <matt.proud@gmail.com>
Co-authored-by: Matthew Dempsky <mdempsky@google.com>
Co-authored-by: Andy Pan <panjf2000@gmail.com>
Co-authored-by: John Bampton <jbampton@gmail.com>
Co-authored-by: Josh Bleecher Snyder <josharian@gmail.com>
Co-authored-by: Johan Brandhorst <johan.brandhorst@gmail.com>
Co-authored-by: Katie Hockman <katie@golang.org>
Co-authored-by: Austin Clements <austin@google.com>
Co-authored-by: Jay Conrod <jayconrod@google.com>
Co-authored-by: KimMachineGun <geon0250@gmail.com>
Co-authored-by: Ian Lance Taylor <iant@golang.org>
Co-authored-by: Naman Gera <namangera15@gmail.com>
Co-authored-by: Manlio Perillo <manlio.perillo@gmail.com>
Co-authored-by: Michael Anthony Knyszek <mknyszek@google.com>
Co-authored-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Co-authored-by: Russ Cox <rsc@golang.org>
Co-authored-by: HowJMay <vulxj0j8j8@gmail.com>
Co-authored-by: Cherry Zhang <cherryyz@google.com>
Co-authored-by: Roland Shoemaker <roland@golang.org>
Co-authored-by: Filippo Valsorda <filippo@golang.org>
Co-authored-by: Keith Randall <khr@golang.org>
Co-authored-by: cuishuang <imcusg@gmail.com>
Co-authored-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Co-authored-by: makdon <makdon@makdon.me>
Co-authored-by: Carlos Amedee <carlos@golang.org>
Co-authored-by: Dmitri Shuralyov <dmitshur@golang.org>
Co-authored-by: Matt Layher <mdlayher@gmail.com>
Co-authored-by: Martin Möhrmann <martin@golang.org>
Co-authored-by: vinckr <vincent@ory.sh>
Co-authored-by: Brad Fitzpatrick <bradfitz@golang.org>
Co-authored-by: nimelehin <nimelehin@gmail.com>
Co-authored-by: Amelia Downs <adowns@vmware.com>
Co-authored-by: nicksherron <nsherron90@gmail.com>
Co-authored-by: Koichi Shiraishi <zchee.io@gmail.com>
Co-authored-by: Archana R <aravind5@in.ibm.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Bryan C. Mills <bcmills@google.com>
Co-authored-by: David Crawshaw <crawshaw@golang.org>
Co-authored-by: Dan Kortschak <dan@kortschak.io>
Co-authored-by: Ayan George <ayan@ayan.net>
Co-authored-by: Kevin Burke <kevin@burke.dev>
Co-authored-by: Michael Matloob <matloob@golang.org>
Co-authored-by: Steven Johnstone <steven.james.johnstone@gmail.com>
Co-authored-by: Tatiana Bradley <tatiana@golang.org>
Co-authored-by: David Taylor <tinystatemachine@gmail.com>
 2022-10-17 15:09:52 -06:00
Maxb
ee9f86141f Fix invalid SNI handling 
SNIExtension was previously marshalling both ip addresses and empty
strings, which are not allowed. See RFC 6066, Section 3.

All of the utls specific testdata replays needed to be rebuilt to
properly accomodate this change since they had previously been including
empty server name extension values

Addresses https://github.com/refraction-networking/utls/issues/96
 2022-02-01 21:01:29 -08:00
Filippo Valsorda
3c6b6127c0 crypto/tls: let HTTP/1.1 clients connect to servers with NextProtos "h2" 
Fixes #46310

Change-Id: Idd5e30f05c439f736ae6f3904cbb9cc2ba772315
Reviewed-on: https://go-review.googlesource.com/c/go/+/325432
Trust: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
 2021-06-10 12:41:37 +00:00
Filippo Valsorda
89df05a1c4 crypto/tls: make cipher suite preference ordering automatic 
We now have a (well, two, depending on AES hardware support) universal
cipher suite preference order, based on their security and performance.
Peer and application lists are now treated as filters (and AES hardware
support hints) that are applied to this universal order.

This removes a complex and nuanced decision from the application's
responsibilities, one which we are better equipped to make and which
applications usually don't need to have an opinion about. It also lets
us worry less about what suites we support or enable, because we can be
confident that bad ones won't be selected over good ones.

This also moves 3DES suites to InsecureCipherSuites(), even if they are
not disabled by default. Just because we can keep them as a last resort
it doesn't mean they are secure. Thankfully we had not promised that
Insecure means disabled by default.

Notable test changes:

  - TestCipherSuiteCertPreferenceECDSA was testing that we'd pick the
    right certificate regardless of CipherSuite ordering, which is now
    completely ignored, as tested by TestCipherSuitePreference. Removed.

  - The openssl command of TestHandshakeServerExportKeyingMaterial was
    broken for TLS 1.0 in CL 262857, but its golden file was not
    regenerated, so the test kept passing. It now broke because the
    selected suite from the ones in the golden file changed.

  - In TestAESCipherReordering, "server strongly prefers AES-GCM" is
    removed because there is no way for a server to express a strong
    preference anymore; "client prefers AES-GCM and AES-CBC over ChaCha"
    switched to ChaCha20 when the server lacks AES hardware; and finally
    "client supports multiple AES-GCM" changed to always prefer AES-128
    per the universal preference list.

    * this is going back on an explicit decision from CL 262857, and
      while that client order is weird and does suggest a strong dislike
      for ChaCha20, we have a strong dislike for software AES, so it
      didn't feel worth making the logic more complex

  - All Client-* golden files had to be regenerated because the
    ClientHello cipher suites have changed.
    (Even when Config.CipherSuites was limited to one suite, the TLS 1.3
    default order changed.)

Fixes #45430
Fixes #41476 (as 3DES is now always the last resort)

Change-Id: If5f5d356c0f8d1f1c7542fb06644a478d6bad1e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/314609
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Trust: Filippo Valsorda <filippo@golang.org>
 2021-05-08 05:15:48 +00:00
Roland Shoemaker
7d3285645e crypto/tls: enforce ALPN overlap when negotiated on both sides 
During the TLS handshake if the server doesn't support any of the
application protocols requested by the client, send the
no_application_protocol alert and abort the handshake on the server
side. This enforces the requirements of RFC 7301.

Change-Id: Iced2bb5c6efc607497de1c40ee3de9c2b393fa5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/289209
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
 2021-05-06 18:57:43 +00:00
maxb
2179f28668
Implement FingerprintClientHello to generate ClientHelloSpec from ClientHello raw bytes (#67) 2020-12-09 22:37:06 -07:00
Roland Shoemaker
18d259497e crypto/tls: de-prioritize AES-GCM ciphers when lacking hardware support 
When either the server or client are lacking hardware support for
AES-GCM ciphers, indicated by the server lacking the relevant
instructions and by the client not putting AES-GCM ciphers at the top
of its preference list, reorder the preference list to de-prioritize
AES-GCM based ciphers when they are adjacent to other AEAD ciphers.

Also updates a number of recorded openssl TLS tests which previously
only specified TLS 1.2 cipher preferences (using -cipher), but not
TLS 1.3 cipher preferences (using -ciphersuites), to specify both
preferences, making these tests more predictable.

Fixes #41181.

Change-Id: Ied896c96c095481e755aaff9ff0746fb4cb9568e
Reviewed-on: https://go-review.googlesource.com/c/go/+/262857
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
 2020-11-10 01:40:27 +00:00
maxb
4ee6244579
Fix omitSNI test (#59) 2020-08-14 13:45:39 -06:00
maxb
dc2ae3bffe
Implement uconn.RemoveSNIExtension() (#51) 
Authored by: https://github.com/max-b
Co-authored-by: Myles Horton <myles@getlantern.org>
Co-authored-by: Willie Forkner <1120829+forkner@users.noreply.github.com>
 2020-07-28 19:25:16 -06:00
Katie Hockman
451074ba19 crypto/tls: rotate session keys in older TLS versions 
Also encode the certificates in a way that's more
consistent with TLS 1.3 (with a 24 byte length prefix).

Note that this will have an additional performance cost
requiring clients to do a full handshake every 7 days
where previously they were able to use the same ticket
indefinitely.

Updates #25256

Change-Id: Ic4d1ba0d92773c490b33b5f6c1320d557cc7347d
Reviewed-on: https://go-review.googlesource.com/c/go/+/231317
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
 2020-05-07 18:33:39 +00:00
Filippo Valsorda
aff697f53d crypto/tls: take key size into account in signature algorithm selection 
Fixes #29793

Change-Id: I6e389d166c2d9a2ba8664a41f4b9569f2481b27f
Reviewed-on: https://go-review.googlesource.com/c/go/+/205177
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
 2019-11-12 01:09:40 +00:00
Filippo Valsorda
6bb85fe4e7 crypto/tls: re-enable RSA-PSS in TLS 1.2 again 
TLS 1.3, which requires RSA-PSS, is now enabled without a GODEBUG
opt-out, and with the introduction of
Certificate.SupportedSignatureAlgorithms (#28660) there is a
programmatic way to avoid RSA-PSS (disable TLS 1.3 with MaxVersion and
use that field to specify only PKCS#1 v1.5 SignatureSchemes).

This effectively reverts 0b3a57b5374bba3fdf88258e2be4c8be65e6a5de,
although following CL 205061 all of the signing-side logic is
conveniently centralized in signatureSchemesForCertificate.

Fixes #32425

Change-Id: I7c9a8893bb5d518d86eae7db82612b9b2cd257d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/205063
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
 2019-11-12 01:09:15 +00:00
Olivier Poitrey
e580b52d55 crypto/tls: send ec_points_format extension in ServerHello 
Follow the recommandation from RFC 8422, section 5.1.2 of sending back the
ec_points_format extension when requested by the client. This is to fix
some clients declining the handshake if omitted.

Fixes #31943

Change-Id: I7b04dbac6f9af75cda094073defe081e1e9a295d
Reviewed-on: https://go-review.googlesource.com/c/go/+/176418
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Olivier Poitrey <rs@rhapsodyk.net>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
 2019-10-31 15:14:45 +00:00
Brad Fitzpatrick
6b5dc9f4ba crypto/tls: remove NPN support 
RELNOTE=yes

Fixes #28362

Change-Id: I43813c0c17bbe6c4cbb4d1f121518c434b3f5aa8
Reviewed-on: https://go-review.googlesource.com/c/go/+/174329
Reviewed-by: Filippo Valsorda <filippo@golang.org>
 2019-10-04 18:07:46 +00:00
Filippo Valsorda
018f13d1a3 crypto/tls: remove SSLv3 support 
SSLv3 has been irreparably broken since the POODLE attack 5 years ago
and RFC 7568 (f.k.a. draft-ietf-tls-sslv3-diediedie) prohibits its use
in no uncertain terms.

As announced in the Go 1.13 release notes, remove support for it
entirely in Go 1.14.

Updates #32716

Change-Id: Id653557961d8f75f484a01e6afd2e104a4ccceaf
Reviewed-on: https://go-review.googlesource.com/c/go/+/191976
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
 2019-08-27 22:24:05 +00:00
Sergey Frolov
ceee37ab69 Merge branch golang crypto/tls upstream 2019-08-22 17:42:47 -06:00
Filippo Valsorda
9a45e56dc1 crypto/tls: disable RSA-PSS in TLS 1.2 again 
Signing with RSA-PSS can uncover faulty crypto.Signer implementations,
and it can fail for (broken) small keys. We'll have to take that
breakage eventually, but it would be nice for it to be opt-out at first.

TLS 1.3 requires RSA-PSS and is opt-out in Go 1.13. Instead of making a
TLS 1.3 opt-out influence a TLS 1.2 behavior, let's wait to add RSA-PSS
to TLS 1.2 until TLS 1.3 is on without opt-out.

Note that since the Client Hello is sent before a protocol version is
selected, we have to advertise RSA-PSS there to support TLS 1.3.
That means that we still support RSA-PSS on the client in TLS 1.2 for
verifying server certificates, which is fine, as all issues arise on the
signing side. We have to be careful not to pick (or consider available)
RSA-PSS on the client for client certificates, though.

We'd expect tests to change only in TLS 1.2:

    * the server won't pick PSS to sign the key exchange
      (Server-TLSv12-* w/ RSA, TestHandshakeServerRSAPSS);
    * the server won't advertise PSS in CertificateRequest
      (Server-TLSv12-ClientAuthRequested*, TestClientAuth);
    * and the client won't pick PSS for its CertificateVerify
      (Client-TLSv12-ClientCert-RSA-*, TestHandshakeClientCertRSAPSS,
      Client-TLSv12-Renegotiate* because "R" requests a client cert).

Client-TLSv13-ClientCert-RSA-RSAPSS was updated because of a fix in the test.

This effectively reverts 88343530720a52c96b21f2bd5488c8fb607605d7.

Testing was made more complex by the undocumented semantics of OpenSSL's
-[client_]sigalgs (see openssl/openssl#9172).

Updates #32425

Change-Id: Iaddeb2df1f5c75cd090cc8321df2ac8e8e7db349
Reviewed-on: https://go-review.googlesource.com/c/go/+/182339
Reviewed-by: Adam Langley <agl@golang.org>
 2019-06-19 19:59:14 +00:00
Filippo Valsorda
28958b0da6 crypto/tls: add support for Ed25519 certificates in TLS 1.2 and 1.3 
Support for Ed25519 certificates was added in CL 175478, this wires them
up into the TLS stack according to RFC 8422 (TLS 1.2) and RFC 8446 (TLS 1.3).

RFC 8422 also specifies support for TLS 1.0 and 1.1, and I initially
implemented that, but even OpenSSL doesn't take the complexity, so I
just dropped it. It would have required keeping a buffer of the
handshake transcript in order to do the direct Ed25519 signatures. We
effectively need to support TLS 1.2 because it shares ClientHello
signature algorithms with TLS 1.3.

While at it, reordered the advertised signature algorithms in the rough
order we would want to use them, also based on what curves have fast
constant-time implementations.

Client and client auth tests changed because of the change in advertised
signature algorithms in ClientHello and CertificateRequest.

Fixes #25355

Change-Id: I9fdd839afde4fd6b13fcbc5cc7017fd8c35085ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/177698
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
 2019-05-17 16:13:45 +00:00
Filippo Valsorda
2e9a42ccc2 Revert "crypto/tls: disable RSA-PSS in TLS 1.2" 
In Go 1.13 we will enable RSA-PSS in TLS 1.2 at the same time as we make
TLS 1.3 enabled by default.

This reverts commit 7ccd3583eddcd79679fb29cfc83a6e6fb6973f1e.

Updates #30055

Change-Id: I6f2ddf7652d1172a6b29f4e335ff3a71a89974bc
Reviewed-on: https://go-review.googlesource.com/c/163080
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
 2019-02-27 07:54:00 +00:00
Sergey Frolov
1188641a16 Correctly handle HelloRetryRequest 2019-02-22 09:51:44 -07:00
Filippo Valsorda
b5bf3890ae crypto/tls: disable RSA-PSS in TLS 1.2 
Most of the issues that led to the decision on #30055 were related to
incompatibility with or faulty support for RSA-PSS (#29831, #29779,
v1.5 signatures). RSA-PSS is required by TLS 1.3, but is also available
to be negotiated in TLS 1.2.

Altering TLS 1.2 behavior based on GODEBUG=tls13=1 feels surprising, so
just disable RSA-PSS entirely in TLS 1.2 until TLS 1.3 is on by default,
so breakage happens all at once.

Updates #30055

Change-Id: Iee90454a20ded8895e5302e8bcbcd32e4e3031c2
Reviewed-on: https://go-review.googlesource.com/c/160998
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
 2019-02-07 18:34:43 +00:00
Sergey Frolov
b84d7d5f05 +tls13 extensions; +Chrome 70, Firefox 63 parrots 
Adds support for following TLS 1.3 extensions:
 - PSKKeyExchangeModes
 - SupportedVersions
 - KeyShare
and uses them to implement newest Chrome and Firefox parrots.

Tests for default Golang uTLS were regenerated because
they previously used TLS-1.2 as max version.
 2018-12-18 17:53:26 -07:00
Sergey Frolov
109af06ea2 Adapt uTLS for new TLS 1.3 code 2018-12-05 15:12:21 -07:00
Sergey Frolov
23b79cad14 Merge TLS 1.3 from upstream 
uTLS not adapted to this change yet,
uTLS changes will be in a separate commmit.
 2018-12-03 12:55:53 -07:00
Filippo Valsorda
daa7ff8195 crypto/tls: fix client certificates support for legacy servers 
signatureSchemesForCertificate was written to be used with TLS 1.3, but
ended up used for TLS 1.2 client certificates in a refactor. Since it
only supported TLS 1.3 signature algorithms, it would lead to no RSA
client certificates being sent to servers that didn't support RSA-PSS.

TestHandshakeClientCertRSAPKCS1v15 was testing *specifically* for this,
but alas the OpenSSL flag -verify accepts an empty certificates list as
valid, as opposed to -Verify...

Fixes #28925

Change-Id: I61afc02ca501d3d64ab4ad77bbb4cf10931e6f93
Reviewed-on: https://go-review.googlesource.com/c/151660
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
 2018-11-30 19:10:38 +00:00
Filippo Valsorda
b4c934319a crypto/tls: prevent the test server from sending session tickets 
Since they are sent after the handshake in TLS 1.3, the client was not
actually consuming them, as it doesn't make any Read calls. They were
then sitting in the kernel receive buffer when the client would call
Close. The kernel would see that and send a RST, which would race the
closeNotify, causing errors.

Also, we get to trim 600 lines of useless test data.

Fixes #28852

Change-Id: I7517feab77dabab7504bfc111098ba09ea07ae5e
Reviewed-on: https://go-review.googlesource.com/c/151659
Reviewed-by: Ian Lance Taylor <iant@golang.org>
 2018-11-29 05:55:51 +00:00
Filippo Valsorda
5db23cd389 crypto/tls: enable TLS 1.3 and update tests 
To disable TLS 1.3, simply remove VersionTLS13 from supportedVersions,
as tested by TestEscapeRoute, and amend documentation. To make it
opt-in, revert the change to (*Config).supportedVersions from this CL.

I did not have the heart to implement the early data skipping feature
when I realized that it did not offer a choice between two
abstraction-breaking options, but demanded them both (look for handshake
type in case of HelloRetryRequest, trial decryption otherwise). It's a
lot of complexity for an apparently small gain, but if anyone has strong
opinions about it let me know.

Note that in TLS 1.3 alerts are encrypted, so the close_notify peeking
to return (n > 0, io.EOF) from Read doesn't work. If we are lucky, those
servers that unexpectedly close connections after serving a single
request will have stopped (maybe thanks to H/2) before they got updated
to TLS 1.3.

Relatedly, session tickets are now provisioned on the client first Read
instead of at Handshake time, because they are, well, post-handshake
messages. If this proves to be a problem we might try to peek at them.

Doubled the tests that cover logic that's different in TLS 1.3.

The benchmarks for TLS 1.2 compared to be0f3c286b5 (before TLS 1.3 and
its refactors, after CL 142817 changed them to use real connections)
show little movement.

name                                       old time/op   new time/op   delta
HandshakeServer/RSA-8                        795µs ± 1%    798µs ± 1%    ~     (p=0.057 n=10+18)
HandshakeServer/ECDHE-P256-RSA-8             903µs ± 0%    909µs ± 1%  +0.68%  (p=0.000 n=8+17)
HandshakeServer/ECDHE-P256-ECDSA-P256-8      198µs ± 0%    204µs ± 1%  +3.24%  (p=0.000 n=9+18)
HandshakeServer/ECDHE-X25519-ECDSA-P256-8    202µs ± 3%    208µs ± 1%  +2.98%  (p=0.000 n=9+20)
HandshakeServer/ECDHE-P521-ECDSA-P521-8     15.5ms ± 1%   15.9ms ± 2%  +2.49%  (p=0.000 n=10+20)
Throughput/MaxPacket/1MB-8                  5.81ms ±23%   6.14ms ±44%    ~     (p=0.605 n=8+18)
Throughput/MaxPacket/2MB-8                  8.91ms ±22%   8.74ms ±33%    ~     (p=0.498 n=9+19)
Throughput/MaxPacket/4MB-8                  12.8ms ± 3%   14.0ms ±10%  +9.74%  (p=0.000 n=10+17)
Throughput/MaxPacket/8MB-8                  25.1ms ± 7%   24.6ms ±16%    ~     (p=0.129 n=9+19)
Throughput/MaxPacket/16MB-8                 46.3ms ± 4%   45.9ms ±12%    ~     (p=0.340 n=9+20)
Throughput/MaxPacket/32MB-8                 88.5ms ± 4%   86.0ms ± 4%  -2.82%  (p=0.004 n=10+20)
Throughput/MaxPacket/64MB-8                  173ms ± 2%    167ms ± 7%  -3.42%  (p=0.001 n=10+19)
Throughput/DynamicPacket/1MB-8              5.88ms ± 4%   6.59ms ±64%    ~     (p=0.232 n=9+18)
Throughput/DynamicPacket/2MB-8              9.08ms ±12%   8.73ms ±21%    ~     (p=0.408 n=10+18)
Throughput/DynamicPacket/4MB-8              14.2ms ± 5%   14.0ms ±11%    ~     (p=0.188 n=9+19)
Throughput/DynamicPacket/8MB-8              25.1ms ± 6%   24.0ms ± 7%  -4.39%  (p=0.000 n=10+18)
Throughput/DynamicPacket/16MB-8             45.6ms ± 3%   43.3ms ± 1%  -5.22%  (p=0.000 n=10+8)
Throughput/DynamicPacket/32MB-8             88.4ms ± 3%   84.8ms ± 2%  -4.06%  (p=0.000 n=10+10)
Throughput/DynamicPacket/64MB-8              175ms ± 3%    167ms ± 2%  -4.63%  (p=0.000 n=10+10)
Latency/MaxPacket/200kbps-8                  694ms ± 0%    694ms ± 0%  -0.02%  (p=0.000 n=9+9)
Latency/MaxPacket/500kbps-8                  279ms ± 0%    279ms ± 0%  -0.09%  (p=0.000 n=10+10)
Latency/MaxPacket/1000kbps-8                 140ms ± 0%    140ms ± 0%  -0.15%  (p=0.000 n=10+9)
Latency/MaxPacket/2000kbps-8                71.1ms ± 0%   71.0ms ± 0%  -0.09%  (p=0.001 n=8+9)
Latency/MaxPacket/5000kbps-8                30.5ms ± 6%   30.1ms ± 6%    ~     (p=0.905 n=10+9)
Latency/DynamicPacket/200kbps-8              134ms ± 0%    134ms ± 0%    ~     (p=0.796 n=9+9)
Latency/DynamicPacket/500kbps-8             54.8ms ± 0%   54.7ms ± 0%  -0.18%  (p=0.000 n=8+10)
Latency/DynamicPacket/1000kbps-8            28.5ms ± 0%   29.1ms ± 8%    ~     (p=0.173 n=8+10)
Latency/DynamicPacket/2000kbps-8            15.3ms ± 6%   15.9ms ±10%    ~     (p=0.905 n=9+10)
Latency/DynamicPacket/5000kbps-8            9.14ms ±21%   9.65ms ±82%    ~     (p=0.529 n=10+10)

name                                       old speed     new speed     delta
Throughput/MaxPacket/1MB-8                 175MB/s ±13%  167MB/s ±64%    ~     (p=0.646 n=7+20)
Throughput/MaxPacket/2MB-8                 241MB/s ±25%  241MB/s ±40%    ~     (p=0.660 n=9+20)
Throughput/MaxPacket/4MB-8                 328MB/s ± 3%  300MB/s ± 9%  -8.70%  (p=0.000 n=10+17)
Throughput/MaxPacket/8MB-8                 335MB/s ± 7%  340MB/s ±17%    ~     (p=0.212 n=9+20)
Throughput/MaxPacket/16MB-8                363MB/s ± 4%  367MB/s ±11%    ~     (p=0.340 n=9+20)
Throughput/MaxPacket/32MB-8                379MB/s ± 4%  390MB/s ± 4%  +2.93%  (p=0.004 n=10+20)
Throughput/MaxPacket/64MB-8                388MB/s ± 2%  401MB/s ± 7%  +3.25%  (p=0.004 n=10+20)
Throughput/DynamicPacket/1MB-8             178MB/s ± 4%  157MB/s ±73%    ~     (p=0.127 n=9+20)
Throughput/DynamicPacket/2MB-8             232MB/s ±11%  243MB/s ±18%    ~     (p=0.415 n=10+18)
Throughput/DynamicPacket/4MB-8             296MB/s ± 5%  299MB/s ±15%    ~     (p=0.295 n=9+20)
Throughput/DynamicPacket/8MB-8             334MB/s ± 6%  350MB/s ± 7%  +4.58%  (p=0.000 n=10+18)
Throughput/DynamicPacket/16MB-8            368MB/s ± 3%  388MB/s ± 1%  +5.48%  (p=0.000 n=10+8)
Throughput/DynamicPacket/32MB-8            380MB/s ± 3%  396MB/s ± 2%  +4.20%  (p=0.000 n=10+10)
Throughput/DynamicPacket/64MB-8            384MB/s ± 3%  403MB/s ± 2%  +4.83%  (p=0.000 n=10+10)

Comparing TLS 1.2 and TLS 1.3 at tip shows a slight (~5-10%) slowdown of
handshakes, which might be worth looking at next cycle, but the latency
improvements are expected to overshadow that.

name                                       old time/op   new time/op   delta
HandshakeServer/ECDHE-P256-RSA-8             909µs ± 1%    963µs ± 0%   +5.87%  (p=0.000 n=17+18)
HandshakeServer/ECDHE-P256-ECDSA-P256-8      204µs ± 1%    225µs ± 2%  +10.20%  (p=0.000 n=18+20)
HandshakeServer/ECDHE-X25519-ECDSA-P256-8    208µs ± 1%    230µs ± 2%  +10.35%  (p=0.000 n=20+18)
HandshakeServer/ECDHE-P521-ECDSA-P521-8     15.9ms ± 2%   15.9ms ± 1%     ~     (p=0.444 n=20+19)
Throughput/MaxPacket/1MB-8                  6.14ms ±44%   7.07ms ±46%     ~     (p=0.057 n=18+19)
Throughput/MaxPacket/2MB-8                  8.74ms ±33%   8.61ms ± 9%     ~     (p=0.552 n=19+17)
Throughput/MaxPacket/4MB-8                  14.0ms ±10%   14.1ms ±12%     ~     (p=0.707 n=17+20)
Throughput/MaxPacket/8MB-8                  24.6ms ±16%   25.6ms ±14%     ~     (p=0.107 n=19+20)
Throughput/MaxPacket/16MB-8                 45.9ms ±12%   44.7ms ± 6%     ~     (p=0.607 n=20+19)
Throughput/MaxPacket/32MB-8                 86.0ms ± 4%   87.9ms ± 8%     ~     (p=0.113 n=20+19)
Throughput/MaxPacket/64MB-8                  167ms ± 7%    169ms ± 2%   +1.26%  (p=0.011 n=19+19)
Throughput/DynamicPacket/1MB-8              6.59ms ±64%   6.79ms ±43%     ~     (p=0.480 n=18+19)
Throughput/DynamicPacket/2MB-8              8.73ms ±21%   9.58ms ±13%   +9.71%  (p=0.006 n=18+20)
Throughput/DynamicPacket/4MB-8              14.0ms ±11%   13.9ms ±10%     ~     (p=0.687 n=19+20)
Throughput/DynamicPacket/8MB-8              24.0ms ± 7%   24.6ms ± 8%   +2.36%  (p=0.045 n=18+17)
Throughput/DynamicPacket/16MB-8             43.3ms ± 1%   44.3ms ± 2%   +2.48%  (p=0.001 n=8+9)
Throughput/DynamicPacket/32MB-8             84.8ms ± 2%   86.7ms ± 2%   +2.27%  (p=0.000 n=10+10)
Throughput/DynamicPacket/64MB-8              167ms ± 2%    170ms ± 3%   +1.89%  (p=0.005 n=10+10)
Latency/MaxPacket/200kbps-8                  694ms ± 0%    699ms ± 0%   +0.65%  (p=0.000 n=9+10)
Latency/MaxPacket/500kbps-8                  279ms ± 0%    280ms ± 0%   +0.68%  (p=0.000 n=10+10)
Latency/MaxPacket/1000kbps-8                 140ms ± 0%    141ms ± 0%   +0.59%  (p=0.000 n=9+9)
Latency/MaxPacket/2000kbps-8                71.0ms ± 0%   71.3ms ± 0%   +0.42%  (p=0.000 n=9+9)
Latency/MaxPacket/5000kbps-8                30.1ms ± 6%   30.7ms ±10%   +1.93%  (p=0.019 n=9+9)
Latency/DynamicPacket/200kbps-8              134ms ± 0%    138ms ± 0%   +3.22%  (p=0.000 n=9+10)
Latency/DynamicPacket/500kbps-8             54.7ms ± 0%   56.3ms ± 0%   +3.03%  (p=0.000 n=10+8)
Latency/DynamicPacket/1000kbps-8            29.1ms ± 8%   29.1ms ± 0%     ~     (p=0.173 n=10+8)
Latency/DynamicPacket/2000kbps-8            15.9ms ±10%   16.4ms ±36%     ~     (p=0.633 n=10+8)
Latency/DynamicPacket/5000kbps-8            9.65ms ±82%   8.32ms ± 8%     ~     (p=0.573 n=10+8)

name                                       old speed     new speed     delta
Throughput/MaxPacket/1MB-8                 167MB/s ±64%  155MB/s ±55%     ~     (p=0.224 n=20+19)
Throughput/MaxPacket/2MB-8                 241MB/s ±40%  244MB/s ± 9%     ~     (p=0.407 n=20+17)
Throughput/MaxPacket/4MB-8                 300MB/s ± 9%  298MB/s ±11%     ~     (p=0.707 n=17+20)
Throughput/MaxPacket/8MB-8                 340MB/s ±17%  330MB/s ±13%     ~     (p=0.201 n=20+20)
Throughput/MaxPacket/16MB-8                367MB/s ±11%  375MB/s ± 5%     ~     (p=0.607 n=20+19)
Throughput/MaxPacket/32MB-8                390MB/s ± 4%  382MB/s ± 8%     ~     (p=0.113 n=20+19)
Throughput/MaxPacket/64MB-8                401MB/s ± 7%  397MB/s ± 2%   -0.96%  (p=0.030 n=20+19)
Throughput/DynamicPacket/1MB-8             157MB/s ±73%  156MB/s ±39%     ~     (p=0.738 n=20+20)
Throughput/DynamicPacket/2MB-8             243MB/s ±18%  220MB/s ±14%   -9.65%  (p=0.006 n=18+20)
Throughput/DynamicPacket/4MB-8             299MB/s ±15%  303MB/s ± 9%     ~     (p=0.512 n=20+20)
Throughput/DynamicPacket/8MB-8             350MB/s ± 7%  342MB/s ± 8%   -2.27%  (p=0.045 n=18+17)
Throughput/DynamicPacket/16MB-8            388MB/s ± 1%  378MB/s ± 2%   -2.41%  (p=0.001 n=8+9)
Throughput/DynamicPacket/32MB-8            396MB/s ± 2%  387MB/s ± 2%   -2.21%  (p=0.000 n=10+10)
Throughput/DynamicPacket/64MB-8            403MB/s ± 2%  396MB/s ± 3%   -1.84%  (p=0.005 n=10+10)

Fixes #9671

Change-Id: Ieb57c5140eb2c083b8be0d42b240cd2eeec0dcf6
Reviewed-on: https://go-review.googlesource.com/c/147638
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
 2018-11-12 20:44:39 +00:00
Filippo Valsorda
b523d280e4 crypto/tls: implement TLS 1.3 client authentication 
Note that the SignatureSchemes passed to GetClientCertificate in TLS 1.2
are now filtered by the requested certificate type. This feels like an
improvement anyway, and the full list can be surfaced as well when
support for signature_algorithms_cert is added, which actually matches
the semantics of the CertificateRequest signature_algorithms in TLS 1.2.

Also, note a subtle behavior change in server side resumption: if a
certificate is requested but not required, and the resumed session did
not include one, it used not to invoke VerifyPeerCertificate. However,
if the resumed session did include a certificate, it would. (If a
certificate was required but not in the session, the session is rejected
in checkForResumption.) This inconsistency could be unexpected, even
dangerous, so now VerifyPeerCertificate is always invoked. Still not
consistent with the client behavior, which does not ever invoke
VerifyPeerCertificate on resumption, but it felt too surprising to
entirely change either.

Updates #9671

Change-Id: Ib2b0dbc30e659208dca3ac07d6c687a407d7aaaf
Reviewed-on: https://go-review.googlesource.com/c/147599
Reviewed-by: Adam Langley <agl@golang.org>
 2018-11-12 20:43:55 +00:00
Filippo Valsorda
166c58b85c crypto/tls: implement TLS 1.3 PSK authentication (server side) 
Added some assertions to testHandshake, but avoided checking the error
of one of the Close() because the one that would lose the race would
write the closeNotify to a connection closed on the other side which is
broken on js/wasm (#28650). Moved that Close() after the chan sync to
ensure it happens second.

Accepting a ticket with client certificates when NoClientCert is
configured is probably not a problem, and we could hide them to avoid
confusing the application, but the current behavior is to skip the
ticket, and I'd rather keep behavior changes to a minimum.

Updates #9671

Change-Id: I93b56e44ddfe3d48c2bef52c83285ba2f46f297a
Reviewed-on: https://go-review.googlesource.com/c/147445
Reviewed-by: Adam Langley <agl@golang.org>
 2018-11-12 20:43:35 +00:00
Filippo Valsorda
5b79a7c982 crypto/tls: implement TLS 1.3 middlebox compatibility mode 
Looks like the introduction of CCS records in the client second flight
gave time to s_server to send NewSessionTicket messages in between the
client application data and close_notify. There seems to be no way of
turning NewSessionTicket messages off, neither by not sending a
psk_key_exchange_modes extension, nor by command line flag.

Interleaving the client write like that tickled an issue akin to #18701:
on Windows, the client reaches Close() before the last record is drained
from the send buffer, the kernel notices and resets the connection,
cutting short the last flow. There is no good way of synchronizing this,
so we sleep for a RTT before calling close, like in CL 75210. Sigh.

Updates #9671

Change-Id: I44dc1cca17b373695b5a18c2741f218af2990bd1
Reviewed-on: https://go-review.googlesource.com/c/147419
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
 2018-11-12 20:43:06 +00:00
Filippo Valsorda
e04a8ac694 crypto/tls: implement TLS 1.3 KeyUpdate messages 
Since TLS 1.3 delivers handshake messages (including KeyUpdate) after
the handshake, the want argument to readRecord had became almost
pointless: it only meant something when set to recordTypeChangeCipherSpec.
Replaced it with a bool to reflect that, and added two shorthands to
avoid anonymous bools in calls.

Took the occasion to simplify and formalize the invariants of readRecord.

The maxConsecutiveEmptyRecords loop became useless when readRecord
started retrying on any non-advancing record in CL 145297.

Replaced panics with errors, because failure is better than undefined
behavior, but contained failure is better than a DoS vulnerability. For
example, I suspect the panic at the top of readRecord was reachable from
handleRenegotiation, which calls readHandshake with handshakeComplete
false. Thankfully it was not a panic in 1.11, and it's allowed now.

Removed Client-TLSv13-RenegotiationRejected because OpenSSL isn't
actually willing to ask for renegotiation over TLS 1.3, the expected
error was due to NewSessionTicket messages, which didn't break the rest
of the tests because they stop too soon.

Updates #9671

Change-Id: I297a81bde5c8020a962a92891b70d6d70b90f5e3
Reviewed-on: https://go-review.googlesource.com/c/147418
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
 2018-11-12 20:42:36 +00:00
Filippo Valsorda
376ff45dc1 crypto/tls: implement TLS 1.3 server handshake (base) 
Implement a basic TLS 1.3 server handshake, only enabled if explicitly
requested with MaxVersion.

This CL intentionally leaves for future CLs:
  - PSK modes and resumption
  - client authentication
  - compatibility mode ChangeCipherSpecs
  - early data skipping
  - post-handshake messages
  - downgrade protection
  - KeyLogWriter support
  - TLS_FALLBACK_SCSV processing

It also leaves a few areas up for a wider refactor (maybe in Go 1.13):
  - the certificate selection logic can be significantly improved,
    including supporting and surfacing signature_algorithms_cert, but
    this isn't new in TLS 1.3 (see comment in processClientHello)
  - handshake_server_tls13.go can be dried up and broken into more
    meaningful, smaller functions, but it felt premature to do before
    PSK and client auth support
  - the monstrous ClientHello equality check in doHelloRetryRequest can
    get both cleaner and more complete with collaboration from the
    parsing layer, which can come at the same time as extension
    duplicates detection

Updates #9671

Change-Id: Id9db2b6ecc2eea21bf9b59b6d1d9c84a7435151c
Reviewed-on: https://go-review.googlesource.com/c/147017
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
 2018-11-02 22:07:43 +00:00
Filippo Valsorda
2c3ff7ba06 crypto/tls: implement TLS 1.3 client handshake (base) 
Implement a basic TLS 1.3 client handshake, only enabled if explicitly
requested with MaxVersion.

This CL intentionally leaves for future CLs:
  - PSK modes and resumption
  - client authentication
  - post-handshake messages
  - downgrade protection
  - KeyLogWriter support

Updates #9671

Change-Id: Ieb6130fb6f25aea4f0d39e3a2448dfc942e1de7a
Reviewed-on: https://go-review.googlesource.com/c/146559
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
 2018-11-02 22:07:02 +00:00
Filippo Valsorda
ed74f7823e crypto/tls: advertise and accept rsa_pss_rsae signature algorithms 
crypto/x509 already supports PSS signatures (with rsaEncryption OID),
and crypto/tls support was added in CL 79736. Advertise support for the
algorithms and accept them as a peer.

Note that this is about PSS signatures from regular RSA public keys.
RSA-PSS only public keys (with RSASSA-PSS OID) are supported in neither
crypto/tls nor crypto/x509. See RFC 8446, Section 4.2.3.

testdata/Server-TLSv12-ClientAuthRequested* got modified because the
CertificateRequest carries the supported signature algorithms.

The net/smtp tests changed because 512 bits keys are too small for PSS.

Based on Peter Wu's CL 79738, who did all the actual work in CL 79736.

Updates #9671

Change-Id: I4a31e9c6e152ff4c50a5c8a274edd610d5fff231
Reviewed-on: https://go-review.googlesource.com/c/146258
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
 2018-11-02 22:05:52 +00:00
Filippo Valsorda
0a9fc9c88a crypto/tls: make ConnectionState.ExportKeyingMaterial a method 
The unexported field is hidden from reflect based marshalers, which
would break otherwise. Also, make it return an error, as there are
multiple reasons it might fail.

Fixes #27125

Change-Id: I92adade2fe456103d2d5c0315629ca0256953764
Reviewed-on: https://go-review.googlesource.com/130535
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
 2018-08-22 03:48:56 +00:00
Sergey Frolov
112951f6d7 Refactor: enable flexible external configuration 2018-06-21 13:55:02 -04:00