This reverts commit 413c125da38990720744c0d98ab65c0d5b1602da.
Reason for revert: Giving this more thought, we've decided that
converting types under the hood may cause unexpected behavior to
users. This is a feature that can always be added after more
consideration has been done, but is not something that can be
removed due to the backwards compatibility promise.
Updates golang/go#45593
Change-Id: I79bab24979d7e4c294e6cb6455d4c7729d6a0efb
Reviewed-on: https://go-review.googlesource.com/c/go/+/350251
Trust: Katie Hockman <katie@golang.org>
Trust: Joe Tsai <joetsai@digital-static.net>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Joe Tsai <joetsai@digital-static.net>
This change refactors some of the code to support skipping a run
of the seed corpus by the go command before runFuzzing occurs.
Previously, the go command would run all seed corpus for all targets
that match the provided `run` argument. This will be redundant when
fuzzing a target. Now, the seed corpus is only run by targets other than
the one that's about to be fuzzed, and the worker handles running and
reporting issues with the seed corpus.
Part of the logic that needed close inspection is what to do if a
failure occurs during a testing-only or coverage-only fail. If the input
is already in the seed corpus, the fuzzing engine shouldn't add it. If
the input is currently in the cache, then it should be written to
testdata. In all cases, if an error occurs, we need to report this to
the user with enough information for them to debug it.
This uncovered some issues with our code when fuzzing without
instrumentation, and when -run=None was provided. There are some logic
fixes in this change, and some small refactors.
Fixesgolang/go#48327Fixesgolang/go#48296
Change-Id: I9ce2be0219c5b09277ddd308df8bc5a46d4558fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/349630
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Print the elapsed time as a nicely formatted duration, and
make small adjustments to the command line output while fuzzing.
Fixesgolang/go#48132
Change-Id: Id95f84c0939171a777448c444d9b87d7af26b654
Reviewed-on: https://go-review.googlesource.com/c/go/+/349970
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
The coordinator needs to marshal data that was provided
via f.Add. However, it was also attempting to marshal data
that was in testdata, which was not needed,
and was causing a panic. This change fixes this.
Fixesgolang/go#48228
Change-Id: I1256c5a287b5a09d2f8cca59beb0f0fc06cc3554
Reviewed-on: https://go-review.googlesource.com/c/go/+/348381
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Change-Id: I70c0229e43dfe37f70b9c79c2e6fe88d7b8d7bd0
Reviewed-on: https://go-review.googlesource.com/c/go/+/347231
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Instead of holding all corpus data/values in memory, only store seed
inputs added via F.Add in memory, and only load corpus entries which
are written to disk when we need them. This should significantly reduce
the memory required by the coordinator process.
Additionally only load the corpus in the coordinator process, since the
worker has no need for it.
Fixes#46669.
Change-Id: Ic3b0c5e929fdb3e2877b963e6b0fa14e140c1e1d
Reviewed-on: https://go-review.googlesource.com/c/go/+/345096
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
When a fuzz worker discovers an input that activates coverage counters
that weren't previously activated, it sends that input back to the
coordinator, as before. If the coordinator also finds that input
provides new coverage (that is, some other input hasn't won the race),
the coordinator now sends the input back to workers for minimization.
The minimization procedure now supports minimizing these interesting
inputs. It attempts to find smaller inputs that preserve at least one
new coverage bit. If minimization succeeds, the coordinator adds the
smaller input to the corpus instead of the original. If minimization
fails, the coordinator adds the original input. If minimization finds
that the original input didn't provide new coverage after all (for
example, a counter was activated by an unrelated background goroutine
and was considered flaky), the input is ignored and not recorded.
Change-Id: I81d98d6ec28abb0ac2a476f73480ceeaff674c08
Reviewed-on: https://go-review.googlesource.com/c/go/+/342997
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
If any error occurs when minimizing a crash, for example, the user
presses ^C because minimization is taking too long, the coordinator
will now write the unminimized crash to testdata.
Change-Id: I0c754125781eb184846e496c728e0505a28639d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/342995
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Previously, when -fuzztime was given a number of executions like
-fuzztime=100x, this was a count for each minimization independent of
-fuzztime. Since there is no bound on the number of minimizations,
this was not a meaningful limit.
With this change, executions of the fuzz function during minimization
count toward the -fuzztime global limit. Executions are further
limited by -fuzzminimizetime.
This change also counts executions during the coverage-only run and
reports errors for those executions.
There is no change when -fuzztime specifies a duration or when
-fuzztime is not set.
Change-Id: Ibcf1b1982f28b28f6625283aa03ce66d4de0a26d
Reviewed-on: https://go-review.googlesource.com/c/go/+/342994
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
When taking a snapshot of coverage counters, round each counter down
to the nearest power of 2.
After coarsening, at most 1 bit per byte will be set. This lets the
coordinator use a coverage array as a mask that distinguish between
code that's executed many times for a given input and code that's
executed once or a few times. For example, if a byte in this array has
the value 12, it means the block has been executed at least 4 times
and at least 8 times with different inputs.
Also change the term "edge" to "bits" or just be more vague about how
coverage is represented.
Also add more code that may be "interesting" in test_fuzz_cache.
Change-Id: I67bf2adb298fb8efd7680b069a476c27e5fdbdae
Reviewed-on: https://go-review.googlesource.com/c/go/+/338829
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
* Benchmark{Marshal,Unmarshal}CorpusFile - measures time it takes to
serialize and deserialize byte slices of various lengths.
* BenchmarkWorkerPing - spins up a worker and measures time it takes
to ping it N times as a rough measure of RPC latency.
* BenchmarkWorkerFuzz - spins up a worker and measures time it takes
to mutate an input and call a trivial fuzz function N times.
Also a few small fixes to make this easier.
Change-Id: Id7f2dc6c6c05005cf286f30e6cc92a54bf44fbf7
Reviewed-on: https://go-review.googlesource.com/c/go/+/333670
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
The types provided in f.Fuzz will be viewed as the
canonical types for fuzzing. If the type is different
for a seed corpus entry, then the testing package
will attempt to convert it. If it can't convert it,
f.Fuzz will fail.
Currently, this allows converting types that may result
in precision loss or a semantically different value.
For example, an int(-1) can be converted to uint even
though the value could be math.MaxUint64. There is a
TODO to consider improving this in the future.
Updates golang/go#45593
Change-Id: I2e752119662f46b68445d42b1ffa46dd30e9faea
Reviewed-on: https://go-review.googlesource.com/c/go/+/325702
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
When GODEBUG=fuzzdebug=1, log additional debug level information about
what the fuzzer is doing. This provides useful information for
investigating the operation and performance of the fuzzing engine, and
is necessary for profiling new fuzzing strategies.
Change-Id: Ic3e24e7a128781377e62785767a218811c3c2030
Reviewed-on: https://go-review.googlesource.com/c/go/+/324972
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
When a worker process finds a crasher, it now sends that result
directly to the coordinator without attempting to minimize it
first. The coordinator stops sending new inputs and sends the
unminimized crasher back to a worker (any worker) for minimization.
This prevents wasted work during minimization and will help us
implement -keepfuzzing later on. We may also be able to minimize
interesting inputs with this approach later.
Since panics are recoverable errors (they don't terminate worker
processes), we no longer attempt to minimize non-recoverable errors.
This didn't work too well before: we lost too much state.
Change-Id: Id142c7e91a33f64584170b0d42d22cb1f22a92d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/321835
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
This reverts commit 54f067812dd870c305daabd22ca190b0f48e672e.
Reason for revert: While this is helpful for the engineering team when we're debugging, it might lead to users feeling like the fuzzer is stuck and that there are a lot of edges that are still yet to be reached. In reality, it's very likely that the compiler will instrument more lines of code than are actually reachable by the fuzz target, so showing the ratio between number of edges hit vs. all edges can be misleading. In the future, we may want to consider making this information viewable by a debug flag or something similar.
Change-Id: Ied696f8bf644445bad22c872b64daa7add605ac6
Reviewed-on: https://go-review.googlesource.com/c/go/+/322632
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
* Introduced -fuzzminimizetime flag to control the number of time or
the number of calls to spend minimizing. Defaults to 60s. Only works
for unrecoverable crashes for now.
* Moved the count (used by -fuzztime=1000x) into shared
memory. Calling workerClient.fuzz resets it, but it will remain
after the worker processes crashes. workerClient.minimize resets it
once before restarting the worker the first time, but the total
number of runs should still be limited during minimization, even
after multiple terminations and restarts.
* Renamed fuzzArgs.Count to Limit to avoid confusion.
* Several other small fixes and refactorings.
Change-Id: I03faa4c94405041f6dfe48568e5ead502f8dbbd2
Reviewed-on: https://go-review.googlesource.com/c/go/+/320171
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>
When instrumented packages intersect with the packages used by the
testing or internal/fuzz packages the coverage counters become noisier,
as counters will be triggered by non-fuzzed harness code.
Ideally counters would be deterministic, as there are many advanced
fuzzing strategies that require mutating the input while maintaining
static coverage.
The simplest way to mitigate this noise is to capture the coverage
counters as closely as possible to the invocation of the fuzz target
in the testing package. In order to do this add a new function which
captures the current values of the counters, SnapshotCoverage. This
function copies the current counters into a static buffer,
coverageSnapshot, which workerServer.fuzz can then inspect when it
comes time to check if new coverage has been found.
This method is not foolproof. As the fuzz target is called in a
goroutine, harness code can still cause counters to be incremented
while the target is being executed. Despite this we do see
significant reduction in churn via this approach. For example,
running a basic target that causes strconv to be instrumented for
500,000 iterations causes ~800 unique sets of coverage counters,
whereas by capturing the counters closer to the target we get ~40
unique sets.
It may be possible to make counters completely deterministic, but
likely this would require rewriting testing/F.Fuzz to not use tRunner
in a goroutine, and instead use it in a blocking manner (which I
couldn't figure out an obvious way to do), or by doing something even
more complex.
Change-Id: I95c2f3b1d7089c3e6885fc7628a0d3a8ac1a99cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/320329
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>
This change updates the go command behavior when
fuzzing to instrument the binary for code coverage,
and uses this coverage in the fuzzing engine to
determine if an input is interesting.
Unfortunately, we can't store and use the coverage
data for a given run of `go test` and re-use it
the next time we fuzz, since the edges could have
changed between builds. Instead, every entry in
the seed corpus and the on-disk corpus is run
by the workers before fuzzing begins, so that the
coordinator can get the baseline coverage for what
the fuzzing engine has already found (or what
the developers have already provided).
Users should run `go clean -fuzzcache` before
using this change, to clear out any existing
"interesting" values that were in the cache.
Previously, every single non-crashing input was
written to the on-disk corpus. Now, only inputs
that actually expand coverage are written.
This change includes a small hack in
cmd/go/internal/load/pkg.go which ensures that the Gcflags
that were explicitly set in cmd/go/internal/test/test.go
don't get cleared out.
Tests will be added in a follow-up change, since
they will be a bit more involved.
Change-Id: Ie659222d44475c6d68fa4a35d37c37cab3619d71
Reviewed-on: https://go-review.googlesource.com/c/go/+/312009
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Usage of f.testContext.match.fullName to generate the test name causes
unbounded memory growth, eventually causing the fuzzer to slow down
as memory pressure increases.
Each time fuzzFn is invoked it generates a unique string and stores it
in a map. With the fuzzer running at around 100k executions per second
this consumed around ~30GB of memory in a handful of minutes.
Instead just use the base name of the test for mutated inputs, a special
name for seeded inputs, and the filename for inputs from the input
corpus.
Change-Id: I083f47df7e82f0c6b0bda244f158233784a13029
Reviewed-on: https://go-review.googlesource.com/c/go/+/316030
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
There was a bug where if the types to fuzz were
different from the types in a file in the on-disk
corpus, then the code would panic. We thought
this case was handled, but the final `continue`
in the nested loop still allowed the invalid
entry to be added to the corpus. Pulling the
validation into a helper function makes this
less brittle.
Change-Id: I401346f890ea30ab7cff9640cb555da2e3ff8cc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/313810
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Assuming that this works for non-recoverable errors, there
will likely be a follow-up CL which refactors the minimization
for recoverable errors to use the same RPC flow (since that
more easily allows the worker to tell the coordinator that
it's minimizing and shouldn't send more inputs to other workers
to fuzz).
Change-Id: I32ac7cec4abe2d4c345c0ee77315233047efb1fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/309509
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
This improves readability a bit, and it should help with compatibility
for future clients when arguments are added or reordered.
Unfortunately, testing still can't import internal/fuzz, so the
interface there can't use this type.
Change-Id: I4cda2347884defcbbfc2bd01ab5b4a901d91549c
Reviewed-on: https://go-review.googlesource.com/c/go/+/308192
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>
-fuzztime now works similarly to -benchtime: if it's given a string
with an "x" suffix (as opposed to "s" or some other unit of
duration), the fuzzing system will generate and run a maximum number
of values.
This CL also implements tracking and printing counts, since most of
the work was already done.
Change-Id: I013007984b5adfc1a751c379dc98c8d46b4a97e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/306909
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
worker.runFuzzing now accepts a Context, used for cancellation instead
of doneC (which is removed). This is passed down through workerClient
RPC methods (ping, fuzz).
workerClient RPC methods now wrap the call method, which handles
marshaling and cancellation.
Both workerClient.call and workerServer.serve should return quickly
when their contexts are cancelled. Turns out, closing the pipe won't
actually unblock a read on all platforms. Instead, we were falling
back to SIGKILL in worker.stop, which works but takes longer than
necessary.
Also fixed missing newline in log message.
Change-Id: I7b5ae54d6eb9afd6361a07759f049f048952e0cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/303429
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>
The -fuzztime flag tells us how much time to spend fuzzing, not
counting time spent running the seed corpus. We shouldn't count time
spent loading the cache either. If the cache is large, the time limit
may be exceeded before the coordinator starts the workers.
Change-Id: If00435faa5d24aabdb9003ebb9337fa2e47f22b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/307310
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
When mutating a byte slice, mutate in place, and only allocate once if
the slice's capacity is less than the maximum size.
mutateBytes already should not allocate; we check a post-condition
that the slice's data pointer does not change.
This speeds up the mutator from 4 ms per value to 200-600 ns. For
example:
goos: darwin
goarch: amd64
pkg: internal/fuzz
cpu: Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
BenchmarkMutatorBytes/1-8 5908735 275.3 ns/op
BenchmarkMutatorBytes/10-8 5198473 282.0 ns/op
BenchmarkMutatorBytes/100-8 4304750 233.9 ns/op
BenchmarkMutatorBytes/1000-8 4623988 295.2 ns/op
BenchmarkMutatorBytes/10000-8 4252104 458.5 ns/op
BenchmarkMutatorBytes/100000-8 1236751 950.8 ns/op
PASS
ok internal/fuzz 12.993s
Change-Id: I4bf2a04be6c648ef440af2c62bf0ffa3d310172c
Reviewed-on: https://go-review.googlesource.com/c/go/+/306675
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>
CoordinateFuzzing now continues to run after discovering a crasher. It
waits until all workers have terminated before returning.
This fixes a deadlock that occurred when multiple workers discovered
crashers concurrently. CoordinateFuzzing would receive one crasher,
close doneC (telling workers to stop), then wait for workers to stop
without receiving more crashers. Other workers would block sending
crashers.
Change-Id: I55a64aac0e6e43f5e36b9d03c15051c3d5debb20
Reviewed-on: https://go-review.googlesource.com/c/go/+/293369
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>
* Appending to the worker environment slice should reallocate it. On
Windows, we pass handles through the environment, and concurrent
workers were writing to the same memory, resulting in
"The handle is invalid" errors.
* Instead of passing a handle to the temporary file, we pass its path
to each worker instead. The worker is responsible for opening and
closing the handle. Previously, all inheritable handles were
inherited by all workers, even though only one was used. This
prevented temporary files from being deleted after a worker stopped,
because other workers would still have open handles to it.
Change-Id: If8b8bcfa5b03fbcadd10ef923b036bb0ee5dc3f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/297034
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
This change makes several refactors to start supporting
structured fuzzing. The mutator can still only mutate
byte slices, and future changes will be made to support
mutating other types. However, it does now support
fuzzing more than one []byte.
This change also makes it so that corpus entries are
encoded in the new file format when being written to
testdata or GOCACHE. Any existing GOCACHE data should
be deleted from your local workstation to allow tests
to pass locally.
Change-Id: Iab8fe01a5dc870f0c53010b9d5b0b479bbdb310d
Reviewed-on: https://go-review.googlesource.com/c/go/+/293810
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
We already read the seed corpus from testdata for the
fuzz target, and pass that corpus to the coordinator.
The coordinator doesn't need to read from testdata
again.
Change-Id: Ia7822e3e02b35d56f6918c7082a7b19901b36644
Reviewed-on: https://go-review.googlesource.com/c/go/+/293189
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
This change rewrites much of the glue code in testing/fuzz.go to work
more analogously to T.Run. This results in improved behavior:
* If a fuzz target returns without calling F.Skip, F.Fail, or F.Fuzz,
'go test' will report an error and exit non-zero.
* Functions registered with F.Cleanup are called.
* The user can re-run individual inputs using -run=FuzzTarget/name
where name is the base name of the seed corpus file. We now print
the 'go test' command after a crash.
This change doesn't correctly handle T.Parallel calls yet, but it
should be easier to do that in the future.
Highlighted parts of this change:
* Instead of creating one F for all targets, create an F for each
target. F (actually common) holds the status, output, and cleanup
function list for each target, so it's important to keep them
separate.
* Run each target in its own goroutine via fRunner. fRunner is
analogous to tRunner. It runs cleanups and catches inappropriate
Goexits and panics.
* Run each input in its own goroutine via T.Run. This enables subtest
filtering with -test.run and ensures functions registered with
T.Cleanup (not F.Cleanup) are run at the appropriate time.
Change-Id: Iab1da14ead8bcb57746f8a76f4aebc625baa5792
Reviewed-on: https://go-review.googlesource.com/c/go/+/290693
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
CorpusEntry is now a struct type with Name and Data fields. In the
future, it may have more fields describing multiple values with
different types added with f.Add.
CorpusEntry must be the same type in testing and
internal/fuzz. However, we don't want to export it from testing, and
testing can't import internal/fuzz. We define it to be a type alias of
a struct type instead of a defined type. We need to define it to the
same thing in both places. We'll get a type error when building cmd/go
if there's a difference.
Change-Id: I9df6cd7aed67a6aa48b77ffb3a84bd302d2e5d94
Reviewed-on: https://go-review.googlesource.com/c/go/+/288534
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
This change moves the worker's *sharedMem into a buffered chan that
acts as a mutex. The mutex can be locked by receiving from the chan;
it can be unlocked by sending *sharedMem back to the chan. Multiple
objects (like worker, workerClient, workerServer) may have references
to the chan and may hold the lock across several operations.
This is intended to fix a segfault that occurred when
workerClient.fuzz accessed shared memory after it was already closed
and unmapped by the worker's goroutine. workerClient.fuzz is executed
in a separate goroutine so the worker can still receive messages from
the coordinator (like being told to stop and clean up).
Change-Id: I4eb9079ba9e5bfcfacfecd0fc8ad9bed17b33bba
Reviewed-on: https://go-review.googlesource.com/c/go/+/285054
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Previously, performing a SIGINT would cause the coordinator
to write a crash to testdata, and would continue to run
despite being interupted.
Also includes a few small cleanups.
Change-Id: Ia3cf7cd231c30ac9ad2a61f4935aa543e241f60d
Reviewed-on: https://go-review.googlesource.com/c/go/+/283634
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
This change also allocates a larger capacity (100 MB) for the shared
memory at the start, rather than beginning as small as possible and
immediately needing to grow while mutating. This means that 100 MB is
the maximum size of a corpus entry currently, since growing the shared
memory is not yet supported.
The code in internal/fuzz/mutator.go and internal/fuzz/pcg.go are copied
from, or heavily inspired by, code originally authored by Dmitry Vyukov
and Josh Bleecher Snyder as part of the go-fuzz project. Thanks to them
for their contributions. See https://github.com/dvyukov/go-fuzz.
Change-Id: I0d51d53976e23933072e760ff78e6c4ad9dcd862
Reviewed-on: https://go-review.googlesource.com/c/go/+/281972
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
fuzz.CoordinateFuzzing and RunFuzzWorker now accept a context.Context
parameter. They should terminate gracefully when the context is
cancelled. The worker should exit quickly without processing more
inputs. The coordinator should save interesting inputs to the cache.
The testing package can't import context directly, so it provides a
timeout argument to testdeps.CoordinateFuzzing instead. The testdeps
wrapper sets the timeout and installs an interrupt handler (for SIGINT
on POSIX and the equivalent on Windows) that cancels the context when
^C is pressed.
Note that on POSIX platforms, pressing ^C causes the shell to deliver
SIGINT to all processes in the active group: so 'go test', the
coordinator, and the workers should all react to that. On Windows,
pressing ^C only interrupts 'go test'. We may want to look at that
separately.
Change-Id: I924d3be2905f9685dae82ff3c047ca3d6b5e2357
Reviewed-on: https://go-review.googlesource.com/c/go/+/279487
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
'go test -fuzz' may now read and write interesting fuzzing values to
directories in $GOCACHE/fuzz. Files in this directory are named
$pkg/$test/$hash where $pkg is the package path containing the fuzz
target, $test is the target name, and $hash is the SHA-256 sum of the
data in the file.
Note that different versions of the same package or packages with the
same path from different modules may share the same directory.
Although files are written into a subdirectory of GOCACHE, they are
not removed automatically, nor are they removed by 'go clean -cache'.
Instead, they may be removed with 'go clean -fuzzcache'. We chose to
nest the fuzzing directory inside GOCACHE to avoid introducing a new
environment variable, since there's no real need for users to specify
a separate directory.
Change-Id: I2032cf8e6c92f715cf36a9fc6a550acf666d2382
Reviewed-on: https://go-review.googlesource.com/c/go/+/275534
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Several small changes, most related to GOFUZZCACHE.
* Use separate channels to send crashers and interesting values to the
coordinator.
* Add a new type, crasherEntry, which is a corpusEntry with an
error message.
* Workers now send fatal errors to the coordinator via errC instead of
returning or closing doneC.
* In CoordinateFuzzing, defer code that closes doneC and waits for
workers to stop. This is the only place where doneC is closed.
* In workerServer and workerClient, always pass input values through
shared memory instead of RPC messages or arguments to avoid
confusion.
* Rename sharedMem.value to valueRef and add valueCopy to make it
clearer whether a reference or copy is needed.
* mutate now operates on shared memory directly.
* mutate will not panic on empty input.
Change-Id: I6e57354875508f0ac4483ed2728f3ba18dc938c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/275533
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>
The coordinator process creates a temporary file for each worker. Both
coordinator and worker map the file into memory and use it for input
values. Access is synchronized with RPC over pipes.
Change-Id: I43c10d7291a8760a616b472d11c017a3a7bb19cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/263153
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
This change also includes a small cleanup of the run()
function and additional tests for error conditions
in fuzz targets.
Change-Id: I2b7722b25a0d071182a84f1dc4b92e82a7ea34d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/256978
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Package fuzz provides common fuzzing functionality for tests built
with "go test" and for programs that use fuzzing functionality in the
testing package.
Change-Id: I3901c6a993a9adb8a93733ae1838b86dd78c7036
Reviewed-on: https://go-review.googlesource.com/c/go/+/259259
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
