Mercurial > dropbear-fuzzcorpus

annotate fetch.sh @ 27:6d889f88a97d default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Update to current corpus 2024-12-14
author Matt Johnston <matt@ucc.asn.au>
date Sat, 14 Dec 2024 20:28:01 +0800
parents c87782d4bd46
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
21
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
1 #!/bin/sh
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
2
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
3 # Fetch new corpus from oss-fuzz and update the repository. Make sure you commit first!
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
4
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
5 # Before running you need to set up gcloud auth
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
6
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
7 set -x
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
8 set -e
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
9
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
10 FUZZERS=$(make list-fuzz-targets)
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
11
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
12 rm -rf dl
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
13 mkdir dl
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
14 for F in $FUZZERS; do
26
c87782d4bd46 Make robust for latest.zip being missing
Matt Johnston <matt@ucc.asn.au>
parents: 23
diff changeset
15 gsutil cp gs://dropbear-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/dropbear_$F/latest.zip dl/$F.zip \
c87782d4bd46 Make robust for latest.zip being missing
Matt Johnston <matt@ucc.asn.au>
parents: 23
diff changeset
16 || gsutil cp gs://dropbear-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/dropbear_$F/public.zip dl/$F.zip
21
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
17 done
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
18
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
19 # need to use "xargs -n 200" to avoid "argument list too long"
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
20 for F in $FUZZERS; do
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
21 # 40 characters is a hash - those filenames came from a previous corpus
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
22 # we want to keep manually named files which are handcrafted seeds
26
c87782d4bd46 Make robust for latest.zip being missing
Matt Johnston <matt@ucc.asn.au>
parents: 23
diff changeset
23 (echo $F/???????????????????????????????????????? | xargs -n 200 rm) || true
21
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
24 # unzip overwrite, quiet
26
c87782d4bd46 Make robust for latest.zip being missing
Matt Johnston <matt@ucc.asn.au>
parents: 23
diff changeset
25 mkdir -p $F
21
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
26 (cd $F; unzip -o -q ../dl/$F.zip)
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
27 done
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
28
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
29 hg addremove $FUZZERS