Mercurial > dropbear-fuzzcorpus

annotate fetch.sh @ 21:1a22de6a6b1a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Script to fetch new corpus and update the repository
author Matt Johnston <matt@ucc.asn.au>
date Sun, 07 Mar 2021 16:06:25 +0800
parents
children 4032153b8b7a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
21
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
1 #!/bin/sh
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
2
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
3 # Fetch new corpus from oss-fuzz and update the repository. Make sure you commit first!
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
4
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
5 # Before running you need to set up gcloud auth
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
6
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
7 set -x
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
8 set -e
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
9
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
10 FUZZERS=$(make list-fuzz-targets)
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
11
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
12 rm -rf dl
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
13 mkdir dl
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
14 for F in $FUZZERS; do
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
15 gsutil cp gs://dropbear-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/dropbear_$F/latest.zip dl/$F.zip
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
16 done
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
17
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
18 # need to use "xargs -n 200" to avoid "argument list too long"
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
19 rm -rf old
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
20 mkdir old
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
21 for F in $FUZZERS; do
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
22 mkdir old/$F
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
23 # 40 characters is a hash - those filenames came from a previous corpus
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
24 # we want to keep manually named files which are handcrafted seeds
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
25 echo $F/???????????????????????????????????????? | xargs -n 200 rm || true
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
26 # unzip overwrite, quiet
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
27 (cd $F; unzip -o -q ../dl/$F.zip)
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
28 done
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
29
1a22de6a6b1a Script to fetch new corpus and update the repository
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
30 hg addremove $FUZZERS