Mercurial > dropbear

annotate libtomcrypt/src/pk/pkcs1/pkcs_1_pss_decode.c @ 1801:4983a6bc1f51

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fuzz: fix crash in newtcpdirect(), don't close the channel too early
author Matt Johnston <matt@ucc.asn.au>
date Fri, 05 Mar 2021 22:51:11 +0800
parents 6dba84798cd5
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
1 /* LibTomCrypt, modular cryptographic library -- Tom St Denis
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
2 *
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
3 * LibTomCrypt is a library that provides various cryptographic
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
4 * algorithms in a highly modular and flexible manner.
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
5 *
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
6 * The library is free for all purposes without any express
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
7 * guarantee it works.
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
8 */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
9 #include "tomcrypt.h"
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
10
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
11 /**
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
12 @file pkcs_1_pss_decode.c
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
13 PKCS #1 PSS Signature Padding, Tom St Denis
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
14 */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
15
1435
f849a5ca2efc update to libtomcrypt 1.17 (with Dropbear changes)
Matt Johnston <matt@ucc.asn.au>
parents: 382
diff changeset
16 #ifdef LTC_PKCS_1
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
17
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
18 /**
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
19 PKCS #1 v2.00 PSS decode
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
20 @param msghash The hash to verify
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
21 @param msghashlen The length of the hash (octets)
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
22 @param sig The signature data (encoded data)
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
23 @param siglen The length of the signature data (octets)
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
24 @param saltlen The length of the salt used (octets)
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
25 @param hash_idx The index of the hash desired
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
26 @param modulus_bitlen The bit length of the RSA modulus
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
27 @param res [out] The result of the comparison, 1==valid, 0==invalid
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
28 @return CRYPT_OK if successful (even if the comparison failed)
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
29 */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
30 int pkcs_1_pss_decode(const unsigned char *msghash, unsigned long msghashlen,
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
31 const unsigned char *sig, unsigned long siglen,
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
32 unsigned long saltlen, int hash_idx,
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
33 unsigned long modulus_bitlen, int *res)
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
34 {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
35 unsigned char *DB, *mask, *salt, *hash;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
36 unsigned long x, y, hLen, modulus_len;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
37 int err;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
38 hash_state md;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
39
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
40 LTC_ARGCHK(msghash != NULL);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
41 LTC_ARGCHK(res != NULL);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
42
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
43 /* default to invalid */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
44 *res = 0;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
45
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
46 /* ensure hash is valid */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
47 if ((err = hash_is_valid(hash_idx)) != CRYPT_OK) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
48 return err;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
49 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
50
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
51 hLen = hash_descriptor[hash_idx].hashsize;
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
52 modulus_bitlen--;
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
53 modulus_len = (modulus_bitlen>>3) + (modulus_bitlen & 7 ? 1 : 0);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
54
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
55 /* check sizes */
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
56 if ((saltlen > modulus_len) ||
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
57 (modulus_len < hLen + saltlen + 2)) {
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
58 return CRYPT_PK_INVALID_SIZE;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
59 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
60
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
61 /* allocate ram for DB/mask/salt/hash of size modulus_len */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
62 DB = XMALLOC(modulus_len);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
63 mask = XMALLOC(modulus_len);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
64 salt = XMALLOC(modulus_len);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
65 hash = XMALLOC(modulus_len);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
66 if (DB == NULL || mask == NULL || salt == NULL || hash == NULL) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
67 if (DB != NULL) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
68 XFREE(DB);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
69 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
70 if (mask != NULL) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
71 XFREE(mask);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
72 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
73 if (salt != NULL) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
74 XFREE(salt);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
75 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
76 if (hash != NULL) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
77 XFREE(hash);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
78 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
79 return CRYPT_MEM;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
80 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
81
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
82 /* ensure the 0xBC byte */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
83 if (sig[siglen-1] != 0xBC) {
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
84 err = CRYPT_INVALID_PACKET;
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
85 goto LBL_ERR;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
86 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
87
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
88 /* copy out the DB */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
89 x = 0;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
90 XMEMCPY(DB, sig + x, modulus_len - hLen - 1);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
91 x += modulus_len - hLen - 1;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
92
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
93 /* copy out the hash */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
94 XMEMCPY(hash, sig + x, hLen);
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
95 /* x += hLen; */
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
96
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
97 /* check the MSB */
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
98 if ((sig[0] & ~(0xFF >> ((modulus_len<<3) - (modulus_bitlen)))) != 0) {
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
99 err = CRYPT_INVALID_PACKET;
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
100 goto LBL_ERR;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
101 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
102
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
103 /* generate mask of length modulus_len - hLen - 1 from hash */
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
104 if ((err = pkcs_1_mgf1(hash_idx, hash, hLen, mask, modulus_len - hLen - 1)) != CRYPT_OK) {
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
105 goto LBL_ERR;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
106 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
107
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
108 /* xor against DB */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
109 for (y = 0; y < (modulus_len - hLen - 1); y++) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
110 DB[y] ^= mask[y];
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
111 }
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
112
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
113 /* now clear the first byte [make sure smaller than modulus] */
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
114 DB[0] &= 0xFF >> ((modulus_len<<3) - (modulus_bitlen));
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
115
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
116 /* DB = PS || 0x01 || salt, PS == modulus_len - saltlen - hLen - 2 zero bytes */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
117
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
118 /* check for zeroes and 0x01 */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
119 for (x = 0; x < modulus_len - saltlen - hLen - 2; x++) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
120 if (DB[x] != 0x00) {
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
121 err = CRYPT_INVALID_PACKET;
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
122 goto LBL_ERR;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
123 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
124 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
125
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
126 /* check for the 0x01 */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
127 if (DB[x++] != 0x01) {
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
128 err = CRYPT_INVALID_PACKET;
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
129 goto LBL_ERR;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
130 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
131
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
132 /* M = (eight) 0x00 || msghash || salt, mask = H(M) */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
133 if ((err = hash_descriptor[hash_idx].init(&md)) != CRYPT_OK) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
134 goto LBL_ERR;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
135 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
136 zeromem(mask, 8);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
137 if ((err = hash_descriptor[hash_idx].process(&md, mask, 8)) != CRYPT_OK) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
138 goto LBL_ERR;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
139 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
140 if ((err = hash_descriptor[hash_idx].process(&md, msghash, msghashlen)) != CRYPT_OK) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
141 goto LBL_ERR;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
142 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
143 if ((err = hash_descriptor[hash_idx].process(&md, DB+x, saltlen)) != CRYPT_OK) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
144 goto LBL_ERR;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
145 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
146 if ((err = hash_descriptor[hash_idx].done(&md, mask)) != CRYPT_OK) {
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
147 goto LBL_ERR;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
148 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
149
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
150 /* mask == hash means valid signature */
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
151 if (XMEM_NEQ(mask, hash, hLen) == 0) {
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
152 *res = 1;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
153 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
154
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
155 err = CRYPT_OK;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
156 LBL_ERR:
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
157 #ifdef LTC_CLEAN_STACK
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
158 zeromem(DB, modulus_len);
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
159 zeromem(mask, modulus_len);
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
160 zeromem(salt, modulus_len);
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
161 zeromem(hash, modulus_len);
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
162 #endif
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
163
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
164 XFREE(hash);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
165 XFREE(salt);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
166 XFREE(mask);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
167 XFREE(DB);
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
168
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
169 return err;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
170 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
171
1435
f849a5ca2efc update to libtomcrypt 1.17 (with Dropbear changes)
Matt Johnston <matt@ucc.asn.au>
parents: 382
diff changeset
172 #endif /* LTC_PKCS_1 */
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
173
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
174 /* ref: $Format:%D$ */
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
175 /* git commit: $Format:%H$ */
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
176 /* commit time: $Format:%ai$ */