Mercurial > dropbear

annotate libtomcrypt/src/pk/rsa/rsa_sign_hash.c @ 1559:92c93b4a3646 fuzz

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fix to be able to compile normal(ish) binaries with --enable-fuzz
author Matt Johnston <matt@ucc.asn.au>
date Wed, 28 Feb 2018 22:02:12 +0800
parents 6dba84798cd5
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
1 /* LibTomCrypt, modular cryptographic library -- Tom St Denis
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
2 *
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
3 * LibTomCrypt is a library that provides various cryptographic
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
4 * algorithms in a highly modular and flexible manner.
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
5 *
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
6 * The library is free for all purposes without any express
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
7 * guarantee it works.
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
8 */
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
9 #include "tomcrypt.h"
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
10
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
11 /**
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
12 @file rsa_sign_hash.c
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
13 RSA PKCS #1 v1.5 and v2 PSS sign hash, Tom St Denis and Andreas Lange
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
14 */
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
15
1435
f849a5ca2efc update to libtomcrypt 1.17 (with Dropbear changes)
Matt Johnston <matt@ucc.asn.au>
parents: 382
diff changeset
16 #ifdef LTC_MRSA
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
17
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
18 /**
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
19 PKCS #1 pad then sign
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
20 @param in The hash to sign
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
21 @param inlen The length of the hash to sign (octets)
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
22 @param out [out] The signature
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
23 @param outlen [in/out] The max size and resulting size of the signature
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
24 @param padding Type of padding (LTC_PKCS_1_PSS, LTC_PKCS_1_V1_5 or LTC_PKCS_1_V1_5_NA1)
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
25 @param prng An active PRNG state
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
26 @param prng_idx The index of the PRNG desired
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
27 @param hash_idx The index of the hash desired
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
28 @param saltlen The length of the salt desired (octets)
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
29 @param key The private RSA key to use
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
30 @return CRYPT_OK if successful
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
31 */
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
32 int rsa_sign_hash_ex(const unsigned char *in, unsigned long inlen,
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
33 unsigned char *out, unsigned long *outlen,
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
34 int padding,
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
35 prng_state *prng, int prng_idx,
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
36 int hash_idx, unsigned long saltlen,
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
37 rsa_key *key)
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
38 {
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
39 unsigned long modulus_bitlen, modulus_bytelen, x, y;
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
40 int err;
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
41
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
42 LTC_ARGCHK(in != NULL);
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
43 LTC_ARGCHK(out != NULL);
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
44 LTC_ARGCHK(outlen != NULL);
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
45 LTC_ARGCHK(key != NULL);
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
46
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
47 /* valid padding? */
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
48 if ((padding != LTC_PKCS_1_V1_5) &&
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
49 (padding != LTC_PKCS_1_PSS) &&
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
50 (padding != LTC_PKCS_1_V1_5_NA1)) {
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
51 return CRYPT_PK_INVALID_PADDING;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
52 }
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
53
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
54 if (padding == LTC_PKCS_1_PSS) {
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
55 /* valid prng ? */
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
56 if ((err = prng_is_valid(prng_idx)) != CRYPT_OK) {
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
57 return err;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
58 }
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
59 }
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
60
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
61 if (padding != LTC_PKCS_1_V1_5_NA1) {
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
62 /* valid hash ? */
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
63 if ((err = hash_is_valid(hash_idx)) != CRYPT_OK) {
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
64 return err;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
65 }
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
66 }
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
67
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
68 /* get modulus len in bits */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
69 modulus_bitlen = mp_count_bits((key->N));
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
70
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
71 /* outlen must be at least the size of the modulus */
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
72 modulus_bytelen = mp_unsigned_bin_size((key->N));
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
73 if (modulus_bytelen > *outlen) {
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
74 *outlen = modulus_bytelen;
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
75 return CRYPT_BUFFER_OVERFLOW;
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
76 }
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
77
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
78 if (padding == LTC_PKCS_1_PSS) {
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
79 /* PSS pad the key */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
80 x = *outlen;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
81 if ((err = pkcs_1_pss_encode(in, inlen, saltlen, prng, prng_idx,
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
82 hash_idx, modulus_bitlen, out, &x)) != CRYPT_OK) {
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
83 return err;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
84 }
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
85 } else {
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
86 /* PKCS #1 v1.5 pad the hash */
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
87 unsigned char *tmpin;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
88
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
89 if (padding == LTC_PKCS_1_V1_5) {
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
90 ltc_asn1_list digestinfo[2], siginfo[2];
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
91 /* not all hashes have OIDs... so sad */
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
92 if (hash_descriptor[hash_idx].OIDlen == 0) {
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
93 return CRYPT_INVALID_ARG;
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
94 }
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
95
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
96 /* construct the SEQUENCE
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
97 SEQUENCE {
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
98 SEQUENCE {hashoid OID
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
99 blah NULL
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
100 }
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
101 hash OCTET STRING
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
102 }
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
103 */
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
104 LTC_SET_ASN1(digestinfo, 0, LTC_ASN1_OBJECT_IDENTIFIER, hash_descriptor[hash_idx].OID, hash_descriptor[hash_idx].OIDlen);
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
105 LTC_SET_ASN1(digestinfo, 1, LTC_ASN1_NULL, NULL, 0);
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
106 LTC_SET_ASN1(siginfo, 0, LTC_ASN1_SEQUENCE, digestinfo, 2);
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
107 LTC_SET_ASN1(siginfo, 1, LTC_ASN1_OCTET_STRING, in, inlen);
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
108
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
109 /* allocate memory for the encoding */
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
110 y = mp_unsigned_bin_size(key->N);
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
111 tmpin = XMALLOC(y);
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
112 if (tmpin == NULL) {
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
113 return CRYPT_MEM;
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
114 }
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
115
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
116 if ((err = der_encode_sequence(siginfo, 2, tmpin, &y)) != CRYPT_OK) {
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
117 XFREE(tmpin);
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
118 return err;
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
119 }
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
120 } else {
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
121 /* set the pointer and data-length to the input values */
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
122 tmpin = (unsigned char *)in;
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
123 y = inlen;
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
124 }
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
125
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
126 x = *outlen;
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
127 err = pkcs_1_v1_5_encode(tmpin, y, LTC_PKCS_1_EMSA, modulus_bitlen, NULL, 0, out, &x);
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
128
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
129 if (padding == LTC_PKCS_1_V1_5) {
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
130 XFREE(tmpin);
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
131 }
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
132
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
133 if (err != CRYPT_OK) {
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
134 return err;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
135 }
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
136 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
137
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
138 /* RSA encode it */
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents: 285
diff changeset
139 return ltc_mp.rsa_me(out, x, out, outlen, PK_PRIVATE, key);
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
140 }
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
141
1435
f849a5ca2efc update to libtomcrypt 1.17 (with Dropbear changes)
Matt Johnston <matt@ucc.asn.au>
parents: 382
diff changeset
142 #endif /* LTC_MRSA */
285
1b9e69c058d2 propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 20dccfc09627970a312d77fb41dc2970b62689c3)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
143
1471
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
144 /* ref: $Format:%D$ */
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
145 /* git commit: $Format:%H$ */
6dba84798cd5 Update to libtomcrypt 1.18.1, merged with Dropbear changes
Matt Johnston <matt@ucc.asn.au>
parents: 1435
diff changeset
146 /* commit time: $Format:%ai$ */