Mercurial > dropbear

annotate libtomcrypt/src/pk/katja/katja_encrypt_key.c @ 1457:32f990cc96b1 fuzz

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fix bad assertion
author Matt Johnston <matt@ucc.asn.au>
date Tue, 23 Jan 2018 23:27:40 +0800
parents f849a5ca2efc
children 6dba84798cd5
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
1 /* LibTomCrypt, modular cryptographic library -- Tom St Denis
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
2 *
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
3 * LibTomCrypt is a library that provides various cryptographic
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
4 * algorithms in a highly modular and flexible manner.
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
5 *
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
6 * The library is free for all purposes without any express
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
7 * guarantee it works.
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
8 *
1435
f849a5ca2efc update to libtomcrypt 1.17 (with Dropbear changes)
Matt Johnston <matt@ucc.asn.au>
parents: 382
diff changeset
9 * Tom St Denis, [email protected], http://libtom.org
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
10 */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
11 #include "tomcrypt.h"
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
12
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
13 /**
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
14 @file katja_encrypt_key.c
1435
f849a5ca2efc update to libtomcrypt 1.17 (with Dropbear changes)
Matt Johnston <matt@ucc.asn.au>
parents: 382
diff changeset
15 Katja LTC_PKCS-style OAEP encryption, Tom St Denis
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
16 */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
17
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
18 #ifdef MKAT
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
19
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
20 /**
1435
f849a5ca2efc update to libtomcrypt 1.17 (with Dropbear changes)
Matt Johnston <matt@ucc.asn.au>
parents: 382
diff changeset
21 (LTC_PKCS #1 v2.0) OAEP pad then encrypt
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
22 @param in The plaintext
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
23 @param inlen The length of the plaintext (octets)
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
24 @param out [out] The ciphertext
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
25 @param outlen [in/out] The max size and resulting size of the ciphertext
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
26 @param lparam The system "lparam" for the encryption
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
27 @param lparamlen The length of lparam (octets)
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
28 @param prng An active PRNG
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
29 @param prng_idx The index of the desired prng
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
30 @param hash_idx The index of the desired hash
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
31 @param key The Katja key to encrypt to
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
32 @return CRYPT_OK if successful
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
33 */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
34 int katja_encrypt_key(const unsigned char *in, unsigned long inlen,
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
35 unsigned char *out, unsigned long *outlen,
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
36 const unsigned char *lparam, unsigned long lparamlen,
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
37 prng_state *prng, int prng_idx, int hash_idx, katja_key *key)
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
38 {
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
39 unsigned long modulus_bitlen, modulus_bytelen, x;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
40 int err;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
41
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
42 LTC_ARGCHK(in != NULL);
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
43 LTC_ARGCHK(out != NULL);
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
44 LTC_ARGCHK(outlen != NULL);
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
45 LTC_ARGCHK(key != NULL);
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
46
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
47 /* valid prng and hash ? */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
48 if ((err = prng_is_valid(prng_idx)) != CRYPT_OK) {
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
49 return err;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
50 }
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
51 if ((err = hash_is_valid(hash_idx)) != CRYPT_OK) {
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
52 return err;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
53 }
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
54
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
55 /* get modulus len in bits */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
56 modulus_bitlen = mp_count_bits((key->N));
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
57
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
58 /* payload is upto pq, so we know q is 1/3rd the size of N and therefore pq is 2/3th the size */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
59 modulus_bitlen = ((modulus_bitlen << 1) / 3);
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
60
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
61 /* round down to next byte */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
62 modulus_bitlen -= (modulus_bitlen & 7) + 8;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
63
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
64 /* outlen must be at least the size of the modulus */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
65 modulus_bytelen = mp_unsigned_bin_size((key->N));
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
66 if (modulus_bytelen > *outlen) {
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
67 *outlen = modulus_bytelen;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
68 return CRYPT_BUFFER_OVERFLOW;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
69 }
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
70
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
71 /* OAEP pad the key */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
72 x = *outlen;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
73 if ((err = pkcs_1_oaep_encode(in, inlen, lparam,
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
74 lparamlen, modulus_bitlen, prng, prng_idx, hash_idx,
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
75 out, &x)) != CRYPT_OK) {
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
76 return err;
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
77 }
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
78
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
79 /* Katja exptmod the OAEP pad */
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
80 return katja_exptmod(out, x, out, outlen, PK_PUBLIC, key);
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
81 }
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
82
1435
f849a5ca2efc update to libtomcrypt 1.17 (with Dropbear changes)
Matt Johnston <matt@ucc.asn.au>
parents: 382
diff changeset
83 #endif /* LTC_MRSA */
382
0cbe8f6dbf9e propagate from branch 'au.asn.ucc.matt.ltc.dropbear' (head 2af22fb4e878750b88f80f90d439b316d229796f)
Matt Johnston <matt@ucc.asn.au>
parents:
diff changeset
84
1435
f849a5ca2efc update to libtomcrypt 1.17 (with Dropbear changes)
Matt Johnston <matt@ucc.asn.au>
parents: 382
diff changeset
85 /* $Source$ */
f849a5ca2efc update to libtomcrypt 1.17 (with Dropbear changes)
Matt Johnston <matt@ucc.asn.au>
parents: 382
diff changeset
86 /* $Revision$ */
f849a5ca2efc update to libtomcrypt 1.17 (with Dropbear changes)
Matt Johnston <matt@ucc.asn.au>
parents: 382
diff changeset
87 /* $Date$ */