Commit Graph

55 Commits

Author SHA1 Message Date
Denys Vlasenko
7b969bb2ad tls: P256: remove NOP macro sp_256_norm_8()
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2022-07-13 16:11:17 +02:00
Denys Vlasenko
27df6aeef2 tls: P256: factor out "multiply then reduce" operation
function                                             old     new   delta
sp_256_mont_mul_and_reduce_8                           -      44     +44
sp_256_ecc_mulmod_8                                  517     442     -75
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 0/1 up/down: 44/-75)            Total: -31 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-12-11 23:27:40 +01:00
Denys Vlasenko
8514b4166d tls: P256: enable 64-bit version of montgomery reduction
After more testing, (1) I'm more sure it is indeed correct, and
(2) it is a significant speedup - we do a lot of those multiplications.

function                                             old     new   delta
sp_512to256_mont_reduce_8                            191     223     +32

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-28 21:43:51 +01:00
Denys Vlasenko
90b0d33044 tls: P256: add 64-bit montgomery reduce (disabled), small optimization in 32-bit code
function                                             old     new   delta
sp_512to256_mont_reduce_8                            191     185      -6

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-28 15:44:08 +01:00
Denys Vlasenko
832626227e tls: P256: add comment on logic in sp_512to256_mont_reduce_8, no code changes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-28 12:55:20 +01:00
Denys Vlasenko
cfb615781d tls: P256: simplify sp_256_mont_inv_8 (no need for a temporary)
function                                             old     new   delta
sp_256_ecc_mulmod_8                                  543     517     -26

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-28 11:15:34 +01:00
Denys Vlasenko
1b93c7c4ec tls: P256: pad struct sp_point to 64 bits (on 64-bit arches)
function                                             old     new   delta
curve_P256_compute_pubkey_and_premaster              198     190      -8

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-28 02:56:02 +01:00
Denys Vlasenko
0b13ab66f4 tls: P256: trivial x86-64 fix
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-27 19:36:23 +01:00
Denys Vlasenko
f92ae1dc4b tls: P256: change logic so that we don't need double-wide vectors everywhere
Change sp_256to512z_mont_{mul,sqr}_8 to not require/zero upper 256 bits.
There is only one place where we actually used that (and that's why there
used to be zeroing memset of top half!). Fix up that place.
As a bonus, 256x256->512 multiply no longer needs to care for
"r overlaps a or b" case.

This shrinks sp_point structure as well, not just temporaries.

function                                             old     new   delta
sp_256to512z_mont_mul_8                              150       -    -150
sp_256_mont_mul_8                                      -     147    +147
sp_256to512z_mont_sqr_8                                7       -      -7
sp_256_mont_sqr_8                                      -       7      +7
sp_256_ecc_mulmod_8                                  494     543     +49
sp_512to256_mont_reduce_8                            243     249      +6
sp_256_point_from_bin2x32                             73      70      -3
sp_256_proj_point_dbl_8                              353     345      -8
sp_256_proj_point_add_8                              544     499     -45
------------------------------------------------------------------------------
(add/remove: 2/2 grow/shrink: 2/3 up/down: 209/-213)           Total: -4 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-27 19:27:03 +01:00
Denys Vlasenko
9c671fe3dd tls: P256: do not open-code copying of struct variables
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-27 18:42:27 +01:00
Denys Vlasenko
dcfd8d3d10 tls: P256: fix sp_256_div2_8 - it wouldn't use a[] if low bit is 0
It worked by chance because the only caller passed both parameters
as two pointers to the same array.
My fault (I made this error when converting from 26-bit code).

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-27 16:24:49 +01:00
Denys Vlasenko
8cbb70365f tls: P256: remove redundant zeroing in sp_256_map_8
Previous change made it obvious that we zero out already-zeroed high bits

function                                             old     new   delta
sp_256_ecc_mulmod_8                                  534     494     -40

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-27 15:50:40 +01:00
Denys Vlasenko
4415f7bc06 tls: P256: explain which functions use double-wide arrays, no code changes
function                                             old     new   delta
sp_512to256_mont_reduce_8                              -     243    +243
sp_256to512z_mont_mul_8                                -     150    +150
sp_256to512z_mont_sqr_8                                -       7      +7
sp_256_mont_sqr_8                                      7       -      -7
sp_256_mont_mul_8                                    150       -    -150
sp_256_mont_reduce_8                                 243       -    -243
------------------------------------------------------------------------------
(add/remove: 3/3 grow/shrink: 0/0 up/down: 400/-400)            Total: 0 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-27 15:47:26 +01:00
Denys Vlasenko
bbda85c74b tls: P256: remove constant-time trick in sp_256_proj_point_add_8
function                                             old     new   delta
sp_256_proj_point_add_8                              576     544     -32

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-27 15:06:57 +01:00
Denys Vlasenko
26c8522522 tls: P256: do not open-code copying of struct variables
function                                             old     new   delta
sp_256_ecc_mulmod_8                                  536     534      -2

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-27 15:00:14 +01:00
Denys Vlasenko
4bc9da1071 tls: P256: 64-bit optimizations
function                                             old     new   delta
sp_256_proj_point_dbl_8                              421     428      +7
sp_256_point_from_bin2x32                             78      84      +6
sp_256_cmp_8                                          38      42      +4
sp_256_to_bin_8                                       28      31      +3
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 4/0 up/down: 20/0)               Total: 20 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-11-27 11:28:11 +01:00
Denys Vlasenko
53b2fdcdba *: add NOINLINEs where code noticeably shrinks
function                                             old     new   delta
display                                               85    1463   +1378  -73 bytes
select_and_cluster                                     -    1088   +1088 -139 bytes
parse_reply                                            -     979    +979 -109 bytes
zbc_num_sqrt                                           -     632    +632 -191 bytes
show_bridge_port                                       -     585    +585  -56 bytes
sp_256_proj_point_add_8                                -     576    +576  -45 bytes
encode_then_append_var_plusminus                       -     554    +554 -118 bytes
read_mode_db                                           -     537    +537  -47 bytes
fbset_main                                          1331     747    -584
sp_256_ecc_mulmod_8                                 1157     536    -621
brctl_main                                          2189    1548    -641
expand_one_var                                      2544    1872    -672
zxc_vm_process                                      6412    5589    -823
send_queries                                        1813     725   -1088
recv_and_process_peer_pkt                           2245    1018   -1227
bb_dump_dump                                        1531      80   -1451
------------------------------------------------------------------------------
(add/remove: 7/0 grow/shrink: 1/8 up/down: 6329/-7107)       Total: -778 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-10 13:50:53 +02:00
Denys Vlasenko
17e6fb06b3 tls: whitespace fix
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-06 21:22:36 +02:00
Denys Vlasenko
5e9c617021 tls: P256: sp_256_sub_8_p256_mod always subtracts in-place, use that
i386:

function                                             old     new   delta
sp_256_mont_reduce_8                                 245     243      -2
sp_256_mont_dbl_8                                     26      24      -2
sp_256_ecc_mulmod_8                                 1161    1157      -4
sp_256_proj_point_dbl_8                              359     353      -6
sp_256_sub_8_p256_mod                                 71      32     -39
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 0/5 up/down: 0/-53)             Total: -53 bytes

non-asm code:

function                                             old     new   delta
sp_256_sub_8_p256_mod                                  -      12     +12
sp_256_mont_reduce_8                                 250     243      -7
sp_256_mont_dbl_8                                     31      24      -7
sp_256_ecc_mulmod_8                                 1171    1157     -14
sp_256_proj_point_dbl_8                              374     353     -21
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 0/4 up/down: 12/-49)            Total: -37 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-06 20:19:30 +02:00
Denys Vlasenko
87e3f2e9f8 tls: P256: x86-64 optimized sp_256_sub_8_p256_mod
function                                             old     new   delta
sp_256_sub_8_p256_mod                                  -      53     +53
sp_256_mont_reduce_8                                 223     217      -6
sp_256_mont_dbl_8                                     38      32      -6
sp_256_ecc_mulmod_8                                 1535    1529      -6
sp_256_proj_point_dbl_8                              469     454     -15
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 0/4 up/down: 53/-33)             Total: 20 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-06 19:59:39 +02:00
Denys Vlasenko
911344a998 tls: P256: x86-64 assembly
function                                             old     new   delta
sp_256_mont_mul_8                                    127     155     +28
sp_256_proj_point_dbl_8                              448     469     +21
sp_256_mont_sub_8                                     23      35     +12
sp_256_mont_dbl_8                                     26      38     +12
sp_256_sub_8                                          44      49      +5
sp_256_ecc_mulmod_8                                 1530    1535      +5
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 6/0 up/down: 83/0)               Total: 83 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-06 17:17:34 +02:00
Denys Vlasenko
22fd8fd3f4 tls: P256: tweak arm assembly (currently disabled)
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-06 16:10:49 +02:00
Denys Vlasenko
d74993d31d tls: P@256: remove "header comment is kept intact" comment
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-06 14:28:47 +02:00
Denys Vlasenko
567eefcaf8 tls: P256: do not dumplicate sp_256_sub_8()
function                                             old     new   delta
sp_256_proj_point_dbl_8                              359     374     +15
sp_256_ecc_mulmod_8                                 1159    1171     +12
sp_256_mont_reduce_8                                 245     250      +5
sp_256_mont_dbl_8                                     26      31      +5
sp_256_sub_8_p256_mod                                 43       -     -43
------------------------------------------------------------------------------
(add/remove: 0/1 grow/shrink: 4/0 up/down: 37/-43)             Total: -6 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-06 14:25:54 +02:00
Denys Vlasenko
00f2cceb6a tls: P256: shrink sp_256_mul_add_8 a bit more
function                                             old     new   delta
sp_256_mont_reduce_8                                 257     245     -12

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-06 10:15:29 +02:00
Denys Vlasenko
c784284615 tls: P256: propagate constants, create dedicated "subtract p256_mod" function
8 instances of this subtraction probably warrant a few bytes more of code.

function                                             old     new   delta
sp_256_sub_8_p256_mod                                  -      71     +71
sp_256_mont_sub_8                                      -      29     +29
sp_256_mont_dbl_8                                      -      26     +26
sp_256_mont_reduce_8                                 262     257      -5
sp_256_ecc_mulmod_8                                 1171    1161     -10
sp_256_proj_point_dbl_8                              374     359     -15
static.sp_256_mont_sub_8                              29       -     -29
static.sp_256_mont_dbl_8                              31       -     -31
------------------------------------------------------------------------------
(add/remove: 3/2 grow/shrink: 0/3 up/down: 126/-90)            Total: 36 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-06 01:11:48 +02:00
Denys Vlasenko
2430fcfd8d tls: optimize sp_256_mont_reduce_8 in P256
The code size decrease is small, but we eliminate ALL multiplies!

function                                             old     new   delta
sp_256_mont_reduce_8                                 268     262      -6

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-06 00:24:03 +02:00
Denys Vlasenko
bbd723ebec tls: optimize sp_256_mul_8 in P256
function                                             old     new   delta
sp_256_mont_mul_8                                    151     150      -1

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-05 23:19:18 +02:00
Denys Vlasenko
3b411ebbfc tls: replace "26-bit" P256 code with 32-bit one.
function                                             old     new   delta
sp_256_ecc_mulmod_8                                    -    1171   +1171
sp_256_mod_mul_norm_8                                  -     834    +834
sp_256_proj_point_dbl_8                                -     374    +374
sp_256_mont_reduce_8                                   -     268    +268
sp_256_mont_mul_8                                      -     151    +151
sp_256_sub_8                                           -      76     +76
sp_256_add_8                                           -      76     +76
sp_256_cmp_8                                           -      38     +38
static.sp_256_mont_dbl_8                               -      31     +31
static.sp_256_mont_sub_8                               -      29     +29
sp_256_to_bin_8                                        -      28     +28
sp_256_point_from_bin2x32                             50      73     +23
sp_256_mont_sqr_8                                      -       7      +7
sp_256_mont_sqr_10                                     7       -      -7
p256_mod                                              40      32      -8
curve_P256_compute_pubkey_and_premaster              186     167     -19
sp_256_sub_10                                         22       -     -22
sp_256_add_10                                         22       -     -22
sp_256_cmp_10                                         24       -     -24
sp_256_norm_10                                        31       -     -31
static.sp_256_mont_sub_10                             49       -     -49
static.sp_256_mont_dbl_10                             52       -     -52
static.sp_256_mul_add_10                              82       -     -82
sp_256_from_bin_10                                   119       -    -119
sp_256_to_bin_10                                     120       -    -120
sp_256_mont_reduce_10                                178       -    -178
sp_256_mont_mul_10                                   214       -    -214
sp_256_proj_point_dbl_10                             451       -    -451
sp_256_ecc_mulmod_10                                1216       -   -1216
sp_256_mod_mul_norm_10                              1305       -   -1305
------------------------------------------------------------------------------
(add/remove: 12/15 grow/shrink: 1/2 up/down: 3106/-3919)     Total: -813 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-05 20:01:38 +02:00
Denys Vlasenko
55578f2fb7 tls: fix the case of sp_256_mont_tpl_10() leaving striay high bits
It has no effect on correctness, but interferes with compating internal state
of different implementations.

function                                             old     new   delta
sp_256_proj_point_dbl_10                             443     451      +8
static.sp_256_mont_sub_10                             46      49      +3
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 2/0 up/down: 11/0)               Total: 11 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-05 19:46:39 +02:00
Denys Vlasenko
81d8af1970 tls: fix (what looks like) a rare corner case bug in P256
function                                             old     new   delta
static.sp_256_mont_sub_10                             30      46     +16

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-05 17:31:33 +02:00
Denys Vlasenko
92402d5e0a tls: remove one overzealous debugging statement
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-05 14:01:52 +02:00
Denys Vlasenko
137864f559 tls: add debugging scaffolding to P256 code
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-05 13:50:11 +02:00
Denys Vlasenko
389329efbe tls: another P256 code shrink
Propagate constant arrays and scalars deeper down call chain.
Use sp_256_mont_mul_10 to implement sp_256_mont_sqr_10.

function                                             old     new   delta
sp_256_mont_mul_10                                     -     214    +214
sp_256_mont_reduce_10                                  -     178    +178
sp_256_mont_sqr_10                                     -       7      +7
static.sp_256_mont_reduce_10                         178       -    -178
static.sp_256_mont_mul_10                            214       -    -214
static.sp_256_mont_sqr_10                            234       -    -234
------------------------------------------------------------------------------
(add/remove: 3/3 grow/shrink: 0/0 up/down: 399/-626)         Total: -227 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-05 13:39:33 +02:00
Denys Vlasenko
e730505034 tls: P256 code shrink
function                                             old     new   delta
sp_256_to_bin_10                                       -     120    +120
sp_256_from_bin_10                                     -     119    +119
sp_256_proj_point_dbl_10                             446     443      -3
curve_P256_compute_pubkey_and_premaster              191     186      -5
sp_256_point_from_bin2x32                             62      50     -12
sp_256_to_bin                                        120       -    -120
static.sp_256_from_bin                               149       -    -149
------------------------------------------------------------------------------
(add/remove: 2/2 grow/shrink: 0/3 up/down: 239/-289)          Total: -50 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-05 13:32:04 +02:00
Denys Vlasenko
7714518f1a tls: code shrink P256 code
function                                             old     new   delta
sp_256_to_bin                                        148     120     -28

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-10-01 13:51:39 +02:00
Denys Vlasenko
ac36e70074 tls: remove unused define
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-09-30 00:03:23 +02:00
Denys Vlasenko
81f9a0035b tls: tweak sp_256_ecc_gen_k_10 for smaller code
function                                             old     new   delta
curve_P256_compute_pubkey_and_premaster              194     191      -3

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-27 18:48:43 +02:00
Denys Vlasenko
840ae69615 tls: shrink sp_256_mod_mul_norm_10 more
function                                             old     new   delta
sp_256_mod_mul_norm_10                              1439    1305    -134

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-27 13:32:19 +02:00
Denys Vlasenko
646e856294 tls: shrink sp_256_mod_mul_norm_10
function                                             old     new   delta
sp_256_mod_mul_norm_10                              1439    1405     -34

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-27 13:09:44 +02:00
Denys Vlasenko
48a18d15df tls: shrink p256_base more
function                                             old     new   delta
static.p256_base_bin                                   -      64     +64
sp_256_point_from_bin2x32                              -      62     +62
static.base_y                                         40       -     -40
static.base_x                                         40       -     -40
curve_P256_compute_pubkey_and_premaster              291     194     -97
------------------------------------------------------------------------------
(add/remove: 2/2 grow/shrink: 0/1 up/down: 126/-177)          Total: -51 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-27 12:24:21 +02:00
Denys Vlasenko
39a3ef51b5 tls: shrink p256_base
function                                             old     new   delta
curve_P256_compute_pubkey_and_premaster              196     291     +95
static.base_y                                          -      40     +40
static.base_x                                          -      40     +40
p256_base                                            244       -    -244
------------------------------------------------------------------------------
(add/remove: 2/1 grow/shrink: 1/0 up/down: 175/-244)          Total: -69 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-27 01:32:05 +02:00
Denys Vlasenko
a2bc52dd44 tls: reorder P256 functions to make more sense
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-27 01:21:26 +02:00
Denys Vlasenko
e52e43c72f tls: whitespace fixes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-27 00:40:40 +02:00
Denys Vlasenko
9a40be433d tls: get rid of constant-time add/sub operations
function                                             old     new   delta
sp_256_sub_10                                          -      22     +22
static.sp_256_mont_reduce_10                         176     178      +2
sp_256_mod_mul_norm_10                              1440    1439      -1
sp_256_proj_point_dbl_10                             453     446      -7
sp_256_ecc_mulmod_10                                1229    1216     -13
static.sp_256_mont_sub_10                             52      30     -22
static.sp_256_cond_sub_10                             32       -     -32
------------------------------------------------------------------------------
(add/remove: 1/1 grow/shrink: 1/4 up/down: 24/-75)            Total: -51 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-26 21:58:04 +02:00
Denys Vlasenko
120401249a tls: fix whitespace in P256 code
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-26 20:24:34 +02:00
Denys Vlasenko
93b886f54b tls: shrink sp_256_mont_inv_10
function                                             old     new   delta
sp_256_ecc_mulmod_10                                1237    1251     +14
p256_mod_2                                            32       -     -32
------------------------------------------------------------------------------
(add/remove: 0/1 grow/shrink: 1/0 up/down: 14/-32)            Total: -18 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-26 18:05:53 +02:00
Denys Vlasenko
6381f3d4f6 tls: stop passing temporary buffer address in P256 code
function                                             old     new   delta
sp_256_proj_point_dbl_10                             435     453     +18
sp_256_ecc_mulmod_10                                1300    1237     -63
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/1 up/down: 18/-63)            Total: -45 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-26 17:41:43 +02:00
Denys Vlasenko
772e18775e tls: shrink sp_256_proj_point_dbl_10
function                                             old     new   delta
sp_256_ecc_mulmod_10                                1329    1300     -29

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-26 17:25:27 +02:00
Denys Vlasenko
b3b1713a58 tls: in P256 replace constant-time compares with usual ones
function                                             old     new   delta
sp_256_cmp_10                                          -      24     +24
sp_256_ecc_mulmod_10                                1332    1329      -3
sp_256_cmp_equal_10                                   30       -     -30
static.sp_256_cmp_10                                  43       -     -43
------------------------------------------------------------------------------
(add/remove: 1/2 grow/shrink: 0/1 up/down: 24/-76)            Total: -52 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2021-04-26 16:53:53 +02:00