Commit Graph

342 Commits

Author SHA1 Message Date
Denys Vlasenko
b762c784ca hush: improve ${var#...}, ${var:+...} and ${var/.../...} - handle quoting
dollar_altvalue1 test partially fails: word splitting of unquoted ${var:+...}
is not correct

function                                             old     new   delta
encode_then_expand_vararg                              -     443    +443
expand_one_var                                      1599    1610     +11
parse_stream                                        2756    2753      -3
encode_string                                        250     242      -8
setup_heredoc                                        308     298     -10
expand_and_evaluate_arith                            106      96     -10
encode_then_expand_string                            142     126     -16
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 1/5 up/down: 454/-47)           Total: 407 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-07-17 14:21:38 +02:00
Denys Vlasenko
46158dc833 shell: add 6856 $IFS tests to testsuites
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 20:24:58 +02:00
Denys Vlasenko
f693b606b7 hush: fix recent breakage from parse_stream() changes
function                                             old     new   delta
parse_stream                                        3808    3821     +13

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 20:00:43 +02:00
Denys Vlasenko
44257ad5d0 hush: fix IFS handling in read
$ echo "X:Y:" | (IFS=": " read x y; echo "|$x|$y|")
|X|Y|
$ echo "X:Y  :  " | (IFS=": " read x y; echo "|$x|$y|")
|X|Y|

function                                             old     new   delta
shell_builtin_read                                  1320    1426    +106

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 17:18:34 +02:00
Denys Vlasenko
9678636911 hush: IFS fixes
$ IFS=": "; x=" "; set x $x; for v; do echo "|$v|"; done
|x|
$ IFS=": "; x=":"; set x $x; for v; do echo "|$v|"; done
|x|
||

function                                             old     new   delta
run_pipe                                            1789    1870     +81
expand_on_ifs                                        310     361     +51
pseudo_exec_argv                                     588     591      +3
builtin_local                                         50      53      +3
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 4/0 up/down: 138/0)             Total: 138 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 16:02:58 +02:00
Denys Vlasenko
34179956f9 hush: fix "$v" expansion in case patterns when v='[a]'
function                                             old     new   delta
run_list                                            1053    1063     +10
setup_redirects                                      311     320      +9
encode_then_expand_string                            135     142      +7
run_pipe                                            1784    1789      +5
expand_assignments                                    81      86      +5
expand_string_to_string                              124     125      +1
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 6/0 up/down: 37/0)               Total: 37 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 13:47:59 +02:00
Denys Vlasenko
680c3016a2 ash: parser: Allow newlines within parameter substitution
Upstream commit:

Date: Thu, 22 Mar 2018 21:41:24 +0800
parser: Allow newlines within parameter substitution

    On Fri, Mar 16, 2018 at 11:27:22AM +0800, Herbert Xu wrote:
    > On Thu, Mar 15, 2018 at 10:49:15PM +0100, Harald van Dijk wrote:
    > >
    > > Okay, it can be trivially modified to something that does work in other
    > > shells (even if it were actually executed), but gets rejected at parse time
    > > by dash:
    > >
    > >   if false; then
    > >     : ${$+
    > >   }
    > >   fi
    >
    > That's just a bug in dash's parser with ${} in general, because
    > it bombs out without the if clause too:
    >
    > 	: ${$+
    > 	}

    This patch fixes the parsing of newlines with parameter substitution.

    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 12:39:18 +02:00
Denys Vlasenko
89e9d5534d hush: do not drop backslash from eval 'echo ok\'
newer bash does not drop it, most other shells too

function                                             old     new   delta
unbackslash                                           39      57     +18
parse_stream                                        2753    2751      -2
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/1 up/down: 18/-2)              Total: 16 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 01:15:33 +02:00
Denys Vlasenko
4709df0f15 hush: fix handling of \<eof> in double-quoted strings
function                                             old     new   delta
encode_string                                        268     250     -18

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-10 14:49:01 +02:00
Denys Vlasenko
bcf56114fa hush: fix eval 'echo ok\'
function                                             old     new   delta
parse_stream                                        2762    2753      -9

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-10 14:40:23 +02:00
Denys Vlasenko
21b7f1b6b6 hush: fix a few more corner cases with empty-expanding cmds
See added testcases

function                                             old     new   delta
run_pipe                                            1723    1784     +61

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-05 15:15:53 +02:00
Denys Vlasenko
41d8f10813 hush: fix corner cases with exec in empty expansions
Cases like these:

var=val exec >redir

var=val `` >redir

function                                             old     new   delta
run_pipe                                            1701    1723     +22
redirect_and_varexp_helper                            56      55      -1
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/1 up/down: 22/-1)              Total: 21 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-05 14:41:21 +02:00
Denys Vlasenko
d358b0b65d hush: fix a bug where we don't properly handle f() { a=A; b=B; }; a= f
function                                             old     new   delta
unset_local_var                                       20     274    +254
leave_var_nest_level                                   -      98     +98
set_vars_and_save_old                                128     164     +36
enter_var_nest_level                                   -      32     +32
builtin_local                                         46      50      +4
pseudo_exec_argv                                     554     544     -10
redirect_and_varexp_helper                            77      64     -13
run_pipe                                            1890    1641    -249
unset_local_var_len                                  267       -    -267
------------------------------------------------------------------------------
(add/remove: 2/1 grow/shrink: 3/3 up/down: 424/-539)         Total: -115 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-05 00:51:55 +02:00
Denys Vlasenko
61407807ab hush: fix for readonly vars in "ro=A ro=B cmd" case
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-04 21:14:28 +02:00
Denys Vlasenko
fbf44854a3 hush: support "f() (cmd)" functions
Many other shells support this construct

function                                             old     new   delta
parse_stream                                        2950    3018     +68

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-03 14:56:52 +02:00
Denys Vlasenko
5fa0505f8a hush: fix "set -e; false || x=1; echo OK"
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-03 11:21:13 +02:00
Denys Vlasenko
abf755615e hush: fix a backslash-removal bug in case
function                                             old     new   delta
run_list                                            1270    1053    -217

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-02 17:25:18 +02:00
Denys Vlasenko
9a95df9046 ash: expand: Fix bugs with words connected to the right of $@
Upstream email:

    This is actually composed of two bugs.  First of all our tracking
    of quotemark is wrong so anything after "$@" becomes quoted.  Once
    we fix that then the problem is that the first space character
    after "$@" is not recognised as an IFS.

    This patch fixes both.

    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-02 14:27:50 +02:00
Denys Vlasenko
216913c290 ash: parser: Add syntax stack for recursive parsing
This closes 10821.

Upstream patch:

    From: Herbert Xu <herbert@gondor.apana.org.au>
    Date: Fri, 9 Mar 2018 00:14:02 +0800
    parser: Add syntax stack for recursive parsing

    Without a stack of syntaxes we cannot correctly these two cases
    together:

            "${a#'$$'}"
            "${a#"${b-'$$'}"}"

    A recursive parser also helps in some other corner cases such
    as nested arithmetic expansion with paratheses.

    This patch adds a syntax stack allocated from the stack using
    alloca.  As a side-effect this allows us to remove the naked
    backslashes for patterns within double-quotes, which means that
    EXP_QPAT also has to go.

    This patch also fixes removes any backslashes that precede right
    braces when they are present within a parameter expansion context,
    and backslashes that precede double quotes within inner double
    quotes inside a parameter expansion in a here-document context.

    The idea of a recursive parser is based on a patch by Harald van
    Dijk.

    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

var_bash3, var_bash4 and var_bash6 tests are updated
with the output given by bash-4.3.43

With this patch, the following tests now pass for ash:

    dollar_repl_slash_bash2.tests
    squote_in_varexp2.tests
    squote_in_varexp.tests
    var_bash4.tests

function                                             old     new   delta
readtoken1                                          2615    2874    +259
synstack_push                                          -      54     +54
evalvar                                              574     571      -3
rmescapes                                            330     310     -20
subevalvar                                          1279    1258     -21
argstr                                              1146    1107     -39
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 1/4 up/down: 313/-83)           Total: 230 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-02 13:15:37 +02:00
Denys Vlasenko
bb6f573ad2 hush: add a comment where we differ from bash wrt heredoc EOF mark handling
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-01 18:55:00 +02:00
Denys Vlasenko
41fddb4372 parser: Fix backquote support in here-document EOF mark
Upstream commit:

    Author: Herbert Xu <herbert@gondor.apana.org.au>
    Date:   Thu Mar 15 18:27:30 2018 +0800
    parser: Fix backquote support in here-document EOF mark

    Currently using backquotes in a here-document EOF mark is broken
    because dash tries to do command substitution on it.  This patch
    fixes it by checking whether we're looking for an EOF mark during
    tokenisation.

    Reported-by: Harald van Dijk <harald@gigawatt.nl>
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

With added fix for quoted-ness of the EOF mark.

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-01 16:38:32 +02:00
Denys Vlasenko
1e5111b0f8 ash,hush: handle a few more bkslash-newline cases
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-01 03:04:55 +02:00
Denys Vlasenko
32e183e63e shells: fix var_LINENO1.tests false positive, add it to ash tests too
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-03-31 20:31:13 +02:00
Denys Vlasenko
220be537a0 ash: use pgetc_eatbnl() in more places
Part of upstream commit:

    Date:   Thu Mar 8 08:37:11 2018 +0100
    Author: Harald van Dijk <harald@gigawatt.nl>
    parser: use pgetc_eatbnl() in more places

    dash has a pgetc_eatbnl function in parser.c which skips any
    backslash-newline combinations. It's not used everywhere it could be.
    There is also some duplicated backslash-newline handling elsewhere in
    parser.c. Replace most of the calls to pgetc() with calls to
    pgetc_eatbnl() and remove the duplicated backslash-newline handling.

    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Not adding "readtoken1(pgetc_eatbnl(), DQSYNTAX..." changes, since
readtoken1() handles the "starts with backslash + newline" case itself.

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-03-31 19:40:56 +02:00
Denys Vlasenko
73523079a2 ash,hush: new test dollar_repl_slash_bash2.tests
This fails for ash. hush works.

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-03-30 23:25:24 +02:00
Denys Vlasenko
9acd63c92c ash,hush: fix "saved" redirected fds still visible in children
Based on a patch by Mark Marshall <mark.marshall@omicronenergy.com>

function                                             old     new   delta
dup_CLOEXEC                                            -      49     +49
fcntl_F_DUPFD                                         46       -     -46

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-03-28 18:35:07 +02:00
Denys Vlasenko
d4802c6243 hush: fix a='a\\'; echo "${a%\\\\}"
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-03-02 20:48:36 +01:00
Denys Vlasenko
55f8133a4f shell: tweak bkslash_in_varexp.tests, add bkslash_in_varexp1.tests
It turns out bkslash_in_varexp.tests was a bash bug :]

ash and hush fail "corrected" bkslash_in_varexp.tests as well,
just not as badly as I thought (hush gets half of the cases right).

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-03-02 18:12:12 +01:00
Denys Vlasenko
744a20d8f9 shell: two new tests, both fail for ash and hush
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-03-02 17:13:22 +01:00
Denys Vlasenko
8de5b9f88b ash : fix double-quoted "\z" handling
function                                             old     new   delta
readtoken1                                          2602    2608      +6

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-02-13 14:44:11 +01:00
Denys Vlasenko
f19e3c1c6c shell: handle $((NUM++...) like bash does. Closes 10706
function                                             old     new   delta
evaluate_string                                      680     729     +49

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-01-28 20:13:33 +01:00
Denys Vlasenko
cba79a87f8 hush: fix two corner cases in ${v/pattern/repl}. Closes 10686
function                                             old     new   delta
expand_one_var                                      1592    1618     +26

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-01-25 14:07:40 +01:00
Denys Vlasenko
6aad1ddcc9 hush: implement $LINENO bashism
function                                             old     new   delta
run_pipe                                            1697    1774     +77
unset_local_var_len                                  258     301     +43
hush_main                                           1009    1051     +42
set_local_var                                        516     554     +38
parse_and_run_file                                    42      75     +33
i_getch                                               96     116     +20
.rodata                                            18858   18876     +18
done_command                                         106     122     +16
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 8/0 up/down: 287/0)             Total: 287 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-01-19 15:52:36 +01:00
Denys Vlasenko
aea52e7b2d shell: echo ${?:0} was fixed sometime ago, enable it in tests
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-01-13 19:28:19 +01:00
Denys Vlasenko
9809a82b59 hush: fix raw ^C handlisg in single-quoted strings
function                                             old     new   delta
parse_stream                                        2719    2754     +35

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-01-13 19:14:27 +01:00
Denys Vlasenko
6606c519ef hush: add command2.tests from ash tests
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-01-12 17:58:20 +01:00
Denys Vlasenko
1f1911239c hush: fix handling of ^C in eval
function                                             old     new   delta
run_list                                            1044    1259    +215
builtin_eval                                          45     126     +81
expand_strvec_to_string                               91       -     -91
------------------------------------------------------------------------------
(add/remove: 0/1 grow/shrink: 2/0 up/down: 296/-91)           Total: 205 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-01-11 13:17:30 +01:00
Denys Vlasenko
932b9971d0 hush: fix handling of raw ^C in scripts: "echo ^C"
function                                             old     new   delta
expand_vars_to_list                                 1133    1187     +54
parse_stream                                        2690    2719     +29

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-01-11 12:39:48 +01:00
Denys Vlasenko
a5060b8364 ash: fix nofork bug where environment is not properly passed to a command
function                                             old     new   delta
listvars                                             144     252    +108
evalcommand                                         1500    1546     +46
showvars                                             142     147      +5
shellexec                                            242     245      +3
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 4/0 up/down: 162/0)             Total: 162 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-11-03 14:16:25 +01:00
Denys Vlasenko
25f3b737dc hush: fix comment parsing in cmd, closes 10421
function                                             old     new   delta
parse_stream                                        2692    2690      -2

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-10-22 15:55:48 +02:00
Denys Vlasenko
9fed83a7cc hush: fix false positive in unset.tests
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-08-29 14:35:58 +02:00
Denys Vlasenko
238ff98bb8 hush: fix "getopts" builtin to not be upset by other builtins calling getopt()
function                                             old     new   delta
builtin_getopts                                      363     403     +40
unset_local_var_len                                  185     215     +30
set_local_var                                        440     466     +26
reset_traps_to_defaults                              151     157      +6
pseudo_exec_argv                                     320     326      +6
install_special_sighandlers                           52      58      +6
pick_sighandler                                       62      65      +3
execvp_or_die                                         85      88      +3
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 8/0 up/down: 120/0)             Total: 120 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-08-29 13:38:30 +02:00
Denys Vlasenko
007ce9f807 shell: tweak getopts tests, no code changes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-08-13 02:59:00 +02:00
Denys Vlasenko
419db0391e hush: implement "silent" optstrings of ":opts"
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-08-11 17:21:14 +02:00
Denys Vlasenko
129e1ce72c hush: add a test which fails due to uclibc bug in getopt()
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-08-11 17:00:39 +02:00
Denys Vlasenko
9a7d0a0191 shell: add OPTARG poisoning to getopt_optarg.tests
ash fails this!

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-08-11 02:37:48 +02:00
Denys Vlasenko
81f962f3df hush: teach getopts to set/unset OPTARG
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-08-11 02:05:21 +02:00
Denys Vlasenko
74d4058928 hush: getopts builtin
function                                             old     new   delta
builtin_getopts                                        -     271    +271
bltins1                                              372     384     +12
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 1/0 up/down: 283/0)             Total: 283 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-08-11 01:32:46 +02:00
Denys Vlasenko
19c9f31af1 nofork: fix a bug uncovered by hush testsuite (forgotten fflush)
function                                             old     new   delta
run_nofork_applet                                    280     287      +7

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-08-03 19:52:47 +02:00
Denys Vlasenko
39701204cf hush: do not accept "if() { echo; }" function def
function                                             old     new   delta
parse_stream                                        2634    2692     +58
msg_and_die_if_script                                  -      21     +21
syntax_error_unexpected_ch                            41      46      +5
syntax_error_at                                       14      18      +4
die_if_script                                         31      28      -3
setup_redirects                                      319     308     -11
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 3/2 up/down: 88/-14)             Total: 74 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2017-08-02 19:46:14 +02:00