Upstream commit:
Date: Thu, 22 Mar 2018 21:41:24 +0800
parser: Allow newlines within parameter substitution
On Fri, Mar 16, 2018 at 11:27:22AM +0800, Herbert Xu wrote:
> On Thu, Mar 15, 2018 at 10:49:15PM +0100, Harald van Dijk wrote:
> >
> > Okay, it can be trivially modified to something that does work in other
> > shells (even if it were actually executed), but gets rejected at parse time
> > by dash:
> >
> > if false; then
> > : ${$+
> > }
> > fi
>
> That's just a bug in dash's parser with ${} in general, because
> it bombs out without the if clause too:
>
> : ${$+
> }
This patch fixes the parsing of newlines with parameter substitution.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Since we check for '\' anyway when we determine whether we can look ahead,
we can just check for *and handle* it there.
function old new delta
parse_stream 2751 2740 -11
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
newer bash does not drop it, most other shells too
function old new delta
unbackslash 39 57 +18
parse_stream 2753 2751 -2
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/1 up/down: 18/-2) Total: 16 bytes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream email:
parser: Fix parameter expansion inside inner double quotes
The parsing of parameter expansion inside inner double quotes
breaks because we never look for ENDVAR while innerdq is true.
echo "${x#"${x+''}"''}
This patch fixes it by pushing the syntax stack if innerdq is
true and we enter a new parameter expansion.
This patch also fixes a corner case where a bad substitution error
occurs within arithmetic expansion.
Reported-by: Denys Vlasenko <vda.linux@googlemail.com>
Fixes: ab1cecb40478 (" parser: Add syntax stack for recursive...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
function old new delta
readtoken1 2880 2898 +18
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream email:
This is actually composed of two bugs. First of all our tracking
of quotemark is wrong so anything after "$@" becomes quoted. Once
we fix that then the problem is that the first space character
after "$@" is not recognised as an IFS.
This patch fixes both.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 9 Mar 2018 23:07:53 +0800
parser: Fix single-quoted patterns in here-documents
The script
x=*
cat <<- EOF
${x#'*'}
EOF
prints * instead of nothing as it should. The problem is that
when we're in sqsyntax context in a here-document, we won't add
CTLESC as we should. This patch fixes it:
Reported-by: Harald van Dijk <harald@gigawatt.nl>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
This closes 10821.
Upstream patch:
From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 9 Mar 2018 00:14:02 +0800
parser: Add syntax stack for recursive parsing
Without a stack of syntaxes we cannot correctly these two cases
together:
"${a#'$$'}"
"${a#"${b-'$$'}"}"
A recursive parser also helps in some other corner cases such
as nested arithmetic expansion with paratheses.
This patch adds a syntax stack allocated from the stack using
alloca. As a side-effect this allows us to remove the naked
backslashes for patterns within double-quotes, which means that
EXP_QPAT also has to go.
This patch also fixes removes any backslashes that precede right
braces when they are present within a parameter expansion context,
and backslashes that precede double quotes within inner double
quotes inside a parameter expansion in a here-document context.
The idea of a recursive parser is based on a patch by Harald van
Dijk.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
var_bash3, var_bash4 and var_bash6 tests are updated
with the output given by bash-4.3.43
With this patch, the following tests now pass for ash:
dollar_repl_slash_bash2.tests
squote_in_varexp2.tests
squote_in_varexp.tests
var_bash4.tests
function old new delta
readtoken1 2615 2874 +259
synstack_push - 54 +54
evalvar 574 571 -3
rmescapes 330 310 -20
subevalvar 1279 1258 -21
argstr 1146 1107 -39
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 1/4 up/down: 313/-83) Total: 230 bytes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Author: Herbert Xu <herbert@gondor.apana.org.au>
Date: Thu Mar 15 18:27:30 2018 +0800
parser: Fix backquote support in here-document EOF mark
Currently using backquotes in a here-document EOF mark is broken
because dash tries to do command substitution on it. This patch
fixes it by checking whether we're looking for an EOF mark during
tokenisation.
Reported-by: Harald van Dijk <harald@gigawatt.nl>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
With added fix for quoted-ness of the EOF mark.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Part of upstream commit:
Date: Thu Mar 8 08:37:11 2018 +0100
Author: Harald van Dijk <harald@gigawatt.nl>
parser: use pgetc_eatbnl() in more places
dash has a pgetc_eatbnl function in parser.c which skips any
backslash-newline combinations. It's not used everywhere it could be.
There is also some duplicated backslash-newline handling elsewhere in
parser.c. Replace most of the calls to pgetc() with calls to
pgetc_eatbnl() and remove the duplicated backslash-newline handling.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Not adding "readtoken1(pgetc_eatbnl(), DQSYNTAX..." changes, since
readtoken1() handles the "starts with backslash + newline" case itself.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Tue Mar 6 17:40:37 2018 +0000
expand: 'nolog' and 'debug' options cause "$-" to wreak havoc
Op 29-03-17 om 20:02 schreef Martijn Dekker:
> Bug: if either the 'nolog' or the 'debug' option is set, trying to
> expand "$-" silently aborts parsing of an entire argument.
>
> $ dash -o nolog -c 'set -fuC; echo "|$- are the options|"; set +o nolog; echo "|$- are the options|"'
> |
> |uCf are the options|
> $ dash -o debug -c 'set -fuC; echo "|$- are the options|"; set +o debug; echo "|$- are the options|"'
> |
> |uCf are the options|
This turned out to be easy to fix. The routine producing the "$-"
expansion failed to skip options for which there is no option letter,
but only a long-form name. In dash, 'nolog' and 'debug' are currently
the only two such options. Patch below.
- Martijn
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In bbox ash, pipefail is the option which exhibited this.
Signed-off-by: Martijn Dekker <martijn@inlv.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Testcase:
set -- a ""; space=" "; printf "<%s>\n" "$@"$space
Before:
<a >
After:
<a>
<>
It usually does not bite since bbox forces -funsigned-char build.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
It usually does not bite since bbox forces -funsigned-char build.
But for some reason void linux people disabled that.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
function old new delta
setjobctl 371 367 -4
setinputfile 226 220 -6
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-10) Total: -10 bytes
Based on patch by Mark Marshall <mark.marshall@omicronenergy.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Based on a patch by Mark Marshall <mark.marshall@omicronenergy.com>
function old new delta
dup_CLOEXEC - 49 +49
fcntl_F_DUPFD 46 - -46
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
It turns out bkslash_in_varexp.tests was a bash bug :]
ash and hush fail "corrected" bkslash_in_varexp.tests as well,
just not as badly as I thought (hush gets half of the cases right).
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
I don't have an example of mishandled script, but the logic looked wrong:
it could sometimes treat newlines as if they are spaces.
function old new delta
parse_stream 2788 2787 -1
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
This implements support for the command_not_found_handle hook function, which is
useful for allowing package managers to suggest packages which could provide the
command.
Unlike bash, however, we ignore exit codes from the hook function and always return
the correct POSIX error code (EX_NOTFOUND).
function old new delta
find_command 911 990 +79
Signed-off-by: William Pitcock <nenolod@dereferenced.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Before this patch, "${v:2:0x100000001}" = "${v:2:1}",
and similarly, constructs like "${v:2:9999999999}" may give wrong result
due to int overflows.
function old new delta
substr_atoi - 43 +43
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
If the numeric argument passed to ash's 'shift' built-in is greater than
'$#' the command performs no operation and exits successfully. It should
return a non-zero exit code instead:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#shift
This is consistent with bash and hush.
function old new delta
shiftcmd 122 120 -2
Signed-off-by: Ingo van Lil <inguin@gmx.de>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
NOEXEC'ed applets which use getopt() need this.
function old new delta
builtin_getopts 403 413 +10
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
It is always nicer to give the user some sort of indication why an
operation failed.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Looks biggish and not particularly useful, but may be easier to just eat
the impact if future backports from dash would be otherwise increasingly
difficult.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Thu, 30 Oct 2014 11:53:35 +0800
[PARSER] Catch variable length expansions on non-existant specials
Currently we only check special variable names that follow directly
after $ or ${. So errors such as ${#&} are not caught. This patch
fixes that by moving the is_special check to just before we print out
the special variable name.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
function old new delta
readtoken1 2630 2635 +5
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
The POSIX standard only requires the 'read' builtin to handle '-r':
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/read.html
However, Bash introduced the option '-d <DELIM>' to override IFS for
just one invocation, and it is quite useful.
We already support this in ash, let's add it to hush, too.
function old new delta
builtin_read 263 284 +21
.rodata 163587 163589 +2
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 2/0 up/down: 23/0) Total: 23 bytes
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
The POSIX standard only requires the read builtin to handle -r:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/read.html
However, Bash introduced the option -d <DELIM> to override IFS for
just one invocation, and it is quite useful.
It is also super easy to implement in BusyBox' ash, so let's do that.
The motivation: This option is used by Git's test suite.
function old new delta
.rodata 163505 163587 +82
shell_builtin_read 1244 1289 +45
readcmd 233 259 +26
builtin_read 258 263 +5
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 4/0 up/down: 158/0) Total: 158 bytes
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
hush -c 'yes | head -1' was not happy.
function old new delta
tryexec 159 169 +10
pseudo_exec_argv 328 338 +10
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
The applets with "<applet> [opts] PROG ARGS" API very quickly exec
another program, noexec is okay for them:
chpst/envdir/envuidgid/softlimit/setuidgid
chroot
chrt
ionice
nice
nohup
setarch/linux32/linux64
taskset
cttyhack
"reset" and "sulogin" applets don't have this form, but also exec
another program at once, thus made noexec too.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Based on patch by Johannes Schindelin <johannes.schindelin@gmx.de>
function old new delta
evalcommand 1447 1500 +53
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
As of 035486c75 (ash: significant overhaul of redirect saving logic,
2017-07-31), the sv_pos variable is no longer used (just assigned to,
with no further effect).
Let's just remove it.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
$ echo $$
18448
$ echo $? <wait here, run "kill -INT 18448" in other shell><press enter>
<=== NOTHING??
$
That empty line does not look right. After this patch:
$ echo $$
18448
$ echo $? <wait here, run "kill -INT 18448" in other shell><press enter>
^C
$
function old new delta
fgetc_interactive 245 246 +1
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
"rm -i FILE" and "yes" can now be interrupted by ^C in hush.
This also now works:
$ usleep 19999999
^C
$ echo $?
130
function old new delta
run_pipe 1668 1711 +43
pseudo_exec_argv 312 321 +9
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 2/0 up/down: 52/0) Total: 52 bytes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
