Date: Wed, 27 May 2020 13:19:10 +1000
eval: Prevent recursive PS4 expansion
Yaroslav Halchenko <yoh@onerussian.com> wrote:
> I like to (ab)use PS4 and set -x for tracing execution of scripts.
> Reporting time and PID is very useful in this context.
>
> I am not 100% certain if bash's behavior (of actually running the command
> embedded within PS4 string, probably eval'ing it) is actually POSIX
> compliant, posh seems to not do that; but I think it is definitely not
> desired for dash to just stall:
>
> - the script:
> #!/bin/sh
> set -x
> export PS4='+ $(date +%T.%N) [$$] '
> echo "lets go"
> sleep 1
> echo "done $var"
>
> - bash:
> /tmp > bash --posix test.sh
> +export 'PS4=+ $(date +%T.%N) [$$] '
> +PS4='+ $(date +%T.%N) [$$] '
> + 09:15:48.982296333 [2764323] echo 'lets go'
> lets go
> + 09:15:48.987829613 [2764323] sleep 1
> + 09:15:49.994485037 [2764323] echo 'done '
> done
>
...
> - dash: (stalls it set -x)
> /tmp > dash test.sh
> +export PS4=+ $(date +%T.%N) [$$]
> ^C^C
This patch fixes the infinite loop caused by repeated expansions
of PS4.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Wed, 27 May 2020 12:19:13 +1000
parser: Get rid of PEOA
PEOA is a special character used to mark an alias as being finished
so that we don't enter an infinite loop with nested aliases. It
complicates the parser because we have to ensure that it is skipped
where necessary and not copied to the resulting token text.
This patch removes it and instead delays the marking of aliases
until the second pgetc. This has the same effect as the current
PEOA code while keeping the complexities within the input code.
This adds ~32 bytes of global data:
function old new delta
__pgetc - 512 +512
freestrings - 95 +95
popfile 86 110 +24
pushstring 141 160 +19
basepf 76 84 +8
syntax_index_table 258 257 -1
S_I_T 30 28 -2
.rodata 104255 104247 -8
pgetc_without_PEOA 13 - -13
xxreadtoken 230 215 -15
popstring 158 120 -38
readtoken1 3110 3045 -65
pgetc 547 22 -525
------------------------------------------------------------------------------
(add/remove: 2/1 grow/shrink: 3/7 up/down: 658/-667) Total: -9 bytes
text data bss dec hex filename
1043102 559 5020 1048681 100069 busybox_old
1043085 559 5052 1048696 100078 busybox_unstripped
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Wed, 29 Apr 2020 00:19:59 +1000
parser: Fix alias expansion after heredoc or newlines
This script should print OK:
alias a="case x in " b=x
a
b) echo BAD;; esac
alias BEGIN={ END=}
BEGIN
cat <<- EOF > /dev/null
$(:)
EOF
END
: <<- EOF &&
$(:)
EOF
BEGIN
echo OK
END
However, because the value of checkkwd is either zeroed when it
shouldn't, or isn't zeroed when it should, dash currently gets
it wrong in every case.
This patch fixes it by saving checkkwd and zeroing it where needed.
function old new delta
readtoken 157 176 +19
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Adding previously skipped "readtoken1(pgetc_eatbnl(), syntax_type..." change
from upstream commit:
Date: Thu Mar 8 08:37:11 2018 +0100
parser: use pgetc_eatbnl() in more places
dash has a pgetc_eatbnl function in parser.c which skips any
backslash-newline combinations. It's not used everywhere it could be.
There is also some duplicated backslash-newline handling elsewhere in
parser.c. Replace most of the calls to pgetc() with calls to
pgetc_eatbnl() and remove the duplicated backslash-newline handling.
Testcase:
PS1='\
:::'
should result in ::: prompt, not <newline>::: prompt
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sun, 17 May 2020 23:36:25 +1000
parser: Save and restore heredoclist in expandstr
On Sun, May 17, 2020 at 01:19:28PM +0100, Harald van Dijk wrote:
> This still does not restore the state completely. It does not clean up any
> pending heredocs. I see:
>
> $ PS1='$(<<EOF "'
> src/dash: 1: Syntax error: Unterminated quoted string
> $(<<EOF ":
> >
>
> That is, after entering the ':' command, the shell is still trying to read
> the heredoc from the prompt.
This patch saves and restores the heredoclist in expandstr.
It also removes a bunch of unnecessary volatiles as those variables
are only referenced in case of a longjmp other than one started by
a signal like SIGINT.
function old new delta
expandstr 268 255 -13
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Tue, 28 Apr 2020 01:15:26 +1000
parser: Fix handling of empty aliases
Dash was incorrectly handling empty aliases. When attempting to use an
empty alias with nothing else, I'm (incorrectly) prompted for more
input:
```
$ alias empty=''
$ empty
>
```
Other shells (e.g., bash, yash) correctly handle the lone, empty alias as an
empty command:
```
$ alias empty=''
$ empty
$
```
The problem here is that we incorrectly enter the loop eating TNLs
in readtoken(). This patch fixes it by setting checkkwd correctly.
function old new delta
list 351 355 +4
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upsteam commit:
Date: Mon, 17 May 2021 15:19:23 +0800
eval: Do not cache value of eflag in evaltree
Patrick Brünn <P.Bruenn@beckhoff.com> wrote:
> Since we are migrating to Debian bullseye, we discovered a new behavior
> with our scripts, which look like this:
>>cleanup() {
>> set +e
>> rmdir ""
>>}
>>set -eu
>>trap 'cleanup' EXIT INT TERM
>>echo 'Hello world!'
>
> With old dash v0.5.10.2 this script would return 0 as we expected it.
> But since commit 62cf6955f8abe875752d7163f6f3adbc7e49ebae it returns
> the last exit code of our cleanup function.
...
Thanks for the report. This is actually a fairly old bug with
set -e that's just been exposed by the exit status change. What's
really happening is that cleanup itself is triggering a set -e
exit incorrectly because evaltree cached the value of eflag prior
to the function call.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Thu, 4 Jun 2020 21:53:55 +1000
eval: Check nflag in evaltree instead of cmdloop
This patch moves the nflag check from cmdloop into evaltree. This
is so that nflag will be in force even if we enter the shell via a
path other than cmdloop, e.g., through sh -c.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Stacy Harper reports that this script:
test() { . /tmp/bb_test; }
echo "export TEST=foo" >/tmp/bb_test
test 2>/dev/null
echo "$TEST"
correctly prints 'foo' in BusyBox 1.33 but hangs in 1.34.
Bisection suggested the problem was caused by commit a1b0d3856 (ash: add
process substitution in bash-compatibility mode). Removing the call to
unwindredir() in cmdloop() introduced in that commit makes the script
work again.
Additionally, these examples of process substitution:
while true; do cat <(echo hi); done
f() { while true; do cat <(echo hi); done }
f
result in running out of file descriptors. This is a regression from
v5 of the process substitution patch caused by changes to evalcommand()
not being transferred to v6.
function old new delta
static.pushredir - 99 +99
evalcommand 1729 1750 +21
exitreset 69 86 +17
cmdloop 372 365 -7
unwindredir 28 - -28
pushredir 112 - -112
------------------------------------------------------------------------------
(add/remove: 1/2 grow/shrink: 2/1 up/down: 137/-147) Total: -10 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Let's adopt Herbert Xu's patch, not waiting for it to reach dash git:
hush already has a similar fix.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
On Sat, Feb 9, 2019 Cristian Ionescu-Idbohrn wrote:
> In my case (at work), I have to watch and prevent people from doing
> unportable things. For me, that's a burden.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Process substitution is a Korn shell feature that's also available
in bash and some other shells. This patch implements process
substitution in ash when ASH_BASH_COMPAT is enabled.
function old new delta
argstr 1386 1522 +136
strtodest - 52 +52
readtoken1 3346 3392 +46
.rodata 183206 183250 +44
unwindredir - 28 +28
cmdloop 365 372 +7
static.spclchars 10 12 +2
cmdputs 380 367 -13
exitreset 86 69 -17
evalcommand 1754 1737 -17
varvalue 675 634 -41
------------------------------------------------------------------------------
(add/remove: 2/0 grow/shrink: 5/4 up/down: 315/-88) Total: 227 bytes
text data bss dec hex filename
953967 4219 1904 960090 ea65a busybox_old
954192 4219 1904 960315 ea73b busybox_unstripped
v2: Replace array of file descriptors with a linked list.
Include tests that were unaccountably omitted from v1.
v3: Update linked list code to the intended version.
v4: Change order of conditional code in cmdputs().
v5: Use existing popredir() mechanism to manage file descriptors.
v6: Rebase to latest version of BusyBox ash. Reduce code churn.
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
"In function 'sprint_status48':
error: format not a string literal and no format arguments"
function old new delta
sprint_status48 160 158 -2
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Otherwise if $HISTFILE is unset or reassigned, bad things can happen.
function old new delta
ash_main 1210 1218 +8
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Based on patch by Joachim Nilsson <troglobit@gmail.com>
function old new delta
pgetc 570 547 -23
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Tue, 2 Jun 2020 23:46:48 +1000
jobs: Fix waitcmd busy loop
We need to clear gotsigchld in waitproc because it is used as
a loop conditional for the waitcmd case. Without it waitcmd
may busy loop after a SIGCHLD.
This patch also changes gotsigchld into a volatile sig_atomic_t
to prevent compilers from optimising its accesses away.
Fixes: 6c691b3e5099 ("jobs: Only clear gotsigchld when waiting...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This change also incorporates other changes to bring us closer to upstream.
function old new delta
dowait 553 636 +83
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Fri, 10 Apr 2020 21:03:09 +1000
jobs: Fix infinite loop in waitproc
After we changed the resetting of gotsigchld so that it is only
done if jp is NULL, we can now get an infinite loop in waitproc
if gotsigchld is set but there is no outstanding child because
everything had been waited for previously without gotsigchld being
zeroed.
This patch fixes it by always zeroing gotsigchld as we did before.
The bug that the previous patch was trying to fix is now resolved
by switching the blocking mode to DOWAIT_NORMAL after the specified
job has been completed so that we really do wait for all outstanding
dead children.
Reported-by: Harald van Dijk <harald@gigawatt.nl>
Fixes: 6c691b3e5099 ("jobs: Only clear gotsigchld when waiting...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
function old new delta
dowait 553 631 +78
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream tried it, and then reverted by:
From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Tue, 26 May 2020 23:19:05 +1000
parser: Fix double-backslash nl in old-style command sub
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
show_history() checks that its argument in non-null so there's
no need to repeat the test at call sites.
function old new delta
historycmd 25 17 -8
builtin_history 29 21 -8
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-16) Total: -16 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Since commit 7eb8eecbb (ash: eval: Add assignment built-in support
again) building BusyBox with the 'command' built-in disabled fails.
parse_command_args() only needs to be called when the 'command'
built-in is run. Which it won't be if it's disabled.
v2: Avoiding infinite loops is good, too. Thanks, Harald van Dijk.
Reported-by: Deweloper <deweloper@wp.pl>
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Wed, 12 Sep 2018 14:27:16 +0800
expand: Fix multiple issues with EXP_DISCARD in evalvar
The commit 3cd538634f71538370f5af239f342aec48b7470b broke parameter
expansion in multiple ways because the EXP_DISCARD flag wasn't set
or tested for various cases:
$ src/dash -c 'var=; echo ${var:+nonempty}'
nonempty
$ src/dash -u -c 'unset foo bar; echo ${foo+${bar}}'
dash: 1: bar: parameter not set
$ src/dash -c 'foo=bar; echo ${foo=BUG}; echo $foo'
barBUG
bar
$
This patch fixes them by introducing a new discard variable that
tracks whether the extra word should be discarded or not when it
is parsed.
Reported-by: Martijn Dekker <martijn@inlv.org>
Fixes: 3cd538634f71 ("expand: Do not reprocess data when...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Reported-by: Martijn Dekker <martijn@inlv.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream patch:
Date: Wed, 30 May 2018 02:06:03 +0800
expand: Do not reprocess data when expanding words
Currently various paths will reprocess data when performing word
expansion. For example, expari will skip backwards looking for
the start of the arithmetic expansion, while evalvar will skip
unexpanded words manually.
This is cumbersome and error-prone. This patch fixes this by
making word expansions proceed in a linear fashion. This means
changing argstr and the various expansion functions such as expari
and subevalvar to return the next character to be expanded.
This is inspired by similar code from FreeBSD. However, we take
things one step further and completely remove the manual word
skipping in evalvar. This is accomplished by introducing a new
EXP_DISCARD flag that tells argstr to only parse and not produce
any actual expansions.
Incidentally, argstr will now always NUL-terminate the expansion
unless the EXP_WORD flag is set. This is because all but one
caller of argstr wants the result to be NUL-termianted.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Also includes two one-line follow-up fixes:
expand: Eat closing brace for length parameter
if (subtype == VSLENGTH) {
+ p++;
if (flag & EXP_DISCARD)
expand: Fix double-decrement in argstr
- newloc = expdest - (char *)stackblock() - end;
+ newloc = q - (char *)stackblock() - end;
and changes in code for bash substring extensions.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Following recent work on evalcommand() a TRACE statement to report
the status of a forked command was left in the wrong place.
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
