Date: Wed, 27 May 2020 13:19:10 +1000
eval: Prevent recursive PS4 expansion
Yaroslav Halchenko <yoh@onerussian.com> wrote:
> I like to (ab)use PS4 and set -x for tracing execution of scripts.
> Reporting time and PID is very useful in this context.
>
> I am not 100% certain if bash's behavior (of actually running the command
> embedded within PS4 string, probably eval'ing it) is actually POSIX
> compliant, posh seems to not do that; but I think it is definitely not
> desired for dash to just stall:
>
> - the script:
> #!/bin/sh
> set -x
> export PS4='+ $(date +%T.%N) [$$] '
> echo "lets go"
> sleep 1
> echo "done $var"
>
> - bash:
> /tmp > bash --posix test.sh
> +export 'PS4=+ $(date +%T.%N) [$$] '
> +PS4='+ $(date +%T.%N) [$$] '
> + 09:15:48.982296333 [2764323] echo 'lets go'
> lets go
> + 09:15:48.987829613 [2764323] sleep 1
> + 09:15:49.994485037 [2764323] echo 'done '
> done
>
...
> - dash: (stalls it set -x)
> /tmp > dash test.sh
> +export PS4=+ $(date +%T.%N) [$$]
> ^C^C
This patch fixes the infinite loop caused by repeated expansions
of PS4.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Wed, 27 May 2020 12:19:13 +1000
parser: Get rid of PEOA
PEOA is a special character used to mark an alias as being finished
so that we don't enter an infinite loop with nested aliases. It
complicates the parser because we have to ensure that it is skipped
where necessary and not copied to the resulting token text.
This patch removes it and instead delays the marking of aliases
until the second pgetc. This has the same effect as the current
PEOA code while keeping the complexities within the input code.
This adds ~32 bytes of global data:
function old new delta
__pgetc - 512 +512
freestrings - 95 +95
popfile 86 110 +24
pushstring 141 160 +19
basepf 76 84 +8
syntax_index_table 258 257 -1
S_I_T 30 28 -2
.rodata 104255 104247 -8
pgetc_without_PEOA 13 - -13
xxreadtoken 230 215 -15
popstring 158 120 -38
readtoken1 3110 3045 -65
pgetc 547 22 -525
------------------------------------------------------------------------------
(add/remove: 2/1 grow/shrink: 3/7 up/down: 658/-667) Total: -9 bytes
text data bss dec hex filename
1043102 559 5020 1048681 100069 busybox_old
1043085 559 5052 1048696 100078 busybox_unstripped
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Wed, 29 Apr 2020 00:19:59 +1000
parser: Fix alias expansion after heredoc or newlines
This script should print OK:
alias a="case x in " b=x
a
b) echo BAD;; esac
alias BEGIN={ END=}
BEGIN
cat <<- EOF > /dev/null
$(:)
EOF
END
: <<- EOF &&
$(:)
EOF
BEGIN
echo OK
END
However, because the value of checkkwd is either zeroed when it
shouldn't, or isn't zeroed when it should, dash currently gets
it wrong in every case.
This patch fixes it by saving checkkwd and zeroing it where needed.
function old new delta
readtoken 157 176 +19
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Adding previously skipped "readtoken1(pgetc_eatbnl(), syntax_type..." change
from upstream commit:
Date: Thu Mar 8 08:37:11 2018 +0100
parser: use pgetc_eatbnl() in more places
dash has a pgetc_eatbnl function in parser.c which skips any
backslash-newline combinations. It's not used everywhere it could be.
There is also some duplicated backslash-newline handling elsewhere in
parser.c. Replace most of the calls to pgetc() with calls to
pgetc_eatbnl() and remove the duplicated backslash-newline handling.
Testcase:
PS1='\
:::'
should result in ::: prompt, not <newline>::: prompt
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sun, 17 May 2020 23:36:25 +1000
parser: Save and restore heredoclist in expandstr
On Sun, May 17, 2020 at 01:19:28PM +0100, Harald van Dijk wrote:
> This still does not restore the state completely. It does not clean up any
> pending heredocs. I see:
>
> $ PS1='$(<<EOF "'
> src/dash: 1: Syntax error: Unterminated quoted string
> $(<<EOF ":
> >
>
> That is, after entering the ':' command, the shell is still trying to read
> the heredoc from the prompt.
This patch saves and restores the heredoclist in expandstr.
It also removes a bunch of unnecessary volatiles as those variables
are only referenced in case of a longjmp other than one started by
a signal like SIGINT.
function old new delta
expandstr 268 255 -13
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Tue, 28 Apr 2020 01:15:26 +1000
parser: Fix handling of empty aliases
Dash was incorrectly handling empty aliases. When attempting to use an
empty alias with nothing else, I'm (incorrectly) prompted for more
input:
```
$ alias empty=''
$ empty
>
```
Other shells (e.g., bash, yash) correctly handle the lone, empty alias as an
empty command:
```
$ alias empty=''
$ empty
$
```
The problem here is that we incorrectly enter the loop eating TNLs
in readtoken(). This patch fixes it by setting checkkwd correctly.
function old new delta
list 351 355 +4
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upsteam commit:
Date: Mon, 17 May 2021 15:19:23 +0800
eval: Do not cache value of eflag in evaltree
Patrick Brünn <P.Bruenn@beckhoff.com> wrote:
> Since we are migrating to Debian bullseye, we discovered a new behavior
> with our scripts, which look like this:
>>cleanup() {
>> set +e
>> rmdir ""
>>}
>>set -eu
>>trap 'cleanup' EXIT INT TERM
>>echo 'Hello world!'
>
> With old dash v0.5.10.2 this script would return 0 as we expected it.
> But since commit 62cf6955f8abe875752d7163f6f3adbc7e49ebae it returns
> the last exit code of our cleanup function.
...
Thanks for the report. This is actually a fairly old bug with
set -e that's just been exposed by the exit status change. What's
really happening is that cleanup itself is triggering a set -e
exit incorrectly because evaltree cached the value of eflag prior
to the function call.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Thu, 4 Jun 2020 21:53:55 +1000
eval: Check nflag in evaltree instead of cmdloop
This patch moves the nflag check from cmdloop into evaltree. This
is so that nflag will be in force even if we enter the shell via a
path other than cmdloop, e.g., through sh -c.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
A refactor of the awk printf code in
e2e3802987
appears to have broken the printf interpretation of two percent signs,
which normally outputs only one percent sign.
The patch below brings busybox awk printf behavior back into alignment
with the pre-e2e380 behavior, the busybox printf util, and other common
(awk and non-awk) printf implementations.
function old new delta
awk_printf 626 672 +46
Signed-off-by: Daniel Thau <danthau at bedrocklinux.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Stacy Harper reports that this script:
test() { . /tmp/bb_test; }
echo "export TEST=foo" >/tmp/bb_test
test 2>/dev/null
echo "$TEST"
correctly prints 'foo' in BusyBox 1.33 but hangs in 1.34.
Bisection suggested the problem was caused by commit a1b0d3856 (ash: add
process substitution in bash-compatibility mode). Removing the call to
unwindredir() in cmdloop() introduced in that commit makes the script
work again.
Additionally, these examples of process substitution:
while true; do cat <(echo hi); done
f() { while true; do cat <(echo hi); done }
f
result in running out of file descriptors. This is a regression from
v5 of the process substitution patch caused by changes to evalcommand()
not being transferred to v6.
function old new delta
static.pushredir - 99 +99
evalcommand 1729 1750 +21
exitreset 69 86 +17
cmdloop 372 365 -7
unwindredir 28 - -28
pushredir 112 - -112
------------------------------------------------------------------------------
(add/remove: 1/2 grow/shrink: 2/1 up/down: 137/-147) Total: -10 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Improved error messages:
- specify when a search fails or a mark isn't set;
- warn when line addresses are out of range or when a range of
lines is reversed.
Addresses are limited to the number of lines in the file so a
command like ':2000000000' (go to the two billionth line) no
longer causes a long pause.
Improved vi compatibility of '+' and '-' operators that aren't
followed immediately by a number:
:4+++= 7
:3-2= 1
:3 - 2= 4 (yes, really!)
In a command like ':,$' the empty address before the separator now
correctly refers to the current line. (The similar case ':1,' was
already being handled.)
And all with a tidy reduction in bloat (32-bit build):
function old new delta
colon 4029 4069 +40
.rodata 99348 99253 -95
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/1 up/down: 40/-95) Total: -55 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
gcc 11.2.1 complains that the tar header checksum might overflow
the checksum field. It won't and using an unsigned int for the
calculation seems to convince the compiler too.
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
A user noted that the following command was slower than they
expected:
busybox shuf -i "1500000000-$(date +%s)" -n 5
At time of writing the range contains 128 million values. On my
system this takes 7.7s whereas 'shuf' from coreutils takes a
handful of milliseconds.
Optimise BusyBox 'shuf' for cases where -n is specified by stopping
shuffling once the required number of lines have been processed.
On my system the time for the example is reduced to 0.4s.
function old new delta
shuf_main 520 540 +20
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/0 up/down: 20/0) Total: 20 bytes
v2: Code shrink. Since outlines <= numlines:
- the loop in shuffle_lines() only needs to test the value of
outlines;
- shuffle_lines() can be called unconditionally.
Update timing to allow for the 13 million seconds elapsed since v1.
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Simplify the function print_literal() which is used to format a
string that may contain unprintable characters or control
characters.
- Unprintable characters were being displayed in normal text rather
than the bold used for the rest of the message. This doesn't seem
particularly helpful and it upsets the calculation of the width
of the message in show_status_line(). Use '?' rather than '.' for
unprintable characters.
- Newlines in the string were displayed as both '^J' and '$', which
is somewhat redundant.
function old new delta
not_implemented 199 108 -91
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-91) Total: -91 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
The '/' and '?' search commands wrap to the other end of the buffer
if the search target isn't found. When searches are used to specify
addresses in colon commands they should do the same.
(In traditional vi and vim this behaviour is controlled by the
'wrapscan' option. BusyBox vi doesn't have this option and always
uses the default behaviour.)
function old new delta
colon 4033 4077 +44
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/0 up/down: 44/0) Total: 44 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
The input buffer is initialised to a reasonable size and extended
if necessary. When this happened the offset into the buffer wasn't
reset to zero so subsequent lines were appended to the long line.
Fix this and add some tests.
function old new delta
rev_main 377 368 -9
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-9) Total: -9 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Run initialisation commands from ~/.exrc. As with EXINIT these
commands are processed before the first file is loaded.
Commands starting with double quotes are ignored. This is how
comments are often included in .exrc.
function old new delta
vi_main 268 406 +138
colon 4033 4071 +38
.rodata 108411 108442 +31
packed_usage 34128 34118 -10
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 3/1 up/down: 207/-10) Total: 197 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Rewrite handling of command line arguments so any number of -c
commands will be processed. Previously only two -c commands
were allowed (or one if EXINIT was set).
Process commands from EXINIT before the first file is read into
memory, as specified by POSIX.
function old new delta
run_cmds - 77 +77
.rodata 108410 108411 +1
vi_main 305 268 -37
edit_file 816 764 -52
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 1/2 up/down: 78/-89) Total: -11 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
