Upstream commit:
Date: Sat, 19 May 2018 02:39:41 +0800
jobs: Only clear gotsigchld when waiting for everything
The gotsigchld flag is always cleared in dowait but not all callers
of dowait will wait for everything. In particular, when jp is set
we only wait until the set job isn't running anymore.
This patch fixes this by only clearing gotsigchld if jp is unset.
It also changes the waitcmd to actually set jp which corresponds
to the behaviour of bash/ksh93/mksh.
The only other caller of dowait that doesn't wait for everything
is the jobless reaper. This is in fact redundant now that we wait
after every simple command. This patch removes it.
Finally as every caller of dowait needs to wait until either the
given job is not running, or until all terminated jobs have been
processed, this patch moves the loop into dowait itself.
Fixes: 03876c0743a5 ("eval: Reap zombies after built-in...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream comment:
Date: Mon, 7 May 2018 00:40:34 +0800
jobs - Do not block when waiting on SIGCHLD
Because of the nature of SIGCHLD, the process may have already been
waited on and therefore we must be prepared for the case that wait
may block. So ensure that it doesn't by using WNOHANG.
Furthermore, multiple jobs may have exited when gotsigchld is set.
Therefore we need to wait until there are no zombies left.
Lastly, waitforjob needs to be called with interrupts off and
the original patch broke that.
Fixes: 03876c0743a5 ("eval: Reap zombies after built-in...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
While at it, removed INT_ON/OFF in waitforjob() - it must be called
from INT_OFF region anyway.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon, 26 Mar 2018 23:55:50 +0800
eval: Reap zombies after built-in commands and functions
Currently dash does not reap dead children after built-in commands
or functions. This means that if you construct a loop consisting
of solely built-in commands and functions, then zombies can hang
around indefinitely.
This patch fixes this by reaping when necessary after each built-in
command and function.
Reported-by: Denys Vlasenko <vda.linux@googlemail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:51 +0800
exec: Never rehash regular built-ins
As regular (including special) built-ins can never be overridden,
we should never remove them from the hash table.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream comment:
Date: Sat, 19 May 2018 02:39:50 +0800
exec: Stricter pathopt parsing
This patch changes the parsing of pathopt. First of all only
%builtin and %func (with arbitrary suffixes) will be recognised.
Any other pathopt will be treated as a normal directory.
Furthermore, pathopt can now be specified before the directory,
rather than after it. In fact, a future version may remove support
for pathopt suffixes.
Wherever the pathopt is placed, an optional % may be placed after
it to terminate the pathopt.
This is so that it is less likely that a genuine directory containing
a % sign is parsed as a pathopt.
Users of padvance outside of exec.c have also been modified:
1) cd(1) will always treat % characters as part of the path.
2) chkmail will continue to accept arbitrary pathopt.
3) find_dot_file will ignore the %builtin pathopt instead of trying
to do a stat in the accompanying directory (which is usually the
current directory).
The patch also removes the clearcmdentry optimisation where we
attempt to only partially flush the table where possible.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:48 +0800
exec: Do not allocate stack string in padvance
Many callers of padvance immediately free the allocated string
so this patch moves the stalloc call to the caller. Instead of
returning the allocated string, padvance now returns the length
to allocate (this may be longer than the actual string length,
even including the NUL). For the case where we would previously
return NULL, we now return -1.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:46 +0800
memalloc: Add growstackto helper
This patch adds the growstackto helper which repeatedly calls
growstackblock until the requested size is reached.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream comment:
Date: Sat, 19 May 2018 02:39:42 +0800
parser: Save/restore here-documents in command substitution
This patch changes the parsing of here-documents within command
substitution, both old style and new style. In particular, the
original here-document list is saved upon the beginning of parsing
command substitution and restored when exiting.
This means that here-documents outside of command substitution
can no longer be filled by text within it and vice-versa.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:40 +0800
mkinit: Split reset into exitreset and reset
Previously reset was called after exitshell. This was changed
so that it was called before exitshell because certain state needed
to be reset in order for the EXIT trap to work.
However, this caused issues because certain other states (such
as local variables) should not be reset. This patch fixes this
by creating a new function exitreset that is called prior to
exitshell and moving reset back to its original location.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon, 29 Apr 2019 19:13:37 +0500
expand: Fix trailing newlines processing in backquote expanding
According to POSIX.1-2008 we should remove newlines only at the end of
the substitution. Newlines-only substitions causes dash to remove
newlines before beggining of the substitution. The following code:
cat <<END
1
$(echo "")
2
END
prints "1<newline>2" instead of expected "1<newline><newline>2".
This patch fixes trailing newlines processing in backquote expanding.
Signed-off-by: Nikolai Merinov <n.merinov@inango-systems.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon, 27 May 2019 13:39:37 +0800
parser: Only accept single-digit parameter expansion outside of braces
This patch should fix the problem.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 15 Dec 2018 18:49:31 +0100
shell: Fix clang warnings about "string plus integer"
Building with clang results in some warnings about integer values being
added to strings.
While the code itself is fine and the warnings are indeed harmless,
fixing them also makes the semantic more explicit: what it is actually
being increased is the address which points to the start of the string
in order to skip the initial character when some conditions are met.
Signed-off-by: Antonio Ospite <ao2@ao2.it>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream comment:
Date: Mon, 19 Nov 2018 18:00:32 +0800
eval: Use the correct expansion mode for fd redirection
It has been reported that
echo test >&$EMPTY_VARIABLE
causes dash to segfault. This is a symptom of the bigger problem
that dash tries to perform pathname expansion as well as field
splitting on the word after >& and <&. This is wrong and this
patch fixes it to use the same expansions as done on a normal
redirection.
Reported-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon, 28 May 2018 17:09:48 +0800
expand: Fix skipping of command substitution when trimming in evalvar
When we are trimming an unset variable in evalvar, any embedded
command substitution that should have been skipped are not. This
can cause them to be evaluated later should there be other command
substitutions in the same input word.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream comment:
Date: Fri, 7 Sep 2018 10:34:14 +0200
main: Print \n upon EOF (CTRL-D) when run interactively
Exiting dash via a ^D instead of with "exit" causes dash to forget to
print a newline.
sh-3.1$ sh
sh-3.1$ ^D
sh-3.1$ dash
$ sh-3.1$
It is more neat and tidy to send a newline similarly to what bash does,
so it doesn't make the next prompt of the parent shell look ugly.
Suggested by jidanni.
Signed-off-by: Gerrit Pape <pape@smarden.org>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
[reworded the patch description]
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
Bug-Debian: http://bugs.debian.org/476422
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sun, 27 May 2018 17:31:57 +0800
expand: Use HOME in tilde expansion when it is empty
Currently if HOME is set to empty tilde expansion will fail, i.e.,
it will remain as a literal tilde. This patch changes it to
return the empty string as required by POSIX.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:49 +0800
builtin: Mark more regular built-ins
This patch marks the following built-ins as regular, meaning that
they cannot be overriden using PATH search:
hash
pwd
type
ulimit
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:45 +0800
jobs: Replace some uses of fmtstr with stpcpy/stpncpy
Some uses of fmtstr, particularly the ones without a format string,
can be replaced with stpcpy or stpncpy. This patch does that so
we don't have to introduce unnecessary format strings in order to
silence compiler warnings.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:44 +0800
output: Fix fmtstr return value
The function fmtstr is meant to return the actual length of output
produced, rather than the untruncated length.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:38 +0800
main: Only set savestatus in exitcmd
Currently exitcmd sets exitstatus and then savestatus if the latter
was previously set. In fact, as exitcmd always raises an exception
and will either end up in the setjmp call in main() or exitshell(),
where exitstatus is always replaced by savestatus if set, we only
need to set savestatus.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon Oct 6 10:39:47 2014 +0800
[BUILTIN] Exit without arguments in a trap should use status outside traps
POSIX now requires that exit without arguments in a trap should
return the last command status prior to executing traps. This
patch implements this behaviour.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Keeping up with upstream (in our case, 'before patch' code is not buggy).
Upstream commit:
Date: Fri, 11 May 2018 23:41:25 +0800
parser: Fix incorrect eating of backslash newlines
With the introduction of synstack->syntax, a number of references
to the syntax variable was missed during the conversion. This
causes backslash newlines to be incorrectly removed in single
quote context.
This patch also combines these calls into a new helper function
pgetc_top.
Fixes: ab1cecb40478 ("parser: Add syntax stack for recursive...")
Reported-by: Leah Neukirchen <leah@vuxu.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Adding previously skipped "readtoken1(pgetc_eatbnl(), DQSYNTAX..." changes
from upstream commit:
Date: Thu Mar 8 08:37:11 2018 +0100
Author: Harald van Dijk <harald@gigawatt.nl>
parser: use pgetc_eatbnl() in more places
dash has a pgetc_eatbnl function in parser.c which skips any
backslash-newline combinations. It's not used everywhere it could be.
There is also some duplicated backslash-newline handling elsewhere in
parser.c. Replace most of the calls to pgetc() with calls to
pgetc_eatbnl() and remove the duplicated backslash-newline handling.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
The dash maintainer recently posted a fix for issues with expanding
PS1. These had already been fixed differently in BusyBox ash. Borrow
a couple of improvements:
- Use a single call to setjmp() to trap errors in both readtoken1()
and expandarg().
- In case of error set the prompt to the literal value of PS1 rather
than the half-digested nonsense in stackblock() which might include
ugly control characters.
function old new delta
expandstr 353 300 -53
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Commit d1a2fa2a4 (ash: catch error in arithmetic expansion in PS1)
catches all exceptions raised by expandarg(). Some exceptions, such as
the EXEXIT raised when command expansion is used, are expected:
export PS1='$(echo "$ ")'
These should be processed normally or the shell hangs at the prompt.
function old new delta
expandstr 344 353 +9
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/0 up/down: 9/0) Total: 9 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Setting PS1 to:
PS1='$((123+))'
causes the shell to enter an infinite error loop:
sh: arithmetic syntax error
Catch any exception raised by expandarg() in expandstr() and allow
processing to continue.
function old new delta
expandstr 262 344 +82
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/0 up/down: 82/0) Total: 82 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Setting PS1 to:
PS1='`xxx(`'
causes the shell to terminate with the error:
sh: syntax error: unexpected end of file (expecting ")")
This happens because old-style backquotes require the input to be reread
and thus call setinputstring() a second time. Prevent the problem by
unwinding all recently opened files in expandstr().
function old new delta
unwindfiles - 22 +22
expandstr 247 262 +15
forkchild 631 625 -6
evalcommand 1694 1685 -9
ash_main 1346 1336 -10
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 1/3 up/down: 37/-25) Total: 12 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Bash 5.0 added the dynamic variable EPOCHSECONDS and EPOCHREALTIME
which return the number of seconds since the Unix Epoch as an
integer or float. These are useful for logging or tracing.
function old new delta
change_epoch - 78 +78
.rodata 175167 175235 +68
varinit_data 264 312 +48
change_seconds - 24 +24
change_realtime - 24 +24
------------------------------------------------------------------------------
(add/remove: 3/0 grow/shrink: 2/0 up/down: 242/0) Total: 242 bytes
text data bss dec hex filename
938508 4203 1888 944599 e69d7 busybox_old
938702 4203 1888 944793 e6a99 busybox_unstripped
v2: Cast tv_sec and tv_usec to unsigned quantities.
Add brackets to macros.
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Commit b28d4c346 (ash: [VAR] Move unsetvar functionality into setvareq)
dropped the code that caused dynamic variables to lose their special
properties when unset. Add it back again.
function old new delta
setvareq 346 360 +14
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/0 up/down: 14/0) Total: 14 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Add a specific define to indicate which bash compatibility code
implements 'wait -n'.
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
The following constructs result in ever-increasing memory usage:
while true; do { true; } </dev/null; done
while true; do ( true; ) </dev/null; done
For comparison, bash displays static memory usage in both cases.
This has been fixed in dash by commit 2bc6caa. The maintainer
writes:
I have simplified evaltree so that it simply sets the stack mark
unconditionally. This allows us to remove the stack marks in the
functions called by evaltree.
Closes BusyBox bug 7748.
function old new delta
evaltree 606 632 +26
evalcommand 1724 1696 -28
evalcase 382 351 -31
evalfor 230 196 -34
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/3 up/down: 26/-93) Total: -67 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Embedded scripts require a shell to be present in the BusyBox
binary. Allow either ash or hush to be used for this purpose.
If both are enabled ash takes precedence.
The size of the binary is unchanged in the default configuration:
both ash and hush are present but support for embedded scripts
isn't compiled into hush.
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
The parser reads from an already freed memory location, thereby causing
unpredictable results, in the following situation:
- ENABLE_ASH_EXPAND_PRMT is enabled
- heredoc is being parsed
- command substitution is used within heredoc
Examples where this bug crops up are (PS2 is set to "> "):
$ cat <<EOF
> `echo abc`
> EOF
-sh: O: not found
$ cat <<EOF
> $(echo abc)
> EOF
-sh: {garbage}: not found
The presumable reason is that setprompt_if() causes a nested expansion when
ENABLE_ASH_EXPAND_PRMT is enabled, therefore leaving "wordtext" in an unusable
state. However, when parseheredoc() is called, "tokpushback" is non-zero, which
causes the next call to xxreadtoken() to return TWORD, causing the caller to
use the invalid "wordtoken" instead of reading the next valid token.
The call chain is:
list()
-> peektoken() [sets tokpushback to 1]
-> parseheredoc()
-> setprompt_if()
-> pushstackmark()
-> expandstr()
-> readtoken1()
[sets lasttoken to TWORD, wordtoken points to expanded prompt]
-> popstackmark() [invalidates wordtoken, leaves lasttoken as is]
-> readtoken1()
-> ...parsebackq
-> list()
-> andor()
-> pipeline()
-> readtoken()
-> xxreadtoken()
[tokpushback non-zero, reuse lasttoken and wordtext]
Note that in almost all other contexts, each call to setprompt_if() is preceded
by setting "tokpushback" to zero. One exception is "oldstyle" backquote parsing
in readtoken1(), but there "tokpushback" is reset afterwards. The other
exception is nlprompt(), but this function is only used within readtoken1()
(but in contexts where no nested calls to xxreadtoken() occur) and xxreadtoken()
(where "tokpushback" is guaranteed to be zero).
function old new delta
parseheredoc 124 131 +7
Signed-off-by: Christoph Schulz <develop@kristov.de>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
BusyBox has support for embedded shell scripts. Two types can be
distinguished: custom scripts and scripts implementing applets.
Custom scripts should be placed in the 'embed' directory at build
time. They are given a default applet configuration and appear
as applets to the user but no further configuration is possible.
Applet scripts are integrated with the BusyBox build system and
are intended to be used to ship standard applets that just happen
to be implemented as scripts. They can be configured at build time
and appear just like native applets.
Such scripts should be placed in the 'applets_sh' directory. A stub
C program should be written to provide the usual applet configuration
details and placed in a suitable subsystem directory. It may be
helpful to have a configuration option to enable any dependencies the
script requires: see the 'nologin' applet for an example.
function old new delta
scripted_main - 41 +41
applet_names 2773 2781 +8
applet_main 1600 1604 +4
i2cdetect_main 672 674 +2
applet_suid 100 101 +1
applet_install_loc 200 201 +1
applet_flags 100 101 +1
packed_usage 33180 33179 -1
tryexec 159 152 -7
evalcommand 1661 1653 -8
script_names 9 - -9
packed_scripts 123 114 -9
complete_cmd_dir_file 826 811 -15
shellexec 271 254 -17
find_command 1007 990 -17
busybox_main 642 624 -18
run_applet_and_exit 100 78 -22
find_script_by_name 51 - -51
------------------------------------------------------------------------------
(add/remove: 1/2 grow/shrink: 6/9 up/down: 58/-174) Total: -116 bytes
text data bss dec hex filename
950034 477 7296 957807 e9d6f busybox_old
949918 477 7296 957691 e9cfb busybox_unstripped
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
When a variable is unset by calling setvar(name, NULL, 0) the code
to initialise the new, empty variable fails to initialise the last
character of the string.
Attempts to read the contents of the unset variable will result
in the uninitialised character at the end of the string being
accessed.
For example, running BusyBox under Valgrind and unsetting PATH:
$ valgrind ./busybox_unstripped sh
==21249== Memcheck, a memory error detector
==21249== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==21249== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==21249== Command: ./busybox_unstripped sh
==21249==
/data2/git/build_fix_8721 $ unset PATH
/data2/git/build_fix_8721 $ 0
==21249== Conditional jump or move depends on uninitialised value(s)
==21249== at 0x451371: path_advance (ash.c:2555)
==21249== by 0x456E22: find_command (ash.c:13407)
==21249== by 0x458425: evalcommand (ash.c:10139)
==21249== by 0x454CBC: evaltree (ash.c:9131)
==21249== by 0x456C80: cmdloop (ash.c:13164)
Closes https://bugs.busybox.net/show_bug.cgi?id=8721
v2: On the dash mailing list Harald van Dijk was kind enough to point
out a flaw in my reasoning and provide an alternative patch. Sadly
his patch adds 2 bytes of bloat. Using xzalloc to zero the whole
string gives a bloat of -3 bytes.
function old new delta
setvar 172 169 -3
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Ensure that login_sh is initialised in procargs even when running
an embedded script.
The argc argument to ash_main isn't unused when embedded scripts
are present.
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
To assist in the deployment of shell scripts it may be convenient
to embed them in the BusyBox binary.
'Embed scripts in the binary' takes any files in the directory
'embed', concatenates them with null separators, compresses them
and embeds them in the binary.
When scripts are embedded in the binary, scripts can be run as
'busybox SCRIPT [ARGS]' or by usual (sym)link mechanism.
embed/nologin is provided as an example.
function old new delta
packed_scripts - 123 +123
unpack_scripts - 87 +87
ash_main 1103 1171 +68
run_applet_and_exit 78 128 +50
get_script_content - 32 +32
script_names - 10 +10
expmeta 663 659 -4
------------------------------------------------------------------------------
(add/remove: 4/0 grow/shrink: 2/1 up/down: 370/-4) Total: 366 bytes
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Wed, 28 Mar 2018 18:37:51 +0800
expand: Do not quote backslashes in unquoted parameter expansion
Here is a better example:
a="/*/\nullx" b="/*/\null"; printf "%s\n" $a $b
dash currently prints
/*/\nullx
/*/\null
bash prints
/*/\nullx
/dev/null
You may argue the bash behaviour is inconsistent but it actually
makes sense. What happens is that quote removal only applies to
the original token as seen by the shell. It is never applied to
the result of parameter expansion.
Now you may ask why on earth does the second line say "/dev/null"
instead of "/dev/\null". Well that's because it is not the quote
removal step that removed the backslash, but the pathname expansion.
The fact that the /de\v does not become /dev even though it exists
is just the result of the optimisation to avoid unnecessarily
calling stat(2). I have checked POSIX and I don't see anything
that forbids this behaviour.
So going back to dash yes I think we should adopt the bash behaviour
for pathname expansion and keep the existing case semantics.
This patch does exactly that. Note that this patch does not work
unless you have already applied
https://patchwork.kernel.org/patch/10306507/
because otherwise the optimisation mentioned above does not get
detected correctly and we will end up doing quote removal twice.
This patch also updates expmeta to handle naked backslashes at
the end of the pattern which is now possible.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
function old new delta
expmeta 618 653 +35
memtodest 146 147 +1
Tested to work with both ASH_INTERNAL_GLOB on and off.
hush does not handle this correctly.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:37 +0800
exec: Return 126 on most errors in shellexec
Currently when shellexec fails on most errors the shell will exit
with exit status 2. This patch changes it to 126 in order to avoid
ambiguities with the exit status from a successful exec.
The errors that result in 127 has also been expanded to include
ENOTDIR, ENAMETOOLONG and ELOOP.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
function old new delta
shellexec 245 254 +9
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Tue, 27 Mar 2018 00:39:35 +0800
eval: Restore input files in evalcommand
When evalcommand invokes a command that modifies parsefile and
then bails out without popping the file, we need to ensure the
input file is restored so that the shell can continue to execute.
Reported-by: Martijn Dekker <martijn@inlv.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
function old new delta
unwindfiles - 20 +20
evalcommand 1635 1653 +18
getoptscmd 584 595 +11
popallfiles 20 10 -10
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 2/1 up/down: 49/-10) Total: 39 bytes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Tue, 3 Apr 2018 00:40:25 +0800
parser: Fix parsing of ${}
dash -c 'echo ${}' should print "Bad subtitution" but instead
fails with "Syntax error: Missing '}'". This is caused by us
reading an extra character beyond the right brace. This patch
fixes it so that this construct only fails during expansion rather
than during parsing.
Fixes: 3df3edd13389 ("[PARSER] Report substition errors at...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
function old new delta
readtoken1 2907 2916 +9
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Fri, 23 Mar 2018 18:58:47 +0800
expand: Fix ghost fields with unquoted $@/$*
You're right. The proper fix to this is to ensure that nulonly
is not set in varvalue for $*. It should only be set for $@ when
it's inside double quotes.
In fact there is another bug while we're playing with $@/$*.
When IFS is set to a non-whitespace character such as :, $*
outside quotes won't remove empty fields as it should.
This patch fixes both problems.
Reported-by: Martijn Dekker <martijn@inlv.org>
Suggested-by: Harald van Dijk <harald@gigawatt.nl>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
function old new delta
argstr 1111 1113 +2
evalvar 571 569 -2
varvalue 579 576 -3
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/2 up/down: 2/-5) Total: -3 bytes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:43 +0800
var: Set IFS to fixed value at start time
This patch forces the IFS variable to always be set to its default
value, regardless of the environment.
It also removes the long unused IFS_BROKEN code.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Wed, 4 Apr 2018 17:54:01 +0800
eval: Variable assignments on functions are no longer persistent
Dirk Fieldhouse <fieldhouse@gmx.net> wrote:
> In POSIX.1-2017 ("simultaneously IEEE Std 1003.1™-2017 and The Open
> Group Technical Standard Base Specifications, Issue 7")
> <http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09>,
> we read under '2.9.1 Simple Commands'
>
> "Variable assignments shall be performed as follows:
> ...
> - If the command name is a standard utility implemented as a function
> (see XBD Utility), the effect of variable assignments shall be as if the
> utility was not implemented as a function.
> ...
> - If the command name is a function that is not a standard utility
> implemented as a function, variable assignments shall affect the current
> execution environment during the execution of the function. It is
> unspecified:
>
> * Whether or not the variable assignments persist after the
> completion of the function
>
> * Whether or not the variables gain the export attribute during
> the execution of the function
>
> * Whether or not export attributes gained as a result of the
> variable assignments persist after the completion of the function (if
> variable assignments persist after the completion of the function)"
POSIX used to require the current dash behaviour. However, you're
right that this is no longer the case.
This patch will remove the persistence of the variable assignment.
I have considered the exporting the variables during the function
execution but have decided against it because:
1) It makes the code bigger.
2) dash has never done this in the past.
3) You cannot use this portably anyway.
Reported-by: Dirk Fieldhouse <fieldhouse@gmx.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
function old new delta
evalcommand 1606 1635 +29
evalcase 313 317 +4
evalfun 280 268 -12
pushlocalvars 48 - -48
------------------------------------------------------------------------------
(add/remove: 0/1 grow/shrink: 2/1 up/down: 33/-60) Total: -27 bytes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sun, 25 Mar 2018 16:38:00 +0800
expand: Fix buffer overflow in expandmeta
The native version of expandmeta allocates a buffer that may be
overrun for two reasons. First of all the size is 1 byte too small
but this is normally hidden because the minimum size is rounded
up to 2048 bytes. Secondly, if the directory level is deep enough,
any buffer can be overrun.
This patch fixes both problems by calling realloc when necessary.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
function old new delta
expmeta 517 635 +118
expandarg 990 996 +6
mklocal 288 290 +2
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 3/0 up/down: 126/0) Total: 126 bytes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Thu, 22 Mar 2018 21:41:24 +0800
parser: Allow newlines within parameter substitution
On Fri, Mar 16, 2018 at 11:27:22AM +0800, Herbert Xu wrote:
> On Thu, Mar 15, 2018 at 10:49:15PM +0100, Harald van Dijk wrote:
> >
> > Okay, it can be trivially modified to something that does work in other
> > shells (even if it were actually executed), but gets rejected at parse time
> > by dash:
> >
> > if false; then
> > : ${$+
> > }
> > fi
>
> That's just a bug in dash's parser with ${} in general, because
> it bombs out without the if clause too:
>
> : ${$+
> }
This patch fixes the parsing of newlines with parameter substitution.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
newer bash does not drop it, most other shells too
function old new delta
unbackslash 39 57 +18
parse_stream 2753 2751 -2
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/1 up/down: 18/-2) Total: 16 bytes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream email:
parser: Fix parameter expansion inside inner double quotes
The parsing of parameter expansion inside inner double quotes
breaks because we never look for ENDVAR while innerdq is true.
echo "${x#"${x+''}"''}
This patch fixes it by pushing the syntax stack if innerdq is
true and we enter a new parameter expansion.
This patch also fixes a corner case where a bad substitution error
occurs within arithmetic expansion.
Reported-by: Denys Vlasenko <vda.linux@googlemail.com>
Fixes: ab1cecb40478 (" parser: Add syntax stack for recursive...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
function old new delta
readtoken1 2880 2898 +18
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream email:
This is actually composed of two bugs. First of all our tracking
of quotemark is wrong so anything after "$@" becomes quoted. Once
we fix that then the problem is that the first space character
after "$@" is not recognised as an IFS.
This patch fixes both.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 9 Mar 2018 23:07:53 +0800
parser: Fix single-quoted patterns in here-documents
The script
x=*
cat <<- EOF
${x#'*'}
EOF
prints * instead of nothing as it should. The problem is that
when we're in sqsyntax context in a here-document, we won't add
CTLESC as we should. This patch fixes it:
Reported-by: Harald van Dijk <harald@gigawatt.nl>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
This closes 10821.
Upstream patch:
From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 9 Mar 2018 00:14:02 +0800
parser: Add syntax stack for recursive parsing
Without a stack of syntaxes we cannot correctly these two cases
together:
"${a#'$$'}"
"${a#"${b-'$$'}"}"
A recursive parser also helps in some other corner cases such
as nested arithmetic expansion with paratheses.
This patch adds a syntax stack allocated from the stack using
alloca. As a side-effect this allows us to remove the naked
backslashes for patterns within double-quotes, which means that
EXP_QPAT also has to go.
This patch also fixes removes any backslashes that precede right
braces when they are present within a parameter expansion context,
and backslashes that precede double quotes within inner double
quotes inside a parameter expansion in a here-document context.
The idea of a recursive parser is based on a patch by Harald van
Dijk.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
var_bash3, var_bash4 and var_bash6 tests are updated
with the output given by bash-4.3.43
With this patch, the following tests now pass for ash:
dollar_repl_slash_bash2.tests
squote_in_varexp2.tests
squote_in_varexp.tests
var_bash4.tests
function old new delta
readtoken1 2615 2874 +259
synstack_push - 54 +54
evalvar 574 571 -3
rmescapes 330 310 -20
subevalvar 1279 1258 -21
argstr 1146 1107 -39
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 1/4 up/down: 313/-83) Total: 230 bytes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Author: Herbert Xu <herbert@gondor.apana.org.au>
Date: Thu Mar 15 18:27:30 2018 +0800
parser: Fix backquote support in here-document EOF mark
Currently using backquotes in a here-document EOF mark is broken
because dash tries to do command substitution on it. This patch
fixes it by checking whether we're looking for an EOF mark during
tokenisation.
Reported-by: Harald van Dijk <harald@gigawatt.nl>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
With added fix for quoted-ness of the EOF mark.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Part of upstream commit:
Date: Thu Mar 8 08:37:11 2018 +0100
Author: Harald van Dijk <harald@gigawatt.nl>
parser: use pgetc_eatbnl() in more places
dash has a pgetc_eatbnl function in parser.c which skips any
backslash-newline combinations. It's not used everywhere it could be.
There is also some duplicated backslash-newline handling elsewhere in
parser.c. Replace most of the calls to pgetc() with calls to
pgetc_eatbnl() and remove the duplicated backslash-newline handling.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Not adding "readtoken1(pgetc_eatbnl(), DQSYNTAX..." changes, since
readtoken1() handles the "starts with backslash + newline" case itself.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Tue Mar 6 17:40:37 2018 +0000
expand: 'nolog' and 'debug' options cause "$-" to wreak havoc
Op 29-03-17 om 20:02 schreef Martijn Dekker:
> Bug: if either the 'nolog' or the 'debug' option is set, trying to
> expand "$-" silently aborts parsing of an entire argument.
>
> $ dash -o nolog -c 'set -fuC; echo "|$- are the options|"; set +o nolog; echo "|$- are the options|"'
> |
> |uCf are the options|
> $ dash -o debug -c 'set -fuC; echo "|$- are the options|"; set +o debug; echo "|$- are the options|"'
> |
> |uCf are the options|
This turned out to be easy to fix. The routine producing the "$-"
expansion failed to skip options for which there is no option letter,
but only a long-form name. In dash, 'nolog' and 'debug' are currently
the only two such options. Patch below.
- Martijn
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In bbox ash, pipefail is the option which exhibited this.
Signed-off-by: Martijn Dekker <martijn@inlv.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
It usually does not bite since bbox forces -funsigned-char build.
But for some reason void linux people disabled that.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
function old new delta
setjobctl 371 367 -4
setinputfile 226 220 -6
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-10) Total: -10 bytes
Based on patch by Mark Marshall <mark.marshall@omicronenergy.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Based on a patch by Mark Marshall <mark.marshall@omicronenergy.com>
function old new delta
dup_CLOEXEC - 49 +49
fcntl_F_DUPFD 46 - -46
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
This implements support for the command_not_found_handle hook function, which is
useful for allowing package managers to suggest packages which could provide the
command.
Unlike bash, however, we ignore exit codes from the hook function and always return
the correct POSIX error code (EX_NOTFOUND).
function old new delta
find_command 911 990 +79
Signed-off-by: William Pitcock <nenolod@dereferenced.org>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Before this patch, "${v:2:0x100000001}" = "${v:2:1}",
and similarly, constructs like "${v:2:9999999999}" may give wrong result
due to int overflows.
function old new delta
substr_atoi - 43 +43
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
If the numeric argument passed to ash's 'shift' built-in is greater than
'$#' the command performs no operation and exits successfully. It should
return a non-zero exit code instead:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#shift
This is consistent with bash and hush.
function old new delta
shiftcmd 122 120 -2
Signed-off-by: Ingo van Lil <inguin@gmx.de>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
