Upstream patch:
Date: Wed, 30 May 2018 02:06:03 +0800
expand: Do not reprocess data when expanding words
Currently various paths will reprocess data when performing word
expansion. For example, expari will skip backwards looking for
the start of the arithmetic expansion, while evalvar will skip
unexpanded words manually.
This is cumbersome and error-prone. This patch fixes this by
making word expansions proceed in a linear fashion. This means
changing argstr and the various expansion functions such as expari
and subevalvar to return the next character to be expanded.
This is inspired by similar code from FreeBSD. However, we take
things one step further and completely remove the manual word
skipping in evalvar. This is accomplished by introducing a new
EXP_DISCARD flag that tells argstr to only parse and not produce
any actual expansions.
Incidentally, argstr will now always NUL-terminate the expansion
unless the EXP_WORD flag is set. This is because all but one
caller of argstr wants the result to be NUL-termianted.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Also includes two one-line follow-up fixes:
expand: Eat closing brace for length parameter
if (subtype == VSLENGTH) {
+ p++;
if (flag & EXP_DISCARD)
expand: Fix double-decrement in argstr
- newloc = expdest - (char *)stackblock() - end;
+ newloc = q - (char *)stackblock() - end;
and changes in code for bash substring extensions.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Following recent work on evalcommand() a TRACE statement to report
the status of a forked command was left in the wrong place.
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sun, 11 Nov 2007 15:27:00 +0800
Expand here-documents in the current shell environment
Previously we always expanded here-documents in a subshell. This is
contrary to the POSIX specification and how other shells behave. What's
more this slows down many expansions due to the extra fork (however, it
must be said that it is possible for it speed up certain expansions by
running it simultaneously with the command on two CPUs).
This patch move the expansion into the current shell environment.
Test case:
unset a
cat <<- EOF > /dev/null
${a=NOT}
EOF
echo ${a}BAD
Old result:
BAD
New result:
NOTBAD
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon, 19 Nov 2018 18:43:58 +0800
parser: Do not push token back before parseheredoc
When we read the first token in list() we use peektoken instead
of readtoken as the following code needs to use the same token
again. However, this is wrong when we're in a here-document as
it will clobber the saved token without resetting the tokpushback
flag.
This patch fixes it by doing the tokpushback after parseheredoc
and setting lasttoken again if parseheredoc was called.
Reported-by: Ron Yorston <rmy@frippery.org>
Fixes: 7c245aa8ed33 ("[PARSER] Simplify EOF/newline handling in...")
Fixes: ee5cbe9fd6bc ("[SHELL] Optimize dash -c "command" to avoid a fork")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Tested-by: Simon Ser <contact@emersion.fr>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Fri, 29 Mar 2019 13:49:59 +0800
parser: Fix old-style command substitution here-document crash
... This is caused by the recent change to
save/restore here-docment list around command substitutions. In
doing so we must finish existing here-documents prior to restoring
the old here-document list. This is done for new-style command
substitutions but not for old-style.
This patch fixes it by doing it for both.
Fixes: 51e2d88d6e51 ("parser: Save/restore here-documents in...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Fri, 1 Jun 2018 18:25:29 +0800
expand: Ensure result is escaped in cvtnum
The minus sign generated from arithmetic expansion is currently
unquoted which causes anomalies when the result is used in where
the quoting matters.
This patch fixes it by explicitly calling memtodest on the result
in cvtnum.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Thu, 31 May 2018 01:51:48 +0800
memalloc: Avoid looping in growstackto
Currently growstackto will repeatedly call growstackblock until
the requisite size is obtained. This is wasteful. This patch
changes growstackblock to take a minimum size instead.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Fri, 14 Dec 2018 13:52:02 +0800
eval: Only restore exit status on exit/return
We unconditionally restore the saved status in exitreset, which
is incorrect as we only want to do it for exitcmd and returncmd.
This patch fixes the problem by introducing EXEND.
Reported-by: Martijn Dekker <martijn@inlv.org>
Fixes: da30b4b78769 ("[BUILTIN] Exit without arguments in a trap...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Fixes exitcode_trap4.tests.
Upstream commit:
Date: Mon, 6 Oct 2014 21:51:26 +0800
Return without arguments in a trap should use status outside traps
POSIX now requires that return without arguments in a trap should
return the last command status prior to executing traps. This
patch implements this behaviour.
Incidentally this also changes the behaviour of return without
arguments in a loop conditional to use the last exit status in
the body as opposed to the last command in the conditional when
there is one.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Since commit d81af7216 (ash: eval: Reap zombies after built-in commands
and functions) if the shell is compiled with SH_STANDALONE and SH_NOFORK
enabled nofork applets potentially return the incorrect status.
The status value returned by evalcommand() in this case is obtained from
exitstatus in a call to waitforjob(NULL). This overwrites the status
set for nonfork applets.
If this commit seems familiar it's essentially a reversion of commit
5ccb0e92fa (ash: return exit status of nofork applets). What was correct
in 2016 is wrong in 2020 and vice versa.
Signed-off-by: Ron Yorston <rmy@pobox.com>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon, 28 May 2018 00:17:39 +0800
expand: Merge syntax/quotes in memtodest with flags
The function arguments syntax and quotes are both derived from
the expansion flags. As syntax is only used by memtodest we do
not need to maintain it outside of the function at all.
The only place that uses something other than BASESYNTAX or DQSYNTAX
is exptilde. However in that case DQSYNTAX has exactly the same
effect as SQSYNTAX.
This patch merges these two arguments into a single flags. The
macro QUOTES_KEEPNUL has been renamed to EXP_KEEPNUL in order
to keep the namespace separate.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Thu, 31 May 2018 01:15:34 +0800
eval: Always set localvar_stop
The variable localvar_stop is set iff vlocal is true. gcc doesn't
get this so we get a spurious warning.
This patch fixes this by always calling pushlocalvars with vlocal
and making it only actually do the push if vlocal is non-zero.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:55 +0800
eval: Replace with listsetvar with mklocal/setvareq
This patch replaces listsetvar with mklocal/setvareq. As we now
determine special built-in status prior to variable assignment, we
no longer have to do a second pass listsetvar. Instead we will
call setvareq directly instead of mklocal when necessary.
In order to do this mklocal can now take a flag in order to mark
a variable for export.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:54 +0800
eval: Fail immediately with redirections errors for simple command
Previously, dash would continue to perform variable expansions
even if a redirection error occured. This patch changes it so
that it fails immediately.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:52 +0800
eval: Add assignment built-in support again
This patch adds assignment built-in support that used to exist
in dash prior to 0.3.8-15. This is because it will soon be part
of POSIX, and the semantics are now much better defined.
Recognition is done at execution time, so even "command -- export"
or "var=export; command $var" should work.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:41 +0800
jobs: Only clear gotsigchld when waiting for everything
The gotsigchld flag is always cleared in dowait but not all callers
of dowait will wait for everything. In particular, when jp is set
we only wait until the set job isn't running anymore.
This patch fixes this by only clearing gotsigchld if jp is unset.
It also changes the waitcmd to actually set jp which corresponds
to the behaviour of bash/ksh93/mksh.
The only other caller of dowait that doesn't wait for everything
is the jobless reaper. This is in fact redundant now that we wait
after every simple command. This patch removes it.
Finally as every caller of dowait needs to wait until either the
given job is not running, or until all terminated jobs have been
processed, this patch moves the loop into dowait itself.
Fixes: 03876c0743a5 ("eval: Reap zombies after built-in...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream comment:
Date: Mon, 7 May 2018 00:40:34 +0800
jobs - Do not block when waiting on SIGCHLD
Because of the nature of SIGCHLD, the process may have already been
waited on and therefore we must be prepared for the case that wait
may block. So ensure that it doesn't by using WNOHANG.
Furthermore, multiple jobs may have exited when gotsigchld is set.
Therefore we need to wait until there are no zombies left.
Lastly, waitforjob needs to be called with interrupts off and
the original patch broke that.
Fixes: 03876c0743a5 ("eval: Reap zombies after built-in...")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
While at it, removed INT_ON/OFF in waitforjob() - it must be called
from INT_OFF region anyway.
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon, 26 Mar 2018 23:55:50 +0800
eval: Reap zombies after built-in commands and functions
Currently dash does not reap dead children after built-in commands
or functions. This means that if you construct a loop consisting
of solely built-in commands and functions, then zombies can hang
around indefinitely.
This patch fixes this by reaping when necessary after each built-in
command and function.
Reported-by: Denys Vlasenko <vda.linux@googlemail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:51 +0800
exec: Never rehash regular built-ins
As regular (including special) built-ins can never be overridden,
we should never remove them from the hash table.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream comment:
Date: Sat, 19 May 2018 02:39:50 +0800
exec: Stricter pathopt parsing
This patch changes the parsing of pathopt. First of all only
%builtin and %func (with arbitrary suffixes) will be recognised.
Any other pathopt will be treated as a normal directory.
Furthermore, pathopt can now be specified before the directory,
rather than after it. In fact, a future version may remove support
for pathopt suffixes.
Wherever the pathopt is placed, an optional % may be placed after
it to terminate the pathopt.
This is so that it is less likely that a genuine directory containing
a % sign is parsed as a pathopt.
Users of padvance outside of exec.c have also been modified:
1) cd(1) will always treat % characters as part of the path.
2) chkmail will continue to accept arbitrary pathopt.
3) find_dot_file will ignore the %builtin pathopt instead of trying
to do a stat in the accompanying directory (which is usually the
current directory).
The patch also removes the clearcmdentry optimisation where we
attempt to only partially flush the table where possible.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:48 +0800
exec: Do not allocate stack string in padvance
Many callers of padvance immediately free the allocated string
so this patch moves the stalloc call to the caller. Instead of
returning the allocated string, padvance now returns the length
to allocate (this may be longer than the actual string length,
even including the NUL). For the case where we would previously
return NULL, we now return -1.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:46 +0800
memalloc: Add growstackto helper
This patch adds the growstackto helper which repeatedly calls
growstackblock until the requested size is reached.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream comment:
Date: Sat, 19 May 2018 02:39:42 +0800
parser: Save/restore here-documents in command substitution
This patch changes the parsing of here-documents within command
substitution, both old style and new style. In particular, the
original here-document list is saved upon the beginning of parsing
command substitution and restored when exiting.
This means that here-documents outside of command substitution
can no longer be filled by text within it and vice-versa.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:40 +0800
mkinit: Split reset into exitreset and reset
Previously reset was called after exitshell. This was changed
so that it was called before exitshell because certain state needed
to be reset in order for the EXIT trap to work.
However, this caused issues because certain other states (such
as local variables) should not be reset. This patch fixes this
by creating a new function exitreset that is called prior to
exitshell and moving reset back to its original location.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon, 29 Apr 2019 19:13:37 +0500
expand: Fix trailing newlines processing in backquote expanding
According to POSIX.1-2008 we should remove newlines only at the end of
the substitution. Newlines-only substitions causes dash to remove
newlines before beggining of the substitution. The following code:
cat <<END
1
$(echo "")
2
END
prints "1<newline>2" instead of expected "1<newline><newline>2".
This patch fixes trailing newlines processing in backquote expanding.
Signed-off-by: Nikolai Merinov <n.merinov@inango-systems.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon, 27 May 2019 13:39:37 +0800
parser: Only accept single-digit parameter expansion outside of braces
This patch should fix the problem.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 15 Dec 2018 18:49:31 +0100
shell: Fix clang warnings about "string plus integer"
Building with clang results in some warnings about integer values being
added to strings.
While the code itself is fine and the warnings are indeed harmless,
fixing them also makes the semantic more explicit: what it is actually
being increased is the address which points to the start of the string
in order to skip the initial character when some conditions are met.
Signed-off-by: Antonio Ospite <ao2@ao2.it>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream comment:
Date: Mon, 19 Nov 2018 18:00:32 +0800
eval: Use the correct expansion mode for fd redirection
It has been reported that
echo test >&$EMPTY_VARIABLE
causes dash to segfault. This is a symptom of the bigger problem
that dash tries to perform pathname expansion as well as field
splitting on the word after >& and <&. This is wrong and this
patch fixes it to use the same expansions as done on a normal
redirection.
Reported-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon, 28 May 2018 17:09:48 +0800
expand: Fix skipping of command substitution when trimming in evalvar
When we are trimming an unset variable in evalvar, any embedded
command substitution that should have been skipped are not. This
can cause them to be evaluated later should there be other command
substitutions in the same input word.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream comment:
Date: Fri, 7 Sep 2018 10:34:14 +0200
main: Print \n upon EOF (CTRL-D) when run interactively
Exiting dash via a ^D instead of with "exit" causes dash to forget to
print a newline.
sh-3.1$ sh
sh-3.1$ ^D
sh-3.1$ dash
$ sh-3.1$
It is more neat and tidy to send a newline similarly to what bash does,
so it doesn't make the next prompt of the parent shell look ugly.
Suggested by jidanni.
Signed-off-by: Gerrit Pape <pape@smarden.org>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
[reworded the patch description]
Signed-off-by: Andrej Shadura <andrew.shadura@collabora.co.uk>
Bug-Debian: http://bugs.debian.org/476422
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sun, 27 May 2018 17:31:57 +0800
expand: Use HOME in tilde expansion when it is empty
Currently if HOME is set to empty tilde expansion will fail, i.e.,
it will remain as a literal tilde. This patch changes it to
return the empty string as required by POSIX.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:49 +0800
builtin: Mark more regular built-ins
This patch marks the following built-ins as regular, meaning that
they cannot be overriden using PATH search:
hash
pwd
type
ulimit
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:45 +0800
jobs: Replace some uses of fmtstr with stpcpy/stpncpy
Some uses of fmtstr, particularly the ones without a format string,
can be replaced with stpcpy or stpncpy. This patch does that so
we don't have to introduce unnecessary format strings in order to
silence compiler warnings.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:44 +0800
output: Fix fmtstr return value
The function fmtstr is meant to return the actual length of output
produced, rather than the untruncated length.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Sat, 19 May 2018 02:39:38 +0800
main: Only set savestatus in exitcmd
Currently exitcmd sets exitstatus and then savestatus if the latter
was previously set. In fact, as exitcmd always raises an exception
and will either end up in the setjmp call in main() or exitshell(),
where exitstatus is always replaced by savestatus if set, we only
need to set savestatus.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Upstream commit:
Date: Mon Oct 6 10:39:47 2014 +0800
[BUILTIN] Exit without arguments in a trap should use status outside traps
POSIX now requires that exit without arguments in a trap should
return the last command status prior to executing traps. This
patch implements this behaviour.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Keeping up with upstream (in our case, 'before patch' code is not buggy).
Upstream commit:
Date: Fri, 11 May 2018 23:41:25 +0800
parser: Fix incorrect eating of backslash newlines
With the introduction of synstack->syntax, a number of references
to the syntax variable was missed during the conversion. This
causes backslash newlines to be incorrectly removed in single
quote context.
This patch also combines these calls into a new helper function
pgetc_top.
Fixes: ab1cecb40478 ("parser: Add syntax stack for recursive...")
Reported-by: Leah Neukirchen <leah@vuxu.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
Adding previously skipped "readtoken1(pgetc_eatbnl(), DQSYNTAX..." changes
from upstream commit:
Date: Thu Mar 8 08:37:11 2018 +0100
Author: Harald van Dijk <harald@gigawatt.nl>
parser: use pgetc_eatbnl() in more places
dash has a pgetc_eatbnl function in parser.c which skips any
backslash-newline combinations. It's not used everywhere it could be.
There is also some duplicated backslash-newline handling elsewhere in
parser.c. Replace most of the calls to pgetc() with calls to
pgetc_eatbnl() and remove the duplicated backslash-newline handling.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
