Subject: dash: test with multiple conditions cannot handle variable whose expanded value is same as an operator
Date: Thu, 05 Jan 2017 00:53:57 +0100
Package: dash
Version: 0.5.7-4+b1
Severity: normal
In Bourne shell script with dash, using either [ ] or test,
test -n "${variable}" && echo "variable is set to -->${variable}<--"
works as expected even if the value of variable is to a value the same
as a test operator eg -ne or -nt
variable="-ne" ; test -n "${variable}" && echo "variable is set to -->${variable}<--"
variable is set to -->-ne<--
However the addition of another test condition with -a or -o results in an
unexpected behavior in that the expanded variable is now treated as an operator
and not just as the string as should be the case.
variable="-ne" ; test -n "${variable}" -a -n "${DISPLAY}" && echo "variable is set to -->${variable}<--"
dash: 1: test: Illegal number: -n
Applying quoted parentheses makes no difference.
variable="-ne" ; test \( -n "${variable}" \) -a \( -n "${DISPLAY}" \) && echo "variable is set to -->${variable}<--"
dash: 2: test: Illegal number: -n
Similarly bad behavior is seen with a variable set to "-nt", which is how I stumbled upon this problem.
variable="-nt" ; test \( -n "${variable}" \) -a \( -n "${DISPLAY}" \) && echo "variable is set to -->${variable}<--"
dash: 3: test: closing paren expected
So if the test is consists of a single condtion, the expanded variable does not cause
problems with the test, and should not affect the test if it is part of a multiple condition test.
Incidentally bash also has this problem.
So the short term kludge in a shell script is to do
if [ -n "${variable_with_problem_value}" ]
then
if [ \( other_condition1 \) -a \( other_condition2 \) ]
then
...
-- System Information:
Debian Release: 8.6
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: armel (armv5tel)
Kernel: Linux 3.16.0-4-kirkwood
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=ANSI_X3.4-1968) (ignored: LC_ALL set to C)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages dash depends on:
ii debianutils 4.4+b1
ii dpkg 1.17.27
ii libc6 2.19-18+deb8u6
dash recommends no packages.
dash suggests no packages.
-- debconf information:
* dash/sh: true
Control: tags -1 + upstream wontfix
Hi!
On Thu, Jan 05, 2017 at 12:53:57AM +0100, J G Miller wrote:
> In Bourne shell script with dash, using either [ ] or test,
> test -n "${variable}" && echo "variable is set to -->${variable}<--"
> works as expected even if the value of variable is to a value the same
> as a test operator eg -ne or -nt
> variable="-ne" ; test -n "${variable}" && echo "variable is set to -->${variable}<--"
> variable is set to -->-ne<--
Yes, the argument after -n is literal. As-expected so far.
> However the addition of another test condition with -a or -o results in an
> unexpected behavior in that the expanded variable is now treated as an operator
> and not just as the string as should be the case.
> variable="-ne" ; test -n "${variable}" -a -n "${DISPLAY}" && echo "variable is set to -->${variable}<--"
> dash: 1: test: Illegal number: -n
After expansion, this is test -n -ne -a -n :0.
Notably, -a, -o, (, and ) are all killed in POSIX Issue 8 (Draft 2.1),
via Austin Group Defect 1330:
https://www.austingroupbugs.net/view.php?id=1330
They were marked obsolescent and optional in Issue 7 and earlier
(because they're awful and underspecified, you should never use them).
There are a few ways of parsing this:
if you're using a full-blown per-spec precedence-obeying parser,
like voreutils test or ksh, then you'll parse it like
( -n '-ne' ) && ( -n ':0' )
which resolves to true:
$ ~/code/voreutils/out/cmd/test -n -ne -a -n :0; echo $?
0
$ ksh -c 'test -n -ne -a -n :0; echo $?'
0
(well, idk if ksh uses a parse tree, but it produces the same result
that voreutils test does, and that does use one).
If you parse it in compatibility mode, like dash, bash,
and GNU coreutils test, then you'll parse it like
( '-n' -ne '-a' ) {garbage}
which is illegal:
$ bash -c 'test -n -ne -a -n :0'
bash: line 1: test: -n: integer expression expected
$ test -n -ne -a -n :0
./dash: 1: test: Illegal number: -n
$ /bin/test -n -ne -a -n :0
/bin/test: invalid integer ‘-n’
(if you make the -ne arguments integers the garbage -n :0 makes it
illegal anyway, and all of the above agree).
> Applying quoted parentheses makes no difference.
> variable="-ne" ; test \( -n "${variable}" \) -a \( -n "${DISPLAY}" \) && echo "variable is set to -->${variable}<--"
> dash: 2: test: Illegal number: -n
This is test ( -n -ne ) -a ( -n :0 ).
This is a much more interesting case: the obvious parse here is, indeed,
( -n '-ne' ) && ( -n ':0' )
and GNU coreutils test, voreutils test, and ksh are in agreement now:
$ /bin/test '(' -n -ne ')' -a '(' -n :0 ')'; echo $?
0
$ ~/code/voreutils/out/cmd/test '(' -n -ne ')' -a '(' -n :0 ')'; echo
$?
0
$ ksh -c "test '(' -n -ne ')' -a '(' -n :0 ')'; echo \$?"
0
Nevertheless, bash and dash still parse this differently:
$ test '(' -n -ne ')' -a '(' -n :0 ')'; echo $?
./dash: 9: test: Illegal number: -n
2
$ bash -c "test '(' -n -ne ')' -a '(' -n :0 ')'; echo \$?"
bash: line 1: test: -n: integer expression expected
2
by substituting the first -n for 0, then the first ) for 0,
then adding a closing paren, yielding:
$ test '(' 0 -ne ')' -a '(' -n :0 ')'; echo $?
./dash: 16: test: Illegal number: )
2
$ bash -c "test '(' 0 -ne ')' -a '(' -n :0 ')'; echo \$?"
bash: line 1: test: ): integer expression expected
2
$ test '(' 0 -ne 0 -a '(' -n :0 ')'; echo $?
./dash: 18: test: closing paren expected
2
$ bash -c "test '(' 0 -ne 0 -a '(' -n :0 ')'; echo \$?"
bash: line 1: test: `)' expected
2
$ test '(' 0 -ne 0 -a '(' -n :0 ')' ')'; echo $?
1
$ bash -c "test '(' 0 -ne 0 -a '(' -n :0 ')' ')'; echo \$?"
1
we can see that they parse it as
( ( '-n' -ne ')' ) && ( -n ':0' )
which is unbalanced and puts -n and ) in integer spots.
> Similarly bad behavior is seen with a variable set to "-nt", which is how I stumbled upon this problem.
> variable="-nt" ; test \( -n "${variable}" \) -a \( -n "${DISPLAY}" \) && echo "variable is set to -->${variable}<--"
> dash: 3: test: closing paren expected
Similarly, this is test ( -n -nt ) -a ( -n :0 ), which, again,
would make most sense to be
( -n '-nt' ) -a ( -n ':0' )
and, similarly, the same suspects agree:
$ /bin/test '(' -n -nt ')' -a '(' -n :0 ')'; echo $?
0
$ ~/code/voreutils/out/cmd/test '(' -n -nt ')' -a '(' -n :0 ')'; echo
$?
0
$ ksh -c "test '(' -n -nt ')' -a '(' -n :0 ')'; echo \$?"
0
and disagree:
$ test '(' -n -nt ')' -a '(' -n :0 ')'; echo $?
./dash: 26: test: closing paren expected
2
$ bash -c "test '(' -n -nt ')' -a '(' -n :0 ')'; echo \$?"
bash: line 1: test: `)' expected
2
and through the same reduction process:
$ test '(' -n -nt ')' -a '(' -n :0 ')' ')'; echo $?
1
$ bash -c "test '(' -n -nt ')' -a '(' -n :0 ')' ')'; echo \$?"
1
we arrive at the same parse:
( ( '-n' -nt ')' ) && ( -n ':0' )
which is similarly unbalanced,
and strace confirms they both stat 0 and ).
> So the short term kludge in a shell script is to do
> if [ -n "${variable_with_problem_value}" ]
> then
> if [ \( other_condition1 \) -a \( other_condition2 \) ]
> then
> ...
The correct solution is
if [ -n "$variable" ] && [ cond ] && [ cond ]
then
which is, since Issue 8, the only solution, and before Issue 7,
the only one that wasn't a damned underspecified mine-field.
IOW Just Don't Put Multiple Conditions In Test(1);
quoth Issue 7, XCU, test, APPLICATION USAGE:
The XSI extensions specifying the -a and -o binary primaries and the
'(' and ')' operators have been marked obsolescent. (Many expressions
using them are ambiguously defined by the grammar depending on the
specific expressions being evaluated.) Scripts using these expressions
should be converted to the forms given below. Even though many
implementations will continue to support these obsolescent forms,
scripts should be extremely careful when dealing with user-supplied
input that could be confused with these and other primaries and
operators. Unless the application developer knows all the cases that
produce input to the script, invocations like:
test "$1" -a "$2"
should be written as:
test "$1" && test "$2"
to avoid problems if a user supplied values such as $1 set to '!' and
$2 set to the null string. That is, in cases where maximal portability
is of concern, replace:
test expr1 -a expr2
with:
test expr1 && test expr2
and replace:
test expr1 -o expr2
with:
test expr1 || test expr2
but note that, in test, -a has higher precedence than -o while "&&"
and "||" have equal precedence in the shell.
Parentheses or braces can be used in the shell command language to
effect grouping.
Parentheses must be escaped when using sh; for example:
test \( expr1 -a expr2 \) -o expr3
This command is not always portable even on XSI-conformant systems
depending on the expressions specified by expr1, expr2, and expr3. The
following form can be used instead:
( test expr1 && test expr2 ) || test expr3
Best,
наб
Source: dash
Source-Version: 0.5.12-1
Done: Andrej Shadura <[email protected]>
We believe that the bug you reported is fixed in the latest version of
dash, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to [email protected],
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Andrej Shadura <[email protected]> (supplier of updated dash package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing [email protected])
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Format: 1.8
Date: Thu, 05 Jan 2023 14:06:02 +0100
Source: dash
Architecture: source
Version: 0.5.12-1
Distribution: unstable
Urgency: medium
Maintainer: Andrej Shadura <[email protected]>
Changed-By: Andrej Shadura <[email protected]>
Closes: 55860781982985020297532610175311024635
Changes:
dash (0.5.12-1) unstable; urgency=medium
.
* New upstream release (Closes: #1017531, #1024635).
* Refresh patches.
* Apply upstream patches for hash, ulimit and manpages
(Closes: #558607, #819829, #850202, #975326).
Checksums-Sha1:
97cd1d5275fe8d1f81c036d7c998133674c07f25 1520 dash_0.5.12-1.dsc
e15444a93853f693774df003f87d9040ab600a5e 246054 dash_0.5.12.orig.tar.gz
1dd0eee42af5fd7ead7de85a79a9a8699bc5bc13 38520 dash_0.5.12-1.debian.tar.xz
Checksums-Sha256:
0861850b279fdcd8ff6740e4813896f2ad20365b5995ef175e7d18f6122633c1 1520 dash_0.5.12-1.dsc
6a474ac46e8b0b32916c4c60df694c82058d3297d8b385b74508030ca4a8f28a 246054 dash_0.5.12.orig.tar.gz
b022cf6c89c9312fe57e9a767390f022b4dbedb5f6541c06f6a8b20187a3da61 38520 dash_0.5.12-1.debian.tar.xz
Files:
0f4256c25db24c637966edb041f35fac 1520 shells optional dash_0.5.12-1.dsc
57222b768b84003ea4b801e5d5e0e52b 246054 shells optional dash_0.5.12.orig.tar.gz
4b932e195cf79623b4895b3f9a3ae0ff 38520 shells optional dash_0.5.12-1.debian.tar.xz
-----BEGIN PGP SIGNATURE-----
iHUEARYIAB0WIQSD3NF/RLIsyDZW7aHoRGtKyMdyYQUCY7bLcgAKCRDoRGtKyMdy
Ybj4AQDGyhSiHNJ+NT4Tq3TN1OImwvvVhfbyPaaUpOiSq19v3AD/YdpLQHB6Ju7W
/UrNXSUeUsP4ijKpbWe4JYF37hn18Aw=
=uLUG
-----END PGP SIGNATURE-----
Debbugs is free software and licensed under the terms of the GNU General
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.