Re: left bit shift (<<) operator now demands line continuation

[Chet Ramey]

On 1/27/10 2:34 PM, jwm@horde.net wrote:

> Bash Version: 4.0
> Patch Level: 33
> Release Status: release
> 
> Description:
> bash 4.0 fails to parse a function that's parsed successfully by dash(1) as
> well as older versions of bash (3.1, 3.2):
> 
> [jwm@boost:pts/6 ~> cat inet
> swap32_posix()
> {
>       local funcname=swap32_posix
>       local arg
>       for arg in "$@"; do
>               echo $((
>                       ($arg & 4278190080) >> 24 |
>                       ($arg & 16711680) >> 8 |
>                       ($arg & 65280) << 8 |
>                       ($arg & 255) << 24
>               ))
>       done
> }
> [jwm@boost:pts/6 ~> bash -n inet
> inet: line 6: unexpected EOF while looking for matching `)'
> inet: line 14: syntax error: unexpected end of file
> [jwm@boost:pts/6 ~> dash -n inet
> [jwm@boost:pts/6 ~> 

This is a tricky one.  Before bash-4.0, bash "parsed" $(...) command
substitutions by counting parens, making sure they matched, and hoping
for the best.  This was not Posix conformant and the source of a number
of bug reports -- Posix says that the shell should be able to handle
"any valid shell script" regardless of the number of unmatched parens
when parsing a command substitution -- but more or less worked.

Beginning with bash-4.0, I changed bash to be more rigorous about parsing
the contents of $(...), for Posix conformance and to eliminate the need
for users to resort to things like (pattern-list) in a case statement.  I
had to make a choice at that point: should command substitution or
arithmetic expansion take precedence?  It's an issue because Posix also
says that the shell needs to handle nested subshells ( $((a);(b)) )
without prioritizing.  I chose to have the contents of $(...) parsed as
a command to support the "any valid shell script" part of the standard.
That works pretty well.

It fails when you have tokens composed of the same characters but with
different meanings in a command and expression context.  The `<<' in your
example is one of those.  (In fact, if you replace the << with <, your
arithmetic expansion is a valid command as well as a valid expression.)

After a lot of discussion last month, the Posix working group acknowledged
that we have to change the standard, and it looks like we will choose to
specify that $(( always introduces an arithmetic expansion, as long as the
$((...)) encloses a valid arithmetic expression.  I will probably change
bash-4.2 to implement that regardless of the state of the standard at that
point.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU    chet@case.edu    http://cnswww.cns.cwru.edu/~chet/