Re: $((expr)) allows the hexadecimal constant "0x"

[Chet Ramey]

On 1/9/24 11:23 PM, Martin D Kealey wrote:
> On Tue, 12 Dec 2023, 05:56 Zachary Santer, <zsanter@gmail.com 
> <mailto:zsanter@gmail.com>> wrote:
>
>     On Mon, Dec 11, 2023 at 9:26 AM Chet Ramey <chet.ramey@case.edu
>     <mailto:chet.ramey@case.edu>> wrote:
>      > Part of the issue here is that distros -- Red Hat, at least -- never
>      > upgrade the version of bash they started with. Red Hat will `support'
>      > bash-4.2 as long as they support RHEL 7.
>     To be fair to the Red Hat people, "my scripts used to work and now they
>     don't" is the issue they're trying to avoid, at least within a release.
>
>
> Which is exactly my point about these random changes "because we think it 
> must be a mistake".

Random is in the eye of the beholder, isn't it? So far everything we've
talked about here is the result of a bug report.

> There are users out there that depend on these supposedly-broken features.

That's an excellent rationale for never fixing any bugs -- because someone
might rely on the incorrect behavior -- but that's just not how it's going
to happen.

> It is disingenuous to justify a change on the grounds of "alignment with 
> the documentation". The manual is not a specification (much less a formal 
> one), and leaves out a great many things that would be required in the 
> latter.

You might want to ask yourself why the documentation gets updated. In this
case, it was to clarify the intent of the original implementation: that
integer constants cannot contain `-', because that is an operator. The 
documentation can change to align with the implementation, so it should be
possible to point to the documentation and say the implementation should
change to align with it, once you figure out the intent.

In any event, once something is in the manual, why should users not be able
to count on it? Consider the opposite case as well: if the man page says
something (for instance, that [base#]n constants require something non-
empty following the optional base#), why assume that something else will
work? It might, for a while, but it's not guaranteed.

If you don't want to rely on the manual, how about relying on POSIX? The
manual doesn't need to be 300 pages long, but the standard is already
there.

> There are /many/ things that are "like" something else, but with 
> extensions that are unexplained and often completely unmentioned.

If you think they need clarifying, report them.

> Moreover, 
> the manual is huge, and not a tutorial, so most users only consult it when 
> they already have a problem. (And those of us who do routinely consult it 
> will perform tests when the manual is unclear.)

So it's too big, but not big enough to specify everything. If the manual
is unclear, or diverges from the implementation, it's worth asking a
question about the intent, instead of making assumptions that may not be
sustainable.

> The source of truth is the implementation, not the manual, and it is quite 
> offensive to imply that users are at fault when they don't assume the 
> reverse.

That assumes, of course, that the implementation is perfect and bug-free,
and is encased in amber and can never change. We know that's not the case.

It's pretty chauvinistic to assume that people who want to read the manual
when they have a question and figure out what something does are the ones
at fault, too. Why are they supposed to assume that the manual doesn't
represent the intent? It's possible there's a problem in the
implementation, right? Should they not be allowed to ask? It's just as
offensive for you to imply they're at fault because they do assume the
reverse of what you assert above.

> We know there are errors and gaps in the documentation, and so do 
> the users.

There are errors and gaps in both the documentation and the code.

Sometimes the documentation doesn't reflect the intent, sometimes the code
doesn't. It's an imperfect world.


> Simply disabling existing behaviours breaks things for people who use them, 
> and doesn't actually "fix" scripts that are already broken. (Those scripts 
> may get a better explanation of why they're wrong, but they don't magically 
> start working as intended by their authors.)

What `existing behavior' has changed as the result of this discussion?

> It's backwards to suggest that bare "0x" should be prohibited because the 
> man page says so.

Why? If the man page says something, why should users not be able to count
on what it says? It shouldn't matter that the language was added to the
man page as the result a different, but similar, issue.

If the man page says that integer constants are like C, then people
shouldn't expect things that aren't valid C to work, even if they work
because the bash implementation is too permissive.

> The words “Integer constants follow the C language 
> definition, without suffixes or character constants” were not added to the 
> manual until June 2019 in the Devel branch (as commit 
> 48492ffae22d692594757e53fb4580ebb1f506cf), and did not land in the Master 
> branch until December 2020 (as commit 
> 8868edaf2250e09c4e9a1c75ffe3274f28f38581).

In this case, the documentation got updated to reflect the implementation,
because bash had not ever allowed integer constants to contain an operator,
and that was the substance of the bug report that prompted the discussion.
But now it's there.

(Yes, I know you've said that discussion wasn't good enough because there
were too few participants.)


> Ubuntu LTS users would then have 
> first had a chance to read them in April 2022.

That seems like an Ubuntu problem. I understand they have to lag -- the
aforementioned RHEL 7 issue is just the worst example of vendors shipping
old releases -- but 16 months seems like a pretty big gap. I don't use
Ubuntu, but I figure they make releases more often than that.


> It is not reasonable to expect users to notice that this one line had been 
> inserted into the manual page but without any actual change of behaviour, 
> and then to be told 5 years later that NOW our scripts are wrong because 
> they don't match the man page.

They aren't valid C integer constants, that's for sure. Bash just accepts
them because it's too permissive.

> If behaviour is going to change, it should be announced in advance, as a 
> preface to the change log. Then at least users will have a chance to find 
> out about it before it bites them, and perhaps provide feedback that such a 
> proposed change would be undesirable.

The change in behavior -- and keep in mind that there's no behavior change
here -- is generally part of the bug discussion.

> So please:
> * EITHER change the man page to match the behaviour (I suggest removing the 
> words «without suffixes or character constants» and substituting «except 
> that no digits are required after any prefix, and type modifier suffix 
> characters ('L', 'S', & 'U') are not accepted. Unlike C, Bash does not 
> recognize quoted characters as integer constants».

It's probably better to say that, aside from the [base#]number syntax,
bash implements decimal, octal, and hex constants as specified in the C
standard, with the single exception that bash tries to be helpful and not
require any hex digits after a 0x prefix, instead of trying to list all the 
exceptions.

> * OR add a "Proposed Future Changes" section to the top of CHANGES, with an 
> entry explaining this change and its rationale, AND delay actually 
> implementing the change until after the announcement makes its way into 
> common LTS channels (let's say, April of even-numbered years).

Ubuntu's not paying me anything, and neither is anyone else. That's part of
the cost of being a volunteer. One of the benefits is that I get to ration
my own capacity, so to speak, and decide which feature requests and changes
to implement, and do that work when I have the capacity. If someone wants
to fund me to do that extra work, or delay implementing new features until
I may not have time to do it, or change releases to accommodate vendor
schedules, I'll listen.

Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/