Operators, their names and properties

[rljacobson]

I've been corresponding with @rocky about his work with the operators. I have a few comments that might be helpful to have on GitHub but that aren't really attached to a single issue or pull request. For background, I am the author of this collection of information about Wolfram Language.

A few comments that might be helpful.

• For convenient reference: Here's an HTML rendering of my operator database. It exists as an Apple Numbers spreadsheet that I am happy to send you. I also have it in Excel and CSV formats online. I see you have it in the Mathics codebase already.
• For reference: Part 1 and Part 2 of my article on finding all Mathematica operators. These articles contain a lot more detail about the various sources of information for Mathematica operators, their limitations, how and why some of the sources shouldn't be trusted, and so on.
• Most of my work documenting details of the language has been done using Mathematica versions 10 through 13, but mostly with version 11. Whenever a new version was released, I'd check that all my data was still correct. There are rarely any changes made to operator data between versions. If I recall correctly, the only changes I encountered are with the newly introduced operator for graph edges. Its semantics changed subtlely after it was introduced. It would be nice to have complete data specific to every major and minor version, but I just don't have access to it.
• The precedence of operators in Mathematica differs between the notebook frontend and the command line interface even within a single version of Mathematica. For example, Therefore and Because have their precedence swapped between notebook and command line.
• Some "operators" are not parsed as operators by Mathematica. That is, they are recognized prior to the stage most other operators are parsed. Off the top of my head, examples include the (pseudo) operators for constructing number literals, the "tick" operator for constructing context paths (namespaces), and the Out and Slot operators. (I seem to recall the Get/Put and ? operators are treated specially in some contexts as well, I suspect more for historical than technical reasons.)
• Mathematica's Precedence function is not to be trusted. It often returns 670 for operators it doesn't know about, but it's wrong in other ways, too. Other sources of data about operators have their own problems. The column "actual precedence" are numbers I assigned by just counting by 10s.

I determined operator information empirically, that is, by giving Mathematica code and observing how Mathematica interprets the code. This can be automated to some extent. I just uploaded a repository containing some of the code I have used for this project if you want to reproduce my results or, say, check results for different versions of Mathematica. In particular, you will find code in that repo that can automatically determine the relative precedence of two operators programmatically.

I no longer have a full Mathematica license, so right now I can only use the free Wolfram Engine to check things.

[rljacobson]

The names Mathematica gives to a few of the operators are problematic in a small number of cases. Some operators don't have official names (typically "pseudo operators"), but in some cases the name Mathematica gives is actually the name of a related function that has different semantics. Examples include what I call ParameterizedFunction (Mathematica calls this Function), FunctionApplyPostfix (Mathematica calls this Postfix), and SqrtBox (Mathematica calls this Sqrt). There appears to be more than one underlying reason why Mathematica gives "bad" names. To find all such instances, just look for operators for which "Unofficial Name" is different from "Name".

[rocky]

The names Mathematica gives to a few of the operators are problematic in a small number of cases. Some operators don't have official names (typically "pseudo operators"), but in some cases the name Mathematica gives is actually the name of a related function that has different semantics. Examples include what I call ParameterizedFunction (Mathematica calls this Function), FunctionApplyPostfix (Mathematica calls this Postfix), and SqrtBox (Mathematica calls this Sqrt). There appears to be more than one underlying reason why Mathematica gives "bad" names. To find all such instances, just look for operators for which "Unofficial Name" is different from "Name".

While I appreciate this information, if you have specific changes to named-characters.yml or operators.yml please put PRs and/or add some issues. Thanks.

[rocky]

Thanks for putting this all together. The amount of done and level of detail is amazing.

• For reference: Part 1

Some comments reading this. The first link in the body of the text "Generalizing PEMDAS: What is an operator?" is broken.

And the title of that article, "PEMDAS" is not a term I think most people know, although it is in wikipedia in an another category. And even here, there are other acronyms in use: BODMA, DOMDAS, BIDMAS and BEDMAS. Nowhere in that article with PEMDAS in the title is PEMDAS defined.

Also, under Data Sources, the alignment of the text between "Source" and "Description" doesn't quite line up in my browser. So I have a hard time knowing when one description ends and the next starts. A suggestion is to just turn these in to subsections or add some line space between entries.

Finally, when I read about WolframLanguageData, I was at a loss to know exactly how to use that because the necessary parameters are not specified. An example would be nice and I think I added that to the YAML documentation in mathics_scanner. However somewhere else in the article it is mentioned how to get the specific information from WolframLanguageData.

[rljacobson]

Thanks. I have started making issues to correct this long-standing bitrot in my blog:

• PEMDAS issue is PEMDAS is never defined in Part 1 of Wolfram Language article series, link broken rljacobson/rljacobson.github.io#1
• Table formatting issue is Table formatting is difficult to read rljacobson/rljacobson.github.io#2

As for WolframLanguageData, I link to its documentation, and I provide an example usage and output in a discussion of the function in the article:

In[1]:= WolframLanguageData["Power", "PrecedenceRanks"]

Out[1]= {{expr1\_expr2\%expr3, Power[Subscript[expr1,expr2],expr3]}->8,
        {expr1^expr2, Power[expr1,expr2]}->21,
        {expr1^expr2, Power[expr1,expr2]}->21,
        {expr1^expr2, Power[expr1,expr2]}->21,
        {Subsuperscript[expr1, expr2, expr3],
            Power[Subscript[expr1,expr2],expr3]}->21,
        {expr1\^expr2\%expr3, Power[Subscript[expr1,expr3],expr2]}->21,
        {\@expr\%n,Power[expr,Power[n,-1]]}->22}

I also show how to use WolframLanguageData to request everything in its database (which downloads 5,000+ database records):

In[1]:= Select[
            WolframLanguageData[
                WolframLanguageData["Entities"],
                {"Name", "PrecedenceRanks", "ShortNotations"}
            ],
            #[[2]] =!= Missing["NotApplicable"] &
        ]

Is there something else you are wanting?

[rocky]

... and Part 2 of my article on finding all Mathematica operators.

A comment on reading Part 2.

After "Two synonyms of the same operator:" there is some code that does not render properly as it does in Part 1. The article would read easier if this were fixed.

[rljacobson]

Created an issue: rljacobson/rljacobson.github.io#3. This same problem likely manifests elsewhere on my blog.

[rljacobson]

The difference between associativity "Non" and associativity "None".

I am afraid I have added to the confusion by stubbornly choosing to use the terms in nonstandard ways. In retrospect, I should have used the terms in the same way other authors use them. I use the terms in my operator tables and blog as follows.

Non Associative: An operator is non associative if the concept of associativity simply does not apply to the operator at all. This occurs primarily in the case of "parenthesizing" operators ("matchfix" operators) and operators that take no argument ("nullary" operators). For these operators, there is never any ambiguity about how to parse repeated/adjacent uses of the operator.

Note that there are a handful of operators that are neither matchfix nor nullary operators that are nontheless non associative under this definition. For example when Span/;; (a weird operator) appears adjacent to another instance of itself, it does not capture the other instance as an argument but instead is parsed as implicit multiplication, that is, ;; ;; is parsed as Power[Span[1, All], 2]. I also have Pattern and Optional operators (both represented by :) listed as non associative, but I wonder if that's an error. In any case, the example of Span demonstrates that non associativity in this sense is not necessarily automatic, that is, the parser author may have to make an explicit choice within the code of a parser to make an operator non associative (in this sense) rather than something else. One might mistakenly think it's automatic in all cases.

Associativity of None: An operator has an associativity of None if adjacent/repeated uses of the operator "chains" the arguments under a single function call. For example, a + b + c + d is parsed as Plus[a, b, c, d], not as nested calls to Plus.

Usage in the literature

As I said, I'm using these terms in nonstandard ways. There is a third case that is very common in other languages but that generally doesn't occur in Mathematica, namely when adjacent/repeated appearances of an operator is an error of syntax. In the literature, usually the phrase non associative means this third case: adjacent uses is a syntax error. The reason this doesn't happen in Mathematica is because Mathematica allows implicit multiplication, and so nonsensical instances of adjacent operators are just promoted to multiplication by default.

There is no single standard term in the literature for when the concept of associativity does not apply. While the phrases "parenthesizing" and "nullary" are common, they are specific cases of associativity being inapplicable and do not describe the general case.

As far as I know there is no standard term for what I refer to as an associativity of None. I have seen the terms chaining, admissive (the operator "admits" multiple operands), cumulative, and other terms to refer to this case. My current preference is chaining.

[rocky]

Thanks for the clarification!