permute function seems not correct

[JensLincke]

The permute benchmark function looks nearly like a recursive permutation function. While implementing it for lox, I wondered why it does not compute the correct permutation 6!. What does it compute?

"The number of permutations of n distinct objects is n factorial, usually written as n!"

Like this:

Starting permute benchmark ...
1 [1, 2, 3]
2 [1, 3, 2]
3 [2, 1, 3]
4 [2, 3, 1]
5 [3, 2, 1]
6 [3, 1, 2]
ERROR Benchmark failed with incorrect result
permute: iterations=1 average: 0.0us total: 0us

The correct permutation of 6! should be 720

class Permute < Benchmark {
    init() {
        super.init();
        this.count = 0;
        this.v = nil;
    }

    benchmark() {
        this.count = 0;
        this.v = [];
        var n = 6;
        for (var i = 0; i < n; i = i + 1) {
            this.v[i] = i + 1;  
        }
        this.permute(0, n);
        return this.count;
    }

    verifyResult(result) {
        return result == 720;
    }

    permute(i, n) {
        var j;
        if (i == n) {
            this.count = this.count + 1;
            // print "" + this.count + " " + this.v;
        } else {
            for (j = i; j < n; j = j + 1) {
                this.swap(i, j);          
                this.permute(i+1, n);
                this.swap(i, j);
            }
        }
    } 

    swap(i, j) {
        var tmp = this.v[i];
        this.v[i] = this.v[j];
        this.v[j] = tmp;
    }
}

[smarr]

Yeah, I fear changing that is a major change to the benchmark.

There are a couple of other rather specific choices in the original code I got from SOM: SOM-st/som-java@3a69207#diff-5241273eecb6c5e75da48d83df5ec1de46647780885655761b830e888ec9d486

1. The array is not initialized, which means nil is swapped. This is a bit pointless, but makes it a good test for the performance of storage strategies.

2. The count is what it is. It's not really counting the number of permutations, but rather the number of invocations of permute.

While I understand the desire to be able to understand and explain things, changing it would mean a quite substantial change to the benchmark, and what it is measuring. And I have no idea what the impact on the various language implementations is.

I am happy to consider a PR, that changes the benchmark and checks what the impact on all languages is, but I would expect that it's not really worth the effort...