David Cornelius discussion

[gcarreno]

Hey David(@corneliusdavid),

As mentioned on the Discords, was able to get a Linux 64b compiled and scp over to the Linux host.
Did not run the full automation on it due to having the Windows VM open and it would not be fair to your entry to be tested on that of a busy system.

Will do that once I close it and then will post results here.

Cheers,
Gus

[gcarreno]

Hey David(@corneliusdavid),

Houston we have a slight problem:

$ ./test_all.sh dcornelius
******** Test ********

===== David Cornelius ======
5c027842e7e4a5c121184450a4cf7372f989ffd8675368c3c46e2037bfc938a3  /home/gcarreno/Programming/1brc-ObjectPascal/results/dcornelius.output
db3d79d31b50daa8c03a1e4f2025029cb137f9971aa04129d8bca004795ae524  Official Output Hash
===========

There's an archived file containing the baseline output on the data folder.

I think something is off with the alphabetic sorting.
But it's just my very naive opinion.

Cheers,
Gus
P.S.: Nonetheless I'll post a cold start time ./bin/dcornelius... for your reference.

[corneliusdavid]

I had noticed a sorting difference between the different techniques as I compared the results of small data sets during my testing but I didn't take the time to figure out which was right. I'm not sure what the problem is as the final sort is done with TStringList.Sort, which I think was used by another entry. I don't know if Unicode characters are not handled properly or if there's a difference in how Delphi's TStringList differs from the one implemented in Lazarus.

[synopse]

I would say that TStringList is using binary UTF-16 sorting of 16-bit WideChar on Delphi, whereas Lazarus sorts with binary UTF-8 content of AnsiChar.

[gcarreno]

The baseline is using a custom compare function on the TStringList.
That method has been adopted by many since GetMem put it on the baseline and on his own entry.
I think that custom compare function is something to look at and understand how it moulds the results.

[corneliusdavid]

Yes, I will as I have time. Thanks!

[gcarreno]

More than welcome!!

[gcarreno]

Hey David(@corneliusdavid),

As promised:

$ time ./bin/dcornelius /tmp/measurements-1_000_000_000.txt dic
{...}
real	6m55.244s
user	6m53.693s
sys	0m1.460s

Cheers,
Gus

[gcarreno]

Hey David(@corneliusdavid),

While messing around with your entry, mainly to get it to use Baseline.Common, I noticed 2 things:

1. Your alphabetic ordering is different from everyone else.
2. The mean of negative numbers are in the hundreds, while the positive ones are fine,

I'm not sure if this was the case before I made your code use Baseline.Common, but even having a better look, I can't suss out what's happening.

There's only one place where you call the rounding and it looks good. I'm quite baffled.

Cheers,
Gus

[corneliusdavid]

Rounding: While I was out of town, I changed the rounding routine in my code to match the updated one in the README (using ceil) but had not merged it yet. I've reverted my code as I'd rather not use external units (e.g. Baseline.Common).

Sorting: I studied Brian Fire's code and found out TStringList uses ANSI sorting; setting UseLocale := False turns that off. So, I've made that change to the TStringList technique. For the TDictionary technique, I'm using TArray.Sort with a TStringComparer. I might have to study that further.

[gcarreno]

I've reverted my code as I'd rather not use external units (e.g. Baseline.Common).

No problem. It's there if one opts for it, but it's not mandatory by no means!!

Sorting:...

Awesome! I'll be awaiting your changes in the future!!

[gcarreno]

Hey David(@corneliusdavid),

WIth the work of Georges(@georges-hatem) we came to the conclusion that the overloading of the Match.Ceil in Delphi breaks when used like this:

function TChallengeCommon.PascalRound(const x: Double): Double;
begin
  Result := Ceil(x * 10) / 10;
end;

Arnaud suggested that it could be mitigated with:

function TChallengeCommon.PascalRound(const x: Double): Double;
begin
  x:= x * 10;
  Result := Ceil(x) / 10;
end;

He also suggested a new Ceil function. I've put that code in the Baseline.Common.pas file.
Have a look in there, before you start pulling your hair out like Georges did. 😄

Cheers,
Gus

[gcarreno]

Hey David(@corneliusdavid),

I've just ran you entry through the testing of the hash automation and it's doing good.
This, of course, before applying the current pending PR.
Thought you'd know!!

Cheers,
Gus

[corneliusdavid]

What do you mean by "it's doing good"? Did the generated hash match or not?
You had run a hash against this previously and it did not match--why would it be different now?

[gcarreno]

I'm confused myself.
I ran the test automation and here's the result:

$ ./test_all.sh dcornelius
******** Test ********

===== David Cornelius ======
4256d19d3e134d79cc6f160d428a1d859ce961167bd01ca528daca8705163910  /home/gcarreno/Programming/1brc-ObjectPascal/results/dcornelius.output
4256d19d3e134d79cc6f160d428a1d859ce961167bd01ca528daca8705163910  Official Output Hash

And this is the version that is in the repo now, without the pending PR.

I must've done something bad. Like forget to compile and scp it over.
It's totally my fault, and I apologise for this profusely!!

Not sure what this new PR is all about, since I'm too tired to even go look, but your hash is now matching.

[corneliusdavid]

Well, that's good news because I was going to give up if this new PR didn't work. Thanks!
Please reject the PR--I don't want to take the chance that I break it again!! (Of course, I could always revert the commit.)

[gcarreno]

Closing PR !!