Optimize merge_sort algorithm for largest data sizes

[SergeyKopienko]

In this PR we extends the approach from #1933 to merge_sort algorithm.

[danhoeflinger]

Not a full review, but enough feedback to group together. I still need time to digest the algorithm, and look into details

[danhoeflinger]

I think more explanation of these magic numbers is warranted. Is it possible to consolidate this magic number with the one in tune_amount_of_base_diagonals?
It seems like it should be equivalent to something like __base_diag_count = bit_floor(ceil(__nsorted/8192)).

[SergeyKopienko]

The parameter __amount_of_base_diagonals has been removed from tune_amount_of_base_diagonals function.
In other parts I propose to save the current state as is.
About __base_diag_count constant: we have this value in two submitters: for merge' and for merge_sort` algorithms.

[danhoeflinger]

So the formula is

base_diags = ceil_div( 32 * 1024, bitceil( ceil_div(256 * 1024 * 1024, __n_sorted) )

We have two magic numbers here, and I don't know why either are chosen. Do they represent some characteristics of target hardware?

From the calculation, in practice, it seems like we are effectively targeting some "reasonable" size between base diagonals, while keeping the number of base diagonals a power of two. This makes some sense from a performance perspective in that for target hardware, there will likely be some fixed distance between base diagonals which optimizes the second kernel's performance. This is all speculation though, since I haven't done any tuning here.

A simpler function which should be similar in effect and the magic number would have more meaning (to me) would be something like this:

__optimal_dist_between_base_diags = 8 * 1024;
__base_diag_count = bit_floor(__n_sorted / __optimal_dist_between_base_diags);

https://godbolt.org/z/TjPMMG44Y

They only differ once __n_sorted grows larger than your 256M magic number, and so the ceiling division does more work to round up to 1.

If the goal is to keep the distance between diagonals the same, then the simpler function does a better job of it (extending to higher numbers). However, this is just my speculation about why this is function is successful at tuning the diagonal count. Please check this with your understanding and the performance of the code (and perhaps try different values of __optimal_dist_between_base_diags if you have time). I also don't know if there is a reason to keep a power of 2 number of diagonals, but you may have good reason to do that.

[danhoeflinger]

Does it make sense that the starting size limit is the same for "merge" as it is for segments of merge_sort?
I would expect that it would be a somewhat lower threshold for merge sort, since we can combine calculations of base diagonals across multiple segments of merge within merge sort in a single kernel.

Perhaps it has more to do with the size between base segments than justifying the diagonal kernel, in which case it perhaps makes sense to have the same threshold. I think a discussion within a tech meeting would be helpful to consider this stuff.

[mmichel11]

Can __dpl_ceiling_div be used here instead of just std::ceil which involves conversion to floating point type?