Creation of arrays with negative lengths?

[jerviscui]

runtime/src/libraries/System.Private.CoreLib/src/System/Threading/ThreadPoolWorkQueue.cs

Line 138 in 2b60d82

var newArray = new object?[m_array.Length << 1];

I have a problem where a continuous left shift goes negative, and the code throws an exception.
Am I missing something?

Thank you everyone.

Successive left shifts in this code will cause the length to become negative, throwing an exception. But there is no defensive code here.

My question is, will this never happen? Hidden logic in the CLR or otherwise?

[En3Tho]

How do you even end up having 2 billion work items in the first place?

Are you just pushing more and more items in an endless loop or something?

Expanding an array more and more won't help you as it is a clear indication of a bug in your code

[huoyaoyuan]

Trying to create array of negative length will throw an exception. It's a safe behavior for handling exceptional cases.

[huoyaoyuan]

How do you even end up having 2 billion work items in the first place?

Security-wise, it's still important to get exceptionally large content correctly rejected, instead of causing security issues like buffer overrun.

[EgorBo]

@jerviscui is the question basically "how can m_array.Length << 1 produce a negative value"? if so, just run this:

byte[] arr = new byte[1_100_000_000];
Console.WriteLine(arr.Length << 1);

[sfiruch]

var newArray = new object?[m_array.Length << 1];
I have a problem where a continuous left shift goes negative, and the code throws an exception. Am I missing something?

m_array.Length is a signed integer. The most-significant bit of a signed integer specifies whether the integer represents a positive or negative number.

If you start with a very large signed integer, which has the second-most-significant-bit set, the left-shift will shift that bit into the most-significant position. This then represents a negative number, because the most-significant bit is set.

This is expected behavior. I don't think it's an issue/bug in .NET.

[jerviscui]

How do you even end up having 2 billion work items in the first place?

Security-wise, it's still important to get exceptionally large content correctly rejected, instead of causing security issues like buffer overrun.

Yes, I agree.
I was wondering if there is hidden logic in the CLR to prevent this.

[jerviscui]

@EgorBo I have re-described problem. Thanks.

[EgorBo]

@EgorBo I have re-described problem. Thanks.

My question is, will this never happen? Hidden logic in the CLR or otherwise?

Well it may happen if you push some unrealistic amount of work item to the pool I guess. We may change the logic to be aligned what List's Grow does, namely

if ((uint)newCapacity > Array.MaxLength) newCapacity = Array.MaxLength;

but it won't protect you from an exception anyway if you're just stress-testing it

[sfiruch]

Security-wise, it's still important to get exceptionally large content correctly rejected, instead of causing security issues like buffer overrun.

Yes, I agree. I was wondering if there is hidden logic in the CLR to prevent this.

What would you expect to happen when there's no more space in the queue?

You probably shouldn't have billions of entries in a ThreadPoolWorkQueue anyway. I think raising an exception is reasonable for such an exceptional case. You pobably try to divide the work too fine-grained. Parallel and .AsParallel() are easy-to-use altternatives that do not suffer from this problem.

If the implementation of ThreadPoolWorkQueue used extreme measures to support an arbitrary number of entries, i.e. more than int.MaxValue, you'll eventually run out of memory, which will rrsult in an exception.

Either way, you get an exception when you try to put billions of elements into queues. You shouldn't do this. The only quesition is perhaps, which Exception you should get. I think the ThreadPoolWorlQueue could produce a more descriptive exception.