.NET stack memory limit

.net core stack size
memory allocation in c#
c# stackalloc
c# get stack size
c# set stack limit
.net heap
visual studio 2017 increase stack size c++
visual studio stack reserve size

I am using C#, .NET 4.0, 64-bit. I need to store in memory 500 million "data points" that are used in computations. I need to decide whether to create these as struct or class objects. Structs seem so much faster.

Is there a memory limit for the stack? If so, how can it be adjusted.

Will storing so much data on a stack affect the overall performance of the system?

(By the way, I am aware of the single-object size limitation in .NET, so that's being addressed -- the data will be stored in multiple collections).


You're asking the wrong question. If stack size matters, you're doing something wrong.

If you use many datapoints, you'll put them in a collection, such as an array. Arrays are always allocated on then heap. An array of structs embeds the individual structs and forms a continuous memory block. (If you have more than 2GB, you need several arrays).

Whereas with reference types, the array will only contain the references, and the objects get allocated individually on the heap. A heap allocation has about 16 bytes of overhead, the reference in the array accounts for another 8. You'll also get worse cache locality due to the indirections, and the GC has to do more work, to crawl all those references.

My conclusion is that if you have many small datapoints, make them a struct, and put them in an array.

Stack vs Heap. What's the difference and why should I care?, Why is the limit on the stack size smaller than the amount of memory actually available to the stack? There are huge files about 100Mb. I want to load them into memory (RAM), process and save somewhere. At the same time I want that a limit of memory usage exists. Example, 100Mb, to my app don't use more then this memory limit. If the limit is exceeded the file is processed parts. My understanding of this:


You are going to store your data in arrays and arrays are always stored on the heap. So it doesn't matter whether or not you use structs or classes to hold those arrays. You may well want to make sure that your data points are value types (i.e. structs) so that arrays of data points can be allocated efficiently in contiguous blocks of memory.

Performance differences between heap and stack allocated memory are most likely to be seen with small objects that are allocated and deallocated in a short space of time. For long-lived objects of the size you describe, I would expect there to be no difference in performance between stack and heap allocated memory.

General: How do I change my default limits for stack size, core file , of the reserved virtual address space, the fewer threads you can create. Dealing with this at present - the .net core binary is not obeying memory limits of the machine it is on. WEther this is because K8S is not communicating the memory limit, or because of a bug in .net core, this answer sadly is not valid. – gburton Nov 28 '19 at 15:50


You could use classes for your data points. In this case, the memory will be allocated on the heap.

But considering that you are talking about 500 million data points, and especially since you are programming in the .NET world with a more restricted memory limit for apps, I would strongly encourage using some kind of embedded database, like sqlite, for example. In this way, you would avoid having all of your data points in memory simultaneously, but only the ones you need for computation now.

why is stack memory size so limited?, In .NET 2.0 and newer you can simply specify thread size in a Where reserve is the maximum memory to allocate for stack the commit value  I have a .Net application running on a 32 bit box. The application is a windows service. It consistently hovers around 600-800 MB range. Is this a problem. If an application crosses 1 GB, is it a m


It's surprising that no one seemed to try to answer the actual question.

I absolutely understand that this is the wrong question to ask 99.9% of the time, but it would still be interesting to know the results (at least I was curious).

The test program

It is really simple using unsafe code and the stackalloc keyword.

class Program
{
    static void Main(string[] args)
    {
        for (int i = 100; i < Int32.MaxValue; i+=10)
        {
            StackCheck(i);
            Console.WriteLine($"Successfully allocated {i} bytes on the stack");
        }
    }

    public static unsafe void StackCheck(int size)
    {
        byte* array = stackalloc byte[size];
    }
}
Results

Mind that this is 100% implementation detail and may differ by CLR, CLR version, operating system or individual machine. In my experiment both the full .NET Framework 4.7.2 and .NET Core 2.1.4 crashed just above the 1MB mark. Interestingly it is not even consistent between runs, the results fluctuate by a few hundred bytes.

Adjusting the stack limit

You cannot change the stack size on an existing thread, but you can set it on new ones:

Thread testThread = new Thread(() =>
{
    for (int i = 1000; i < Int32.MaxValue; i+=1000)
    {
        StackCheck(i);
        Console.WriteLine($"Successfully allocated {i} bytes on the stack");
    }
}, 200_000_000);
testThread.Start();
testThread.Join();

Obviously the whole stack is allocated when you create the thread, if you set it too large, the Thread constructor will throw an OutOfMemoryException.

But again, this test was done to mainly to satisfy my own curiosity, as others stated don't do this unless you really really know what you're doing.

Default stack size for pthreads, A stack allocated memory block created during the method execution is Limit the amount of memory you allocate with stackalloc : C# Copy. Update: The 2Gb single-object memory limit has been lifted on 64 bit with the release of .NET 4.5. You'll need to set gcAllowVeryLargeObjects in your app.config. The maximum number of elements in an array is still 2^32-1, though.


Increasing the Size of your Stack (.NET Memory Management: Part 3), The reserved memory size represents the total stack allocation in virtual they do remove pages from the system total commit limit, which is the  With .NET 4.5, 10 years memory limit of 2 GB is over Posted on 22/07/2012 by Bahrudin Hrnjica There are numerous new features coming with .NET 4.5 and here, on this blog, you can find several posts about it.


stackalloc expression, Even though with the .NET framework we don't have to actively worry about memory management and garbage collection (GC), we still have to keep memory​  In C++ you can simply specify the linker’s /stack option but in C# you have to jump through a few hoops in order to change stack size. The Easiest Way ( .NET 2.0 ) In .NET 2.0 and newer you can simply specify thread size in a thread’s constructor. Unfortunately, this method is only compatible only with Windows XP and newer operating systems.


Thread Stack Size, Stack is used for static memory allocation and Heap for dynamic memory allocation, both stored in the computer's RAM . Variables allocated on the stack are  Thread Stack Size. 05/31/2018; 2 minutes to read; In this article. Each new thread or fiber receives its own stack space consisting of both reserved and initially committed memory. The reserved memory size represents the total stack allocation in virtual memory. As such, the reserved size is limited to the virtual address range.