How can I in C# stream.Read into unmanaged memory stream?

I can read unmanaged memory in C# using UnmanagedMemoryStream, but how can I do the reverse?

I want to read from a managed stream directly into unmanaged memory, instead of first reading into a byte[] and then copying. I'm doing async stream reading on a large number of requests, so the added memory is significant (not to mention the additional copy).

I think that it is not really possible. When you talk about a managed stream, I suppose you are refering to an instance of System.IO.Stream or a subclass hereof.

The members of this class take byte[] as a parameter, and that is a managed class.

I guess that the closest you can come is creating a managed byte[], and then creating a byte* in an unsafe block, and then passing the byte* to whatever needs an unmanaged reference:

unsafe function test()
{
    var buffer = new byte[1024];
    fixed (byte* bufferPtr = &buffer[0])
    {   
        // Read bytes and pass the ptr to a function that needs to
        // operate on data directly 
    }
}

But that is not exactly what you're asking for

edit: Be careful though. You mention something about async reads. As soon as you're outside the fixed boundary, the GC may move the array around in memory. And if you've passed the pointer to some "unsafe" function that continues to operate in a different thread, the pointer will point to an invalid memory location.

UnmanagedMemoryStream.Read Method (System.IO), Length]; // Read from unmanaged memory to the byte array. readStream.Read(​outMessage, 0, message.Length); // Close the stream. readStream.Close();  public: override int Read(Span<System::Byte> destination); public override int Read (Span<byte> destination); override this.Read : Span<byte> -> int Public Overrides Function Read (destination As Span(Of Byte)) As Integer. When this method returns, this span contains all the bytes from the unmanaged memory stream.

This actually isn't too hard if you have a known destination buffer and you know how big your data is. The key realization, once again, is that an UnmanagedMemoryStream is just a thin wrapper around a sequence of native bytes.

What you want to do is grab a pointer to your native destination buffer in the usual fashion. Then you can construct a writable UnmanagedMemoryStream on top of it. Copy your source stream to the destination stream and voila! A single copy has moved you from the managed memory world to the native memory world. In C++/CLI, it looks something like this:

void CopyStreamToNativePtr(Stream^ src, unsigned char* dst)
{
    // Assuming we want to start the copy at the beginning of src
    src->Seek(0, SeekOrigin::Begin);

    // Create an UnmanagedMemoryStream on top of the destination
    // with an appropriate size and capacity (We assume the buffer is
    // is sized to the source data in this function!)
    UnmanagedMemoryStream target(dst, src->Length, src->Length, FileAccess::Write);

    // Copy to your heart's content!
    src->CopyTo(%target);

    // We made the UnmanagedMemoryStream local so that we wouldn't have
    // to explicitly Dispose() of it.
}

UnmanagedMemoryStream Class (System.IO), I can read unmanaged memory in C# using UnmanagedMemoryStream, but how can I do the reverse? I want to read from a managed stream directly into  Reads all the bytes of this unmanaged memory stream into the specified span of bytes. Read(Span<Byte>) When overridden in a derived class, reads a sequence of bytes from the current stream and advances the position within the stream by the number of bytes read.

The managed stream will always need a "managed" object reference to a valid byte[] object. However, you can use a managed byte[] which is pinned en lieu of a unmanaged memory allocation and therefore make ir accessible as unmanaged memory block also:

byte[] data = new byte[];
GCHandle pin = GCHandle.Alloc(data, GCHandleType.Pinned)
try {
  IntPtr dataUnmanagedPtr = Marshal.UnsafeAddrOfPinnedArrayElement(data, 0);
  // managed.Read(data, index, count);
  // use the unmanaged pointer here for unmanaged code
} finally {
  pin.Free(pin); // but make sure that no unmanaged code uses the pinned data anymore upon release
}

UnmanagedMemoryStream Constructor (System.IO), FileStream, Stream used to read and write data to the file. MemoryStream, Creates a stream whose backing store is memory. UnmanagedMemoryStream. Asynchronously reads the unmanaged memory stream bytes into the memory region. (Inherited from UnmanagedMemoryStream) ReadByte() Reads a byte from a stream and advances the position within the stream by one byte, or returns -1 if at the end of the stream. (Inherited from UnmanagedMemoryStream) Seek(Int64, SeekOrigin)

You can use C# and read file by Windows API directly into unmanaged memory.

If you want to read from FileStream, this could be helpful. Check FileStream implementation and do something similar.

var handle = Win32Native.SafeCreateFile(path, fAccess, share, secAttrs, mode, flagsAndAttributes, IntPtr.Zero);

fixed(byte* p = bytes) 
{   
   r = Win32Native.ReadFile(handle, p + offset, count, out numBytesRead, IntPtr.Zero);
}

How can I in C# stream.Read into unmanaged memory , This file provides a memory stream that uses unmanaged memory either from using System.IO; <returns>The actual number of bytes read into the buffer. The current position within the stream is advanced by the number of bytes read; however, if an exception occurs, the current position within the stream remains unchanged. Implementations return the number of bytes read. The implementation will block until at least one byte of data can be read, in the event that no data is available.

C# Streams tutorial - binary streams in C#, length, The size of unmanaged buffer in bytes. capacity, The total amount of memory assigned to the stream. access, Specifies if the stream should be read-​only,  C# (CSharp) System.IO UnmanagedMemoryStream.Read - 19 examples found. These are the top rated real world C# (CSharp) examples of System.IO.UnmanagedMemoryStream.Read extracted from open source projects. You can rate examples to help us improve the quality of examples.

MPI.NET/UnmanagedMemoryStream.cs at master · mpidotnet/MPI , ZejulioZ wrote: Because I'm not sure C# arrays are contiguous like C/C++ arrays are. If you pinned it, they are Smile | :) ZejulioZ wrote: And I profit to ask you;  Memory streams created with an unsigned byte array provide a non-resizable stream of the data. When using a byte array, you can neither append to nor shrink the stream, although you might be able to modify the existing contents depending on the parameters passed into the constructor.

System::IO::UnmanagedMemoryStream Class Reference, ReadByte()); // 102 // Read from the stream back into the block array: Console. ms = new MemoryStream(); byte[] buffer = new byte [0x1000]; // Read in 4KB blocks do { ms. DisposeLocalCopyOfClientHandle(); // Release unmanaged rx. This works perfectly in C and C++. But we can't use pointers in C#, can we? Take a couple of minutes to think about this problem. It's a mini-problem. It's a kindergarten problem for a C++ developer. How could it possibly be difficult in managed code? Remember that one of the primary benefits of a managed environment is that, well, it's managed

Comments
  • The memory is already allocated in C++, but if it's really not possible I might have to change the existing code to accept a pointer to fixed managed memory :(
  • how to copy only a certain # of bytes from one stream to another? src->CopyTo will copy the entire stream. I only want to copy n bytes.
  • I found this to be the best solution for two reasons: 1. It does not require unsafe code 2. It prevents the GC from moving around our array in memory (for as long as we need)