Most efficient way of copying a raw byte array into an empty byte vector

copy array to vector c++
memcpy
std::copy
uint8_t array to vector
std::vector
c++ copy struct to vector
c move array to vector
c++ vector push_back array

I have a scenario in which I need to copy the contents of a raw dynamically allocated uint8_t array into a vector (which is guaranteed to be empty whenever this scenario happens).

vector<uint8_t> myVector;
const uint8_t* myRawArray;

It is really important to me that the copy operation is as efficient as possible and portable (various compiler versions might be used).

One approach I thought of using is this:

myVector.reserve(byteCount);
myVector.insert(myVector.begin(), myRawArray, myRawArray + byteCount);

Any ideas on how the speed of that compares to this one:

myVector.resize(byteCount);
memcpy(myVector.data(), myRawArray, byteCount);

I guess memcpy should be fast but then I am forced to use resize which needs to zero-out the memory, so I guess it will slow it down a bit..

Also, any other suggestions?


If you don't need to create the vector before the copy takes place, you could always pass the raw array to the constructor of your vector:

std::vector<uint8_t> myVector(myRawArray, myRawArray + byteCount);

If you do need to construct the vector beforehand, the following is an option:

std::vector<uint8_t> myVector;
// ... do some stuff ...
// Now, we're ready for the copy, and byteCount is known.
myVector.reserve(byteCount);
std::copy(myRawArray, myRawArray + byteCount, std::back_inserter(myVector));

I would suggest using std::copy unless memcpy is proven to be faster. std::copy is safer and more idiomatic in C++ code, but don't be afraid to use memcpy if it really is proven to be faster. The speed difference will most likely change with different compilers.

I hope this helps.

How do you copy the contents of an array to a std::vector in C++ , To cut a long story short Method 4, using vector::insert, is the best for Apart from being an ugly solution (byte copying!) remember that this can  Direct ByteBuffers provide very efficient I/O, but getting data into and out of them is more expensive than byte[] arrays. Thus, the fastest choice is going to be application dependent. Amazingly, in my tests, if the buffer size is at least 2048 bytes, it is actually faster to fill a byte[] array, copy it into a direct ByteBuffer, then write


memcpy() is usually written in assembly and it is very optimized so you should know that memcpy will be fast. vector::insert is usually implemented as having a call to memcpy under the hood but it does need to check if there is enough space in the vector for the insertions to take place without any reallocations. I have not profiled this but I bet the first version with the call to reserve is faster.

An alternative to this would be to use std::copy which has been found to be slightly faster than using memcpy in some cases, you can be sure that if possible it also makes a call to memcpy or does something better. So performance issues should not be a problem with it. It will also take care of increasing the size of the vector to match your requirement.

std::memcpy, Copies count bytes from the object pointed to by src to the object pointed to by dest . Both objects are reinterpreted as arrays of unsigned char. It is usually more efficient than std::strcpy, which must scan the data it Several C++ compilers transform suitable memory-copying loops to std::memcpy calls. The memory in vector is guaranteed to be allocated contiguously, and unsigned char is POD, therefore it is totally safe to memcpy into it (assuming you don't copy more than you have allocated, of course).


Thanks all for your input to my issue I have resolve the problem by doing the following changes to my structure and implementing it like this

   struct YUV_Buffer
   {
void *pCacheBuf = nullptr;
int frameID = 0;
int height = 0;
int width = 0;
void CopyBuf(BYTE * pBuf, int sizBuf)
{
    pCacheBuf = new BYTE[sizBuf];
    memcpy(pCacheBuf, pBuf, sizBuf);
}

YUV_Buffer(BYTE * pBuf, int nFrameID, int nHeight, int nWidth)
    : frameID(nFrameID), height(nHeight), width(nWidth)
{
    CopyBuf(pBuf, 8 * 1024 * 1024);
}

YUV_Buffer(const YUV_Buffer & yuvbuf)
    :frameID(yuvbuf.frameID), height(yuvbuf.height), width(yuvbuf.width)
{
    CopyBuf((BYTE*)yuvbuf.pCacheBuf, 8 * 1024 * 1024);
}
~YUV_Buffer() {
    delete[]pCacheBuf;
    pCacheBuf = NULL;
}
 };

I then implement it like this:

  YUV_Buffer nBuffer = YUV_Buffer((BYTE*)pSysFrame, pmfxInSurface->Data.FrameOrder, pmfxInSurface->Info.CropH, pmfxInSurface->Info.CropW);
mBuffer.emplace_back(nBuffer);

Hope this may help others also complements to sarabande from expert-exchange for there help and input.

Regards Nigel

Convert an array to a vector in C++, In this post, we will discuss how to convert an array to a vector in C++. Another efficient solution is to use the std::insert function which takes three parameters  StringBuilder is a very useful and memory efficient way to concatenate strings, but there is no obvious similar class for byte arrays. This class adds that, and provides a flexible binary data storage medium at the same time.


QByteArray Class, QByteArray can be used to store both raw bytes (including '\0's) and For most purposes, QString is the class you want to use. One way to initialize a QByteArray is simply to pass a const char * to its constructor. If the byte array being appended to is not empty, a deep copy of the data is performed, taking linear time. I have a scenario in which I need to copy the contents of a raw dynamically allocated uint8_t array into a vector (which is guaranteed to be empty whenever this scenario happens). vector<uint8_t> myVector; const uint8_t* myRawArray;


std::vec::Vec, vec[0], 7); vec.extend([1, 2, 3].iter().copied()); for x in &vec { println! In Rust, it's more common to pass slices as arguments rather than vectors when you Option<Vec<u8>> { // Per the FFI method's docs, "32768 bytes is always enough". let is also more efficient, because it can backshift the elements of the array in bulk. See Convert between rune array/slice and string. Convert string to bytes. When you convert a string to a byte slice, you get a new slice that contains the same bytes as the string. b := []byte("ABC€") fmt.Println(b) // [65 66 67 226 130 172] Note that the character € is encoded in UTF-8 using 3 bytes.


Convert byte[] array to File using Java, Remove first element from ArrayList in Java · Best Books to Learn Java for To convert byte[] to file getBytes() method of String class is used, and simple write()  and a local array of bytes inside a function (declared the same way) that gets re-declared every time the function gets called. The local array is called sequence2. Inside the function, I want to only copy a certain portion of sequence1 into sequence2 and would like sequence2 to be various sizes per function call based on how many characters