mongodb move documents from one collection to another collection

mongoose move document to another collection
mongodb move collection to another database
mongodb copy documents from one collection to another java
mongodb copy collection to another collection
mongodb create collection from another collection
copy indexes from one collection to another mongodb
copy collection mongodb compass
mongodb 4.2 clone collection

How can documents be moved from one collection to another collection in MongoDB?? For example: I have lot of documents in collection A and I want to move all 1 month older documents to collection B (these 1 month older documents should not be in collection A).

Using aggregation we can do copy. But what I am trying to do is moving of documents. What method can be used to move documents?


Update 2

Please do NOT upvote this answer any more. As written @jasongarber's answer is better in any aspect.

Update

This answer by @jasongarber is a safer approach and should be used instead of mine.


Provided I got you right and you want to move all documents older than 1 month, and you use mongoDB 2.6, there is no reason not to use bulk operations, which are the most efficient way of doing multiple operations I am aware of:

> var bulkInsert = db.target.initializeUnorderedBulkOp()
> var bulkRemove = db.source.initializeUnorderedBulkOp()
> var date = new Date()
> date.setMonth(date.getMonth() -1)
> db.source.find({"yourDateField":{$lt: date}}).forEach(
    function(doc){
      bulkInsert.insert(doc);
      bulkRemove.find({_id:doc._id}).removeOne();
    }
  )
> bulkInsert.execute()
> bulkRemove.execute()

This should be pretty fast and it has the advantage that in case something goes wrong during the bulk insert, the original data still exists.


Edit

In order to prevent too much memory to be utilized, you can execute the bulk operation on every x docs processed:

> var bulkInsert = db.target.initializeUnorderedBulkOp()
> var bulkRemove = db.source.initializeUnorderedBulkOp()
> var x = 10000
> var counter = 0
> var date = new Date()
> date.setMonth(date.getMonth() -1)
> db.source.find({"yourDateField":{$lt: date}}).forEach(
    function(doc){
      bulkInsert.insert(doc);
      bulkRemove.find({_id:doc._id}).removeOne();
      counter ++
      if( counter % x == 0){
        bulkInsert.execute()
        bulkRemove.execute()
        bulkInsert = db.target.initializeUnorderedBulkOp()
        bulkRemove = db.source.initializeUnorderedBulkOp()
      }
    }
  )
> bulkInsert.execute()
> bulkRemove.execute()

mongodb move documents from one collection to another collection , Copies all documents from collection into newCollection using server-side JavaScript. If newCollection does not exist, MongoDB creates it. Parameter, Type � In MongoDB, the command does not exist to copy a collection from one database to another. To achieve it, use the below concept −db.yourCollectionName.find().f


The bulk operations @markus-w-mahlberg showed (and @mark-mullin refined) are efficient but unsafe as written. If the bulkInsert fails, the bulkRemove will still continue. To make sure you don't lose any records when moving, use this instead:

function insertBatch(collection, documents) {
  var bulkInsert = collection.initializeUnorderedBulkOp();
  var insertedIds = [];
  var id;
  documents.forEach(function(doc) {
    id = doc._id;
    // Insert without raising an error for duplicates
    bulkInsert.find({_id: id}).upsert().replaceOne(doc);
    insertedIds.push(id);
  });
  bulkInsert.execute();
  return insertedIds;
}

function deleteBatch(collection, documents) {
  var bulkRemove = collection.initializeUnorderedBulkOp();
  documents.forEach(function(doc) {
    bulkRemove.find({_id: doc._id}).removeOne();
  });
  bulkRemove.execute();
}

function moveDocuments(sourceCollection, targetCollection, filter, batchSize) {
  print("Moving " + sourceCollection.find(filter).count() + " documents from " + sourceCollection + " to " + targetCollection);
  var count;
  while ((count = sourceCollection.find(filter).count()) > 0) {
    print(count + " documents remaining");
    sourceDocs = sourceCollection.find(filter).limit(batchSize);
    idsOfCopiedDocs = insertBatch(targetCollection, sourceDocs);

    targetDocs = targetCollection.find({_id: {$in: idsOfCopiedDocs}});
    deleteBatch(sourceCollection, targetDocs);
  }
  print("Done!")
}

db.collection.copyTo() — MongoDB Manual, here is to move the records of a Mongo collection from one Mongo server to another. One use case for this is to copy a collection, or part of a collection, from a a directory for the database db_1 and within that two files for our collection_b . The above command is used to push collection data from one db to another db in same mongo instance. First, let me breakdown the command: db.source_collection.find({condition}) - To fetch the data from source_collection based on the given condition forEach() - iterates the data one by one


Insert and remove:

var documentsToMove = db.collectionA.find({});
documentsToMove.forEach(function(doc) {
    db.collectionB.insert(doc);
    db.collectionA.remove(doc);
});

note: this method might be quite slow for large collections or collections holding large documents.

How to Carefully Migrate a Collection From One Mongo Server Onto , You can copy documents from one collection to another (keeping the existing documents in the destination) using various methods: * MongoDB's export and� Starting in version 4.2, MongoDB removes the eval command. The deprecated db.collection.copyTo(), which wraps the eval command, can only be run against MongoDB 4.0 or earlier versions. For behavior and example, refer to the 4.0 or earlier version of the manual.


$out is use to create the new collection with data , so use $out

db.oldCollection.aggregate([{$out : "newCollection"}])

then use drop

db.oldCollection.drop()

MongoDB: How can I copy all documents from collection A to , db.[CollectionName].find().forEach(function(d){ db.getSiblingDB([Database])[ CollectionName].insert(d); });. Comment on gist. Sign in to comment or sign up to � When you have documents where a one-to-many relationship exists, it can be helpful to use an embedded, or nested, document structure. Working with an embedded document in a MongoDB collection isn’t difficult as long as you understand how this unique structure should be used.


May be from the performance point of view it's better to remove a lot of documents using one command(especially if you have indexes for query part) rather than deleting them one-by-one.

For example:

db.source.find({$gte: start, $lt: end}).forEach(function(doc){
   db.target.insert(doc);
});
db.source.remove({$gte: start, $lt: end});

MongoDb copy data from one collection to another collection � GitHub, How to Copy a MongoDB Collection to Another Database you can choose how the copy operation should treat (existing) document _ids. To delete all documents in a collection, pass an empty document ({}). justOne: boolean: Optional. To limit the deletion to just one document, set to true. Omit to use the default value of false and delete all documents matching the deletion criteria. writeConcern: document: Optional. A document expressing the write concern. Omit to use the


The Fastest Way to Copy MongoDB Collections, In MongoDB, the command does not exist to copy a collection from one The query to create a collection with a document is as follows −. mongoexport --collection collection1_backup --out collection1.json to import collection from json file . mongoimport --db test --collection collection1 --file collection1.json to import single collection from backup/dump file one need to convert *.bson file to *.json by using . bsondump collection1_backup.bson > collection1_backup.json


How to copy a collection from one database to another in MongoDB?, I have a part of a document in mongo collection that we originally thought would Above is the data I have, Now I am trying to add one field at filters, array with� movePrimary fails if the destination shard contains a conflicting collection namespace. For example: An administrator issues movePrimary to change the primary shard for the hr database. A user or application issues a write operation against an unsharded collection in hr while movePrimary is moving that collection. The write operation creates


Move Part of Mongo Document to Another Collection : mongodb, destination collection a clone of the source collection. Additionally, I know from this: mongodb move documents from one collection to another collection that I� If you need to move a collection between two MongoDB databases, there is no need to dump and restore your data using mongodump / mongorestore. In a way, it's pretty similar to what we can do with MySQL's "RENAME TABLE" command: db.runCommand({renameCollection:"sourcedb.mycol",to:"targetdb.mycol"})