How to handle DuplicateKeyError in MongoDB (pyMongo)?

pymongo autoreconnect
read data from mongodb python
pymongo exceptions
mongodb duplicate key error
pymongo find
mongodb python driver
mongodb python api example
pymongo e11000 duplicate key error

Could anybody please tell me how to handle DuplicateKeyError in MongoDB?

I am writing a python script, where I move several docs from two different collections into a third one. There is a small overlap between the two collections due to having a few identical documents (with identical ObjectId). This results in the following:

DuplicateKeyError: E11000 duplicate key error collection: admin.collection_test index: id dup key: { : ObjectId('593a920b529e170d4b8fbf72') }

In order to get rid of the error I use:

try:
    do something
except pymongo.errors.DuplicateKeyError:
    pass

I expect by using of the "try-except" to move all non-crossing documents to the third collection, but instead the script just peacefully stops running once a first overlap (an already existing document in the collection) appears. Would appreciate any help a lot!

for doc in documents:
    client.update_one({'_id': doc['_id']}, doc, upsert=True)

You can use update_one with upsert=True. This updates doc with new doc if doc exists already otherwise it creates new doc.

How to handle DuplicateKeyError in MongoDB (pyMongo)?, How to handle DuplicateKeyError in MongoDB (pyMongo)?. pymongo autoreconnect read data from mongodb python pymongo errors mongodb duplicate key  If a document does not have a value for the indexed field in a unique index, the index will save a null value for this document. Because of the unique feature, MongoDB will only permit one document that lacks the indexed field. So just remove that null document and it will work.

If you're iterating over the documents, try using continue instead of pass.

for doc in documents:
    try:
        # insert into new collection
    except pymongo.errors.DuplicateKeyError:
        # skip document because it already exists in new collection
        continue

errors – Exceptions raised by the pymongo package, In order to auto-reconnect you must handle this exception, recognizing that the operation Raised when an insert or update fails due to a duplicate key error. There is an answer for preventing addition of duplicate documents in mongoDB in general at How to stop insertion of Duplicate documents in a mongodb collection. The idea is to use update with upsert=True instead of insert_one. So while inserting the code for pymongo would be. db[collection_name].update(document,document,upsert=True)

CONNECT TO DATABASE
connection = pymongo.MongoClient("192.168.2.202", 27017)
CREATE DATABASE
database = connection['my_database']
CREATE COLLECTION
collection = database['my_collection']
INSERT DOCUMENTS IN COLLECTION
url="http://some/api/url/path/?format=json"
data = {
    '_id': url,
    'timestamp': datetime.datetime.now(),
    'data': {
        'XX': 1,
        'YY': 2,
        'ZZ':  3
    }
}
TO AVOID DUPLICATES - THIS WILL CREATE NEW DOCUMENT IF SAME ID NOT EXIST
collection.update_one({'_id': url},  {"$set": data}, upsert=True)

Tutorial, The first step when working with PyMongo is to create a MongoClient to the DuplicateKeyError: E11000 duplicate key error index: test_database.profiles. This answer is obsolete : the dropDups option was removed in MongoDB 3.0, so a different approach will be required in most cases.For example, you could use aggregation as suggested on: MongoDB duplicate documents even after adding unique key.

pymongo.errors.DuplicateKeyError Python Example, This page provides Python code examples for pymongo.errors. safe insert for second unique item' # except DuplicateKeyError: # pass # Test Remove upon intermittent Mongo errors # If, however, we get a DuplicateKeyError, suppress it  I have a MongoDB replica set with 3 members and a Python application which stores data in it. I can handle pymongo's AutoReconnect exception when using single document inserts with a wrapper as fo

mongodb/mongo-php-library, BulkWriteException {#618 #message: "E11000 duplicate key error index: Consider: <?php $m = new MongoDB\Driver\Manager; $c = new  MongoDB stores data in BSON format. BSON strings are UTF-8 encoded so PyMongo must ensure that any strings it stores contain only valid UTF-8 data. Regular strings (<type ‘str’>) are validated and stored unaltered. Unicode strings (<type ‘unicode’>) are encoded UTF-8 first.

Flask: Building Python Web Services, If we add the indices through the MongoDB shell, it means that someone has to Therefore, we'll create an independent Python script just to make the indices. to our application so that it can deal with the case of a new request being made DuplicateKeyError: return False If we try to insert a request into our database  The following are code examples for showing how to use pymongo.errors.DuplicateKeyError().They are from open source Python projects. You can vote up the examples you like or vote down the ones you don't like.

Comments
  • your ObjectId already exist for this collection for another document