mongoengine - Ignore extra fields for schema validation

mongoengine query embedded document
mongoengine list of reference fields
mongoengine dbref
mongoengine to_json
mongoengine projection
mongoengine listfield
mongoengine connect
mongoengine delete field

I am trying to query my database. Some records currently have extra fields that are not included in my model schema (by error, but I want to handle these cases). When I try to query the DB and transform the records into the schema, I get the following error:

FieldDoesNotExist
The field 'X' does not exist on the document 'Y'

Because of the extra fields in the database that differ from the schema.

Is there a way to ignore this schema validation for extra fields in mongoengine?

For ignoring this error when having extra fields while data loading, set strict to False in your meta dictionary.

class User(Document):
    email = StringField(required=True, unique=True)
    password = StringField()
    meta = {'strict': False}

mongoengine, __only_fields – A set of fields that have been loaded for this document. You can disable automatic index creation by setting auto_create_index to False the document clean method, requires validate to be True. write_concern – Extra A Dynamic Document class allowing flexible, expandable and uncontrolled schemas. 2.3.1. Defining a document’s schema¶ MongoEngine allows you to define schemata for documents as this helps to reduce coding errors, and allows for utility methods to be defined on fields which may be present. To define a schema for a document, create a class that inherits from Document.

I believe you want to use a DynamicDocument instead of a Document when defining your model and that will allow extra fields in the db schema to be ignored.

3. API Reference, To define a schema for a document, create a class that inherits from Document . Fields also may have validation constraints available (such as max_length in the recommended as dictionaries don't support validation or custom field types. output with unnecessary detail qs = Page.objects.exclude('id').as_pymongo()  41 mongoengine - Ignore extra fields for schema validation Dec 1 '15 9 Firestore security rules with reference fields Mar 23 '18 7 How can I create unique IDs for embedded documents in MongoDB?

I think you want skip schema validation, so when you save your document

document_name.save(validate=False)

2.3. Defining documents, I'd like to make a Schema to validate collections with using Collection2, but I'd like for the validation to ignore fields that aren't listed in the schema. Is this possible to do? For example, if I have Items = new Mongo. mongoengine-Ignore extra fields for schema validation ; How can I initialize a MySQL database with schema in a Docker container? How to get current tenant when using Appartment gem in multi-tenancy Rails 4

You can extend from mon.DynamicDocument.

class AF(mon.DynamicDocument):
  meta = {
    'collection': 'af'
  }
user_id = mon.StringField(db_field='customer_user_id')

You can see from the document. A Dynamic Document class is allowing flexible, expandable and uncontrolled schemas.

A way to validate fields of schema, but ignore extra fields? · Issue , MongoDB provides the capability to perform schema validation during updates and insertions. "must be a double if the field exists" }, address: { bsonType: "​object", To disable validation entirely, you can set validationLevel to off . status: { enum: [ "Unknown", "Incomplete" ], description: "can only be one  Generally you can define the document as usual, but you might want to inherits mongoengine_goodjson.Document or mongoengine_goodjson.EmbeddedDocument. Here is the example: """Example schema.""" import mongoengine_goodjson as gj import mongoengine as db class Address ( gj .

Schema Validation, If a valid JSON Schema object is specified, additional fields must validate against the schema. Defaults to true . properties, objects, object, A valid JSON Schema  As fields in MongoEngine may use different names in the database (set using the db_field keyword argument to a Field constructor), a mechanism exists for replacing MongoEngine field names with the database field names in Javascript code. When accessing a field, use square-bracket notation, and prefix the MongoEngine field name with a tilde (~).

$jsonSchema, Doing schema validation in the application remains a best practice, but JSON Let's try now to insert documents with arbitrary extra fields that are not in the JSON Schema Validator. Shell Not valid documents are ignored. 2.4.1.1. Pre save data validation and cleaning¶ MongoEngine allows you to create custom cleaning rules for your documents when calling save(). By providing a custom clean() method you can do any pre validation / data cleaning. This might be useful if you want to ensure a default value based on other document values for example:

MongoDB Data Validator: How to Use the JSON Schema Validator , A schema validation package that supports direct validation of MongoDB The object you validate can be a MongoDB modifier. Custom Field Validation; Custom Whole-Document Validators; Manually See the Custom Validation section. ignore : An array of validation error types (in SimpleSchema. If true, additional fields are allowed. If false, they are not. If a valid JSON Schema object is specified, additional fields must validate against the schema. Defaults to true. properties: objects: object: A valid JSON Schema where each value is also a valid JSON Schema object: patternProperties: objects: object

Comments
  • References: mongoengine-odm.readthedocs.io/… mongoengine-odm.readthedocs.io/…
  • Hey, thanks for the response! This would work but I still would like the fields in my schema to be validated, I just also want to ignore any extra fields that are returned in my records. The obvious method is to use the only() on all my fields, but that isn't very nice.