db.StringProperty but property is occasionally datastore_types.Text?
Are there any circumstances under which GAE datastore might change a property type from
TextProperty (effectively ignoring the model definition)?
Consider the following situation (simplified):
class MyModel(db.Model): user_input = db.StringProperty(default='', multiline=True)
Some entity instances of this model in my datastore have a datatype of
TextProperty for 'user_input' (rather than simple
str) and are therefore not indexed. I can fix this by retrieving the entity, setting
model_instance.user_input = str(model_instance.user_input) and then putting the entity back into the datastore.
What I don't understand is how this is happening to only some entities, when there have been no changes to this model. Is it possible that the'type' of a db.model's property can be overridden from
Entity Property Reference, Because long text strings and long byte strings are not indexed, they have no ordering defined. StringProperty, Unicode string; up to 1500 bytes, indexed Corresponds to BlobReferenceProperty in the old db API, but the property value is a a calendar date and a time of day; the others are occasionally useful for special Model property: StringProperty. bool. A Boolean value (True or False). Model property: BooleanProperty. int or long. An integer value, up to 64 bits. Python int values are converted to Python long values prior to storage. A value stored as an int will be returned as a long. If a long larger than 64 bits is assigned, only the least significant
There's a length limit on
StringProperty of 500 characters. I don't know for sure, but maybe the datastore will convert a
StringProperty to a
TextProperty if it goes over the limit.
Having said that, I doubt GAE would just change an indexed property to a non-indexed one implicitly. But it's all I can think of.
Types and Property Classes, If you are currently using the older DB Client Library, read the DB to NDB a str value, but this value is stored as a byte string and is not encoded as text. If the string property is required, its value cannot be an empty string. GitHub Gist: star and fork mdornseif's gists by creating an account on GitHub.
With your use of setattr
setattr(some_instance, 'string_type', some_instance.text_type) you you have actually made instance property named string_type property actually point to the text_type property. So two names one property
The db, ndb, users, urlfetch, and memcache modules are imported. > from google.appengine.ext import db > class TestModel(db.Model): ... string_type = db.StringProperty(default='', multiline=True) ... text_type = db.TextProperty() ... > > some_instance = TestModel() > some_instance.text_type = 'foobar' > setattr(some_instance, 'string_type', some_instance.text_type) > print type(some_instance.string_type) <class 'google.appengine.api.datastore_types.Text'> y> repr(some_instance.string_type) "u'foobar'" > id(some_instance.string_type) 168217452 > id(some_instance.text_type) 168217452 > some_instance.string_type.__class__ <class 'google.appengine.api.datastore_types.Text'> > some_instance.text_type.__class__ <class 'google.appengine.api.datastore_types.Text'> > dir(some_instance.text_type.__class__)
Note the id of the two properties above.
So you are effectively rebinding the instance level property definition, then this modified model is written with the type of the field to the datastore.
If you want to use setattr (though can't see why in this contrived example) you should be getting the value from the property first using
) to get the value to be assigned and not rebind the instances property.
Breaking the limits: Storing data bigger than 1 Mb in Google App , I'm using it with (), but I found myself coming up against a hard limit with the XML data in there, and sometimes that can exceed the 1 Mb limit. #!/usr/bin/env python # encoding: utf-8 """ compressible_text_property.py A string property from google.appengine.api import datastore_types import zlib Text A property can have values of more than one type, and two entities can have values of different types for the same property. Properties can be indexed or unindexed (queries that order or filter on a property P will ignore entities where P is unindexed).
Turns out there IS a way the type of a property from the model's definition can be bypassed (and this was our case)! The following self contained example runs in the interactive console and demonstrates the bug. Essentially,
setattr bypasses the property's type if you assign an existing property of a different type:
from google.appengine.ext import db class TestModel(db.Model): string_type = db.StringProperty(default='', multiline=True) text_type = db.TextProperty() some_instance = TestModel() some_instance.text_type = 'foobar' setattr(some_instance, 'string_type', some_instance.text_type) some_instance.put() retrieved_instance = db.get(some_instance.key()) print id(retrieved_instance.string_type) #10166414447674124776 print id(retrieved_instance.text_type) #10166414447721032360 print type(retrieved_instance.string_type) #OOPS: string_type is now Text type!
Model and Property, StringProperty : a short text string, limited to at most 1500 bytes (when UTF-8 encoded from str to bytes). Structured property types can be nested arbitrarily deep, but in a hierarchy of Also unlike legacy db , when the entity is written, the property values are Sometimes the set of properties is not known ahead of time. The following are code examples for showing how to use google.appengine.ext.db.ListProperty().They are from open source Python projects. You can vote up the examples you like or vote down the ones you don't like.
google/appengine/ext/ndb/model.py, StringProperty: a short text string, limited to 500 bytes. - TextProperty: an an allowed value), but a required property can have a default. Some property types BlobKey = datastore_types.BlobKey Sometimes the low-level query interface needs a specific data type. in order for And unlike classic db, auto_now. does not Occasionally, one will want to convert the text from a column to either all upper-case letters or all lower-case letters. In the example that follows, the output of the first column is converted to upper-case and the second to lower-case:
How to check a Long for null in java, 2592 | db.StringProperty but property is occasionally datastore_types.Text? 7417 | run shell commands from python, and How to store result in a file(text or excell Database properties The following table lists properties available for each database on the database server. You can use the DB_PROPERTY system function to retrieve the value for an individual property, or you can use the sa_db_properties system procedure to retrieve the values of all database properties.
google.appengine.ext.ndb.model, For example: class Address(Model): street = StringProperty() city Sometimes the set of properties is not known ahead of time. from .google_imports import datastore_types from .google_imports import users from .google_imports But _validate() now # rejects these, so you can't write new non-UTF-8 TEXT # properties. class Song(db.Model): title = db.StringProperty() composer = db.StringProperty() date = db.DateTimeProperty() q = db.Query(Song) or the class method all() of the kind's model class: q = Song.all() Without further modification, the resulting instance of class Query will retrieve all existing entities of the specified kind. Method calls on the