可变JSON对象类型不跟踪更改SQLAlchemy (Sqlalchemy)

Question

好的:我正在用和PostgreSQL构建一个Flask应用程序，我想在我的数据库中创建一个可变的JSON类型列。在SQLAlchemy中有许多在线示例(以及这里的许多问题)定制可变对象类型。我找到了一个：金刚 (带有一个完整的作者在这里)，它处理可变的JSON对象类型。从理论上讲，它提供了JsonObject，它是一个JSON对象类型，在基本级别上具有dict和list的更改跟踪，而NestedJsonObject是一个JSON对象类型，对dict和list具有嵌套的更改跟踪。看起来很甜。

我不能为我的生活使它工作，虽然我有充分的信心，它确实如此。注意:上面的作者写的并不是一个在列中实现对象类型的例子，所以由于我是一个新手，我很有可能把下一个部分搞错了。然而，我已经咨询了sqlalchemy.ext.mutable参考，这似乎是正确的。

这是我的models.py：

from application import app
from flask_sqlalchemy import SQLAlchemy
from flask_login import UserMixin
import sqlalchemy_json # I'm using Alembic for migrations and make this import in my script.py.mako, too, in case that matters
from sqlalchemy_json import NestedJsonObject

db = SQLAlchemy(app)

class User(UserMixin, db.Model): #flask-sqlalchemy provides a Base declaration with db.Model
    __tablename__ = 'users'

    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(80), unique=True)
    data = db.Column(NestedJsonObject)

    def __init__(self, name, data):
        self.name = name
        self.data = data

    def __repr__(self):
        return "<User(id='%d', name='%s', connections='%r')>" % (self.id, self.name, self.data)

...

考虑到这一点，这是与我的用户合作的一个例子：

data = {}
data['a']= {'b': 'c', 'd': 'e'}
user = User( ... , data)
db.session.add(user)
db.session.commit()
#<User( ... data='{u'a': {u'b': u'c', u'd':u'e'}}')>

最初的承诺是有效的。但随后的承诺不包括：

user = db.session.query(User).filter( ... ).first()
user.data['foo']={}
db.session.commit()
#<User( ... data='{u'a': {u'b': u'c', u'd':u'e'}}')>
user.data['foo']['bar'] = {'x': 'x', 'z': 'z'}
db.session.commit()
#<User( ... data='{u'a': {u'b': u'c', u'd':u'e'}}')>

你说对了。为了清楚起见，下面是模块中的两个文件和作者的注释：

alchemy.py

请注意文件底部的.associate_with：

# Third-party modules
try:
  import simplejson as json
except ImportError:
  import json

import sqlalchemy
from sqlalchemy.ext import mutable

# Custom modules
from . import track


class NestedMutable(mutable.MutableDict, track.TrackedDict):
  """SQLAlchemy `mutable` extension dictionary with nested change tracking."""
  def __setitem__(self, key, value):
    """Ensure that items set are converted to change-tracking types."""
    super(NestedMutable, self).__setitem__(key, self.convert(value, self))

  @classmethod
  def coerce(cls, key, value):
    """Convert plain dictionary to NestedMutable."""
    if isinstance(value, cls):
      return value
    if isinstance(value, dict):
      return cls(value)
    return super(cls).coerce(key, value)


class _JsonTypeDecorator(sqlalchemy.TypeDecorator):
  """Enables JSON storage by encoding and decoding on the fly."""
  impl = sqlalchemy.String

  def process_bind_param(self, value, dialect):
    return json.dumps(value)

  def process_result_value(self, value, dialect):
    return json.loads(value)


class JsonObject(_JsonTypeDecorator):
  """JSON object type for SQLAlchemy with change tracking as base level."""


class NestedJsonObject(_JsonTypeDecorator):
  """JSON object type for SQLAlchemy with nested change tracking."""


mutable.MutableDict.associate_with(JsonObject)
NestedMutable.associate_with(NestedJsonObject)

track.py

#!/usr/bin/python
"""This module contains the tracked object classes.
TrackedObject forms the basis for both the TrackedDict and the TrackedList.
A function for automatic conversion of dicts and lists to their tracked
counterparts is also included.
"""

# Standard modules
import itertools
import logging


class TrackedObject(object):
  """A base class for delegated change-tracking."""
  _type_mapping = {}

  def __init__(self, *args, **kwds):
    self.logger = logging.getLogger(type(self).__name__)
    self.logger.debug('%s: __init__', self._repr())
    self.parent = None
    super(TrackedObject, self).__init__(*args, **kwds)

  def changed(self, message=None, *args):
    """Marks the object as changed.
    If a `parent` attribute is set, the `changed()` method on the parent will
    be called, propagating the change notification up the chain.
    The message (if provided) will be debug logged.
    """
    if message is not None:
      self.logger.debug('%s: %s', self._repr(), message % args)
    self.logger.debug('%s: changed', self._repr())
    if self.parent is not None:
      self.parent.changed()

  @classmethod
  def register(cls, origin_type):
    """Registers the class decorated with this method as a mutation tracker.
    The provided `origin_type` is mapped to the decorated class such that
    future calls to `convert()` will convert the object of `origin_type` to an
    instance of the decorated class.
    """
    def decorator(tracked_type):
      """Adds the decorated class to the `_type_mapping` dictionary."""
      cls._type_mapping[origin_type] = tracked_type
      return tracked_type
    return decorator

  @classmethod
  def convert(cls, obj, parent):
    """Converts objects to registered tracked types
    This checks the type of the given object against the registered tracked
    types. When a match is found, the given object will be converted to the
    tracked type, its parent set to the provided parent, and returned.
    If its type does not occur in the registered types mapping, the object
    is returned unchanged.
    """
    obj_type = type(obj)
    for origin_type, replacement in cls._type_mapping.iteritems():
      if obj_type is origin_type:
        new = replacement(obj)
        new.parent = parent
        return new
    return obj

  @classmethod
  def convert_iterable(cls, iterable, parent):
    """Returns a generator that performs `convert` on every of its members."""
    return (cls.convert(item, parent) for item in iterable)

  @classmethod
  def convert_iteritems(cls, iteritems, parent):
    """Returns a generator like `convert_iterable` for 2-tuple iterators."""
    return ((key, cls.convert(value, parent)) for key, value in iteritems)

  @classmethod
  def convert_mapping(cls, mapping, parent):
    """Convenience method to track either a dict or a 2-tuple iterator."""
    if isinstance(mapping, dict):
      return cls.convert_iteritems(mapping.iteritems(), parent)
    return cls.convert_iteritems(mapping, parent)

  def _repr(self):
    """Simple object representation."""
    return '<%(namespace)s.%(type)s object at 0x%(address)0xd>' % {
        'namespace': __name__,
        'type': type(self).__name__,
        'address': id(self)}


@TrackedObject.register(dict)
class TrackedDict(TrackedObject, dict):
  """A TrackedObject implementation of the basic dictionary."""
  def __init__(self, source=(), **kwds):
    super(TrackedDict, self).__init__(itertools.chain(
        self.convert_mapping(source, self),
        self.convert_mapping(kwds, self)))

  def __setitem__(self, key, value):
    self.changed('__setitem__: %r=%r', key, value)
    super(TrackedDict, self).__setitem__(key, self.convert(value, self))

  def __delitem__(self, key):
    self.changed('__delitem__: %r', key)
    super(TrackedDict, self).__delitem__(key)

  def clear(self):
    self.changed('clear')
    super(TrackedDict, self).clear()

  def pop(self, *key_and_default):
    self.changed('pop: %r', key_and_default)
    return super(TrackedDict, self).pop(*key_and_default)

  def popitem(self):
    self.changed('popitem')
    return super(TrackedDict, self).popitem()

  def update(self, source=(), **kwds):
    self.changed('update(%r, %r)', source, kwds)
    super(TrackedDict, self).update(itertools.chain(
        self.convert_mapping(source, self),
        self.convert_mapping(kwds, self)))


@TrackedObject.register(list)
class TrackedList(TrackedObject, list):
  """A TrackedObject implementation of the basic list."""
  def __init__(self, iterable=()):
    super(TrackedList, self).__init__(self.convert_iterable(iterable, self))

  def __setitem__(self, key, value):
    self.changed('__setitem__: %r=%r', key, value)
    super(TrackedList, self).__setitem__(key, self.convert(value, self))

  def __delitem__(self, key):
    self.changed('__delitem__: %r', key)
    super(TrackedList, self).__delitem__(key)

  def append(self, item):
    self.changed('append: %r', item)
    super(TrackedList, self).append(self.convert(item, self))

  def extend(self, iterable):
    self.changed('extend: %r', iterable)
    super(TrackedList, self).extend(self.convert_iterable(iterable, self))

  def remove(self, value):
    self.changed('remove: %r', value)
    return super(TrackedList, self).remove(value)

  def pop(self, index):
    self.changed('pop: %d', index)
    return super(TrackedList, self).pop(index)

  def sort(self, cmp=None, key=None, reverse=False):
    self.changed('sort')
    super(TrackedList, self).sort(cmp=cmp, key=key, reverse=reverse)

另外，

init.py

from .alchemy import NestedJsonObject, NestedMutable, JsonObject

__all__ = (
    'NestedJsonObject',
    'NestedMutable',
    'JsonObject'
)

谢谢你读了这么多。如果你有什么建议，请告诉我。如果这是一个副本(我自己找不到，但它们可能存在)，请标记为这样。而且，如果您自己发现上面的代码是对您的项目的一个很好的补充，请支持模块的作者。我不是他/她。这是他/她的执照：

Copyright (c) 2014, Elmer de Looff <elmer.delooff@gmail.com>
All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Answer 1

在网上查看后，sqlalchemy(感谢edelooff)是最好的选择，但是原产一只支持可变的dict，这叉子更新通过torotil解决问题，谢谢torotil！

并在TrackedObject中放置记录器造成深拷贝问题，最好移到模块上。我用一个新的存储库来解决这个问题：Cysnake4713

此外，我使用原产地sqlalchemy.sql.sqltypes.JSON而不是自定义JsonObject，似乎工作很好。

只需重新编写associate_with部件：

NestedMutable.associate_with(db.JSON)