Python2.7+ Django 1.7 + PostgreSQL 9.3:当我试图将一些文本保存到数据库时，我得到了一个UnicodeEncodeError。怎么回事？

Question

在我的models.py文件中，我定义了一个模型，大致如下：

class MyThing(models.Model):
    thing_id     = models.CharField(max_length=20, unique=True, null=False)
    code_snippet = models.TextField(null=True)

我试图用从Google中提取的一堆数据填充这个表。其中一段是JavaScript的片段。从这个API返回的项目之一有一个JS片段，它正在抛出一个错误。

我得到的错误是：

UnicodeEncodeError：'ascii‘编解码器不能在213-214位置编码字符:序数不在范围内(128个)

本节中出现了一些令人不快的文本：

type="text/javascript" charset="utf-\xad\u20108"></script>

我几乎可以肯定是\xad，它一定是某种unicode转义序列。

我检查了我的PostgreSQL编码，它使用的是UTF-8。

我还将这一行放在我的models.py的顶部，但这并没有什么区别：

from __future__ import unicode_literals

这到底是怎么回事？为什么我的unicode感知数据库不能保存这个字符串？另外，我什么时候开始使用'ascii' codec的？

--编辑:下面的全堆栈跟踪

ERROR/MainProcess] Task apps.r2d_service.tasks.sync_jsnippets_task[e74712bf-2e04-4bed-b08c-f24f9ebb3049] raised unexpected: UnicodeEncodeError('ascii', u'<script type="text/javascript">\n    window._sm_plcmnt = "finyo_5225_5119";\n    var _sm_viewed    = false;\n</script>\n<script src="https://cdn.company412media.com/ng/js/augur.js" type="text/javascript" charset="utf-\xad\u20108"></script>\n<script src=\'https://cdn.company412media.com/ng/pub.js\'></script>\n<script type="text/javascript">\n$$(\'#bib_actions_table tr:nth-child(1) a\').each(function (tab) {\n  tab.observe(\'click\', function (e, el) {\n    if (!_sm_viewed) {\n      SM_Augur.init();\n      _sm_viewed = true;\n    }\n  });\n});\n\n</script>\n', 213, 215, 'ordinal not in range(128)')
Traceback (most recent call last):
  File "/home/vagrant/.virtualenvs/myapp/local/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/vagrant/.virtualenvs/myapp/local/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/vagrant/myapp/apps/r2d_service/tasks.py", line 25, in sync_jsnippets_task
    sync_jsnippets()
  File "/home/vagrant/myapp/apps/r2d_service/sync.py", line 44, in sync_jsnippets
    myapp_creative.sync(r2d_creative)
  File "/home/vagrant/myapp/apps/r2d_service/models.py", line 244, in sync
    self.save()
  File "/home/vagrant/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/db/models/base.py", line 590, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/vagrant/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/db/models/base.py", line 618, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/home/vagrant/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/db/models/base.py", line 680, in _save_table
    forced_update)
  File "/home/vagrant/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/db/models/base.py", line 724, in _do_update
    return filtered._update(values) > 0
  File "/home/vagrant/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/db/models/query.py", line 600, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/home/vagrant/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1004, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/home/vagrant/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
    cursor.execute(sql, params)
  File "/home/vagrant/.virtualenvs/myapp/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 90, in execute
    logger.debug('(%.3f) %s; args=%s' % (duration, sql, params),
UnicodeEncodeError: 'ascii' codec can't encode characters in position 213-214: ordinal not in range(128)

Answer 1

您需要使用base64转换这些字符串，然后可以将其保存到数据库中。

import base64

base64.b64encode("stings")