自定义命令从Django shell中将照片上传到Photologue？

Question

我已经成功地使用了Photologue来呈现定期创建的数据绘图图像的图库。当然，现在这个能力已经建立了，大量的数据地块正在被创建，它们需要被共享！

编写上传图像并使用manage.py从Django shell添加到图库的过程是下一步；然而，作为一个业余的Django，我遇到了一些困难。

下面是我目前开发的自定义命令addphoto.py：

from django.core.management.base import BaseCommand, CommandError
from django.utils import timezone
from photologue.models import Photo, Gallery

import os
from datetime import datetime
import pytz

class Command(BaseCommand):

    help = 'Adds a photo to Photologue.'

    def add_arguments(self, parser):
        parser.add_argument('imagefile', type=str)
        parser.add_argument('--title', type=str)
        parser.add_argument('--date_added', type=str, help="datetime string in 'YYYY-mm-dd HH:MM:SS' format [UTC]")
        parser.add_argument('--gallery', type=str)

    def handle(self, *args, **options):

        imagefile = options['imagefile']

        if options['title']:
            title = options['title']
        else:
            base = os.path.basename(imagefile)
            title = os.path.splitext(base)[0]
        if options['date_added']:
            date_added = datetime.strptime(options['date_added'],'%Y-%m-%d %H:%M:%S').replace(tzinfo=pytz.UTC)
        else:
            date_added = timezone.now()

        p = Photo(image=imagefile, title=title, date_added=date_added)
        p.save()

不幸的是，当使用--traceback执行时，结果如下：

./manage.py addphoto '../dataplots/monitoring/test.png' --traceback
Failed to read EXIF DateTimeOriginal
Traceback (most recent call last):
  File "/home/user/mysite/photologue/models.py", line 494, in save
    exif_date = self.EXIF(self.image.file).get('EXIF DateTimeOriginal', None)
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/db/models/fields/files.py", line 51, in _get_file
    self._file = self.storage.open(self.name, 'rb')
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/core/files/storage.py", line 38, in open
    return self._open(name, mode)
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/core/files/storage.py", line 300, in _open
    return File(open(self.path(name), mode))
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/core/files/storage.py", line 405, in path
    return safe_join(self.location, name)
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/utils/_os.py", line 78, in safe_join
    'component ({})'.format(final_path, base_path))
django.core.exceptions.SuspiciousFileOperation: The joined path (/home/user/mysite/dataplots/monitoring/test.png) is located outside of the base path component (/home/user/mysite/media)
Traceback (most recent call last):
  File "./manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line
    utility.execute()
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/core/management/__init__.py", line 355, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/home/user/mysite/photologue/management/commands/addphoto.py", line 36, in handle
    p.save()
  File "/home/user/mysite/photologue/models.py", line 553, in save
    super(Photo, self).save(*args, **kwargs)
  File "/home/user/mysite/photologue/models.py", line 504, in save
    self.pre_cache()
  File "/home/user/mysite/photologue/models.py", line 472, in pre_cache
    self.create_size(photosize)
  File "/home/user/mysite/photologue/models.py", line 411, in create_size
    if self.size_exists(photosize):
  File "/home/user/mysite/photologue/models.py", line 364, in size_exists
    if self.image.storage.exists(func()):
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/core/files/storage.py", line 392, in exists
    return os.path.exists(self.path(name))
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/core/files/storage.py", line 405, in path
    return safe_join(self.location, name)
  File "/home/user/mysite/venv/lib/python3.5/site-packages/django/utils/_os.py", line 78, in safe_join
    'component ({})'.format(final_path, base_path))
django.core.exceptions.SuspiciousFileOperation: The joined path (/home/user/mysite/dataplots/monitoring/cache/test_thumbnail.png) is located outside of the base path component (/home/user/mysite/media)

显然，图像文件的副本没有放在media/目录中。另外，虽然image、title和date_added列是在网站数据库的photologue_photos表中填充的，但slug列不是。

如何将文件上载到MEDIA_ROOT目录？

下面是来自Photologue Photo文件中的ImageModel和models.py模型的相关片段，以供参考：

class Photo(ImageModel):
    title = models.CharField(_('title'),
                         max_length=250,
                         unique=True)
    slug = models.SlugField(_('slug'),
                        unique=True,
                        max_length=250,
                        help_text=_('A "slug" is a unique URL-friendly title for an object.'))
    caption = models.TextField(_('caption'),
                               blank=True)
    date_added = models.DateTimeField(_('date added'),
                                      default=now)
    is_public = models.BooleanField(_('is public'),
                                    default=True,
                                    help_text=_('Public photographs will be displayed in the default views.'))
    sites = models.ManyToManyField(Site, verbose_name=_(u'sites'),
                                   blank=True)

    objects = PhotoQuerySet.as_manager()

    def save(self, *args, **kwargs):
        if self.slug is None:
            self.slug = slugify(self.title)
        super(Photo, self).save(*args, **kwargs)


class ImageModel(models.Model):
    image = models.ImageField(_('image'),
                          max_length=IMAGE_FIELD_MAX_LENGTH,
                          upload_to=get_storage_path)
    date_taken = models.DateTimeField(_('date taken'),
                                  null=True,
                                  blank=True,
                                  help_text=_('Date image was taken; is obtained from the image EXIF data.'))
    view_count = models.PositiveIntegerField(_('view count'),
                                         default=0,
                                         editable=False)
    crop_from = models.CharField(_('crop from'),
                             blank=True,
                             max_length=10,
                             default='center',
                             choices=CROP_ANCHOR_CHOICES)
    effect = models.ForeignKey('photologue.PhotoEffect',
                           null=True,
                           blank=True,
                           related_name="%(class)s_related",
                           verbose_name=_('effect'))

    class Meta:
        abstract = True

    def __init__(self, *args, **kwargs):
        super(ImageModel, self).__init__(*args, **kwargs)
        self._old_image = self.image

    def save(self, *args, **kwargs):
        image_has_changed = False
        if self._get_pk_val() and (self._old_image != self.image):
            image_has_changed = True
            # If we have changed the image, we need to clear from the cache all instances of the old
            # image; clear_cache() works on the current (new) image, and in turn calls several other methods.
            # Changing them all to act on the old image was a lot of changes, so instead we temporarily swap old
            # and new images.
            new_image = self.image
            self.image = self._old_image
            self.clear_cache()
            self.image = new_image  # Back to the new image.
            self._old_image.storage.delete(self._old_image.name)  # Delete (old) base image.
        if self.date_taken is None or image_has_changed:
            # Attempt to get the date the photo was taken from the EXIF data.
            try:
                exif_date = self.EXIF(self.image.file).get('EXIF DateTimeOriginal', None)
                if exif_date is not None:
                    d, t = exif_date.values.split()
                    year, month, day = d.split(':')
                    hour, minute, second = t.split(':')
                    self.date_taken = datetime(int(year), int(month), int(day),
                                           int(hour), int(minute), int(second))
            except:
                logger.error('Failed to read EXIF DateTimeOriginal', exc_info=True)
        super(ImageModel, self).save(*args, **kwargs)
        self.pre_cache()

以下是请求的get_storage_path函数：

# Look for user function to define file paths
PHOTOLOGUE_PATH = getattr(settings, 'PHOTOLOGUE_PATH', None)
if PHOTOLOGUE_PATH is not None:
    if callable(PHOTOLOGUE_PATH):
        get_storage_path = PHOTOLOGUE_PATH
    else:
        parts = PHOTOLOGUE_PATH.split('.')
        module_name = '.'.join(parts[:-1])
        module = import_module(module_name)
        get_storage_path = getattr(module, parts[-1])
else:
    def get_storage_path(instance, filename):
        fn = unicodedata.normalize('NFKD', force_text(filename)).encode('ascii', 'ignore').decode('ascii')
        return os.path.join(PHOTOLOGUE_DIR, 'photos', fn)

Answer 1

下面是我创建的定制addphoto.py命令的工作版本。

为了便于导入，图像文件需要在MEDIA_ROOT/photologue/photos中。该命令使用./manage.py addphoto 'photologue/photos/test.png'执行。请注意，有一个--gallery选项可以将图像添加到图片库，前提是该图库的slug。

from django.core.management.base import BaseCommand, CommandError
from django.utils import timezone
from photologue.models import Photo, Gallery

import os
from datetime import datetime
import pytz

class Command(BaseCommand):

    help = 'Adds a photo to Photologue.'

    def add_arguments(self, parser):
        parser.add_argument('imagefile',
                            type=str)
        parser.add_argument('--title',
                            type=str)
        parser.add_argument('--date_added',
                            type=str,
                            help="datetime string in 'YYYY-mm-dd HH:MM:SS' format [UTC]")
        parser.add_argument('--gallery',
                            type=str)

    def handle(self, *args, **options):
        imagefile = options['imagefile']
        base = os.path.basename(imagefile)

        if options['title']:
            title = options['title']
        else:
            title = os.path.splitext(base)[0]
        if options['date_added']:
            date_added = datetime.strptime(options['date_added'],'%Y-%m-%d %H:%M:%S').replace(tzinfo=pytz.UTC)
        else:
            date_added = timezone.now()

        try:
            p = Photo(image=imagefile, title=title, slug=title, date_added=date_added)
        except:
            raise CommandError('Photo "%s" could not be added' % base)
        p.save()
        self.stdout.write(self.style.SUCCESS('Successfully added photo "%s"' % p))

        if options['gallery']:
            try:
                g = Gallery.objects.get(slug=options['gallery'])
            except:
                raise CommandError('Gallery "%s" does not exist' % options['gallery'])
            p.galleries.add(g.pk)
            p.save()
            self.stdout.write(self.style.SUCCESS('Successfully added photo to gallery "%s"' % g))

Answer 2

关于问题的一个部分:保存slug列时，Photo列为空。

当Photo被保存时，应该自动填充它--正如您在if self.slug is None: self.slug = slugify(self.title)上面的照片源代码的复制和粘贴所表明的那样。

这意味着实际上并不是从管理命令中调用Photologue源代码--您可以通过将一些快速调试代码添加到Photologue代码的本地副本(例如save()方法中的print()语句)中来检查它，并检查它是否正在运行。

Answer 3

编辑/更新2:我想我知道为什么你的弹格字段是空的。我认为问题在于self.slug不是None，即使字段不包含字符串或空字符串(参见这个答案)。因此，尝试将if self.slug is None更新为：

class Photo(ImageModel):
    ...
    def save(self, *args, **kwargs):
        # test if slug is not truthy, including empty string or None-ness
        if not self.slug:
            self.slug = slugify(self.title)
        super(Photo, self).save(*args, **kwargs)

编辑/更新1:签出这个答案。它来自Django 1.4 (我知道是古代的)，但是它应该能解决你的问题。如果您在创建MEDIA_ROOT实例之前复制或移动要添加到Photo中的文件，那么您应该可以继续工作。这是一个答案展示了如何在python中复制文件。我建议您将自定义命令更改为：

from django.core.management.base import BaseCommand, CommandError
from django.utils import timezone
from photologue.models import Photo, Gallery

# so you can access settings.MEDIA_ROOT
from django.conf import settings

# so you can copy file to MEDIA_ROOT if need be
from shutil import copyfile

import os
from datetime import datetime
import pytz

class Command(BaseCommand):

    help = 'Adds a photo to Photologue.'

    def add_arguments(self, parser):
        parser.add_argument('imagefile', type=str)

        # where the imagefile is currently located, assumes MEDIA_ROOT
        parser.add_argument('--media_source', type=str)
        parser.add_argument('--title', type=str)
        parser.add_argument('--date_added', type=str, help="datetime string in 'YYYY-mm-dd HH:MM:SS' format [UTC]")
        parser.add_argument('--gallery', type=str)

    def handle(self, *args, **options):

        # the path of the file relative to media_source
        imagefile = options['imagefile']

        # if file is not in media root, copy it to there
        if options['media_source']:
            media_source = os.path.realpath(options['media_source'])
            media_target = os.path.realpath(settings.MEDIA_ROOT)
            if media_source != media_target:
                copyfile(imagefile, os.path.join(media_target, imagefile)
        # note: if media_source was not provided, assume file is already in MEDIA_ROOT

        if options['title']:
            title = options['title']
        else:
            base = os.path.basename(imagefile)
            title = os.path.splitext(base)[0]
        if options['date_added']:
            date_added = datetime.strptime(options['date_added'],'%Y-%m-%d %H:%M:%S').replace(tzinfo=pytz.UTC)
        else:
            date_added = timezone.now()

        p = Photo(image=imagefile, title=title, date_added=date_added)
        p.save()

现在，您的imagefile相对于您的媒体源(../dataplots)，被复制到MEDIA_ROOT，并且一切都应该按计划工作。以下是您的命令将是什么样子

manage.py addphoto 'monitoring/test.png' --media_source='../dataplots'

它应该将数据图复制到MEDIA_ROOT中，然后按照预期创建Photo实体。

原始答案：

您能在下面的一行中发布什么get_storage_path吗？

class ImageModel(models.Model):
     image = models.ImageField(_('image'),
                          max_length=IMAGE_FIELD_MAX_LENGTH,
                          upload_to=get_storage_path)  # <-- here

在我知道这是什么之前，这个答案将是不完整的，但我想我看到了你的问题。查看正在运行的命令：

./manage.py addphoto '../dataplots/monitoring/test.png' --traceback

您的imagefile参数是../dataplots/monitoring/test.png。如果get_storage_path返回相同的路径，并且../位于路径的开头，那么您将指定一个上传路径，该路径不是在MEDIA_ROOT目录中，而是在其父目录中。我认为它正在尝试上传到MEDIA_ROOT/../dataplots/monitoring/test.png，如您的回溯中的第一个SuspiciousFileOperation所示：

# note: linebreaks and indentation added for readability
django.core.exceptions.SuspiciousFileOperation: 
    The joined path (/home/user/mysite/dataplots/monitoring/test.png) 
    is located outside of the base path component (/home/user/mysite/media)

因此，它试图上传到MEDIA_ROOT/imagefile，但imagefile以../开头，这是不应该允许的。

如果这确实是问题所在(在发布get_storage_path代码之前很难说)，那么解决问题的方法有很多。也许最快的解决方法就是将dataplots目录移动到与manage.py相同的目录中。

mv ../dataplots ./dataplots

这应该可以立即解决您的问题，因为您不再需要../了，但是您可能不需要项目目录中的所有这些数据图，所以这是一个快速但很弱的解决方案。根本的问题是源文件的路径和上传源文件的路径不应该是相等的。我认为您应该将命令参数更改为包含image_source和image_destination，或者可以包含source_media_root和imagefile，其中../是source_media_root的一部分，imagefile相对于source_media_root和MEDIA_ROOT中所需的目标位置。有很多解决方案，但在我知道get_storage_path是什么之前，我无法为其中一个提供适当的代码(假设它是一个函数，可以以某种方式返回或返回imagefile参数)。