手动将行添加到StreamingHttpResponse (Django)

Question

我正在使用Django的StreamingHttpResponse动态地流一个大型的CSV文件。根据医生们，迭代器被传递给响应的streaming_content参数：

import csv
from django.http import StreamingHttpResponse

def get_headers():
    return ['field1', 'field2', 'field3']

def get_data(item):
    return {
        'field1': item.field1,
        'field2': item.field2,
        'field3': item.field3,
    }

# StreamingHttpResponse requires a File-like class that has a 'write' method
class Echo(object):
    def write(self, value):
        return value


def get_response(queryset):
    writer = csv.DictWriter(Echo(), fieldnames=get_headers())
    writer.writeheader() # this line does not work

    response = StreamingHttpResponse(
        # the iterator
        streaming_content=(writer.writerow(get_data(item)) for item in queryset),
        content_type='text/csv',
    )
    response['Content-Disposition'] = 'attachment;filename=items.csv'

    return response

我的问题是:如何在CSV编写器上手动编写行？手动调用writer.writerow( data )或writer.writeheader() (也在内部调用writerow())似乎没有写入数据集，而是只在输出数据集中写入来自streaming_content的生成/流数据。

Answer 1

答案是使用生成器函数生成结果，而不是动态计算它们(在StreamingHttpResponse的streaming_content参数内)，并使用我们创建的伪缓冲区(Echo )来将一行写入响应：

import csv
from django.http import StreamingHttpResponse

def get_headers():
    return ['field1', 'field2', 'field3']

def get_data(item):
    return {
        'field1': item.field1,
        'field2': item.field2,
        'field3': item.field3,
    }

# StreamingHttpResponse requires a File-like class that has a 'write' method
class Echo(object):
    def write(self, value):
        return value

def iter_items(items, pseudo_buffer):
    writer = csv.DictWriter(pseudo_buffer, fieldnames=get_headers())
    yield pseudo_buffer.write(get_headers())

    for item in items:
        yield writer.writerow(get_data(item))

def get_response(queryset):
    response = StreamingHttpResponse(
        streaming_content=(iter_items(queryset, Echo())),
        content_type='text/csv',
    )
    response['Content-Disposition'] = 'attachment;filename=items.csv'
    return response

Answer 2

建议的解决方案实际上可能导致不正确/不匹配的CSV(头与数据不匹配)。您可能希望用以下内容替换受影响的部分：

header = dict(zip(fieldnames, fieldnames))
yield writer.writerow(header)

而不是。这来自于writeheader https://github.com/python/cpython/blob/08045391a7aa87d4fbd3e8ef4c852c2fa4e81a8a/Lib/csv.py#L141:L143的实现。

出于某种原因，它对yield表现不好

希望这对将来的人有帮助:)

还请注意，如果使用python 3.8+，则不需要修复，因为这个PR：https://bugs.python.org/issue27497