在python3中结合使用luigi.LocalTarget和PdfPages

Question

下面的代码片段适用于python2，但不适用于python3。此代码片段旨在允许luigi工作流写入多页PDF，同时仍使用允许原子性的LocalTarget上下文管理器。

import luigi
from matplotlib.backends.backend_pdf import PdfPages
import matplotlib.pyplot as plt

test = luigi.LocalTarget('test.pdf')
with test.open('wb') as fh, PdfPages(fh) as outf:
    plt = plt.plot([1, 2, 3], [4, 5, 6])

这在python2中有效，但在python3中会导致以下错误：

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-ba62e5b716d2> in <module>
----> 1 with test.open('wb') as fh, PdfPages(fh) as outf:
      2     plt = plt.plot([1, 2, 3], [4, 5, 6])

~/miniconda3/envs/cat3.7/lib/python3.7/site-packages/matplotlib/backends/backend_pdf.py in __init__(self, filename, keep_empty, metadata)
   2386 
   2387         """
-> 2388         self._file = PdfFile(filename, metadata=metadata)
   2389         self.keep_empty = keep_empty
   2390 

~/miniconda3/envs/cat3.7/lib/python3.7/site-packages/matplotlib/backends/backend_pdf.py in __init__(self, filename, metadata)
    445         self.fh = fh
    446         self.currentstream = None  # stream object to write to, if any
--> 447         fh.write(b"%PDF-1.4\n")    # 1.4 is the first version to have alpha
    448         # Output some eight-bit chars as a comment so various utilities
    449         # recognize the file as binary by looking at the first few

TypeError: write() argument must be str, not bytes

如何在python3中保留这种原子功能？

Answer 1

对不起，我知道这不是你想要的答案，但是我在Luigi仓库的LocalTarget定义中找到了这一行：

def open(self, mode='r'):
    rwmode = mode.replace('b', '').replace('t', '')
    ...

https://github.com/spotify/luigi/blob/master/luigi/local_target.py#L159

看起来他们没有做任何字节写入(至少在当前版本中)。我肯定会在Github问题上向他们提出这一点。

Answer 2

我不是LocalTarget工作方面的专家，所以我不知道删除b标志是否有原因，或者这是否是一个错误。

解决此问题的一种方法是使用temporary_path函数包装代码：

import luigi

class BinaryFileExample(luigi.Task):
    def output(self):
        return luigi.LocalTarget("simple_binary_file.extension")

    def run(self):
        with self.output().temporary_path() as my_binary_file_path:
            with open(my_binary_file_path, 'wb') as inner_file:
                newFileBytes = [123, 3, 255, 0, 100]
                for byte in newFileBytes:
                    inner_file.write(byte.to_bytes(1, byteorder='big'))