如何从JSON格式的PDF文件中提取基于部件的文本？

Question

从这个文件https://github.com/pymupdf/PyMuPDF-Utilities/blob/master/text-extraction/Dart.pdf

​

我想得到这样的类型的结果：

{
    "file": {
        "title": "Dart Programming Language Specification",
        "1 Scope": {
            "text": "This Ecma standard specifies the syntax and semantics of the Dart program-ming language. It does not specify the APIs of the Dart libraries except where those library elements are essential to the correct functioning of the language itself (e.g, the existence of class Object with methods such as noSuchlethod, runtimeType."
        },
        "2 Conformance": {
            "text": "A conforming implementation of the Dart programming language must pro-vide and support all the APIs (libraries, types, functions, getters, setters, whether top-level, static, instance or local) mandated in this specification. A conforming implementation is permitted to provide additional APIs, but not additional syntax, except for experimental features in support of null-aware cascades that are likelv to be introduced in the next revision of this specification."
        },
        "3 Normative References": [
            {
                "text": "The following referenced documents are indispensable for the application of this document. For dated references, only the edition cited applies. undated references, the latest edition of the referenced document (including any amendments) applies.",
                "1": "The Unicode Standard, Version 5.0, as amended by      Unicode 5.1.0, or successor.",
                "2": "Dart API Reference, https://api.dartlang.org/"
            }
        ]
        ...
    }
}

1. 我的第一个想法是使用Detectron2和库(如深度保护和布局解析器 )，使用OCR技术(尤其是OCR技术)，使用深度学习来执行布局检测。但是经过一些测试后，对文本布局的检测似乎没有被考虑在内。我只能提取标题，文本框和表格的“整体布局”。随后必须根据箱子的坐标对检测进行排序。
2. 我的第二个想法是首先使用保留布局的文本提取方法将PDF文件转换为文本文件。有几种解决方案，比如PyMuPDF。然后对生成包含部分(标题、字幕、文本等)的大纲的文本文件执行处理。作为字典，根据，文件的空格。但这个解决方案似乎不健全，因为我可以有pdf文件，其中没有缩进的部分标题。

是否有一种方法可以执行的布局检测:布局的每个部分与、文本和这些相关的子部件

Answer 1

一旦有了文本，就可以将其转换为json文本，例如https://www.npmjs.com/package/text-2-json (PDF二进制文件首先需要转换为文本)。

为了保持缩进，文本字符串需要替换行(RTL或LTR)两边的空隙，一种方法是使用文本HTML作为输出，而不是纯文本。当然不要使用OCR，如果您已经有PDF结构和样式的字符。运行这个PDF2HTM输出，看看它有多好地模拟PDF (类似于PDF.js文本的方法，但没有js)。正文为10 so，标题为14.3pt

​

body {background-color:slategray}
div {position:relative;background-color:white;margin:1em auto;box-shadow:1px 1px 8px -2px black}
p {position:absolute;white-space:pre;margin:0}

<div id="page1" style="width:612.0pt;height:792.0pt">
<p style="top:91.9pt;left:133.8pt;line-height:10.0pt"><i><span style="font-family:LMRomanSlant10,serif;font-size:10.0pt">Dart Programming Language Specification</span></i></p>
<p style="top:91.9pt;left:472.5pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">6</span></p>
<p style="top:123.3pt;left:133.8pt;line-height:14.3pt"><b><span style="font-family:LMRoman12,serif;font-size:14.3pt">1</span></b></p>
<p style="top:123.3pt;left:158.0pt;line-height:14.3pt"><b><span style="font-family:LMRoman12,serif;font-size:14.3pt">Scope</span></b></p>
<p style="top:132.3pt;left:498.4pt;line-height:2.0pt"><span style="font-family:LMRoman5,serif;font-size:2.0pt;color:#ffffff">ecmaScope</span></p>
<p style="top:148.6pt;left:148.7pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">This Ecma standard specifies the syntax and semantics of the Dart program-</span></p>
<p style="top:160.6pt;left:133.8pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">ming language. It does not specify the APIs of the Dart libraries except where</span></p>
<p style="top:172.5pt;left:133.8pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">those library elements are essential to the correct functioning of the language</span></p>
<p style="top:184.5pt;left:133.8pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">itself (e.g., the existence of class</span><tt><span style="font-family:LMMono10,monospace;font-size:10.0pt"> Object</span></tt><span style="font-family:LMRoman10,serif;font-size:10.0pt"> with methods such as</span><tt><span style="font-family:LMMono10,monospace;font-size:10.0pt"> noSuchMethod</span></tt><span style="font-family:LMRoman10,serif;font-size:10.0pt">,</span></p>
<p style="top:196.4pt;left:133.8pt;line-height:10.0pt"><tt><span style="font-family:LMMono10,monospace;font-size:10.0pt">runtimeType</span></tt><span style="font-family:LMRoman10,serif;font-size:10.0pt">).</span></p>
<p style="top:225.7pt;left:133.8pt;line-height:14.3pt"><b><span style="font-family:LMRoman12,serif;font-size:14.3pt">2</span></b></p>
<p style="top:225.7pt;left:158.0pt;line-height:14.3pt"><b><span style="font-family:LMRoman12,serif;font-size:14.3pt">Conformance</span></b></p>
<p style="top:234.7pt;left:498.4pt;line-height:2.0pt"><span style="font-family:LMRoman5,serif;font-size:2.0pt;color:#ffffff">ecmaConformance</span></p>
<p style="top:251.0pt;left:148.7pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">A conforming implementation of the Dart programming language must pro-</span></p>
<p style="top:262.9pt;left:133.8pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">vide and support all the APIs (libraries, types, functions, getters, setters, whether</span></p>
<p style="top:274.9pt;left:133.8pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">top-level, static, instance or local) mandated in this specification.</span></p>
<p style="top:286.8pt;left:148.7pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">A conforming implementation is permitted to provide additional APIs, but</span></p>
<p style="top:298.8pt;left:133.8pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">not additional syntax, except for experimental features in support of null-aware</span></p>
<p style="top:310.8pt;left:133.8pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">cascades that are likely to be introduced in the next revision of this specification.</span></p>
<p style="top:340.0pt;left:133.8pt;line-height:14.3pt"><b><span style="font-family:LMRoman12,serif;font-size:14.3pt">3</span></b></p>
<p style="top:340.0pt;left:158.0pt;line-height:14.3pt"><b><span style="font-family:LMRoman12,serif;font-size:14.3pt">Normative References</span></b></p>
<p style="top:349.0pt;left:498.4pt;line-height:2.0pt"><span style="font-family:LMRoman5,serif;font-size:2.0pt;color:#ffffff">ecmaNormativeReferences</span></p>
<p style="top:365.3pt;left:148.7pt;line-height:10.0pt"><span style="font-family:LMRoman10,serif;font-size:10.0pt">The following referenced documents are indispensable for the application</span></p>

​

注意：有一些2.0pt的白色文本RH边(标记？)

1 Scope ecmaScope

在缩进的html中这将是白色的，并且可以增加非缩进xml/stext/txt提取的复杂性，(将下面的文本视图移动为文本)，所以最好的缩进源如下所示

mutool convert -pretty -o dart.html dart.pdf

然而，这将是单行，就像在pdf。第二个最好的选择是pdftotext.exe -layout dart.pdf，但是您需要解析为文本。

     Dart Programming Language Specification                                           6


     1     Scope                                                                                 ecmaScope




         This Ecma standard specifies the syntax and semantics of the Dart program-
     ming language. It does not specify the APIs of the Dart libraries except where
     those library elements are essential to the correct functioning of the language
     itself (e.g., the existence of class Object with methods such as noSuchMethod,
     runtimeType).


     2     Conformance                                                                           ecmaConformance




        A conforming implementation of the Dart programming language must pro-
     vide and support all the APIs (libraries, types, functions, getters, setters, whether
     top-level, static, instance or local) mandated in this specification.
        A conforming implementation is permitted to provide additional APIs, but
     not additional syntax, except for experimental features in support of null-aware
     cascades that are likely to be introduced in the next revision of this specification.


     3     Normative References                                                                  ecmaNormativeReferences