Python requests.get返回错误json.decoder.JSONDecodeError

Question

我试图从无法控制的服务器读取大量数据。

错误：

json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 1794)

误差图像

此.json()调用引发错误json.decoder.JSONDecodeError json.decoder。

import requests
Data = requests.get(Data_Url ,headers=session["headers"]).json()
print(Data) 

这个.text返回我无法操作的字符串中的数据。

import requests
Data = requests.get(Data_Url ,headers=session["headers"]).text
print(Data) 

如下所示，数据格式似乎是

{}
{}
{}

如何操作request.get {},{}**?**响应，使之具有JSON格式，并由{},{}**？**分隔

{ "Data" : [{},{}]}

{"resourceType":"Person","id":"cg3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"3g3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"pg3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"GA3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"zQ3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"qQ3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"Fw3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"Nw3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"hw3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"DSQ3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}

Answer 1

您从.text获得的结果不是有效的JSON (或Python文字)格式。在研究了结果之后，我确定返回的字符串中的每一行都遗漏了结束时的"}]}]}“字符，从而纠正了这个问题。

下面的代码将它们添加到每一行中，然后使用ast.literal_eval()函数对其进行解析/计算，从而将其转换为Python字典。然后使用一个列表理解将它们放入列表中。换句话说，您不必费心将它们嵌套在字典中，比如您提议的{"Data": [{}, {}, ...]} (除非您确实出于某种未知的原因)。

from ast import literal_eval
import json
from pprint import pprint

requests_get_text = """\
{"resourceType":"Person","id":"cg3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"3g3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"pg3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"GA3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"zQ3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"qQ3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"Fw3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"Nw3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"hw3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
{"resourceType":"Person","id":"DSQ3","extension":[{"extension":[{"valueCoding":{"system":"http://terminology.hl7.org/","code":"UNK","display":"Unknown"}
"""

# Convert result from requests.get().text into valid JSON/Python format.
data = [literal_eval(f'{line}' '}]}]}\n')
                    for line in requests_get_text.splitlines()]
pprint(data, sort_dicts=False)

输出：

[{'resourceType': 'Person',
  'id': 'cg3',
  'extension': [{'extension': [{'valueCoding': {'system': 'http://terminology.hl7.org/',
                                                'code': 'UNK',
                                                'display': 'Unknown'}}]}]},
 {'resourceType': 'Person',
  'id': '3g3',
  'extension': [{'extension': [{'valueCoding': {'system': 'http://terminology.hl7.org/',
                                                'code': 'UNK',
                                                'display': 'Unknown'}}]}]},
 {'resourceType': 'Person',
  'id': 'pg3',
  'extension': [{'extension': [{'valueCoding': {'system': 'http://terminology.hl7.org/',
                                                'code': 'UNK',
                                                'display': 'Unknown'}}]}]},
 {'resourceType': 'Person',
  'id': 'GA3',
  'extension': [{'extension': [{'valueCoding': {'system': 'http://terminology.hl7.org/',
                                                'code': 'UNK',
                                                'display': 'Unknown'}}]}]},
 {'resourceType': 'Person',
  'id': 'zQ3',
  'extension': [{'extension': [{'valueCoding': {'system': 'http://terminology.hl7.org/',
                                                'code': 'UNK',
                                                'display': 'Unknown'}}]}]},
 {'resourceType': 'Person',
  'id': 'qQ3',
  'extension': [{'extension': [{'valueCoding': {'system': 'http://terminology.hl7.org/',
                                                'code': 'UNK',
                                                'display': 'Unknown'}}]}]},
 {'resourceType': 'Person',
  'id': 'Fw3',
  'extension': [{'extension': [{'valueCoding': {'system': 'http://terminology.hl7.org/',
                                                'code': 'UNK',
                                                'display': 'Unknown'}}]}]},
 {'resourceType': 'Person',
  'id': 'Nw3',
  'extension': [{'extension': [{'valueCoding': {'system': 'http://terminology.hl7.org/',
                                                'code': 'UNK',
                                                'display': 'Unknown'}}]}]},
 {'resourceType': 'Person',
  'id': 'hw3',
  'extension': [{'extension': [{'valueCoding': {'system': 'http://terminology.hl7.org/',
                                                'code': 'UNK',
                                                'display': 'Unknown'}}]}]},
 {'resourceType': 'Person',
  'id': 'DSQ3',
  'extension': [{'extension': [{'valueCoding': {'system': 'http://terminology.hl7.org/',
                                                'code': 'UNK',
                                                'display': 'Unknown'}}]}]}]