pandasdmx:导致SSL错误的sdmx.Request -握手错误

Question

我正在使用pandasdmx库，并且想要访问IMF数据源。下面的代码生成一个SSLError。

from pathlib import Path

import pandasdmx as sdmx

http_proxy = "my/proxy_server/address"
proxies = {
    "http": http_proxy,
    "https": http_proxy,
}
ssl_cert = Path("path/to/my/ssl_certificate")  # .pem file
data_source = "IMF"

src = sdmx.Request(
    data_source,
    proxies=proxies,
    verify=ssl_cert,
    backend="sqlite",
    fast_save=True,
    expire_after=600,
)

flow_msg = src.dataflow()


Traceback (most recent call last):
  File "C:\Users\D292498\AppData\Local\conda\conda\envs\py38\lib\site-packages\IPython\core\interactiveshell.py", line 3331, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-f5bd59ed1595>", line 1, in <module>
    runfile('C:/Users/D292498/.PyCharmCE2019.2/config/scratches/scratch_1.py', wdir='C:/Users/D292498/.PyCharmCE2019.2/config/scratches')
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.2\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.2\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/D292498/.PyCharmCE2019.2/config/scratches/scratch_1.py", line 24, in <module>
    flow_msg = src.dataflow()
  File "C:\Users\D292498\AppData\Local\conda\conda\envs\py38\lib\site-packages\pandasdmx\api.py", line 392, in get
    raise e from None
  File "C:\Users\D292498\AppData\Local\conda\conda\envs\py38\lib\site-packages\pandasdmx\api.py", line 389, in get
    response = self.session.send(req)
  File "C:\Users\D292498\AppData\Local\conda\conda\envs\py38\lib\site-packages\requests_cache\core.py", line 109, in send
    return send_request_and_cache_response()
  File "C:\Users\D292498\AppData\Local\conda\conda\envs\py38\lib\site-packages\requests_cache\core.py", line 97, in send_request_and_cache_response
    response = super(CachedSession, self).send(request, **kwargs)
  File "C:\Users\D292498\AppData\Local\conda\conda\envs\py38\lib\site-packages\requests\sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\D292498\AppData\Local\conda\conda\envs\py38\lib\site-packages\requests\adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='sdmxcentral.imf.org', port=443): Max retries exceeded with url: /ws/public/sdmxapi/rest/dataflow/IMF/latest (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))

但是，我可以很容易地在浏览器中加载网站/xml文档https://sdmxcentral.imf.org/ws/public/sdmxapi/rest/dataflow/IMF/latest。此外，使用常规的request库似乎可以正常工作。下面的代码生成status code 200。

from pathlib import Path

import requests

proxy_url = "my/proxy_server/address"
proxies = {"http": proxy_url, "https": proxy_url}
ssl_cert = Path("path/to/my/ssl_certificate")  # .pem file
web_address = "https://sdmxcentral.imf.org/ws/public/sdmxapi/rest/dataflow/IMF/latest"

r = requests.get(web_address, proxies=proxies, verify=ssl_cert)
print(r.status_code)  # 200 OK

Answer 1

显然，sdmx.Request()没有正确地转发verify参数。

因此，对基础request库的调用将使用其标准CA证书。因此，我们需要将证书信息添加到该特定文件中。

下面几行代码应该可以解决这个问题：

from pathlib import Path

import certifi

ssl_cert = Path("path/to/my/ssl_certificate")  # .pem file

cafile = certifi.where()

with open(ssl_cert, 'rb') as infile:
    customca = infile.read()

with open(cafile, 'ab') as outfile:
    outfile.write(customca)

请在https://incognitjoe.github.io/adding-certs-to-requests.html上找到更多细节。

Answer 2

显然，sdmx.Request()没有正确地转发verify参数。

在sdmx1-a中，更积极维护的pandaSDMX-this的分支是noted in an issue on 2020-11-20和fixed in v1.6.0 on 2020-12-16。现在支持verify，因此不再需要在另一个答案中手动添加证书。