CERTIFICATE_VERIFY_FAILED错误

Question

错误：

异常位置: /usr/lib/python3.8/urllib/request.py，第1357行，在do_open中

运行: Python 3.8.10，Django 4.0.3，Ubuntu 20.04，Apache 2

我使用Django作为一个简单的联系人表单应用程序，该应用程序目前工作正常，不会抛出任何错误。当我使用这个Django库https://github.com/tiesjan/django-hcaptcha-field向表单添加hCaptcha时，会发生C1错误。

这个问题似乎与django- hCaptcha字段无关；当访问hCaptcha API时，它似乎是Ubuntu证书。

我看过许多关于这样的问题(尤指)。https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error，尽管它是用于OS的)，并询问Ubuntu 证书问题。

我试过所有这些“修正”：

pip install pyOpenSSL --upgrade
apt-get install --reinstall python3-certifi
pip install --upgrade certifi --force
apt install --reinstall openssl
apt install ca-certificates
update-ca-certificates --fresh
export SSL_CERT_DIR=/etc/ssl/certs

我“强制”更新了我的--让我们加密SSLs。

我试过更新证书：

wget --quiet https://curl.haxx.se/ca/cacert.pem
export SSL_CERT_FILE=$HOME/cacert.pem

我的Python代码中没有任何东西需要import ssl

What其他的我可以试试吗？

诊断产出：

dpkg -l | grep cert返回

ica-certificates    20210119~20.04.2    all    Common CA certificates
certbot    0.40.0-1ubuntu0.1    all    automatically configure HTTPS using Let's Encrypt
dirmngr    2.2.19-3ubuntu2.1    amd64    GNU privacy guard - network certificate management service
python-certbot-apache    0.36.0-1    all    transitional dummy package
python3-certbot    0.40.0-1ubuntu0.1    all   main library for certbot
python3-certbot-apache   0.39.0-1    all    Apache plugin for Certbot
ipython3-certifi    2019.11.28-1    all    root certificates for validating SSL certs and verifying TLS hosts (python3)
ssl-cert    1.0.39    all    simple debconf wrapper for OpenSSL

dpkg -l | grep openssl返回

libxmlsec1-openssl:amd64    1.2.28-2    amd64        Openssl engine for the XML security library
openssl    1.1.1f-1ubuntu2.12    amd64        Secure Sockets Layer toolkit - cryptographic utility
perl-openssl-defaults:amd64    4    amd64        version compatibility baseline for Perl OpenSSL packages
python3-openssl    19.0.0-1build1    all          Python 3 wrapper around the OpenSSL library

whereis openssl返回

openssl: /usr/bin/openssl /usr/local/bin/openssl /usr/include/openssl /usr/share/man/man1/openssl.1ssl.gz

which openssl /usr/bin/openssl返回

/usr/local/bin/openssl
/usr/bin/openssl

ldd $(which wget)返回

linux-vdso.so.1 (0x00007ffd9f10f000)
libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007efdb3e3d000)
libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007efdb3e34000)
libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007efdb3e12000)
libssl.so.1.1 => /usr/local/lib/libssl.so.1.1 (0x00007efdb3d7a000)
libcrypto.so.1.1 => /usr/local/lib/libcrypto.so.1.1 (0x00007efdb3a8e000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007efdb3a72000)
libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x00007efdb3a5d000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007efdb386b000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007efdb3848000)
/lib64/ld-linux-x86-64.so.2 (0x00007efdb3f6b000)
libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007efdb36c6000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007efdb36c0000)

dpkg -l | grep python3-certifi返回

python3-certifi    2019.11.28-1    all  root certificates for validating SSL certs and verifying TLS hosts (python3)

追溯：

Request Method: POST
Request URL: https://example.com/contact/contact/contact/

Django Version: 4.0.3
Python Version: 3.8.10
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'contactform.apps.ContactformConfig',
 'encrypted_files',
 'hcaptcha_field']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback (most recent call last):
  File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/usr/lib/python3.8/http/client.py", line 1425, in connect
    self.sock = self._context.wrap_socket(self.sock,
  File "/usr/lib/python3.8/ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/lib/python3.8/ssl.py", line 1040, in _create
    self.do_handshake()
  File "/usr/lib/python3.8/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()

During handling of the above exception ([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)), another exception occurred:
  File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/var/www/html/example.com/public_html/contact/contactform/views.py", line 26, in contact
    if form.is_valid():
  File "/usr/local/lib/python3.8/dist-packages/django/forms/forms.py", line 205, in is_valid
    return self.is_bound and not self.errors
  File "/usr/local/lib/python3.8/dist-packages/django/forms/forms.py", line 200, in errors
    self.full_clean()
  File "/usr/local/lib/python3.8/dist-packages/django/forms/forms.py", line 433, in full_clean
    self._clean_fields()
  File "/usr/local/lib/python3.8/dist-packages/django/forms/forms.py", line 445, in _clean_fields
    value = field.clean(value)
  File "/usr/local/lib/python3.8/dist-packages/django/forms/fields.py", line 199, in clean
    self.validate(value)
  File "/usr/local/lib/python3.8/dist-packages/hcaptcha_field/fields.py", line 129, in validate
    response = opener.open(request, timeout=hcaptcha_settings.TIMEOUT)
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open
    response = self._open(req, data)
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 1397, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
    raise URLError(err)

Exception Type: URLError at /contact/contact/
Exception Value: 

fields.py of hcaptcha_field of https://github.com/tiesjan/django-hcaptcha-field

import json
import logging
import ssl # added #############
import certifi # added #############
from urllib.error import HTTPError
from urllib.parse import urlencode
from urllib.request import build_opener, Request, ProxyHandler

from django import forms
from django.core.exceptions import ValidationError
from django.utils.translation import gettext_lazy as _

from hcaptcha_field.settings import hcaptcha_settings
from hcaptcha_field.widgets import hCaptchaWidget


LOGGER = logging.getLogger('hcaptcha_field')


DATA_ATTRIBUTE_CONFIG = frozenset([
    'theme',
    'size',
    'tabindex',
    'callback',
    'expired-callback',
    'chalexpired-callback',
    'open-callback',
    'close-callback',
    'error-callback',
])


QUERY_PARAMETER_CONFIG = frozenset([
    'onload',
    'render',
    'hl',
    'recaptchacompat'
])


class hCaptchaField(forms.Field):
    widget = hCaptchaWidget
    default_error_messages = {
        'error_hcaptcha': _(
            # Translators: Error shown when an internal server error occurred.
            'Something went wrong while verifying the hCaptcha. '
            'Please try again.'
        ),
        'invalid_hcaptcha': _(
            # Translators: Error shown when visitor did not pass the hCaptcha check.
            'hCaptcha could not be verified.'
        ),
        'required': _(
            # Translators: Error shown when visitor forgot to fill in the hCaptcha.
            'Please prove you are human.'
        ),
    }

    def __init__(self, sitekey=None, **kwargs):
        """
        Initializer for `hCaptchaField` class. It determines data attributes
        for the widget class and constructs a widget if none is given. This
        constructed widget receives the URL of the JavaScript resource for the
        hCaptcha integration and the `sitekey` of the site to protect.
        """
        # Retrieve settings
        DEFAULT_CONFIG = hcaptcha_settings.DEFAULT_CONFIG
        JS_API_URL = hcaptcha_settings.JS_API_URL
        SITEKEY = hcaptcha_settings.SITEKEY

        # Determine widget data attributes
        self.widget_data_attrs = {}
        for setting in DATA_ATTRIBUTE_CONFIG:
            if setting in kwargs:
                self.widget_data_attrs[setting] = kwargs.pop(setting)
            elif setting in DEFAULT_CONFIG:
                self.widget_data_attrs[setting] = DEFAULT_CONFIG[setting]

        # If the `widget` argument is not given, instantiate `self.widget` with
        # the hCaptcha API url and the sitekey
        if 'widget' not in kwargs:
            # Determine hCaptcha API url
            query_params = {}
            for setting in QUERY_PARAMETER_CONFIG:
                if setting in kwargs:
                    query_params[setting] = kwargs.pop(setting)
                elif setting in DEFAULT_CONFIG:
                    query_params[setting] = DEFAULT_CONFIG[setting]
            if query_params:
                js_api_url = '%s?%s' % (JS_API_URL, urlencode(query_params))
            else:
                js_api_url = JS_API_URL

            # Determine hCaptcha sitekey
            self.sitekey = sitekey or SITEKEY

            # Instantiate widget
            kwargs['widget'] = self.widget(
                    js_api_url=js_api_url, sitekey=self.sitekey)

        super().__init__(**kwargs)

    def widget_attrs(self, widget):
        """
        Returns the widget attributes, including all the data attributes
        determined in the initializer.
        """
        attrs = super().widget_attrs(widget)
        for key, value in self.widget_data_attrs.items():
            attrs['data-%s' % key] = value
        return attrs

    def validate(self, value):
        """
        Validates the field by verifying the value of the hidden field
        `h-captcha-response` with their API endpoint.
        """
        super().validate(value)

        # Build request
        opener = build_opener(ProxyHandler(hcaptcha_settings.PROXIES))
        post_data = urlencode({
            'secret': hcaptcha_settings.SECRET,
            'response': value,
            'sitekey': self.sitekey,
        }).encode('utf-8')
        request = Request(hcaptcha_settings.VERIFY_URL, post_data)

        # Perform request
        try:
            context=ssl.create_default_context(cafile=certifi.where()) # added ############
            response = opener.open(request, timeout=hcaptcha_settings.TIMEOUT)
        except HTTPError:
            LOGGER.exception("Failed to verify response with hCaptcha API.")
            raise ValidationError(
                self.error_messages['error_hcaptcha'],
                code='error_hcaptcha'
            )

        # Check response
        response_data = json.loads(response.read().decode('utf-8'))
        if not response_data.get('success'):
            LOGGER.error("Failed to pass hCaptcha check: %s", response_data)
            raise ValidationError(
                self.error_messages['invalid_hcaptcha'],
                code='invalid_hcaptcha'
            )

Answer 1

看起来，为了使用context证书，需要将HTTPSHandler传递给certifi。

尝试用以下方法修补/usr/local/lib/python3.8/dist-packages/hcaptcha_field/fields.py的本地副本。

在导入中，正如您已经完成的一样，添加：

import certifi
import ssl

此外，扩展来自urllib.request的导入以包括HTTPSHandler：

from urllib.request import build_opener, Request, ProxyHandler, HTTPSHandler

同样，正如您已经做的那样，创建一个使用certifi证书的SSL上下文。不过，我会在import块下添加此权限，而不是深入模块代码中：

context=ssl.create_default_context(cafile=certifi.where())

然后，在这条新context=...行的正下方，使用context创建一个新的HTTPSHandler实例：

https_handler = HTTPSHandler(context=context)

然后，修改使用opener调用定义build_opener(...)的行，以包含新的HTTPSHandler实例：

opener = build_opener(https_handler, ProxyHandler(hcaptcha_settings.PROXIES))

如果我正确地读取了build_opener文档，那么build_opener机器在尝试打开D19时已经使用了HTTPSHandler的默认实例。希望将这个默认实例替换为这个新的https_handler实例，它将使用包含certifi证书的SSL上下文操作，从而允许urlopen调用工作。

如果这样做有效，那么向上游项目发出一个问题/PR可能是个好主意，可以集成这个逻辑，或者提供默认行为，如果certifi证书可用，就自动提取它们，或者包含一个配置选项，让用户连接到certifi证书中。

UPDATE 2022-04-14:由于这确实解决了问题，我将添加一些更详细的信息，说明为什么需要这个修复程序，以及它是如何工作的。

HTTPS要求您访问的网站提供一个验证其身份的证书。否则，你怎么知道你的交通没有受到干扰，中间人的攻击，等等？网站在浏览网站时向您提供的证书由证书颁发机构颁发给网站的操作员。这个权威机构作为一个可信的第三方，为网站运营商提供“担保”，“是的，你可以从他们那里信任这个证书”。

有几个这样的发证机构。这些当局颁发自己的证书(“证书颁发机构证书”或"CA证书“)，这些证书与个别网站提供的证书完全不同。您的浏览器(或URL打开程序)需要访问这些CA证书，以便完成网站显示的证书的“可信第三方”验证。OP在这里面临的The关键错误是URL找不到一组很好的CA证书用于第三方验证：

消息unable to get local issuer certificate意味着urlopen无法在local系统上找到所需的issuer certificate (CA证书)。

<#>因此，解决方案是将urlopen指向由certifi包提供的CA证书：

• 为了告诉opener在哪里找到它们，您必须在从build_opener创建opener时提供它们的位置。
• build_opener基于处理程序对象的集合工作，因此CA证书信息必须包含在适当的处理程序中。
• 唯一包含有关SSL上下文信息的处理程序(我可以在文档中找到)是HTTPSHandler。
• HTTPSHandler获取与SSL相关的配置信息的方式是在创建新实例时使用context关键字参数，该实例接受“SSL上下文”对象。
• ssl.create_default_context是一个用于创建SSL上下文对象的工厂函数(据我所理解)，可以通过各种方式定制。
• 它的可定制方式之一是使用CA证书存储区的位置(通过cafile参数)
• certifi通过certifi.where()函数在磁盘上提供其管理的CA证书存储的位置。

如果你从下到上扫描这些子弹，它们就能很好地映射到答案中的每一个步骤。