使用网页重放记录HTTPS网页

Question

我正在尝试用网页重播记录一个实验的网站。记录HTTP请求可以正常工作，但是当我尝试记录HTTPS请求时，我会得到一个SSLError。

在OSX上，我使用这个命令记录网页

sudo ./replay.py --record archive.wpr

我已经将系统HTTP代理设置为localhost:80，将HTTPS代理设置为localhost:443

回溯-

2015-02-24 03:57:26,227 ERROR Client failed to make request
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 63226)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 599, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 334, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 655, in __init__
    self.handle()
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 340, in handle
    self.handle_one_request()
  File "/Users/akshay/Projects/web-page-replay/httpproxy.py", line 162, in handle_one_request
    self.raw_requestline = self.rfile.readline(65537)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 476, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 714, in recv
    return self.read(buflen)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 608, in read
    v = self._sslobj.read(len or 1024)
SSLError: [SSL: HTTPS_PROXY_REQUEST] https proxy request (_ssl.c:1750)
----------------------------------------

更新--我尝试使用遥测记录页面集，在记录https页面时得到了SSL握手错误。

2015-03-01 11:12:04,929 WARNING HTTP server started on 127.0.0.1:50781
2015-03-01 11:12:04,930 WARNING HTTPS server started on 127.0.0.1:50782
2015-03-01 11:12:04,981 ERROR Client failed to make request
2015-03-01 11:12:04,981 ERROR Client failed to make request
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 599, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 334, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 655, in __init__
    self.handle()
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 340, in handle
    self.handle_one_request()
  File "/Users/akshay/Downloads/telemetry/src/third_party/webpagereplay/httpproxy.py", line 162, in handle_one_request
    self.raw_requestline = self.rfile.readline(65537)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 476, in readline
    data = self._sock.recv(self._rbufsize)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 714, in recv
    return self.read(buflen)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 608, in read
    v = self._sslobj.read(len or 1024)
SSLError: [SSL: SSL_HANDSHAKE_FAILURE] ssl handshake failure (_ssl.c:1750)
2015-03-01 11:12:05,782 ERROR Client failed to make request
2015-03-01 11:12:07,611 WARNING Retrying fetch GET https://www.google.com/ [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:08,844 WARNING Retrying fetch GET https://www.google.com/ [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:09,539 WARNING Retrying fetch GET https://www.google.com/ [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:10,146 CRITICAL Could not fetch GET https://www.google.com/ [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:11,386 WARNING Retrying fetch GET https://www.google.com/favicon.ico [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:13,428 WARNING Retrying fetch GET https://www.google.com/favicon.ico [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:14,417 WARNING Retrying fetch GET https://www.google.com/favicon.ico [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
2015-03-01 11:12:18,349 CRITICAL Could not fetch GET https://www.google.com/favicon.ico [('accept-encoding', 'gzip, deflate, '), ('host', 'www.google.com')]: hostname '74.125.236.209' doesn't match 'www.google.com'
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 50785)
----------------------------------------

Answer 1

正如我所看到的，python的WebPageReplay实现现在已被废弃，所以您可以使用Go实现go/README.md

开发人员建议使用以下Chrome命令行参数(以及所有类似于Chrome的浏览器)来绕过浏览器的TLS检查：

--ignore-certificate-errors-spki-list=PhrPvGIaAMmd29hj8BCZOq096yj7uMpRNHpn5PDxI6I=

在PhrPvGIaAMmd29hj8BCZOq096yj7uMpRNHpn5PDxI6I=中，它是WebPageReplay提供的证书的主题，默认情况下，它用于记录在wpr存档中的所有域。

对于不支持--ignore-certificate-errors-spki-list的其他浏览器，您可以在wpr存档中动态地或预生成由域名生成证书。