curl -u和python请求之间有什么区别吗？

Question

所以有一个网页，我想退到python.Requests

https://ororo.tv/api/v2/episodes/9

这需要基本的身份验证。如果我这样做的话

 curl -u test@example.com:password https://ororo.tv/api/v2/episodes/9

但是，在使用Requests库尝试在python中这样做时，我会得到我想要的响应，如下所示

>>> r = requests.get('https://ororo.tv/api/v2/episodes/9', auth=('test@example.com', 'password'))
>>> r
<Response [520]>

我总是收到520个回复。有人能告诉我，我可能做错了什么吗？

Answer 1

是的，有细微的差别。在被发送的头中有一些细微的差异，而这些显然对这个API很重要。

如果您将查询的URL更改为使用http://httpbin.org/get (在线HTTP测试服务HTTPBin.org的终结点)，您可以看到curl和requests发送的内容不同：

$ curl -u test@example.com:password http://httpbin.org/get
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Authorization": "Basic dGVzdEBleGFtcGxlLmNvbTpwYXNzd29yZA==",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.51.0"
  },
  "origin": "84.92.98.170",
  "url": "http://httpbin.org/get"
}
$ python -c "import requests; print(requests.get('http://httpbin.org/get', auth=('test@example.com', 'password')).text)"
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Authorization": "Basic dGVzdEBleGFtcGxlLmNvbTpwYXNzd29yZA==",
    "Host": "httpbin.org",
    "User-Agent": "python-requests/2.11.1"
  },
  "origin": "84.92.98.170",
  "url": "http://httpbin.org/get"
}

为了突出这些差异：

• requests发送一个额外的标题，Accept-Encoding，设置为gzip, deflate
• User-Agent标头不同；两者都反映当前代理。

您必须看到这些标头中的哪一个导致了https://ororo.tv/api/v2站点上的问题。当我更正URL使用v2和https时，就像curl命令一样，和设置了User-Agent头，然后得到一个有效的响应：

>>> headers = {'User-Agent': 'curl/7.51.0'}
>>> r = requests.get('https://ororo.tv/api/v1/episodes/9',
                     auth=('test@example.com', 'password'), headers=headers)
>>> r
<Response [200]>
>>> from pprint import pprint
>>> pprint(r.json())
{'airdate': '2005-10-13',
 'download_url': 'https://static-uk2.ororo.tv/uploads/video/file/9/Everybody.Hates.Chris.S01E04.DVDRip.Everybody.Hates.Sausage_1480525209.mp4?attachment=true&wmsAuthSign=aWQ9ODAzNDI3Kyt2aWRlbys5JnNlcnZlcl90aW1lPTIvOC8yMDE3IDI6Mjc6MDQgUE0maGFzaF92YWx1ZT1kbEpGM3c1bldSOXBOMUg5V2N1S0NnPT0mdmFsaWRtaW51dGVzPTk2MCZzdHJtX2xlbj05NQ%3D%3D',
 'id': 9,
 'name': 'Everybody Hates Sausage',
 'number': '4',
 'plot': 'When Julius buys a big crate of sausage, he makes everyone eat it '
         'with every meal. But Tonya refuses to, causing friction between the '
         'her and Rochelle. While at school, Chris is sentenced to 3 days of '
         'detention after a rumor goes round about him beating up the school '
         'bully, Joey.',
 'resolution': 'SD',
 'season': 1,
 'show_name': 'Everybody hates Chris',
 'subtitles': [{'lang': 'en',
                'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/4867/Everybody.Hates.Chris.S01E04.DVDRip.Everybody.Hates.Sausage.eng.vtt'},
               {'lang': 'ru',
                'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/28629/Everybody.Hates.Chris.S01E04.DVDRip.Everybody.Hates.Sausage.vtt'},
               {'lang': 'es',
                'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/55744/1x04_EH_Sausage.vtt'},
               {'lang': 'pt',
                'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/124429/Everybody_Hates_Chris_104_-_Everybody_Hates_Sausage.vtt'},
               {'lang': 'cs',
                'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/217213/Everybody_Hates_Chris_104_-_Everybody_Hates_Sausages.vtt'},
               {'lang': 'tr',
                'url': 'https://uploads.ororo-mirror.tv/uploads/subtitle/file/192405/Everybody_Hates_Chris_S01E04_-_Everybody_Hates_Sausages-tur.vtt'}],
 'updated_at': 1480640069,
 'url': 'https://static-gra.ororo.tv/uploads/video/file/9/Everybody.Hates.Chris.S01E04.DVDRip.Everybody.Hates.Sausage_1480525209.smil/playlist.m3u8?wmsAuthSign=aWQ9ODAzNDI3Kyt2aWRlbys5JnNlcnZlcl90aW1lPTIvOC8yMDE3IDI6Mjc6MDQgUE0maGFzaF92YWx1ZT1FajlGK2JPMEd3aU1Lc3lnN1M4NlpBPT0mdmFsaWRtaW51dGVzPTk2MCZzdHJtX2xlbj05Ng%3D%3D'}

Answer 2

您尝试使用的api可能要求您在使用类似于Python的请求时，以特定的方式格式化请求，可能需要头文件和base64编码的身份验证。

请参阅此示例，该示例将向您展示如何同时发送base64编码的身份验证头以及一些数据：

import requests
import base64

username = "some_username"
password = "some_password"

request_url = "https://ororo.tv/api/v2/episodes/9"

# In this example, I'm passing some data along with the request.
# these are generally what you would expect to pass along in an encoded url:
# /api?some_url_param_key=some_url_param_value

data = {}
data["some_url_param_key"] = "some_url_param_value"

# This is an example header, not necessarily what you need, 
# but it should serve as a good starting point.

headers = {}
headers["Authorization"] = "Basic " + base64.encodestring(username + ":" + password).replace('\n', '')
headers["Accept"] = "*/*"
headers["Content-Type"] = "application/x-www-form-urlencoded"
headers["User-Agent"] = "runscope/0.1"

# You can use post() in some cases where you would expect to use get().
# Every API is its own unique snowflake and expects different inputs.
# Try opening up the Chrome console and run the request in the 
# browser, where you know it works. Examine the headers and response
# in cases where the API you're accessing doesn't provide you 
# with the necessary inputs. 

result = requests.post(request_url, headers=headers, data=data)
print result