我正在尝试使用Python中的Google搜索来抓取网络,但我希望它能考虑到我过去的搜索。有没有办法登录然后用Python在Google上搜索?
这是我使用的代码的一部分:
from googlesearch import search
for j in search(query, stop=n, lang=lang):
#request al browser
req = urllib.request.Request(j, headers={'User-Agent': "Magic Browser"})
cj = CookieJar()
try:
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
except:
pass发布于 2019-12-04 23:32:50
尝试这段代码(请通知: googlesearch不支持代理,但我知道如何在需要时添加此功能):
from googlesearch import search
query = "apple iphone news 2019"
for i in search(\
query, # The query you want to run
tld = 'com.ua', # The top level domain
lang = 'en', # The language
num = 10, # Number of results per page
start= 0, # First result to retrieve
stop = None, # Last result to retrieve
pause= 2.0, # Lapse between HTTP requests
):
my_results_list.append(i)
print(i)请密切关注与'tld‘参数-这必须等于顶级谷歌域名(域名级别从右向左上升)。例如,我的领地谷歌服务器DNS eq 'www.google.com.ua‘and 'tld'='com.ua’。有关每个地区的谷歌服务器的完整列表,请参阅this list
更新:对于支持代理:
proxy_support = urllib.request.ProxyHandler({'http' : 'http://IP-Address:Port',
'https': 'http://IP-Address:port'})
opener = urllib.request.build_opener(proxy_support)
urllib.request.install_opener(opener)https://stackoverflow.com/questions/59153281
复制相似问题