带有身份验证或cookie的Python scrapy

Question

我有以下网络爬虫脚本是正确的工作，我需要的是一个集成身份验证或在每个请求发送cookies的方式

import scrapy
from scrapy.spiders import CrawlSpider, Rule
from scrapy.linkextractors import LinkExtractor
class TheFriendlyNeighbourhoodSpider(CrawlSpider):
    name = 'TheFriendlyNeighbourhoodSpider'    
    allowed_domains = ['one.google.com']
    start_urls = ['https://one.google.com/about']   
    custom_settings = {
    'LOG_LEVEL': 'INFO'
    }
    rules = (
        Rule(LinkExtractor(), callback='parse_item', follow=True),
    )
    def parse_item(self, response):
        print(response.url)

Answer 1

request_with_cookies = Request(url="http://www.example.com",
                           cookies=[{'name': 'currency',
                                    'value': 'USD',
                                    'domain': 'example.com',
                                    'path': '/currency'}])

有关更多信息，请访问：https://docs.scrapy.org/en/latest/topics/request-response.html