如何遍历csv文件中的启动urls

Question

所以基本上，在我第一次运行蜘蛛的时候，由于某种原因，它起了作用，但在那之后，它只抓取了一个URL。

-My程序正在抓取我想要从列表中抓取的部分。

-Converts部件列表也是来自文件的URL。

-Runs并获取我想要的数据并将其输入到csv文件中。

问题是:只有从一个URL获得输出不知道从哪里开始，我已经检查了其他资源并尝试制作一个start_request。结果仍然一样。

所以，基本上，我如何才能让它使用所有的start_urls，并迭代它们中的每一个，而不仅仅是最后一个？

这是蜘蛛：

import csv
import xlrd
import scrapy

wb = xlrd.open_workbook(r'C:\Users\Jatencio\PycharmProjects\testy\test.xlsx')
ws = wb.sheet_by_index(0)
mylist = ws.col_values(0)
print(mylist)

li = []
for el in mylist:
    baseparts = el[:5]
    url1 = 'https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=' + baseparts + '&pageSize=500&pkeyword=' + baseparts
    li.append(url1)
final = list(set(li))


file = open('templist.csv','w+',newline='')
with file:
    write = csv.writer(file, delimiter =',')
    write.writerows(x.split(',') for x in final)

class DigikeSpider(scrapy.Spider):
    name = 'digike'
    allowed_domains = ['digikey.com']
    custom_settings = {
        "USER_AGENT": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36"

    }

    with open('templist.csv') as file:
        start_urls = [line.strip() for line in file]

    def parse(self, response):
        data = {}
        parts1 = []
        # parts=response.css('Table#productTable.productTable')
        for p in response.css('tbody#lnkPart > tr'):

            if p.css('td.tr-mfgPartNumber span::text').get() not in mylist:
                continue

            else:
                parts1 = p.css('td.tr-mfgPartNumber span::text').get()

            if p.css('td.tr-minQty.ptable-param span.desktop::text').get():
                quantity = p.css('td.tr-minQty.ptable-param span.desktop::text').get()
                quantity = quantity.strip()
                cleaned_quantity = int(quantity.replace(',', ''))
            else:
                quantity = 'No quantity'

            if p.css('td.tr-unitPrice.ptable-param center::text').get() == 'Active':
                p.css('td.tr-mfgPartNumber span::text').remove()

            else:
                pass

            if p.css('td.tr-unitPrice.ptable-param center::text').get() == 'Obsolete':
                p.css('td.tr-mfgPartNumber span::text').remove()

            else:
                pass

            if p.css('td.tr-unitPrice.ptable-param center::text').get() == 'Discontinued at Digi-Key':
                p.css('td.tr-mfgPartNumber span::text').remove()

            else:
                pass

            if p.css('td.tr-unitPrice.ptable-param span::text').get():
                price = p.css('td.tr-unitPrice.ptable-param span::text').get()
                cleaned_price = price.strip()
            else:
                price = 'No Price'

            if p.css('td.tr-qtyAvailable.ptable-param span.desktop::text').get():
                stock = p.css('td.tr-qtyAvailable.ptable-param span.desktop::text').get()
                cleaned_stock = stock.strip()

            else:
                pass
            if p.css('#part-status ::text').get():
                status = p.css('#part-status ::text').get()
                cleaned_status = status.strip()

            else:
                pass

            yield {
                'Part': parts1,
                'Quantity': cleaned_quantity,
                'Price': cleaned_price,
                'Stock': cleaned_stock,
                'Status': cleaned_status,

            }

输出

2020-07-30 10:12:11 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=IS62L&pageSize=500&pkeyword=IS62L> (referer:
 None)
2020-07-30 10:12:11 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY622&pageSize=500&pkeyword=CY622> (referer:
 None)
2020-07-30 10:12:11 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY622&pageSize=500&pkeyword=CY622
> (referer: None)
Traceback (most recent call last):
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\utils\defer.py", line 120, in iter_errback
    yield next(it)
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\utils\python.py", line 346, in __next__
    return next(self.data)
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\utils\python.py", line 346, in __next__
    return next(self.data)
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
    for x in result:
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 340, in <genexpr>
    return (_set_referer(r) for r in result or ())
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "C:\Users\Jatencio\PycharmProjects\testy\testdigi\testdigi\spiders\digike.py", line 93, in parse
    'Quantity': cleaned_quantity,
UnboundLocalError: local variable 'cleaned_quantity' referenced before assignment
2020-07-30 10:12:15 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=IS62C&pageSize=500&pkeyword=IS62C> (referer:
 None)
2020-07-30 10:12:17 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=IS62W&pageSize=500&pkeyword=IS62W> (referer:
 None)
2020-07-30 10:12:17 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY621&pageSize=500&pkeyword=CY621> (referer:
 None)
2020-07-30 10:12:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY621&pageSize=500&pkeyword=CY621>
{'Part': 'CY62128ELL-45SXIT', 'Quantity': 1000, 'Price': '$2.29429', 'Stock': '1,000 - Immediate', 'Status': 'Active'}
2020-07-30 10:12:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY621&pageSize=500&pkeyword=CY621>
{'Part': 'CY62157EV30LL-45ZSXIT', 'Quantity': 1000, 'Price': '$6.44254', 'Stock': '2,000 - Immediate', 'Status': 'Active'}
2020-07-30 10:12:17 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY621&pageSize=500&pkeyword=CY621>

2020-07-30 10:12:17 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY621&pageSize=500&pkeyword=CY621
> (referer: None)
Traceback (most recent call last):
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\parsel\selector.py", line 368, in remove
    parent = self.root.getparent()
AttributeError: 'str' object has no attribute 'getparent'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\utils\defer.py", line 120, in iter_errback
    yield next(it)
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\utils\python.py", line 346, in __next__
    return next(self.data)
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\utils\python.py", line 346, in __next__
    return next(self.data)
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 29, in process_spider_output
    for x in result:
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 340, in <genexpr>
    return (_set_referer(r) for r in result or ())
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\core\spidermw.py", line 64, in _evaluate_iterable
    for r in iterable:
  File "C:\Users\Jatencio\PycharmProjects\testy\testdigi\testdigi\spiders\digike.py", line 55, in parse
    p.css('td.tr-mfgPartNumber span::text').remove()
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\parsel\selector.py", line 164, in remove
    x.remove()
  File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\parsel\selector.py", line 371, in remove
    raise CannotRemoveElementWithoutRoot(
parsel.selector.CannotRemoveElementWithoutRoot: The node you're trying to remove has no root, are you trying to remove a pseudo-element? Try to use 'li' as a selector instead of 'li::text' or '//li' instead of '//li/text()', for exampl
e.
2020-07-30 10:12:17 [scrapy.core.engine] INFO: Closing spider (finished)
2020-07-30 10:12:17 [scrapy.extensions.feedexport] INFO: Stored csv feed (40 items) in: DigiKeyPartsList.csv

打印(Start_urls)：start_urls'https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=IS62W&pageSize=500&pke yword=IS62W'，'https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY621&pageSize=500&pkeyword=CY621'，'https://www.digikey.com/products/en/integrated-circuits-ics /memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY622&pageSize=500&pkeyword=CY622‘

新输出

2020-07-30 12:51:31 [scrapy.utils.log] INFO: Scrapy 2.2.1 started (bot: testdigi)
2020-07-30 12:51:31 [scrapy.utils.log] INFO: Versions: lxml 4.5.2.0, libxml2 2.9.5, cssselect 1.1.0, parsel 1.6.0, w3lib 1.22.0, Twisted 20.3.0, Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:37:02) [MSC v.1924 64 bit (AMD64)], pyO
penSSL 19.1.0 (OpenSSL 1.1.1g  21 Apr 2020), cryptography 3.0, Platform Windows-10-10.0.17134-SP0
2020-07-30 12:51:31 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.selectreactor.SelectReactor
2020-07-30 12:51:31 [scrapy.crawler] INFO: Overridden settings:
{'BOT_NAME': 'testdigi',
 'NEWSPIDER_MODULE': 'testdigi.spiders',
 'SPIDER_MODULES': ['testdigi.spiders'],
 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '
               '(KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36'}
2020-07-30 12:51:31 [scrapy.extensions.telnet] INFO: Telnet Password: 4abf97dccc166f2d
2020-07-30 12:51:31 [py.warnings] WARNING: c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\scrapy\extensions\feedexport.py:210: ScrapyDeprecationWarning: The `FEED_URI` and `FEED_FORMAT` settings have been deprecated in
favor of the `FEEDS` setting. Please see the `FEEDS` setting docs for more details
  exporter = cls(crawler)

2020-07-30 12:51:31 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.feedexport.FeedExporter',
 'scrapy.extensions.logstats.LogStats']
2020-07-30 12:51:32 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2020-07-30 12:51:32 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2020-07-30 12:51:32 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2020-07-30 12:51:32 [scrapy.core.engine] INFO: Spider opened
2020-07-30 12:51:32 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2020-07-30 12:51:32 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2020-07-30 12:51:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=IS62L&pageSize=500&pkeyword=IS62L> (referer:
 None)
2020-07-30 12:51:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=IS62C&pageSize=500&pkeyword=IS62C> (referer:
 None)
2020-07-30 12:51:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY622&pageSize=500&pkeyword=CY622> (referer:
 None)
2020-07-30 12:51:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY622&pageSize=500&pkeyword=CY622>
{'Part': 'CY62256NLL-55ZXIT', 'Quantity': 'No quantity', 'Price': '$1.11989', 'Stock': '0', 'Status': 'Obsolete'}
2020-07-30 12:51:33 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=IS62W&pageSize=500&pkeyword=IS62W> (referer:
 None)
2020-07-30 12:51:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY622&pageSize=500&pkeyword=CY622>
{'Part': 'CY62256VNLL-70ZXIT', 'Quantity': 'No quantity', 'Price': 'No Price', 'Stock': '0', 'Status': 'Obsolete'}
2020-07-30 12:51:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY622&pageSize=500&pkeyword=CY622>
{'Part': 'CY62256NLL-55SNXIT', 'Quantity': 'No quantity', 'Price': 'No Price', 'Stock': '0', 'Status': 'Obsolete'}
2020-07-30 12:51:33 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY622&pageSize=500&pkeyword=CY622>
{'Part': 'CY62256VNLL-70SNXIT', 'Quantity': 'No quantity', 'Price': 'No Price', 'Stock': '0', 'Status': 'Obsolete'}
2020-07-30 12:51:34 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.digikey.com/products/en/integrated-circuits-ics/memory/774?FV=-8%7C774%2C7%7C1&quantity=0&ColumnSort=0&page=1&k=CY621&pageSize=500&pkeyword=CY621> (referer:
 None)
{'Part': 'CY62148EV30LL-45ZSXIT', 'Quantity': 1000, 'Price': 'No Price', 'Stock': '0', 'Status': 'Active'}
2020-07-30 12:51:34 [scrapy.core.engine] INFO: Closing spider (finished)
2020-07-30 12:51:34 [scrapy.extensions.feedexport] INFO: Stored csv feed (46 items) in: DigiKeyPartsList.csv
2020-07-30 12:51:34 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 2145,
 'downloader/request_count': 5,
 'downloader/request_method_count/GET': 5,
 'downloader/response_bytes': 289446,
 'downloader/response_count': 5,
 'downloader/response_status_count/200': 5,
 'elapsed_time_seconds': 2.311786,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2020, 7, 30, 16, 51, 34, 681758),
 'item_scraped_count': 46,
 'log_count/DEBUG': 51,
 'log_count/INFO': 11,
 'log_count/WARNING': 1,
 'response_received_count': 5,
 'scheduler/dequeued': 5,
 'scheduler/dequeued/memory': 5,
 'scheduler/enqueued': 5,
 'scheduler/enqueued/memory': 5,
 'start_time': datetime.datetime(2020, 7, 30, 16, 51, 32, 369972)}
2020-07-30 12:51:34 [scrapy.core.engine] INFO: Spider closed (finished)

(venv) C:\Users\Jatencio\PycharmProjects\testy\testdigi\testdigi>

Answer 1

现在，通过执行日志，我可以告诉您，蜘蛛中有两个问题，似乎都与start_urls无关。

第一个例外：

File "C:\Users\Jatencio\PycharmProjects\testy\testdigi\testdigi\spiders\digike.py", line 93, in parse
    'Quantity': cleaned_quantity,
UnboundLocalError: local variable 'cleaned_quantity' referenced before assignment

在定义cleaned_quantity之前，您要引用它。问题在于：

        if p.css('td.tr-minQty.ptable-param span.desktop::text').get():
            quantity = p.css('td.tr-minQty.ptable-param span.desktop::text').get()
            quantity = quantity.strip()
            cleaned_quantity = int(quantity.replace(',', ''))
        else:
            quantity = 'No quantity'

如果您的If语句解析为false，则永远不会定义cleaned_quantity，并且在您试图组装项时会引发错误：

        yield {
            'Part': parts1,
            'Quantity': cleaned_quantity,
            'Price': cleaned_price,
            'Stock': cleaned_stock,
            'Status': cleaned_status,
        }

这种情况只发生在几次迭代中，而不是全部。

第二个例外：

 File "C:\Users\Jatencio\PycharmProjects\testy\testdigi\testdigi\spiders\digike.py", line 55, in parse
    p.css('td.tr-mfgPartNumber span::text').remove()
[...]
 File "c:\users\jatencio\pycharmprojects\testy\venv\lib\site-packages\parsel\selector.py", line 371, in remove
    raise CannotRemoveElementWithoutRoot(
parsel.selector.CannotRemoveElementWithoutRoot: The node you're trying to remove has no root, are you trying to remove a pseudo-element? Try to use 'li' as a selector instead of 'li::text' or '//li' instead of '//li/text()', for example.

这里的问题是在parsel所调用的伪元素中使用.remove()方法，您只能使用它从HTML树中删除实际的元素，所以我认为这应该可以解决这个问题：

改变这一点：

p.css('td.tr-mfgPartNumber span::text').remove()

对此：

p.css('td.tr-mfgPartNumber span').remove()

这是所有使用remove 方法的行的情况。。

如果这解决了你的问题，请告诉我。