Newspaper3k、用户代理和抓取

Question

我正在制作由作者、发布日期和新闻文章的主文本组成的文本文件。我有代码可以这样做，但我需要Newspaper3k首先从这些文章中识别相关信息。由于用户代理规范已被是一个问题在此之前，所以我也指定了用户代理。这是我的代码，你可以跟着走。这是Python的version 3.9.0。

import time, os, random, nltk, newspaper 

from newspaper import Article, Config

user_agent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124  Safari/537.36'

config = Config()
config.browser_user_agent = user_agent

url = 'https://www.eluniversal.com.mx/estados/matan-3-policias-durante-ataque-en-nochistlan-zacatecas'
article = Article(url, config=config)
article.download()
#article.html #
article.parse()
article.nlp()

article.authors
article.publish_date
article.text 

为了更好地理解为什么这个例子特别令人费解，请用这个替换我上面提供的链接，然后重新运行代码。使用此链接，代码现在正确运行，返回作者、日期和文本。上面代码中的链接是没有的。我在这里忽略了什么？

Answer 1

显然，报纸要求我们指定我们感兴趣的语言。由于一些奇怪的原因，这里的代码仍然没有提取作者，但这对我来说已经足够了。这是代码，如果其他人会从中受益的话。

#
# Imports our modules
#

import time, os, random, nltk, newspaper
from newspaper import Article
from googletrans import Translator
translator = Translator()

# The link we're interested in

url = 'https://www.eluniversal.com.mx/estados/matan-3-policias-durante-ataque-en-nochistlan-zacatecas'


#
# Extracts the meta-data
#

article = Article(url, language='es')
article.download()
article.parse()
article.nlp()

#
# Makes these into strings so they'll get into the list
#

authors = str(article.authors)
date = str(article.publish_date)
maintext = translator.translate(article.summary).text


# Makes the list we'll append

elements = [authors+ "\n", date+ "\n", maintext+ "\n", url]

for x in elements:
    print(x)