把<link>变成<link/>

Question

我正在尝试从一些RSS下载和解析文本，比如http://rss.sciencedirect.com/publication/science/03043878。下面是一个简单的例子：

import urllib.request
import urllib.parse
import requests

from bs4 import BeautifulSoup

def main():

    soup = BeautifulSoup(urllib.request.urlopen('http://rss.sciencedirect.com/publication/science/03043878'),"html.parser").encode("ascii")        
    print(soup)

if __name__ == '__main__':
    main()

在原始的html中(如果您直接查看网站)，链接的前面是<link>，后面是</link>。但是，打印出来的漂亮汤用<link/>取代了<link>，并完全放弃了</link>。你知道我可能做错了什么吗，或者这是个bug？

PS尝试将编码更改为utf-8，但仍然发生。

Answer 1

您正在解析RSS。RSS是XML。因此，将features="xml“传递给BeautifulSoup构造函数。

import urllib.request
from bs4 import BeautifulSoup 

def main():
    doc = BeautifulSoup(urllib.request.urlopen('http://rss.sciencedirect.com/publication/science/03043878'), "xml")
    # If you want to print it as ascii (as per your original post).
    print (doc.prettify('ascii'))
    # To write it to an file as ascii (as per your original post).
    with open("ascii.txt", "wb") as file:
        file.write(doc.prettify('ascii'))
    # To write it to an file as utf-8 (as the original RSS).
    with open("utf-8.txt", "wb") as file:
        file.write(doc.prettify('utf-8'))
    # If you want to print the links.
    for item in doc.findAll('link'):
        print(item)

if __name__ == '__main__':
    main()

文件和终端中的输出：

... <link>
http://rss.sciencedirect.com/action/redirectFile?&amp;zone=main&amp;currentActivity=feed&amp;usageType=outward&amp;url=http%3A%2F%2Fwww.sciencedirect.com%2Fscience%3F_ob%3DGatewayURL%26_origin%3DIRSSSEARCH%26_method%3DcitationSearch%26_piikey%3DS0304387817300512%26_version%3D1%26md5%3D16ed8e2672e8048590d3c41993306b0f
</link> ...

Answer 2

解析器无法正确评估链接。对于这个问题，您应该使用xml而不是html.parser作为解析器。

soup = BeautifulSoup(urllib.request.urlopen('http://rss.sciencedirect.com/publication/science/03043878'),"xml")
print(len(soup.find_all("link")))

输出52个链接。