查找<caption class=“表-标题”>

Question

因此，我编写了一个脚本来从网站上抓取表格，并将其保存到Excel工作表中：

import pandas as pd
import requests
from bs4 import BeautifulSoup
from pandas import ExcelWriter
import os.path
path = "C:...."
url= 'https://zoek.officielebekendmakingen.nl/kst-35570-2.html'
page = requests.get(url)
soup = BeautifulSoup(page.text, 'html.parser')

tables_df = pd.read_html(url, attrs = {'class': 'kio2 portrait'})

tables = soup.find_all('table', class_="kio2 portrait")

titles = []
for table in tables:
    print(table)
    title = table.find_all("caption", class_="table-title")
    titles.append(title)
titles = []

writer = pd.ExcelWriter('output.xlsx')
for i, df in enumerate(tables_df, 1):
    df.to_excel(writer, index=True,sheet_name=f'sheetName_{i}')
writer.save()

这是可行的，但是现在我想要找到这些表的所有标题，这样我就可以给每个工作表提供这个标题。例如，第一个表包含我感兴趣的以下文本：

<table cellpadding="0" cellspacing="0" class="kio2 portrait" summary="Tabel 1.1 Budgettaire kerngegevens"><caption class="table-title">Tabel 1.1 Budgettaire kerngegevens</caption>

现在我想比较一下<caption class="table-title">和</caption>之间的区别。或者，也可以使用summary元素。我如何才能做到这一点？我已经在代码中尝试过了，但我还没有找到任何东西。

Answer 1

尝试：

import requests
import pandas as pd
from bs4 import BeautifulSoup
from pandas import ExcelWriter


url = "https://zoek.officielebekendmakingen.nl/kst-35570-2.html"
soup = BeautifulSoup(requests.get(url).text, "html.parser")

writer = pd.ExcelWriter("output.xlsx")
for i, table in enumerate(soup.find_all("table", class_="kio2 portrait"), 1):
    df = pd.read_html(str(table))[0]

    caption = table.get("summary", "").replace(":", "").strip()
    # some tables doesn't contain summary, so make generic sheet name:
    if not caption:
        caption = f"table {i}"

    df.to_excel(writer, sheet_name=caption)

writer.save()

这将创建具有185张工作表的output.xlsx (至少在我的Libreoffice中打开它)：

​

​