使用美汤从网页中提取特定字符串

Question

我有一系列我正在循环的URL，希望在页面顶部(示例)检索“称为”行中的所有内容。然而，并不是每个体育场都有这样的(示例)。

1 Arrowhead Drive, Kansas City, MO 64129
Years Active: 1972-2021 (406 games)
===> Known As: Arrowhead Stadium (1972-2020), GEHA Field at Arrowhead Stadium (2021-2021) <===
Surfaces: astroturf (1972-1993), grass (1994-2021)

下面是返回它所在的div中的所有内容的代码，但我不确定如何继续并返回我想要的特定p标记中的文本。

url_base = "https://www.pro-football-reference.com/stadiums/KAN00.htm"
response = requests.get(url_base)
soup = bs(response.text, "html.parser")
div = soup.find("div", {"id":"meta"})

<div id="meta">
<div>
<h1 itemprop="name">GEHA Field at Arrowhead Stadium History</h1>
<p>1 Arrowhead Drive, Kansas City, MO 64129</p><p><b>Years Active:</b> 1972-2021 (<a href="https://stathead.com/football/tgl_finder.cgi?request=1&amp;match=career&amp;year_min=1950&amp;year_max=2022&amp;game_type=E&amp;game_num_min=0&amp;game_num_max=99&amp;week_num_min=0&amp;week_num_max=99&amp;temperature_gtlt=lt&amp;stadium_id=KAN00&amp;c5val=1.0&amp;order_by=pass_td">406 games</a>)<p><b>Known As:</b> Arrowhead Stadium (1972-2020), GEHA Field at Arrowhead Stadium (2021-2021)<p><b>Surfaces:</b> astroturf (1972-1993), grass (1994-2021)<p><b>Teams:</b><ul><li><a href="/teams/kan/">Kansas City Chiefs</a> (1972-2022)</li><li> Regular Season: <a href="https://stathead.com/football/tgl_finder.cgi?request=1&amp;match=game&amp;team_id=kan&amp;year_min=1950&amp;year_max=2022&amp;game_type=R&amp;game_num_min=0&amp;game_num_max=99&amp;week_num_min=0&amp;week_num_max=99&amp;temperature_gtlt=lt&amp;stadium_id=KAN00&amp;c5val=1.0&amp;order_by=pass_td">233-155-1</a></li><li> Playoffs: <a href="https://stathead.com/football/tgl_finder.cgi?request=1&amp;match=game&amp;team_id=kan&amp;year_min=1950&amp;year_max=2022&amp;game_type=P&amp;game_num_min=0&amp;game_num_max=99&amp;week_num_min=0&amp;week_num_max=99&amp;temperature_gtlt=lt&amp;stadium_id=KAN00&amp;c5val=1.0&amp;order_by=pass_td">9-8</a></li></ul></p>
<button class="opener" data-class="open" data-id="info" data-type="hide_after" id="meta_more_button">More venue info</button>
<script>
// see sr.menus.js:sr_menus_checkInfoCookie to explain
function sr_menus_checkInfoCookie_inline(browserType) {
    var el_info = document.getElementById('info');
    var el_button = document.getElementById('meta_more_button');
    var bling_len = 0;    
    if (!el_button || !el_info || !el_info.classList) { console.log('no meta_button'); return; }
    var el = el_button;
    var siblingsHidden = 0;
    while (el = el.previousSibling) { if ((el.nodeType === 1) && (el.offsetWidth <= 0 || el.offsetHeight <= 0)) { siblingsHidden++; } }
    var button_cookie = false;
    if (browserType === 'desktop') {  button_cookie = vjs_readCookie('meta_more_button');   }
    // We allow up to four of bling lines or additional player bio data entries in mobile.
    if (el_info && el_button && (button_cookie || (siblingsHidden + bling_len <= 4))) {el_button.parentNode.removeChild(el_button); el_info.classList.add('open');  }
    else { el_button.classList.add('show');  }
}
if (Modernizr.desktop || Modernizr.laptop) { sr_menus_checkInfoCookie_inline('desktop'); } else { sr_menus_checkInfoCookie_inline('mobile'); }
var sr_menus_checkInfoCookie_run_inline = true;
</script>
</p></p></p></div>
</div>

Answer 1

您可以选择文本“称为”的<b>标记，然后获得下一个文本兄弟：

txt = (
    soup.select_one('b:-soup-contains("Known As")')
    .find_next_sibling(text=True)
    .strip()
)
print(txt)

指纹：

Arrowhead Stadium (1972-2020), GEHA Field at Arrowhead Stadium (2021-2021)