用Python查找Selenium中的元素

Question

我一直在收集Youtube游戏中的直播频道/观众名单。我使用selenium和Python一起强制网站向下滚动页面，这样它就可以加载更多的11个频道。供参考，这是我正在工作的网页。

我已经找到了我想要的数据的位置，但是我很难让selenium去那里。我遇到麻烦的地方是这样：

<div class="style-scope ytg-gaming-video-renderer" id="video-metadata"><span class="title ellipsis-2 style-scope ytg-gaming-video-renderer"><ytg-nav-endpoint class="style-scope ytg-gaming-video-renderer x-scope ytg-nav-endpoint-2"><a href="/watch?v=FFKSD1HHrdA" tabindex="0" class="style-scope ytg-nav-endpoint" target="_blank">
              Live met Bo3
            </a></ytg-nav-endpoint></span>
    <div class="channel-info small layout horizontal center style-scope ytg-gaming-video-renderer">
        <ytg-owner-badges class="style-scope ytg-gaming-video-renderer x-scope ytg-owner-badges-0">
            <template class="style-scope ytg-owner-badges" is="dom-repeat"></template>
        </ytg-owner-badges>
        <ytg-formatted-string class="style-scope ytg-gaming-video-renderer">
            <ytg-nav-endpoint class="style-scope ytg-formatted-string x-scope ytg-nav-endpoint-2"><a href="/channel/UCD8Q9V5wgo8o0XGfUqsRrDQ" tabindex="0" class="style-scope ytg-nav-endpoint" target="_blank">Rico Eeman</a>
            </ytg-nav-endpoint>
        </ytg-formatted-string>
    </div><span class="ellipsis-1 small style-scope ytg-gaming-video-renderer" id="video-viewership-info" hidden=""></span>
    <div id="metadata-badges" class="small style-scope ytg-gaming-video-renderer">
        <ytg-live-badge-renderer class="style-scope ytg-gaming-video-renderer x-scope ytg-live-badge-renderer-1">
            <template class="style-scope ytg-live-badge-renderer" is="dom-if"></template>

            <span aria-label="" class="text layout horizontal center style-scope ytg-live-badge-renderer">4 watching</span>
            <template class="style-scope ytg-live-badge-renderer" is="dom-if"></template>
        </ytg-live-badge-renderer>
    </div>
</div>

目前，我正在努力：

#This part works fine. I can use the unique ID
meta_data = driver.find_element_by_id('video-metadata')

#This part is also fine. Once again, it has an ID.
viewers = meta_data.find_element_by_id('metadata-badges')
print(viewers.text)

但是，我在获取通道名称(在本例中是'Rico Eeman'，它位于第一个嵌套的div标记下)时遇到了困难。因为它是一个复合类名，所以我无法按类名查找元素，因此尝试以下xpath无法工作：

name = meta_data.find_element_by_xpath('/div[@class="channel-info small layout horizontal center style-scope ytg-gaming-video-renderer"]/ytg-formatted-string'

name = meta_data.find_element_by_xpath('/div[1])

它们都会引发未找到错误的元素。我真不知道在这里该怎么办。有人有可行的解决方案吗？

Answer 1

名称id不在<ytg-formatted-string>标记中，它在其中一个后代中。试一试

meta_data.find_element_by_css_selector('.style-scope.ytg-formatted-string.x-scope.ytg-nav-endpoint-2 > a')

或使用xpath

meta_data.find_element_by_xpath('//ytg-nav-endpoint[@class="style-scope ytg-formatted-string x-scope ytg-nav-endpoint-2"]/a')

Answer 2

这将获得所有的名称，即使您的xpath使用video-metadata不会获得所有的名称，每个用户的id也会被重复，所以您需要find_elements并迭代返回的元素：

names = dr.find_elements_by_css_selector("a.style-scope.ytg-nav-endpoint[href^='/channel/']")
print([name.get_attribute("text") for name in names])

这给了你：

['NinjaNation Gaming', 'DURX DANIEL', 'DEMON', 'Perfection', 'The one and only jd', 'Violator Games', 'KingLuii718', 'NinjaNation Gaming', 'DURX DANIEL', 'DEMON', 'Perfection']