文档建议反馈控制台
首页
学习
活动
专区
圈层
工具
MCP广场
文章/答案/技术大牛
发布
社区首页 >问答首页 >如何在美汤中选择某一类下的所有数据

问如何在美汤中选择某一类下的所有数据
EN

Stack Overflow用户
提问于 2020-10-23 18:58:16
回答 2查看 63关注 0票数 0

当我使用findAll、类和标记时，代码找不到任何东西

这是我的尝试：

代码语言：javascript
复制
import requests
from bs4 import BeautifulSoup

# get the data
data = requests.get('https://www.twitch.tv/directory/game/Counter-Strike%3A%20Global%20Offensive/clips?range=24hr/')

# load data into bs4
soup = BeautifulSoup(data.text, 'html.parser')

clip = soup.find_all("a", ["tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit"])

print(clip)

有什么问题吗？

python
beautifulsoup
EN

回答 2

Stack Overflow用户

发布于 2020-10-23 19:15:31

数据是通过Ajax动态加载的，但是您可以使用requests/json模块来加载它：

代码语言：javascript
复制
import json
import requests

url = 'https://gql.twitch.tv/gql#origin=twilight'

params = [{"operationName":"ClipsCards__Game","variables":{"gameName":"Counter-Strike: Global Offensive","limit":20,"criteria":{"languages":[],"filter":"LAST_WEEK"}},"extensions":{"persistedQuery":{"version":1,"sha256Hash":"0d8d0eba9fc7ef77de54a7d933998e21ad7a1274c867ec565ac14ffdce77b1f9"}}}]
headers = {'Client-Id': 'kimne78kx3ncx6brgo4mv6wki5h1ko'}

data =  requests.post(url, headers=headers, json=params).json()

# uncomment this to print all data:
# print(json.dumps(data, indent=4))

for g in data[0]['data']['game']['clips']['edges']:
    print('{:<60} {:<10} {}'.format(g['node']['title'], g['node']['viewCount'], g['node']['url']))

打印：

代码语言：javascript
复制
12 y/o ace                                                   121409     https://clips.twitch.tv/GoldenBlitheBillKeepo
Dupreeh feeling himself                                      51384      https://clips.twitch.tv/TenderBlatantCattleWoofer
ZyWoo Deagle Ace                                             49288      https://clips.twitch.tv/FurryWiseLemurLitty
ART + GAU JOGO DA TEAM ONE                                   34676      https://clips.twitch.tv/FamousDeliciousCasetteSMOrc
flusha insane 1v4                                            32522      https://clips.twitch.tv/NaiveSavoryLadiesOneHand
UNLUCKY XDD                                                  31698      https://clips.twitch.tv/AveragePoisedDelicataCmonBruh
lindinho clutchzero 1x3                                      24808      https://clips.twitch.tv/SmokyIronicMangetoutDatBoi
Stewie2k Selfless Play                                       24693      https://clips.twitch.tv/TallViscousSlothBloodTrail
bedzie magik flesh                                           20393      https://clips.twitch.tv/TangibleAverageDugongNononoCat
moe vs autimatic on stream                                   20311      https://clips.twitch.tv/CrowdedYummyNigiriPastaThat
Tarik insane 1v5 his first EU FPL game                       20104      https://clips.twitch.tv/HappyTangibleChowderRiPepperonis
Snax zaskakuje przeciwnika i wygrywa przegraną rundę!        19614      https://clips.twitch.tv/SpinelessFragileEchidnaImGlitch
Welcome to EU Tarik                                          18400      https://clips.twitch.tv/StormySavageDotterelYee
csgo'd                                                       17318      https://clips.twitch.tv/EmpathicAwkwardOtterHeyGuys
epic sauce                                                   15501      https://clips.twitch.tv/AttractiveRacyPhoneYouWHY
NiKo 1v2 2x 1taps                                            15088      https://clips.twitch.tv/SuspiciousJoyousSalsifyPogChamp
morda                                                        14394      https://clips.twitch.tv/PuzzledLightMonkeyMingLee
RAIN 3k vs sAw                                               14387      https://clips.twitch.tv/TastySavoryKangarooPRChase
Synchronous grenade throwing                                 13881      https://clips.twitch.tv/TransparentFrailCoffeePipeHype
sneaky fail ;p                                               13668      https://clips.twitch.tv/SquareRockyBunnyCclamChamp
票数 1
EN

Stack Overflow用户

发布于 2020-10-23 19:03:23

尝试使用selenium而不是requests来获取网站的html代码。下面是你如何做到这一点：

代码语言：javascript
复制
from bs4 import BeautifulSoup
from selenium import webdriver
import time

driver = webdriver.Chrome()

driver.get('https://www.twitch.tv/directory/game/Counter-Strike%3A%20Global%20Offensive/clips?range=24hr/')

time.sleep(5)

data = driver.page_source

driver.close()

soup = BeautifulSoup(data, 'html.parser')

clip = soup.find_all("a", class_ = "tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit")

print(clip)

输出：

代码语言：javascript
复制
/*
* 提示：该行代码过长，系统自动注释不进行高亮。一键复制会移除系统注释 
* [<a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/natosaphix/clip/GoldenBlitheBillKeepo"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="12 y/o ace">12 y/o ace</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/dreamhackcs/clip/TenderBlatantCattleWoofer"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="Dupreeh feeling himself">Dupreeh feeling himself</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/dreamhackcs3/clip/FurryWiseLemurLitty"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="ZyWoo Deagle Ace">ZyWoo Deagle Ace</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/artcsgo/clip/FamousDeliciousCasetteSMOrc"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="ART + GAU JOGO DA TEAM ONE">ART + GAU JOGO DA TEAM ONE</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/dreamhackcs/clip/NaiveSavoryLadiesOneHand"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="flusha insane 1v4">flusha insane 1v4</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/pago3/clip/AveragePoisedDelicataCmonBruh"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="UNLUCKY XDD">UNLUCKY XDD</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/deercheerup/clip/SmokyIronicMangetoutDatBoi"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="lindinho clutchzero 1x3">lindinho clutchzero 1x3</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/esl_csgo/clip/TallViscousSlothBloodTrail"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="Stewie2k Selfless Play">Stewie2k Selfless Play</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/pago3/clip/TangibleAverageDugongNononoCat"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="bedzie magik flesh">bedzie magik flesh</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/tarik/clip/CrowdedYummyNigiriPastaThat"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="moe vs autimatic on stream">moe vs autimatic on stream</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/tarik/clip/HappyTangibleChowderRiPepperonis"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="Tarik insane 1v5 his first EU FPL game">Tarik insane 1v5 his first EU FPL game</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/tebtv/clip/SpinelessFragileEchidnaImGlitch"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="Snax zaskakuje przeciwnika i wygrywa przegraną rundę!">Snax zaskakuje przeciwnika i wygrywa przegraną rundę!</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/snax/clip/UninterestedPoisedBadgerNononoCat"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="snax o neo i tazie">snax o neo i tazie</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/niko/clip/StormySavageDotterelYee"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="Welcome to EU Tarik">Welcome to EU Tarik</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/esl_csgo/clip/EmpathicAwkwardOtterHeyGuys"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="csgo'd">csgo'd</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/dextersjourney/clip/AttractiveRacyPhoneYouWHY"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="epic sauce">epic sauce</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/dreamhackcs2/clip/SuspiciousJoyousSalsifyPogChamp"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="NiKo 1v2 2x 1taps">NiKo 1v2 2x 1taps</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/innocenttt/clip/PuzzledLightMonkeyMingLee"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="morda">morda</h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/dreamhackcs2/clip/TastySavoryKangarooPRChase"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="RAIN 3k vs sAw ">RAIN 3k vs sAw </h3></div></a>, <a class="tw-full-width tw-interactive tw-link tw-link--hover-underline-none tw-link--inherit" href="/csgomc_ru/clip/TransparentFrailCoffeePipeHype"><div class="tw-align-items-start tw-flex"><h3 class="tw-ellipsis tw-font-size-5" title="Synchronous grenade throwing">Synchronous grenade throwing</h3></div></a>]
*/

我认为您正在尝试从这些anchor标记中抓取视频链接和标题。为此，请将以下行添加到您的代码中：

代码语言：javascript
复制
for a in clip:
    print('-'*100)
    print(f"Title = {a.div.h3['title']}")
    print(f"Link = https://www.twitch.tv/{a['href']}")

输出：

代码语言：javascript
复制
----------------------------------------------------------------------------------------------------
Title = 12 y/o ace
Link = https://www.twitch.tv//natosaphix/clip/GoldenBlitheBillKeepo
----------------------------------------------------------------------------------------------------
Title = Dupreeh feeling himself
Link = https://www.twitch.tv//dreamhackcs/clip/TenderBlatantCattleWoofer
----------------------------------------------------------------------------------------------------
Title = ZyWoo Deagle Ace
Link = https://www.twitch.tv//dreamhackcs3/clip/FurryWiseLemurLitty
----------------------------------------------------------------------------------------------------
Title = ART + GAU JOGO DA TEAM ONE
Link = https://www.twitch.tv//artcsgo/clip/FamousDeliciousCasetteSMOrc
----------------------------------------------------------------------------------------------------
Title = flusha insane 1v4
Link = https://www.twitch.tv//dreamhackcs/clip/NaiveSavoryLadiesOneHand
----------------------------------------------------------------------------------------------------
Title = UNLUCKY XDD
Link = https://www.twitch.tv//pago3/clip/AveragePoisedDelicataCmonBruh
----------------------------------------------------------------------------------------------------
Title = lindinho clutchzero 1x3
Link = https://www.twitch.tv//deercheerup/clip/SmokyIronicMangetoutDatBoi
----------------------------------------------------------------------------------------------------
Title = Stewie2k Selfless Play
Link = https://www.twitch.tv//esl_csgo/clip/TallViscousSlothBloodTrail
----------------------------------------------------------------------------------------------------
Title = bedzie magik flesh
Link = https://www.twitch.tv//pago3/clip/TangibleAverageDugongNononoCat
----------------------------------------------------------------------------------------------------
Title = moe vs autimatic on stream
Link = https://www.twitch.tv//tarik/clip/CrowdedYummyNigiriPastaThat
----------------------------------------------------------------------------------------------------
Title = Tarik insane 1v5 his first EU FPL game
Link = https://www.twitch.tv//tarik/clip/HappyTangibleChowderRiPepperonis
----------------------------------------------------------------------------------------------------
Title = Snax zaskakuje przeciwnika i wygrywa przegraną rundę!
Link = https://www.twitch.tv//tebtv/clip/SpinelessFragileEchidnaImGlitch
----------------------------------------------------------------------------------------------------
Title = Welcome to EU Tarik
Link = https://www.twitch.tv//niko/clip/StormySavageDotterelYee
----------------------------------------------------------------------------------------------------
Title = csgo'd
Link = https://www.twitch.tv//esl_csgo/clip/EmpathicAwkwardOtterHeyGuys
----------------------------------------------------------------------------------------------------
Title = epic sauce
Link = https://www.twitch.tv//dextersjourney/clip/AttractiveRacyPhoneYouWHY
----------------------------------------------------------------------------------------------------
Title = NiKo 1v2 2x 1taps
Link = https://www.twitch.tv//dreamhackcs2/clip/SuspiciousJoyousSalsifyPogChamp
----------------------------------------------------------------------------------------------------
Title = morda
Link = https://www.twitch.tv//innocenttt/clip/PuzzledLightMonkeyMingLee
----------------------------------------------------------------------------------------------------
Title = RAIN 3k vs sAw 
Link = https://www.twitch.tv//dreamhackcs2/clip/TastySavoryKangarooPRChase
----------------------------------------------------------------------------------------------------
Title = Synchronous grenade throwing
Link = https://www.twitch.tv//csgomc_ru/clip/TransparentFrailCoffeePipeHype
----------------------------------------------------------------------------------------------------
Title = sneaky fail ;p 
Link = https://www.twitch.tv//pashabiceps/clip/SquareRockyBunnyCclamChamp
票数 0
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：

https://stackoverflow.com/questions/64498718

复制
相关文章

相似问题

领券

腾讯云开发者

扫码关注腾讯云开发者

扫码关注腾讯云开发者

领取腾讯云代金券

热门产品

热门推荐

更多推荐

Copyright © 2013 - 2026 Tencent Cloud. All Rights Reserved. 腾讯云 版权所有 

深圳市腾讯计算机系统有限公司 ICP备案/许可证号：粤B2-20090059 粤公网安备44030502008569号

腾讯云计算（北京）有限责任公司 京ICP证150476号 |  京ICP备11018762号

问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档