使用selenium获取完整的html源代码

Question

我想在电报中得到一篇文章的点击数。当我得到页面源时，javaScript函数还没有执行，因此没有启用数据。有什么帮助吗？

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

from selenium.webdriver.common.by import By
import selenium.webdriver.support.expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait    

driver = webdriver.Firefox()

driver.get("https://t.me/varzesh3/106369")

wait = WebDriverWait(driver, 30)
element = wait.until(EC.visibility_of_any_elements_located((By.CLASS_NAME,'tgme_widget_message_views')))

我甚至用Chrome驱动程序试过，但没有改变。

Answer 1

这里有个更好的主意。我可以使用bs4和url https://t.me/varzesh3/106369?embed=1。

这样做更好，因为当你通过这个网址发送请求时，你现在有了iframe。

这是密码：

def generate_post_views(link):
   # https://t.me/varzesh3/106432
   html = requests.get(link + '?embed=1')
   plain_text = html.text
   soup = BeautifulSoup(plain_text, "html.parser")
   try:
       views = soup.find('span', {'class':'tgme_widget_message_views'}).get_text()
   except AttributeError:
       views = 0
   return views

Answer 2

它不起作用，因为要检查的元素在iframe中。您需要首先切换到IFrame。

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

from selenium.webdriver.common.by import By
import selenium.webdriver.support.expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait    

driver = webdriver.Chrome()

driver.get("https://t.me/varzesh3/106369")
iframe = driver.find_element(By.ID,"telegram-post-varzesh3-106369")
driver.switch_to.frame(iframe)
wait = WebDriverWait(driver, 30)
element = wait.until(EC.visibility_of_any_elements_located((By.CLASS_NAME,'tgme_widget_message_views')))