我不能刮擦div‘’一些文本“class =”，我认为文本会导致错误

Question

我怎样才能像(<div data-v-28872a74="" class="col-lg-10 col-md-10 col-sm-12 col-12 offset-lg-1 offset-md-1 offset-sm-0 offset-0">)那样刮html。

我尝试过soup.find_all('div', class_ = 'col-lg-10 col-md-10 col-sm-12 col-12 offset-lg-1 offset-md-1 offset-sm-0 offset-0')，但是输出只是[]。

实际代码：

div数据-v-28872a74=“class=”-lg-10 col 10 col 12 col-12偏移-lg-1偏移量-md-1偏移量-sm-0偏移量-0‘

import requests
from bs4 import BeautifulSoup as bs
url = 'https://remart.az/yasayis-kompleksi?cities=1&districts='

result = requests.get(url)
soup = bs(result.text, 'html.parser')
code= soup.find_all('div', class_ = 'col-lg-10 col-md-10  col-sm-12 col-12  offset-lg-1 offset-md-1 offset-sm-0 offset-0')
print(code)

这第二个代码会刮掉urls，但在下一个代码中，我看到了同样的问题。

driver = webdriver.Chrome(r'C:\Program Files (x86)\chromedriver_win32\chromedriver.exe')
driver.get('https://remart.az/yasayis-kompleksi?cities=1&districts=')
time.sleep(3)

aze = driver.find_element(By.XPATH, '//*[@id="app"]/div[2]/div[1]/div[2]/div[6]/button')


for a in range(1,2):
    aze.click()
    time.sleep(1)
    

soup = bs(driver.page_source, "html.parser")
aezexx = soup.find_all('div', class_ = 'bitem')
for parent in aezexx:
    a_tag = parent.find("a")
    URRL = a_tag.attrs['href']
    print(URRL)

soup = bs(driver.page_source, "html.parser")
aezexx = soup.find_all('div', class_ = 'bitem')
for parent in aezexx:
    a_tag = parent.find("a")
    URRL = a_tag.attrs['href']
    result = requests.get(URRL)
    soup = bs(result.text, 'html.parser')
    are = soup.find_all("div", class_ = 'bottom-panel-descripton cut-text')
    for aes in are:
        azzzz = aes.find_all('p')
        print(azzzz) 

Answer 1

尝试：

import re
import json
import requests
import pandas as pd
from ast import literal_eval

url = "https://remart.az/yasayis-kompleksi?cities=1&districts="
html_doc = requests.get(url).text

data = re.search(r'window\.__INITIAL_STATE__ = (".*")', html_doc).group(1)
data = json.loads(literal_eval(data))

df = pd.DataFrame(data)
del df["descr"]
df["city"] = df["city"].str["name"]
df["district"] = df["district"].str["name"]

print(df.head())

指纹：

    id                  name  status  company_id    end_date land_area  contact_person                                         website                      email                                                   phones  housing_count  block_count floor_count  apartment_count apartments_on_floor_count elevator_count                                       address  city_id  district_id orient_ids          lat          lng  underground_garage underground_garage_floor_count  underground_garage_place_count objects_floor_count objects_area infr_items infr_additional_items  credit interest_rate  maximum_installment_period minimum_initial_deposit  payment_graph  mortgage mortgage_interest_rate  mortgage_duration mortgage_initial_deposit partner_banks created_at updated_at  seen                 title                               description                                                                                                                                                                                                                                                                                                                                                                              keywords  kupcha  currency  village_id  metro_id  recommended  country_id foreign_price                                    image_cover                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             image_condo image_construction location_info infrastructure_info                                                                                        full_payment_comment credit_comment mortgage_comment documents_comment                  slug min_price min_price_apart  city      district
0  464  Golden Rose Boutique       1         296  2022-03-31      0.20             NaN                                            None                       None                                        +994 50 241 21 12            3.0          3.0           7             42.0                         2              1            Necef Nerimanov küç., 1979 məhellə        1            3       None         None         None                   0                           None                             NaN                None         None       None                  None       1          0.00                        12.0                   20.00              1         0                   8.00              240.0                    40.00          None       None       None   0.0  Golden Rose Boutique  Golden Rose Boutique - premium menziller  golden rose boutique, golden_rose_boutique, golden-rose-boutique, lalafo, korter, bina, yeniemlak, residence yaşayış kompleksi, yeniemlak, kreditlə yaşayış kompleksindəki mənzillər, yaşayış kompleksindəki mənzillərin qiyməti, yaşayış kompleksində mənzillərin alınması, kredit, ipoteka, Bakı, satış, yeni bina, yeni tikililər, mənzillər, mənzillər, otaqlar , modern, layihe       0         1         NaN       1.0            0           1          None  condos/September2021/xMhblXRcyffXvVk90AaK.jpg                                                                     ["condos\/September2021\/RadK9ymfGUCGzJqQsrpT.jpg","condos\/September2021\/F1JEiifMCxfUAPTFQATP.jpg","condos\/September2021\/qTRHpWZS9O3PczUBPMDr.jpg","condos\/September2021\/mzJp8VNyylp1GsPRyCWb.jpg","condos\/September2021\/59vJ3NLfluOjNmndQcj2.jpg","condos\/September2021\/hNrBMAuI04cyzxUrXHxc.jpg","condos\/September2021\/kzbi7vDumwFdvdprzS4i.jpg","condos\/September2021\/r90BM5Du1EGRzoQ6c3i5.jpg","condos\/September2021\/LVXvZeCNjOFij11wi5ag.jpg"]               None                                                                     <p>Mənzillərin 1 kvadrat metrinin nağd qiyməti 2450 manatdan başlayır</p>                                                    golden-rose-boutique   2450.00     237037.5000  Bakı  Nərimanov r.
1  463        Central Towers       1           9  2024-12-12      None             NaN                               srconstruction.az     info@srconstruction.az                                     *1144, 050 988 11 44            3.0          3.0          16            178.0                       4-5              2                                                      1           12       None  40.38121100  49.82461600                   0                           None                             NaN                None         None       None                  None       0          None                         NaN                    None              1         1                  10.00               20.0                    30.00          None       None       None   0.0                                                                                                                                                                                                                                                                                                                                                                                                                                                             0         1         NaN      11.0            0           1          None  condos/September2021/6o8EAc6jDh99QjedN4ao.jpg                                                                     ["condos\/September2021\/zpLYa1KtFyZ7O56HyifJ.jpg","condos\/September2021\/qZctiglmI8WAPnTbjxAa.jpg","condos\/September2021\/FZvCsxXwtY6IGm1Mckkr.jpg","condos\/September2021\/ZIK9JgbiPr2k7p2pPRqk.jpg","condos\/September2021\/TTevd9TqrF6Zas25WIYl.jpg","condos\/September2021\/avYtaaEoT7cBkRADL5B7.jpg","condos\/September2021\/ocFd2JG7LbstKI12uCTY.jpg","condos\/September2021\/SF1p6C9nDzCBay94aBvl.jpg","condos\/September2021\/BeAlNNHu2Om5Btw4Od3p.jpg"]                 []                                                                                                                                                                                                        central-towers   1500.00     122830.0000  Bakı    Yasamal r.
2  462               SkyHome       1          18  2023-12-12      1.00             NaN  kristalabsheron.az/az/project/index/53/skyhome  office@kristalabsheron.az                                                    *1544            3.0          3.0       16-18              NaN                      None              2                                                      1            5       None  40.38811800  49.81547200                   0                           None                             NaN                None         None       None                  None       0          1.00                         1.0                    1.00              1         1                  10.00               20.0                    20.00          None       None       None   0.0                                                                                                                                                                                                                                                                                                                                                                                                                                                             0         1         NaN       2.0            0           1          None     condos/August2021/KLh7WNZqsWizOytX6ABU.jpg                                                                                                                                                                                                                                                                                                                                                                                          ["condos\/August2021\/YkSXADyK9Q75mjasBwvJ.jpg","condos\/August2021\/hem8X1Mhq6loKwzPTkab.jpg","condos\/August2021\/nZY36EpVaixrOBNZKm26.jpg"]               None                                                                                                                                                                                                               skyhome   1750.00      94675.0000  Bakı     Nizami r.
3  461              Yuqa MTK       1         271  2021-08-08      None             NaN                                       resant.az  realestate@resantgroup.az                                *4445,  +994 50 505 13 33            1.0          1.0          16             96.0                         6              2                                                      1            3       None  40.40601900  49.86819800                   0                           None                             NaN                None         None       None                  None       1          0.00                        24.0                   30.00              1         0                   None                NaN                     None          None       None       None   1.0                                                                                                                                                                                                                                                                                                                                                                                                                                                             1         1         NaN       NaN            0           1          None     condos/August2021/HtY9SPYpvCMy2AzSIYo6.jpg  ["condos\/August2021\/rSB7TAGpKy5bWG4YlpMa.jpg","condos\/August2021\/moXSg5i7ovKaM4Mxaxog.jpg","condos\/August2021\/HfTYzb3miKSLUp3nx6ZK.jpg","condos\/August2021\/WVQErnMZWlNIm08aCZxE.jpg","condos\/August2021\/AABftVGeTNkAcLOPjk20.jpg","condos\/August2021\/oj8f2wmWDmTxK2TvuJaX.jpg","condos\/August2021\/7VVuckWYC1pEquPmKX1c.jpg","condos\/August2021\/JC2T54WRukjFNoPRp63y.jpg","condos\/August2021\/erAitfcyK2LAdzZUuYwY.jpg","condos\/August2021\/hrMR8D5hlk0EAHmk18Y4.jpg","condos\/August2021\/jVldUFH35AssuMj0ZBdx.jpg"]               None                                                                                                                                                                                                              yuqa-mtk   2000.00     180000.0000  Bakı  Nərimanov r.
4  460               Zəfər 1       1         211  2023-09-09      None             NaN                              zefer1.rezidens.az           info@rezidens.az  +994 50 292 11 11, +994 55 292 11 11, +994 70 292 11 11            1.0          1.0          14             65.0                         5              2  Bakıxanov qəsəbəsi, S. Mehmandarov küçəsi, 5        1            7       None  40.38286700  49.96533800                   0                           None                             NaN                None         None       None                  None       1          0.00                        36.0                   50.00              1         0                   None                NaN                     None          None       None       None   1.0               Zəfər 1       Az mənzilli bina - Zəfər 1 layihəsi                 zəfər 1, zəfər_1, zəfər-1, biznes klass kompleksi, lalafo, korter, bina, yeniemlak, residence yaşayış kompleksi, yeniemlak, kreditlə yaşayış kompleksindəki mənzillər, yaşayış kompleksindəki mənzillərin qiyməti, yaşayış kompleksində mənzillərin alınması, kredit, ipoteka, Bakı, satış, yeni bina, yeni tikililər, mənzillər, mənzillər, otaqlar , modern, layihe       0         1         NaN      15.0            0           1          None     condos/August2021/7FKhwnY5qCd4W3owxKV1.jpg                                                                                                                                                                                              ["condos\/August2021\/8XZe2oTNyYEOUR6xDmx9.jpg","condos\/August2021\/WHfgTRvFM2lyYuPtvdSy.jpg","condos\/August2021\/ypBFEfcJLzjirVYs3QtD.jpg","condos\/August2021\/QrNYHSt3BvB7uRZ8SBFV.jpg","condos\/August2021\/QjXVTS03mGrSAwbJbknM.jpg","condos\/August2021\/uKtJRJoB9H2bQuWYqMcO.jpg","condos\/August2021\/pp7xqqrGoLznhxZ8pPJu.jpg"]               None                                    <p>Baxış istiqamətindən və mərtəbədən asılı olmayaraq qiymətlər 1,250 manatdan başlayır</p>\n<p>&nbsp;</p>                                                                 zefer-1      0.00          0.0000  Bakı    Sabunçu r.