分析数据，Python

Question

我对Python很感兴趣。我需要解析弗雷德(https://fred.stlouisfed.org/series/A191RI1A225NBEA)的数据。因此，我需要让Python访问这个网站，然后单击“下载”按钮，然后单击“CSV(数据)”按钮，然后将这些数据保存在Python中。实际上，这个任务有一个很大的问题，我尝试使用selenium：

selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome('/Users/davidabramyan/Downloads/chromedriver 3')
browser = browser.get('https://fred.stlouisfed.org/series/A191RI1A225NBEA')
elm = browser.find_element_by_link_text('Download')
browser.implicitly_wait(5)
elm.click()

并使用BeautifulSoup：

response = urllib.request.urlopen(url)
return response.read()

def parse(html):
soup = BeautifulSoup(html, 'lxml')
table = soup.find('span', class_='pull-right col-xs-1')
rows = table.find('fg-download-menu')
print(table.prettify())

def main():
parse(get_html('https://fred.stlouisfed.org/series/A191RI1A225NBEA#0'))

if __name__ == '__main__':
main()

但不起作用。我认为美丽的汤更好，但在那里我不知道怎么做。你能帮我一下吗？提前谢谢你！

Answer 1

您不能使用BeautifulSoup，因为下载"CSV (数据)“的url是由ajax生成的

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait

browser = webdriver.Chrome('/Users/davidabramyan/Downloads/chromedriver 3')
# make sure above is chromedriver executable path not directory
wait = WebDriverWait(browser, 10)
browser.get('https://fred.stlouisfed.org/series/A191RI1A225NBEA')

dlButton = browser.find_element_by_id("download-button")
dlButton.click()
wait.until(lambda driver: browser.execute_script("return $('#download-data-csv').attr('href') != '#'"))
dlButton = browser.find_element_by_id("download-data-csv")
dlButton.click()

# get url to download with python
# csv_download_url = dlButton.get_attribute('href')
# csv_file = urllib2.urlopen(download)
# ....