首页
学习
活动
专区
圈层
工具
发布
社区首页 >问答首页 >我需要运行一个程序,一天一次自动抓取数据

我需要运行一个程序,一天一次自动抓取数据
EN

Stack Overflow用户
提问于 2020-04-19 02:12:45
回答 1查看 93关注 0票数 0

我创建了一个程序,它可以抓取一个在线表格,并将其写成Excel文件。为了使这些数据有用,我需要在每天的同一时间抓取它一次(财务数据)。

我计划将代码打包到一个Windows可执行文件中,并让它在我的VPS上自主运行。下面是代码。我该如何将程序循环到每天上午9点整运行一次,并用当前日期创建一个Excel文件?另外,我该如何将程序打包成.exe文件呢?任何帮助都是非常感谢的。

代码语言:javascript
复制
'''
   This program will scrape the Dukascopy sentiment iframe and create a table of all currency pairs from the historical sentiment index.
   it also writes the data to an excel file.
'''
import requests
import re
import json
import pandas as pd

params = {
    'path': 'historical_sentiment_index/data',
    'type': 'swfx',
    'jsonp': '_callbacks____2k8ym6sn2'
}

headers = {
    'Referer': 'https://freeserv.dukascopy.com/2.0/?path=historical_sentiment_index/index&showHeader=false&tableBorderColor=%234bafe9&liquidity=consumers&availableInstruments=EUR/JPY%2CUSD/RUB%2CUSD/HKD%2CAUD/CAD%2CAUD/CHF%2CAUD/JPY%2CAUD/NZD%2CAUD/SGD%2CCAD/CHF%2CCAD/HKD%2CCAD/JPY%2CCHF/JPY%2CCHF/PLN%2CCHF/SGD%2CEUR/AUD%2CEUR/CAD%2CEUR/CHF%2CEUR/DKK%2CEUR/GBP%2CEUR/HKD%2CEUR/NOK%2CEUR/NZD%2CEUR/PLN%2CEUR/RUB%2CEUR/SEK%2CEUR/SGD%2CEUR/TRY%2CGBP/AUD%2CGBP/CAD%2CGBP/CHF%2CGBP/JPY%2CGBP/NZD%2CHKD/JPY%2CNZD/CAD%2CNZD/CHF%2CNZD/JPY%2CSGD/JPY%2CTRY/JPY%2CUSD/CNH%2CUSD/DKK%2CUSD/MXN%2CUSD/NOK%2CUSD/PLN%2CUSD/SEK%2CUSD/SGD%2CUSD/TRY%2CUSD/ZAR%2CZAR/JPY&availableCurrencies=AUD%2CCAD%2CCHF%2CGBP%2CHKD%2CJPY%2CMXN%2CNOK%2CNZD%2CPLN%2CRUB%2CSEK%2CSGD%2CTRY%2CUSD%2CZAR%2CEUR%2CXAG%2CXAU&sort=volume&order=asc&last=true&sixhours=true&oneday=true&fivedays=true&width=100%25&height=1385&adv=popup&lang=en',
}


def main(url):
    with requests.Session() as req:
        r = req.get(
            "https://freeserv.dukascopy.com/2.0/index.php", params=params, headers=headers)
        match = re.search(r"2k8ym6sn2\((\[.*?])", r.text).group(1)
        data = json.loads(match)
        global df
        df = pd.DataFrame(data).set_index("name")
        #print(df)  # For Full DataFrame view.
        #df.to_csv("data.csv")  # to save it in CSV file
        #print(df.loc['AUD/CAD']) # you can have it as list or dict with to_list() or to_dict()

main("https://freeserv.dukascopy.com/2.0/index.php")


#Creating a dataframe of the needed currency pairs to analyze
df2 = df.loc[['EUR/USD',
'USD/CAD','GBP/USD','USD/CHF','USD/JPY','GBP/USD','CAD/CHF','AUD/CAD','AUD/CHF','AUD/JPY','AUD/NZD','AUD/SGD','CAD/CHF','CAD/HKD','CAD/JPY',
'CHF/JPY','CHF/SGD','EUR/AUD','EUR/CAD','EUR/CHF','EUR/HKD','EUR/JPY','EUR/NZD','EUR/SEK','EUR/SGD','EUR/TRY','GBP/AUD','GBP/CAD','GBP/CHF',
'GBP/JPY','GBP/NZD','HKD/JPY','NZD/CAD','NZD/CHF','NZD/JPY','SGD/JPY','TRY/JPY','USD/CNH','USD/HKD','USD/RUB','USD/SEK','USD/SGD','USD/TRY',
'USD/ZAR','XAU/USD','XAG/USD']]

#Dropping the liquidity provider sentiment
df2 = df2.drop(['last_long','sixhours_long','oneday_long','fivedays_long'],axis=1)
#Renaming the columns for clarity
df2.columns = ['Last update','6hours','1day','5days']
#Sort the table into alphabetical order
df2.sort_values(by='name',axis='index',ascending=True,inplace=True)

#Writing the data into an excel file on the desktop
output_path = r'C:/Users/II/Desktop/Sentiment Data/April,18,2020 - Sentiment Data.xlsx'
from pandas import ExcelWriter
from pandas import ExcelFile

writer = ExcelWriter(output_path)
df2.to_excel(writer,'Sheet1',index=True,)
writer.save()
EN

回答 1

Stack Overflow用户

发布于 2020-04-19 13:48:58

只需将以下代码添加到脚本中

代码语言:javascript
复制
import datetime
files_save = str(datetime.date.today().strftime("%d %B %Y"))+'.xlsx'

并使用Task Scheduler安排您的日程

请确保您以管理员身份登录,或者您具有与管理员相同的访问权限。

Start->Control Action->Create ->System Security->Administrative Tools->Task Scheduler and Taska name ->Type单击Next

按照向导进行操作。

票数 0
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/61293975

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档