昨天的某招聘平台又给我发广告短信,我一不小心点进去了,然后别的没看见,直接弹框免费领取7天免费会员,免费领那就领呗,白嫖的事情,何乐而不为呢?
重点来了,会员可以尊享专栏免费看。

看到这里你以为我又来发广告?要去取关了,那可以去取关了,不然的话,接下来的内容估计对你有难度。
好了废话少说,你看见啥了没有。对,专栏免费看!这难道不是福利吗?啥,只能看7天?
一直卖关子,还能看到这里的,一定是铁粉了,那肯定不会亏待你们的,我们正式开始,免费7天会员为啥是终生免费会员。
很简单啊,免费7天的时间内,我们把所有的内容复制粘贴下来,保存到本地不就可以永久免费看了吗。
你以为资深CV工程师干的啥,那就是ctrl+c 和ctrl+v。可是作为一个比较懒的人来说,这种重复性的操作,还需要手动来,是对一个程序员来说,伤害不大,但是侮辱性极强。
老粉丝之前看过我写的Python爬取网页保存为PDF 就应该知道接下来的操作,还是换汤不换药。
竟然能看见就应该是下载的,具体的原理不细讲了,看看上面的蓝字文章。
重点福利来了:
import os
import time
import requests
import json
import pdfkit
html_template = """
<html style="font-size: 37.5px;" lang="en">
<head>
<meta charset="UTF-8">
<link rel="stylesheet" type="text/css" href="https://s0.lgstatic.com/kw-web-fed/188.80868283.css" crossorigin="anonymous">
</head>
<body style="font-size: 12px; height: 100%; width: 100%;page-break-before: always;page-break-inside: avoid;" >
<div id="app" class="">
<div data-v-3bc7277d="" data-v-6cc00bca="" class="pc-detail-wrap">
<div data-v-3bc7277d="" class="right-content-wrap" style="height: 270px;">
<div data-v-39db28e6="" data-v-3bc7277d="" class="main-wrap" style="">
<div data-v-39db28e6="" class="main-title">
{theme}
</div>
<div data-v-39db28e6="" class="main-container">
<div data-v-39db28e6="" class="rich-text-wrap">
{content}
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>
"""
# 从浏览器中把相关value复制出来
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36',
'Cookie': '浏览器的Cookie',
'Host': 'gate.lagou.com',
'Origin': 'https://kaiwu.lagou.com',
'Authorization': '浏览器的Authorization',
'Referer': 'https://kaiwu.lagou.com/course/courseInfo.htm?courseId=440',
'TE': 'Trailers',
'X-L-REQ-HEADER': '{deviceType:1}'
}
# 当前目录文件夹,需要替换成这个文件的目录路径
folderName = 'E:\\pythonWorkSpace\\hello\\lagoujiaoyu'
# 通过url获取到html保存到本地
def parse_url_to_html(courseName, courseId):
url = "https://gate.lagou.com/v1/neirong/kaiwu/getCourseLessonDetail?lessonId=" + str(courseId)
response = requests.get(url, headers=headers, timeout=30, allow_redirects=False)
jsonObject = json.loads(response.text)
theme = jsonObject['content']['theme']
textContent = jsonObject['content']['textContent']
html = html_template.format(theme=theme, content=textContent)
name = theme.replace(" ", "").replace("|", "").replace(":", "-").replace("?", "") + ".html"
with open(folderName + "/" + courseName + "/" + name, 'w', encoding="utf-8") as f:
f.write(str(html))
return name
# html 转成pdf
def html_to_pdf(htmlname, pdfname):
options = {
'page-size': 'Letter',
'margin-top': '0.75in',
'margin-right': '0.75in',
'margin-bottom': '0.75in',
'margin-left': '0.75in',
'encoding': "UTF-8",
'custom-header': [
('Accept-Encoding', 'gzip')
],
'cookie': [
('cookie-name1', 'cookie-value1'),
('cookie-name2', 'cookie-value2'),
],
'outline-depth': 10,
}
#wkhtmltopdf 需要替换成自己电脑的安装路径
config = pdfkit.configuration(wkhtmltopdf='d:\\Program Files\\wkhtmltopdf\\bin\\wkhtmltopdf.exe')
pdfkit.from_file(folderName + "/" + courseName + "/" + htmlname,
folderName + "/" + courseName + "/" + pdfname,
configuration=config,
options=options)
os.remove(folderName + "/" + courseName + "/" + htmlname)
# 通过课程id获取下面的文章ids
def getCourseId(courseId):
url = "https://gate.lagou.com/v1/neirong/kaiwu/getCourseLessons?courseId=" + str(courseId)
response = requests.get(url, headers=headers, timeout=30, allow_redirects=False)
jsonObject = json.loads(response.text)
# 创建专栏文件夹
courseName = jsonObject['content']['courseName']
courseName = courseName.replace(" ", "")
# 保存文章id
courseIdList = set()
courseSectionList = jsonObject['content']['courseSectionList']
for courseSection in courseSectionList:
for courseLesson in courseSection['courseLessons']:
courseIdList.add(courseLesson['id'])
return courseName, courseIdList
# 创建文件夹
def newFolder(folder_name):
isExists = os.path.exists(folderName + "/" + folder_name)
if not isExists:
try:
os.mkdir(folderName + "/" + folder_name)
except Exception:
pass
if __name__ == '__main__':
# 这是输入课程id bookId=18, 在浏览器的路径中找courseId=后面的那个数
bookId = 500
courseName, courseIdList = getCourseId(bookId)
print("{}电子书下载开始".format(courseName))
newFolder(courseName)
for courseId in courseIdList:
name = parse_url_to_html(courseName, courseId)
html_to_pdf(name, name.replace(".html", ".pdf"))
print(name)
time.sleep(1)
print("{}电子书下载完成".format(courseName))使用步骤:









注意事项: