抓取Netflix

Question

试图以编程的方式从Netflix中获取最后一次查看的数据，但在登录阶段出现了一些问题。我当前的代码只会导致Netflix向后吐出一个We were unable to process your request.页面：

var request = require('request').defaults({jar: true});
var cheerio = require('cheerio');

var url = "https://www.netflix.com/Login?locale=en-GB&nextpage=https%3A%2F%2Fwww.netflix.com%2FWiViewingActivity";

request(url, function (error, response, body) {
    if (!error && response.statusCode == 200) {
        var $ = cheerio.load(body);
        var authCode = $("#login-form > input").attr("value");
        request.post(url+"?email=myemail%40gmail.com&password=mypassword&RememberMe=on&authURL="+authCode, {
        }, function(err, response, body){
            console.log(body);
        });
    }
})

有什么想法吗？

令人惊讶的是，谷歌对于Scraping Netflix几乎什么都没有。

Answer 1

想清楚了，需要：

1. 发送任何类型的用户代理字符串。
2. 使用请求的form参数发送表单数据
3. 手动发送曲奇

下面是我的最后一段代码，它获取了最新的观看项目：

var request = require('request').defaults({jar: true});
var cheerio = require('cheerio');

var url = "https://www.netflix.com/Login";

request(url, function (error, response, body) {
    if (!error && response.statusCode == 200) {
        var $ = cheerio.load(body);
        var authCode = $("#login-form > input").attr("value");
        request.post({url: url, 
            form: {
                "email": "email@gmail.com",
                "password": "password",
                "authURL": authCode,
                "RememberMe": "on"
            },
            headers:{
                'User-Agent': "NodeScrape"
            }
        }, function(err, response, body){
            var cookies = response.headers['set-cookie'];
            request({url: "https://www.netflix.com/WiViewingActivity", headers: {'Cookie': cookies, 'User-Agent': "NodeScrape"}}, function(error, response, body){
                var $ = cheerio.load(body);
                console.log($(".seriestitle").eq(0).text());
            });
        });
    }
})