Groovy HttpBuilder与cookie有关的问题

Question

我需要编写一个web客户端，它可以访问一个遗留的web应用程序，登录到它，从一个/widget页面中提取一些信息，并根据这个页面的HTML做一些工作。我选择使用Groovy/HttpBuilder解决方案，原因超出了这个问题的范围。

唯一的缺点(据我所知)是HttpBuilder不支持在请求之间保留cookie。这是一个主要问题，因为(Java) web应用程序使用JSESSIONID cookie来确定用户是否登录、是否具有权限等等。

因此，首先，如果我上面的断言是不正确的，而且HttpBuilder确实支持跨请求保留cookie，那么请纠正我，也许这里的答案是一个解决方案，它向我展示了如何利用HttpBuilder的这一部分。在这种情况下，我下面所有的代码都是毫无意义的。

假设我是正确的，而且这不是由HttpBuilder处理的，我发现这个极好的解决方案由于某些原因而无法工作，因此我提出了问题。

我对该代码的修改(见上面的链接)如下：

TaskAutomator.groovy
====================
package com.me.myapp.tasker

import groovyx.net.http.ContentType
import groovyx.net.http.Method

class TaskAutomator {
    static void main(String[] args) {
        TaskAutomator tasker = new TaskAutomator()
        String result = tasker.doWork("http://myapp.example.com")

        println result
    }

    String doWork(String baseUrl) {
        CookieRetainingHttpBuilder cookiedBuilder = new CookieRetainingHttpBuilder(baseUrl)
        Map logins = [username: 'user', password: '12345']

        // Go to the main page where we will get back the HTML for a login screen.
        // We don't really care about the response here, so long as its HTTP 200.
        cookiedBuilder.request(Method.GET, ContentType.HTML, "", null)

        // Log in to the app, where, on success, we will get back the HTML for a the
        // "Main Menu" screen users see when they log in. We don't really care about
        // the response here, so long as its HTTP 200.
        cookiedBuilder.request(Method.POST, ContentType.HTML, "/auth", logins)

        // Finally, now that our JSESSIONID cookies is authenticated, go to the widget page
        // which is what we actually care about interacting with.
        def response = cookiedBuilder.request(Method.GET, ContentType.HTML, "/widget", null)

        // Test to make sure the response is what I think it is.
        print response

        String result

        // TODO: Now actually do work based off the response.

        result
    }
}

CookieRetainingHttpBuilder
==========================
package com.me.myapp.tasker

import groovyx.net.http.ContentType
import groovyx.net.http.HTTPBuilder
import groovyx.net.http.HttpResponseDecorator
import groovyx.net.http.Method

class CookieRetainingHttpBuilder {
    private String baseUrl
    private HTTPBuilder httpBuilder
    private List<String> cookies

    CookieRetainingHttpBuilder(String baseUrl) {
        this.baseUrl = baseUrl
        this.httpBuilder = initializeHttpBuilder()
        this.cookies = []
    }

    public def request(Method method, ContentType contentType, String url, Map<String, Serializable> params) {
        httpBuilder.request(method, contentType) { request ->
            uri.path = url
            uri.query = params
            headers['Cookie'] = cookies.join(';')
        }
    }

    private HTTPBuilder initializeHttpBuilder() {
        def httpBuilder = new HTTPBuilder(baseUrl)

        httpBuilder.handler.success = { HttpResponseDecorator resp, reader ->
            resp.getHeaders('Set-Cookie').each {
                String cookie = it.value.split(';')[0]
                cookies.add(cookie)
            }

            reader
        }

        httpBuilder
    }
}

当我运行这段代码时，我得到了下面的堆栈跟踪(我已经剔除了一些不有趣的部分，因为它非常大)：

Exception in thread "main" groovyx.net.http.HttpResponseException: Not Found
    at groovyx.net.http.HTTPBuilder.defaultFailureHandler(HTTPBuilder.java:642)
        ... (lines omitted for brevity)
    at groovyx.net.http.HTTPBuilder$1.handleResponse(HTTPBuilder.java:494)
        ... (lines omitted for brevity)
    at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:506)
    at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:425)
    at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:374)
    at groovyx.net.http.HTTPBuilder$request.call(Unknown Source)
    at com.me.myapp.tasker.CookieRetainingHttpBuilder.request(CookieRetainingHttpBuilder.groovy:20)
        ... (lines omitted for brevity)
    at com.me.myapp.tasker.TaskAutomator.doWork(TaskAutomator.groovy:23)
        ... (lines omitted for brevity)
    at com.me.myapp.tasker.TaskAutomator.main(TaskAutomator.groovy:13)

CookieRetainingHttpBuilder:20是request的这一行

httpBuilder.request(method, contentType) { request ->

有人能明白我为什么要得到这个吗？，另外，我想在TaskAutomater#doWork(...)方法中确认我的方法/策略。CookieRetainingHttpBuilder 是我对“正确”的使用，意思是：

1. 转到主/登录页面
2. POSTing登录页面和登录
3. 转到小部件页面

或者这里是否有一种更好/更有效地使用HttpBuilder的不同方式(记住，CookieRetainingHttpBuilder毕竟只是HttpBuilder的一个包装器)。

Answer 1

我认为这个错误可能是由于缺少导入而出现的，或者可能是HttpBuilder的旧版本。查看HttpBuilder.Class，我看到了这一点，这为我的建议提供了参考：

protected java.lang.Object parseResponse(org.apache.http.HttpResponse resp, java.lang.Object contentType) throws groovyx.net.http.HttpResponseException { /* compiled code */ }

我相当肯定您可以在您的headers.'Set-Cookie设置中使用httpBuilder。语法与您所拥有的不同，但是更改很小而且很简单，这是我在使用HttpBuilder时使用的基本方法。

@Grab(group = 'org.codehaus.groovy.modules.http-builder', module = 'http-builder', version = '0.7)
    import groovyx.net.http.HTTPBuilder
    import org.apache.http.HttpException
    import static groovyx.net.http.ContentType.TEXT
    import static groovyx.net.http.Method.GET

    def http = new HTTPBuilder(urlToHit)
    http.request(urlToHit, GET, TEXT) { req ->

    headers.'User-Agent' = ${userAgent}
    headers.'Set-Cookie' = "${myCookie}"

    response.success = { resp, reader ->
        html = reader.getText()
    }

    response.failure = { resp, reader ->
        System.err.println "Failure response: ${resp.status}"
        throw new HttpException()        
    }        
}

另外要注意的是，您没有故障处理。我不知道这是否会引起一个例外，但它可能值得研究。

编辑建议，我正在合并我的答案(谢谢你让我知道.我不知道什么是适当的礼仪)。

这是我想出来的。我尽了最大努力重用你发布的代码。我尽我所能发表意见。如果你有什么问题，告诉我。

@Grab(group = 'org.codehaus.groovy.modules.http-builder', module = 'http-builder', version = '0.7')
import static groovyx.net.http.ContentType.HTML
import static groovyx.net.http.Method.POST
import static groovyx.net.http.Method.GET
import groovyx.net.http.ContentType
import groovyx.net.http.HTTPBuilder
import groovyx.net.http.URIBuilder
import groovyx.net.http.Method
import org.apache.http.HttpException

/**
 * This class defines the methods used for getting and using cookies
 * @param baseUrl The URL we will use to make HTTP requests. In this example, it is https://www.pinterest.com
 */

class CookieRetainingHttpBuilder {

    String baseUrl

    /**
     * This method makes an http request and adds cookies to the array list for later use
     * @param method The method used to make the http request. In this example, we use GET and POST
     * @param contentType The content type we are requesting. In this example, we are getting HTML
     * @param url The URI path for the appropriate page. For example, /login/ is for the login page
     * @param params The URI query used for setting parameters. In this example, we are using login credentials
     */

    public request (Method method, ContentType contentType, String url, Map<String, Serializable> params) {

        List<String> cookies = new ArrayList<>()

        def http = new HTTPBuilder(baseUrl)

        http.request(baseUrl, method, contentType) { req ->

            URIBuilder uriBuilder = new URIBuilder(baseUrl)
            uriBuilder.query = params
            uriBuilder.path = url

            headers.'Accept' = HTML
            headers.'User-Agent' = "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36"
            headers.'Set-Cookie' = cookies.join(";")

            response.success = { resp, reader ->

                resp.getHeaders('Set-Cookie').each {
                    def cookie = it.value.split(";").toString()
                    cookies.add(cookie)
                }

                return reader

            }

            response.failure = { resp, reader ->
                System.err.println "Failure response: ${resp.status}"
                throw new HttpException()
            }

        }

    }

}

/**
 * This class contains the method to make HTTP requests in the proper sequence
 * @param base The base URL
 * @param user The username of the site being logged in to
 * @param pass The password for the username
 */

class TaskAutomator {

    private static String base = "http://myapp.example.com"
    private static String user = "thisIsMyUser"
    private static String pass = "thisIsMyPassword"

    /**
     * This method contains the functions in proper order to set cookies and login to a site
     * @return response Returns the HTML from the final GET request
     */

    static String doWork () {

        CookieHandler.setDefault(new CookieManager());

        CookieRetainingHttpBuilder cookiedBuilder = new CookieRetainingHttpBuilder(baseUrl: base)
        Map logins = [username: user, password: pass]

        // Go to the main page where we will get back the HTML for a login screen.
        // We don't really care about the response here, so long as its HTTP 200.
        cookiedBuilder.request(GET, HTML, "", null)

        // Log in to the app, where, on success, we will get back the HTML for a the
        // "Main Menu" screen users see when they log in. We don't really care about
        // the response here, so long as its HTTP 200.
        cookiedBuilder.request(POST, HTML, "/login/", logins)

        // Finally, now that our JSESSIONID cookies is authenticated, go to the widget page
        // which is what we actually care about interacting with.
        def response = cookiedBuilder.request(GET, HTML, "/", null)

        // Test to make sure the response is what I think it is.
        return response

        // TODO: Now actually do work based off the response.

    }

}

TaskAutomator tasker = new TaskAutomator()
String result = tasker.doWork()
println result