成功连接nutch 1.12与solr 6.5,并爬行未经认证的网站.当我试图抓取经过认证的网站时,我无法继续。有人能帮我克服它吗。
错误:
java.lang.RuntimeException: java.lang.IllegalArgumentException: No form exists: user-login
at org.apache.nutch.protocol.httpclient.Http.resolveCredentials(Http.java:485)
at org.apache.nutch.protocol.httpclient.Http.getResponse(Http.java:180)
at org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(HttpBase.java:261)
at org.apache.nutch.fetcher.FetcherThread.run(FetcherThread.java:295)
Caused by: java.lang.IllegalArgumentException: No form exists: user-login
at org.apache.nutch.protocol.httpclient.HttpFormAuthentication.getLoginFormParams(HttpFormAuthentication.java:183)
at org.apache.nutch.protocol.httpclient.HttpFormAuthentication.login(HttpFormAuthentication.java:95)
at org.apache.nutch.protocol.httpclient.Http.resolveCredentials(Http.java:483)httpclient-auth.xml:
<auth-configuration>
<credentials authMethod="formAuth"
loginUrl="<url>"
loginFormId="user-login"
loginRedirect="true">
<loginPostData>
<field name="name"
value="*<name>*"/>
<field name="pass"
value="*<password>*"/>
<field name="op"
value="Log in"/>
</loginPostData>
</credentials>
</auth-configuration>搜索了几个链接但没能解决。
谢谢。
发布于 2017-07-06 11:46:13
你如何识别需要提交的表格?在httpclient-auth.xml文件中放置以下内容:
loginFormId="user-login" 所以,它试图通过id找到登录表单的插件,这意味着表单必须有user-login id,将这个值更改为表单登录的正确id,您正在尝试爬行。
https://stackoverflow.com/questions/44947345
复制相似问题