文档建议反馈控制台
首页
学习
活动
专区
圈层
工具
MCP广场
文章/答案/技术大牛
发布
社区首页 >问答首页 >Goutte -gu包/DomCrawler-从HTML表中抓取-许多复杂的

问Goutte -gu包/DomCrawler-从HTML表中抓取-许多复杂的
EN

Stack Overflow用户
提问于 2016-11-07 16:33:38
回答 1查看 1.9K关注 0票数 0

我开始使用Goutte从网站获取我需要的信息，它的精彩，它节省了我的大量时间和麻烦。但另一方面，有时会出现异常现象，我不知道是什么原因造成的。这是我现在正在刮的一页：http://lifecenter.sgst.cn/hit/search.jsp?key1=%27a%27&key2=%27b%27&key3=%27c%27&key4=%27d%27&key5=C0305

我一直在浏览这个网站，下面的链接和网站上的每一页都有自己的表格。这个表看起来很简单，下面是其中一些的HTML：

代码语言：javascript
复制
<table align="center" border="1" cellpadding="4" cellspacing="1" width="927">
<tbody><tr bgcolor="#3399FF">
<th align="center">ENTRY</th>
<th align="center">COMPOUND</th>
<th align="center">HERB</th>
<th align="center">TARGET ID</th>
<th align="center">TARGET NAME</th>
<th align="center">TARGET TYPE</th>
</tr>
</tbody><tbody id="table2">


<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0595">HIT001882</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0595&amp;protein=Katanin p60 ATPase-containing subunit A1">T0595</a></td>
<td align="center">Katanin p60 ATPase-containing subunit A1</td>
<td align="center">Direct Target</td>
</tr>

<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0596">HIT001883</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0596&amp;protein=Adenosine receptor A2a">T0596</a></td>
<td align="center">Adenosine receptor A2a</td>
<td align="center">Direct Target</td>
</tr>

<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0040">HIT001885</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0040&amp;protein=Nitric oxide synthase, inducible">T0040</a></td>
<td align="center">Nitric oxide synthase, inducible</td>
<td align="center">Direct Target</td>
</tr>

<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0054">HIT001884</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0054&amp;protein=Prostaglandin G/H synthase 2">T0054</a></td>
<td align="center">Prostaglandin G/H synthase 2</td>
<td align="center">Indirect Target</td>
</tr>

</tbody>
</table>

所以我做的第一件事就是选择<tbody>，因为我需要的所有数据都在那里：

代码语言：javascript
复制
    $tbody = $compoundPage->filter('tbody#table2');

当我跑步时，一切看起来都很好：

代码语言：javascript
复制
> exit(dump($compoundPage->filter('tbody#table2')->html()));

这是我得到的：

代码语言：javascript
复制
<tr>\n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0595">HIT001882</a></td>\r\n
    <td align="center">linalool</td>\r\n
    <td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>\r\n
    <td align="center"><a href="Protein_ID.jsp?pid=T0595&amp;protein=Katanin%20p60%20ATPase-containing%20subunit%20A1">T0595</a></td>\r\n
    <td align="center">Katanin p60 ATPase-containing subunit A1</td>\r\n
    <td align="center">Direct Target</td>\r\n
</tr>\n
<tr>\n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0596">HIT001883</a></td>\r\n
    <td align="center">linalool</td>\r\n
    <td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>\r\n
    <td align="center"><a href="Protein_ID.jsp?pid=T0596&amp;protein=Adenosine%20receptor%20A2a">T0596</a></td>\r\n
    <td align="center">Adenosine receptor A2a</td>\r\n
    <td align="center">Direct Target</td>\r\n
</tr>\n
<tr>\n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0040">HIT001885</a></td>\r\n
    <td align="center">linalool</td>\r\n
    <td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Radix Bupleuri chinensis(</td>\r\n
    <td align="center"><a href="Protein_ID.jsp?pid=T0040&amp;protein=Nitric%20oxide%20synthase,%20inducible">T0040</a></td>\r\n
    <td align="center">Nitric oxide synthase, inducible</td>\r\n
    <td align="center">Direct Target</td>\r\n
</tr>\n
<tr>\n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0054">HIT001884</a></td>\r\n
    <td align="center">linalool</td>\r\n
    <td align="center">Ardisiae japonicae</td>\r\n
    <td align="center"><a href="Protein_ID.jsp?pid=T0054&amp;protein=Prostaglandin%20G/H%20synthase%202">T0054</a></td>\r\n
    <td align="center">Prostaglandin G/H synthase 2</td>\r\n
    <td align="center">Indirect Target</td>\r\n
    </tr>

我看到四排。

但是现在我需要遍历行，所以我必须更进一步：

代码语言：javascript
复制
$tbody = $compoundPage->filter('tbody#table2 > tr');

当我运行时：exit(dump($compoundPage->filter('tbody#table2 > tr')->html()));

它只输出一行：

代码语言：javascript
复制
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0595">HIT001882</a></td>\r\n
<td align="center">linalool</td>\r\n
<td align="center">Ardisiae japonicae(ai di cha|矮地茶);</td>\r\n
<td align="center"><a href="Protein_ID.jsp?pid=T0595&amp;protein=Katanin%20p60%20ATPase-containing%20subunit%20A1">T0595</a></td>\r\n
<td align="center">Katanin p60 ATPase-containing subunit A1</td>\r\n
<td align="center">Direct Target</td>\r\n

但当我像这样数行的时候：

退出(转储($compoundPage->filter(‘tbody#table2>tr’)->count()；

它说有4行。

当我迭代这些行时会出现这个问题，如下所示：

代码语言：javascript
复制
    $data = $rows->each(function($row,$i) use ($client) {
      $tds = $row->filter('td');

      $keys = array('entry','compound','herb','target_id','target_name','target_type');

      foreach($keys as $td_i => $key) {
          $data[$key] = $tds->eq($i)->text();
      }

  });

首先，当我计算td元素时，它说只有1。如果我试图对td节点执行任何操作，就会得到以下错误：https://gyazo.com/94042ba2e3101ba918a71cc9e8e3be5a。

如果我这么做：

退出(转储($row->filter(‘td’)；

在循环中，它告诉我有24个节点：

代码语言：javascript
复制
Crawler {#1121 ▼
  #uri: "http://lifecenter.sgst.cn/hit/search.jsp?key1='a'&key2='b'&key3='c'&key4='d'&key5=C0305"
   -defaultNamespacePrefix: "default"
   -namespaces: []
   -baseHref: "http://lifecenter.sgst.cn/hit/search.jsp?key1='a'&key2='b'&key3='c'&key4='d'&key5=C0305"
   -document: DOMDocument {#52 ▶}
   -nodes: array:24 [▼
     0 => DOMElement {#1123 ▶}
     1 => DOMElement {#1124 ▶}
     2 => DOMElement {#1125 ▶}
     3 => DOMElement {#1126 ▶}
     4 => DOMElement {#1127 ▶}
     5 => DOMElement {#1128 ▶}
     6 => DOMElement {#1129 ▶}
     7 => DOMElement {#1130 ▶}
     8 => DOMElement {#1131 ▶}
     9 => DOMElement {#1132 ▶}
     ... (omitted the rest cuz I have to do this indentation manually, its so time consuming.

  ]
  -isHtml: true
php
guzzle
goutte
domcrawler
EN

回答 1

Stack Overflow用户

发布于 2016-11-07 16:44:35

我让它起作用了，我补充说：

$compoundPage->filter('tbody#table2')->filter('tr')；= $rows

而不是这样：

$rows = $compoundPage->filter('tbody#table2 > tr')；

现在它正常工作了。我仍然不知道问题出在哪里，因为这两种选择方法应该是相同的，而且我很肯定我以前尝试过这种方法。在循环遍历表时，我遇到了许多问题，我发现在以下几个方面肯定有不同：

$rows = $compoundPage->filter('tbody#table2 > tr')；

和

$compoundPage->filter('tbody#table2')->filter('tr')；= $rows

我见过CSS选择方法多次破坏脚本。

票数 0
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：

https://stackoverflow.com/questions/40470037

复制
相关文章

相似问题

领券

腾讯云开发者

扫码关注腾讯云开发者

扫码关注腾讯云开发者

领取腾讯云代金券

热门产品

热门推荐

更多推荐

Copyright © 2013 - 2026 Tencent Cloud. All Rights Reserved. 腾讯云 版权所有 

深圳市腾讯计算机系统有限公司 ICP备案/许可证号：粤B2-20090059 粤公网安备44030502008569号

腾讯云计算（北京）有限责任公司 京ICP证150476号 |  京ICP备11018762号

问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档