使用HtmlAgilityPack

Question

我试图从HTML页面中获取一个链接和另一个元素，但我真的不知道该怎么做。这就是我现在所拥有的：

var client = new HtmlWeb(); // Initialize HtmlAgilityPack's functions.
var url = "http://p.thedgtl.net/index.php?tag=-1&title={0}&author=&o=u&od=d&page=-1&"; // The site/page we are indexing.
var doc = client.Load(string.Format(url, textBox1.Text)); // Index the whole DB.
var nodes = doc.DocumentNode.SelectNodes("//a[@href]"); // Get every url.

string authorName = "";
string fileName = "";
string fileNameWithExt;

foreach (HtmlNode link in nodes)
{
    string completeUrl = link.Attributes["href"].Value; // The complete plugin download url.

    #region Get all jars

    if (completeUrl.Contains(".jar")) // Check if the url contains .jar
    {
        fileNameWithExt = completeUrl.Substring(completeUrl.LastIndexOf('/') + 1); // Get the filename with extension.
        fileName = fileNameWithExt.Remove(fileNameWithExt.LastIndexOf('.')); ; // Get the filename without extension.
        Console.WriteLine(fileName);
    }

    #endregion

    #region Get all Authors

    if (completeUrl.Contains("?author=")) // Check if the url contains .jar
    {
        authorName = completeUrl.Substring(completeUrl.LastIndexOf('=') + 1); // Get the filename with extension.
        Console.WriteLine(authorName);
    }

    #endregion
}

我试着把所有的文件名和作者放在一起，但现在所有的东西都是随机放置的，为什么？

有人能帮我一下吗？谢谢!

Answer 1

如果你看一下HTML，非常不幸的是它不是格式良好的。有许多开放标签，HAP的结构方式不像浏览器，它将文档的大部分解释为深度嵌套。因此，您不能像在浏览器中那样简单地迭代表的各行，它会变得更复杂。

在处理这类文档时，您必须对查询进行相当大的更改。不是搜索子元素，而是搜索子元素来调整更改。

var title = System.Web.HttpUtility.UrlEncode(textBox1.Text);
var url = String.Format("http://p.thedgtl.net/index.php?title={0}", title);

var web = new HtmlWeb();
var doc = web.Load(url);

// select the rows in the table
var xpath = "//div[@class='content']/div[@class='pluginList']/table[2]";
var table = doc.DocumentNode.SelectSingleNode(xpath);

// unfortunately the `tr` tags are not closed so HAP interprets
// this table having a single row with multiple descendant `tr`s
var rows = table.Descendants("tr")
    .Skip(1); // skip header row

var query =
    from row in rows
    // there may be a row with an embedded ad
    where row.SelectSingleNode("td/script") == null
    // each row has 6 columns so we need to grab the next 6 descendants
    let columns = row.Descendants("td").Take(6).ToList()
    let titleText = columns[1].Elements("a").Select(a => a.InnerText).FirstOrDefault()
    let authorText = columns[2].Elements("a").Select(a => a.InnerText).FirstOrDefault()
    let downloadLink = columns[5].Elements("a").Select(a => a.GetAttributeValue("href", null)).FirstOrDefault()
    select new
    {
        Title = titleText ?? "",
        Author = authorText ?? "",
        FileName = Path.GetFileName(downloadLink ?? ""),
    };

因此，现在您可以遍历查询并为每一行写出所需的内容。

foreach (var item in query)
{
    Console.WriteLine("{0} ({1})", item.FileName, item.Author);
}