解析http链接中的信息，以便将文件移动和重新排序到类别文件夹中

Question

要爬行和重命名文件名，我使用以下PowerShell代码

gci *.pdf | foreach { (iwr "https://arxiv.org/abs/$($_.BaseName)") -match 'primary-subject">(.*?)</span>'; $matches[1] }

但是我需要订购并将它们移到相关的主题文件夹中。

，这是HTTP 链接。

文本文件是这样格式化的。

[1] arXiv:1611.00024 [pdf, other]
    Symmetry, Outer Bounds, and Code Constructions: A Computer-Aided Investigation on the Fundamental Limits of Caching
    Chao Tian
    Comments: 34 pages, 7 figures; submitted to IEEE Trans. Information Theory
    Subjects: Information Theory (cs.IT)
[2]  arXiv:1611.00044 [pdf, ps, other]
    Optimal Signaling for Secure Communications over Gaussian MIMO Wiretap Channels
    Sergey Loyka, Charalambos D. Charalambous
    Comments: accepted by IEEE Trans. Info. Theory
    Subjects: Information Theory (cs.IT)
[3]  arXiv:1611.00057 [pdf, ps, other]
    Holomorphy of adjoint L functions for quasisplit A2
    Joseph Hundley
    Comments: 18 pages
    Subjects: Number Theory (math.NT)
[4]  arXiv:1611.00066 [pdf, other]
    Many Haken Heegaard splittings
    Alessandro Sisto
    Comments: 12 pages, 3 figures
    Subjects: Geometric Topology (math.GT)
[6]  arXiv:1611.00069 [pdf, other]
    On singular square metrics with vanishing Douglas curvature
    Changtao Yu, Hongmei Zhu
    Comments: 12 pages, 2 figures
    Subjects: Differential Geometry (math.DG); Metric Geometry (math.MG)
[7]  arXiv:1611.00071 [pdf, ps, other]
    Eigenvalues of rotations and braids in spherical fusion categories
    Daniel Barter, Corey Jones, Henry Tucker
    Subjects: Quantum Algebra (math.QA); Representation Theory (math.RT)

我需要从每个文件文本块中提取主题类别名称。

Subjects: Information Theory (cs.IT)
Subjects: Information Theory (cs.IT)
Subjects: Number Theory (math.NT)
Subjects: Geometric Topology (math.GT)
Subjects: Differential Geometry (math.DG)
Subjects: Quantum Algebra (math.QA)

我不想包括第二个标签

Metric Geometry (math.MG)
Representation Theory (math.RT)

所以最后我应该把这个文件夹做成这样

[folder] Information Theory (cs.IT)
[folder] Number Theory (math.NT)
[folder] Geometric Topology (math.GT)
[folder] Differential Geometry (math.DG)
[folder] Quantum Algebra (math.QA)

并将最后的文件移到类似以下的相关主题文件夹中

[folder] Information Theory (cs.IT)
    |
    |__ [file] Symmetry, Outer Bounds, and Code Constructions...
    |__ [file] Optimal Signaling for Secure Communications...

[folder] Number Theory (math.NT)
    |
    |__ [file] Holomorphy of adjoint L functions...

Answer 1

我不知道这是不是你的搜索但是我的新代码

Function Clean-InvalidFileNameChars {
  param(
    [Parameter(Mandatory=$true,
      Position=0,
      ValueFromPipeline=$true,
      ValueFromPipelineByPropertyName=$true)]
    [String]$Name
  )

  $invalidChars = [IO.Path]::GetInvalidFileNameChars() -join ''
  $re = "[{0}]" -f [RegEx]::Escape($invalidChars)
  $res=($Name -replace $re)
  return $res.Substring(0, [math]::Min(260, $res.Length))
}

Function Clean-InvalidPathChars {
  param(
    [Parameter(Mandatory=$true,
      Position=0,
      ValueFromPipeline=$true,
      ValueFromPipelineByPropertyName=$true)]
    [String]$Name
  )

  $invalidChars = [IO.Path]::GetInvalidPathChars() -join ''
  $re = "[{0}]" -f [RegEx]::Escape($invalidChars)
  $res=($Name -replace $re)
  return $res.Substring(0, [math]::Min(248, $res.Length))
}


$rootpath="c:\temp2"

$rootpathresult="c:\tempresult"

$template=@'
[3]  arXiv:1611.00057 [pdf, ps, other] 
Title: {title*:Holomorphy of adjoint $L$ functions for quasisplit A2} 
Authors: Joseph Hundley 
Comments: 18 pages 
Subjects: {subject:Number Theory (math.NT)} 
[4]  arXiv:1611.00066 [pdf, other] 
Title: {title*:Many Haken Heegaard splittings} 
Authors: Alessandro Sisto 
Comments: 12 pages, 3 figures 
Subjects: {subject:Geometric Topology (math.GT)} 
[5]  arXiv:1611.00067 [pdf, ps, other] 
Title: {title*:Subsumed homoclinic connections and infinitely many coexisting attractors in piecewise-linear maps} 
Authors: David J.W. Simpson, Christopher P. Tuffley 
Subjects: {subject:Dynamical Systems (math.DS)} 
[21]  arXiv:1611.00114 [pdf, ps, other] 
Title: {title*:Faces of highest weight modules and the universal Weyl polyhedron} 
Authors: Gurbir Dhillon, Apoorva Khare 
Comments: We recall preliminaries and results from the companion paper arXiv:1606.09640 
Subjects: {subject:Representation Theory (math.RT)}; Combinatorics (math.CO); Metric Geometry (math.MG)
'@

#extract utils data and clean 
$listbook=gci $rootpath -File -filter *.pdf | foreach { New-Object psobject -Property @{file=$_.fullname; books= ((iwr "https://arxiv.org/abs/$($_.BaseName)").ParsedHtml.body.outerText | ConvertFrom-String -TemplateContent $template)}} | select file -ExpandProperty books |  select file,  @{N="Subject";E={Clean-InvalidPathChars $_.subject}}, @{N="Title";E={Clean-InvalidFileNameChars $_.title}}  



#build dirs and copy+rename file
$listbook | %{$newpath="$rootpathresult\$($_.subject)"; New-Item -ItemType Directory -Path "$newpath" -Force;  Copy-Item $_.file "$newpath\$($_.title).pdf" -Force}

Answer 2

Function Clean-InvalidFileNameChars {
  param(
    [Parameter(Mandatory=$true,
      Position=0,
      ValueFromPipeline=$true,
      ValueFromPipelineByPropertyName=$true)]
    [String]$Name
  )

  $invalidChars = [IO.Path]::GetInvalidFileNameChars() -join ''
  $re = "[{0}]" -f [RegEx]::Escape($invalidChars)
  $res=($Name -replace $re)
  return $res.Substring(0, [math]::Min(260, $res.Length))
}

Function Clean-InvalidPathChars {
  param(
    [Parameter(Mandatory=$true,
      Position=0,
      ValueFromPipeline=$true,
      ValueFromPipelineByPropertyName=$true)]
    [String]$Name
  )

  $invalidChars = [IO.Path]::GetInvalidPathChars() -join ''
  $re = "[{0}]" -f [RegEx]::Escape($invalidChars)
  $res=($Name -replace $re)
  return $res.Substring(0, [math]::Min(248, $res.Length))
}


$res=Invoke-WebRequest "https://arxiv.org/list/math/1611"
$rootpath="c:\temp"

$template=@'
[3]  arXiv:1611.00057 [pdf, ps, other] 
Title: {title*:Holomorphy of adjoint $L$ functions for quasisplit A2} 
Authors: Joseph Hundley 
Comments: 18 pages 
Subjects: {subject:Number Theory (math.NT)} 
[4]  arXiv:1611.00066 [pdf, other] 
Title: {title*:Many Haken Heegaard splittings} 
Authors: Alessandro Sisto 
Comments: 12 pages, 3 figures 
Subjects: {subject:Geometric Topology (math.GT)} 
[5]  arXiv:1611.00067 [pdf, ps, other] 
Title: {title*:Subsumed homoclinic connections and infinitely many coexisting attractors in piecewise-linear maps} 
Authors: David J.W. Simpson, Christopher P. Tuffley 
Subjects: {subject:Dynamical Systems (math.DS)} 
[21]  arXiv:1611.00114 [pdf, ps, other] 
Title: {title*:Faces of highest weight modules and the universal Weyl polyhedron} 
Authors: Gurbir Dhillon, Apoorva Khare 
Comments: We recall preliminaries and results from the companion paper arXiv:1606.09640 
Subjects: {subject:Representation Theory (math.RT)}; Combinatorics (math.CO); Metric Geometry (math.MG)
'@

#get date and cut with format template, group by Subject and clean Title and Subject for transformation to dir and file name
$grousubject=$res.ParsedHtml.body.outerText | ConvertFrom-String -TemplateContent $template | select  @{N="Subject";E={Clean-InvalidPathChars $_.subject}}, @{N="Title";E={Clean-InvalidFileNameChars $_.title}} | group Subject 

#create dir and files
$grousubject | %{$path= "$rootpath\$($_.Name)" ; $_.group.title | %{New-Item -ItemType File -Path "$path\$_" -Force}   }