自动下载站点动态生成的http文件

Question

我正在尝试自动下载report http://www.wesm.ph。但是，页面按需生成文件，下载URL将生成如下内容：

http://www.wesm.ph/download.php?download=TUJBT1JURF8yMDE3LTA3LTI2XzIwMTctMDctMjZfR19MVVpPTi5jc3Y=

有可能实现自动化吗？谢谢。

Answer 1

此脚本将获取最新的文件。

将$folderPath设置为保存CSV的文件夹。

$folderPath = 'C:\Users\Michael\Downloads'

# If I can't download something from the last two weeks 
# then use this flag to track the error. 
$downloadSucceeded = $false

function giveBinaryEqualFile ([string] $myInput, [string] $fileName)
{
    $Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
    [System.IO.File]::WriteAllText($fileName, $myInput, $Utf8NoBomEncoding)
}

function generateAddress ([DateTime] $myDate)
{
    $localTimeZone = [System.TimeZoneInfo]::Local
    $PhilippineTimeZone = [System.TimeZoneInfo]::FindSystemTimeZoneById("China Standard Time")

    $PhilippineNow = [System.TimeZoneInfo]::ConvertTime($myDate, $localTimeZone, $PhilippineTimeZone)

    # Address
    $address = "http://www.wesm.ph/download.php?download="

    # What is the file name? 
    $dateInName = Get-Date -Date $PhilippineNow -Format 'yyyy-MM-dd'
    $nameInURL = "MBAORTD_{0}_{0}_G_LUZON.csv" -f $dateInName
    $fileName  =     "RTD_{0}_{0}_G_LUZON.csv" -f $dateInName

    # Base64 Encode
    $byteArray = [System.Text.Encoding]::UTF8.GetBytes($nameInURL)
    $encodedFileName = [System.Convert]::ToBase64String($byteArray)

    # URL
    $url = $address + $encodedFileName

    # Object 
    $properties = @{
        'address'  = $url
        'fileName' = $fileName
    }

    New-Object PSObject -Property $properties
}


# Try to download the latest file. 
# Search the last two weeks. 
:latest for($i=0; $i -ge -14; $i--)
{
    $localNow = (Get-Date).AddDays($i)
    $name = generateAddress $localNow
    $myRequest = Invoke-WebRequest -Uri $name.address

    # Skip this URL if the file length is zero. 
    if ($myRequest.RawContentLength -eq 0)
    {  continue latest  }

    # Skip this URL if we get the 404 page. 
    foreach ($element in $myRequest.AllElements) 
    {   
        if ($element.class -eq 'error')
        {  continue latest  }
    }

    # We did not see an error message. 
    # We must have the file. 

    # Save the file. 
    $myPath = Join-Path $folderPath ($name.fileName)
    if (Test-Path -Path $myPath )
    {
        Write-Host "$($name.fileName) already exists. Exiting. "
        exit
    }
    else
    {  giveBinaryEqualFile ($myRequest.Content) $myPath  }

    # Record success. 
    $downloadSucceeded = $true

    # Leave the loop. 
    break latest
}

if ($downloadSucceeded)
{
    Write-Host "The download succeeded."
    Write-Host "File Name: $($name.fileName)"
}
else
{
    Write-Host "The download failed."
    Write-Host "No files available from the last two weeks. "
}

使用网页浏览器下载文件和使用HtmlWebResponseObject下载会产生不同的文件。内容是一样的。但是编码是不同的。并且PowerShell格式化程序会添加一个换行符。所以我Removed the BOM和newline。您还可以重用我的giveBinaryEqualFile()函数来修复其他脚本中的格式问题。

确保我们使用的是菲律宾的time zone。Another example。

Encode the URL per Obsidian Age的评论。

并使用labels尽早跳出循环。