Python -从Azure笔记本中的C驱动器访问文件

Question

我在C盘里有h5文件。我无法在Azure的h5中上传数据集，因为它的大小为468 MB。我如何从代码本身读取它。如果没有Azure，在本地机器上有jupyter笔记本，我可以使用以下代码进行访问：

使用h5py.File('SVHN_single_grey1.h5'，'r')作为hdf：

这在Azure中不起作用，因为它无法访问计算机上的本地文件。

Answer 1

如果你可以让你的H5文件通过url直接从互联网访问，你可以尝试使用下面的代码在Azure Notebook中读取它。

import requests
from io import BytesIO
import h5py

r = requests.get("<an url for accessing your H5 file, such as https://host:port/.../SVHN_single_grey1.h5>")
f = BytesIO(r.content)
with h5py.File(f) as hdf:
    ...

如果没有，您必须首先将H5文件作为资源url发布到internet服务，然后通过上面的代码使用它。我推荐使用Azure官方工具azcopy来帮助将其作为blob上传到Azure Blob Storage，请参考官方教程Tutorial: Migrate on-premises data to cloud storage by using AzCopy了解更多详细信息。然后，您可以按照下面的示例代码再次阅读它。

from azure.storage.blob.baseblobservice import BaseBlobService
from azure.storage.blob import BlobPermissions
from datetime import datetime, timedelta
import requests
from io import BytesIO
import h5py

account_name = '<your account name>'
account_key = '<your account key>'
container_name = '<your container name>'
blob_name = '<your blob name>'

blob_service = BaseBlobService(
    account_name=account_name,
    account_key=account_key
)

sas_token = blob_service.generate_blob_shared_access_signature(container_name, blob_name, permission=BlobPermissions.READ, expiry=datetime.utcnow() + timedelta(hours=1))
# print(sas_token)
url_with_sas = blob_service.make_blob_url(container_name, blob_name, sas_token=sas_token)
# print(url_with_sas)

r = requests.get(url_with_sas)
f = BytesIO(r.content)
with h5py.File(f) as hdf:
    ...

或者，下面是另一个也能正常工作的示例代码。

from azure.storage.blob.baseblobservice import BaseBlobService
from io import BytesIO
import h5py

account_name = '<your account name>'
account_key = '<your account key>'
container_name = '<your container name>'
blob_name = '<your blob name>'

blob_service = BaseBlobService(
    account_name=account_name,
    account_key=account_key
)

stream = BytesIO()
blob_service.get_blob_to_stream(container_name, blob_name, stream)
with h5py.File(stream) as hdf:
    ...