PySpark中数据库的认知文本分析:不支持字符串和字典/对象文档输入的混合

Question

我的dummy.txt中有以下文本

“雷德蒙德是美国华盛顿州国王县的一座城市，位于西雅图以东15英里处。

该文档已上载到我的datalake文件夹“/dbfs/mnt/lake/RAW/export/dumy.txt”

我用以下代码读取数据：

with open("/dbfs/mnt/lake/RAW/export/dummy.txt", "rb") as fd:
    documents = fd.read()

然后，我将认知文本分析应用于dummy.txt文件中的数据，如下所示：

from azure.core.credentials import AzureKeyCredential
from azure.ai.textanalytics import TextAnalyticsClient

credential = AzureKeyCredential("xxxxxxxxxxxxxxxxxxxx")
endpoint= "https://xxxxxxxx.cognitiveservices.azure.com/"
text_analytics_client = TextAnalyticsClient(endpoint, credential)


response = text_analytics_client.extract_key_phrases(documents, language="en")
result = [doc for doc in response if not doc.is_error]

for doc in result:
    print(doc.key_phrases)

我应该得到以下信息：

['King County', 'United States', 'Redmond', 'city', 'Washington', 'Seattle']

但我得到以下类型错误：

Mixing string and dictionary/object document input unsupported

有人能告诉我该怎么做才能解决这个问题吗？

Answer 1

您可以通过以下方式安装客户端库：

pip install azure-ai-textanalytics==5.1.0

创建一个新的Python文件并复制下面的代码。记住用资源的键替换键变量，用资源的端点替换端点变量。

key = "paste-your-key-here"
endpoint = "paste-your-endpoint-here"

from azure.ai.textanalytics import TextAnalyticsClient
from azure.core.credentials import AzureKeyCredential

# Authenticate the client using your key and endpoint 
def authenticate_client():
    ta_credential = AzureKeyCredential(key)
    text_analytics_client = TextAnalyticsClient(
            endpoint=endpoint, 
            credential=ta_credential)
    return text_analytics_client

    client = authenticate_client()

    def key_phrase_extraction_example(client):

    try:
        documents = ["My cat might need to see a veterinarian."]

        response = client.extract_key_phrases(documents = documents)[0]

        if not response.is_error:
            print("\tKey Phrases:")
            for phrase in response.key_phrases:
                print("\t\t", phrase)
        else:
            print(response.id, response.error)

    except Exception as err:
        print("Encountered exception. {}".format(err))
        
    key_phrase_extraction_example(client)

产出-

Key Phrases:
    cat
    veterinarian

参考- https://learn.microsoft.com/en-us/azure/cognitive-services/language-service/key-phrase-extraction/quickstart?pivots=programming-language-python