azure计算机视觉检测文本

Question

我正在使用azure计算机视觉从图像中提取文本，它按预期工作，但现在我面临着一个挑战，我必须从图像中检索特定的文本，而不是从图像中检索所有文本，并且我想要提取的文本可能在不同的图像中不同。(也许这些图像可以有一个共同的文本) Azure计算机视觉有什么方法可以帮助我做到这一点吗？示例:配料因为有很多照片，配料与简单的文本混合在一起，我如何才能只提取配料：

​

谢谢

Answer 1

Azure Cognitive Service Vision API能够将连续的文本检测为“区域”，区域由“行”组成，行包含“词”。一种简单的方法是检测某个关键字(如“配料”)是否出现在某个区域中，如果出现，则保存同一区域中该关键字之后出现的所有单词(使用Levenshtein距离以允许某些偏差)：

import requests
import numpy as np
from PIL import Image
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import matplotlib.patches as patches
import Levenshtein  # install with pip install python-Levenshtein

def draw_bounding_box(ax, coordinates, color, size):
    coordinates_array = coordinates.split(",")
    x = int(coordinates_array[0])
    y = int(coordinates_array[1])
    w = int(coordinates_array[2])
    h = int(coordinates_array[3])
    try:
        rect = patches.Rectangle((x, y), w, h, linewidth=size, edgecolor=color, facecolor='none')
        ax.add_patch(rect)
    except:
        pass

def text_detection(service_endpoint, subscription_key, filename):
    with open(filename, 'rb') as f:
        data = f.read()
    headers = {'Ocp-Apim-Subscription-Key': subscription_key, 'Content-Type': 'application/octet-stream'}
    params = {'language': 'fr', 'detectOrientation': 'false'}
    response = requests.post(service_endpoint, headers=headers, params=params, data=data)

    analysis = response.json()
    regions = analysis['regions']

    im = np.array(Image.open(filename), dtype=np.uint8)
    fig, ax = plt.subplots(1)
    ax.imshow(im)
    ingredients = ""

    for region in regions:
        in_ingredients = False
        draw_bounding_box(ax, region["boundingBox"], "g", 3)
        for line in region["lines"]:
            draw_bounding_box(ax, line["boundingBox"], "b", 2)
            for word in line["words"]:
                text = word["text"]
                if Levenshtein.distance(text.lower(), "ingredients") <= 2:
                    in_ingredients = True
                if in_ingredients:
                    ingredients += text + " "

    plt.show()
    print(ingredients)

service_endpoint = "https://<region>.api.cognitive.microsoft.com/vision/v3.1/ocr"
subscription_key = "<your key>"
image_path = "/path/to/image.jpg"

text_detection(service_endpoint, subscription_key, image_path)

输出(绿色边界框为区域，蓝色边界框为线条)：

​

​

INGRÉDIENTS : Nouilles 88 % : farine de blé ; huile de palme amidon modifié de pomme de terre ; sel , poudres à lever E500,E501. Assaisonnement 12 % : sel ; exhausteurs degoù g\utamate monosodique, E635 ; maltodextrine , sucre, carotte lactose ; curry 5 % (curcuma, cumin, cannelle, fenugrec, muscade, cardamome, clou de girofle, gingembre, piment, coriandre, poivre noir) ; extrait de poulet en poudre arômes (dont blé, lait) ; curcuma en poudre ; poireau 2,2 asse de poulet ; extrait de levure ; ail en poudre ; saucede sojaen poudre (soja, blé, maltodextrine, sel) ; ciboulette deshydratée % ; coriandre en poudre ; Piment ; poivrenar colorants : extrait de paprika, E150c. de céleri, crustacés, mollusques, moutarde, œufs, Poissons, graines de sésame et sulfites

另一个例子：

​

​

INGREDIENTS: WATER, TOMATO PASTE), SUGAR! SOYBEAN OIL, DISTILLED OF: MODIÅED CORN STARCH, DEHYDRATED ONION, PHOSPHORIC ACID, XANTHAN GUM, NATURAL FLAVOR, DEHYDRATED GARLIC, POTASSIUM SORBATE AND CALCIUM DISODIUM EDTA (PRESERVATIVES), CITRIC ACID, GUAR GUM, RED 40, YELLOW 6, BLUE 1. OUR PROMISE, QUALITY & SATISFACTION 100% GUARANTEED OR YOUR MONEY BACK

这种方法肯定不是万无一失的。如果关键字出现在文本的另一部分中，或者如果配料表被分割，它将失败，如下例所示：

​

​

ingredient to be naturally derived if it is unchanged from its natural state or has undergone processing yet still retains greater than 50% of its molecular structure from its original plant or mineral source. This formula consists of 96% naturally derived ingredients