在弹性搜索索引模板中使用Ingest附件插件

Question

我正在尝试更新我当前的弹性搜索模式，该模式位于1.3.2上，是最新的一个。对于其中一个索引，当前模式如下所示：

curl -XPOST localhost:9200/_template/<INDEXNAME> -d '{
    "template" : "*-<INDEXNAME_TYPE>",
    "index.mapping.attachment.indexed_chars": -1,
    "mappings" : {
        "post" : {
            "properties" : {
                "sub" : { "type" : "string" },
                "sender" : { "type" : "string" },
                "dt" : { "type" : "date", "format" : "EEE, d MMM yyyy HH:mm:ss Z" },
                "body" : { "type" : "string"},
                "attachments" : {
                    "type" : "attachment",
                    "path" : "full",
                    "fields" : {
                        "attachments" : {
                            "type" : "string",
                            "term_vector" : "with_positions_offsets",
                            "store" : true
                        },
                        "name" : {"store" : "yes"},
                        "title" : {"store" : "yes"},
                        "date" : {"store" : "yes"},
                        "content_type" : {"store" : "yes"},
                        "content_length" : {"store" : "yes"}
                    }
                }
            }
        }
    }
}'

与我的旧版本的弹性搜索，有一个“地图-附件”插件安装。我知道“映射器-附件”插件已经被"Ingest附件处理器“所取代，并且按照插件网站的例子，我理解了它们的例子，我需要创建一个管道，

PUT _ingest/pipeline/attachment
  {
    "description" : "Extract attachment information from arrays",
    "processors" : [
      {
        "foreach": {
          "field": "attachments",
          "processor": {
            "attachment": {
              "target_field": "_ingest._value.attachment",
              "field": "_ingest._value.data",
              "indexed_chars" : -1
            }
          }
        }
      }
    ]
  }

  PUT my-index-000001/_doc/my_id?pipeline=attachment
  {
    "sub" : "This is a test post",
    "sender" : "jane.doe@gmail.com",
    "dt" : "Sat, 15 Jan 2022 08:50:00 AEST"
    "body" : "Test Body",
    "fromaddr": "jane.doe@gmail.com",
    "toaddr": "larne.jones@gmail.com",
    "attachments" : [
      {
        "filename" : "ipsum.txt",
        "data" : "dGhpcyBpcwpqdXN0IHNvbWUgdGV4dAo="
      },
      {
        "filename" : "test.txt",
        "data" : "VGhpcyBpcyBhIHRlc3QK"
      }
    ]
  } 

如何使用这个新的附件处理器来创建我以前拥有的索引模板？

注意:对于我的索引和模式，对于每个"post"，将有一个或多个附件，

Answer 1

答案是，与以前的版本不同，我不能使用附件的数据类型。因此，按照elastic.co网站和我自己的问题的例子，答案就在我的问题本身。

• 第一:创建管道，如问题中所示
• 第二，创建模式(见下文)
• 第三，插入问题中所示的数据。当将数据插入索引时，使用pipeline=attachment作为管道的名称，插件将将给定的附件解析到上面的模式中。

curl -XPOST localhost:9200/_template/<INDEXNAME> -d '{
    "template" : "*-<INDEXNAME_TYPE>",
    "index.mapping.attachment.indexed_chars": -1,
    "mappings" : {
        "post" : {
            "properties" : {
                "sub" : { "type" : "string" },
                "sender" : { "type" : "string" },
                "dt" : { "type" : "date", "format" : "EEE, d MMM yyyy HH:mm:ss Z" },
                "body" : { "type" : "string"},
                "attachments" : {
                    "properties" : {
                        "attachment" : {
                            "properties" : {
                                "content" : { 
                                    "type" : "text",
                                    "store": true,
                                    "term_vector": "with_positions_offsets"
                                 },
                                "content_length" : { "type" : "long" },
                                "content_type" : { "type" : "keyword" },
                                "language" : { "type" : "keyword"},
                                "date" : { "type" : "date", "format" : "EEE, d MMM yyyy HH:mm:ss Z" }
                            }
                        },
                        "content" : { "type": "keyword" },
                        "name" : { "type" : "keyword" }
                    }
                }
            }
        }
    }
}'