mapper [clientip]有不同的[规范]值，不能从禁用更改为启用]

Question

与其他类型的现有映射冲突的Mapper：\n[mapper clientip有不同的规范值，不能从禁用更改为启用]

Elasticsearch 6.2.4

我们使用logstash+elasticsearch来获取cloudfront使用的一些带宽指标。logstash是这样分析事物的：https://gist.github.com/chrisan/1c5ce5beacfc0e124d39fa842f051857#file-logstash-api-conf

这会生成索引，如：https://gist.github.com/chrisan/1c5ce5beacfc0e124d39fa842f051857#file-indicies

具有映射，如：https://gist.github.com/chrisan/1c5ce5beacfc0e124d39fa842f051857#file-mappings

我被要求获得不同的IP地址，并尝试使用聚合查询：

{
  "size": 0,
    "aggs" : {
        "distinct_ips" : {
            "filter" : { "term": { "company" : "XXX" } },
            "aggs" : {
                "cardinality" : { "cardinality": {"field": "clientip"   } }
            }
        }
    }
}

但这一结果是：

{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [clientip] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
            }
        ],
        "type": "search_phase_execution_exception",
        "reason": "all shards failed",
        "phase": "query",
        "grouped": true,
        "failed_shards": [
            {
                "shard": 0,
                "index": "logstash-2018.01.01",
                "node": "dO1JCnAnSmmk5EfDmfYgqQ",
                "reason": {
                    "type": "illegal_argument_exception",
                    "reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [clientip] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
                }
            }
        ]
    },
    "status": 400
}

所以我试着用

PUT /*/_PUT/_doc?update_all_types

{
  "properties": {
    "clientip": {
      "type":     "text",
      "fielddata": true
    }
  }
}

返回：

{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Mapper for [clientip] conflicts with existing mapping in other types:\n[mapper [clientip] has different [norms] values, cannot change from disable to enabled]"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "Mapper for [clientip] conflicts with existing mapping in other types:\n[mapper [clientip] has different [norms] values, cannot change from disable to enabled]"
    },
    "status": 400
}

我做错了什么？

Answer 1

发现后，必须删除所有内容，然后用

{
        "template": "logstash",
        "order": 0,
        "version": 60001,
        "index_patterns": [
            "logstash-*"
        ],
        "settings": {
            "index": {
                "refresh_interval": "5s"
            }
        },
        "mappings": {
            "_default_": {
                "dynamic_templates": [
                    {
                        "message_field": {
                            "path_match": "message",
                            "match_mapping_type": "string",
                            "mapping": {
                                "type": "text",
                                "norms": false
                            }
                        }
                    },
                    {
                        "string_fields": {
                            "match": "*",
                            "match_mapping_type": "string",
                            "mapping": {
                                "type": "text",
                                "norms": false,
                                "fields": {
                                    "keyword": {
                                        "type": "keyword",
                                        "ignore_above": 256
                                    }
                                }
                            }
                        }
                    }
                ],
                "properties": {
                    "@timestamp": {
                        "type": "date"
                    },
                    "@version": {
                        "type": "keyword"
                    },
                    "clientip": {
                        "type":     "text",
                        "fields": {
                        "keyword": { 
                          "type": "keyword"
                        }
                      }
                    },
                    "company": { 
                      "type": "text",
                      "fields": {
                        "keyword": { 
                          "type": "keyword"
                        }
                      }
                    },
                    "geoip": {
                        "dynamic": true,
                        "properties": {
                            "ip": {
                                "type": "ip"
                            },
                            "location": {
                                "type": "geo_point"
                            },
                            "latitude": {
                                "type": "half_float"
                            },
                            "longitude": {
                                "type": "half_float"
                            }
                        }
                    }
                }
            }
        },
        "aliases": {}
    }

然后我就可以用

"aggs": {
   "distinct_ips": {
     "cardinality": {
       "field": "clientip.keyword"
     }
   }
 }