文章/答案/技术大牛

发布

社区首页 >问答首页 >使用正则表达式的Kibana不能按预期工作

问使用正则表达式的Kibana不能按预期工作
EN

Stack Overflow用户

提问于 2018-10-21 02:23:06

回答 1查看 329关注 0票数 2

我对Kibana和Elastic search的使用非常陌生，但我有它的工作，可以搜索字符串等没有问题。

但是我想在这个正则表达式上进行搜索

threshold of 1000 ms: ([5-9],\d+|[1-9][0-9],\d+|[1-9][0-9][0-9],\d+)

其工作原理与使用https://regex101.com/和Notepad++测试的结果相同。但是无论我怎么尝试，我都不能让它在Kibana中运行。

我尝试将索引字段改为关键字而不是文本，但仍然不起作用。

即使是像这样的简单正则表达式

message:/192.168.99.[0-9]{3}/

并不像你所期望的那样工作。

有没有人能给我个建议，让它正常工作？快把我逼疯了。

我应该提一下，我使用的是Filebeat 6.4.2、Elasticsearch 6.4.2和Kibana 6.4.2

谢谢

映射

"mappings": {
  "doc": {
    "date_detection": false,
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "@version": {
        "type": "long"
      },
      "CorrelationId": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "InteractionNumber": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "RequestorId": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "alertType": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "app": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "beat": {
        "properties": {
          "hostname": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "name": {
            "type": "keyword",
            "ignore_above": 1024
          },
          "version": {
            "type": "keyword",
            "ignore_above": 1024
          }
        }
      },
      "host": {
        "properties": {
          "name": {
            "type": "keyword",
            "ignore_above": 1024
          }
        }
      },
      "input": {
        "properties": {
          "type": {
            "type": "keyword",
            "ignore_above": 1024
          }
        }
      },
      "level": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "logger_name": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "message": {
        "type": "keyword"
      },
      "nodeId": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "offset": {
        "type": "long"
      },
      "pegathread": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "prospector": {
        "properties": {
          "type": {
            "type": "keyword",
            "ignore_above": 1024
          }
        }
      },
      "source": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "source_host": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "src-env": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "src-node": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "src-vm": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "stack": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "tenantid": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "thread_name": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "userid": {
        "type": "keyword",
        "ignore_above": 1024
      },
      "x-requested-with": {
        "type": "keyword",
        "ignore_above": 1024
      }
    }
  }
}

消息字段内容示例

2018-10-20 23:10:21,068 GMT*8*PEGA0001*8087*1000*8ce767fc2b32*NA*NA*HKVZWM7PHSLMGR3ZXP4OEKEBG3DFFS30K*Test.User*Case-CAS-FS-Work-Svc*Solution:01.03.01*00cb8b6febb234d359369e54a60a865f*Y*3*HKVZWM7PHSLMGR3ZXP4OEKEBG3DFFS30K*35*http-apr-8080-exec-26*STANDARD*com.pega.pegarules.session.internal.engineinterface.service.HttpAPI*192.168.99.100|192.168.99.1*Activity=Pega-UI-CommandPalette.pzGetPaletteOptions*Rule-Obj-Activity:pzGetPaletteOptions*PEGA-UI-COMMANDPALETTE PZGETPALETTEOPTIONS #20161123T194957.445 GMT Step: 2 Circum: 0*NA*****pxRDBIOElapsed=0.03;pxRDBIOCount=4;pxRunStreamCount=811;pxTotalReqCPU=2.81;pxRunModelCount=270;pxOutputBytes=584,268;pxRunWhenCount=1,904;pxDeclarativePageLoadElapsed=6.84;pxRulesExecuted=3,471;pxOtherCount=314;pxDBInputBytes=3,553,909;pxTotalReqTime=8.09;pxActivityCount=967;pxAlertCount=1;pxOtherFromCacheCount=66;pxInteractions=1;pxLegacyRuleAPIUsedCount=1;pxRuleCount=254;pxInputBytes=101;pxRuleIOElapsed=0.09;pxRulesUsed=4,262;pxDeclarativePageLoadCount=6;pxRuleFromCacheCount=254;pxOtherIOElapsed=0.99;pxTrackedPropertyChangesCount=106;pxOtherIOCount=255;*NA*NA*NA*NA*NA*pyActivity=Pega-UI-CommandPalette.pzGetPaletteOptions;primaryPageClass=Data-Portal-DesignerStudio;*HTTP interaction has exceeded the elapsed time alert threshold of 1000 ms: 8088 ms.*

添加了字符串形式的消息类型，这是_analyze结果:-

{
  "token": "threshold",
  "start_offset": 1276,
  "end_offset": 1285,
  "type": "<ALPHANUM>",
  "position": 128
},
{
  "token": "of",
  "start_offset": 1286,
  "end_offset": 1288,
  "type": "<ALPHANUM>",
  "position": 129
},
{
  "token": "1000",
  "start_offset": 1289,
  "end_offset": 1293,
  "type": "<NUM>",
  "position": 130
},
{
  "token": "ms",
  "start_offset": 1294,
  "end_offset": 1296,
  "type": "<ALPHANUM>",
  "position": 131
},
{
  "token": "8088",
  "start_offset": 1298,
  "end_offset": 1302,
  "type": "<NUM>",
  "position": 132
},
{
  "token": "ms",
  "start_offset": 1303,
  "end_offset": 1305,
  "type": "<ALPHANUM>",
  "position": 133
}

elasticsearch

elastic-stack

kibana-6

回答 1

Stack Overflow用户

发布于 2018-10-23 15:56:09

在版本6.2.4上测试

添加了下面的索引和映射，如下所示

    PUT test
{
  "mappings": {
    "_doc": {
      "properties": {
        "message": {
          "type": "text"
        },
        "message2": {
          "type": "keyword"
        }
      }
    }
  }
}

将2个文档添加到索引中，如下所示

PUT test/_doc/1?refresh
{
  "message": "hellothere",
  "message2":"2018-10-20 23:10:21,068 GMT*8*PEGA0001*8087*1000*8ce767fc2b32*NA*NA*HKVZWM7PHSLMGR3ZXP4OEKEBG3DFFS30K*Test.User*Case-CAS-FS-Work-Svc*Solution:01.03.01*00cb8b6febb234d359369e54a60a865f*Y*3*HKVZWM7PHSLMGR3ZXP4OEKEBG3DFFS30K*35*http-apr-8080-exec-26*STANDARD*com.pega.pegarules.session.internal.engineinterface.service.HttpAPI*192.168.99.100|192.168.99.1*Activity=Pega-UI-CommandPalette.pzGetPaletteOptions*Rule-Obj-Activity:pzGetPaletteOptions*PEGA-UI-COMMANDPALETTE PZGETPALETTEOPTIONS #20161123T194957.445 GMT Step: 2 Circum: 0*NA*****pxRDBIOElapsed=0.03;pxRDBIOCount=4;pxRunStreamCount=811;pxTotalReqCPU=2.81;pxRunModelCount=270;pxOutputBytes=584,268;pxRunWhenCount=1,904;pxDeclarativePageLoadElapsed=6.84;pxRulesExecuted=3,471;pxOtherCount=314;pxDBInputBytes=3,553,909;pxTotalReqTime=8.09;pxActivityCount=967;pxAlertCount=1;pxOtherFromCacheCount=66;pxInteractions=1;pxLegacyRuleAPIUsedCount=1;pxRuleCount=254;pxInputBytes=101;pxRuleIOElapsed=0.09;pxRulesUsed=4,262;pxDeclarativePageLoadCount=6;pxRuleFromCacheCount=254;pxOtherIOElapsed=0.99;pxTrackedPropertyChangesCount=106;pxOtherIOCount=255;*NA*NA*NA*NA*NA*pyActivity=Pega-UI-CommandPalette.pzGetPaletteOptions;primaryPageClass=Data-Portal-DesignerStudio;*HTTP interaction has exceeded the elapsed time alert threshold of 1000 ms: 8088 ms.*" 
} 


PUT test/_doc/2?refresh
{
  "message": "2018-10-20 23:10:21,068 GMT*8*PEGA0001*8087*1000*8ce767fc2b32*NA*NA*HKVZWM7PHSLMGR3ZXP4OEKEBG3DFFS30K*Test.User*Case-CAS-FS-Work-Svc*Solution:01.03.01*00cb8b6febb234d359369e54a60a865f*Y*3*HKVZWM7PHSLMGR3ZXP4OEKEBG3DFFS30K*35*http-apr-8080-exec-26*STANDARD*com.pega.pegarules.session.internal.engineinterface.service.HttpAPI*192.168.99.100|192.168.99.1*Activity=Pega-UI-CommandPalette.pzGetPaletteOptions*Rule-Obj-Activity:pzGetPaletteOptions*PEGA-UI-COMMANDPALETTE PZGETPALETTEOPTIONS #20161123T194957.445 GMT Step: 2 Circum: 0*NA*****pxRDBIOElapsed=0.03;pxRDBIOCount=4;pxRunStreamCount=811;pxTotalReqCPU=2.81;pxRunModelCount=270;pxOutputBytes=584,268;pxRunWhenCount=1,904;pxDeclarativePageLoadElapsed=6.84;pxRulesExecuted=3,471;pxOtherCount=314;pxDBInputBytes=3,553,909;pxTotalReqTime=8.09;pxActivityCount=967;pxAlertCount=1;pxOtherFromCacheCount=66;pxInteractions=1;pxLegacyRuleAPIUsedCount=1;pxRuleCount=254;pxInputBytes=101;pxRuleIOElapsed=0.09;pxRulesUsed=4,262;pxDeclarativePageLoadCount=6;pxRuleFromCacheCount=254;pxOtherIOElapsed=0.99;pxTrackedPropertyChangesCount=106;pxOtherIOCount=255;*NA*NA*NA*NA*NA*pyActivity=Pega-UI-CommandPalette.pzGetPaletteOptions;primaryPageClass=Data-Portal-DesignerStudio;*HTTP interaction has exceeded the elapsed time alert threshold of 1000 ms: 8088 ms.*",
  "message2":"hellothere" 
}

在0个结果中搜索message2: /192.168.99.[0-9]{3}/结果

在doc#2中搜索message: /192.168.99.[0-9]{3}/结果

在doc#1中搜索message2: /.*192.168.99.[0-9]{3}.*/结果

在0个结果中搜索message: /pegarules.session/结果。

但是在doc#1中搜索message: /.*pegarules.session.*/结果，因为倒排索引有"token": "com.pega.pegarules.session.internal.engineinterface.service.httpapi"

在doc#1`中搜索message2: /.*pegarules.session.*/结果

因此，消息字段(类型text)被标记化，正则表达式搜索通配符标记模式将返回结果。

其中，as表示不分析message2字段(类型为keyword)，而是按原样放入倒排索引。除非添加贪婪量词(.*)，否则正则表达式搜索像192.168.99.[0-9]{3}这样的模式不会返回任何结果

Lucene正则表达式引擎与Perl不兼容，但支持的运算符范围较小，因此它可能无法像正则正则表达式那样工作并匹配结果。

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-regexp-query.html#regexp-syntax

票数 2

页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持

原文链接：

https://stackoverflow.com/questions/52908780

复制

相似问题

问使用正则表达式的Kibana不能按预期工作
EN

回答 1

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问使用正则表达式的Kibana不能按预期工作EN

回答 1

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问使用正则表达式的Kibana不能按预期工作
EN