Graylog服务器无法通过TCP :：GELFDispatcher读取Gelf消息-无法处理GELF消息:：无法解压缩GELF消息有效负载

Question

我正在尝试通过tcp将以下json写入graylog服务器：

{"facility":"GELF","file":"","full_message":"Test Message Tcp","host":"FShabesta-12097","level":3,"line":"","short_message":"Test Message Tcp","timestamp":63501046329911952.0,"version":"1.0","_LoggerName":"Test Logger"}

下面是通过网络传输的实际go字节：

1F-8B-08-00-00-00-00-00-04-00-7D-8C-BB-0E-82-30-14-86-5F-85-9C-19-49-5B-04-53-76-70-41-17-D9-49-25-07-68-D2-52-42-2B-89-31-BE-BB-87-60-1C-DD-FE-EB-F7-82-5E-75-DA-E8-F0-84-02-CE-65-5D-41-0C-BD-36-48-6E-53-0F-63-5A-8B-DE-AB-61-4B-1A-F4-21-BA-EC-36-6A-BA-99-16-A3-F3-81-9A-EA-36-AA-3B-B5-EA-C0-05-93-27-2A-0C-AE-68-A0-48-49-E9-E9-8B-F3-A3-5B-C2-7F-5E-D0-76-C3-D8-19-8A-3C-CD-18-67-C7-3C-15-52-72-2E-33-91-B0-18-56-5C-BC-76-13-9D-79-C2-68-DF-D6-6E-18-70-B9-2A-FB-03-EE-09-BC-3F-55-2B-04-1D-DB-00-00-00

而灰度日志服务器抛出以下异常：

2013-04-08 18:37:21,341 WARN : org.graylog2.inputs.gelf.GELFDispatcher - Could not handle GELF message.
java.lang.IllegalStateException: Failed to decompress the GELF message payload
        at org.graylog2.gelf.GELFMessage.getJSON(GELFMessage.java:150)
        at org.graylog2.gelf.GELFProcessor.messageReceived(GELFProcessor.java:62)
        at org.graylog2.inputs.gelf.GELFDispatcher.messageReceived(GELFDispatcher.java:77)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:458)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:439)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.EOFException
        at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:264)
        at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:171)
        at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:78)
        at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:90)
        at org.graylog2.plugin.Tools.decompressGzip(Tools.java:159)
        at org.graylog2.gelf.GELFMessage.getJSON(GELFMessage.java:139)
        ... 15 more
2013-04-08 18:37:21,343 WARN : org.graylog2.inputs.gelf.GELFDispatcher - Could not handle GELF message.
java.lang.IllegalStateException: GELF message is too short. Not even the type header would fit.
        at org.graylog2.gelf.GELFMessage.getGELFType(GELFMessage.java:128)
        at org.graylog2.inputs.gelf.GELFDispatcher.messageReceived(GELFDispatcher.java:67)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:458)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:439)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
2013-04-08 18:37:21,344 WARN : org.graylog2.inputs.gelf.GELFDispatcher - Could not handle GELF message.
java.lang.IllegalStateException: GELF message is too short. Not even the type header would fit.
        at org.graylog2.gelf.GELFMessage.getGELFType(GELFMessage.java:128)
        at org.graylog2.inputs.gelf.GELFDispatcher.messageReceived(GELFDispatcher.java:67)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:458)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:439)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
2013-04-08 18:37:21,344 WARN : org.graylog2.inputs.gelf.GELFDispatcher - Could not handle GELF message.
java.lang.IllegalStateException: GELF message is too short. Not even the type header would fit.
        at org.graylog2.gelf.GELFMessage.getGELFType(GELFMessage.java:128)
        at org.graylog2.inputs.gelf.GELFDispatcher.messageReceived(GELFDispatcher.java:67)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:458)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:439)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
2013-04-08 18:37:21,345 WARN : org.graylog2.inputs.gelf.GELFDispatcher - Could not handle GELF message.
java.lang.IllegalStateException: GELF message is too short. Not even the type header would fit.
        at org.graylog2.gelf.GELFMessage.getGELFType(GELFMessage.java:128)
        at org.graylog2.inputs.gelf.GELFDispatcher.messageReceived(GELFDispatcher.java:67)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:458)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:439)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
2013-04-08 18:37:21,346 WARN : org.graylog2.inputs.gelf.GELFDispatcher - Could not handle GELF message.
java.lang.IllegalStateException: GELF message is too short. Not even the type header would fit.
        at org.graylog2.gelf.GELFMessage.getGELFType(GELFMessage.java:128)
        at org.graylog2.inputs.gelf.GELFDispatcher.messageReceived(GELFDispatcher.java:67)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:458)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:439)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
2013-04-08 18:37:21,348 WARN : org.graylog2.inputs.gelf.GELFDispatcher - Could not handle GELF message.
java.lang.IllegalStateException: JSON is null/could not be parsed (invalid JSON)
        at org.graylog2.gelf.GELFProcessor.parse(GELFProcessor.java:89)
        at org.graylog2.gelf.GELFProcessor.messageReceived(GELFProcessor.java:62)
        at org.graylog2.inputs.gelf.GELFDispatcher.messageReceived(GELFDispatcher.java:77)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:458)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:439)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
2013-04-08 18:37:21,349 WARN : org.graylog2.inputs.gelf.GELFDispatcher - Could not handle GELF message.
java.lang.IllegalStateException: GELF message is too short. Not even the type header would fit.
        at org.graylog2.gelf.GELFMessage.getGELFType(GELFMessage.java:128)
        at org.graylog2.inputs.gelf.GELFDispatcher.messageReceived(GELFDispatcher.java:67)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:458)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:439)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
2013-04-08 18:37:21,350 WARN : org.graylog2.inputs.gelf.GELFDispatcher - Could not handle GELF message.
java.lang.IllegalStateException: GELF message is too short. Not even the type header would fit.
        at org.graylog2.gelf.GELFMessage.getGELFType(GELFMessage.java:128)
        at org.graylog2.inputs.gelf.GELFDispatcher.messageReceived(GELFDispatcher.java:67)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:458)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:439)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)

写入UDP端口时会通过完全相同的消息。

负责在网络上写入字节的C#代码片段：

public void Send(byte[] bytes, int length, IPEndPoint ipEndPoint)
{
    using (var tcpClient = new TcpClient(ipEndPoint.Address.ToString(), ipEndPoint.Port))
    {
        var stream = tcpClient.GetStream();
        stream.Write(bytes, 0, length);
        stream.Close();
    }
}

任何提示/建议都非常感谢。

Graylog jira linkGithub issue link。

Answer 1

enter link description here

“目前，GELF TCP仅支持未压缩和非分块的有效负载。在同一TCP连接中发送时，每个消息需要用空字节(\0)分隔。

(!)注意

由于使用空字节(\0)作为帧分隔符，GELF TCP不支持压缩。“

这看起来在GELF上仍然不支持通过TCP进行压缩