为亚马逊OpenSearch安装Serilog

Question

我正在尝试设置Serilog，以便将日志从ASP.NET Core WebAPI发送到亚马逊OpenSearch的本地实例。我在控制台上看到了日志，但是在OpenSearch中没有显示任何内容。

安装了第三方图书馆：

• Serilog.AspNetCore (6.0.0-dev-00265)
• Serilog.Enrichers.Environment (2.2.1-dev-00787)
• Serilog.Sinks.Elasticsearch (9.0.0-β7)

通过运行OpenSearch (没有安全插件)：

https://opensearch.org/docs/2.0/opensearch/install/docker/#sample-docker-compose-file-for-development

Program.cs

var logger = new LoggerConfiguration()
      .WriteTo.Console()
      .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri("http://localhost:9200"))
      {
          AutoRegisterTemplate = true,
          MinimumLogEventLevel = LogEventLevel.Information,  
          FailureCallback = FailureCallback,
          EmitEventFailure = EmitEventFailureHandling.RaiseCallback | EmitEventFailureHandling.ThrowException
      })
      .CreateLogger();

builder.Logging.ClearProviders();
builder.Logging.AddSerilog(logger);

控制器类：

_logger.LogWarning("Example warning");
_logger.LogError("Example error");

FailureCallback是空的。OpenSearch控制台没有显示任何问题。

可能出什么事了？

Answer 1

我已经尝试过您的设置，下面是一些结果(注意，只使用稳定版本的软件)：

beta)

• Serilog.Sinks.Elasticsearch

• .NET CoreV6.0(非诉8.4.1
• Serilog.AspNetCore 5.0.0

)

码头工人-使用的组合：

version: '3'
services:
  opensearch-node1:
    image: opensearchproject/opensearch:2.0.1
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      - "DISABLE_INSTALL_DEMO_CONFIG=true" # disables execution of install_demo_configuration.sh bundled with security plugin, which installs demo certificates and security configurations to OpenSearch
      - "DISABLE_SECURITY_PLUGIN=true" # disables security plugin entirely in OpenSearch by setting plugins.security.disabled: true in opensearch.yml
      - "discovery.type=single-node" # disables bootstrap checks that are enabled when network.host is set to a non-loopback address
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    volumes:
      - opensearch-data1:/usr/share/opensearch/data
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net

  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:2.0.1
    container_name: opensearch-dashboards
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      - 'OPENSEARCH_HOSTS=["http://opensearch-node1:9200"]'
      - "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" # disables security dashboards plugin in OpenSearch Dashboards
    networks:
      - opensearch-net

volumes:
  opensearch-data1:

networks:
  opensearch-net:

Program.cs

using Serilog;
using Serilog.Events;
using Serilog.Sinks.Elasticsearch;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
builder.Services.AddRazorPages();

builder.Logging.ClearProviders();

Serilog.Debugging.SelfLog.Enable(msg => Console.WriteLine(msg));

ServicePointManager.ServerCertificateValidationCallback =
    (source, certificate, chain, sslPolicyErrors) => true;

var logger = new LoggerConfiguration()
    .WriteTo.Console()
    .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri("http://localhost:9200"))
    {
        AutoRegisterTemplate = true,
        MinimumLogEventLevel = LogEventLevel.Information,
        FailureCallback = e => Console.WriteLine("Unable to submit event " + e.MessageTemplate),
        EmitEventFailure = EmitEventFailureHandling.RaiseCallback | EmitEventFailureHandling.ThrowException,
        TypeName =  "_doc",
        InlineFields = false

    })
    .CreateLogger();

builder.Logging.ClearProviders();
builder.Logging.AddSerilog(logger);

    

var app = builder.Build();

// Configure the HTTP request pipeline.
if (!app.Environment.IsDevelopment())
{
    app.UseExceptionHandler("/Error");
    // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
    app.UseHsts();
}


app.UseHttpsRedirection();
app.UseStaticFiles();

app.UseRouting();

app.UseAuthorization();

app.MapRazorPages();

app.Run();

有几件事应该有助于排除故障。

1. Serilog.Debugging.SelfLog.Enable(msg => Console.WriteLine(msg)); -将显示来自Serilog的真正错误(很可能您将看到SSL)

1. ServicePointManager.ServerCertificateValidationCallback = (source, certificate, chain, sslPolicyErrors) => true; -暂时停止任何SSL问题(稍后可以修复)

1. 我捕捉到的下一件事是由Serilog生成的_type字段的问题，它不被弹性> v8.2所接受，这很可能是因为您的缓冲区保存了旧记录。

虽然最新的测试版Serilog采用了"compatibility.override_main_response_version=true“”_doc“，但adopted 2.0.1还有另一个带有TypeName=设置的bug (请参阅这里的详细信息) https://github.com/opensearch-project/OpenSearch/pull/3530 --基本上我建议将AWS opensearch回滚到v2.。

之后，希望它能奏效:)

Answer 2

对我来说，问题在于我没有为IndexFormat属性(在ElasticSearchSinkOptions对象中)提供值。相反，我把它放在端点中，如果您通过REST插入数据，就应该这样做。总之，下面的代码为我解决了这个问题：

var jsonFormatter = new CompactJsonFormatter();

var loggerConfig = new LoggerConfiguration()
  .Enrich.FromLogContext()
  .WriteTo.Map("Name", "**error**", (name, writeTo) =>
  {
    var currentYear = DateTime.Today.Year;
    var currentWeek = calendar.GetWeekOfYear(DateTime.Now, 
                                CalendarWeekRule.FirstDay, 
                                DayOfWeek.Monday);
    writeTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri("<opensearch endpoint>"))
    {
      CustomFormatter = jsonFormatter,
      TypeName = "_doc",
      IndexFormat = $"my-index-{currentYear}-{currentWeek}",
      MinimumLogEventLevel = LogEventLevel.Information,
      EmitEventFailure = EmitEventFailureHandling.RaiseCallback | 
                          EmitEventFailureHandling.ThrowException,
      FailureCallback = e =>
        Console.WriteLine(
          "An error occured in Serilog ElasticSearch sink: " +
          $"{e.Exception.Message} | {e.Exception.InnerException?.Message}")
    });
  });
Log.Logger = loggerConfig.CreateLogger();

当然，您还需要正确地设置OpenSearch，以便它可以将策略自动应用到索引，等等。