Please add log_400_event or log_level (debug) options

[AlKapkone]

Problem

2022-04-01 11:51:33 +0000 [warn]: #0 send an error event to @ERROR: error_class=Fluent::Plugin::ElasticsearchErrorHandler::ElasticsearchError error="400 - Rejected by Elasticsearch" location=nil tag="app.fpm" time=2022-04-01 11:51:26.872353958 +0000

I want to know the reason why, so I need the log_level debug or at least the log_es_400_reason parameter.
Like we have it here - https://banzaicloud.com/docs/one-eye/logging-operator/configuration/plugins/outputs/elasticsearch/#elasticsearch-log_es_400_reason

Using Fluentd and ES plugin versions

    repository: ghcr.io/banzaicloud/fluentd
    tag: v1.14.4-alpine-2

• OS version - "Alpine Linux v3.14"
• Fluentd 1.14.4
• AWS ES Service plugin version fluent-plugin-aws-elasticsearch-service (2.4.1)
• ES version - AWS OpenSearch 1.1(latest)

[AlKapkone]

@cosmo0920 Hello, sorry to bother you, but do we have a way how to troubleshoot my issue? I just want to know why AWS OpenSearch reject my logs. Thanks.

[ServerNinja]

We are running into an issue where this option would be very helpful.

[cosmo0920]

Already inherited: https://github.com/atomita/fluent-plugin-aws-elasticsearch-service/blob/master/lib/fluent/plugin/out_aws-elasticsearch-service.rb#L10

[AlKapkone]

Already inherited: https://github.com/atomita/fluent-plugin-aws-elasticsearch-service/blob/master/lib/fluent/plugin/out_aws-elasticsearch-service.rb#L10

Hi, @cosmo0920 . Actually, I still didn't know how to enable debug and use this log_es_400_reason parameter.
Could you help me with that?
My config is next now:

      awsElasticsearch:
        endpoint:
          url: https://logs.example.com
          region: us-east-1
          assume_role_arn:
            value: arn:aws:iam::12345678:role/logs-full-access
        index_name: app-${kube_app_name}-%Y.%m.%d
        buffer:
          tags: tag, time, kube_app_name
          timekey: 3s
          timekey_wait: 5s
          timekey_use_utc: true
          flush_thread_count: 8

[AlKapkone]

I'll try to clarify and add more context: I use banzai-cloud logging-operator - https://banzaicloud.com/docs/one-eye/logging-operator/configuration/plugins/outputs/aws_elasticsearch/
I use the fluent-plugin-aws-elasticsearch-service plugin as an output for my logs.

[AlKapkone]

So I suppose this log_es_400_reason option should be added from the banzai logging-operator side?
Am I right ?

[cosmo0920]

Yes. Adding log_es_400_reason parameter support is required here:
https://github.com/banzaicloud/logging-operator/blob/c9722ee319584ff75ebda49ef4095b6bf127d7d4/pkg/sdk/model/output/aws_elasticsearch.go
Like as this PR: kube-logging/logging-operator#858

[cosmo0920]

And this issue is not related this repo but, for tracking issue, we leave this issue as-is for now.