Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can i suppress type name for OpenSearchservice v2.3 #84

Open
hanedatatsufumi opened this issue Mar 10, 2023 · 2 comments
Open

Can i suppress type name for OpenSearchservice v2.3 #84

hanedatatsufumi opened this issue Mar 10, 2023 · 2 comments

Comments

@hanedatatsufumi
Copy link

Problem

The _type field has been removed from Amazon OpenSearchService since v2.0.

https://opensearch.org/docs/latest/breaking-changes/

For example, in the fluent-plugin-elasticsearch plugin, the suppress_type_name setting can be used to prevent the _type field from being sent to OpenSearchService, but is there a similar setting planned to be released for the fluent-plugin-aws-elasticsearch-service plugin?

https://github.com/uken/fluent-plugin-elasticsearch/#suppress_type_name

Alternatively, could you please provide a workaround?

Note that even if type_name "" is set, the _type field is still sent to OpenSearchService and the sent logs are rejected.

...

Steps to replicate

  <match os.**>
    @type aws-elasticsearch-service
    include_tag_key true
    type_name ""
    target_index_key "@log_name"
    tag_key "@log_name"
    logstash_format true
    logstash_prefix "projecta"
    logstash_dateformat "%Y.%m"
    <buffer tag>
      @type "memory"
      flush_thread_count 4
    </buffer>
    <endpoint>
      url "https://search-xxxxxxxxxx.ap-northeast-1.es.amazonaws.com"
      region "ap-northeast-1"
      access_key_id "xxxxxx"
      secret_access_key xxxxxx
    </endpoint>
  </match>

Expected Behavior or What you need to ask

...

Using Fluentd and ES plugin versions

  • OS version
    4.14.301-224.520.amzn2.x86_64
  • Fluentd
    td-agent 1.11.1
  • AWS ES Service plugin version
    fluent-plugin-aws-elasticsearch-service (2.4.0)
  • IAM policies for AWS ES Service (optional)
  • ES version (optional)
    OpenSearchservice v2.3
@agonzalezfer
Copy link

agonzalezfer commented Apr 17, 2023

I have the same problem in this moment... Any solution? I think to use to fluent-plugin-opensearch...

@superkashyap
Copy link

@hanedatatsufumi any fix for it ? facing the same problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants