We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When I use a parser, I have nothing insert in my ES index
Exemple based on an "Activities" table with many "Sessions". config :
<source> type postgres_replicator host "#{ENV['POSGRESQL_HOST']}" port "#{ENV['POSGRESQL_PORT']}" username "#{ENV['POSGRESQL_USER']}" password "#{ENV['POSGRESQL_PASSWORD']}" database "#{ENV['POSGRESQL_DB']}" sql SELECT "Activities"."id", "Activities"."name", json_object_agg("Sessions"."date", "Sessions"."status") as rawSessions FROM "Activities" INNER JOIN "Sessions" on "Activities"."id" = "Sessions"."activityId" GROUP BY "Activities"."id" primary_keys id primary_keys id interval 1m tag replicator.pipeline.Activities.${event}.${primary_keys} </source> <filter replicator.pipeline.Activities.**> @type parser format json key_name rawSessions hash_value_field sessions reserve_data true </filter> <filter replicator.pipeline.Activities.**> @type record_transformer remove_keys rawSessions </filter> <match replicator.pipeline.Activities.**> @type elasticsearch user "#{ENV['ES_USER']}" password "#{ENV['ES_PASSWORD']}" host "#{ENV['ES_HOST']}" scheme https index_name myindex type_name activities id_key id time_key updatedAt logstash_format false scheme https port "#{ENV['ES_PORT']}" </match>
This one works fine
<source> type postgres_replicator host "#{ENV['POSGRESQL_HOST']}" port "#{ENV['POSGRESQL_PORT']}" username "#{ENV['POSGRESQL_USER']}" password "#{ENV['POSGRESQL_PASSWORD']}" database "#{ENV['POSGRESQL_DB']}" sql SELECT "Activities"."id", "Activities"."name", json_object_agg("Sessions"."date", "Sessions"."status") as rawSessions FROM "Activities" INNER JOIN "Sessions" on "Activities"."id" = "Sessions"."activityId" GROUP BY "Activities"."id" primary_keys id primary_keys id interval 1m tag replicator.pipeline.Activities.${event}.${primary_keys} </source> <filter replicator.pipeline.Activities.**> @type parser format json key_name rawSessions hash_value_field sessions reserve_data true </filter> <filter replicator.pipeline.Activities.**> @type record_transformer remove_keys rawSessions </filter> <match replicator.pipeline.Activities.**> @type stdout </match>
also this one work.
<source> type postgres_replicator host "#{ENV['POSGRESQL_HOST']}" port "#{ENV['POSGRESQL_PORT']}" username "#{ENV['POSGRESQL_USER']}" password "#{ENV['POSGRESQL_PASSWORD']}" database "#{ENV['POSGRESQL_DB']}" sql SELECT "Activities"."id", "Activities"."name", json_object_agg("Sessions"."date", "Sessions"."status") as rawSessions FROM "Activities" INNER JOIN "Sessions" on "Activities"."id" = "Sessions"."activityId" GROUP BY "Activities"."id" primary_keys id primary_keys id interval 1m tag replicator.pipeline.Activities.${event}.${primary_keys} </source> <filter replicator.pipeline.Activities.**> @type record_transformer remove_keys rawSessions </filter> <match replicator.pipeline.Activities.**> @type elasticsearch user "#{ENV['ES_USER']}" password "#{ENV['ES_PASSWORD']}" host "#{ENV['ES_HOST']}" scheme https index_name myindex type_name activities id_key id time_key updatedAt logstash_format false scheme https port "#{ENV['ES_PORT']}" </match>
The problem is only when I'm using parser with elasticsearch output. No log even in debug that clue the problem.
The text was updated successfully, but these errors were encountered:
No branches or pull requests
When I use a parser, I have nothing insert in my ES index
Exemple based on an "Activities" table with many "Sessions".
config :
This one works fine
also this one work.
The problem is only when I'm using parser with elasticsearch output.
No log even in debug that clue the problem.
The text was updated successfully, but these errors were encountered: