Skip to content

Commit b4d9556

Browse files
[8.15](backport #41640) libbeat: increase total_fields.limit to 12500 (#41672)
It increased the `index.mapping.total_fields.limit` from `10000` to `12500` in order to avoid ingestion failures caused by too many field in the index. Since 8.15.0 the limit started to be hit. The field count being exceeded is on the index, counting all mapped fields and the dynamic fields. That's why a small event might trigger the error, the event contains new fields to be mapped which would exceed the total field limit if mapped. (cherry picked from commit 42dd93b) --------- Co-authored-by: Anderson Queiroz <[email protected]>
1 parent d273d28 commit b4d9556

File tree

3 files changed

+3
-2
lines changed

3 files changed

+3
-2
lines changed

CHANGELOG.next.asciidoc

+1
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
1010

1111
*Affecting all Beats*
1212

13+
- Fix metrics not being ingested, due to "Limit of total fields [10000] has been exceeded while adding new fields [...]". The total fields limit has been increased to 12500. No significant performance impact on Elasticsearch is anticipated. {pull}41640[41640]
1314

1415
*Auditbeat*
1516

libbeat/template/load_test.go

+1-1
Original file line numberDiff line numberDiff line change
@@ -170,7 +170,7 @@ func TestFileLoader_Load(t *testing.T) {
170170
"refresh_interval": "5s",
171171
"mapping": mapstr.M{
172172
"total_fields": mapstr.M{
173-
"limit": 10000,
173+
"limit": defaultTotalFieldsLimit,
174174
},
175175
},
176176
"query": mapstr.M{

libbeat/template/template.go

+1-1
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ import (
3535
var (
3636
// Defaults used in the template
3737
defaultDateDetection = false
38-
defaultTotalFieldsLimit = 10000
38+
defaultTotalFieldsLimit = 12500
3939
defaultMaxDocvalueFieldsSearch = 200
4040

4141
defaultFields []string

0 commit comments

Comments
 (0)