maxNumberOfFailuresToShow doesn't seem to work as expected.

[samixchoumi]

Bug description

Looking at the example from the doc here : maxNumberOfFailuresToShow, it doesn't seem to work as expected when I set maxNumberOfFailuresToShow to 0 in my cli_config.json file like that : "maxNumberOfFailuresToShow": 0,

How to reproduce (please include a snippet of your Playwright Slack Reporter configuration)

1. Set the config maxNumberOfFailuresToShow to '0'
2. Run your test with some failing tests cases.
3. See some failure reported in your Slack.

Not sure if it's useful, but I use :

"sendUsingWebhook": {
    "webhookUrl": "my slack webhook URL"
}

Expected behavior

No failure shown in the Slack report.

Environment & setup

• OS: macOS
• Node.js version: v21.1.0

playwright-slack-report version

"playwright-slack-report": "^1.1.72"

[ryanrosello-og]

yeah this is a legit bug. Seems this bit of the code https://github.com/ryanrosello-og/playwright-slack-report/blob/main/src/SlackReporter.ts#L77 is treating zero as falsy so its always falling back to showing 10 failures even though you specified 0 as its value.

should be straight-forward fix:

this.maxNumberOfFailuresToShow
  = slackReporterConfig.maxNumberOfFailuresToShow !== undefined ? slackReporterConfig.maxNumberOfFailuresToShow : 10;

leave it with me. I can take a look

[samixchoumi]

Okey ! Waiting for the fix then ! Thanks 🙏🏻

[samixchoumi]

Hello, @ryanrosello-og FYI, I tested by replacing the value to 1, I still have multiple error shown in the slack reporting and still have the message : There are too many failures to display - 5 out of 28 failures shown

My cli_config.json looks like that :

{
  "sendResults": "always",
  "slackLogLevel": "error",
  "sendUsingWebhook": {
    "webhookUrl": "$SLACK_WEBHOOK_URL"
  },
  "meta": [
    { "key": "branch", "value" : "$CI_COMMIT_BRANCH"},
    { "key": "commit", "value" : "$CI_COMMIT_SHA"}
  ],
  "maxNumberOfFailuresToShow": 1,
  "disableUnfurl": true
}

[samixchoumi]

@ryanrosello-og Hello, I updated to the latest version : "playwright-slack-report": "^1.1.80",
I still have multiple errors appearing when setting "maxNumberOfFailuresToShow": 0,

See my cli_config.json file :

{
  "sendResults": "always",
  "slackLogLevel": "error",
  "sendUsingWebhook": {
    "webhookUrl": "$SLACK_WEBHOOK_URL"
  },
  "meta": [
    { "key": "Pipeline", "value" : "$CI_PIPELINE_URL"}
  ],
  "maxNumberOfFailuresToShow": 0,
  "disableUnfurl": true
}

Here is a resume of the slack report :

:performing_arts: Playwright Results
:white_check_mark: 24 | :x: 59 | :large_yellow_circle: 0 | :fast_forward: 0
Pipeline :    $CI_PIPELINE_URL

 72 |     // Set currency
> 74 |     await this.page.getByText(currency).click();
    |                                         ^
TimeoutError: locator.click: Timeout 10000ms exceeded.
Call log:
 - waiting for getByText('USD ($)')

:warning: There are too many failures to display - 5 out of 59 failures shown

[ryanrosello-og]

@samixchoumi it turns out your config is using the wrong key, use "maxNumberOfFailures": 0,
instead of "maxNumberOfFailuresToShow": 0,

When you provided maxNumberOfFailuresToShow, it was causing it to fallback to the default value (5) see ....

https://github.com/ryanrosello-og/playwright-slack-report/blob/main/src/cli/cli_schema.ts#L30

[samixchoumi]

Oh, I though I needed to use this parameter : https://github.com/ryanrosello-og/playwright-slack-report?tab=readme-ov-file#maxnumberoffailurestoshow
I update my config file to see if it's fixed then. Thanks

[samixchoumi]

It works !
Thanks (maybe updating the documentation to avoid conflict between both parameter ?)

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.