Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Extra file JSON object passed via CLI --extra-files parameter is improperly parsed #2519

Open
witalloliveira opened this issue Mar 28, 2025 · 0 comments
Assignees
Labels
priority: p3 Desirable enhancement or fix. May not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@witalloliveira
Copy link

Environment details

  • OS: macOS 15.3 (Darwin 24.3.0)
  • Node.js version: v22.13.0
  • npm version: v10.9.2
  • release-please version: 17.0.0

Issue:

When passing an object via the --extra-files parameter (for example, an object with "type": "yaml", "path": "scripts/helm/dev-ext/sa-east-1/values.yaml", "jsonpath": "$.rollout-Canary.image.tag" and "glob": false), Release Please appears to parse each property separately as a file path. Instead of treating the JSON object as a single extra file configuration, its keys and values are fetched individually from the target branch.

Reproduction:

  1. Run the following command (using zsh):
release-please release-pr \
  --repo-url="https://github.com/myOrg/myRepo" \
  --token "****************************************" \
  --release-type=simple \
  --pull-request-header="My custom pull request header" \
  --extra-files='[{"type":"yaml","path":"scripts/helm/dev-ext/sa-east-1/values.yaml","jsonpath":"$.rollout-Canary.image.tag","glob":false}]'
  1. Observe in the logs that Release Please tries to fetch:
    • {type:"yaml"
    • path:"scripts/helm/dev-ext/sa-east-1/values.yaml"
    • jsonpath:"$.rollout-Canary.image.tag"
    • glob:false}
      as separate files from branch main.
❯ type: simple
❯ targetBranch: main
❯ commits: 2
✔ Considering: 2 commits
❯ component: 
❯ pull request title pattern: undefined
❯ componentNoSpace: false
✔ Looking for open release pull requests
✔ found 0 open release pull requests.
✔ Looking for snoozed release pull requests
✔ found 0 snoozed release pull requests.
❯ Fetching CHANGELOG.md from branch main
❯ Fetching version.txt from branch main
⚠ file version.txt did not exist
❯ Fetching {type:"yaml" from branch main
⚠ file {type:"yaml" did not exist
❯ Fetching path:"scripts/helm/dev-ext/sa-east-1/values.yaml" from branch main
⚠ file path:"scripts/helm/dev-ext/sa-east-1/values.yaml" did not exist
❯ Fetching jsonpath:"$.rollout-Canary.image.tag" from branch main
⚠ file jsonpath:"$.rollout-Canary.image.tag" did not exist
❯ Fetching glob:false} from branch main
⚠ file glob:false} did not exist
❯ Fetching .release-please-manifest.json from branch main
⚠ file .release-please-manifest.json did not exist
✔ Starting GitHub PR workflow...

Expected Behavior:

The JSON array containing the extra file object should be parsed correctly. The CLI should use the "type" field to determine the updater (e.g., GenericYaml for "yaml" type) and apply the "jsonpath" to update the file content, rather than treating the object properties as independent file paths.

The CLI should pick up the --extra-files parameter as a valid JSON array containing a single object. That object should then be processed by the extraFilePaths and extraFileUpdates methods (see attached snippet from base.ts), determining the file paths and updater type without fetching individual object properties from the branch.

Context:

Looking at the source code in base.ts, the method extraFilePaths(extraFile: ExtraFile) expects an object but then processes it in a way that suggests the entire object should represent a single extra file. The behavior above indicates that the passed JSON isn’t being recognized as an object with keys but rather as separate string inputs.

Requirements

I'm using the CLI with the --extra-files parameter instead of a configuration file or manifest because in our CI pipeline we don't expect to have both files in the repository. We need this behavior to override options locally. I have also tested using a configuration file, but every time the CLI still attempts to fetch the file remotely from the main branch. I'm open to learning if there's a way to override some config or manifest options using local files, but currently, my goal is to pass the extra file data directly via the CLI.

Additional Context

Below is a brief excerpt from base.ts where the extra files are processed:

// Helper to convert extra files with globs to the file paths to add
private async extraFilePaths(extraFile: ExtraFile): Promise<string[]> {
  if (typeof extraFile !== 'object') {
    return [extraFile];
  }
  if (!extraFile.glob) {
    return [extraFile.path];
  }
  if (extraFile.path.startsWith('/')) {
    return (
      await this.github.findFilesByGlobAndRef(
        extraFile.path.slice(1),
        this.targetBranch
      )
    ).map(file => `/${file}`);
  } else if (this.path === ROOT_PROJECT_PATH) {
    return this.github.findFilesByGlobAndRef(
      extraFile.path,
      this.targetBranch
    );
  } else {
    return this.github.findFilesByGlobAndRef(
      extraFile.path,
      this.targetBranch,
      this.path
    );
  }
}

Also this following code when the GenericYaml CompositeUpdate is called:

  protected async extraFileUpdates(
    version: Version,
    versionsMap: VersionsMap,
    dateFormat: string
  ): Promise<Update[]> {
    const extraFileUpdates: Update[] = [];
    for (const extraFile of this.extraFiles) {
      if (typeof extraFile === 'object') {
        const paths = await this.extraFilePaths(extraFile);
        for (const path of paths) {
          switch (extraFile.type) {
            case 'generic':
              extraFileUpdates.push({
                path: this.addPath(path),
                createIfMissing: false,
                updater: new Generic({
                  version,
                  versionsMap,
                  dateFormat: dateFormat,
                }),
              });
              break;
            case 'json':
              extraFileUpdates.push({
                path: this.addPath(path),
                createIfMissing: false,
                updater: new GenericJson(extraFile.jsonpath, version),
              });
              break;
            case 'yaml':
              extraFileUpdates.push({
                path: this.addPath(path),
                createIfMissing: false,
                updater: new GenericYaml(extraFile.jsonpath, version),
              });
              break;
            
           [...]

            default:
              throw new Error(
                `unsupported extraFile type: ${
                  (extraFile as {type: string}).type
                }`
              );
          }
        }
      } else if (extraFile.endsWith('.json')) {
        extraFileUpdates.push({
          path: this.addPath(extraFile),
          createIfMissing: false,
          updater: new CompositeUpdater(
            new GenericJson('$.version', version),
            new Generic({version, versionsMap, dateFormat: dateFormat})
          ),
        });
      } else if (extraFile.endsWith('.yaml') || extraFile.endsWith('.yml')) {
        extraFileUpdates.push({
          path: this.addPath(extraFile),
          createIfMissing: false,
          updater: new CompositeUpdater(
            new GenericYaml('$.version', version),
            new Generic({version, versionsMap, dateFormat: dateFormat})
          ),
        });

      [...]

      } else {
        extraFileUpdates.push({
          path: this.addPath(extraFile),
          createIfMissing: false,
          updater: new Generic({version, versionsMap, dateFormat: dateFormat}),
        });
      }
    }
    return extraFileUpdates;
  }

This issue suggests that the CLI does not recognize the JSON object as a whole, and each property ends up being treated as a separate file.

Any help or guidance to resolve this behavior would be appreciated.

@witalloliveira witalloliveira added priority: p3 Desirable enhancement or fix. May not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Mar 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p3 Desirable enhancement or fix. May not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

2 participants