Initial draft of CodeTF v3

[drdavella]

This draft provides a starting point for the discussion about a new, remediation-focused version of CodeTF.

The previous version of CodeTF is oriented around batch changes to files. Our new architecture requires a format that is instead oriented around the remediation of individual findings, each of which may require changes to multiple files.

This version retains much of the metadata structures contained within v2 but reorganized to better reflect the realities of remediation.

[nahsra]

It's so hard for me to reason about a schema, can we make an example that shows all the fields in use? We could use that to validate the schema does what we want, as well.

[nahsra]

Should this have this header?

[nahsra]

I would expect this to be a single fix. I thought we liked the idea of fix orchestration being done upstream. This implies there should be some orchestration here.

This means callers would have to change how they call codemods -- with a single input, but that significantly de-scopes the codemodder implementations.

[nahsra]

I think it's worth pointing out all this findingMetadata should be from the tool itself -- which begs the question -- in the scope of a single issue being fixed, the caller already knows this -- so, do we need to send it back? Don't they already know it?

[drdavella]

Before I push additional changes, let me try to summarize the proposed general direction:

• We want to return to the philosophy of codemods as "small, sharp tools" where each codemod has only a very small scope of responsibility. In this new iteration, that responsibility boils down to fixing a single finding
• Because of this descoping of responsibility, CodeTF needs to represent very little information about the the given finding itself. The caller is responsible for indicating which vulnerability is being addressed, and so holds all of the relevant information already. However, I would like to argue that representing, at minimum, finding ID, tool, and possibly also rule ID, is useful as a record of the transformation that occurred and potentially will be useful for debugging
• As another result of descoping, we expect that codemodder will be asked to fix only one issue at a time, which means we need to represent only one fix result in CodeTF. While I understand the argument and there are many advantages to this approach, I would also like to leave flexibility for batch processing, which means I would like to allow an array of fixes, even if in most cases it contains only a single result
• We still need to communicate several other statuses in addition to successful fixes: failures (could be parsing, LLM errors, etc.), skipped (don't have a codemod/rule for the fix), and "declined" (refuse to fix because deemed not remediable or another reason). Each of these states should be able to convey high-level metadata about the reasoning, even if we no longer need more specific fields like failedFiles
• Responsibility for any kind of fix patch composition lives outside the scope of codemodder itsef