Add fix quality metadata to ChangeSet spec

[drdavella]

No description provided.

[drdavella]

After further discussion we might be looking for three separate ratings:

1. Safety (can I take this change without breaking my code)
2. Effectiveness (does it fix the problem without introducing syntactic or semantic changes)
3. Cleanliness (does this code introduce stylistic issues, add/remove comments, etc.)

It's worth noting that these are not strictly independent categories so we need to be careful when scoring.

From a user perspective, I might argue that they really only care about two dimensions:

1. Is the fix correct?
2. Will it break my app?

However, those two could ultimately be derived from the first three criteria.

[nahsra]

From a user perspective, I might argue that they really only care about two dimensions:
Is the fix correct?
Will it break my app?

I'd argue these are just different ways of "safe" and "effective".

[nahsra]

Are the 3 things implementation details of the 1 thing? I'm developing more conviction about:

• Safe
• Effective
• Palatability

I think "safe" encapsulates a whole set of concerns about off-target effects on the code compilation and code functioning.

I think "effective" encapsulates everything about the fix effectiveness, completeness, durability, etc.

I think "palatability" is about all the stylistic / formatting / bikeshedding / team norm kind of stuff that we will, frankly, only have limited insight into. If we're going to be opinionated about how implementors should feel about this change by exposing these, we must come to agreement about the factors now, and I'm happy to do that.

[nahsra]

Shouldn't this also be at the "change" level, rather than the changeset? I might be misunderstanding, but each individual change will have its own score, right? Or are we asking CodeTF providers to estimate a "global score"?

[drdavella]

Shouldn't this also be at the "change" level, rather than the changeset? I might be misunderstanding, but each individual change will have its own score, right? Or are we asking CodeTF providers to estimate a "global score"?

My assumption so far has been that this evaluation applies per fix and we have effectively been assuming a 1:1 mapping between fixed finding and changeset entry.