Oak Attestation Integration Tests

This crate provides comprehensive integration tests and utilities for Oak's attestation system. It serves three main purposes:

1. Attestation Output Analysis:

   • A binary utility that analyzes attestation output for breaking changes.
   • It saves new snapshots when the output changes.
   • Snapshots are ordered starting at 00000, progressing to 00001 and so on.
   • This utility must be run and its output committed whenever changes are made that alter attestation output.

2. Verification Backwards Compatibility Checks:

   • Tests the current verification library against snapshots of past attestation outputs, ensuring that verification succeeds.

3. Attestation Integration Testing:

   • Tests covering the full workflow from generating an attestation to verifying it.

CI Failure Guide

If you encounter a CI failure related to this crate, it likely means one of four things (from most to least likely):

1. Attestation Output Snapshot Update Required:

   • You've made changes that alter the attestation output, but haven't created a snapshot.
   • Action: Run update_testdata_assert_no_breaking_changes locally, commit the new snapshot, and push your changes.

2. Breaking Changes in Attestation Output Detected:

   • The attestation output has changed in a way that could break verification for older versions.
   • Action: Reimplement the change in a non breaking way. Changes should be additive. Implement them by adding new fields, not by altering or deleting current ones. Consult with the team if assistance needed.

3. Breaking Changes in Verification Library Detected:

   • The verification library has been modified in a way that older evidence in snapshot no longer verifies.
   • Action: This is a serious issue that requires careful consideration. Review the breaking changes to ensure they are necessary. Usually it's possible to implement changes in a way that maintains compatability with verifying old version of evidence. If that is not possible, consult with the team to determine the best course of action.

4. Current Attestation-Verification Pairing Failure:

   • The integration tests pairing the current attestation logic with the current verification logic are failing.
   • Action: This indicates a mismatch between the current attestation output and what the verification process expects. Debug the changes in both the attestation and verification components to identify the source of the discrepancy.

Future Work and Considerations

1. Snapshot Versioning:

   • Currently, snapshots are incrementally numbered (000000, 000001, etc.).
   • This approach simplifies identifying the latest snapshot but may lead to merge conflicts if multiple changes occur simultaneously.
   • Future consideration: Implement a more robust versioning system that reduces the likelihood of conflicts.

2. Snapshot Retention Policy:

   • Snapshots should realistically represent the distribution of evidence "in the wild", potentially with a conservative approach.
   • We should establish a policy for deleting old snapshots based on the phasing out of the logic that generated them.
   • Deletion criteria:
     • 100% certainty that the logic generating a snapshot has been phased out.
     • Or, ~99% phased out with acceptance of breaking the remaining 1% of clients.
   • This approach ensures that our test suite remains relevant to real-world scenarios while allowing for careful pruning of outdated evidence.
   • TODO: b/372045744 - Develop a mechanism to track the lifecycle of evidence-generating logic and implement a process for safe snapshot deletion.

3. Breaking Changes to Attestation Outputs:

   • In the future, we may need to allow breaking changes (e.g., removing fields) to attestation outputs.
   • This would be based on the assumption that no clients in production are still using old versions of the verification library that read these fields.
   • Challenges:
     • Our current testing doesn't verify against old client versions.
     • We only check that old fields remain set, without knowledge of their continued relevance.
   • Future work:
     • Develop a strategy for safely implementing breaking changes.
     • Consider implementing a deprecation period for fields before removal.
     • Explore ways to test compatibility with older client versions or maintain a registry of field usage.

Crate Organization

This crate is separate from individual attestation/verification components due to its dependencies. It relies on a mix of crates, some of which are not available in google3.

For tests specific to individual aspects of attestation/verification that don't have external dependencies, place them in the relevant component crate. This ensures they can run in google3 if the crate is imported.