Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZIR-353: Keccak circuit is fully deterministic #205

Merged
merged 21 commits into from
Mar 17, 2025
Merged

Conversation

jacobdweightman
Copy link
Contributor

@jacobdweightman jacobdweightman commented Mar 14, 2025

This PR contains a lot of changes to complete the Picus proof of deteterminism for the Keccak circuit.

  • Add "early optimizations" to reduce size of generated code. These ultimately don't affect the validity polynomial of the circuit, but make a large difference in --emit=zhltopt code size.
    • Elide redundant struct members (new)
    • Mux common code hoisting (new)
    • Field DCE (pre-existing, added earlier in pipeline)
  • Add Picus-specific optimizations to emitter
    • Inline constants
    • Inline expressions used once
    • AliasLayoutOps used "across" mux arms are translated into linear combinations instead of arm-local assertions
  • Add PicusInput directive to declare actual circuit inputs (which otherwise look nondeterministic)
  • Add PicusHintEq directive to help Picus reason about "non-local" constraints
  • Add AssertRange and AssumeRange directives
  • Fix handling of @layout members in Picus translation
  • Update Picus model of globals: these are always deterministic, since the verifier is given a unique, concrete assignment for them
  • Give externs the picus_inline attribute by default
  • Eliminated some unused signals
  • Added CI job that checks the Keccak circuit is deterministic

@github-actions github-actions bot changed the title Keccak circuit is fully deterministic ZIR-353: Keccak circuit is fully deterministic Mar 14, 2025
@jacobdweightman jacobdweightman force-pushed the jacob/picus-v2 branch 9 times, most recently from 2114b03 to 9447bf2 Compare March 14, 2025 16:52
@flaub
Copy link
Member

flaub commented Mar 14, 2025

Do these changes cause the KECCAK_CONTROL_ROOT to change?

@jacobdweightman
Copy link
Contributor Author

I just reran the zirgen repo's cargo bootstrap keccak and the risc0 repo's cargo xtask bootstrap, and I only see source location comment and member name changes. I pushed these changes to a draft PR in case you would like to review, @flaub

AttrTypeReplacer replacer;
replacer.addReplacement([&](StructType t) { return typeReplacer(replacer, t); });
replacer.addReplacement([&](LayoutType t) { return typeReplacer(replacer, t); });
// replacer.addReplacement(attrReplacer); // TODO: I should optimize the attributes too!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this TODO left intentionally?

SmallVector<std::pair<size_t, size_t>> merges;
for (size_t i = 0; i < members.size(); i++) {
for (size_t j = i + 1; j < members.size(); j++) {
if (members[i] == members[j]) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This nested loop makes me nervous, and I wonder if you've considered using a multimap instead. Or can we be sure that N is always small enough that it won't matter?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pass looks at the return value of a constructor, and if it is a PackOp then it searches for members that are the same. This is O(number_of_members^2), but in practice our components tend not to have very many members. I'm not entirely sure how a multimap helps with this, could you say a bit more?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking you'd iterate over the members once to build a map from members to indexes, then iterate over the map to deduplicate - but if the number of members is small, it doesn't matter.

// operations must also be in the first mux arm.
auto it = mux.getRegion(0).op_begin();
while (it != mux->getRegion(0).op_end()) {
Operation& op = *(it++);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks to my eyes like a decomposed for loop, and I wonder whether iterating over Region::getOps wouldn't be simpler? Perhaps there's a good reason it's done this way, but a comment explaining the unusual structure might be nice.

Copy link
Contributor Author

@jacobdweightman jacobdweightman Mar 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because hoisting moves operations out of arm 0, this iteration scheme avoids losing track of which operation in the region we're looking at, so using a for loop doesn't work. I've updated the comment to reflect this, I realize this was not as clear as it could be, so thanks for the feedback!

@@ -224,6 +224,14 @@ auto reduce(std::array<T1, N> elems, T2 start, const BoundLayout<T3>& b, F f) {
// All the extern handling
#define INVOKE_EXTERN(ctx, name, ...) extern_##name(ctx, ##__VA_ARGS__)

// TODO: release mode that removes assertions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this todo still relevant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately. I'm planning to come back to that in a followup.

@flaub
Copy link
Member

flaub commented Mar 17, 2025

I also ran bootstrap locally to see if this would break anything, and since it's just source location changes in comments, I approved.

@jacobdweightman
Copy link
Contributor Author

Thanks for the feedback, @mars-risc0 and @flaub! I'm in a bit of a hurry to get this merged, so I'm going to follow up on a flag for stripping out asserts and the multimap suggestion.

@jacobdweightman jacobdweightman merged commit e85a176 into main Mar 17, 2025
12 checks passed
@jacobdweightman jacobdweightman deleted the jacob/picus-v2 branch March 17, 2025 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants