AbstractInterpreter: Add a hook to opt into more extensive stmt info #39716
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I'm in the process of updating Cthulhu to make use of an AbstractInterpreter instead of using raw inference queries. Over time Cthulhu has grown many heuristics for trying to reverse engineer what inference did in order to present this to the user. Of course, these heuristics are incomplete and worse, we do already mostly have this information neatly packaged in the stmt_info field, since optimization needs the same information. Switching Cthulhu to an AbstractInterpreter gives it access to this info (as well as allowing it to maintain its own caches, fixing various weirdness around statement order dependence with the base inference caches). However, there are a few places in inference where we drop stmt info that Cthulhu would like to read because the optimizer doesn't need it. This adds a hook for Cthulhu (and other tooling that would like to receive it) to get more verbose stmt info out of inference. My end goal here is to turn Cthulhu entirely into a viewer for inference data structures, and not have it contain any heuristics that duplicate inference functionality. I think we're pretty close to that, but a few tweaks are required. Hopefully the end result should be a much more functional and precise Cthulhu (as well as being easier to maintain, since we get to delete half the code).
Keno
requested review from
vchuravy,
vtjnash,
JeffBezanson,
timholy and
aviatesk
Keno
added a commit
to JuliaDebug/Cthulhu.jl
that referenced
this pull request
Feb 17, 2021
Rips out all the heuristics that are duplicating the analysis that inference already does to figure out what is getting called. This should be much more precise and general. We may want to revisit the whole CallInfo setup in the future to maybe use the stmt info more directly, but this should be good for the moment. Depends on JuliaLang/julia#39716 (so WIP until that is merged).
Keno
added a commit
to JuliaDebug/Cthulhu.jl
that referenced
this pull request
Feb 17, 2021
Rips out all the heuristics that are duplicating the analysis that inference already does to figure out what is getting called. This should be much more precise and general. We may want to revisit the whole CallInfo setup in the future to maybe use the stmt info more directly, but this should be good for the moment. Depends on JuliaLang/julia#39716 (so WIP until that is merged).
vtjnash
approved these changes
Feb 19, 2021
aviatesk
added a commit
to aviatesk/JET.jl
that referenced
this pull request
Feb 20, 2021
Keno
added a commit
to JuliaDebug/Cthulhu.jl
that referenced
this pull request
Mar 3, 2021
Rips out all the heuristics that are duplicating the analysis that inference already does to figure out what is getting called. This should be much more precise and general. We may want to revisit the whole CallInfo setup in the future to maybe use the stmt info more directly, but this should be good for the moment. Depends on JuliaLang/julia#39716 (so WIP until that is merged).
ElOceanografo
pushed a commit
to ElOceanografo/julia
that referenced
this pull request
May 4, 2021
…uliaLang#39716) I'm in the process of updating Cthulhu to make use of an AbstractInterpreter instead of using raw inference queries. Over time Cthulhu has grown many heuristics for trying to reverse engineer what inference did in order to present this to the user. Of course, these heuristics are incomplete and worse, we do already mostly have this information neatly packaged in the stmt_info field, since optimization needs the same information. Switching Cthulhu to an AbstractInterpreter gives it access to this info (as well as allowing it to maintain its own caches, fixing various weirdness around statement order dependence with the base inference caches). However, there are a few places in inference where we drop stmt info that Cthulhu would like to read because the optimizer doesn't need it. This adds a hook for Cthulhu (and other tooling that would like to receive it) to get more verbose stmt info out of inference. My end goal here is to turn Cthulhu entirely into a viewer for inference data structures, and not have it contain any heuristics that duplicate inference functionality. I think we're pretty close to that, but a few tweaks are required. Hopefully the end result should be a much more functional and precise Cthulhu (as well as being easier to maintain, since we get to delete half the code).
antoine-levitt
pushed a commit
to antoine-levitt/julia
that referenced
this pull request
May 9, 2021
…uliaLang#39716) I'm in the process of updating Cthulhu to make use of an AbstractInterpreter instead of using raw inference queries. Over time Cthulhu has grown many heuristics for trying to reverse engineer what inference did in order to present this to the user. Of course, these heuristics are incomplete and worse, we do already mostly have this information neatly packaged in the stmt_info field, since optimization needs the same information. Switching Cthulhu to an AbstractInterpreter gives it access to this info (as well as allowing it to maintain its own caches, fixing various weirdness around statement order dependence with the base inference caches). However, there are a few places in inference where we drop stmt info that Cthulhu would like to read because the optimizer doesn't need it. This adds a hook for Cthulhu (and other tooling that would like to receive it) to get more verbose stmt info out of inference. My end goal here is to turn Cthulhu entirely into a viewer for inference data structures, and not have it contain any heuristics that duplicate inference functionality. I think we're pretty close to that, but a few tweaks are required. Hopefully the end result should be a much more functional and precise Cthulhu (as well as being easier to maintain, since we get to delete half the code).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'm in the process of updating Cthulhu to make use of an AbstractInterpreter
instead of using raw inference queries. Over time Cthulhu has grown many
heuristics for trying to reverse engineer what inference did in order to
present this to the user. Of course, these heuristics are incomplete and
worse, we do already mostly have this information neatly packaged in
the stmt_info field, since optimization needs the same information.
Switching Cthulhu to an AbstractInterpreter gives it access to this info
(as well as allowing it to maintain its own caches, fixing various
weirdness around statement order dependence with the base inference
caches). However, there are a few places in inference where we drop
stmt info that Cthulhu would like to read because the optimizer doesn't
need it. This adds a hook for Cthulhu (and other tooling that would
like to receive it) to get more verbose stmt info out of inference.
As an example, this adds statement info for return_type, which is only
produced on opt in. There is a few more of these, but this shows the
flavor of what I'm going for.
My end goal here is to turn Cthulhu entirely into a viewer for inference
data structures, and not have it contain any heuristics that duplicate
inference functionality. I think we're pretty close to that, but
a few tweaks are required. Hopefully the end result should be a much
more functional and precise Cthulhu (as well as being easier to
maintain, since we get to delete half the code).