MethodAnalysis.jl
This package facilitates introspection of Julia's internals, with a particular focus on its MethodInstances and their backedges.
Julia's internals are not subject to the same interface compatibility guarantee that the rest of the language enjoys.
Demonstrations
A few demonstrations will give you a taste of what can be done with this package.
Collecting all submodules of Base
julia> using MethodAnalysis
julia> mods = Module[];
julia> visit(Base) do obj
if isa(obj, Module)
push!(mods, obj)
return true # descend into submodules
end
false # but don't descend into anything else (MethodTables, etc.)
end
julia> Base.FastMath ∈ mods
true
You can do this more easily with the convenience utility child_modules
.
Collecting all Methods of functions defined in Core.Compiler
visit
also descends into functions, methods, and MethodInstances:
julia> meths = []
Any[]
julia> visit(Core.Compiler) do item
isa(item, Method) && push!(meths, item)
true # walk through everything
end
julia> first(methods(Core.Compiler.typeinf_ext)) ∈ meths
true
Methods are found by visiting the function. This has an important consequence: if PkgB
defines a new method for PkgA.f
, you won't find that method by visiting PkgB
: you have to visit PkgA.f
(which you can find by visiting PkgA
). This is a consequence of how Julia stores Methods, not a limitation of MethodAnalysis.
Thus, to find all methods defined in PkgB
, you have to traverse the entire system (visit() do ... end
), and check the meth.module
field of every Method to determine which module created it.
For methods that accept keyword arguments, Julia creates "hidden" methods for filling in the default values. Prior to Julia 1.9, you could find these by visiting the module that owns the "parent" function. On Julia 1.9 and above, these instead get added as methods of Core.kwcall
. Consequently, these methods cannot be found by visiting the module that owns the parent function.
Getting a MethodInstance for a particular set of types
julia> foo(::AbstractVector) = 1
foo (generic function with 1 method)
julia> methodinstance(foo, (Vector{Int},)) # we haven't called it yet, so it's not compiled
julia> foo([1,2])
1
julia> methodinstance(foo, (Vector{Int},))
MethodInstance for foo(::Vector{Int64})
Collecting a subset of MethodInstances for a particular function
Let's collect all single-argument compiled instances of findfirst
:
julia> mis = Core.MethodInstance[];
julia> visit(findfirst) do item
isa(item, Core.MethodInstance) && length(Base.unwrap_unionall(item.specTypes).parameters) == 2 && push!(mis, item)
true
end
julia> mis
1-element Vector{Core.MethodInstance}:
MethodInstance for findfirst(::BitVector)
We checked that the length was 2, rather than 1, because the first parameter is the function type itself:
julia> mis[1].specTypes
Tuple{typeof(findfirst), BitVector}
There's also a convenience shortcut:
julia> mis = methodinstances(findfirst)
Getting the backedges for a function
Let's see all the compiled instances of Base.setdiff
and their immediate callers:
julia> direct_backedges(setdiff)
6-element Vector{Any}:
MethodInstance for setdiff(::Base.KeySet{Any, Dict{Any, Any}}, ::Base.KeySet{Any, Dict{Any, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Dict{Any, Any})
MethodInstance for setdiff(::Base.KeySet{Any, Dict{Any, Any}}, ::Base.KeySet{Any, Dict{Any, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
MethodInstance for setdiff(::Base.KeySet{Char, Dict{Char, Any}}, ::Base.KeySet{Any, Dict{Any, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
MethodInstance for setdiff(::Base.KeySet{Any, Dict{Any, Any}}, ::Base.KeySet{Char, Dict{Char, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
MethodInstance for setdiff(::Base.KeySet{Char, Dict{Char, Any}}, ::Base.KeySet{Char, Dict{Char, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
MethodInstance for setdiff(::Vector{Base.UUID}, ::Vector{Base.UUID}) => MethodInstance for deps_graph(::Pkg.Types.Context, ::Dict{Base.UUID, String}, ::Dict{Base.UUID, Pkg.Types.VersionSpec}, ::Dict{Base.UUID, Pkg.Resolve.Fixed})
Printing backedges as a tree
MethodAnalysis uses AbstractTrees to display the complete set of backedges:
julia> mi = methodinstance(findfirst, (BitVector,))
MethodInstance for findfirst(::BitVector)
julia> MethodAnalysis.print_tree(mi)
MethodInstance for findfirst(::BitVector)
├─ MethodInstance for prune_graph!(::Graph)
│ └─ MethodInstance for var"#simplify_graph!#111"(::Bool, ::typeof(simplify_graph!), ::Graph, ::Set{Int64})
│ └─ MethodInstance for simplify_graph!(::Graph, ::Set{Int64})
│ └─ MethodInstance for simplify_graph!(::Graph)
│ ├─ MethodInstance for trigger_failure!(::Graph, ::Vector{Int64}, ::Tuple{Int64, Int64})
│ │ ⋮
│ │
│ └─ MethodInstance for resolve_versions!(::Context, ::Vector{PackageSpec})
│ ⋮
│
├─ MethodInstance for update_solution!(::SolutionTrace, ::Graph)
│ └─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│ ├─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│ │ ├─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│ │ │ ├─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│ │ │ │ ⋮
│ │ │ │
│ │ │ └─ MethodInstance for maxsum(::Graph)
│ │ │ ⋮
│ │ │
│ │ └─ MethodInstance for maxsum(::Graph)
│ │ └─ MethodInstance for resolve(::Graph)
│ │ ⋮
│ │
│ └─ MethodInstance for maxsum(::Graph)
│ └─ MethodInstance for resolve(::Graph)
│ ├─ MethodInstance for trigger_failure!(::Graph, ::Vector{Int64}, ::Tuple{Int64, Int64})
│ │ ⋮
│ │
│ └─ MethodInstance for resolve_versions!(::Context, ::Vector{PackageSpec})
│ ⋮
│
└─ MethodInstance for selective_eval_fromstart!(::typeof(finish_and_return!), ::Frame, ::BitVector, ::Bool)
└─ MethodInstance for selective_eval_fromstart!(::Frame, ::BitVector, ::Bool)
Finding the callers of a method
To find already-compiled callers of sum(::Vector{Int})
# Collect all MethodInstances
mis = methodinstances();
# Create a function that returns `true` for the correct set of argument types
argmatch(argtyps) = length(argtyps) == 1 && argtyps[1] === Vector{Int}
# Find the calls that match
findcallers(sum, argmatch, mis)
There are more options, see the help for findcallers
.
API reference
visit
MethodAnalysis.visit
— Functionvisit(operation, obj; print::Bool=false)
Scan obj
and all of its "sub-objects" (e.g., functions if obj::Module
, methods if obj::Function
, etc.) recursively. operation(x)
should return true
if visit
should descend into "sub-objects" of x
.
If print
is true
, each visited object is printed to standard output.
Example
To collect all MethodInstances of a function,
julia> mis = Core.MethodInstance[];
julia> visit(findfirst) do x
if isa(x, Core.MethodInstance)
push!(mis, x)
return false
end
true
end
julia> length(mis)
34
The exact number of MethodInstances will depend on what code you've run in your Julia session.
visit(operation; print::Bool=false)
Scan all loaded modules with operation
. See visit(operation, obj)
for further detail.
Example
Collect all loaded modules, even if they are internal.
julia> mods = Module[];
julia> visit() do x
if isa(x, Module)
push!(mods, x)
return true
end
false
end
julia> mods 113-element Array{Module,1}: Random Random.DSFMT [...]
MethodAnalysis.visit_backedges
— Functionvisit_backedges(operation, obj)
Visit the backedges of obj
and apply operation
to each. operation
may need to be able to handle two call forms, operation(mi)
and operation(sig=>mi)
, where mi
is a MethodInstance
and sig
is a Tuple
-type. The latter arises from either invoke
calls or MethodTable
backedges.
operation(edge)
should return true
if the backedges of edge
should in turn be visited, false
otherwise. However, no MethodInstance
will be visited more than once.
The set of visited objects includes obj
itself. For example, visit_backedges(operation, f::Function)
will visit all methods of f
, and this in turn will visit all MethodInstances of these methods.
MethodAnalysis.visit_withmodule
— Functionvisit_withmodule(operation; print::Bool=false)
visit_withmodule(operation, obj, mod; print::Bool=false)
Similar to visit
, except that operation
should have signature operation(x, mod)
where mod
is either:
- the module in which
x
was found, or nothing
ifx
is itself a top-level module.
If you're visiting underneath a specific object obj
, you must supply mod
, the module (or nothing
) in which obj
would be found.
backedges
MethodAnalysis.all_backedges
— Functionall_backedges(mi::MethodInstance)
Return a list of all backedges (direct and indirect) of mi
.
MethodAnalysis.direct_backedges
— Functiondirect_backedges(f::Function; skip=true)
Collect all backedges for a function f
as pairs instance=>caller
or sig=>caller
pairs. The latter occur for MethodTable backedges. If skip
is true
, any caller
listed in a MethodTable backedge is omitted from the instance backedges.
direct_backedges(mi::MethodInstance)
A vector of all direct backedges of mi
. This is equivalent to mi.backedges
except that it's "safe," meaning it returns an empty list even when mi.backedges
is not defined.
MethodAnalysis.terminal_backedges
— Functionterminal_backedges(mi::MethodInstance)
Obtain the "ultimate callers" of mi
, i.e., the reason(s) mi
was compiled.
MethodAnalysis.with_all_backedges
— Functionwith_all_backedges(itr)
Return all MethodInstances detected when iterating through items in itr
and any their backedges. The result includes both MethodTable and MethodInstance backedges.
utilities
MethodAnalysis.methodinstance
— Functionmi = methodinstance(f, types)
mi = methodinstance(tt::Type{<:Tuple})
Return the MethodInstance
mi
for function f
and the given types
, or for the complete signature tt
. If no version compiled for these types exists, returns nothing
.
Examples
julia> f(x, y::String) = 2x; f(x, y::Number) = x + y;
julia> f(1, "hi"); f(1, 1.0);
julia> methodinstance(f, (Int, String))
MethodInstance for f(::Int64, ::String)
julia> methodinstance(Tuple{typeof(f), Int, String})
MethodInstance for f(::Int64, ::String)
MethodAnalysis.methodinstances
— Functionmethodinstances()
methodinstances(top)
Collect all MethodInstance
s, optionally restricting them to a particular module, function, method, or methodlist.
Examples
julia> sin(π/2)
1.0
julia> sin(0.8f0)
0.7173561f0
julia> methodinstances(sin)
2-element Vector{Core.MethodInstance}:
MethodInstance for sin(::Float64)
MethodInstance for sin(::Float32)
julia> m = which(convert, (Type{Bool}, Real))
convert(::Type{T}, x::Number) where T<:Number in Base at number.jl:7
julia> methodinstances(m)
68-element Vector{Core.MethodInstance}:
MethodInstance for convert(::Type{UInt128}, ::Int64)
MethodInstance for convert(::Type{Int128}, ::Int64)
MethodInstance for convert(::Type{Int64}, ::Int32)
MethodInstance for convert(::Type{UInt64}, ::Int64)
⋮
Note the method m
was broader than the signature we queried with, and the returned MethodInstance
s reflect that breadth. See methodinstances
for a more restrictive subset, and methodinstances_owned_by
for collecting MethodInstances owned by specific modules.
methodinstances(f, types)
methodinstances(tt::Type{<:Tuple})
Return all MethodInstances whose signature is a subtype of types
.
Example
julia> methodinstances(convert, (Type{Bool}, Real))
2-element Vector{Core.MethodInstance}:
MethodInstance for convert(::Type{Bool}, ::Bool)
MethodInstance for convert(::Type{Bool}, ::Int64)
Compare this to the result from methodinstance
.
MethodAnalysis.methodinstances_owned_by
— Functionmis = methodinstances_owned_by(mod::Module; include_child_modules::Bool=true, kwargs...)
Return a list of MethodInstance
s that are owned by mod
. If include_child_modules
is true
, this includes sub-modules of mod
, in which case kwargs
are passed to child_modules
.
The primary difference between methodinstances(mod)
and methodinstances_owned_by(mod)
is that the latter excludes MethodInstances
that belong to re-exported dependent packages.
MethodAnalysis.child_modules
— Functionmods = child_modules(mod::Module; external::Bool=false)
Return a list that includes mod
and all sub-modules of mod
. By default, modules loaded from other sources (e.g., packages or those defined by Julia itself) are excluded, even if exported (or @reexport
ed, see https://github.com/simonster/Reexport.jl), unless you set external=true
.
Examples
julia> module Outer
module Inner
export Base
end
end
Main.Outer
julia> child_modules(Outer)
2-element Vector{Module}:
Main.Outer
Main.Outer.Inner
julia> child_modules(Outer.Inner)
1-element Vector{Module}:
Main.Outer.Inner
Extended help
In the example above, because of the export Base
, the following visit
-based implementation would also collect Base
and all of its sub-modules:
julia> mods = Module[]
Module[]
julia> visit(Outer) do item
if item isa Module
push!(mods, item)
return true
end
return false
end
julia> Base ∈ mods
true
julia> length(mods) > 20
true
MethodAnalysis.call_type
— Functioncall_type(tt)
Split a signature type like Tuple{typeof(f),ArgTypes...}
back out to (f, Tuple{ArgTypes...})
MethodAnalysis.findcallers
— Functioncallers = findcallers(f, argmatch::Union{Nothing,Function}, mis; callhead=:call | :iterate)
Find "static" callers of a function f
with inferred argument types for which argmatch(types)
returns true. Optionally pass nothing
for argmatch
to allow any calls to f
. mis
is the list of MethodInstance
s you want to check, for example obtained from methodinstances
.
callhead
controls whether you're looking for an ordinary (:call
) or a splatted (varargs) call (:iterate
).
callers
is a vector of CallMatch
objects.
Examples
f(x) = rand()
function g()
a = f(1.0)
z = Any[7]
b = f(z[1]::Integer)
c = f(z...)
return nothing
end
g()
mis = union(methodinstances(f), methodinstances(g))
# callhead = :call is the default
callers1 = findcallers(f, argtyps->length(argtyps) == 1 && argtyps[1] === Float64, mis)
# Get the partial-inference case with known number of arguments (this is definitely :call)
callers2 = findcallers(f, argtyps->length(argtyps) == 1 && argtyps[1] === Integer, mis; callhead=:call)
# Get the splat call
callers3 = findcallers(f, argtyps->length(argtyps) == 1 && argtyps[1] === Vector{Any}, mis; callhead=:iterate)
findcallers
is available on Julia 1.6 and higher
findcallers
is not guaranteed to find all calls. Calls can be "obfuscated" by many mechanisms, including calls from top level, calls where the function is a runtime variable, etc.
MethodAnalysis.hasbox
— Functionhasbox(mi::MethodInstance)
Return true
if the code for mi
has a Core.Box
. This often arises from a limitation in Julia's type-inference, see https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-captured.
MethodAnalysis.worlds
— Functionminmaxs = worlds(mi::MethodInstance)
Collect the (min,max) world-age pairs for all CodeInstances associated with mi
.
types
MethodAnalysis.CallMatch
— TypeCallMatch
A structure to summarize a "matching" caller/callee pair. The fields are:
mi
: theMethodInstance
for the callersrc
: its correspondingCodeInfo
sparams
: the type parameters for the caller, givenmi
's signature (alternatively usemi.sparam_vals
)line
: the statement number (in SSAValue sense) on which the call occursargtypes
: the caller's inferred types passed as arguments to the callee