MethodAnalysis.jl

This package facilitates introspection of Julia's internals, with a particular focus on its MethodInstances and their backedges.

Warning

Julia's internals are not subject to the same interface compatibility guarantee that the rest of the language enjoys.

Demonstrations

A few demonstrations will give you a taste of what can be done with this package.

Collecting all submodules of Base

julia> using MethodAnalysis

julia> mods = Module[];

julia> visit(Base) do obj
           if isa(obj, Module)
               push!(mods, obj)
               return true     # descend into submodules
           end
           false   # but don't descend into anything else (MethodTables, etc.)
       end

julia> Base.FastMath ∈ mods
true

You can do this more easily with the convenience utility child_modules.

Collecting all Methods of functions defined in Core.Compiler

visit also descends into functions, methods, and MethodInstances:

julia> meths = []
Any[]

julia> visit(Core.Compiler) do item
           isa(item, Method) && push!(meths, item)
           true   # walk through everything
       end

julia> first(methods(Core.Compiler.typeinf_ext)) ∈ meths
true
Note

Methods are found by visiting the function. This has an important consequence: if PkgB defines a new method for PkgA.f, you won't find that method by visiting PkgB: you have to visit PkgA.f (which you can find by visiting PkgA). This is a consequence of how Julia stores Methods, not a limitation of MethodAnalysis.

Thus, to find all methods defined in PkgB, you have to traverse the entire system (visit() do ... end), and check the meth.module field of every Method to determine which module created it.

For methods that accept keyword arguments, Julia creates "hidden" methods for filling in the default values. Prior to Julia 1.9, you could find these by visiting the module that owns the "parent" function. On Julia 1.9 and above, these instead get added as methods of Core.kwcall. Consequently, these methods cannot be found by visiting the module that owns the parent function.

Getting a MethodInstance for a particular set of types

julia> foo(::AbstractVector) = 1
foo (generic function with 1 method)

julia> methodinstance(foo, (Vector{Int},))   # we haven't called it yet, so it's not compiled


julia> foo([1,2])
1

julia> methodinstance(foo, (Vector{Int},))
MethodInstance for foo(::Vector{Int64})

Collecting a subset of MethodInstances for a particular function

Let's collect all single-argument compiled instances of findfirst:

julia> mis = Core.MethodInstance[];

julia> visit(findfirst) do item
           isa(item, Core.MethodInstance) && length(Base.unwrap_unionall(item.specTypes).parameters) == 2 && push!(mis, item)
           true
       end

julia> mis
1-element Vector{Core.MethodInstance}:
 MethodInstance for findfirst(::BitVector)

We checked that the length was 2, rather than 1, because the first parameter is the function type itself:

julia> mis[1].specTypes
Tuple{typeof(findfirst), BitVector}

There's also a convenience shortcut:

julia> mis = methodinstances(findfirst)

Getting the backedges for a function

Let's see all the compiled instances of Base.setdiff and their immediate callers:

julia> direct_backedges(setdiff)
6-element Vector{Any}:
     MethodInstance for setdiff(::Base.KeySet{Any, Dict{Any, Any}}, ::Base.KeySet{Any, Dict{Any, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Dict{Any, Any})
     MethodInstance for setdiff(::Base.KeySet{Any, Dict{Any, Any}}, ::Base.KeySet{Any, Dict{Any, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
   MethodInstance for setdiff(::Base.KeySet{Char, Dict{Char, Any}}, ::Base.KeySet{Any, Dict{Any, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
   MethodInstance for setdiff(::Base.KeySet{Any, Dict{Any, Any}}, ::Base.KeySet{Char, Dict{Char, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
 MethodInstance for setdiff(::Base.KeySet{Char, Dict{Char, Any}}, ::Base.KeySet{Char, Dict{Char, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
                                   MethodInstance for setdiff(::Vector{Base.UUID}, ::Vector{Base.UUID}) => MethodInstance for deps_graph(::Pkg.Types.Context, ::Dict{Base.UUID, String}, ::Dict{Base.UUID, Pkg.Types.VersionSpec}, ::Dict{Base.UUID, Pkg.Resolve.Fixed})

Printing backedges as a tree

MethodAnalysis uses AbstractTrees to display the complete set of backedges:

julia> mi = methodinstance(findfirst, (BitVector,))
MethodInstance for findfirst(::BitVector)

julia> MethodAnalysis.print_tree(mi)
MethodInstance for findfirst(::BitVector)
├─ MethodInstance for prune_graph!(::Graph)
│  └─ MethodInstance for var"#simplify_graph!#111"(::Bool, ::typeof(simplify_graph!), ::Graph, ::Set{Int64})
│     └─ MethodInstance for simplify_graph!(::Graph, ::Set{Int64})
│        └─ MethodInstance for simplify_graph!(::Graph)
│           ├─ MethodInstance for trigger_failure!(::Graph, ::Vector{Int64}, ::Tuple{Int64, Int64})
│           │  ⋮
│           │
│           └─ MethodInstance for resolve_versions!(::Context, ::Vector{PackageSpec})
│              ⋮
│
├─ MethodInstance for update_solution!(::SolutionTrace, ::Graph)
│  └─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│     ├─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│     │  ├─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│     │  │  ├─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│     │  │  │  ⋮
│     │  │  │
│     │  │  └─ MethodInstance for maxsum(::Graph)
│     │  │     ⋮
│     │  │
│     │  └─ MethodInstance for maxsum(::Graph)
│     │     └─ MethodInstance for resolve(::Graph)
│     │        ⋮
│     │
│     └─ MethodInstance for maxsum(::Graph)
│        └─ MethodInstance for resolve(::Graph)
│           ├─ MethodInstance for trigger_failure!(::Graph, ::Vector{Int64}, ::Tuple{Int64, Int64})
│           │  ⋮
│           │
│           └─ MethodInstance for resolve_versions!(::Context, ::Vector{PackageSpec})
│              ⋮
│
└─ MethodInstance for selective_eval_fromstart!(::typeof(finish_and_return!), ::Frame, ::BitVector, ::Bool)
   └─ MethodInstance for selective_eval_fromstart!(::Frame, ::BitVector, ::Bool)

Following calls forward

direct_calls reports what a MethodInstance calls, the opposite direction from direct_backedges. It includes callees later removed by optimization:

julia> module Demo
           inner(x) = 2x
           outer(x) = inner(x) + 1
       end
Main.Demo

julia> Demo.outer(3)
7

julia> direct_calls(methodinstance(Demo.outer, (Int,)))
2-element Vector{CallEdge}:
 CallEdge(direct, MethodInstance for Main.Demo.inner(::Int64))
 CallEdge(direct, MethodInstance for +(::Int64, ::Int64))

callgraph follows edges transitively. Use follow to stop traversal at selected callees:

julia> inmodule(t) = (m = t isa Method ? t : t.def; m isa Method && m.module === MyPkg);

julia> graph = callgraph(MyPkg; follow=inmodule);

To omit external callees while retaining package methods reached through functions such as map, use keep. The graph connects retained methods across omitted callees with :indirect edges:

julia> graph = callgraph(MyPkg; keep=inmodule);

Finding the callers of a method

To find already-compiled callers of sum(::Vector{Int})

# Collect all MethodInstances
mis = methodinstances();
# Create a function that returns `true` for the correct set of argument types
argmatch(argtyps) = length(argtyps) == 1 && argtyps[1] === Vector{Int}
# Find the calls that match
findcallers(sum, argmatch, mis)

There are more options, see the help for findcallers.

API reference

visit

MethodAnalysis.visitFunction
visit(operation, obj; print::Bool=false)

Scan obj and all of its "sub-objects" (e.g., functions if obj::Module, methods if obj::Function, etc.) recursively. operation(x) should return true if visit should descend into "sub-objects" of x.

If print is true, each visited object is printed to standard output.

Example

To collect all MethodInstances of a function,

julia> mis = Core.MethodInstance[];

julia> visit(findfirst) do x
           if isa(x, Core.MethodInstance)
               push!(mis, x)
               return false
           end
           true
       end

julia> length(mis)
34

The exact number of MethodInstances will depend on what code you've run in your Julia session.

source
visit(operation; print::Bool=false)

Scan all loaded modules with operation. See visit(operation, obj) for further detail.

Example

Collect all loaded modules, even if they are internal.

julia> mods = Module[];

julia> visit() do x
           if isa(x, Module)
               push!(mods, x)
               return true
           end
           false
       end

julia> mods 113-element Array{Module,1}: Random Random.DSFMT [...]

source
MethodAnalysis.visit_backedgesFunction
visit_backedges(operation, obj)

Visit the backedges of obj and apply operation to each. operation may need to be able to handle two call forms, operation(mi) and operation(sig=>mi), where mi is a MethodInstance and sig is a Tuple-type. The latter arises from either invoke calls or MethodTable backedges.

operation(edge) should return true if the backedges of edge should in turn be visited, false otherwise. However, no MethodInstance will be visited more than once.

The set of visited objects includes obj itself. For example, visit_backedges(operation, f::Function) will visit all methods of f, and this in turn will visit all MethodInstances of these methods.

source
MethodAnalysis.visit_withmoduleFunction
visit_withmodule(operation; print::Bool=false)
visit_withmodule(operation, obj, mod; print::Bool=false)

Similar to visit, except that operation should have signature operation(x, mod) where mod is either:

  • the module in which x was found, or
  • nothing if x is itself a top-level module.

If you're visiting underneath a specific object obj, you must supply mod, the module (or nothing) in which obj would be found.

source

backedges

MethodAnalysis.direct_backedgesFunction
direct_backedges(f::Function; skip=true)

Collect all backedges for a function f as pairs instance=>caller or sig=>caller pairs. The latter occur for MethodTable backedges. If skip is true, any caller listed in a MethodTable backedge is omitted from the instance backedges.

source
direct_backedges(mi::MethodInstance)

A vector of the callers of mi. This is equivalent to mi.backedges except that it's "safe," meaning it returns an empty list even when mi.backedges is not defined. A caller is listed once per backedge, so it appears more than once if it reaches mi through more than one call form. Use backedge_pairs to tell those forms apart.

source
MethodAnalysis.with_all_backedgesFunction
with_all_backedges(itr)

Return all MethodInstances detected when iterating through items in itr and any their backedges. The result includes both MethodTable and MethodInstance backedges.

source
MethodAnalysis.backedge_pairsFunction
prs = backedge_pairs(backedges::Vector{Any})

Decode a MethodInstance's backedges field into (invokesig, caller) tuples. invokesig is nothing unless the call was made through invoke, in which case it is the signature named there. A caller may appear more than once, once per distinct invokesig.

source

forward call edges

MethodAnalysis.direct_callsFunction
edges = direct_calls(ci::CodeInstance)
edges = direct_calls(mi::MethodInstance)

Return the CallEdges recorded while compiling ci, or their union over all of mi's codeinstances.

Calls through a value typed only as Function are not recorded. InteractiveUtils.@trace_dispatch can identify their runtime targets.

Compat

Requires at least Julia 1.12.

Example

julia> f(x) = sum(abs2, x);

julia> f([1.0]);

julia> direct_calls(methodinstance(f, (Vector{Float64},)))
1-element Vector{CallEdge}:
 CallEdge(direct, MethodInstance for sum(::typeof(abs2), ::Vector{Float64}))
source
MethodAnalysis.visit_callsFunction
visit_calls(operation, obj)

Walk forward call edges, starting from obj and calling operation(caller, edge) for each CallEdge reached. caller is the CallTarget the edge was recorded on.

operation(caller, edge) should return true to visit the callee's edges and false to skip them. No callee is visited more than once.

obj may be a MethodInstance, a CodeInstance, a collection of either, or anything methodinstances accepts, such as a function or a module.

source
MethodAnalysis.callgraphFunction
graph = callgraph(obj; follow=Returns(true), keep=Returns(true))

Build the forward call graph reachable from obj, as a dictionary mapping each CallTarget to its CallEdges. See visit_calls for the accepted forms of obj.

follow(callee) controls traversal. Rejected callees remain in the graph but have no outgoing edges.

keep(callee) decides which callees appear in graph; the starting points always do. A rejected callee is still traversed. Kept targets reached through rejected callees are connected to the nearest kept caller by :indirect edges.

Examples

To keep all calls among methods of MyPkg, use

inmodule(t) = (m = t isa Method ? t : t.def; m isa Method && m.module === MyPkg)
callgraph(MyPkg; keep=inmodule)

Extended help

Each caller has at most one :indirect edge per callee. A direct edge takes precedence over an indirect edge for the same pair.

follow controls traversal, whereas keep filters the resulting graph. Because keep cannot include targets beyond a callee rejected by follow, use keep alone to restrict a graph to one module:

source

utilities

MethodAnalysis.methodinstanceFunction
mi = methodinstance(f, types)
mi = methodinstance(tt::Type{<:Tuple})

Return the MethodInstance mi for function f and the given types, or for the complete signature tt. If no version compiled for these types exists, returns nothing.

Examples

julia> f(x, y::String) = 2x; f(x, y::Number) = x + y;

julia> f(1, "hi"); f(1, 1.0);

julia> methodinstance(f, (Int, String))
MethodInstance for f(::Int64, ::String)

julia> methodinstance(Tuple{typeof(f), Int, String})
MethodInstance for f(::Int64, ::String)
source
MethodAnalysis.methodinstancesFunction
methodinstances()
methodinstances(top)

Collect all MethodInstances, optionally restricting them to a particular module, function, method, or methodlist.

Examples

julia> sin(π/2)
1.0

julia> sin(0.8f0)
0.7173561f0

julia> methodinstances(sin)
2-element Vector{Core.MethodInstance}:
 MethodInstance for sin(::Float64)
 MethodInstance for sin(::Float32)

julia> m = which(convert, (Type{Bool}, Real))
convert(::Type{T}, x::Number) where T<:Number in Base at number.jl:7

julia> methodinstances(m)
68-element Vector{Core.MethodInstance}:
 MethodInstance for convert(::Type{UInt128}, ::Int64)
 MethodInstance for convert(::Type{Int128}, ::Int64)
 MethodInstance for convert(::Type{Int64}, ::Int32)
 MethodInstance for convert(::Type{UInt64}, ::Int64)
 ⋮

Note the method m was broader than the signature we queried with, and the returned MethodInstances reflect that breadth. See methodinstances for a more restrictive subset, and methodinstances_owned_by for collecting MethodInstances owned by specific modules.

source
methodinstances(f, types)
methodinstances(tt::Type{<:Tuple})

Return all MethodInstances whose signature is a subtype of types.

Example

julia> methodinstances(convert, (Type{Bool}, Real))
2-element Vector{Core.MethodInstance}:
 MethodInstance for convert(::Type{Bool}, ::Bool)
 MethodInstance for convert(::Type{Bool}, ::Int64)

Compare this to the result from methodinstance.

source
MethodAnalysis.methodinstances_owned_byFunction
mis = methodinstances_owned_by(mod::Module; include_child_modules::Bool=true, kwargs...)

Return a list of MethodInstances that are owned by mod. If include_child_modules is true, this includes sub-modules of mod, in which case kwargs are passed to child_modules.

The primary difference between methodinstances(mod) and methodinstances_owned_by(mod) is that the latter excludes MethodInstances that belong to re-exported dependent packages.

source
MethodAnalysis.codeinstancesFunction
cis = codeinstances(mi::MethodInstance; owner=nothing)

Collect the CodeInstances cached for mi. By default only results from Julia's own compiler are returned; pass owner to select those produced by an external AbstractInterpreter instead.

source
MethodAnalysis.child_modulesFunction
mods = child_modules(mod::Module; external::Bool=false)

Return a list that includes mod and all sub-modules of mod. By default, modules loaded from other sources (e.g., packages or those defined by Julia itself) are excluded, even if exported (or @reexported, see https://github.com/simonster/Reexport.jl), unless you set external=true.

Examples

julia> module Outer
       module Inner
       export Base
       end
       end
Main.Outer

julia> child_modules(Outer)
2-element Vector{Module}:
 Main.Outer
 Main.Outer.Inner

julia> child_modules(Outer.Inner)
1-element Vector{Module}:
 Main.Outer.Inner

Extended help

In the example above, because of the export Base, the following visit-based implementation would also collect Base and all of its sub-modules:

julia> mods = Module[]
Module[]

julia> visit(Outer) do item
           if item isa Module
               push!(mods, item)
               return true
           end
           return false
       end

julia> Base ∈ mods
true

julia> length(mods) > 20
true
source
MethodAnalysis.findcallersFunction
callers = findcallers(f, argmatch::Union{Nothing,Function}, mis; callhead=:call | :iterate)

Find "static" callers of a function f with inferred argument types for which argmatch(types) returns true. Optionally pass nothing for argmatch to allow any calls to f. mis is the list of MethodInstances you want to check, for example obtained from methodinstances.

callhead controls whether you're looking for an ordinary (:call) or a splatted (varargs) call (:iterate).

callers is a vector of CallMatch objects.

Examples

f(x) = rand()
function g()
    a = f(1.0)
    z = Any[7]
    b = f(z[1]::Integer)
    c = f(z...)
    return nothing
end

g()
mis = union(methodinstances(f), methodinstances(g))

# callhead = :call is the default
callers1 = findcallers(f, argtyps->length(argtyps) == 1 && argtyps[1] === Float64, mis)

# Get the partial-inference case with known number of arguments (this is definitely :call)
callers2 = findcallers(f, argtyps->length(argtyps) == 1 && argtyps[1] === Integer, mis; callhead=:call)

# Get the splat call
callers3 = findcallers(f, argtyps->length(argtyps) == 1 && argtyps[1] === Vector{Any}, mis; callhead=:iterate)
Compat

findcallers is available on Julia 1.6 and higher

Warn

findcallers is not guaranteed to find all calls. Calls can be "obfuscated" by many mechanisms, including calls from top level, calls where the function is a runtime variable, etc.

source
MethodAnalysis.hasboxFunction
hasbox(mi::MethodInstance)

Return true if the code for mi has a Core.Box. This often arises from a limitation in Julia's type-inference, see https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-captured.

source
MethodAnalysis.worldsFunction
minmaxs = worlds(mi::MethodInstance)

Collect the (min,max) world-age pairs for all CodeInstances associated with mi.

source

types

MethodAnalysis.CallMatchType
CallMatch

A structure to summarize a "matching" caller/callee pair. The fields are:

  • mi: the MethodInstance for the caller
  • src: its corresponding CodeInfo
  • sparams: the type parameters for the caller, given mi's signature (alternatively use mi.sparam_vals)
  • line: the statement number (in SSAValue sense) on which the call occurs
  • argtypes: the caller's inferred types passed as arguments to the callee. An argument that is itself a type is reported as Type{X}, so predicates can be written the same way on every Julia version.
source
MethodAnalysis.CallEdgeType
CallEdge

One call from a caller to a callee, as recorded by type inference.

Extended help

Fields:

  • callee: the CallTarget, or nothing for :nomatch edges
  • code: the callee's CodeInstance if one was recorded, otherwise nothing
  • kind: :direct, :invoke, :dispatch, :nomatch, or :indirect (see below)
  • sig: the call signature, a Tuple type; nothing for :direct edges
  • covered: whether callee is the only possible target for sig

kind may be:

  • :direct: inference resolved the call to a single callee.
  • :invoke: an invoke call, where sig names the method explicitly.
  • :dispatch: dispatch on sig may reach callee. There is one edge per applicable method. If covered is false, other methods may also be reached.
  • :nomatch: no method applies to sig, so reaching this call site raises a MethodError. callee and code are nothing.
  • :indirect: callgraph omitted the intermediate callees between the caller and callee. sig and covered describe the original call site.

See direct_calls for call sites that do not produce an edge.

source
MethodAnalysis.CallTargetType

A callee recorded on a forward edge: a MethodInstance when the callee has been specialized, or a Method when it has not.

source