MethodAnalysis.jl

This package facilitates introspection of Julia's internals, with a particular focus on its MethodInstances and their backedges.

Warning

Julia's internals are not subject to the same interface compatibility guarantee that the rest of the language enjoys.

Demonstrations

A few demonstrations will give you a taste of what can be done with this package.

Collecting all submodules of Base

julia> using MethodAnalysis

julia> mods = Module[];

julia> visit(Base) do obj
           if isa(obj, Module)
               push!(mods, obj)
               return true     # descend into submodules
           end
           false   # but don't descend into anything else (MethodTables, etc.)
       end

julia> Base.FastMath ∈ mods
true

You can do this more easily with the convenience utility child_modules.

Collecting all Methods of functions defined in Core.Compiler

visit also descends into functions, methods, and MethodInstances:

julia> meths = []
Any[]

julia> visit(Core.Compiler) do item
           isa(item, Method) && push!(meths, item)
           true   # walk through everything
       end

julia> first(methods(Core.Compiler.typeinf_ext)) ∈ meths
true
Note

Methods are found by visiting the function. This has an important consequence: if PkgB defines a new method for PkgA.f, you won't find that method by visiting PkgB: you have to visit PkgA.f (which you can find by visiting PkgA). This is a consequence of how Julia stores Methods, not a limitation of MethodAnalysis.

Thus, to find all methods defined in PkgB, you have to traverse the entire system (visit() do ... end), and check the meth.module field of every Method to determine which module created it.

For methods that accept keyword arguments, Julia creates "hidden" methods for filling in the default values. Prior to Julia 1.9, you could find these by visiting the module that owns the "parent" function. On Julia 1.9 and above, these instead get added as methods of Core.kwcall. Consequently, these methods cannot be found by visiting the module that owns the parent function.

Getting a MethodInstance for a particular set of types

julia> foo(::AbstractVector) = 1
foo (generic function with 1 method)

julia> methodinstance(foo, (Vector{Int},))   # we haven't called it yet, so it's not compiled


julia> foo([1,2])
1

julia> methodinstance(foo, (Vector{Int},))
MethodInstance for foo(::Vector{Int64})

Collecting a subset of MethodInstances for a particular function

Let's collect all single-argument compiled instances of findfirst:

julia> mis = Core.MethodInstance[];

julia> visit(findfirst) do item
           isa(item, Core.MethodInstance) && length(Base.unwrap_unionall(item.specTypes).parameters) == 2 && push!(mis, item)
           true
       end

julia> mis
1-element Vector{Core.MethodInstance}:
 MethodInstance for findfirst(::BitVector)

We checked that the length was 2, rather than 1, because the first parameter is the function type itself:

julia> mis[1].specTypes
Tuple{typeof(findfirst), BitVector}

There's also a convenience shortcut:

julia> mis = methodinstances(findfirst)

Getting the backedges for a function

Let's see all the compiled instances of Base.setdiff and their immediate callers:

julia> direct_backedges(setdiff)
6-element Vector{Any}:
     MethodInstance for setdiff(::Base.KeySet{Any, Dict{Any, Any}}, ::Base.KeySet{Any, Dict{Any, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Dict{Any, Any})
     MethodInstance for setdiff(::Base.KeySet{Any, Dict{Any, Any}}, ::Base.KeySet{Any, Dict{Any, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
   MethodInstance for setdiff(::Base.KeySet{Char, Dict{Char, Any}}, ::Base.KeySet{Any, Dict{Any, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
   MethodInstance for setdiff(::Base.KeySet{Any, Dict{Any, Any}}, ::Base.KeySet{Char, Dict{Char, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
 MethodInstance for setdiff(::Base.KeySet{Char, Dict{Char, Any}}, ::Base.KeySet{Char, Dict{Char, Any}}) => MethodInstance for keymap_merge(::Dict{Char, Any}, ::Union{Dict{Any, Any}, Dict{Char, Any}})
                                   MethodInstance for setdiff(::Vector{Base.UUID}, ::Vector{Base.UUID}) => MethodInstance for deps_graph(::Pkg.Types.Context, ::Dict{Base.UUID, String}, ::Dict{Base.UUID, Pkg.Types.VersionSpec}, ::Dict{Base.UUID, Pkg.Resolve.Fixed})

Printing backedges as a tree

MethodAnalysis uses AbstractTrees to display the complete set of backedges:

julia> mi = methodinstance(findfirst, (BitVector,))
MethodInstance for findfirst(::BitVector)

julia> MethodAnalysis.print_tree(mi)
MethodInstance for findfirst(::BitVector)
├─ MethodInstance for prune_graph!(::Graph)
│  └─ MethodInstance for var"#simplify_graph!#111"(::Bool, ::typeof(simplify_graph!), ::Graph, ::Set{Int64})
│     └─ MethodInstance for simplify_graph!(::Graph, ::Set{Int64})
│        └─ MethodInstance for simplify_graph!(::Graph)
│           ├─ MethodInstance for trigger_failure!(::Graph, ::Vector{Int64}, ::Tuple{Int64, Int64})
│           │  ⋮
│           │
│           └─ MethodInstance for resolve_versions!(::Context, ::Vector{PackageSpec})
│              ⋮
│
├─ MethodInstance for update_solution!(::SolutionTrace, ::Graph)
│  └─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│     ├─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│     │  ├─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│     │  │  ├─ MethodInstance for converge!(::Graph, ::Messages, ::SolutionTrace, ::NodePerm, ::MaxSumParams)
│     │  │  │  ⋮
│     │  │  │
│     │  │  └─ MethodInstance for maxsum(::Graph)
│     │  │     ⋮
│     │  │
│     │  └─ MethodInstance for maxsum(::Graph)
│     │     └─ MethodInstance for resolve(::Graph)
│     │        ⋮
│     │
│     └─ MethodInstance for maxsum(::Graph)
│        └─ MethodInstance for resolve(::Graph)
│           ├─ MethodInstance for trigger_failure!(::Graph, ::Vector{Int64}, ::Tuple{Int64, Int64})
│           │  ⋮
│           │
│           └─ MethodInstance for resolve_versions!(::Context, ::Vector{PackageSpec})
│              ⋮
│
└─ MethodInstance for selective_eval_fromstart!(::typeof(finish_and_return!), ::Frame, ::BitVector, ::Bool)
   └─ MethodInstance for selective_eval_fromstart!(::Frame, ::BitVector, ::Bool)

Finding the callers of a method

To find already-compiled callers of sum(::Vector{Int})

# Collect all MethodInstances
mis = methodinstances();
# Create a function that returns `true` for the correct set of argument types
argmatch(argtyps) = length(argtyps) == 1 && argtyps[1] === Vector{Int}
# Find the calls that match
findcallers(sum, argmatch, mis)

There are more options, see the help for findcallers.

API reference

visit

MethodAnalysis.visitFunction
visit(operation, obj; print::Bool=false)

Scan obj and all of its "sub-objects" (e.g., functions if obj::Module, methods if obj::Function, etc.) recursively. operation(x) should return true if visit should descend into "sub-objects" of x.

If print is true, each visited object is printed to standard output.

Example

To collect all MethodInstances of a function,

julia> mis = Core.MethodInstance[];

julia> visit(findfirst) do x
           if isa(x, Core.MethodInstance)
               push!(mis, x)
               return false
           end
           true
       end

julia> length(mis)
34

The exact number of MethodInstances will depend on what code you've run in your Julia session.

source
visit(operation; print::Bool=false)

Scan all loaded modules with operation. See visit(operation, obj) for further detail.

Example

Collect all loaded modules, even if they are internal.

julia> mods = Module[];

julia> visit() do x
           if isa(x, Module)
               push!(mods, x)
               return true
           end
           false
       end

julia> mods 113-element Array{Module,1}: Random Random.DSFMT [...]

source
MethodAnalysis.visit_backedgesFunction
visit_backedges(operation, obj)

Visit the backedges of obj and apply operation to each. operation may need to be able to handle two call forms, operation(mi) and operation(sig=>mi), where mi is a MethodInstance and sig is a Tuple-type. The latter arises from either invoke calls or MethodTable backedges.

operation(edge) should return true if the backedges of edge should in turn be visited, false otherwise. However, no MethodInstance will be visited more than once.

The set of visited objects includes obj itself. For example, visit_backedges(operation, f::Function) will visit all methods of f, and this in turn will visit all MethodInstances of these methods.

source
MethodAnalysis.visit_withmoduleFunction
visit_withmodule(operation; print::Bool=false)
visit_withmodule(operation, obj, mod; print::Bool=false)

Similar to visit, except that operation should have signature operation(x, mod) where mod is either:

  • the module in which x was found, or
  • nothing if x is itself a top-level module.

If you're visiting underneath a specific object obj, you must supply mod, the module (or nothing) in which obj would be found.

source

backedges

MethodAnalysis.direct_backedgesFunction
direct_backedges(f::Function; skip=true)

Collect all backedges for a function f as pairs instance=>caller or sig=>caller pairs. The latter occur for MethodTable backedges. If skip is true, any caller listed in a MethodTable backedge is omitted from the instance backedges.

source
direct_backedges(mi::MethodInstance)

A vector of all direct backedges of mi. This is equivalent to mi.backedges except that it's "safe," meaning it returns an empty list even when mi.backedges is not defined.

source
MethodAnalysis.with_all_backedgesFunction
with_all_backedges(itr)

Return all MethodInstances detected when iterating through items in itr and any their backedges. The result includes both MethodTable and MethodInstance backedges.

source

utilities

MethodAnalysis.methodinstanceFunction
mi = methodinstance(f, types)
mi = methodinstance(tt::Type{<:Tuple})

Return the MethodInstance mi for function f and the given types, or for the complete signature tt. If no version compiled for these types exists, returns nothing.

Examples

julia> f(x, y::String) = 2x; f(x, y::Number) = x + y;

julia> f(1, "hi"); f(1, 1.0);

julia> methodinstance(f, (Int, String))
MethodInstance for f(::Int64, ::String)

julia> methodinstance(Tuple{typeof(f), Int, String})
MethodInstance for f(::Int64, ::String)
source
MethodAnalysis.methodinstancesFunction
methodinstances()
methodinstances(top)

Collect all MethodInstances, optionally restricting them to a particular module, function, method, or methodlist.

Examples

julia> sin(π/2)
1.0

julia> sin(0.8f0)
0.7173561f0

julia> methodinstances(sin)
2-element Vector{Core.MethodInstance}:
 MethodInstance for sin(::Float64)
 MethodInstance for sin(::Float32)

julia> m = which(convert, (Type{Bool}, Real))
convert(::Type{T}, x::Number) where T<:Number in Base at number.jl:7

julia> methodinstances(m)
68-element Vector{Core.MethodInstance}:
 MethodInstance for convert(::Type{UInt128}, ::Int64)
 MethodInstance for convert(::Type{Int128}, ::Int64)
 MethodInstance for convert(::Type{Int64}, ::Int32)
 MethodInstance for convert(::Type{UInt64}, ::Int64)
 ⋮

Note the method m was broader than the signature we queried with, and the returned MethodInstances reflect that breadth. See methodinstances for a more restrictive subset, and methodinstances_owned_by for collecting MethodInstances owned by specific modules.

source
methodinstances(f, types)
methodinstances(tt::Type{<:Tuple})

Return all MethodInstances whose signature is a subtype of types.

Example

julia> methodinstances(convert, (Type{Bool}, Real))
2-element Vector{Core.MethodInstance}:
 MethodInstance for convert(::Type{Bool}, ::Bool)
 MethodInstance for convert(::Type{Bool}, ::Int64)

Compare this to the result from methodinstance.

source
MethodAnalysis.methodinstances_owned_byFunction
mis = methodinstances_owned_by(mod::Module; include_child_modules::Bool=true, kwargs...)

Return a list of MethodInstances that are owned by mod. If include_child_modules is true, this includes sub-modules of mod, in which case kwargs are passed to child_modules.

The primary difference between methodinstances(mod) and methodinstances_owned_by(mod) is that the latter excludes MethodInstances that belong to re-exported dependent packages.

source
MethodAnalysis.child_modulesFunction
mods = child_modules(mod::Module; external::Bool=false)

Return a list that includes mod and all sub-modules of mod. By default, modules loaded from other sources (e.g., packages or those defined by Julia itself) are excluded, even if exported (or @reexported, see https://github.com/simonster/Reexport.jl), unless you set external=true.

Examples

julia> module Outer
       module Inner
       export Base
       end
       end
Main.Outer

julia> child_modules(Outer)
2-element Vector{Module}:
 Main.Outer
 Main.Outer.Inner

julia> child_modules(Outer.Inner)
1-element Vector{Module}:
 Main.Outer.Inner

Extended help

In the example above, because of the export Base, the following visit-based implementation would also collect Base and all of its sub-modules:

julia> mods = Module[]
Module[]

julia> visit(Outer) do item
           if item isa Module
               push!(mods, item)
               return true
           end
           return false
       end

julia> Base ∈ mods
true

julia> length(mods) > 20
true
source
MethodAnalysis.findcallersFunction
callers = findcallers(f, argmatch::Union{Nothing,Function}, mis; callhead=:call | :iterate)

Find "static" callers of a function f with inferred argument types for which argmatch(types) returns true. Optionally pass nothing for argmatch to allow any calls to f. mis is the list of MethodInstances you want to check, for example obtained from methodinstances.

callhead controls whether you're looking for an ordinary (:call) or a splatted (varargs) call (:iterate).

callers is a vector of CallMatch objects.

Examples

f(x) = rand()
function g()
    a = f(1.0)
    z = Any[7]
    b = f(z[1]::Integer)
    c = f(z...)
    return nothing
end

g()
mis = union(methodinstances(f), methodinstances(g))

# callhead = :call is the default
callers1 = findcallers(f, argtyps->length(argtyps) == 1 && argtyps[1] === Float64, mis)

# Get the partial-inference case with known number of arguments (this is definitely :call)
callers2 = findcallers(f, argtyps->length(argtyps) == 1 && argtyps[1] === Integer, mis; callhead=:call)

# Get the splat call
callers3 = findcallers(f, argtyps->length(argtyps) == 1 && argtyps[1] === Vector{Any}, mis; callhead=:iterate)
Compat

findcallers is available on Julia 1.6 and higher

Warn

findcallers is not guaranteed to find all calls. Calls can be "obfuscated" by many mechanisms, including calls from top level, calls where the function is a runtime variable, etc.

source
MethodAnalysis.hasboxFunction
hasbox(mi::MethodInstance)

Return true if the code for mi has a Core.Box. This often arises from a limitation in Julia's type-inference, see https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-captured.

source
MethodAnalysis.worldsFunction
minmaxs = worlds(mi::MethodInstance)

Collect the (min,max) world-age pairs for all CodeInstances associated with mi.

source

types

MethodAnalysis.CallMatchType
CallMatch

A structure to summarize a "matching" caller/callee pair. The fields are:

  • mi: the MethodInstance for the caller
  • src: its corresponding CodeInfo
  • sparams: the type parameters for the caller, given mi's signature (alternatively use mi.sparam_vals)
  • line: the statement number (in SSAValue sense) on which the call occurs
  • argtypes: the caller's inferred types passed as arguments to the callee
source