Skip to content

WIP: Front end for EnzymeMLIR ProbProg pass #1236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
Draft

WIP: Front end for EnzymeMLIR ProbProg pass #1236

wants to merge 12 commits into from

Conversation

sbrantq
Copy link
Member

@sbrantq sbrantq commented May 2, 2025

WIP: For now just a minimal generate API.

@sbrantq sbrantq requested review from wsmoses and abajpai15 May 2, 2025 03:33
@sbrantq sbrantq changed the title Front end for EnzymeMLIR ProbProg pass WIP: Front end for EnzymeMLIR ProbProg pass May 2, 2025
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

JuliaFormatter

[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶


[JuliaFormatter] reported by reviewdog 🐶

op_ty_results = IR.Type[result_0, ]


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.init", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶


[JuliaFormatter] reported by reviewdog 🐶

op_ty_results = IR.Type[output, ]


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.placeholder", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶


[JuliaFormatter] reported by reviewdog 🐶

op_ty_results = IR.Type[output, ]
operands = Value[cache, ]


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.pop", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶


[JuliaFormatter] reported by reviewdog 🐶

operands = Value[cache, value, ]


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.push", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶

function sample(inputs::Vector{Value}; outputs::Vector{IR.Type}, fn, name=nothing, location=Location())
op_ty_results = IR.Type[outputs..., ]
operands = Value[inputs..., ]


[JuliaFormatter] reported by reviewdog 🐶

attributes = NamedAttribute[namedattribute("fn", fn), ]


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.sample", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶


[JuliaFormatter] reported by reviewdog 🐶

operands = Value[gradient, value, ]


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.set", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶

function simulate(inputs::Vector{Value}; trace::IR.Type, fn, name=nothing, location=Location())
op_ty_results = IR.Type[trace, ]
operands = Value[inputs..., ]


[JuliaFormatter] reported by reviewdog 🐶

attributes = NamedAttribute[namedattribute("fn", fn), ]


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.simulate", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶

function trace(inputs::Vector{Value}, oldTrace=nothing::Union{Nothing, Value}; constraints=nothing::Union{Nothing, Value}, newTrace::IR.Type, weights::Vector{IR.Type}, fn, name=nothing, location=Location())
op_ty_results = IR.Type[newTrace, weights..., ]
operands = Value[inputs..., ]


[JuliaFormatter] reported by reviewdog 🐶

attributes = NamedAttribute[namedattribute("fn", fn), ]


[JuliaFormatter] reported by reviewdog 🐶

push!(attributes, operandsegmentsizes([length(inputs), (oldTrace==nothing) ? 0 : 1(constraints==nothing) ? 0 : 1]))


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.trace", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶

operands = Value[trace, sample, ]


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.addSampleToTrace", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶

function insertChoiceToMap(choiceMap::Value, choice::Value; newChoiceMap::IR.Type, name=nothing, location=Location())
op_ty_results = IR.Type[newChoiceMap, ]
operands = Value[choiceMap, choice, ]


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.insertChoiceToMap", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false

create_operation,
context,
IndexType
import ...IR: NamedAttribute, Value, Location, Block, Region, Attribute, create_operation, context, IndexType
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
import ...IR: NamedAttribute, Value, Location, Block, Region, Attribute, create_operation, context, IndexType
import ...IR:
NamedAttribute,
Value,
Location,
Block,
Region,
Attribute,
create_operation,
context,
IndexType

import ..Dialects: namedattribute, operandsegmentsizes
import ...API


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change

"""
`addTo`

TODO
"""
function addTo(values::Vector{Value}; location=Location())
op_ty_results = IR.Type[]
operands = Value[values...,]
operands = Value[values..., ]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
operands = Value[values..., ]
operands = Value[values...,]

Comment on lines +19 to +22

create_operation(
"enzyme.addTo", location;
operands, owned_regions, successors, attributes,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
create_operation(
"enzyme.addTo", location;
operands, owned_regions, successors, attributes,
return create_operation(
"enzyme.addTo",
location;
operands,
owned_regions,
successors,
attributes,

results=op_ty_results,
result_inference=false,
result_inference=false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
result_inference=false
result_inference=false,

Comment on lines +131 to +135
!isnothing(library_call) && push!(attributes, namedattribute("library_call", library_call))

create_operation(
"enzyme.genericAdjoint", location;
operands, owned_regions, successors, attributes,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
!isnothing(library_call) && push!(attributes, namedattribute("library_call", library_call))
create_operation(
"enzyme.genericAdjoint", location;
operands, owned_regions, successors, attributes,
!isnothing(library_call) &&
push!(attributes, namedattribute("library_call", library_call))
return create_operation(
"enzyme.genericAdjoint",
location;
operands,
owned_regions,
successors,
attributes,

results=op_ty_results,
result_inference=false,
result_inference=false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
result_inference=false
result_inference=false,

)
end


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change

Comment on lines +143 to +144
op_ty_results = IR.Type[result_0, ]
operands = Value[gradient, ]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
op_ty_results = IR.Type[result_0, ]
operands = Value[gradient, ]
op_ty_results = IR.Type[result_0,]
operands = Value[gradient,]

Comment on lines +148 to +151

create_operation(
"enzyme.get", location;
operands, owned_regions, successors, attributes,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
create_operation(
"enzyme.get", location;
operands, owned_regions, successors, attributes,
return create_operation(
"enzyme.get",
location;
operands,
owned_regions,
successors,
attributes,

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

JuliaFormatter

[JuliaFormatter] reported by reviewdog 🐶

function trace(inputs::Vector{Value}, oldTrace=nothing::Union{Nothing, Value}; constraints=nothing::Union{Nothing, Value}, newTrace::IR.Type, weights::Vector{IR.Type}, fn, name=nothing, location=Location())
op_ty_results = IR.Type[newTrace, weights..., ]
operands = Value[inputs..., ]


[JuliaFormatter] reported by reviewdog 🐶

attributes = NamedAttribute[namedattribute("fn", fn), ]


[JuliaFormatter] reported by reviewdog 🐶

push!(attributes, operandsegmentsizes([length(inputs), (oldTrace==nothing) ? 0 : 1(constraints==nothing) ? 0 : 1]))


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.trace", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶

operands = Value[trace, sample, ]


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.addSampleToTrace", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶

function insertChoiceToMap(choiceMap::Value, choice::Value; outputs::IR.Type, name=nothing, location=Location())
op_ty_results = IR.Type[outputs, ]
operands = Value[choiceMap, choice, ]


[JuliaFormatter] reported by reviewdog 🐶

create_operation(
"enzyme.insertChoiceToMap", location;
operands, owned_regions, successors, attributes,


[JuliaFormatter] reported by reviewdog 🐶

result_inference=false


[JuliaFormatter] reported by reviewdog 🐶

@test Array(model_compiled(seed1, μ1, σ1, shape)) Array(model_compiled(seed1, μ1, σ1, shape))
@test mean(Array(model_compiled(seed1, μ1, σ1, shape))) 0.0 atol = 0.05 rtol = 0.05
@test mean(Array(model_compiled(seed2, μ2, σ2, shape))) 1000.0 atol = 0.05 rtol = 0.05


[JuliaFormatter] reported by reviewdog 🐶

Array(model_compiled(seed1, μ1, σ1, shape)) .≈ Array(model_compiled(seed2, μ2, σ2, shape))

return result
end

end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
end
end

results=op_ty_results,
result_inference=false,
result_inference=false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
result_inference=false
result_inference=false,

)
end


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change

function init(; result_0::IR.Type, location=Location())
op_ty_results = IR.Type[result_0,]
op_ty_results = IR.Type[result_0, ]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
op_ty_results = IR.Type[result_0, ]
op_ty_results = IR.Type[result_0,]

Comment on lines +164 to +167

create_operation(
"enzyme.init", location;
operands, owned_regions, successors, attributes,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
create_operation(
"enzyme.init", location;
operands, owned_regions, successors, attributes,
return create_operation(
"enzyme.init",
location;
operands,
owned_regions,
successors,
attributes,

"enzyme.set", location;
operands, owned_regions, successors, attributes,
results=op_ty_results,
result_inference=false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
result_inference=false
result_inference=false,

Comment on lines +261 to +263
function simulate(inputs::Vector{Value}; outputs::Vector{IR.Type}, fn, name=nothing, location=Location())
op_ty_results = IR.Type[outputs..., ]
operands = Value[inputs..., ]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
function simulate(inputs::Vector{Value}; outputs::Vector{IR.Type}, fn, name=nothing, location=Location())
op_ty_results = IR.Type[outputs..., ]
operands = Value[inputs..., ]
function simulate(
inputs::Vector{Value}; outputs::Vector{IR.Type}, fn, name=nothing, location=Location()
)
op_ty_results = IR.Type[outputs...,]
operands = Value[inputs...,]

operands = Value[inputs..., ]
owned_regions = Region[]
successors = Block[]
attributes = NamedAttribute[namedattribute("fn", fn), ]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
attributes = NamedAttribute[namedattribute("fn", fn), ]
attributes = NamedAttribute[namedattribute("fn", fn),]

Comment on lines +268 to +271

create_operation(
"enzyme.simulate", location;
operands, owned_regions, successors, attributes,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
create_operation(
"enzyme.simulate", location;
operands, owned_regions, successors, attributes,
return create_operation(
"enzyme.simulate",
location;
operands,
owned_regions,
successors,
attributes,

"enzyme.simulate", location;
operands, owned_regions, successors, attributes,
results=op_ty_results,
result_inference=false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[JuliaFormatter] reported by reviewdog 🐶

Suggested change
result_inference=false
result_inference=false,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant