HostIR integration to FusionExecutorCache PR 1/n #3707
Conversation
nsarka
force-pushed
the
nsarka/hostir-integration
branch
from
94b4d48 to
5852eb7
Compare
|
Thanks, updated. |
nsarka
force-pushed
the
nsarka/hostir-integration
branch
from
581d0c9 to
4409faa
Compare
|
!test |
nsarka
force-pushed
the
nsarka/hostir-integration
branch
from
2590d88 to
bab9bff
Compare
| std::vector<Val*> inputs, | ||
| std::vector<Val*> outputs); |
There was a problem hiding this comment.
| std::vector<Val*> inputs, | |
| std::vector<Val*> outputs); | |
| const std::vector<Val*>& inputs, | |
| const std::vector<Val*>& outputs); |
so it's easier to avoid unnecessary copies -- no std::move needed.
| return ss.str(); | ||
| } | ||
|
|
||
| int64_t LaunchKernel::getIndex() const { |
There was a problem hiding this comment.
Style nit: I'd inline trivial implementation like this in the class declaration so the readers don't need to jump to cpp to understand what the method does.
|
|
||
| int64_t getIndex() const; | ||
|
|
||
| int64_t hic_executor_index_; |
| hic->addInput(launch_kernel_inputs.back()); | ||
| hic->addOutput(launch_kernel_outputs.back()); |
There was a problem hiding this comment.
| hic->addInput(launch_kernel_inputs.back()); | |
| hic->addOutput(launch_kernel_outputs.back()); | |
| hic->addInput(tv2); | |
| hic->addOutput(tv3); |
| hic->addOutput(launch_kernel_outputs.back()); | ||
|
|
||
| auto launch_kernel = IrBuilder::create<LaunchKernel>( | ||
| 0, launch_kernel_inputs, launch_kernel_outputs); |
There was a problem hiding this comment.
| 0, launch_kernel_inputs, launch_kernel_outputs); | |
| 0, {tv2}, {tv3}); |
This way, it's less code and you can remove launch_kernel_inputs/outputs.
|
|
||
| IrCloner ir_cloner(hic.get()); | ||
| auto tv2 = ir_cloner.clone(tv0); | ||
| auto tv3 = ir_cloner.clone(tv1); |
There was a problem hiding this comment.
Naming nit: I'd rename tv0/1/2/3 to something more meaningful, e.g., in, out, hic_in, hic_out. This way, the reader knows their meaning without reading how they are defined.
| HostIrEvaluator hie(std::move(hic)); | ||
|
|
||
| at::Tensor output = at::empty({32, 32}, options); | ||
| auto outputs = hie.runWithInput({{tv2, t0}, {tv3, output}}); |
There was a problem hiding this comment.
Therefore, this output is not used by HostIrEvaluator? What happens if you remove {tv3, output}?
That said, sooner or later, we will make output allocations explicit in host IR (see this section), and will run a KernelExecutor with pre-allocated outputs.
In this PR I add a vector of KernelExecutor* to HostIrContainer, and new HostIR called LaunchKernel that takes an index to the vector and launches the kernel. I also add a test to verify correctness.
This is the first in a series of PRs for integration with FusionExecutorCache.