tvm-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lianmin Zheng via TVM Discuss <>
Subject [TVM Discuss] [Development] [DISCUSS] and Documentation of AlterLayout Pass
Date Thu, 21 Mar 2019 06:48:47 GMT

@yzhliu  You are right. At that time, we thought `AlterOpLayout` does not have dependency
problem and can be done in a single forward pass, so we tried to do a lot of things in a single
pass, which includes operator substitution, layout inference, and layout-transformation insertion.
I agree that some semantics of this pass are confusing.

There are several use cases for this pass
1. **Replace an operator with a new data flow graph**
    By registering the `@register_alter_op_layout`, users can replace an operator with a data
flow graph. But there is a constraint that the new data flow should have the same number of
inputs as the original operator.

    Weight pre-packing for dense and winograd conv2d lies in this catagory.
2. **Alternate the layout of a conv2d**
   In the registered `@register_alter_op_layout`, if we return a conv2d operator with the
same attributes, except for the different layout. Then this pass will try to insert the necessary
layout transforms. The tricky part is how to insert minimal transformations. The current implementation
uses a greedy method which propagates the layout of conv2d to pooling, broadcast and elemwise
The layout correction rule is done by registering `FInferCorrectLayout`. You can see two types
of implementation. For conv2d, dense, they are "active" operators, the layout correction starts
from their attributes `layout`. For pooling, broadcast and elemwise operators, they are "passive"
operators, they need to modify themselves to fit the input layout (i.e. propagate layout)

**Why mutation on const is required?**
Because I want to provide a minimal interface for users to register `FInferCorrectLayout`,
whose input and output are only layouts. However, during layout correction (or layout propagation),
some "passive" operators like pooling have to modify their attributes because their attribues
has the `layout` filed. So I ended up by modifying the const attributes in the `FInferCorrectLayout`
If we want to eliminate this mutation, ideally we should use `FTVMAlterOpLayout` for these
operators. But registering entire rewrite rules `FTVMAlterOpLayout` for all operators is very
redundant. Because during rewrite, one operator can even be replaced by a data flow graph,
the rewrite rule is not straightforward and not only depends on the single op. There are too
much common parts of `FTVMAlterOpLayout`.

In terms of refactoring, maybe we can decouple these two usages and implement them as several
separate passes. I am happy to follow up on your RFC.

[Visit Topic](
to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](

Tianqi Chen, UW, Seattle, WA, 98105, United States
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message