tvm-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tianqi Chen <notificati...@github.com>
Subject [dmlc/tvm] [RFC][DISCUSS] Tuple-related Fusion (#3039)
Date Wed, 17 Apr 2019 19:03:39 GMT
Recently, there are a few problems arise wrt to the fusor and tuples. The main problem is the
incompatibility of the calling convention when dealing with tuples.

- At the lowest level, the primitive function is not aware of the tuple, and we simply flatten
everything including inputs and outputs when there is a tuple input or return values.
- The function that calls into the primitive function is aware of tuples.

This creates tension between at which point we should run the fusion algorithm. Originally,
both Tuple and TupleGetItem are opaque, which means we should not fuse them. Then throughout
the course, we started to fuse tuple-related node because they are useful as **intermediate
values**, and can be optimized away in primitive operations. 

However, tuple itself is bad as return values and can cause a bunch of problems for low-level
code gen. In particular, we can get functions as follows(note the duplication in return values)

```
fn (%data: Tensor[(1, 32, 32, 3), float32]) -> (Tensor[(1, 32, 32, 3), float32], Tensor[(1,
32, 32, 3), float32]) {
  %3 = fn (%p0: Tensor[(1, 32, 32, 3), float32], __dict__=meta[StrMap][0]) -> (Tensor[(1,
32, 32, 3), float32], Tensor[(1, 32, 32, 3), float32]) {
    %0 = log(%p0)
    %1 = (%0, %0)
    %1
  }
  %3 = %2(%data)
  %3
}
```

Ideally, what we want is to fuse TupleNode to follow up consumer nodes if they are intermediate
nodes, but not do so when they are the return values. So we won't suffer from this problem(as
TupleNode itself cannot be the final master node). 

Let us use this thread to do some consolidated discussion on the related topic 
 


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/3039
Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message