tvm-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-tvm] siju-samuel edited a comment on pull request #5507: [CRT]fix to reduce RAM size during loading model
Date Thu, 07 May 2020 22:18:44 GMT

siju-samuel edited a comment on pull request #5507:
URL: https://github.com/apache/incubator-tvm/pull/5507#issuecomment-625525365


   @tqchen. Thanks for the review.
   I work in a device with very limited ram availability. Actually even a few kbs is crucial
for me.
   
   Im using a quantized model and the pool is 80% unit8 data. So i dont want to allocate all
pool with float and 32bit. If i consider the bit i can restrict all pool to allocate 32bit.
   
   So while allocating space, to compute `size` we either need `bits` or `dytpe`. Now both
are not available in `TVMGraphRuntimePoolEntry`. So i added bits in `TVMGraphRuntimePoolEntry`
   
   As you suggested, we can compute `size` while finding maximum space needed and save to
 `pool_entry[sid].size` 
   But it requires either 
   1. while making TVMNDArray_Empty, always use `dtype.code = kDLInt` & `dtype.bits =
8`.
   
   2. while making TVMNDArray_Empty, use Float & 32bit (like before this change) and reduce
the `size` as per `bits` and align to 4bytes.
   
   I would like to go with 2nd approach. I havnet done the modification yet. I will do the
modifications, test and will update this PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message