From dev-return-5602-archive-asf-public=cust-asf.ponee.io@singa.apache.org Tue Jun 2 05:45:07 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id EA38918064C for ; Tue, 2 Jun 2020 07:45:06 +0200 (CEST) Received: (qmail 58322 invoked by uid 500); 2 Jun 2020 05:45:06 -0000 Mailing-List: contact dev-help@singa.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@singa.apache.org Delivered-To: mailing list dev@singa.apache.org Received: (qmail 58312 invoked by uid 99); 2 Jun 2020 05:45:06 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Jun 2020 05:45:06 +0000 From: =?utf-8?q?GitBox?= To: dev@singa.apache.org Subject: =?utf-8?q?=5BGitHub=5D_=5Bsinga=5D_XJDKC_edited_a_comment_on_pull_request_?= =?utf-8?q?=23697=3A_New_Model_Layer_Operator_API?= Message-ID: <159107670584.10999.97102649900539270.asfpy@gitbox.apache.org> Date: Tue, 02 Jun 2020 05:45:05 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit In-Reply-To: References: XJDKC edited a comment on pull request #697: URL: https://github.com/apache/singa/pull/697#issuecomment-637290809 > > > > Summary: > > > > ``` > > > > * set hx.creator and cx.creator to None. (Still can't use the graph to train correctly but can be executed normally) > > > > > > > > * create ReLU layer instance > > > > ``` > > > > > > > > > update conv and linear layers to include an argument for `activation` > > > > ``` > > > > * create Loss layer instance > > > > > > > > * remove set_attribute function, just copy the initial value from tensor directly. Raise warning in __setattr__ when the types do not match > > > > ``` > > > > > > > > > Shall we totally disable reassignment, like self.W=..., because it may affect the graph as it replace the tensor W with another tensor? > > > > ``` > > > > * remove on_device function, get device info from input tensors > > > > ``` > > > > > > But I'm not sure where the reassignment is used in the whole project. Maybe it's used in many places. I think reassignment is still very common. > > I see. > Will reassignment have any side effect to the computational graph? Reassignment will not be buffered in the graph. If we want to update a tensor in the graph, we may need to copy(will create an operator) the new value to the original tensor. Just like we use Axpy to update parameters. Another strategy is to swap pointers of the two blocks. ```python a = Tensor() # block1->ptr1->addr1 b = Tensor() # block2->ptr2->addr2 a = b # inc ref_count of block2, copy the data in block2 to block1. ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org