From dev-return-2472-archive-asf-public=cust-asf.ponee.io@systemml.apache.org Thu May 10 23:44:21 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id E0DB018063A for ; Thu, 10 May 2018 23:44:20 +0200 (CEST) Received: (qmail 59747 invoked by uid 500); 10 May 2018 21:44:20 -0000 Mailing-List: contact dev-help@systemml.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@systemml.apache.org Delivered-To: mailing list dev@systemml.apache.org Received: (qmail 59731 invoked by uid 99); 10 May 2018 21:44:19 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 May 2018 21:44:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id E644FC00E1 for ; Thu, 10 May 2018 21:44:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.898 X-Spam-Level: * X-Spam-Status: No, score=1.898 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 7mPU4GaXI-wO for ; Thu, 10 May 2018 21:44:16 +0000 (UTC) Received: from mail-vk0-f48.google.com (mail-vk0-f48.google.com [209.85.213.48]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id C4C455F24C for ; Thu, 10 May 2018 21:44:15 +0000 (UTC) Received: by mail-vk0-f48.google.com with SMTP id e67-v6so1811615vke.7 for ; Thu, 10 May 2018 14:44:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Vn+NYzq9LNO5HNBgRB7P4o2ss1Nzy7OBNCK5hPaSxBs=; b=l5scFkV+d5XZ+zDyAuRH8FYX63ypuJlVdXCRyHJxlPyzBA6WSrFeyL3gz+wm+3wp5l Dcpu9RbqGeeLeqsAFWhX5pzkl/OoXwgzhkhRePQREm+Li62EfjsEWE8AFEE/isMuFxxe bPrdkbGY5f+UD43lID++Rk+JXN9gwGRxTm4a8y6P2bOba5v/7ap9v3ca1rHphdpunWPo z2r/UwfIdkrA9R9lX2UcN5Cr5LT0j0R/jtVJR7rFDSh2MsZ/97Kg7a0vFfIp/dj/l3hi fSZUKlCoNnQRpPA4zaqOmQGPLvuBOuvjwxfSoztqLB7IjGAoxhoPnrytQQdb9fGbJ4k6 mV0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Vn+NYzq9LNO5HNBgRB7P4o2ss1Nzy7OBNCK5hPaSxBs=; b=HZQZxjsz+H0MgRZJLn3ObQEMPlV50rxthllIrmtP/AR/6s9Qf23VydaPiNe5Z8uK4d Lp5KuJxCYFUXxs9W6omUIq6vFe2uBfZOBQCWWY3JlcQOjgkNF7871LEgZxePh/p95LXR 3WhHhg4225tQSd3wyfw81R2bWfjp5VDxZSlWWtLEU5MJ+PPlU7rSqDImvos2YPLabsg1 Fuyy7VF0JFUFb/sxJnu9K4ciT/0hc4eoIMMX4ZyRY03uw/LlQQnvjMtAt84sdBo0MZ4K fINGSiSQfyeCs8n7J/wz4uDFt9eWJz0Xjv8mk8gCuTMEMc6ce+9za/c1F54zhrOq4oF1 suqQ== X-Gm-Message-State: ALKqPwd+Yx4H3w6rFyvrmEDJIKcOSQlDhzMvJZ+sTievJsq+JWZK2lGi grVbhKN18bimKn9Wb+5QibG39FJDY8KK4PT+Yvk= X-Google-Smtp-Source: AB8JxZo9WJYwF5K0wDup4d+Kz29bpTfAZVvIHrLrgPcsaN4LRB5oXSZ8zfyeKipbH0h+KEBgscKvt5wjjeSAgCyqTJU= X-Received: by 2002:a1f:fc82:: with SMTP id a124-v6mr2342688vki.43.1525988654449; Thu, 10 May 2018 14:44:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.176.85.93 with HTTP; Thu, 10 May 2018 14:44:14 -0700 (PDT) In-Reply-To: References: From: Guobao Li Date: Thu, 10 May 2018 23:44:14 +0200 Message-ID: Subject: Re: Questions about MNIST LeNet example To: dev@systemml.apache.org Cc: Matthias Boehm Content-Type: multipart/alternative; boundary="000000000000ae07b6056be0eb72" --000000000000ae07b6056be0eb72 Content-Type: text/plain; charset="UTF-8" Thanks Matthias! It will be great for passing the model to the paramserv function. Regards, Guobao 2018-05-10 21:47 GMT+02:00 Matthias Boehm : > just FYI: we now have support for list and named-list data types in > SystemML, which allow passing the entire model as a single handle. For > example, you can define the following > > l1 = list(W1, b1, W2, b2, W3, b3, W4, b4), or > l2 = list(a=W1, b=b1, c=W2, d=b2, e=W3, f=b3, g=W4, h=b4) > > and access entries via l1[7] or l2['g'] accordingly. We're still > working on additional features to make the integration with IPA, > functions, and size/type propagation smoother, but the basic > functionality is already available. > > Regards, > Matthias > > On Sun, May 6, 2018 at 1:08 PM, Matthias Boehm wrote: > > Hi Guobao, > > > > that sounds very good. In general, the "model" refers to the > > collection of all weights and bias matrices of a given architecture. > > Similar to a classic regression model, we can view the weights as the > > "slope", i.e., multiplicative terms, while the biases are the > > "intercept", i.e., additive terms that shift the layer output. Both > > are subject to training and thus part of the model. > > > > This implies that the number of matrices in the model depends on the > > architecture. Hence, we have two choices here: (a) allow for a > > variable number of inputs and outputs, or (b) create a struct-like > > data type that allows passing the collection of matrices via a single > > handle. We've discussed the second option in other contexts as well > > because this would also be useful for reducing the number of > > parameters passed through function calls. I'm happy to help out > > integrating these struct-like data types if needed. > > > > Great to see that you're in the process of updating the related JIRAs. > > Let us know whenever you think you're ready with an initial draft - > > then I'd make a detailed pass over it. > > > > Furthermore, I would recommend to experiment with running these > > existing mnist lenet examples (which is one of our baselines moving > > forward): > > * Download the "infinite MNIST" data generator > > (http://leon.bottou.org/projects/infimnist), and generate a moderately > > sized dataset (e.g., 256K instances). > > * Convert the input into SystemML's binary block format. The generator > > produces the data in libsvm format and we provide a data converter > > (see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our > > internal binary representation. > > * Run the basic mnist lenet example for a few epochs. > > * Install the native BLAS libraries mkl or openblas and try using it > > for the above example to ensure its setup and configured correctly. > > > > > > Regards, > > Matthias > > > > On Sun, May 6, 2018 at 3:24 AM, Guobao Li > wrote: > >> Hi Matthias, > >> > >> I'm currently reading the dml script MNIST LeNet example and got some > >> questions. I hope that you could help me out of them. > >> > >> 1) Is it possible to define a matrix containing the variables? Because > I'm > >> wondering how to represent the model as a parameter for the "paramserv" > >> function. > >> 2) What is the role of bias? Why we need it? > >> > >> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and > hope > >> to get some feedback. Thanks! > >> > >> Regards, > >> Guobao > >> > --000000000000ae07b6056be0eb72--