From user-return-567-archive-asf-public=cust-asf.ponee.io@madlib.apache.org Fri Jul 27 16:41:49 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 7D238180657 for ; Fri, 27 Jul 2018 16:41:48 +0200 (CEST) Received: (qmail 52439 invoked by uid 500); 27 Jul 2018 14:41:47 -0000 Mailing-List: contact user-help@madlib.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@madlib.apache.org Delivered-To: mailing list user@madlib.apache.org Received: (qmail 52429 invoked by uid 99); 27 Jul 2018 14:41:47 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Jul 2018 14:41:47 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id EE78218071F for ; Fri, 27 Jul 2018 14:41:46 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.075 X-Spam-Level: *** X-Spam-Status: No, score=3.075 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FORGED_HOTMAIL_RCVD2=1.187, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=hotmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id e_PKNVEInqXd for ; Fri, 27 Jul 2018 14:41:46 +0000 (UTC) Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-oln040092001091.outbound.protection.outlook.com [40.92.1.91]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 91C2B5F2F0 for ; Fri, 27 Jul 2018 14:41:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8GL7jk9efxn7Vzmar37W8ZbNjj4mcarJWnwdJud6hPM=; b=aWRRiZlm4lI1pMnWUQq5T+T/hH8iL4FfBS1ax5iFZy+UU8cb3hQasCdCchco9z7931TYFtBViRJ8fk+skQsXb7aOg43/kaDbvXyBhKYdW51q0bfmarbKP89RZngzhDnQAh1XmixRaNg2SllD40NT/Na5be5RiWBboxnFTBDDb6CnLvWz48Zu6FTY42XXPEK4Stffp7SSCJmnJ9KFsVtD00PcdysNwHBmbP7ZdxRSGxa+M3Wm9T/v1M5dsFvL8F5R9SyXBSo/Bt2PoPPB4RomLbuSXd8bRmbAz0mMfgJHR/OhUZkg7PvjuyliwkeNQCYcR+ihkPH+nrZHerzUzIffCQ== Received: from BY2NAM01FT058.eop-nam01.prod.protection.outlook.com (10.152.68.55) by BY2NAM01HT209.eop-nam01.prod.protection.outlook.com (10.152.69.125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.20.995.12; Fri, 27 Jul 2018 14:41:38 +0000 Received: from DM5PR12MB1801.namprd12.prod.outlook.com (10.152.68.60) by BY2NAM01FT058.mail.protection.outlook.com (10.152.68.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.995.12 via Frontend Transport; Fri, 27 Jul 2018 14:41:38 +0000 Received: from DM5PR12MB1801.namprd12.prod.outlook.com ([fe80::874:480f:2189:8199]) by DM5PR12MB1801.namprd12.prod.outlook.com ([fe80::874:480f:2189:8199%9]) with mapi id 15.20.0995.019; Fri, 27 Jul 2018 14:41:38 +0000 From: LUYAO CHEN To: "user@madlib.apache.org" Subject: Out of memory for neural network Thread-Topic: Out of memory for neural network Thread-Index: AQHUJbc9HmFp6rTjzUu0dZXca+Nvvw== Date: Fri, 27 Jul 2018 14:41:38 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-incomingtopheadermarker: OriginalChecksum:F88A71ED94EBA08DA27144F73270581056640EB749DCC7BCEA27D545D0A771DE;UpperCasedChecksum:32E4D11FEC01AA19DF575BF15687670A64248D39D1FA5EFA0B1ECFD07505A166;SizeAsReceived:6875;Count:44 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [kHsAouE7bccVDgjIiw5RqS6qO37yPcjzZZnnBgqvwFc=] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;BY2NAM01HT209;6:74S+hS8040SwR1foLpW0pu5Mes24FRdgxYdg+uFMGYcFH3mV5dqwLvfE9qxEtE3vrpWpB/8m/yPCXV5hqBjGjl3bHG9dL6izRlvrE7y5n1lxz2TBs/J0/thmBEP49m73+y9q950RlPWQq9V+4kAMuOFfrWvlwb0Qdo0046UDYNW70NfHQQpvXikDDzBeWZ1H+/luoIABxEAISTQKfvRdSv5/kMxaUQNhvMAh2asBmp2mJSMJm3RlKPb7qZ+oOQAxelyflTb+xlEhEhQ2PXqSicxVaQIUvH/5eIJVOuGuDjxORlG8970BOulCiZlos9HehjaMoJZ8+05hPFQdliDR/JbGG+rke+rUFzbNCgc+8Ncxv4HCbwsljupXokL+e8zTnDhhHRu9WIw+ndRjPTsl6fJdlLp9Pv4yuf27eJ5/CX4QrXd7XKxRyMSdT6gyXOIhMv7R4wRAoa1C0vVjLSebtw==;5:CnJk6K8zOazgFK8JQLlCj/h22LKPVzjafBFre6hNAdFlysts7lDHrtUgUojsj5BYGGmPXLt8ouRimu4VQ9KYcyogWfjMXmJ+5G+Zkglw0GbFwS1CVdDteUOQhEfrOdrLQ/IEVwv2oI3rS9HwxgfQtKLq1pPfZqjSjQfsoVuN8OQ=;7:ivEoswmgrEoaX1rRYIUDrF2cUYgfsc5AMAnlUveDsQdErl8zilXWaQgm9NUcKrCB17uODSajC1S+3HQksKTfeO17O1JcFqfSjqtBsGp0MlDZk5htMyCsOzU0VLwGepHv/eH5Wc38oaJSk8zqFV6k1ch1b0/uygoGx0afG/8U+7BcZRfx8iD6eeIi6vz8tQpDJnG915UlH/CYJQrKmWlkEHS7Xnt5SiZ7mOv0iCSheWSnVHCfDAovQ0/BfTxAizWE x-incomingheadercount: 44 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(7020095)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1603101448)(1601125500)(1701031045);SRVR:BY2NAM01HT209; x-ms-traffictypediagnostic: BY2NAM01HT209: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(82015058);SRVR:BY2NAM01HT209;BCL:0;PCL:0;RULEID:;SRVR:BY2NAM01HT209; x-forefront-prvs: 07467C4D33 x-forefront-antispam-report: SFV:NSPM;SFS:(7070007)(199004)(189003)(56003)(6916009)(86362001)(68736007)(83332001)(2501003)(14444005)(105586002)(106356001)(6606003)(99286004)(256004)(14454004)(413944005)(6436002)(73972006)(2351001)(5250100002)(81156014)(8676002)(8936002)(1730700003)(74316002)(20460500001)(82202002)(5640700003)(53346004)(7696005)(104016004)(55016002)(102836004)(476003)(97736004)(9686003)(54896002)(53336002)(19627405001)(5660300001)(25786009)(6346003)(2900100001)(486006)(26005)(6506007)(33656002)(87572001)(400700002)(27310500001)(15852004);DIR:OUT;SFP:1901;SCL:1;SRVR:BY2NAM01HT209;H:DM5PR12MB1801.namprd12.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: hotmail.com does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=luyao_chen@hotmail.com; x-microsoft-antispam-message-info: d1JH5t5pW4pJWpzHwixpINlCy5zB0mgndRxKsK44CvMx7osheyLcvMUBaIknwc77J42BLaTHoupXA5/qlP4xUsMFchrPAO4H4m7dmHc3yVPuJnJXVdDtXrNg1W0X0uBQmX2IEiqpkaD4KRbLLXWmrYeE0+ASYj536IpSxIKbgarsvxMqRsCzYHIjjmmARB0uzPRkxYbKhoJKlFfglcQG/fpMRP7n4lqOSNd82J959tQ= Content-Type: multipart/alternative; boundary="_000_DM5PR12MB1801181941C80040E6FDDC5FFE2A0DM5PR12MB1801namp_" MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 9a4e3081-9524-43cf-bfc3-dcaef82d5da1 X-MS-Exchange-CrossTenant-Network-Message-Id: 83ee25e6-5c25-429b-9c85-08d5f3cf0f4e X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 9a4e3081-9524-43cf-bfc3-dcaef82d5da1 X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Jul 2018 14:41:38.2758 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2NAM01HT209 --_000_DM5PR12MB1801181941C80040E6FDDC5FFE2A0DM5PR12MB1801namp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Dear user community, I would report a problem regarding in neural network. I am using a 16G RAM = machine. After some iterations (~100 ). I got the below error ERROR: spiexceptions.OutOfMemory: out of memory DETAIL: Failed on request of size 32800. CONTEXT: Traceback (most recent call last): PL/Python function "mlp_classification", line 36, in grouping_col PL/Python function "mlp_classification", line 45, in wrapper PL/Python function "mlp_classification", line 325, in mlp PL/Python function "mlp_classification", line 580, in update PL/Python function "mlp_classification" Below is the command , SELECT madlib.mlp_classification( 'train_data_sub', -- Source table 'mlp_model', -- Destination table 'features', -- Input features 'positive', -- Label ARRAY[5], -- Number of units per layer 'learning_rate_init=3D0.003, n_iterations=3D500, tolerance=3D0', -- Optimizer params 'tanh', -- Activation function NULL, -- Default weight (1) FALSE, -- No warm start true, -- verbose 'case_icd' -- Grouping ); Is that a problem or just caused by the data size? Regards, Luyao Chen --_000_DM5PR12MB1801181941C80040E6FDDC5FFE2A0DM5PR12MB1801namp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Dear user community,


I would report a problem regardin= g in neural network. I am using a 16G RAM machine. <= /p>


After some iterations (~100= ). I got the below error 


ERROR:  spiexceptions.OutOfMemory: out of memory
DETAIL:  Failed on request of size 32800.
CONTEXT:  Traceback (most recent call last):
  PL/Python function "mlp_classification", line 36, in = <module>
    grouping_col
  PL/Python function "mlp_classification", line 45, in = wrapper
  PL/Python function "mlp_classification", line 325, in= mlp
  PL/Python function "mlp_classification", line 580, in= update
PL/Python function "mlp_classification"

Below is the command ,

SELECT madlib.mlp_classification(
    'train_data_sub',      -- Source table&nb= sp;
    'mlp_model',      -- Destination table
    'features',     -- Input features
    'positive',     -- Label
    ARRAY[5],         -- Number of = units per layer
    'learning_rate_init=3D0.003,
    n_iterations=3D500,
    tolerance=3D0',     -- Optimizer params
    'tanh',           -- Activ= ation function
    NULL,             -- = Default weight (1)
    FALSE,            -- No wa= rm start
    true,             -- = verbose
    'case_icd'         -- Grouping<= /div>
);

Is that a problem or just caused by the data size? 

Regards,
Luyao Chen

--_000_DM5PR12MB1801181941C80040E6FDDC5FFE2A0DM5PR12MB1801namp_--