From dev-return-3469-archive-asf-public=cust-asf.ponee.io@madlib.apache.org Sat Jul 7 00:54:29 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 3A895180627 for ; Sat, 7 Jul 2018 00:54:29 +0200 (CEST) Received: (qmail 91942 invoked by uid 500); 6 Jul 2018 22:54:28 -0000 Mailing-List: contact dev-help@madlib.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@madlib.apache.org Delivered-To: mailing list dev@madlib.apache.org Received: (qmail 91929 invoked by uid 99); 6 Jul 2018 22:54:27 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Jul 2018 22:54:27 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 942DFE0BC7; Fri, 6 Jul 2018 22:54:27 +0000 (UTC) From: hpandeycodeit To: dev@madlib.apache.org Reply-To: dev@madlib.apache.org References: In-Reply-To: Subject: [GitHub] madlib pull request #288: Jira:1239: Converts features from multiple columns... Content-Type: text/plain Message-Id: <20180706225427.942DFE0BC7@git1-us-west.apache.org> Date: Fri, 6 Jul 2018 22:54:27 +0000 (UTC) Github user hpandeycodeit commented on a diff in the pull request: https://github.com/apache/madlib/pull/288#discussion_r200788091 --- Diff: src/ports/postgres/modules/cols_vec/cols2vec.py_in --- @@ -0,0 +1,110 @@ +""" +@file cols2vec.py_in + +@brief Utility to convert Columns to array + +""" + +import plpy +from utilities.control import MinWarning +from utilities.utilities import split_quoted_delimited_str +from utilities.utilities import _string_to_array +from utilities.validate_args import columns_exist_in_table +from utilities.validate_args import is_var_valid +from utilities.validate_args import get_cols +from utilities.validate_args import quote_ident +from utilities.utilities import py_list_to_sql_string + + +m4_changequote(`') + + +def validate_cols2vec_args(source_table, output_table, + list_of_features, list_of_features_to_exclude, cols_to_output, **kwargs): + """ + Function to validate input parameters + """ + if list_of_features.strip() != '*': + if not (list_of_features and list_of_features.strip()): + plpy.error("Features to include is empty") + + if list_of_features.strip() != '*': + if not columns_exist_in_table( + source_table, split_quoted_delimited_str(list_of_features)): + plpy.error( + "Invalid columns to list_of_features ({0})".format(list_of_features)) + + if cols_to_output and cols_to_output.strip() != '*': + if not columns_exist_in_table( + source_table, _string_to_array(cols_to_output)): + plpy.error("Invalid columns to output list ({0})". + format(cols_to_output)) + + +def cols2vec(schema_madlib, source_table, output_table, list_of_features, + list_of_features_to_exclude=None, cols_to_output=None, **kwargs): + """ + Args: + @param schema_madlib: Name of MADlib schema + @param model: Name of table containing the tree model + @param source_table: Name of table containing prediction data + @param output_table: Name of table to output the results + @param list_of_features: Comma-separated string of column names or + expressions to put into feature array. + Can also be a '*' implying all columns + are to be put into feature array. + @param list_of_features_to_exclude: Comma-separated string of column names + to exclude from the feature array + @param cols_to_output: Comma-separated string of column names + from the source table to keep in the output table, + in addition to the feature array. + + Returns: + None + + """ + + with MinWarning('warning'): + validate_cols2vec_args(source_table, output_table, list_of_features, + list_of_features_to_exclude, cols_to_output, **kwargs) + + all_cols = '' + feature_cols = '' + if list_of_features.strip() == '*': + all_cols = get_cols(source_table, schema_madlib) + all_col_set = set(list(all_cols)) + exclude_set = set(split_quoted_delimited_str( + list_of_features_to_exclude)) + feature_set = all_col_set - exclude_set + feature_cols = py_list_to_sql_string( + list(feature_set), "text", False) + filtered_list_of_features = ",".join( + feat for feat in list(feature_set)) + else: + feature_list = split_quoted_delimited_str(list_of_features) + feature_exclude = split_quoted_delimited_str( + list_of_features_to_exclude) + return_set = set(feature_list) - set(feature_exclude) --- End diff -- Updated the above code as well along Also the order of features will remain same. ---