Return-Path: X-Original-To: apmail-flink-dev-archive@www.apache.org Delivered-To: apmail-flink-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B689917DBC for ; Wed, 18 Mar 2015 15:08:13 +0000 (UTC) Received: (qmail 53944 invoked by uid 500); 18 Mar 2015 15:08:01 -0000 Delivered-To: apmail-flink-dev-archive@flink.apache.org Received: (qmail 53846 invoked by uid 500); 18 Mar 2015 15:08:00 -0000 Mailing-List: contact dev-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list dev@flink.apache.org Received: (qmail 53823 invoked by uid 99); 18 Mar 2015 15:08:00 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Mar 2015 15:08:00 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,T_RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 18 Mar 2015 15:07:59 +0000 Received: (qmail 52413 invoked by uid 99); 18 Mar 2015 15:07:39 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Mar 2015 15:07:39 +0000 Date: Wed, 18 Mar 2015 15:07:39 +0000 (UTC) From: "Till Rohrmann (JIRA)" To: dev@flink.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (FLINK-1736) Add CountVectorizer to machine learning library MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org Till Rohrmann created FLINK-1736: ------------------------------------ Summary: Add CountVectorizer to machine learning library Key: FLINK-1736 URL: https://issues.apache.org/jira/browse/FLINK-1736 Project: Flink Issue Type: Improvement Components: Machine Learning Library Reporter: Till Rohrmann A {{CountVectorizer}} feature extractor [1] assigns each occurring word in a corpus an unique identifier. With this mapping it can vectorize models such as bag of words or ngrams in a efficient way. The unique identifier assigned to a word acts as the index of a vector. The number of word occurrences is represented as a vector value at a specific index. The advantage of the {{CountVectorizer}} compared to the FeatureHasher is that the mapping of words to indices can be obtained which makes it easier to understand the resulting feature vectors. The {{CountVectorizer}} could be generalized to support arbitrary feature values. The {{CountVectorizer}} should be implemented as a {{Transfomer}}. Resources: [1] [http://scikit-learn.org/stable/modules/feature_extraction.html#common-vectorizer-usage] -- This message was sent by Atlassian JIRA (v6.3.4#6332)