Mailing-List: contact dev-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@mahout.apache.org
Received-SPF: pass (nike.apache.org: domain of guxiaobo1982@gmail.com
 designates 209.85.160.170 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CACOCgcm9EPBvsDwUhX4n8yg4n=cVCFtGQ8vOhT9JRpay2ck4hw@mail.gmail.com>
References: 
 <CACOCgcnUDt6c=mR60Fj7BusZvgNVMTVtAFh10dJT3G_=8yXmFQ@mail.gmail.com>
	<CADqrX8KnrDRT1eSd=Wuf_SN0jMr8XNvGgZMBU34wZXY4czo=TQ@mail.gmail.com>
	<CACOCgcn3x6UtMsqmh=AOML2OP7ig2jyZFckxKn=HUFsJKj7+SQ@mail.gmail.com>
	<CADqrX8K4_7x1B9ZxbbQfmeZGaWnZBuSQbZ9nM2a2j=ANHSrhsw@mail.gmail.com>
	<CACOCgc=W9bKvHkM29Fom+M8ceH52qnU0v=FGD4wphvbD-nWjLQ@mail.gmail.com>
	<CACOCgcm9EPBvsDwUhX4n8yg4n=cVCFtGQ8vOhT9JRpay2ck4hw@mail.gmail.com>
Date: Fri, 15 Jul 2011 22:55:58 +0800
Message-ID: 
 <CACOCgcnubT5w4nvw09ob6L5a6qurbQ1JyQJjBOBvqArTbrFJ7w@mail.gmail.com>
Subject: Re: File format question about Random forest.
From: Xiaobo Gu <guxiaobo1982@gmail.com>
To: dev@mahout.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Do the -p and -f option of org.apache.mahout.df.tools.Describe have to
be HDFS URLs, can they be local file system paths?


On Fri, Jul 15, 2011 at 9:28 PM, Xiaobo Gu <guxiaobo1982@gmail.com> wrote:
> Can we make the file descriptor as following:
>
> 1. make a small csv file with the same format as the actual dataset,
> say a CSV file with header and only one record,
> 2. Use java weka.core.converters.CSVLoader filename.csv >
> filename.arff =A0to convert the small CSV into a ARFF file, see
> http://maya.cs.depaul.edu/classes/ect584/weka/preprocess.html
> 3. Use org.apache.mahout.df.tools.Describe =A0to generate a descriptor
>
>
> The only consern here is: does the small CSV file with one record
> sufficient enough to generate the ARFF file header, or do we have to
> use the whole file to avoid losing information?
>
>
> Xiaobo Gu
>
>
>
>
> On Fri, Jul 15, 2011 at 9:10 PM, Xiaobo Gu <guxiaobo1982@gmail.com> wrote=
:
>> But if we use CSV files, how can we generate descriptors for datasets?
>>
>> Cheers
>>
>> Xiaobo Gu
>>
>> On Thu, Jul 14, 2011 at 1:27 AM, deneche abdelhakim <adeneche@gmail.com>=
 wrote:
>>> I guess yes. as long as you don't use quotes or double quotes to embed =
the
>>> fields.
>>>
>>> On Wed, Jul 13, 2011 at 2:58 PM, Xiaobo Gu <guxiaobo1982@gmail.com> wro=
te:
>>>
>>>> So for simple datasets, which only have numeric and character
>>>> lable(without blank) category columns, =A0can we just use CSV tools to
>>>> save it as a standard CSV file without header?
>>>>
>>>>
>>>> On Wed, Jul 13, 2011 at 3:53 AM, deneche abdelhakim <adeneche@gmail.co=
m>
>>>> wrote:
>>>> > the current implementation doesn't support the ARFF format
>>>> out-of-the-box,
>>>> > as described in the Wiki you need to remove the header of the file a=
nd
>>>> leave
>>>> > only the data. Actually, this implementation is fully compatible wit=
h
>>>> UCI's
>>>> > datasets which are comma separated text files. You'll also need to c=
all
>>>> the
>>>> > dataset description tool (see the wiki) in order to generate a prope=
r
>>>> > description file (contains the nature of each attribute: Numerical o=
r
>>>> > Categorical).
>>>> >
>>>> > Yes you can use BuildForest and TestForest to generate and use Rando=
m
>>>> forest
>>>> > models from the command line
>>>> >
>>>> > On Tue, Jul 12, 2011 at 2:19 PM, Xiaobo Gu <guxiaobo1982@gmail.com>
>>>> wrote:
>>>> >
>>>> >> Hi,
>>>> >>
>>>> >> The Random Forest partial implementation in
>>>> >>
>>>> https://cwiki.apache.org/confluence/display/MAHOUT/Partial+Implementat=
ion
>>>> >> use the ARFF file format, is ARFF the only supportted file format w=
hen
>>>> >> using the BuildForest and TestForest program, and are BuildForest a=
nd
>>>> >> TestForest program are official tools to build Random Forest models
>>>> >> from the command line?
>>>> >>
>>>> >> Regards,
>>>> >>
>>>> >> Xiaobo Gu
>>>> >>
>>>> >
>>>>
>>>
>>
>