Mailing-List: contact user-help@mahout.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@mahout.apache.org
Received-SPF: pass (nike.apache.org: domain of charly.lizarralde@gmail.com
 designates 74.125.82.170 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <SNT106-W55423AD247A27F5A432A2392620@phx.gbl>
References: <SNT106-W55423AD247A27F5A432A2392620@phx.gbl>
From: Charly Lizarralde <charly.lizarralde@gmail.com>
Date: Tue, 30 Oct 2012 18:07:58 -0300
Message-ID: 
 <CABfgGt9bg74mEKNLTQK=A1-uM5ypac3u77DqriiC9tZa7G4+pg@mail.gmail.com>
Subject: Re: Converting one large text file with multiple documents to
 SequenceFile format
To: user@mahout.apache.org
Content-Type: multipart/alternative; boundary=f46d044282406c1ba004cd4d2f34

--f46d044282406c1ba004cd4d2f34
Content-Type: text/plain; charset=ISO-8859-1

I had the exact same issue and I tried to use the seqdirectory command with
a different filter class but It did not work. It seems there's a bug in the
mahout-0.6 code.

It ended up as writing a custom map-reduce program that performs just that.

Greetiings!
Charly

On Tue, Oct 30, 2012 at 5:00 PM, Nick Woodward <levar1@hotmail.com> wrote:

>
> I have done a lot of searching on the web for this, but I've found
> nothing, even though I feel like it has to be somewhat common. I have used
> Mahout's 'seqdirectory' command to convert a folder containing text files
> (each file is a separate document) in the past. But in this case there are
> so many documents (in the 100,000s) that I have one very large text file in
> which each line is a document. How can I convert this large file to
> SequenceFile format so that Mahout understands that each line should be
> considered a separate document?  Would it be better if the file was
> structured like so....docId1 {tab} document textdocId2 {tab} document
> textdocId3 {tab} document text...
>
> Thank you very much for any help.Nick
>

--f46d044282406c1ba004cd4d2f34--