hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Illecker <millec...@apache.org>
Subject Re: Combine all Writables into a new package
Date Mon, 21 Oct 2013 16:54:56 GMT
+1 for hama-io or hama-commons

But we should discuss the parts that can be combined in a common module.

I would suggest the following:

*hama-commons module*

-- io --

CombineFileInputFormat.java

CombineFileSplit.java

FileInputFormat.java

FileOutputFormat.java

FileSplit.java

HashPartitioner.java

InputFormat.java

InputSplit.java

LineRecordReader.java

Messagable.java

MonitorManager.java

NullInputFormat.java

NullOutputFormat.java

OutputCollector.java

OutputFormat.java

Partitioner.java

PartitioningRunner.java

RecordReader.java

RecordWriter.java

SequenceFileInputFormat.java

SequenceFileOutputFormat.java

SequenceFileRecordReader.java

SequenceFileRecordWriter.java

TrackedRecordReader.java

TextInputFormat.java

TextOutputFormat.java


-- io.writable --

MatrixWritable.java

TextArrayWritable.java

VectorWritable.java


-- math --

CrossEntropy.java

DenseDoubleMatrix.java

DenseDoubleVector.java

DoubleDoubleFunction.java

DoubleDoubleVectorFunction.java

DoubleFunction.java

DoubleMatrix.java

DoubleVector.java

DoubleVectorFunction.java

Function.java

FunctionFactory.java

IdentityFunction.java

Sigmoid.java

SquaredError.java

Tanh.java

Tuple.java


*hama-core module submitted by Suraj*


-- bsp --

BSP.java

BSPInterface.java

BSPJob.java

BSPJobClient.java

BSPJobContext.java

BSPJobID.java

BSPPeer.java

Counters.java

Combiner.java

ID.java

LocalBSPRunner.java


-- bps.master --

BSPMaster.java


-- bsp.peer.ipc --


-- bsp.peer.ipc.message --

BSPMessageBundle.java


-- bsp.peer --

BSPPeerImpl.java


-- bsp.task --

BSPTask.java

BSPTaskRunner.java

Task.java

TaskAttemptContext.java

TaskAttemptID.java

TaskCompletionEvent.java

TaskID.java

TaskInProgress.java

TaskLog.java

TaskLogAppender.java

TaskLogServlet.java

TaskRunner.java

TaskStatus.java


-- bsp.ipc --

Directive.java

DirectiveException.java

DirectiveHandler.java

DispatchTasksDirective.java


-- bsp.ipc.action --

CommitTaskAction.java

GroomServerAction.java

JobChangeEvent.java

KillJobAction.java

KillTaskAction.java

LaunchTaskAction.java

RecoverTaskAction.java

ReinitGroomAction.java

ReportGroomStatusDirective.java

UpdatePeerAction.java

ClusterStatus.java


-- bsp.groom --

GroomServer.java

GroomServerManager.java

GroomServerStatus.java


-- bsp.job --

JobInProgress.java

JobInProgressListener.java

JobProfile.java

JobStatus.java

RunningJob.java


-- bsp.scheduler --

Schedulable.java

SimpleTaskScheduler.java

TaskScheduler.java

FCFSQueue.java

Queue.java

QueueManager.java


-- bsp.superstep --

Superstep.java

SuperstepBSP.java


Thanks!


2013/10/21 Chia-Hung Lin <clin4j@googlemail.com>

> +1 for hama-io or hama-commons
>
>
> On 21 October 2013 21:35, Tommaso Teofili <tommaso.teofili@gmail.com>
> wrote:
> > what about creating a module for that (Writables and InputFormats for
> now)
> > hama-io / hama-commons that can be used by both (containing math stuff as
> > well) ?
> >
> > Tommaso
> >
> >
> > 2013/10/21 Martin Illecker <millecker@apache.org>
> >
> >> VectorWritable and MatrixWritable have both some dependencies
> >> to org.apache.hama.ml.math.
> >> (DenseDoubleVector, DoubleVector and DenseDoubleMatrix, DoubleMatrix)
> >>
> >> If we move VectorWritable and MatrixWritable to core (e.g.,
> >> org.apache.hama.io.writable), we have to move org.apache.hama.ml.math as
> >> well.
> >> I think that's not possible because of other classes in hama-ml
> depending
> >> on ml.math.
> >>
> >> Temporary I will have to copy VectorWritable to the core to use it in a
> >> test case.
> >>
> >>
> >> 2013/10/21 Tommaso Teofili <tommaso.teofili@gmail.com>
> >>
> >> > 2013/10/21 Martin Illecker <millecker@apache.org>
> >> >
> >> > > Hello,
> >> > >
> >> > > regarding to my Hama Pipes test case [1], I want to use
> VectorWritable
> >> > > inside the hama-core module.
> >> > > Therefore I would need a dependency to hama-ml but this will cause
a
> >> > cyclic
> >> > > dependency.
> >> > >
> >> > > So is it possible to move both writables, VectorWritable and
> >> > > MatrixWritable, from org.apache.hama.ml.writable into a new package?
> >> > > e.g., org.apache.hama.io.writable based on [2]
> >> > >
> >> >
> >> > I think this really makes sense.
> >> >
> >> >
> >> > >
> >> > > Regarding to [3] we can also move TextArrayWritable
> >> > > from org.apache.hama.bsp into this new package.
> >> > >
> >> > > Do you think we can move the writables of
> org.apache.hama.ml.writable
> >> to
> >> > > the core module?
> >> > >
> >> >
> >> > +1
> >> >
> >> >
> >> > > And can we do the package refactoring [2] of org.apache.hama.bsp
> >> > submitted
> >> > > by Suraj?
> >> > >
> >> >
> >> > +1 here too.
> >> >
> >> > Tommaso
> >> >
> >> >
> >> > >
> >> > > Thanks!
> >> > >
> >> > > Martin
> >> > >
> >> > > [1] https://issues.apache.org/jira/browse/HAMA-808
> >> > > [2]
> >> > https://issues.apache.org/jira/secure/attachment/12609417/bsplist.txt
> >> > > [3] https://issues.apache.org/jira/browse/HAMA-727
> >> > >
> >> >
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message