batchee-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Romain Manni-Bucau <rmannibu...@gmail.com>
Subject Re: shipping a new release?
Date Tue, 17 Mar 2015 21:42:17 GMT
that's even worse. Let me try to be more explicit.

You need to manage:
- checkpoint (file + line number I think but if your chunk is really the
folder then it can be something else)
- avoid concurrent reading. Common way is to add foo.txt.inprogress while
reading (or in another folder) and move foo.txt in done/ once done

Last point: this has no link with jsefa IMO but could be an extras
component (FolderReader) delegating to another reader.




Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber
<http://www.tomitribe.com>

2015-03-17 20:56 GMT+01:00 Mark Struberg <struberg@yahoo.de>:

> I guess you mean the checkpoint we manage? Currently this is the line
> number, right?
>
> LieGrue,
> strub
>
>
>
>
>
> On Tuesday, 17 March 2015, 18:55, Romain Manni-Bucau <
> rmannibucau@gmail.com> wrote:
>
>
> >
> >
> >+1 but take care that's harder than a listFiles(), you need a inprogress
> and done markers
> >
> >
> >
> >Romain Manni-Bucau
> >@rmannibucau |  Blog | Github | LinkedIn | Tomitriber
> >
> >2015-03-17 18:53 GMT+01:00 Reinhard Sandtner <reinhard.sandtner@gmail.com
> >:
> >
> >hey,
> >>
> >>it would be nice to have a jsefaReader which takes a list of files or a
> folder as parameter. currently you can only read a single file
> >>
> >>wdyt?
> >>
> >>lg
> >>reini
> >>
> >>> Am 17.03.2015 um 10:30 schrieb Romain Manni-Bucau <
> rmannibucau@gmail.com>:
> >>>
> >>> no time for it myself but +1
> >>>
> >>>
> >>> Romain Manni-Bucau
> >>> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> >>> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> >>> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber
> >>> <http://www.tomitribe.com>
> >>
> >>>
> >>> 2015-03-17 10:27 GMT+01:00 Mark Struberg <struberg@yahoo.de>:
> >>>
> >>>> Hi folks!
> >>>>
> >>>> What do you think about shipping a new release?
> >>>> We've fixed quite a few bugs.
> >>>>
> >>>> LieGrue,
> >>>> strub
> >>>>
> >>
> >>
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message