falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suhas Vasu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-145) Feed eviction be implemented in appropriate Storage implementation
Date Wed, 01 Oct 2014 10:20:34 GMT

    [ https://issues.apache.org/jira/browse/FALCON-145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154633#comment-14154633

Suhas Vasu commented on FALCON-145:

The file EvictionHelper was deleted as part of FALCON-731, it is being reintroduced here.
That shouldn't be the case.

Also in FALCON-731, we have
-        EvictionHelper.logInstancePaths(path.getFileSystem(getConf()), path, instancePaths.toString());
+        EvictedInstanceSerDe.serializeEvictedInstancePaths(
+                path.getFileSystem(getConf()), path, instancePaths);

but here it is 
+            EvictionHelper.logInstancePaths(logFilePath.getFileSystem(new Configuration()),
+                    instancePaths.toString());
+            EvictedInstanceSerDe.serializeEvictedInstancePaths(
+                    logFilePath.getFileSystem(new Configuration()), logFilePath, instancePaths);

This shouldn't be the case

We should ideally be deleting all references to EvictionHelper. 
Would suggest to give a fresh patch from the trunk.

> Feed eviction be implemented in appropriate Storage implementation
> ------------------------------------------------------------------
>                 Key: FALCON-145
>                 URL: https://issues.apache.org/jira/browse/FALCON-145
>             Project: Falcon
>          Issue Type: Improvement
>            Reporter: Venkatesh Seetharam
>            Assignee: Ajay Yadav
>         Attachments: FALCON-145-v2.patch, FALCON-145-v3.patch, FALCON-145-v4.patch, falcon-145.patch
> Since the feed storage is abstracted in Storage class either as FileSystemStorage or
CatalogStorage, moreover, behaviors for listing partitions and drop partitions are listed
there, why do we need to hardcode the eviction behavior and instance deletion discovery for
filesystem need to happen in FeedEvictor ? Why can't be implemented in appropriate Storage
implementation. That way FeedEvictor would simpler and lot cleaner. 
> This can apply to table replication as well for import and export of partitions.

This message was sent by Atlassian JIRA

View raw message