streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Blackmon (JIRA)" <>
Subject [jira] [Created] (STREAMS-207) Customize metadata fields written/read to/from HDFS/S3
Date Mon, 03 Nov 2014 17:06:34 GMT
Steve Blackmon created STREAMS-207:

             Summary: Customize metadata fields written/read to/from HDFS/S3
                 Key: STREAMS-207
             Project: Streams
          Issue Type: Improvement
            Reporter: Steve Blackmon

These persisters currently expect a 4-column format.  It should be possible via configuration
to omit one or more of the four principle fields (id, timestamp, metadata, document) and have
the reader and writer operations succeed.  The default behavior should remain write and read
of all four fields.

This will allow streams to be constructed from files containing only ids, only documents,
or only metadata without processor-based workarounds.

This message was sent by Atlassian JIRA

View raw message