beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephen Sisk (JIRA)" <>
Subject [jira] [Created] (BEAM-1644) IO ITs: shared directory for kubernetes resources?
Date Tue, 07 Mar 2017 22:04:38 GMT
Stephen Sisk created BEAM-1644:

             Summary: IO ITs: shared directory for kubernetes resources?
                 Key: BEAM-1644
             Project: Beam
          Issue Type: Bug
          Components: sdk-java-extensions
            Reporter: Stephen Sisk
            Assignee: Davor Bonaci

It is the case that different IOs will be created that connect to the same data stores - HadoopInputFormat
in particular uses ES and cassandra, which are also used in their respective IOs as well.
Jdbc is likely to have the same type of overlap.

It would be nice to share TestPipelineOption and kubernetes/docker scripts so that we don't
need to repeat them in each module. 

As such, we should put them in a shared location. It seems like the best bet is to create
a new module for test resources. 

* Create a new module under io, test-common (? I'm not in love with that name) 
* All the IOs with ITs can put their TestPipelineOption classes there
* it will have a kubernetes directory with the various kubernetes scripts in sub directories

Further thoughts:
If 2 IOs need different configurations of a data store, this would make that harder. In that
case, there's nothing stopping us from having two different sets of the kubernetes scripts
in the shared location

This message was sent by Atlassian JIRA

View raw message