ambari-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From oleew...@apache.org
Subject ambari git commit: AMBARI-21155. Design Ambari Infra Component & workflows (oleewere)
Date Wed, 31 May 2017 18:07:33 GMT
Repository: ambari
Updated Branches:
  refs/heads/trunk 753f8aacc -> 23e23afd8


AMBARI-21155. Design Ambari Infra Component & workflows (oleewere)


Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/23e23afd
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/23e23afd
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/23e23afd

Branch: refs/heads/trunk
Commit: 23e23afd884449812d05c490804dc4845b01ac21
Parents: 753f8aa
Author: oleewere <oleewere@gmail.com>
Authored: Wed May 31 12:21:34 2017 +0200
Committer: oleewere <oleewere@gmail.com>
Committed: Wed May 31 20:03:27 2017 +0200

----------------------------------------------------------------------
 ambari-infra/ambari-infra-manager/README.md     |  92 ++-
 .../ambari-infra-manager/docs/api/swagger.yaml  | 784 +++++++++++++++++++
 .../docs/images/batch-1.png                     | Bin 0 -> 20521 bytes
 .../docs/images/batch-2.png                     | Bin 0 -> 29388 bytes
 .../docs/images/batch-3.png                     | Bin 0 -> 14105 bytes
 .../docs/images/batch-4.png                     | Bin 0 -> 23277 bytes
 .../infra/common/InfraManagerConstants.java     |   2 +-
 .../infra/conf/InfraManagerApiDocConfig.java    |  35 +-
 .../conf/batch/InfraManagerBatchConfig.java     |   8 +-
 .../ambari/infra/job/dummy/DummyItemWriter.java |  13 +
 .../infra/job/dummy/DummyJobListener.java       |  39 +
 .../infra/job/dummy/DummyStepListener.java      |  41 +
 .../apache/ambari/infra/rest/JobResource.java   |   2 +-
 13 files changed, 1002 insertions(+), 14 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/README.md
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/README.md b/ambari-infra/ambari-infra-manager/README.md
index d3527c4..dd2854c 100644
--- a/ambari-infra/ambari-infra-manager/README.md
+++ b/ambari-infra/ambari-infra-manager/README.md
@@ -18,13 +18,99 @@ limitations under the License.
 -->
 
 # Ambari Infra Manager
-TODO
-## Build & Run Application
+
+## Overview
+
+Ambari Infra Manager is a REST based management application for Ambari Infra services (like
Infra Solr). The API is built on top of [Spring Batch] (http://docs.spring.io/spring-batch/reference/html/)
+
+### Architecture
+![batch-1](docs/images/batch-1.png)
+
+### Job execution overview
+![batch-2](docs/images/batch-2.png)
+
+### Job workflow
+![batch-3](docs/images/batch-3.png)
+
+### Step workflow
+![batch-4](docs/images/batch-4.png)
+
+(images originally from [here] (http://docs.spring.io/spring-batch/reference/html/))
+
+## API documentation
+
+Infra Manager uses [Swagger] (http://swagger.io/), generated yaml file can be downloaded
from [here] (docs/api/swagger.yaml)
+
+
+## Development guide
+
+### Adding a new custom job
+
+As Infra Manager is a Spring based application and using Java configurations, if it is needed
to add a new custom Job, the Jobs/Steps/Configurations are need to be on the classpath. Spring
beans are registered only in a specific package, so for writing a plugin, all the added Java
classes needs to be added inside "org.apache.ambari.infra" package.
+
+For the plugin it will be needed to add all Spring & Spring batch dependencies. For adding
a new Job you will need to define a new Configuration object. There you can define your own
jobs/steps/writers/readers/processors, as you can see in that example:
+```java
+@Configuration
+@EnableBatchProcessing
+public class MyJobConfig {
+
+  @Inject
+  private StepBuilderFactory steps;
+
+  @Inject
+  private JobBuilderFactory jobs;
+  
+  
+  @Bean(name = "dummyStep")
+  protected Step dummyStep(ItemReader<DummyObject> reader,
+                         ItemProcessor<DummyObject, String> processor,
+                         ItemWriter<String> writer) {
+    return steps.get("dummyStep").listener(new DummyStepListener()).<DummyObject, String>
chunk(2)
+      .reader(reader).processor(processor).writer(writer).build();
+  }
+  
+  @Bean(name = "dummyJob")
+  public Job job(@Qualifier("dummyStep") Step dummyStep) {
+    return jobs.get("dummyJob").listener(new DummyJobListener()).start(dummyStep).build();
+  }
+
+}
+```
+As you can see it will require to implement [ItemWriter] (https://docs.spring.io/spring-batch/apidocs/org/springframework/batch/item/ItemWriter.html),
[ItemReader] (http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/item/ItemReader.html)
and [ItemProcessor] (https://docs.spring.io/spring-batch/apidocs/org/springframework/batch/item/ItemProcessor.html)
+
+### Schedule custom jobs
+
+It can be needed based on business requirements to schedule jobs (e.g. daily) instead of
run manually through the REST API. It can be done with adding a custom bean to "org.apache.ambari.infra"
package with using [@Scheduled] (http://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/annotation/Scheduled.html):
+```java
+@Named
+public class MySchedulerObject {
+
+   @Inject
+   private JobService jobService; // or JobOperator jobOperator if spring-batch-admin manager
dependecy is not included
+   
+   @Value("${infra-manager.batch.my.param:defaultString}")
+   private String myParamFromLogSearchProperties;
+   
+   @Scheduled(cron = "*/5 * * * * MON-FRI")
+   public void doSomething() {
+      // setup job params
+      jobService.launch(jobName, jobParameters, TimeZone.getDefault());
+   }
+   
+   @Scheduled(cron = "${infra.manager.my.prop}")
+   public void doSomethingBasedOnInfraProperty() {
+      // do something ...
+   }
+}
+```
+
+You can put your cron expression inside infra-manager.properties file just make it configuratble.
+### Build & Run Application
 ```bash
 mvn clean package exec:java
 ```
 
-## Build & Run Application in docker container
+### Build & Run Application in docker container
 ```bash
 cd docker
 ./infra-manager-docker.sh

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/docs/api/swagger.yaml
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/docs/api/swagger.yaml b/ambari-infra/ambari-infra-manager/docs/api/swagger.yaml
new file mode 100644
index 0000000..824629f
--- /dev/null
+++ b/ambari-infra/ambari-infra-manager/docs/api/swagger.yaml
@@ -0,0 +1,784 @@
+---
+swagger: "2.0"
+info:
+  description: "Manager component for Ambari Infra"
+  version: "1.0.0"
+  title: "Infra Manager REST API"
+  license:
+    name: "Apache 2.0"
+    url: "http://www.apache.org/licenses/LICENSE-2.0.html"
+basePath: "/api/v1"
+tags:
+- name: "jobs"
+schemes:
+- "http"
+- "https"
+paths:
+  /jobs:
+    get:
+      tags:
+      - "jobs"
+      summary: "Get all jobs"
+      description: ""
+      operationId: "getAllJobs"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "page"
+        in: "query"
+        required: false
+        type: "integer"
+        default: 0
+        format: "int32"
+      - name: "size"
+        in: "query"
+        required: false
+        type: "integer"
+        default: 20
+        format: "int32"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            type: "array"
+            items:
+              $ref: "#/definitions/JobInfo"
+  /jobs/executions:
+    delete:
+      tags:
+      - "jobs"
+      summary: "Stop all job executions."
+      description: ""
+      operationId: "stopAll"
+      produces:
+      - "application/json"
+      parameters: []
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            type: "integer"
+            format: "int32"
+  /jobs/executions/{jobExecutionId}:
+    get:
+      tags:
+      - "jobs"
+      summary: "Get job and step details for job execution instance."
+      description: ""
+      operationId: "getExectionInfo"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "jobExecutionId"
+        in: "path"
+        required: true
+        type: "integer"
+        format: "int64"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            $ref: "#/definitions/JobExecutionDetailsResponse"
+    delete:
+      tags:
+      - "jobs"
+      summary: "Stop or abandon a running job execution."
+      description: ""
+      operationId: "stopOrAbandonJobExecution"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "jobExecutionId"
+        in: "path"
+        required: true
+        type: "integer"
+        format: "int64"
+      - name: "operation"
+        in: "query"
+        required: true
+        type: "string"
+        enum:
+        - "STOP"
+        - "ABANDON"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            $ref: "#/definitions/JobExecutionInfoResponse"
+  /jobs/executions/{jobExecutionId}/context:
+    get:
+      tags:
+      - "jobs"
+      summary: "Get execution context for specific job."
+      description: ""
+      operationId: "getExecutionContextByJobExecId"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "jobExecutionId"
+        in: "path"
+        required: true
+        type: "integer"
+        format: "int64"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            $ref: "#/definitions/ExecutionContextResponse"
+  /jobs/executions/{jobExecutionId}/steps/{stepExecutionId}:
+    get:
+      tags:
+      - "jobs"
+      summary: "Get step execution details."
+      description: ""
+      operationId: "getStepExecution"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "jobExecutionId"
+        in: "path"
+        required: true
+        type: "integer"
+        format: "int64"
+      - name: "stepExecutionId"
+        in: "path"
+        required: true
+        type: "integer"
+        format: "int64"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            $ref: "#/definitions/StepExecutionInfoResponse"
+  /jobs/executions/{jobExecutionId}/steps/{stepExecutionId}/execution-context:
+    get:
+      tags:
+      - "jobs"
+      summary: "Get the execution context of step execution."
+      description: ""
+      operationId: "getStepExecutionContext"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "jobExecutionId"
+        in: "path"
+        required: true
+        type: "integer"
+        format: "int64"
+      - name: "stepExecutionId"
+        in: "path"
+        required: true
+        type: "integer"
+        format: "int64"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            $ref: "#/definitions/StepExecutionContextResponse"
+  /jobs/executions/{jobExecutionId}/steps/{stepExecutionId}/progress:
+    get:
+      tags:
+      - "jobs"
+      summary: "Get progress of step execution."
+      description: ""
+      operationId: "getStepExecutionProgress"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "jobExecutionId"
+        in: "path"
+        required: true
+        type: "integer"
+        format: "int64"
+      - name: "stepExecutionId"
+        in: "path"
+        required: true
+        type: "integer"
+        format: "int64"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            $ref: "#/definitions/StepExecutionProgressResponse"
+  /jobs/info/names:
+    get:
+      tags:
+      - "jobs"
+      summary: "Get all job names"
+      description: ""
+      operationId: "getAllJobNames"
+      produces:
+      - "application/json"
+      parameters: []
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            type: "array"
+            uniqueItems: true
+            items:
+              type: "string"
+  /jobs/{jobName}:
+    post:
+      tags:
+      - "jobs"
+      summary: "Start a new job instance by job name."
+      description: ""
+      operationId: "startJob"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "jobName"
+        in: "path"
+        required: true
+        type: "string"
+      - name: "params"
+        in: "query"
+        required: false
+        type: "string"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            $ref: "#/definitions/JobExecutionInfoResponse"
+  /jobs/{jobName}/executions:
+    get:
+      tags:
+      - "jobs"
+      summary: "Get the id values of all the running job instances."
+      description: ""
+      operationId: "getExecutionIdsByJobName"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "jobName"
+        in: "path"
+        required: true
+        type: "string"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            type: "array"
+            uniqueItems: true
+            items:
+              type: "integer"
+              format: "int64"
+  /jobs/{jobName}/info:
+    get:
+      tags:
+      - "jobs"
+      summary: "Get job details by job name."
+      description: ""
+      operationId: "getJobDetails"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "page"
+        in: "query"
+        required: false
+        type: "integer"
+        default: 0
+        format: "int32"
+      - name: "size"
+        in: "query"
+        required: false
+        type: "integer"
+        default: 20
+        format: "int32"
+      - name: "jobName"
+        in: "path"
+        required: true
+        type: "string"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            $ref: "#/definitions/JobDetailsResponse"
+  /jobs/{jobName}/{jobInstanceId}/executions:
+    get:
+      tags:
+      - "jobs"
+      summary: "Get execution for job instance."
+      description: ""
+      operationId: "getExecutionsForInstance"
+      produces:
+      - "application/json"
+      parameters:
+      - name: "jobName"
+        in: "path"
+        required: true
+        type: "string"
+      - name: "jobInstanceId"
+        in: "path"
+        required: true
+        type: "integer"
+        format: "int64"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            type: "array"
+            items:
+              $ref: "#/definitions/JobExecutionInfoResponse"
+    post:
+      tags:
+      - "jobs"
+      summary: "Restart job instance."
+      description: ""
+      operationId: "restartJobInstance"
+      produces:
+      - "application/json"
+      parameters:
+      - in: "body"
+        name: "body"
+        required: false
+        schema:
+          $ref: "#/definitions/JobExecutionRestartRequest"
+      responses:
+        200:
+          description: "successful operation"
+          schema:
+            $ref: "#/definitions/JobExecutionInfoResponse"
+definitions:
+  JobExecutionData:
+    type: "object"
+    properties:
+      id:
+        type: "integer"
+        format: "int64"
+      executionContext:
+        $ref: "#/definitions/ExecutionContext"
+      jobInstance:
+        $ref: "#/definitions/JobInstance"
+      jobId:
+        type: "integer"
+        format: "int64"
+      jobParameters:
+        $ref: "#/definitions/JobParameters"
+      failureExceptions:
+        type: "array"
+        items:
+          $ref: "#/definitions/Throwable"
+      endTime:
+        type: "string"
+        format: "date-time"
+      exitStatus:
+        $ref: "#/definitions/ExitStatus"
+      createTime:
+        type: "string"
+        format: "date-time"
+      lastUpdated:
+        type: "string"
+        format: "date-time"
+      jobConfigurationName:
+        type: "string"
+      startTime:
+        type: "string"
+        format: "date-time"
+      status:
+        type: "string"
+        enum:
+        - "COMPLETED"
+        - "STARTING"
+        - "STARTED"
+        - "STOPPING"
+        - "STOPPED"
+        - "FAILED"
+        - "ABANDONED"
+        - "UNKNOWN"
+      stepExecutionDataList:
+        type: "array"
+        items:
+          $ref: "#/definitions/StepExecutionData"
+  JobInstance:
+    type: "object"
+    properties:
+      id:
+        type: "integer"
+        format: "int64"
+      version:
+        type: "integer"
+        format: "int32"
+      jobName:
+        type: "string"
+      instanceId:
+        type: "integer"
+        format: "int64"
+  StepExecutionData:
+    type: "object"
+    properties:
+      id:
+        type: "integer"
+        format: "int64"
+      jobExecutionId:
+        type: "integer"
+        format: "int64"
+      executionContext:
+        $ref: "#/definitions/ExecutionContext"
+      stepName:
+        type: "string"
+      terminateOnly:
+        type: "boolean"
+        default: false
+      failureExceptions:
+        type: "array"
+        items:
+          $ref: "#/definitions/Throwable"
+      endTime:
+        type: "string"
+        format: "date-time"
+      exitStatus:
+        $ref: "#/definitions/ExitStatus"
+      lastUpdated:
+        type: "string"
+        format: "date-time"
+      commitCount:
+        type: "integer"
+        format: "int32"
+      readCount:
+        type: "integer"
+        format: "int32"
+      filterCount:
+        type: "integer"
+        format: "int32"
+      writeCount:
+        type: "integer"
+        format: "int32"
+      readSkipCount:
+        type: "integer"
+        format: "int32"
+      writeSkipCount:
+        type: "integer"
+        format: "int32"
+      processSkipCount:
+        type: "integer"
+        format: "int32"
+      rollbackCount:
+        type: "integer"
+        format: "int32"
+      startTime:
+        type: "string"
+        format: "date-time"
+      status:
+        type: "string"
+        enum:
+        - "COMPLETED"
+        - "STARTING"
+        - "STARTED"
+        - "STOPPING"
+        - "STOPPED"
+        - "FAILED"
+        - "ABANDONED"
+        - "UNKNOWN"
+  StackTraceElement:
+    type: "object"
+    properties:
+      methodName:
+        type: "string"
+      fileName:
+        type: "string"
+      lineNumber:
+        type: "integer"
+        format: "int32"
+      className:
+        type: "string"
+      nativeMethod:
+        type: "boolean"
+        default: false
+  JobExecutionDetailsResponse:
+    type: "object"
+    properties:
+      jobExecutionInfoResponse:
+        $ref: "#/definitions/JobExecutionInfoResponse"
+      stepExecutionInfoList:
+        type: "array"
+        items:
+          $ref: "#/definitions/StepExecutionInfoResponse"
+  StepExecutionContextResponse:
+    type: "object"
+    properties:
+      executionContextMap:
+        type: "object"
+        additionalProperties:
+          type: "object"
+      jobExecutionId:
+        type: "integer"
+        format: "int64"
+      stepExecutionId:
+        type: "integer"
+        format: "int64"
+      stepName:
+        type: "string"
+  StepExecutionProgress:
+    type: "object"
+    properties:
+      estimatedPercentCompleteMessage:
+        $ref: "#/definitions/MessageSourceResolvable"
+      estimatedPercentComplete:
+        type: "number"
+        format: "double"
+  ExitStatus:
+    type: "object"
+    properties:
+      exitCode:
+        type: "string"
+      exitDescription:
+        type: "string"
+      running:
+        type: "boolean"
+        default: false
+  ExecutionContextResponse:
+    type: "object"
+    properties:
+      jobExecutionId:
+        type: "integer"
+        format: "int64"
+      executionContextMap:
+        type: "object"
+        additionalProperties:
+          type: "object"
+  StepExecutionHistory:
+    type: "object"
+    properties:
+      stepName:
+        type: "string"
+      count:
+        type: "integer"
+        format: "int32"
+      commitCount:
+        $ref: "#/definitions/CumulativeHistory"
+      rollbackCount:
+        $ref: "#/definitions/CumulativeHistory"
+      readCount:
+        $ref: "#/definitions/CumulativeHistory"
+      writeCount:
+        $ref: "#/definitions/CumulativeHistory"
+      filterCount:
+        $ref: "#/definitions/CumulativeHistory"
+      readSkipCount:
+        $ref: "#/definitions/CumulativeHistory"
+      writeSkipCount:
+        $ref: "#/definitions/CumulativeHistory"
+      processSkipCount:
+        $ref: "#/definitions/CumulativeHistory"
+      duration:
+        $ref: "#/definitions/CumulativeHistory"
+      durationPerRead:
+        $ref: "#/definitions/CumulativeHistory"
+  TimeZone:
+    type: "object"
+    properties:
+      displayName:
+        type: "string"
+      id:
+        type: "string"
+      dstsavings:
+        type: "integer"
+        format: "int32"
+      rawOffset:
+        type: "integer"
+        format: "int32"
+  MessageSourceResolvable:
+    type: "object"
+    properties:
+      arguments:
+        type: "array"
+        items:
+          type: "object"
+      codes:
+        type: "array"
+        items:
+          type: "string"
+      defaultMessage:
+        type: "string"
+  ExecutionContext:
+    type: "object"
+    properties:
+      dirty:
+        type: "boolean"
+        default: false
+      empty:
+        type: "boolean"
+        default: false
+  StepExecutionInfoResponse:
+    type: "object"
+    properties:
+      id:
+        type: "integer"
+        format: "int64"
+      jobExecutionId:
+        type: "integer"
+        format: "int64"
+      jobName:
+        type: "string"
+      name:
+        type: "string"
+      startDate:
+        type: "string"
+      startTime:
+        type: "string"
+      duration:
+        type: "string"
+      durationMillis:
+        type: "integer"
+        format: "int64"
+      exitCode:
+        type: "string"
+      status:
+        type: "string"
+  JobExecutionInfoResponse:
+    type: "object"
+    properties:
+      id:
+        type: "integer"
+        format: "int64"
+      stepExecutionCount:
+        type: "integer"
+        format: "int32"
+      jobId:
+        type: "integer"
+        format: "int64"
+      jobName:
+        type: "string"
+      startDate:
+        type: "string"
+      startTime:
+        type: "string"
+      duration:
+        type: "string"
+      jobExecutionData:
+        $ref: "#/definitions/JobExecutionData"
+      jobParameters:
+        type: "object"
+        additionalProperties:
+          type: "object"
+      jobParametersString:
+        type: "string"
+      restartable:
+        type: "boolean"
+        default: false
+      abandonable:
+        type: "boolean"
+        default: false
+      stoppable:
+        type: "boolean"
+        default: false
+      timeZone:
+        $ref: "#/definitions/TimeZone"
+  JobInfo:
+    type: "object"
+    properties:
+      name:
+        type: "string"
+      executionCount:
+        type: "integer"
+        format: "int32"
+      launchable:
+        type: "boolean"
+        default: false
+      incrementable:
+        type: "boolean"
+        default: false
+      jobInstanceId:
+        type: "integer"
+        format: "int64"
+  JobExecutionRestartRequest:
+    type: "object"
+    properties:
+      jobName:
+        type: "string"
+      jobInstanceId:
+        type: "integer"
+        format: "int64"
+      operation:
+        type: "string"
+        enum:
+        - "RESTART"
+  Throwable:
+    type: "object"
+    properties:
+      cause:
+        $ref: "#/definitions/Throwable"
+      stackTrace:
+        type: "array"
+        items:
+          $ref: "#/definitions/StackTraceElement"
+      message:
+        type: "string"
+      localizedMessage:
+        type: "string"
+      suppressed:
+        type: "array"
+        items:
+          $ref: "#/definitions/Throwable"
+  JobParameters:
+    type: "object"
+    properties:
+      parameters:
+        type: "object"
+        additionalProperties:
+          $ref: "#/definitions/JobParameter"
+      empty:
+        type: "boolean"
+        default: false
+  CumulativeHistory:
+    type: "object"
+    properties:
+      count:
+        type: "integer"
+        format: "int32"
+      min:
+        type: "number"
+        format: "double"
+      max:
+        type: "number"
+        format: "double"
+      standardDeviation:
+        type: "number"
+        format: "double"
+      mean:
+        type: "number"
+        format: "double"
+  JobInstanceDetailsResponse:
+    type: "object"
+    properties:
+      jobInstance:
+        $ref: "#/definitions/JobInstance"
+      jobExecutionInfoResponseList:
+        type: "array"
+        items:
+          $ref: "#/definitions/JobExecutionInfoResponse"
+  JobParameter:
+    type: "object"
+    properties:
+      identifying:
+        type: "boolean"
+        default: false
+      value:
+        type: "object"
+      type:
+        type: "string"
+        enum:
+        - "STRING"
+        - "DATE"
+        - "LONG"
+        - "DOUBLE"
+  StepExecutionProgressResponse:
+    type: "object"
+    properties:
+      stepExecutionProgress:
+        $ref: "#/definitions/StepExecutionProgress"
+      stepExecutionHistory:
+        $ref: "#/definitions/StepExecutionHistory"
+      stepExecutionInfoResponse:
+        $ref: "#/definitions/StepExecutionInfoResponse"
+  JobDetailsResponse:
+    type: "object"
+    properties:
+      jobInfo:
+        $ref: "#/definitions/JobInfo"
+      jobInstanceDetailsResponseList:
+        type: "array"
+        items:
+          $ref: "#/definitions/JobInstanceDetailsResponse"

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/docs/images/batch-1.png
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/docs/images/batch-1.png b/ambari-infra/ambari-infra-manager/docs/images/batch-1.png
new file mode 100644
index 0000000..d763852
Binary files /dev/null and b/ambari-infra/ambari-infra-manager/docs/images/batch-1.png differ

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/docs/images/batch-2.png
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/docs/images/batch-2.png b/ambari-infra/ambari-infra-manager/docs/images/batch-2.png
new file mode 100644
index 0000000..1de3479
Binary files /dev/null and b/ambari-infra/ambari-infra-manager/docs/images/batch-2.png differ

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/docs/images/batch-3.png
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/docs/images/batch-3.png b/ambari-infra/ambari-infra-manager/docs/images/batch-3.png
new file mode 100644
index 0000000..7f1123c
Binary files /dev/null and b/ambari-infra/ambari-infra-manager/docs/images/batch-3.png differ

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/docs/images/batch-4.png
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/docs/images/batch-4.png b/ambari-infra/ambari-infra-manager/docs/images/batch-4.png
new file mode 100644
index 0000000..beb610a
Binary files /dev/null and b/ambari-infra/ambari-infra-manager/docs/images/batch-4.png differ

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java
b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java
index 77f7008..105f20e 100644
--- a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/common/InfraManagerConstants.java
@@ -25,7 +25,7 @@ public final class InfraManagerConstants {
   public static final String PROTOCOL_SSL = "https";
   public static final String ROOT_CONTEXT = "/";
   public static final String WEB_RESOURCE_FOLDER = "webapp";
-  public static final String DEFAULT_DATA_FOLDER_LOCATION = "/usr/ambari-infra-manager/data";
+  public static final String DEFAULT_DATA_FOLDER_LOCATION = "/opt/ambari-infra-manager/data";
   public static final String DATA_FOLDER_LOCATION_PARAM = "dataFolderLocation";
   public static final Integer SESSION_TIMEOUT = 60 * 30;
 }

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerApiDocConfig.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerApiDocConfig.java
b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerApiDocConfig.java
index 22e2263..4c76742 100644
--- a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerApiDocConfig.java
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/InfraManagerApiDocConfig.java
@@ -21,12 +21,22 @@ package org.apache.ambari.infra.conf;
 import io.swagger.jaxrs.config.BeanConfig;
 import io.swagger.jaxrs.listing.ApiListingResource;
 import io.swagger.jaxrs.listing.SwaggerSerializers;
+import io.swagger.models.Info;
+import io.swagger.models.License;
 import org.springframework.context.annotation.Bean;
 import org.springframework.context.annotation.Configuration;
 
 @Configuration
 public class InfraManagerApiDocConfig {
 
+  private static final String DESCRIPTION = "Manager component for Ambari Infra";
+  private static final String VERSION = "1.0.0";
+  private static final String TITLE = "Infra Manager REST API";
+  private static final String LICENSE = "Apache 2.0";
+  private static final String LICENSE_URL = "http://www.apache.org/licenses/LICENSE-2.0.html";
+  private static final String RESOURCE_PACKAGE = "org.apache.ambari.infra.rest";
+  private static final String BASE_PATH = "/api/v1";
+
   @Bean
   public ApiListingResource apiListingResource() {
     return new ApiListingResource();
@@ -41,14 +51,25 @@ public class InfraManagerApiDocConfig {
   public BeanConfig swaggerConfig() {
     BeanConfig beanConfig = new BeanConfig();
     beanConfig.setSchemes(new String[]{"http", "https"});
-    beanConfig.setBasePath("/api/v1");
-    beanConfig.setTitle("Infra Manager REST API");
-    beanConfig.setDescription("Manager component for Ambari Infra");
-    beanConfig.setLicense("Apache 2.0");
-    beanConfig.setLicenseUrl("http://www.apache.org/licenses/LICENSE-2.0.html");
+    beanConfig.setBasePath(BASE_PATH);
+    beanConfig.setTitle(TITLE);
+    beanConfig.setDescription(DESCRIPTION);
+    beanConfig.setLicense(LICENSE);
+    beanConfig.setLicenseUrl(LICENSE_URL);
     beanConfig.setScan(true);
-    beanConfig.setVersion("1.0.0");
-    beanConfig.setResourcePackage("org.apache.ambari.infra.rest");
+    beanConfig.setVersion(VERSION);
+    beanConfig.setResourcePackage(RESOURCE_PACKAGE);
+
+    License license = new License();
+    license.setName(LICENSE);
+    license.setUrl(LICENSE_URL);
+
+    Info info = new Info();
+    info.setDescription(DESCRIPTION);
+    info.setTitle(TITLE);
+    info.setVersion(VERSION);
+    info.setLicense(license);
+    beanConfig.setInfo(info);
     return beanConfig;
   }
 }

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java
b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java
index c3d8db6..95f87f5 100644
--- a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/conf/batch/InfraManagerBatchConfig.java
@@ -20,7 +20,9 @@ package org.apache.ambari.infra.conf.batch;
 
 import org.apache.ambari.infra.job.dummy.DummyItemProcessor;
 import org.apache.ambari.infra.job.dummy.DummyItemWriter;
+import org.apache.ambari.infra.job.dummy.DummyJobListener;
 import org.apache.ambari.infra.job.dummy.DummyObject;
+import org.apache.ambari.infra.job.dummy.DummyStepListener;
 import org.springframework.batch.admin.service.JdbcSearchableJobExecutionDao;
 import org.springframework.batch.admin.service.JdbcSearchableJobInstanceDao;
 import org.springframework.batch.admin.service.JdbcSearchableStepExecutionDao;
@@ -68,6 +70,7 @@ import org.springframework.jdbc.core.JdbcTemplate;
 import org.springframework.jdbc.datasource.DriverManagerDataSource;
 import org.springframework.jdbc.datasource.init.DataSourceInitializer;
 import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator;
+import org.springframework.scheduling.annotation.EnableAsync;
 import org.springframework.scheduling.annotation.EnableScheduling;
 import org.springframework.transaction.PlatformTransactionManager;
 
@@ -78,6 +81,7 @@ import java.net.MalformedURLException;
 @Configuration
 @EnableBatchProcessing
 @EnableScheduling
+@EnableAsync
 public class InfraManagerBatchConfig {
 
   @Value("classpath:org/springframework/batch/core/schema-drop-sqlite.sql")
@@ -225,13 +229,13 @@ public class InfraManagerBatchConfig {
   protected Step dummyStep(ItemReader<DummyObject> reader,
                        ItemProcessor<DummyObject, String> processor,
                        ItemWriter<String> writer) {
-    return steps.get("dummyStep").<DummyObject, String> chunk(2)
+    return steps.get("dummyStep").listener(new DummyStepListener()).<DummyObject, String>
chunk(2)
       .reader(reader).processor(processor).writer(writer).build();
   }
 
   @Bean(name = "dummyJob")
   public Job job(@Qualifier("dummyStep") Step dummyStep) {
-    return jobs.get("dummyJob").start(dummyStep).build();
+    return jobs.get("dummyJob").listener(new DummyJobListener()).start(dummyStep).build();
   }
 
   @Bean

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java
b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java
index f495795..9a78706 100644
--- a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyItemWriter.java
@@ -22,8 +22,15 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.batch.item.ItemWriter;
 
+import java.io.File;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Date;
 import java.util.List;
 
+import static org.apache.ambari.infra.common.InfraManagerConstants.DATA_FOLDER_LOCATION_PARAM;
+
 public class DummyItemWriter implements ItemWriter<String> {
 
   private static final Logger LOG = LoggerFactory.getLogger(DummyItemWriter.class);
@@ -32,5 +39,11 @@ public class DummyItemWriter implements ItemWriter<String> {
   public void write(List<? extends String> values) throws Exception {
     LOG.info("DummyItem writer called (values: {})... wait 1 seconds", values.toString());
     Thread.sleep(1000);
+    String outputDirectoryLocation = String.format("%s%s%s%s", System.getProperty(DATA_FOLDER_LOCATION_PARAM),
File.separator, "dummyOutput-", new Date().getTime());
+    Path pathToDirectory = Paths.get(outputDirectoryLocation);
+    Path pathToFile = Paths.get(String.format("%s%s%s", outputDirectoryLocation, File.separator,
"dummyOutput.txt"));
+    Files.createDirectories(pathToDirectory);
+    LOG.info("Write to file: ", pathToFile.getFileName().toAbsolutePath());
+    Files.write(pathToFile, values.toString().getBytes());
   }
 }

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyJobListener.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyJobListener.java
b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyJobListener.java
new file mode 100644
index 0000000..0bbfb55
--- /dev/null
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyJobListener.java
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.ambari.infra.job.dummy;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.batch.core.JobExecution;
+import org.springframework.batch.core.JobExecutionListener;
+
+public class DummyJobListener implements JobExecutionListener {
+
+  private static final Logger LOG = LoggerFactory.getLogger(DummyJobListener.class);
+
+  @Override
+  public void beforeJob(JobExecution jobExecution) {
+    LOG.info("Dummy - before job execution");
+  }
+
+  @Override
+  public void afterJob(JobExecution jobExecution) {
+    LOG.info("Dummy - after job execution");
+  }
+}

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyStepListener.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyStepListener.java
b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyStepListener.java
new file mode 100644
index 0000000..548e650
--- /dev/null
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/job/dummy/DummyStepListener.java
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.ambari.infra.job.dummy;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.batch.core.ExitStatus;
+import org.springframework.batch.core.StepExecution;
+import org.springframework.batch.core.StepExecutionListener;
+
+public class DummyStepListener implements StepExecutionListener {
+
+  private static final Logger LOG = LoggerFactory.getLogger(DummyStepListener.class);
+
+  @Override
+  public void beforeStep(StepExecution stepExecution) {
+    LOG.info("Dummy step - before step execution");
+  }
+
+  @Override
+  public ExitStatus afterStep(StepExecution stepExecution) {
+    LOG.info("Dummy step - after step execution");
+    return stepExecution.getExitStatus();
+  }
+}

http://git-wip-us.apache.org/repos/asf/ambari/blob/23e23afd/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/JobResource.java
----------------------------------------------------------------------
diff --git a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/JobResource.java
b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/JobResource.java
index 7023957..0e20b54 100644
--- a/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/JobResource.java
+++ b/ambari-infra/ambari-infra-manager/src/main/java/org/apache/ambari/infra/rest/JobResource.java
@@ -98,7 +98,7 @@ public class JobResource {
 
   @GET
   @Produces({"application/json"})
-  @Path("/info/{jobName}")
+  @Path("{jobName}/info")
   @ApiOperation("Get job details by job name.")
   public JobDetailsResponse getJobDetails(@BeanParam @Valid JobRequest jobRequest) throws
NoSuchJobException {
     return jobManager.getJobDetails(jobRequest.getJobName(), jobRequest.getPage(), jobRequest.getSize());


Mime
View raw message