hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chandni Singh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8867) Retrieve the status of resource localization
Date Wed, 23 Jan 2019 20:46:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750413#comment-16750413
] 

Chandni Singh commented on YARN-8867:
-------------------------------------

Tested patch with the following spec:
{code}
{
  "version": "1.0.2",
  "components" :
  [
    {
      "name": "sleeper",
      "number_of_containers": 2,
      "launch_command": "sleep 8000",
      "resource": {
        "cpus": 1,
        "memory": "256"
      },
      "configuration" : {
         "files": [
          {
            "type": "STATIC",
            "src_file": "file1.txt",
            "dest_file": "changed_name.txt"
          }
        ]
       }
    }
  ]
}
{code}

The status of the service with {{PENDING}} localization status
{code}
{
    "components": [
        {
            "configuration": {
                "env": {},
                "files": [
                    {
                        "dest_file": "changed_name.txt",
                        "properties": {},
                        "src_file": "file1.txt",
                        "type": "STATIC"
                    }
                ],
                "properties": {}
            },
            "containers": [
                {
                    "bare_host": "10.22.8.153",
                    "component_instance_name": "sleeper-0",
                    "id": "container_1548273183277_0001_01_000002",
                    "launch_time": 1548275812327,
                    "localization_statuses": [
                        {
                            "dest_file": "changed_name.txt",
                            "state": "PENDING"
                        }
                    ],
                    "state": "RUNNING_BUT_UNREADY"
                }
            ],
            "decommissioned_instances": [],
            "dependencies": [],
            "launch_command": "sleep 8000",
            "name": "sleeper",
            "number_of_containers": 2,
            "quicklinks": [],
            "resource": {
                "additional": {},
                "cpus": 1,
                "memory": "256"
            },
            "restart_policy": "ALWAYS",
            "run_privileged_container": false,
            "state": "FLEXING"
        }
    ],
    "configuration": {
        "env": {},
        "files": [],
        "properties": {}
    },
    "dependencies": [],
    "id": "application_1548273183277_0001",
    "kerberos_principal": {},
    "lifetime": -1,
    "name": "test1",
    "quicklinks": {},
    "state": "STARTED",
    "version": "1.0.2"
}
{code}

Once localization is completed, the status output of the service reflects it:
{code}
{
    "components": [
        {
            "configuration": {
                "env": {},
                "files": [
                    {
                        "dest_file": "changed_name.txt",
                        "properties": {},
                        "src_file": "file1.txt",
                        "type": "STATIC"
                    }
                ],
                "properties": {}
            },
            "containers": [
                {
                    "bare_host": "10.22.8.153",
                    "component_instance_name": "sleeper-0",
                    "hostname": "HW12119.local",
                    "id": "container_1548273183277_0001_01_000002",
                    "ip": "10.22.8.153",
                    "launch_time": 1548275812327,
                    "localization_statuses": [
                        {
                            "dest_file": "changed_name.txt",
                            "state": "COMPLETED"
                        }
                    ],
                    "state": "RUNNING_BUT_UNREADY"
                },
                {
                    "bare_host": "10.22.8.153",
                    "component_instance_name": "sleeper-1",
                    "hostname": "HW12119.local",
                    "id": "container_1548273183277_0001_01_000003",
                    "ip": "10.22.8.153",
                    "launch_time": 1548275813334,
                    "localization_statuses": [
                        {
                            "dest_file": "changed_name.txt",
                            "state": "COMPLETED"
                        }
                    ],
                    "state": "RUNNING_BUT_UNREADY"
                }
            ],
            "decommissioned_instances": [],
            "dependencies": [],
            "launch_command": "sleep 8000",
            "name": "sleeper",
            "number_of_containers": 2,
            "quicklinks": [],
            "resource": {
                "additional": {},
                "cpus": 1,
                "memory": "256"
            },
            "restart_policy": "ALWAYS",
            "run_privileged_container": false,
            "state": "FLEXING"
        }
    ],
    "configuration": {
        "env": {},
        "files": [],
        "properties": {}
    },
    "dependencies": [],
    "id": "application_1548273183277_0001",
    "kerberos_principal": {},
    "lifetime": -1,
    "name": "test1",
    "quicklinks": {},
    "state": "STARTED",
    "version": "1.0.2"
}
{code}

[~eyang] let me know if this looks good to you?

> Retrieve the status of resource localization
> --------------------------------------------
>
>                 Key: YARN-8867
>                 URL: https://issues.apache.org/jira/browse/YARN-8867
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Chandni Singh
>            Assignee: Chandni Singh
>            Priority: Major
>         Attachments: YARN-8867.001.patch, YARN-8867.002.patch, YARN-8867.003.patch, YARN-8867.004.patch,
YARN-8867.005.patch, YARN-8867.006.patch, YARN-8867.007.patch, YARN-8867.wip.patch
>
>
> Refer YARN-3854.
> Currently NM does not have an API to retrieve the status of localization. Unless the
client can know when the localization of a resource is complete irrespective of the type of
the resource, it cannot take any appropriate action. 
> We need an API in {{ContainerManagementProtocol}} to retrieve the status on the localization.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message