ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vitaly Brodetskyi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-4283) Pass Through API for API forwarding from the Ambari Server (Falcon/Jobs API).
Date Thu, 16 Jan 2014 19:21:19 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873805#comment-13873805
] 

Vitaly Brodetskyi commented on AMBARI-4283:
-------------------------------------------

[~mahadev].  Here are some thoughts about implementation of this functionality:
The main goal of this functionality(as i understand) to forward http requests of all types,
that we got from API call. And then, return back result, that we got after url execution.

We can implement this functionality by API. But i think it's not correct. Because, to my mind,
we can't map this functionality to our API architecture. In our API architecture, one of the
main places, takes resources. But in our proxy, there are no resources, as i understand. At
all, we need only service and provider, which will forward requests and responces. 

But if we need to implement it in API we can do the next. I'm not sure that names were choosen
correctly(reason described before). 
We should create new resource type and resourcedefinition:
{code}
Resource.Type.Proxy
{code}

{code}
public class ProxyResourceDefinition extends BaseResourceDefinition {

  public ProxyResourceDefinition () {
    super(Resource.Type.Proxy);
  }
  
  @Override
  public String getPluralName() {
    return "urls";
  }

  @Override
  public String getSingularName() {
    return "url";
  }

  @Override
  public Set<SubResourceDefinition> getSubResourceDefinitions() {
    return Collections.emptySet();
  }

}
{code}

After resource type and resourcedefinition were created, we need to add them to ResourceInstanceFactoryImpl.getResourceDefinition(...)
It will look like:
{code}
case Proxy:
   resourceDefinition = new ProxyResourceDefinition();
   break;
{code}

Then we can create new service class, something like ProxyService. It will look like:
{code}
@Path("/proxy/")
public class ProxyService extends BaseService {

  @GET
  @Path("{url}")
  @Produces("text/plain")
  public Response proceedGetUrl(@Context HttpHeaders headers, @Context UriInfo ui,
      @PathParam("url") String url) {
    return handleRequest(headers, null, ui, Request.Type.GET, createProxyResource(url));
  }
  
  
  @POST
  @Path("{url}")
  @Produces("text/plain")
  public Response proceedPostURL(String body, @Context HttpHeaders headers, @Context UriInfo
ui,
     @PathParam("url") String url) {
    return handleRequest(headers, body, ui, Request.Type.POST, createProxyResource(url));
  }
  
   
  @PUT
  @Path("{url}")
  @Produces("text/plain")
  public Response proceedPutURL(String body, @Context HttpHeaders headers, @Context UriInfo
ui,
     @PathParam("url") String url) {
    return handleRequest(headers, body, ui, Request.Type.PUT, createProxyResource(url));
  }
   
   
  @DELETE
  @Path("{url}")
  @Produces("text/plain")
  public Response proceedDeleteURL(@Context HttpHeaders headers, @Context UriInfo ui,
     @PathParam("url") String url) {
    return handleRequest(headers, null, ui, Request.Type.DELETE, createProxyResource(url));
  }

  private ResourceInstance createProxyResource(String url) {
    return createResource(Resource.Type.Proxy,
        Collections.singletonMap(Resource.Type.Proxy, url));
  }
}
{code}

And now we can create provider class for our new resource. For example ProxyResourceProvider.
Then we should add this provider to AbstractControllerResourceProvider.getResourceProvider(...)
{code}
case Proxy:
        return new ProxyResourceProvider(propertyIds, keyPropertyIds, managementController);
{code}

In ProxyPropertyProvider we should implement main functionality. Our main task is to execute
URL request, which was sent for us by API, and return response for user. In our project we
have URLStreamProvider class, which can help us to execute URL request. This class is used
by Ganglia and Nagios to get some properties/info. Here are some code of the main method:
{code}
@Override
  public InputStream readFrom(String spec, String requestMethod, String params) throws IOException
{
    if (LOG.isDebugEnabled()) {
      LOG.debug("readFrom spec:" + spec);
    }
    
    HttpURLConnection connection = spec.startsWith("https") ? 
        (HttpURLConnection)getSSLConnection(spec)
        : (HttpURLConnection)getConnection(spec);

    String appCookie = appCookieManager.getCachedAppCookie(spec);
    if (appCookie != null) {
      LOG.debug("Using cached app cookie for URL:" + spec);
      connection.setRequestProperty(COOKIE, appCookie);
    }
    connection.setConnectTimeout(connTimeout);
    connection.setReadTimeout(readTimeout);
    connection.setDoOutput(true);
    connection.setRequestMethod(requestMethod);
    
    if (params != null)
      connection.getOutputStream().write(params.getBytes());
    
    int statusCode = connection.getResponseCode();
    if (statusCode == HttpStatus.SC_UNAUTHORIZED ) {
      String wwwAuthHeader = connection.getHeaderField(WWW_AUTHENTICATE);
      if (LOG.isInfoEnabled()) {
        LOG.info("Received WWW-Authentication header:" + wwwAuthHeader + ", for URL:" + spec);
      }
      if (wwwAuthHeader != null && 
          wwwAuthHeader.trim().startsWith(NEGOTIATE)) {
        //connection.getInputStream().close();
        connection = spec.startsWith("https") ? 
            (HttpURLConnection)getSSLConnection(spec)
            : (HttpURLConnection)getConnection(spec);
        appCookie = appCookieManager.getAppCookie(spec, true);
        connection.setRequestProperty(COOKIE, appCookie);
        connection.setConnectTimeout(connTimeout);
        connection.setReadTimeout(readTimeout);
        connection.setDoOutput(true);
        
        return connection.getInputStream();
      } else {
        // no supported authentication type found
        // we would let the original response propogate
        LOG.error("Unsupported WWW-Authentication header:" + wwwAuthHeader+ ", for URL:" +
spec);
        return connection.getInputStream();
      }
    } else {
      // not a 401 Unauthorized status code
      // we would let the original response propogate
      return connection.getInputStream();
    }
  }
{code} 

We can send any URL and any HTTP request type for this method, and it will return InputStream
for us. I've tested it with simple GET request(http://dev01.hortonworks.com:50070/listPaths/user)
and got correct answer:
{code}
<?xml version="1.0" encoding="UTF-8"?>
<listing time="2014-01-16T14:02:52+0000" recursive="no" path="/user" exclude="" filter=".*"
version="2.2.0.2.0.6.0-101">
 <directory path="/user" modified="2014-01-16T10:41:16+0000" accesstime="1970-01-01T00:00:00+0000"
permission="drwxr-xr-x" owner="hdfs" group="hdfs"/>
 <directory path="/user/ambari-qa" modified="2014-01-16T10:44:12+0000" accesstime="1970-01-01T00:00:00+0000"
permission="drwxrwx---" owner="ambari-qa" group="hdfs"/>
</listing>
{code}

Then, UI can easy parse this response and show this info for user, in comfortable way.
**************************************************************************************************************************************************

Another way to implement this feature, it's Servlets. I think it will not take a lot of time
to implement proxy on it . It can be configured for any url call (using filters). Servlet
works fast enough. Servlet will work more faster then API(much less class/method calls and
objects). In servlet, we can use URLStreamProvider class code to send http requests and get
responses. And then, we will return response for user. About 'dispatcher forwarding', i think
will not work correctly, because as i remember we can forward only in scope of current host.

What do you think about it?

> Pass Through API for API forwarding from the Ambari Server (Falcon/Jobs API).
> -----------------------------------------------------------------------------
>
>                 Key: AMBARI-4283
>                 URL: https://issues.apache.org/jira/browse/AMBARI-4283
>             Project: Ambari
>          Issue Type: Task
>          Components: agent
>    Affects Versions: 1.5.0
>            Reporter: Vitaly Brodetskyi
>            Assignee: Vitaly Brodetskyi
>             Fix For: 1.5.0
>
>
> Pass Through API for API forwarding from the Ambari Server (Falcon/Jobs API).



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message