hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Sankauskas <pe...@motally.com>
Subject How do I make an async call to Hive in Java?
Date Tue, 02 Feb 2010 17:42:42 GMT
Hi everyone,

I would like to execute a Hive query on the server in an asynchronous
manner. The Hive query will likely take a long time to complete, so I would
prefer not to block on the call. I am currently using Thirft to make a
blocking call (blocks on client.execute()), but I have not seen an example
of how to make a non-blocking call. Here is the blocking code:

    TSocket transport = new TSocket("hive.example.com", 10000);
    transport.setTimeout(999999999);
    TBinaryProtocol protocol = new TBinaryProtocol(transport);
    Client client = new ThriftHive.Client(protocol);
    transport.open();
    client.execute(hql);  // Omitted HQL

    List<String> rows;
    while ((rows = client.fetchN(1000)) != null) {
        for (String row : rows) {
            // Do stuff with row
        }
    }

    transport.close();

The code above is missing try/catch blocks to keep it short.

I can see that Thirft has things like "TNonblockingSocket" but I cannot find
any examples of how to use it, or if Hive even supports it. Does anyone have
any ideas how to do an async call? Is there a better way?

Thanks!

Kind regards,
Peter Sankauskas

Mime
View raw message