Hive JobTracking？

Question

我已经检查过这些方法，但我似乎找不到如何推断完成百分比的方法(就像eventListener!)蜂巢的一份工作。请帮帮我！编辑-我认为你可以从客户端得到"I'm done mapping...so I'm 50% completed“(如果我提交了一个覆盖外部表的命令)。OpsCenter with Brisk (由Datastax提供)就是做这件事的。

import java.util.List;

import org.apache.hadoop.hive.metastore.api.MetaException;
import org.apache.hadoop.hive.service.HiveServerException;
import org.apache.hadoop.hive.service.ThriftHive;
import org.apache.hadoop.hive.service.ThriftHive.Client;
import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.transport.TSocket;

public class Hive {

    static Client client;
    static TSocket transport;

    public static void main(String args[]) throws HiveServerException,
            TException, MetaException {

        transport = new TSocket("hiveserver",
                10000);
        transport.setTimeout(999999999);

        TBinaryProtocol protocol = new TBinaryProtocol(transport);
        client = new ThriftHive.Client(protocol);
        transport.open();

        System.out.println("Starting map job...");

        Thread mapReduceThread = new Thread(new HiveQuery(
                "SELECT COUNT(*) FROM myHiveTable"));
        mapReduceThread.start();

        System.out.println("Waiting on map...");
    }

    private static class HiveQuery implements Runnable {

        private String hql;

        public HiveQuery(String hql) {
            this.setHql(hql);
        }

        public void run() {
            long start = System.currentTimeMillis();

            // Blocking
            try {
                client.execute(this.getHql());
            } catch (HiveServerException e) {
                e.printStackTrace();
            } catch (TException e) {
                e.printStackTrace();
            }

            List<String> responseList = null;
            try {
                responseList = client.fetchAll();
            } catch (HiveServerException e) {
                e.printStackTrace();
            } catch (TException e) {
                e.printStackTrace();
            }

            long elapsedTimeMillis = System.currentTimeMillis() - start;
            float elapsedTime = elapsedTimeMillis / 1000F;

            System.out.println("Job took: " + elapsedTime + " seconds");

            for (String response : responseList) {
                System.out.println("Response: " + response);
            }

            transport.close();
            System.out.println("Closed transport");
            System.exit(0);
        }

        public void setHql(String hql) {
            this.hql = hql;
        }

        public String getHql() {
            return hql;
        }
    }

}

Answer 1

你所说的外推是什么意思？

我认为你不能通过Hadoop中的完成百分比数字来推断任何事情，除非它上升了，那么就发生了更多的工作。你不能说“哦，那意味着我还有20秒”或类似的话。