Hama学习笔记(6)-获取各个peer(task)的信息、确定master task
有时候在bsp job中需要确定一个master task,这就需要获取各个peer(task)的信息,如host name、端口号等等。
hama中获取peer的主机名和端口号很方便:
@SuppressWarnings("unchecked") private final static <KEYIN, VALUEIN, KEYOUT, VALUEOUT, M extends Writable> void runBSP( final BSPJob job, BSPPeerImpl<KEYIN, VALUEIN, KEYOUT, VALUEOUT, M> bspPeer, final BytesWritable rawSplit, final BSPPeerProtocol umbilical) throws Exception { BSP<KEYIN, VALUEIN, KEYOUT, VALUEOUT, M> bsp = (BSP<KEYIN, VALUEIN, KEYOUT, VALUEOUT, M>) ReflectionUtils .newInstance(job.getConfiguration().getClass("bsp.work.class", BSP.class), job.getConfiguration()); // The policy is to throw the first exception and log the remaining. Exception firstException = null; try { bsp.setup(bspPeer); bsp.bsp(bspPeer); } catch (Exception e) { LOG.error("Error running bsp setup and bsp function.", e); firstException = e; } finally { try { bsp.cleanup(bspPeer); } catch (Exception e) { LOG.error("Error cleaning up after bsp executed.", e); if (firstException == null) firstException = e; } finally { try { bspPeer.close(); } catch (Exception e) { LOG.error("Error closing BSP Peer.", e); if (firstException == null) firstException = e; } if (firstException != null) throw firstException; } } }
可见,调用setup之后紧接着就是调用bsp方法,setup中不加sync的话,可能master task正在发送消息,其他的peer已经进入bsp函数执行第一个超步了,这时他们试图读取master task发来的消息是读不到的。当然啦,也可以发第一轮消息发送放在第一个超步中。