读书人

Hadoop学习十八:Hadoop-Hdfs DataXce

发布时间: 2013-07-08 14:13:00 作者: rapoo

Hadoop学习十八:Hadoop-Hdfs DataXceiverServer源码其它操作及总结

一.?其它操作

  public static final byte OP_REPLACE_BLOCK = (byte) 83;  public static final byte OP_COPY_BLOCK = (byte) 84;  public static final byte OP_BLOCK_CHECKSUM = (byte) 85;

?

二.REPLACE_BLOCK

?Hadoop学习十八:Hadoop-Hdfs DataXceiverServer源码其它操作及小结

Receive a block and write it to disk, it then notifies the namenode to?remove the copy from the source.sourceID:read delete hint.DatanodeInfo proxySource:read proxy source.拷贝数据块和写数据块不同的是发起拷贝请求方不提供数据,只提供一个数据源proxySource,接收方需要自己同proxySource建立连接;写请求发起方提供数据。拷贝请求不需要链式流,因为只发生在两个节点上。

?

三.copyBlock

Hadoop学习十八:Hadoop-Hdfs DataXceiverServer源码其它操作及小结

?Read a block from the disk and then sends it to a destination.读取block,发生给请求方。

四.getBlockChecksum

Hadoop学习十八:Hadoop-Hdfs DataXceiverServer源码其它操作及小结

Get block checksum (MD5 of CRC32).

?

五.总结

    readBlock(DataInputStream in):从in里读取blockId和generationStamp,读取此block数据,发送给请求方(Socket s = ss.accept())。writeBlock(DataInputStream in):从in里读取blockId和generationStamp,将要从此block读数据;?读取DatanodeInfo targets[],组成链式流;读取block数据到targets[0],继续发生给下一个targets[1]...replaceBlock(DataInputStream in):从in里读取blockId和generationStamp,将要从此block读数据;读取proxySource,与proxySource建立连接开始读取block数据;读取sourceID,notifies the namenode to?remove the copy from the source.copyBlock(DataInputStream in):从in里读取blockId和generationStamp,读取此block的数据发送给请求方。getBlockChecksum(DataInputStream in):从in里读取blockId和generationStamp,读取此block的检验值和MD5码。

读书人网 >软件架构设计

热点推荐