一个hadoop执行问题
刚刚学习hadoop,搭建起集群在运行hadoop自带例子时报如下错误。请高手帮我解决下
hadoop jar c:/hadoop/hadoop-0.20.2/hadoop-0.20.2-examples.jar wordcount in/test.txt out12/06/08 15:57:08 INFO input.FileInputFormat: Total input paths to process : 112/06/08 15:57:08 INFO mapred.JobClient: Running job: job_201206081518_000112/06/08 15:57:09 INFO mapred.JobClient: map 0% reduce 0%12/06/08 15:57:18 INFO mapred.JobClient: map 100% reduce 0%12/06/08 15:57:32 INFO mapred.JobClient: Task Id : attempt_201206081518_0001_r_000000_0, Status : FAILEDjava.io.IOException: Can not get the relative path: base = hdfs://MASTER:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_0 child = hdfs://master:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_0/part-r-00000 at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:200) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:146) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:165) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:118) at org.apache.hadoop.mapred.Task.commit(Task.java:779) at org.apache.hadoop.mapred.Task.done(Task.java:691) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414) at org.apache.hadoop.mapred.Child.main(Child.java:170)12/06/08 15:57:47 INFO mapred.JobClient: Task Id : attempt_201206081518_0001_r_000000_1, Status : FAILEDjava.io.IOException: Can not get the relative path: base = hdfs://MASTER:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_1 child = hdfs://master:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_1/part-r-00000 at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:200) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:146) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:165) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:118) at org.apache.hadoop.mapred.Task.commit(Task.java:779) at org.apache.hadoop.mapred.Task.done(Task.java:691) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414) at org.apache.hadoop.mapred.Child.main(Child.java:170)12/06/08 15:58:02 INFO mapred.JobClient: Task Id : attempt_201206081518_0001_r_000000_2, Status : FAILEDjava.io.IOException: Can not get the relative path: base = hdfs://MASTER:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_2 child = hdfs://master:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_2/part-r-00000 at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:200) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:146) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:165) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:118) at org.apache.hadoop.mapred.Task.commit(Task.java:779) at org.apache.hadoop.mapred.Task.done(Task.java:691) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414) at org.apache.hadoop.mapred.Child.main(Child.java:170)12/06/08 15:58:20 INFO mapred.JobClient: Job complete: job_201206081518_000112/06/08 15:58:20 INFO mapred.JobClient: Counters: 1212/06/08 15:58:20 INFO mapred.JobClient: Job Counters12/06/08 15:58:20 INFO mapred.JobClient: Launched reduce tasks=412/06/08 15:58:20 INFO mapred.JobClient: Launched map tasks=112/06/08 15:58:20 INFO mapred.JobClient: Data-local map tasks=112/06/08 15:58:20 INFO mapred.JobClient: Failed reduce tasks=112/06/08 15:58:20 INFO mapred.JobClient: FileSystemCounters12/06/08 15:58:21 INFO mapred.JobClient: HDFS_BYTES_READ=2512/06/08 15:58:21 INFO mapred.JobClient: FILE_BYTES_WRITTEN=9312/06/08 15:58:21 INFO mapred.JobClient: Map-Reduce Framework12/06/08 15:58:21 INFO mapred.JobClient: Combine output records=512/06/08 15:58:21 INFO mapred.JobClient: Map input records=212/06/08 15:58:21 INFO mapred.JobClient: Spilled Records=512/06/08 15:58:21 INFO mapred.JobClient: Map output bytes=4512/06/08 15:58:21 INFO mapred.JobClient: Combine input records=512/06/08 15:58:21 INFO mapred.JobClient: Map output records=5
1 楼 yaoxiaowei0909 2012-06-08 我觉得是hdfs://MASTER:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_0 child = hdfs://master:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_0/part-r-00000 atmaster 大小写的问题。但是不知道在哪里改? 2 楼 yaoxiaowei0909 2012-06-18 问题解决。core-site.xml里,我写的是master。写iP就对了