分享

HDFS Shell基本操作

yaojiank 发表于 2013-10-25 10:45:16 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 2 11946
本帖最后由 pig2 于 2014-5-2 13:59 编辑


浏览
HDFS
目录
/opt/hadoop$ bin/hadoop fs -ls
上传文件到
HDFS
上传
/opt/hadoop$ bin/hadoop fs -put conf input
检查
/opt/hadoop$ bin/hadoop fs -ls
/opt/hadoop$ bin/hadoop fs -ls input

HDFS
下载文件到本机
下载
/opt/hadoop$ bin/hadoop fs -get input fromHDFS
检查
/opt/hadoop$ ls -al | grep fromHDFS
/opt/hadoop$ ls -al fromHDFS
删除文档
/opt/hadoop$ bin/hadoop fs -ls input
/opt/hadoop$ bin/hadoop fs -rm input/masters
查看文档
/opt/hadoop$ bin/hadoop fs -ls input
/opt/hadoop$ bin/hadoop fs -cat input/slaves
Example
$ bin/hadoop fs -ls input
Found 4 items
-rw-r--r--
2 hadooper supergroup
115045564 2009-04-02 11:51 /user/hadooper/input/1.txt-rw-r--r--
2 hadooper supergroup
987864 2009-04-02 11:51 /user/hadooper/input/2.txt-rw-r--r--
2 hadooper supergroup
1573048 2009-04-02 11:51 /user/hadooper/input/3.txt-rw-r--r--
2 hadooper supergroup
25844527 2009-04-02 11:51 /user/hadooper/input/4.txt
$ bin/hadoop fs -ls hdfs://gm1.nchc.org.tw:9000/user/hadooper/input
Found 4 items
-rw-r--r--
2 hadooper supergroup
115045564 2009-04-02 11:51 /user/hadooper/input/1.txt-rw-r--r--
2 hadooper supergroup
987864 2009-04-02 11:51 /user/hadooper/input/2.txt-rw-r--r--
2 hadooper supergroup
1573048 2009-04-02 11:51 /user/hadooper/input/3.txt-rw-r--r--
2 hadooper supergroup
25844527 2009-04-02 11:51 /user/hadooper/input/4.txt
$ bin/hadoop fs -cat quota/hadoop-env.sh
$ bin/hadoop fs -chgrp -R hadooper own
$ bin/hadoop fs -chmod -R 755 own
$ bin/hadoop fs -chown -R hadooper own
$ bin/hadoop fs -put input dfs_input
$ bin/hadoop fs -get dfs_input input1
$ bin/hadoop fs -cp own hadooper
$ bin/hadoop fs -du input

Found 4 items
115045564
hdfs://gm1.nchc.org.tw:9000/user/hadooper/input/1.txt987864
hdfs://gm1.nchc.org.tw:9000/user/hadooper/input/2.txt1573048
hdfs://gm1.nchc.org.tw:9000/user/hadooper/input/3.txt25844527
hdfs://gm1.nchc.org.tw:9000/user/hadooper/input/4.txt
$ bin/hadoop fs -dus input

hdfs://gm1.nchc.org.tw:9000/user/hadooper/input
143451003
$ bin/hadoop fs -expunge
$ echo "this is one; " >> in1/input
$ echo "this is two; " >> in1/input2
$ bin/hadoop fs -put in1 in1
$ bin/hadoop fs -getmerge in1 merge.txt
$ cat ./merge.txt

$ bin/hadoop fs -ls
$ bin/hadoop fs -lsr /
$ bin/hadoop fs -mkdir a b c
$ bin/hadoop fs -moveFromLocal in1 in2
$ bin/hadoop fs -mv in2 in3
$ bin/hadoop fs -rm in1/input
$ bin/hadoop fs -rmr in1
$ bin/hadoop fs -setrep -w 2 -R input
Replication 2 set:
$ bin/hadoop fs -stat input
2009-04-02 03:51:29
将文件最后1K的内容输出
$ bin/hadoop fs -tail input/1.txt
检查文件是否存在
$ bin/hadoop fs -test -e /user/hadooper/input/5.txt
$ bin/hadoop fs -test -z /user/hadooper/input/5.txt
test: File does not exist: /user/hadooper/input/5.txt
$ bin/hadoop fs -test -d /user/hadooper/input/5.txt

test: File does not exist: /user/hadooper/input/5.txt
$ hadoop fs -text macadr-eth1.txt.gz
00:1b:fc:61:75:b1
00:1b:fc:58:9c:23
建立一个空文件
$ bin/hadoop fs -touchz b/kk
$ bin/
hadoop fs -test -z b/kk
$ echo $?
1
$ bin/hadoop fs -test -z b/a.txt.zip
$ echo $?
0

已有(2)人评论

跳转到指定楼层
dgxl 发表于 2013-10-25 10:45:16
总结不错,就是如果能稍稍排版一下更好。:)
回复

使用道具 举报

starrycheng 发表于 2013-10-25 10:45:16
这是在搞啥啊
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条