Hadoop: Perintah Shell - Revision history

Onnowpurbo: /* upload: hadoop fs -cat */

2015-11-09T10:25:29Z

upload: hadoop fs -cat

← Older revision		Revision as of 10:25, 9 November 2015
Line 32:		Line 32:
	To see contents of a file:		To see contents of a file:

	$ hadoop fs -cat /user/hdfs/dir1/myfile		$ hadoop fs -cat /user/hdfs/dir1/myfile
	This is myfile		This is myfile





	==hadoop fs -mkdir==		==hadoop fs -mkdir==

Onnowpurbo at 10:24, 9 November 2015

2015-11-09T10:24:29Z

← Older revision		Revision as of 10:24, 9 November 2015
Line 22:		Line 22:
	To list the contents of a directory:		To list the contents of a directory:

	$ hadoop fs -ls /user		$ hadoop fs -ls /user
	~~Found 8 items~~
	~~drwxr-xr-x - cloudera cloudera 0 2015-03-28 23:43 /user/cloudera~~
	~~drwxr-xr-x - hdfs supergroup 0 2015-03-14 20:11 /user/hdfs~~
	~~drwxr-xr-x - mapred hadoop 0 2015-03-15 14:08 /user/history~~
	~~drwxrwxrwx - hive hive 0 2014-12-18 04:33 /user/hive~~
	~~drwxrwxr-x - hue hue 0 2015-03-21 15:34 /user/hue~~
	~~drwxrwxrwx - oozie oozie 0 2014-12-18 04:34 /user/oozie~~
	~~drwxr-xr-x - sample sample 0 2015-03-14 22:05 /user/sample~~
	~~drwxr-xr-x - spark spark 0 2014-12-18 04:34 /user/spark~~



			Found 1 items
			drwxr-xr-x - hduser supergroup 0 2015-11-09 16:35 /user/hduser


Line 52:		Line 43:
	To create a directory in HDFS:		To create a directory in HDFS:

	$ hadoop fs -mkdir /user/hdfs/dir1		$ hadoop fs -mkdir /user/hdfs/dir1
	mkdir: Permission denied: user=cloudera, access=WRITE, inode="/user/hdfs":hdfs:supergroup:drwxr-xr-x		mkdir: Permission denied: user=cloudera, access=WRITE, inode="/user/hdfs":hdfs:supergroup:drwxr-xr-x

	$ sudo su hdfs		$ sudo su hdfs
	bash-4.1$ hadoop fs -mkdir /user/hdfs/dir1		bash-4.1$ hadoop fs -mkdir /user/hdfs/dir1


Line 66:		Line 57:
	To copy src files from local file system to the Hadoop data file system:		To copy src files from local file system to the Hadoop data file system:

	$ hadoop fs -put <local_src> <HDFS_dest_Path>		$ hadoop fs -put <local_src> <HDFS_dest_Path>

	bash-4.1$ whoami		bash-4.1$ whoami
	hdfs		hdfs

	bash-4.1$ hadoop fs -put myfile /user/hdfs/dir1		bash-4.1$ hadoop fs -put myfile /user/hdfs/dir1


Line 77:		Line 68:


	download: hadoop fs -get		==download: hadoop fs -get==

	To copy files from the Hadoop data file system to the local file system:		To copy files from the Hadoop data file system to the local file system:

	$ hadoop fs -get /user/hdfs/dir1/myfile .		$ hadoop fs -get /user/hdfs/dir1/myfile .


Line 91:		Line 82:
	To copy a file from source to destination:		To copy a file from source to destination:

	bash-4.1$ hadoop fs -cp /user/hdfs/dir1/myfile /user/hdfs/dir2		bash-4.1$ hadoop fs -cp /user/hdfs/dir1/myfile /user/hdfs/dir2

	bash-4.1$ hadoop fs -ls /user/hdfs/dir2/myfile		bash-4.1$ hadoop fs -ls /user/hdfs/dir2/myfile
	-rw-r--r-- 1 hdfs supergroup 7 2015-03-29 11:43 /user/hdfs/dir2/myfile		-rw-r--r-- 1 hdfs supergroup 7 2015-03-29 11:43 /user/hdfs/dir2/myfile


Line 106:		Line 97:
	hadoop fs -copyFromLocal <localsrc> URI		hadoop fs -copyFromLocal <localsrc> URI

	bash-4.1$ pwd		bash-4.1$ pwd
	/home/cloudera/workspace/temp		/home/cloudera/workspace/temp

	bash-4.1$ ls		bash-4.1$ ls
	myfile myfile2		myfile myfile2

	bash-4.1$ hadoop fs -copyFromLocal myfile2 /user/hdfs/dir2/		bash-4.1$ hadoop fs -copyFromLocal myfile2 /user/hdfs/dir2/

	bash-4.1$ hadoop fs -cat /user/hdfs/dir2/myfile2		bash-4.1$ hadoop fs -cat /user/hdfs/dir2/myfile2
	This is myfile2		This is myfile2


Line 125:		Line 116:
	To copy a file to Local file system from HDFS		To copy a file to Local file system from HDFS

	$ pwd		$ pwd
	/home/cloudera/workspace/temp		/home/cloudera/workspace/temp

	$ ls		$ ls
	myfile myfile2		myfile myfile2

	$ hadoop fs -copyToLocal /user/hdfs/dir1/myfile myfile3		$ hadoop fs -copyToLocal /user/hdfs/dir1/myfile myfile3

	$ ls		$ ls
	myfile myfile2 myfile3		myfile myfile2 myfile3

Onnowpurbo: New page: Sumber: http://www.bogotobogo.com/Hadoop/BigData_Hadoop_fs_commands_list.php ==HDFS shell commands== Hadoop Distributed File System (HDFS) is designed to reliably store very large file...

2015-11-09T05:54:26Z

New page: Sumber: http://www.bogotobogo.com/Hadoop/BigData_Hadoop_fs_commands_list.php ==HDFS shell commands== Hadoop Distributed File System (HDFS) is designed to reliably store very large file...

New page

Sumber: http://www.bogotobogo.com/Hadoop/BigData_Hadoop_fs_commands_list.php

==HDFS shell commands==

Hadoop Distributed File System (HDFS) is designed to reliably store very large files across machines in a large cluster.

Hadoop file system (fs) shell commands are used to perform various file operations such as copying a file, viewing the contents of the file, changing ownership of files, changing permissions, creating directories etc.

In this tutorial, we'll see basic Hadoop HDFS operations managed through shell commands which are useful to manage files on HDFS clusters.

==hadoop fs -ls==

To list the contents of a directory:

$ hadoop fs -ls /user
Found 8 items
drwxr-xr-x - cloudera cloudera 0 2015-03-28 23:43 /user/cloudera
drwxr-xr-x - hdfs supergroup 0 2015-03-14 20:11 /user/hdfs
drwxr-xr-x - mapred hadoop 0 2015-03-15 14:08 /user/history
drwxrwxrwx - hive hive 0 2014-12-18 04:33 /user/hive
drwxrwxr-x - hue hue 0 2015-03-21 15:34 /user/hue
drwxrwxrwx - oozie oozie 0 2014-12-18 04:34 /user/oozie
drwxr-xr-x - sample sample 0 2015-03-14 22:05 /user/sample
drwxr-xr-x - spark spark 0 2014-12-18 04:34 /user/spark

==upload: hadoop fs -cat==

To see contents of a file:

$ hadoop fs -cat /user/hdfs/dir1/myfile
This is myfile

==hadoop fs -mkdir==

To create a directory in HDFS:

$ hadoop fs -mkdir /user/hdfs/dir1
mkdir: Permission denied: user=cloudera, access=WRITE, inode="/user/hdfs":hdfs:supergroup:drwxr-xr-x

$ sudo su hdfs
bash-4.1$ hadoop fs -mkdir /user/hdfs/dir1

==upload: hadoop fs -put==

To copy src files from local file system to the Hadoop data file system:

$ hadoop fs -put <local_src> <HDFS_dest_Path>

bash-4.1$ whoami
hdfs

bash-4.1$ hadoop fs -put myfile /user/hdfs/dir1

download: hadoop fs -get

To copy files from the Hadoop data file system to the local file system:

$ hadoop fs -get /user/hdfs/dir1/myfile .

==copy: hadoop fs -cp==

To copy a file from source to destination:

bash-4.1$ hadoop fs -cp /user/hdfs/dir1/myfile /user/hdfs/dir2

bash-4.1$ hadoop fs -ls /user/hdfs/dir2/myfile
-rw-r--r-- 1 hdfs supergroup 7 2015-03-29 11:43 /user/hdfs/dir2/myfile

==copyFromLocal: hadoop fs -copyFromLocal==

To copy a file from Local file system to HDFS

hadoop fs -copyFromLocal <localsrc> URI

bash-4.1$ pwd
/home/cloudera/workspace/temp

bash-4.1$ ls
myfile myfile2

bash-4.1$ hadoop fs -copyFromLocal myfile2 /user/hdfs/dir2/

bash-4.1$ hadoop fs -cat /user/hdfs/dir2/myfile2
This is myfile2

==copyToLocal: hadoop fs -copyToLocal==

To copy a file to Local file system from HDFS

$ pwd
/home/cloudera/workspace/temp

$ ls
myfile myfile2

$ hadoop fs -copyToLocal /user/hdfs/dir1/myfile myfile3

$ ls
myfile myfile2 myfile3

==hadoop fs -rm==

To remove a file from HDFS

bash-4.1$ hadoop fs -ls /user/hdfs/dir2
Found 2 items
-rw-r--r-- 1 hdfs supergroup 7 2015-03-29 11:43 /user/hdfs/dir2/myfile
-rw-r--r-- 1 hdfs supergroup 16 2015-03-29 11:52 /user/hdfs/dir2/myfile2

bash-4.1$ hadoop fs -rm /user/hdfs/dir2/myfile2

bash-4.1$ hadoop fs -ls /user/hdfs/dir2
-rw-r--r-- 1 hdfs supergroup 7 2015-03-29 11:43 /user/hdfs/dir2/myfile

==directory remove: hadoop fs -rm -r==

To remove a directory from HDFS

bash-4.1$ hadoop fs -ls /user/hdfs/
Found 3 items
drwxr-xr-x - hdfs supergroup 0 2015-03-28 14:08 /user/hdfs/.Trash
drwxr-xr-x - hdfs supergroup 0 2015-03-29 10:55 /user/hdfs/dir1
drwxr-xr-x - hdfs supergroup 0 2015-03-29 12:12 /user/hdfs/dir2

bash-4.1$ hadoop fs -rm -r /user/hdfs/dir2

bash-4.1$ hadoop fs -ls /user/hdfs/
drwxr-xr-x - hdfs supergroup 0 2015-03-28 14:08 /user/hdfs/.Trash
drwxr-xr-x - hdfs supergroup 0 2015-03-29 10:55 /user/hdfs/dir1

==Referensi==

* http://www.bogotobogo.com/Hadoop/BigData_Hadoop_fs_commands_list.php