Difference between revisions of "Hadoop: Sampel Dataset untuk test Hadoop"

From OnnoWiki
Jump to navigation Jump to search
Line 15: Line 15:
 
If you have Hadoop installed on your machine,you can use the following two ways to generate data.
 
If you have Hadoop installed on your machine,you can use the following two ways to generate data.
  
==3.hadoop jar /usr/lib/hadoop-0.20-mapreduce/hadoop-examples.jar randomwriter /random-data==
+
==hadoop-examples.jar randomwriter /random-data==
 +
 
 +
cd /usr/local/hadoop
 +
hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar randomwriter /random-data
 +
 
 +
Membuat 10 GB data per node didalam folder /random-data di HDFS.
  
generates 10 GB data per node under folder /random-data in HDFS.
 
               
 
 
==hadoop-examples.jar randomtextwriter /random-text-data==
 
==hadoop-examples.jar randomtextwriter /random-text-data==
  

Revision as of 18:00, 9 November 2015

Sumber: http://www.hadooplessons.info/2013/06/data-sets-for-practicing-hadoop.html


To practise Hadoop you can use below ways to generate the big data (GB),So that you can get the real feel/power of the Hadoop.


1.clearbits.net

From clearbits.net, you can get quarterly full data set of stack exchange so that you can use it while you are practising the hadoop . it contains around 10 GB data.

2.grouplens.org

grouplens.org collected different rating data sets ,you can use it for practicing the hadoop.

If you have Hadoop installed on your machine,you can use the following two ways to generate data.

hadoop-examples.jar randomwriter /random-data

cd /usr/local/hadoop
hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar randomwriter /random-data

Membuat 10 GB data per node didalam folder /random-data di HDFS.

hadoop-examples.jar randomtextwriter /random-text-data

cd /usr/local/hadoop
hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar randomtextwriter /random-text-data

Membuat 10 GB text data per node dibawah folder /random-text-data di HDFS

5. Amazon

provides so many data sets ,you can use them.

6. Stackoverflow

Check answers of the same question on stackoverflow


7. University of Waitako

many data sets available for practicing machine learning.

8. Quora

See answers for the similar question on Quora.

If you know any free data sets ,please share in comments





Referensi