Introduction to Hadoop

Ravikiran Paladugu

Ravi pursuing his doctoral degree in the area of Data Science and currently working as a Storage and Hadoop consultant specializing in Apache Hadoop, EMC and NetApp products, USA. My experience with Big data includes design, deployment and administration of Hadoop and related components by providing end-to-end data protection and identifying performance bottlenecks in the infrastructure.

Loading Facebook Comments ...

20 Responses

  1. Sukumar Enuguri says:

    I’d love to learn more hadoop…

  2. Ramdev says:

    sukumar, you will see more on this space.

  3. ramesh reddy says:

    Hi Ramdev,

    Currently am working as sr unix admin(Solaris-primary Linux-Secondary)
    i would like to learn hadoop , but which topic is suit for unox admins like HDFS, JOBkeer etc.
    Could you please suggest .. where to start and what to learn.

  4. Ramdev says:

    Ramesh, Undestanding with HDFS, MAP Reduce and Hadoop Cluster maintence will be additional asset for unix admin, if they chose to work in cloud based environment.

  5. ramesh reddy says:

    Hi Ramdev,

    Please provide some websites for HDFS, MAP Reduce and Hadoop Clusters documentation

  6. Abhishek says:

    Hey Rama…Super article..:)
    Can u pls share/send steps to install it …i am working on it with 200+ data nodes and started from scratch…right now I m scratching my own head ..but will keep on updating my findings and glitches here..if u have any apart from cloudera..pls send. Many Thanks..and hats off to ur website..it’s awesome..!!

  7. Hi Abhishek,

    I am working on drafting a article to summarize the general steps of installing Hadoop. I bet it would be hectic when you are just starting with this technology, but as you get experienced it will be great fun. I would go for scripting when talking about 200+ nodes.

  8. Piper says:

    I personally thouroughly liked this topic unixadminschool.com » Introduction to Hadoop totally a very lovely browse appreciation

  9. Thanks for the feedback Piper!

  10. Rahul says:

    Hi Ramdev,

    currently i’m working as a Solaris/Linux Admin from 2 Years. and now i’m looking for Oracle DBA will it good for me specially my future?please suggest me.

  11. Ramdev says:

    Hi Rahul, for me personally the career growth is same for both DBA and Sysadmin and infact they grow parallel in infrastructure management services. If at all if you are looking for future technologies and interested in database related stuff, i would recommend cassandra and bigdata skills.  

  12. Kiran says:

    I am interested to know the Hadoop concepts for Unix SA’s. As per my knowledge there are 2 ways in Hadoop.
    1. Hadoop Development 2. Hadoop Admin. For Unix SA’s Hadoop Admin is very good suitable.
    In current market (Jobs) how is Hadoop growth. Will it continue ? I

  13. david blowe says:

    Hadoop – very nice ! I am broadening my knowledge greatly ! Thanks ! If you have more on Isilon and Hadoop that would be good .Excellent ! Did I understand correctly that Hadoop allows or makes a triad/triangle of DN servers so if one goes down then the data can still be obtained from the other two ? Or it “just” has 3copies on one big server like I think Isilon is or Panasas too ?

  14. Kiran MS says:

    Step:2 . By default we make it as 128mb blocks
    Make me clear on this … i see that by default it should be 64 MB right ?

  15. Ramdev says:

    @Kiran, i believe for the values mentioned in this article based cloudera hadoop.
    Are you using apache default hadoop version?

  16. Kiran MS says:

    Okie….
    I have an issue.. I am trying to move the files from Linux to Hadoop. No errors in it. But the problem is ..We are unable to view the Dir…

    Pls find the following o/p

    hadoop@hadoop-VirtualBox:~$ pwd
    /home/hadoop

    hadoop@hadoop-VirtualBox:~$ ls -l Hadoop_Files
    -rw-rw-r– 1 hadoop hadoop 9 Feb 2 11:31 Hadoop_Files

    hadoop@hadoop-VirtualBox:~$ hadoop fs -put Hadoop_Files /user/hadoop

    hadoop@hadoop-VirtualBox:~$ hadoop fs -ls
    Found 4 items
    -rw-r–r– 1 hadoop supergroup 22 2014-02-01 12:15 /user/hadoop/Apache_Hadoop
    -rw-r–r– 1 hadoop supergroup 9 2014-02-02 11:31 /user/hadoop/Hadoop_Files
    -rw-r–r– 1 hadoop supergroup 8445 2014-02-01 12:25 /user/hadoop/examples.desktop
    -rw-r–r– 1 hadoop supergroup 22 2014-02-01 12:16 /user/hadoop/test

    hadoop@hadoop-VirtualBox:~$ hadoop fs -cat /user/hadoop/Hadoop_Files
    Test File

    hadoop@hadoop-VirtualBox:~$ hadoop fs -lsr
    -rw-r–r– 1 hadoop supergroup 22 2014-02-01 12:15 /user/hadoop/Apache_Hadoop
    -rw-r–r– 1 hadoop supergroup 9 2014-02-02 11:31 /user/hadoop/Hadoop_Files
    -rw-r–r– 1 hadoop supergroup 8445 2014-02-01 12:25 /user/hadoop/examples.desktop
    -rw-r–r– 1 hadoop supergroup 22 2014-02-01 12:16 /user/hadoop/test
    hadoop@hadoop-VirtualBox:~$

    hadoop@hadoop-VirtualBox:~$ jps
    2543 DataNode
    3780 Jps
    3085 TaskTracker
    2857 JobTracker
    2773 SecondaryNameNode
    2316 NameNode
    hadoop@hadoop-VirtualBox:~

    From localhost:50070 we are able see the Dir but not in command line…

  17. Kiran MS says:

    from localhost:50070

    Name Type Size Replication Block Size Modification Time Permission Owner Group
    tmp dir

    2014-02-01 12:14 rwxr-xr-x hadoop supergroup
    user dir

    2014-02-01 12:25 rwxr-xr-x hadoop supergroup

    Name Type Size Replication Block Size Modification Time Permission Owner Group
    hadoop dir

    2014-02-02 11:31 rwxr-xr-x hadoop supergrou

    Name Type Size Replication Block Size Modification Time Permission Owner Group
    Apache_Hadoop file 0.02 KB 1 64 MB 2014-02-01 12:15 rw-r–r– hadoop supergroup
    Hadoop_Files file 0.01 KB 1 64 MB 2014-02-02 11:31 rw-r–r– hadoop supergroup
    examples.desktop file 8.25 KB 1 64 MB 2014-02-01 12:25 rw-r–r– hadoop supergroup
    test file 0.02 KB 1 64 MB 2014-02-01 12:16 rw-r–r– hadoop supergroup

  18. PRAFUL says:

    Thanks for Showing interest in HADOOP, Please go hadoop Admin part… it will very help for you . RAJA 9818868839

  19. I am planning to get hadoop administratio training from you .

    If possible, please let me know the details

Leave a Reply

Your email address will not be published.

[contact-form to='ramkumar.ramadevu@gmail.com' subject='New Learning Request Submitted'][contact-field label='Name' type='name' required='1'/][contact-field label='Email' type='email' required='1'/][contact-field label='Learning Request' type='textarea' required='1'/][contact-field label='Are you Looking for ' type='radio' required='1' options='Paid Training,Free Training'/][/contact-form]

What is your Learning Goal for Next Six Months ? Talk to us