compile hadoop 2.8 on Ubuntu 16.04

Compile hadoop 2.8 on Ubuntu 16.04

  1. Install some dependencies:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    sudo apt -y install gcc*
    sudo apt -y install cmake
    sudo apt -y install glibc-headers
    sudo apt -y install gcc-c++
    sudo apt -y install zip-devel
    sudo apt -y install openssl-devel
    sudo apt -y install svn
    sudo apt -y install git
    sudo apt -y install ncurses-devel
    sudo apt -y install lzo-devel
    sudo apt -y install autoconf
    sudo apt -y install libtool
    sudo apt -y install automake
  2. Install protobuf:

    1
    2
    3
    4
    5
    6
    7
    8
    wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz
    tar zxvf protobuf-2.5.0.tar.gz
    cd protobuf-2.5.0
    ./configure
    make
    make check
    sudo make install
    sudo ldconfig
  3. Compile Hadoop:

    1
    2
    3
    4
    wget http://apache.spinellicreations.com/hadoop/common/hadoop-2.8.1/hadoop-2.8.1-src.tar.gz
    tar -zxvf hadoop-2.8.1-src.tar.gz
    cd hadoop-2.8.1-src
    mvn package -Dmaven.javadoc.skip=true -Pdist,native -DskipTests -Dtar
  4. The compiled hadoop tarball locates at hadoop-2.8.1-src/hadoop-dist/target/hadoop-2.8.1.tar.gz