Bryan

            BlogJava :: 首頁 :: 聯系 :: 聚合  :: 管理
            37 Posts :: 3 Stories :: 24 Comments :: 0 Trackbacks
          Please refer to hadoop-2.7.0-src\BUILDING.txt and install all the required components.

          1. install jdk1.7.0_25 and configure the JAVA_HOME and PATH, CLASSPATH, install apache-maven-3.3.3  and config MAVEN_HOME and also the PATH
          use java -version, javac -version, mvn -version to validate the installation.

          2. install windows sdk with the following URL : http://www.microsoft.com/en-us/download/details.aspx?id=8279
          Windows SDK 7.1 Command Prompt will appear

          3. download and install windows git package Git-1.9.5-preview20150319.exe from site http://git-scm.com/download/win,and add 
          C:\Program Files (x86)\Git\bin to the path so that shell command(shell command in pom.xml) can be used.

          4. download and install protoc-2.5.0-win32.zip, place in system32 folder or added to the path.
          use command protoc --version to test

          5.download cmake and install http://www.cmake.org/download/
          cmake -version

          6. get hadoop source hadoop 2.7.0 and 
          open Windows SDK 7.1 Command Prompt and build hadoop with the following command
          set Platform=x64
          cd C:\hadoop\hadoop-2.7.0-src
          mvn package -Pdist,native-win -DskipTests -Dtar

          The following are the files I have listed which need to install, some of them is for resolving issues of hadoop maven build  and if you are not facing them, you can ignore.

          hadoop-2.7.0-src.tar.gz  --->required
          Git-1.9.5-preview20150319.exe  --->required
          apache-maven-3.3.3  --->required
          cmake-3.2.2-win32-x86.exe  --->required
          winsdk_web.exe  --->required
          protoc-2.5.0-win32.zip  --->required
          hadoop-2.7.0-src  --->required
          VC-Compiler-KB2519277.exe -----> sp1
          ammintrin.h
          MicrosoftFixit.ProgramInstallUninstall.RNP.6635582324812150.4.1.Run.exe  ------> tool to uninstall the newer version of visual c++ if it exists

          The following are the issues encountered when I compile Hadoop 2.7.0

          Installation Blockers:
          A newer version of Microsoft Visual C++ 2010 Redistributable has been detected on the machine.
          Final Result: Installation failed with error code: (0x000013EC), "A StopBlock was hit or a System Requirement was not met." (Elapsed time: 0 00:00:01).

          Please refer to the link(
          http://stackoverflow.com/questions/19366006/error-when-installing-windows-sdk-7-1
          ) when errors are found. Try to uninstall the newer version first. It does have a newer version in my laptop,but  I could not find it in windows uninstaller, but this tool (
          http://support.microsoft.com/mats/Program_Install_and_Uninstall
          )can.


          LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or c
          orrupt [D:\hadoop\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\
          winutils\winutils.vcxproj]


          Answer:
          download and install Microsoft Visual C++ 2010 Service Pack 1 Compiler Update for the Windows SDK 7.1
          https://www.microsoft.com/en-us/download/details.aspx?id=4422

          C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\intrin.h(26): f
          atal error C1083: Cannot open include file: 'ammintrin.h': No such file or direc
          tory [D:\hadoop\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\na
          tive\native.vcxproj]
            C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\intrin.h(26): f
          atal error C1083: Cannot open include file: 'ammintrin.h': No such file or direc
          tory [D:\hadoop\hadoop-2.7.0-src\hadoop-common-project\hadoop-common\src\main\na
          tive\native.vcxproj]

              1 Warning(s)
              2 Error(s)

          download file ammintrin.h from the internet and place it in folder C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include


          posted on 2015-05-23 17:00 Life is no respector of any genius. 閱讀(2618) 評論(8)  編輯  收藏 所屬分類: hadoop

          Feedback

          # re: Hadoop2.7.0 build on win7 64 2015-07-10 22:14 Life is no respector of any genius.
          hadoop2.7.1
          http://pan.baidu.com/s/1dDwF4op  回復  更多評論
            

          # re: Hadoop2.7.0 build on win7 64 2015-07-12 07:16 Bryan
          When runing hadoop2.7.1 on SunOS, the following error happens, and after one day of analysis, It is due to that "df -k -P" fails on "SunOS plmdev 5.10 Generic_141444-09 sun4v sparc SUNW,SPARC-Enterprise-T5220"

          2015-07-11 22:27:11,205 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Stopping services started for active state
          2015-07-11 22:27:11,205 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 1
          2015-07-11 22:27:11,217 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 5 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 163
          2015-07-11 22:27:11,230 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /data/ALMR/hadoop/data/dfs/namenode/current/edits_inprogress_0000000000000000001 -> /data/ALMR/hadoop/data/dfs/namenode/current/edits_0000000000000000001-0000000000000000002
          2015-07-11 22:27:11,241 INFO org.apache.hadoop.ipc.Server: Stopping server on 9000
          2015-07-11 22:27:11,338 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Stopping services started for active state
          2015-07-11 22:27:11,338 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Stopping services started for standby state
          2015-07-11 22:27:11,354 INFO org.mortbay.log: Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:50070
          2015-07-11 22:27:11,366 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode metrics system...
          2015-07-11 22:27:11,369 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system stopped.
          2015-07-11 22:27:11,370 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system shutdown complete.
          2015-07-11 22:27:11,409 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
          ExitCodeException exitCode=1:
          at org.apache.hadoop.util.Shell.runCommand(Shell.java:554)
          at org.apache.hadoop.util.Shell.run(Shell.java:456)
          at org.apache.hadoop.fs.DF.getFilesystem(DF.java:76)
          at org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker$CheckedVolume.<init>(NameNodeResourceChecker.java:69)
          at org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker.addDirToCheck(NameNodeResourceChecker.java:165)
          at org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker.<init>(NameNodeResourceChecker.java:134)
          at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startCommonServices(FSNamesystem.java:1058)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:678)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:664)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
          at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
          2015-07-11 22:27:11,424 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
          2015-07-11 22:27:11,430 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
          /************************************************************
          SHUTDOWN_MSG: Shutting down NameNode at plmdev/135.251.35.155
          ************************************************************/
            回復  更多評論
            

          # re: Hadoop2.7.0 build on win7 64 2015-08-01 22:23 Life is no respector of any genius.
          continue with the above message:

          It seems that I am making a mistake, there are different versions of du df and id command, and finally, I am using /usr/local/bin, and this problem disappears.


          SYNOPSIS
          /usr/bin/du [-adr] [-k | -h] [-H | -L] [-o | -s] [file
          ...]

          /usr/xpg4/bin/du [-a | -s] [-k | -h] [-H | -L] [-rx]
          [file ...]

          Also, map reduce jobs get stuck in accepted state and could not get it run, finally I found that the disk space is greater than 90% or /tmp reaches 90% threshold.  回復  更多評論
            

          # re: Hadoop2.7.0 build on win7 64 2015-08-03 05:58 Life is no respector of any genius.
          http://git-scm.com/download/linux

          unix package tool
          http://www.opencsw.org/packages/CSWbashcmplt/  回復  更多評論
            

          # re: Hadoop2.7.0 build on win7 64 2015-10-17 20:45 Life is no respector of any genius.
          hadoop 2.6.1
          http://yun.baidu.com/s/1o6zRwgy

          hadoop-eclipse-plugin-2.6.1.jar
          http://pan.baidu.com/s/1qMSJO  回復  更多評論
            

          # re: Hadoop2.7.0 build on win7 64 2015-12-20 11:15 Life is no respector of any genius.
          windows 7 build of hadoop
          hadoop 2.6.2
          http://pan.baidu.com/s/1n0vYQ

          hadopop 2.6.3
          http://pan.baidu.com/s/1c1y9OaO
            回復  更多評論
            

          # re: Hadoop2.7.0 build on win7 64 2016-01-02 09:53 Life is no respector of any genius.
          http://javadev.org/sitemap/  回復  更多評論
            

          # re: Hadoop2.7.0 build on win7 64 2016-02-01 08:38 tongdayong
          hadoop 2.7.2
          http://pan.baidu.com/s/1i4plIfF  回復  更多評論
            


          只有注冊用戶登錄后才能發表評論。


          網站導航:
           
          主站蜘蛛池模板: 科技| 高陵县| 宜兰县| 大荔县| 苗栗市| 剑河县| 伊金霍洛旗| 黑龙江省| 泽库县| 田阳县| 集安市| 汝阳县| 恩平市| 南皮县| 上高县| 华蓥市| 芦山县| 大邑县| 新郑市| 民权县| 剑川县| 永善县| 丹巴县| 婺源县| 桦南县| 西宁市| 日喀则市| 沙坪坝区| 元朗区| 清原| 涟源市| 通辽市| 柞水县| 大安市| 墨竹工卡县| 沂源县| 潼南县| 吉隆县| 锡林郭勒盟| 五家渠市| 新巴尔虎左旗|