�??xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:trackback="http://madskills.com/public/xml/rss/module/trackback/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/"><channel><title>&#31934;&#21697;&#22269;&#20135;&#40644;&#97;&#8744;&#29255;&#39640;&#28165;&#22312;&#32447;&#13;&#44;&#31934;&#21697;&#20813;&#36153;&#22269;&#20135;&#19968;&#21306;&#20108;&#21306;&#19977;&#21306;&#22235;&#21306;&#44;&#27611;&#29255;&#20813;&#36153;&#22312;&#32447;</title><link>http://www.aygfsteel.com/bacoo/category/35981.html</link><description>心怀未来�Q�开创未来！</description><language>zh-cn</language><lastBuildDate>Wed, 07 Jan 2009 23:15:44 GMT</lastBuildDate><pubDate>Wed, 07 Jan 2009 23:15:44 GMT</pubDate><ttl>60</ttl><item><title>InputFormat学习</title><link>http://www.aygfsteel.com/bacoo/archive/2009/01/07/250221.html</link><dc:creator>so true</dc:creator><author>so true</author><pubDate>Wed, 07 Jan 2009 01:40:00 GMT</pubDate><guid>http://www.aygfsteel.com/bacoo/archive/2009/01/07/250221.html</guid><wfw:comment>http://www.aygfsteel.com/bacoo/comments/250221.html</wfw:comment><comments>http://www.aygfsteel.com/bacoo/archive/2009/01/07/250221.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.aygfsteel.com/bacoo/comments/commentRss/250221.html</wfw:commentRss><trackback:ping>http://www.aygfsteel.com/bacoo/services/trackbacks/250221.html</trackback:ping><description><![CDATA[<p>InputFormat�Q�就是�ؓ了能够从一个jobconf中得��C��个split集合�Q�InputSplit[]�Q�，然后再�ؓ�q�个split集合配上一个合适的RecordReader�Q�getRecordReader�Q�来��d��每个split中的数据�?/p>
<p>InputSplit�Q���承自Writable接口�Q�因此一个InputSplit实则包含了四个接口函敎ͼ���d��写（readFields和write�Q�，getLength能够�l�出�q�个split中所记录的数据大���，getLocations能够得到�q�个split位于哪些��L��之上�Q�blkLocations[blkIndex].getHosts()�Q�，�q�里需要说明的是一个block要么对应一个split�Q�要么对应多个split�Q�因此每个split都可以从它所属的block中获取主��Z��息，而且我猜���block的大���应该是split的整数倍，否则有可能一个split跨越两个block�?/p>
<p>对于RecordReader�Q�其实这个接口主要就是�ؓ了维护一�l?lt;K,V&gt;键值对�Q��Q何一个实��C��该接口的�cȝ��构造函数都需要是&#8220;(Configuration conf, Class&lt; ? extends InputSplit&gt; split)&#8221;的�Ş式，因�ؓ一个RecordReader是有针对性的�Q�就是针�Ҏ���U�split来进行的�Q�因此必���d��与某�U�split�l�定��h��。这个接口中最重要的方法就是next�Q�在利用next�q�行��d��K和V�Ӟ��需要先通过createKey和createValue来创建K和V的对象，然后再传�l�next作�ؓ参数�Q���得next对�Ş参中的数据成员进行修攏V�?/p>
<p>一个file�Q�FileStatus�Q�分成多个block存储�Q�BlockLocation[]�Q�，每个block都有固定的大���（file.getBlockSize()�Q�，然后计算出每个split所需的大���（computeSplitSize(goalSize, minSize, blockSize)�Q�，然后���长度�ؓlength�Q�file.getLen()�Q�的file分割为多个split�Q�最后一个不���一个split大小的部分单独�ؓ其分配一个split�Q�最后返回这个file分割的最�l�结果（return splits.toArray(new FileSplit[splits.size()])�Q��?/p>
<p>一个job�Q�会得到输入的文件�\径（conf.get("mapred.input.dir", "")�Q�，然后据此可以得到一个Path[]�Q�对于每个Path�Q�都可以得到一个fs�Q�FileSystem fs = p.getFileSystem(job)�Q�，然后再得��C��个FileStatus[]�Q�FileStatus[] matches = fs.globStatus(p, inputFilter)�Q�，再把里面的每个FileStatus拿出来，判断其是否�ؓdir�Q�如果是的话���FileStatus stat:fs.listStatus(globStat.getPath(), inputFilter)�Q�然后再���stat加入到最�l�的�l�果集中result�Q�如果是文�g的话�Q�那���q��接加入到�l�果集中。说得简�z�一些，���是一个job会得到input.dir中的所有文�Ӟ��每个文�g都用FileStatus来记录�?/p>
<p>MultiFileSplit的官�Ҏ���q�是&#8220;A sub-collection of input files. Unlike {@link FileSplit}, MultiFileSplit class does not represent a split of a file, but a split of input files into smaller sets. The atomic unit of split is a file.&#8221;�Q�一个MultiFileSplit中含有多个小文�g�Q�每个文件应该只隶属于一个block�Q�然后getLocations���p��回所有小文�g对应的block的getHosts�Q�getLength�q�回所有文件的大小��d���?/p>
<p>对于MultiFileInputFormat�Q�它的getSplits�q�回的是一个MultiFileSplit的集合，也就是一个个的小文�g���，举个���单的例子��׃��很清楚了�Q�假定这个job中有5个小文�g�Q�大���分别�ؓ2�Q?�Q?�Q?�Q?�Q�假定我们期望split的��L��目�ؓ3的话�Q�先���出个double avgLengthPerSplit = ((double)totLength) / numSplits�Q�结果应该�ؓ5�Q�然后再切分�Q�因此得到的三个文�g����ؓ�Q�{文�g1�?}、{文�g3}、{文�g4�?}。如果这五个文�g的大���分别�ؓ2�Q?�Q?�Q?�Q?�Q�那么应该得到四个文件簇为：{文�g1}、{文�g2}、{文�g3�?}、{文�g5}。此外，�q�个�cȝ��getRecordReader依然是个abstract的方法，因此其子�c�d�����d��实现�q�个函数�?/p>
<img src ="http://www.aygfsteel.com/bacoo/aggbug/250221.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.aygfsteel.com/bacoo/" target="_blank">so true</a> 2009-01-07 09:40 <a href="http://www.aygfsteel.com/bacoo/archive/2009/01/07/250221.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>配置分布式hadoop时ssh斚w��该注意的事项</title><link>http://www.aygfsteel.com/bacoo/archive/2008/11/15/240625.html</link><dc:creator>so true</dc:creator><author>so true</author><pubDate>Fri, 14 Nov 2008 17:25:00 GMT</pubDate><guid>http://www.aygfsteel.com/bacoo/archive/2008/11/15/240625.html</guid><wfw:comment>http://www.aygfsteel.com/bacoo/comments/240625.html</wfw:comment><comments>http://www.aygfsteel.com/bacoo/archive/2008/11/15/240625.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.aygfsteel.com/bacoo/comments/commentRss/240625.html</wfw:commentRss><trackback:ping>http://www.aygfsteel.com/bacoo/services/trackbacks/240625.html</trackback:ping><description><![CDATA[<p>配置ssh无密码访问：<br />
比如�Q�A是server�Q�B是client�Q�现在B希望通过ssh无密码访问A�Q�那么就需要把B的公匙放到A的authorized_keys文�g中�?/p>
<p>1。首先需要A支持�q�种讉K��模式�Q?br />
配置A�?etc/ssh/sshd_config�Q�将�q�两��设�|�如下：<br />
RSAAuthentication yes<br />
PubkeyAuthentication yes</p>
<p>2。B生��id_rsa.pub�Q��ƈ���这个文件中的内�Ҏ���l�用&#8220;&gt;&gt;&#8221;��d��到A的authorized_keys文�g末尾�?/p>
<p>3。在B上，ssh A的ip/A的hostname���可以实现无密码登陆A�?/p>
<p>但是�q�么做是有前提的�Q�很多�h都忽略了�q�个前提�Q�导致费了很多周折都没有成功�Q�就像我似的�Q�我���p��了很多时间才扑ֈ�问题所在�?br />
因�ؓA或B机器里都有很多个账户�Q�在B上键入ssh命��o后，我们�q�没有制定连接到A上的那个帐户�Q�那么这里面默认的潜规则是什么呢�Q�就是你在B上ssh�Ӟ��当前使用的那个帐��P��假如名字是haha�Q�就会作��Z��期待�q�接到A上的帐户�Q�我们可以显�C�的通过ssh -l haha [hostname]或者ssh haha@[hostname]�q�种方式来连接到A上的haha帐户�Q�如果用隐士规则的话�Q�那么系�l�就是依据你在B上当前��用的帐户来作为A上被�q�接的帐戗��?br />
因此�Q�要实现无密码访问的前提���是�Q�A和B上有同样的帐户名�U�ͼ�完全一��_��包括大小写。（我就很郁��P��因�ؓ我在windows下用cygwin和一个linux机器�q�接�Q�windows下的帐户�W�一个字母大写了�Q�而linux的帐��L���W�一个字母是���写的，��D��我费了很长时间都没有发现问题症结所在）。其实，�q�也���是��Z��么在配置hadoop分布式计���时�Q�必��要求的每个机器上都必须有一个完全一��L��用户名�?/p>
<p>既然说到了后面的�q�些注意事项�Q�那么也要提醒大�Ӟ��在上面给出的三个步骤中的�W?步，必须是在�{�同的帐户下得到的id_rsa.pub文�g�Q�否则还是不行�?/p>
<img src ="http://www.aygfsteel.com/bacoo/aggbug/240625.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.aygfsteel.com/bacoo/" target="_blank">so true</a> 2008-11-15 01:25 <a href="http://www.aygfsteel.com/bacoo/archive/2008/11/15/240625.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item><item><title>一个简单shell脚本</title><link>http://www.aygfsteel.com/bacoo/archive/2008/11/15/240624.html</link><dc:creator>so true</dc:creator><author>so true</author><pubDate>Fri, 14 Nov 2008 17:23:00 GMT</pubDate><guid>http://www.aygfsteel.com/bacoo/archive/2008/11/15/240624.html</guid><wfw:comment>http://www.aygfsteel.com/bacoo/comments/240624.html</wfw:comment><comments>http://www.aygfsteel.com/bacoo/archive/2008/11/15/240624.html#Feedback</comments><slash:comments>0</slash:comments><wfw:commentRss>http://www.aygfsteel.com/bacoo/comments/commentRss/240624.html</wfw:commentRss><trackback:ping>http://www.aygfsteel.com/bacoo/services/trackbacks/240624.html</trackback:ping><description><![CDATA[<p>今天能写�����样一个shell脚本�Q�其实�ƈ没有费太大力气，因此�q�不是说我几�l�周折终有结果而兴奋，而是觉得自己现在�l�于可以�t�实下来做自己喜�Ƣ做的事情，能够专注的去学该学的东西而兴奋。之前学了很多杂七杂八的东西�Q�因为目标不明确�Q�很痛苦�Q�究其根本，是因��Z��知道自己���从事什么职业，只知道自己想从事IT�q�行�Q�但具体的工作方向却不知道，因此啥都要学习，�q�个�q�程对于我来说很痛苦。因为我是一个比较喜�Ƣ踏�t�实实做事的人，不做��׃��做，做就要做得很好。我之前看过一���关于论�q�程序员���躁的文章，写得太精彩了。而里面提到的很多���躁的做法都在我�w�上得到了印证，�q�让我很郁闷。现在，工作定了�Q�我知道该学点啥了，目标专注了，太美好了�?/p>
<p>借用Steven Jobs的一番话来说���是�Q?/p>
<p>The only way to be truely satisfied is to do what you believe is great work, and the only way to do great work is to love what you do!</p>
<p>我觉得一个�h能做到这一步，真的很幸���，自己��d��力，��L��搏，��d��现自��q��价��|��让自己对自己的表现满意，�q�是我经常对自己说的一句话�?/p>
<p>现在的我�Q�工作定了，奛_��也定了，也就是媳妇定了，我需要做的就是去奋斗�Q�去努力�Q�去拼搏�?/p>
<p>我很感谢自己能遇到这样一个媳妇，能支持我�Q�关心我�Q�我不知道自�׃��后会不会很成功，但是我知道有了这个好内柱�Q�我做什么都�t�实。我知道�Q�有了她�Q�我太幸���，我也一定会带给她幸���的�Q�I promise!</p>
<p>&nbsp;</p>
<p>好了�Q�下面就把代码脓出来吧，呵呵�Q?/p>
<p>#!/bin/sh</p>
<p>cd /hadoop/logs</p>
<p>var="`ls *.log`"<br />
cur=""<br />
name=""<br />
file=log_name.txt</p>
<p>if [ -e $file ]; then<br />
&nbsp;rm $file<br />
fi</p>
<p>for cur in $var<br />
do<br />
&nbsp;name=`echo $cur | cut -d'-' -f3`<br />
&nbsp;<br />
&nbsp;#cat $cur | grep ^2008 | awk '{print $0 " [`echo $name`]"}' &gt;&gt; $file<br />
&nbsp;cat $cur | grep ^2008 | sed "s/^.*$/&amp;[$name]/" &gt;&gt; $file<br />
&nbsp;#awk '{print $0 " [`echo $name`]"}' &gt;&gt; $file<br />
done</p>
<p>cp $file __temp.txt <br />
sort __temp.txt &gt;$file<br />
rm __temp.txt</p>
<p>�q�行的结果是�Q?/p>
<p>2008-11-14 10:08:47,671 INFO org.apache.hadoop.dfs.NameNode: STARTUP_MSG: [namenode]<br />
2008-11-14 10:08:48,140 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=NameNode, port=9000[namenode]<br />
2008-11-14 10:08:48,171 INFO org.apache.hadoop.dfs.NameNode: Namenode up at: bacoo/192.168.1.34:9000[namenode]<br />
2008-11-14 10:08:48,171 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=NameNode, sessionId=null[namenode]<br />
2008-11-14 10:08:48,234 INFO org.apache.hadoop.dfs.NameNodeMetrics: Initializing NameNodeMeterics using context object:org.apache.hadoop.metrics.spi.NullContext[namenode]<br />
2008-11-14 10:08:48,875 INFO org.apache.hadoop.dfs.FSNamesystemMetrics: Initializing FSNamesystemMeterics using context object:org.apache.hadoop.metrics.spi.NullContext[namenode]<br />
2008-11-14 10:08:48,875 INFO org.apache.hadoop.fs.FSNamesystem: fsOwner=Zhaoyb,None,root,Administrators,Users,Debugger,Users[namenode]<br />
2008-11-14 10:08:48,875 INFO org.apache.hadoop.fs.FSNamesystem: isPermissionEnabled=true[namenode]<br />
2008-11-14 10:08:48,875 INFO org.apache.hadoop.fs.FSNamesystem: supergroup=supergroup[namenode]<br />
2008-11-14 10:08:48,890 INFO org.apache.hadoop.fs.FSNamesystem: Registered FSNamesystemStatusMBean[namenode]<br />
2008-11-14 10:08:48,953 INFO org.apache.hadoop.dfs.Storage: Edits file edits of size 4 edits # 0 loaded in 0 seconds.[namenode]<br />
2008-11-14 10:08:48,953 INFO org.apache.hadoop.dfs.Storage: Image file of size 80 loaded in 0 seconds.[namenode]<br />
2008-11-14 10:08:48,953 INFO org.apache.hadoop.dfs.Storage: Number of files = 0[namenode]<br />
2008-11-14 10:08:48,953 INFO org.apache.hadoop.dfs.Storage: Number of files under construction = 0[namenode]<br />
2008-11-14 10:08:48,953 INFO org.apache.hadoop.fs.FSNamesystem: Finished loading FSImage in 657 msecs[namenode]<br />
2008-11-14 10:08:49,000 INFO org.apache.hadoop.dfs.StateChange: STATE* Leaving safe mode after 0 secs.[namenode]<br />
2008-11-14 10:08:49,000 INFO org.apache.hadoop.dfs.StateChange: STATE* Network topology has 0 racks and 0 datanodes[namenode]<br />
2008-11-14 10:08:49,000 INFO org.apache.hadoop.dfs.StateChange: STATE* UnderReplicatedBlocks has 0 blocks[namenode]<br />
2008-11-14 10:08:49,609 INFO org.mortbay.util.Credential: Checking Resource aliases[namenode]<br />
2008-11-14 10:08:50,015 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4[namenode]<br />
2008-11-14 10:08:50,015 INFO org.mortbay.util.Container: Started HttpContext[/logs,/logs][namenode]<br />
2008-11-14 10:08:50,015 INFO org.mortbay.util.Container: Started HttpContext[/static,/static][namenode]<br />
2008-11-14 10:08:54,656 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@17f11fb[namenode]<br />
2008-11-14 10:08:55,453 INFO org.mortbay.util.Container: Started WebApplicationContext[/,/][namenode]<br />
2008-11-14 10:08:55,468 INFO org.apache.hadoop.fs.FSNamesystem: Web-server up at: 0.0.0.0:50070[namenode]<br />
2008-11-14 10:08:55,468 INFO org.mortbay.http.SocketListener: Started SocketListener on 0.0.0.0:50070[namenode]<br />
2008-11-14 10:08:55,468 INFO org.mortbay.util.Container: Started org.mortbay.jetty.Server@61a907[namenode]<br />
2008-11-14 10:08:55,484 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting[namenode]<br />
2008-11-14 10:08:55,484 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 9000: starting[namenode]<br />
2008-11-14 10:08:55,515 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 9000: starting[namenode]<br />
2008-11-14 10:08:55,515 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 9000: starting[namenode]<br />
2008-11-14 10:08:55,515 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 9000: starting[namenode]<br />
2008-11-14 10:08:55,515 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9000: starting[namenode]<br />
2008-11-14 10:08:55,515 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 9000: starting[namenode]<br />
2008-11-14 10:08:55,531 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 9000: starting[namenode]<br />
2008-11-14 10:08:55,531 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 9000: starting[namenode]<br />
2008-11-14 10:08:55,531 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 9000: starting[namenode]<br />
2008-11-14 10:08:55,531 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 9000: starting[namenode]<br />
2008-11-14 10:08:55,531 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 9000: starting[namenode]<br />
2008-11-14 10:08:56,015 INFO org.apache.hadoop.dfs.NameNode.Secondary: STARTUP_MSG: [secondarynamenode]<br />
2008-11-14 10:08:56,156 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=SecondaryNameNode, sessionId=null[secondarynamenode]<br />
2008-11-14 10:08:56,468 WARN org.apache.hadoop.dfs.Storage: Checkpoint directory \tmp\hadoop-SYSTEM\dfs\namesecondary is added.[secondarynamenode]<br />
2008-11-14 10:08:56,546 INFO org.mortbay.util.Credential: Checking Resource aliases[secondarynamenode]<br />
2008-11-14 10:08:56,609 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4[secondarynamenode]<br />
2008-11-14 10:08:56,609 INFO org.mortbay.util.Container: Started HttpContext[/logs,/logs][secondarynamenode]<br />
2008-11-14 10:08:56,609 INFO org.mortbay.util.Container: Started HttpContext[/static,/static][secondarynamenode]<br />
2008-11-14 10:08:56,953 INFO org.mortbay.jetty.servlet.XMLConfiguration: No WEB-INF/web.xml in file:/E:/cygwin/hadoop/webapps/secondary. Serving files and default/dynamic servlets only[secondarynamenode]<br />
2008-11-14 10:08:56,953 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@b1a4e2[secondarynamenode]<br />
2008-11-14 10:08:57,062 INFO org.mortbay.util.Container: Started WebApplicationContext[/,/][secondarynamenode]<br />
2008-11-14 10:08:57,078 INFO org.apache.hadoop.dfs.NameNode.Secondary: Secondary Web-server up at: 0.0.0.0:50090[secondarynamenode]<br />
2008-11-14 10:08:57,078 INFO org.mortbay.http.SocketListener: Started SocketListener on 0.0.0.0:50090[secondarynamenode]<br />
2008-11-14 10:08:57,078 INFO org.mortbay.util.Container: Started org.mortbay.jetty.Server@18a8ce2[secondarynamenode]<br />
2008-11-14 10:08:57,078 WARN org.apache.hadoop.dfs.NameNode.Secondary: Checkpoint Period&nbsp;&nbsp; :3600 secs (60 min)[secondarynamenode]<br />
2008-11-14 10:08:57,078 WARN org.apache.hadoop.dfs.NameNode.Secondary: Log Size Trigger&nbsp;&nbsp;&nbsp; :67108864 bytes (65536 KB)[secondarynamenode]<br />
2008-11-14 10:08:59,828 INFO org.apache.hadoop.mapred.JobTracker: STARTUP_MSG: [jobtracker]<br />
2008-11-14 10:09:00,015 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=JobTracker, port=9001[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 9001: starting[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 9001: starting[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 9001: starting[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 9001: starting[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 9001: starting[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 9001: starting[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 9001: starting[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 9001: starting[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 9001: starting[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 9001: starting[jobtracker]<br />
2008-11-14 10:09:00,031 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 9001: starting[jobtracker]<br />
2008-11-14 10:09:00,125 INFO org.mortbay.util.Credential: Checking Resource aliases[jobtracker]<br />
2008-11-14 10:09:01,703 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4[jobtracker]<br />
2008-11-14 10:09:01,703 INFO org.mortbay.util.Container: Started HttpContext[/logs,/logs][jobtracker]<br />
2008-11-14 10:09:01,703 INFO org.mortbay.util.Container: Started HttpContext[/static,/static][jobtracker]<br />
2008-11-14 10:09:02,312 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@1cd280b[jobtracker]<br />
2008-11-14 10:09:08,359 INFO org.mortbay.util.Container: Started WebApplicationContext[/,/][jobtracker]<br />
2008-11-14 10:09:08,375 INFO org.apache.hadoop.mapred.JobTracker: JobTracker up at: 9001[jobtracker]<br />
2008-11-14 10:09:08,375 INFO org.apache.hadoop.mapred.JobTracker: JobTracker webserver: 50030[jobtracker]<br />
2008-11-14 10:09:08,375 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=[jobtracker]<br />
2008-11-14 10:09:08,375 INFO org.mortbay.http.SocketListener: Started SocketListener on 0.0.0.0:50030[jobtracker]<br />
2008-11-14 10:09:08,375 INFO org.mortbay.util.Container: Started org.mortbay.jetty.Server@16a9b9c[jobtracker]<br />
2008-11-14 10:09:12,984 INFO org.apache.hadoop.mapred.JobTracker: Starting RUNNING[jobtracker]<br />
2008-11-14 10:09:56,894 INFO org.apache.hadoop.dfs.DataNode: STARTUP_MSG: [datanode]<br />
2008-11-14 10:10:02,516 INFO org.apache.hadoop.mapred.TaskTracker: STARTUP_MSG: [tasktracker]<br />
2008-11-14 10:10:08,768 INFO org.apache.hadoop.dfs.Storage: Formatting ...[datanode]<br />
2008-11-14 10:10:08,768 INFO org.apache.hadoop.dfs.Storage: Storage directory /hadoop/hadoopfs/data is not formatted.[datanode]<br />
2008-11-14 10:10:11,343 INFO org.apache.hadoop.dfs.DataNode: Registered FSDatasetStatusMBean[datanode]<br />
2008-11-14 10:10:11,347 INFO org.apache.hadoop.dfs.DataNode: Opened info server at 50010[datanode]<br />
2008-11-14 10:10:11,352 INFO org.apache.hadoop.dfs.DataNode: Balancing bandwith is 1048576 bytes/s[datanode]<br />
2008-11-14 10:10:16,430 INFO org.mortbay.util.Credential: Checking Resource aliases[tasktracker]<br />
2008-11-14 10:10:17,976 INFO org.mortbay.util.Credential: Checking Resource aliases[datanode]<br />
2008-11-14 10:10:20,068 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4[datanode]<br />
2008-11-14 10:10:20,089 INFO org.mortbay.util.Container: Started HttpContext[/logs,/logs][datanode]<br />
2008-11-14 10:10:20,089 INFO org.mortbay.util.Container: Started HttpContext[/static,/static][datanode]<br />
2008-11-14 10:10:20,725 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4[tasktracker]<br />
2008-11-14 10:10:20,727 INFO org.mortbay.util.Container: Started HttpContext[/logs,/logs][tasktracker]<br />
2008-11-14 10:10:20,727 INFO org.mortbay.util.Container: Started HttpContext[/static,/static][tasktracker]<br />
2008-11-14 10:10:27,078 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/localhost[jobtracker]<br />
2008-11-14 10:10:32,171 INFO org.apache.hadoop.dfs.StateChange: BLOCK* NameSystem.registerDatanode: node registration from 192.168.1.167:50010 storage DS-1556534590-127.0.0.1-50010-1226628640386[namenode]<br />
2008-11-14 10:10:32,187 INFO org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/192.168.1.167:50010[namenode]<br />
2008-11-14 10:13:57,171 WARN org.apache.hadoop.dfs.Storage: Checkpoint directory \tmp\hadoop-SYSTEM\dfs\namesecondary is added.[secondarynamenode]<br />
2008-11-14 10:13:57,187 INFO org.apache.hadoop.fs.FSNamesystem: Number of transactions: 5 Total time for transactions(ms): 0 Number of syncs: 3 SyncTimes(ms): 4125 [namenode]<br />
2008-11-14 10:13:57,187 INFO org.apache.hadoop.fs.FSNamesystem: Roll Edit Log from 192.168.1.34[namenode]<br />
2008-11-14 10:13:57,953 INFO org.apache.hadoop.dfs.NameNode.Secondary: Downloaded file fsimage size 80 bytes.[secondarynamenode]<br />
2008-11-14 10:13:57,968 INFO org.apache.hadoop.dfs.NameNode.Secondary: Downloaded file edits size 288 bytes.[secondarynamenode]<br />
2008-11-14 10:13:58,593 INFO org.apache.hadoop.fs.FSNamesystem: fsOwner=Zhaoyb,None,root,Administrators,Users,Debugger,Users[secondarynamenode]<br />
2008-11-14 10:13:58,593 INFO org.apache.hadoop.fs.FSNamesystem: isPermissionEnabled=true[secondarynamenode]<br />
2008-11-14 10:13:58,593 INFO org.apache.hadoop.fs.FSNamesystem: supergroup=supergroup[secondarynamenode]<br />
2008-11-14 10:13:58,640 INFO org.apache.hadoop.dfs.Storage: Edits file edits of size 288 edits # 5 loaded in 0 seconds.[secondarynamenode]<br />
2008-11-14 10:13:58,640 INFO org.apache.hadoop.dfs.Storage: Number of files = 0[secondarynamenode]<br />
2008-11-14 10:13:58,640 INFO org.apache.hadoop.dfs.Storage: Number of files under construction = 0[secondarynamenode]<br />
2008-11-14 10:13:58,718 INFO org.apache.hadoop.dfs.Storage: Image file of size 367 saved in 0 seconds.[secondarynamenode]<br />
2008-11-14 10:13:58,796 INFO org.apache.hadoop.fs.FSNamesystem: Number of transactions: 0 Total time for transactions(ms): 0 Number of syncs: 0 SyncTimes(ms): 0 [secondarynamenode]<br />
2008-11-14 10:13:58,921 INFO org.apache.hadoop.dfs.NameNode.Secondary: Posted URL 0.0.0.0:50070putimage=1&amp;port=50090&amp;machine=192.168.1.34&amp;token=-16:145044639:0:1226628551796:1226628513000[secondarynamenode]<br />
2008-11-14 10:13:59,078 INFO org.apache.hadoop.fs.FSNamesystem: Number of transactions: 0 Total time for transactions(ms): 0 Number of syncs: 0 SyncTimes(ms): 0 [namenode]<br />
2008-11-14 10:13:59,078 INFO org.apache.hadoop.fs.FSNamesystem: Roll FSImage from 192.168.1.34[namenode]<br />
2008-11-14 10:13:59,265 WARN org.apache.hadoop.dfs.NameNode.Secondary: Checkpoint done. New Image Size: 367[secondarynamenode]<br />
2008-11-14 10:29:02,171 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 0 time(s).[secondarynamenode]<br />
2008-11-14 10:29:04,187 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 1 time(s).[secondarynamenode]<br />
2008-11-14 10:29:06,109 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 2 time(s).[secondarynamenode]<br />
2008-11-14 10:29:08,015 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 3 time(s).[secondarynamenode]<br />
2008-11-14 10:29:10,031 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 4 time(s).[secondarynamenode]<br />
2008-11-14 10:29:11,937 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 5 time(s).[secondarynamenode]<br />
2008-11-14 10:29:13,843 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 6 time(s).[secondarynamenode]<br />
2008-11-14 10:29:15,765 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 7 time(s).[secondarynamenode]<br />
2008-11-14 10:29:17,671 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 8 time(s).[secondarynamenode]<br />
2008-11-14 10:29:19,593 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 9 time(s).[secondarynamenode]<br />
2008-11-14 10:29:21,078 ERROR org.apache.hadoop.dfs.NameNode.Secondary: Exception in doCheckpoint: [secondarynamenode]<br />
2008-11-14 10:29:21,171 ERROR org.apache.hadoop.dfs.NameNode.Secondary: java.io.IOException: Call failed on local exception[secondarynamenode]<br />
2008-11-14 10:34:23,156 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 0 time(s).[secondarynamenode]<br />
2008-11-14 10:34:25,078 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 1 time(s).[secondarynamenode]<br />
2008-11-14 10:34:27,078 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 2 time(s).[secondarynamenode]<br />
2008-11-14 10:34:29,078 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 3 time(s).[secondarynamenode]<br />
2008-11-14 10:34:31,000 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 4 time(s).[secondarynamenode]<br />
2008-11-14 10:34:32,906 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 5 time(s).[secondarynamenode]<br />
2008-11-14 10:34:34,921 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 6 time(s).[secondarynamenode]<br />
2008-11-14 10:34:36,828 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 7 time(s).[secondarynamenode]<br />
2008-11-14 10:34:38,640 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 8 time(s).[secondarynamenode]<br />
2008-11-14 10:34:40,546 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Bacoo/192.168.1.34:9000. Already tried 9 time(s).[secondarynamenode]<br />
2008-11-14 10:34:41,468 ERROR org.apache.hadoop.dfs.NameNode.Secondary: Exception in doCheckpoint: [secondarynamenode]<br />
2008-11-14 10:34:41,468 ERROR org.apache.hadoop.dfs.NameNode.Secondary: java.io.IOException: Call failed on local exception[secondarynamenode]<br />
2008-11-14 10:38:43,359 INFO org.apache.hadoop.dfs.NameNode.Secondary: SHUTDOWN_MSG: [secondarynamenode]<br />
</p>
我相信，�q�样���可以按照时间的��序�Q�把生��的日志好好理一遍顺序了�Q�而且每一个步骤后面还都有了各自对应的node�c�d���?
  <img src ="http://www.aygfsteel.com/bacoo/aggbug/240624.html" width = "1" height = "1" /><br><br><div align=right><a style="text-decoration:none;" href="http://www.aygfsteel.com/bacoo/" target="_blank">so true</a> 2008-11-15 01:23 <a href="http://www.aygfsteel.com/bacoo/archive/2008/11/15/240624.html#Feedback" target="_blank" style="text-decoration:none;">发表评论</a></div>]]></description></item></channel></rss>                                     <footer>
<div class="friendship-link">
<a href="http://www.aygfsteel.com/" title="&#29408;&#29408;&#20037;&#20037;&#20122;&#27954;&#27431;&#32654;&#19987;&#21306;&#95;&#20013;&#25991;&#23383;&#24149;&#20122;&#27954;&#32508;&#21512;&#20037;&#20037;&#50;&#48;&#50;&#95;&#22269;&#20135;&#31934;&#21697;&#20122;&#27954;&#31532;&#20116;&#21306;&#22312;&#32447;&#95;&#26085;&#26412;&#20813;&#36153;&#32593;&#31449;&#35270;&#39057;">&#29408;&#29408;&#20037;&#20037;&#20122;&#27954;&#27431;&#32654;&#19987;&#21306;&#95;&#20013;&#25991;&#23383;&#24149;&#20122;&#27954;&#32508;&#21512;&#20037;&#20037;&#50;&#48;&#50;&#95;&#22269;&#20135;&#31934;&#21697;&#20122;&#27954;&#31532;&#20116;&#21306;&#22312;&#32447;&#95;&#26085;&#26412;&#20813;&#36153;&#32593;&#31449;&#35270;&#39057;</a>

</div>

</footer>


��վ֩���ģ�壺
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">Ӣ����</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">�ʳ�</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">̩˳��</a>|
<a href="http://" target="_blank">������������</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">����</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">��ͼʲ��</a>|
<a href="http://" target="_blank">�ں���</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">�����</a>|
<a href="http://" target="_blank">��������</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">��ɳ��</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">տ����</a>|
<a href="http://" target="_blank">��ɽ��</a>|
<a href="http://" target="_blank">�ﶫ��</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">�Ĵ�ʡ</a>|
<a href="http://" target="_blank">����</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">����</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">�¸���</a>|
<a href="http://" target="_blank">�Ƹ���</a>|
<a href="http://" target="_blank">������</a>|
<a href="http://" target="_blank">�ٹ���</a>|
<a href="http://" target="_blank">��͡��</a>|
<a href="http://" target="_blank">��ʯ��</a>|
<script>
(function(){
    var bp = document.createElement('script');
    var curProtocol = window.location.protocol.split(':')[0];
    if (curProtocol === 'https') {
        bp.src = 'https://zz.bdstatic.com/linksubmit/push.js';
    }
    else {
        bp.src = 'http://push.zhanzhang.baidu.com/push.js';
    }
    var s = document.getElementsByTagName("script")[0];
    s.parentNode.insertBefore(bp, s);
})();
</script>
</body>