锘??xml version="1.0" encoding="utf-8" standalone="yes"?> 鑳屾櫙 錛?/p> 銆銆鍦ㄤ嬌鐢ㄦ悳绱㈠紩鎿庡拰鐢?shù)鍟嗙殑鎼滅储鍔熻兘鏃跺Q屽ぇ瀹朵竴瀹氶亣鍒拌繃榪欐牱鐨勬儏鏅細(xì)鎴戞兂鎼滅儲(chǔ)鐢?shù)濯?jiǎng)“鍗佷簩鐢熻倴”錛屽彲涓嶅皬蹇冭緭鎴?#8220;鍗佷簩鐢熸晥”浜嗭紝涓嶇敤鎷呭績鎼滀笉鍒頒綘鎯寵鐨勭粨鏋滐紝鍥犱負(fù)寤虹珛鍦ㄥぇ鏁版嵁涓婄殑鎼滅儲(chǔ)寮曟搸浼?xì)甯綘鑷姩绾犻敊锛尀p繖涓緥瀛怗oogle鍜孊aidu榪斿洖緇欐垜鐨勫垎鍒槸錛?/p> 鏄劇ず浠ヤ笅鏌ヨ瀛楄瘝鐨勭粨鏋滐細(xì) 鍗佷簩鐢熻倴 鍜?span style="color: #ff6600;"> 鎮(zhèn)ㄨ鎵劇殑鏄笉鏄? 鍗佷簩鐢熻倴 錛屼粬浠兘鍋氬埌浜嗚嚜鍔ㄧ籂閿欙紝鍏充簬鑷姩綰犻敊鎴戜箣鍓嶄篃鍐欒繃涓綃囬檵鏂?/strong>錛屽綋鏃舵槸鑷繁瀹炵幇鐨凬-Gram妯″瀷,浣嗘槸鏁堟灉涓嶆槸澶ソ錛屼富瑕佹槸閽堝涓嶅悓鐨勮鏂欏簱綆楁硶鐨勭簿紜害鏄笉涓鏍風(fēng)殑錛屾垜鎯蟲崲涓畻娉曡瘯璇曠湅錛岀洰鍓嶄富嫻佺殑璁$畻涓查棿鐨勮窛紱?鐩稿弽鐨勶紝浣犱篃鍙互鐞嗚В涓虹浉浼煎害)鏄?span>Levenshtein錛屽綋瑕佸疄鐜版椂錛屽彂鐜發(fā)ucene宸茬粡鍋氫簡榪欎釜浜嬶紝閭e挶灝辯珯鍦ㄥ法浜虹殑鑲╄唨涓婃垚闀垮惂銆?/span> 寮曠敤鍖咃細(xì) 銆銆lucene-core-3.1.0.jar + lucene-spellchecker-3.1.0.jar,浣犲彲浠ュ湪榪欓噷寰楀埌 浣跨敤紺轟緥錛?/strong> 銆銆鍦ㄧ被SpellCorrector鐨刴ain鏂規(guī)硶涓姞鍏ヤ互涓嬩唬鐮?/strong> 濂戒簡錛屾帴涓嬫潵灝辯洿鎺ヨ繍琛屽惂錛屼緥濡傛垜鎼滅儲(chǔ)“鍗佷簩鐢熸晥”錛屽垯鎻愮ず璇存槸涓嶆槸瑕佹壘“鍗佷簩鐢熻倴” 緗戠粶涓婄殑Heritrix涓枃璧勬簮姣旇緝?yōu)畱锛屾暣鐞嗕竴涓嬶細(xì) 涓枃錛?/span> l 銆婂紑鍙戣嚜宸辯殑鎼滅儲(chǔ)寮曟搸 Lucene 2.0 + Heritrix銆嬩綔鑰呴偙鍝?/span>&絎︽粩婊旂殑BLOG http://lucenebook.spaces.live.com/ l 銆婂紑鍙戣嚜宸辯殑鎼滅儲(chǔ)寮曟搸 Lucene 2.0 + Heriterx銆?/span>絎崄绔犳墿灞?/span>Heritrix璇曡绔犺妭 (鍙互鑰冭檻寮鍙戠殑錛屾瘮杈冩湁鐢?/span>) http://book.csdn.net/bookfiles/312/10031212848.shtml l Heritrix絎旇 http://wiki.hoodong.com/wiki/jRwNBCFgWA1dYB0NC l Heritrix crawler vs Nutch crawler http://www.dbanotes.net/web/heritrix_crawler_vs_nutch_crawler.html l 澶╀笅緇村-鐖櫕紼嬪簭 http://www.allwiki.com/wiki/Heritrix#Heritrix.E7.9A.84.E5.B1.80.E9.99.90 鑻辨枃錛?/span> l Heritrix涓婚〉 l HTMLParser涓婚〉 鍏抽敭瀛楋細(xì)Heritrix 127.0.0.1 IP 涓繪満 Heritrix榛樿緇戝畾鐨?/span>IP鏄?/span>127.0.0.1銆?/span> 鍦?/span>org.archive.crawler.Heritrix涓?/span> … final private static Collection<String> LOCALHOST_ONLY = Collections.unmodifiableList(Arrays.asList(new String[] { "127.0.0.1" })); … private static Collection<String> guiHosts = LOCALHOST_ONLY; protected static String doCmdLineArgs(final String [] args) throws Exception { … // Now look at options passed. for (int i = 0; i < options.length; i++) { switch(options[i].getId()) { … case 'b': Heritrix.guiHosts = parseHosts(options[i].getValue()); break; … default: assert false: options[i].getId(); } } … } 棣栧厛瀹氫箟浜嗛粯璁?/span>IP錛?/span>127.0.0.1錛岀劧鍚庤祴緇?/span>guiHost涓繪満鍙橀噺銆傚綋鎸囧畾-b鎴?/span>--bind鍙傛暟鏃訛紝鎵嶄細(xì)鎶婃寚瀹氱殑IP璧嬬粰涓繪満鍙橀噺銆?/span> 鍙﹀錛屼腑闂磋繕鏈変竴姝ュ弬鏁板鐞嗭紝瀵逛簬--xxxx鍙傛暟浼?xì)铦{涓?/span>-x鐨勫艦寮忕粺涓澶勭悊錛屾墍浠?/span>--bind鍜?/span>-b鏈変竴鏍風(fēng)殑鏁堟灉銆?/span> 鍏抽敭瀛楋細(xì)Heritrix 鍚姩 鍙傛暟 bind admin properties Heritrix鐨勫惎鍔ㄥ弬鏁幫紝闄や簡--bind澶栵紝閮藉彲浠ュ湪heritrix.properties璁劇疆錛岃屼笉鐢ㄦ瘡嬈¢兘鍦ㄥ懡浠よ涓緭鍏ャ?/span> 濡傚父鐢ㄧ殑--port, --admin絳夈?/span> heritrix.cmdline.admin = admin:admin heritrix.cmdline.port = 8080 heritrix.cmdline.run = false heritrix.cmdline.nowui = false heritrix.cmdline.order = heritrix.cmdline.jmxserver = false heritrix.cmdline.jmxserver.port = 8081 鍏抽敭瀛楋細(xì)Heritrix 涓枃 涔辯爜 GB2312 Extractor 緇ф壙浠?/span>org.archive.crawler.extractor.Extractor鐨勫瓙綾伙紝鍦╡xtract鏂規(guī)硶涓彲浠ヤ粠鍙傛暟CrawlURI涓彇鍑鴻瑙f瀽鐨勫唴瀹廣?/span> curi.getHttpRecorder().getReplayCharSequence.toString() 鏈変腑鏂囨椂錛屼笉鍋氬鐞嗕細(xì)杈撳嚭涔辯爜銆傚彲浠ュ湪鍙栧埌鐨凥ttpRecorder鍚庤緗紪鐮侊細(xì) HttpRecorder hr = curi.getHttpRecorder(); if ( hr == null ) { throw new IOException( "Why is recorder null here?" ); } hr.setCharacterEncoding( "gb2312" ); cs = hr.getReplayCharSequence(); System.out.println( cs.toString() ); 鍘熸枃鍑鴻嚜錛?div>http://blog.chinaunix.net/uid-8464637-id-2461166.html
2 File dict = new File("");
3 Directory directory = FSDirectory.open(dict);
4
5 //瀹炰緥鍖栨嫾鍐欐鏌ュ櫒
6 SpellChecker sp = new SpellChecker(directory);
7
8
9 //鍒涘緩璇嶅吀
10 File dictionary = new File(SpellCorrecter.class.getResource("dictionary.txt").getFile());
11
12 //瀵硅瘝鍏歌繘琛岀儲(chǔ)寮?/span>
13 sp.indexDictionary(new PlainTextDictionary(dictionary));
14
15
16 //鏈夐敊鍒瓧鐨勬悳绱?/span>
17 String search = "闈炲父鍕挎壈";
18
19
20 //寤鴻涓暟錛岃繖閲屾垜鍙兂瑕佹渶鎺ヨ繎鐨勯偅涓涓紝浣犲彲浠ヨ緗垚鍒殑鏁板瓧錛屽3
21 int suggestionNumber = 1;
22
23
24 //鑾峰彇寤鴻鐨勫叧閿瓧
25 String[] suggestions = sp.suggestSimilar(search, suggestionNumber);
26
27
28 //鏄劇ず緇撴灉
29 System.out.println("鎼滅儲(chǔ):" + search);
30
31
32 for (String word : suggestions) {
33 System.out.println("浣犺鎵劇殑鏄笉鏄?" + word);
34 }
娉細(xì)榪欎箣鍓嶄綘闇瑕佹湁涓鏂欏簱錛屾垜榪欓噷鏄釜瀛樻斁姝g‘瑙嗛鍚嶇О鐨勬枃浠訛紝鏍煎紡濡備笅錛?/div>
椹伴绔炶墖鐜嬬浜岄儴
閽撻噾榫?br /> 娼囨箻璺竴鍙?br /> 鎴忛噷鎴忓絎簩瀛?br /> 鑽夊師鐙肩埖澹箰
鎷晳澶у叺鐟炴仼鍙傝冩枃鐚?/h3>
]]>Heritrix璧勬簮
Heritrix緇戝畾涓繪満IP
Heritrix鍚姩鍙傛暟
鍏充簬Heritrix鐨?/span>Extractor涓枃涔辯爜