The NoteBook of EricKong

            BlogJava :: 首頁 :: 聯系 :: 聚合  :: 管理
            611 Posts :: 1 Stories :: 190 Comments :: 0 Trackbacks

          本文是參考logstash官方文檔實踐的筆記,搭建環境和所需組件如下:

          • Redhat 5.7 64bit / CentOS 5.x
          • JDK 1.6.0_45
          • logstash 1.3.2 (內帶kibana)
          • elasticsearch 0.90.10
          • redis 2.8.4

          搭建的集中式日志分析平臺流程如下: 

          安裝logstash+kibana+elasticsearch+redis搭建日志分析平臺 - 傲風 - 0與1構筑世界,程序員創造時代

           

          elasticsearch

          1、下載elasticsearch。

          wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.10.tar.gz 

          2、解壓后,進入bin目錄。執行如下命令,讓elasticsearch以前臺方式啟動:

          ./elasticsearch -f 
          [2014-01-16 16:21:31,825][INFO ][node                     ] [Saint Elmo] version[0.90.10], pid[32269], build[0a5781f/2014-01-10T10:18:37Z] [2014-01-16 16:21:31,826][INFO ][node                     ] [Saint Elmo] initializing ... [2014-01-16 16:21:31,836][INFO ][plugins                  ] [Saint Elmo] loaded [], sites [] [2014-01-16 16:21:35,425][INFO ][node                     ] [Saint Elmo] initialized [2014-01-16 16:21:35,425][INFO ][node                     ] [Saint Elmo] starting ... [2014-01-16 16:21:35,578][INFO ][transport                ] [Saint Elmo] bound_address {inet[/0.0.0.0:9300]}, publish_address {inet[/10.0.2.15:9300]} 

          Redis

          1、其安裝方式可以參考我的另一篇文章Redis編譯安裝。

          2、進入其bin目錄,執行如下命令,使之在控制臺輸出debug信息:

          ./redis-server --loglevel verbose 
          [32470] 16 Jan 16:45:57.330 * The server is now ready to accept connections on port 6379 [32470] 16 Jan 16:45:57.330 - 0 clients connected (0 slaves), 283536 bytes in use 

          logstash日志生成器(shipper)

          1、新建一個配置文件:shipper.conf,其內容如下:

          input {     stdin {         type => "example"     } }  output {     stdout {         codec => rubydebug     }     redis {         host => "127.0.0.1"         port => 6379         data_type => "list"         key => "logstash"     } } 

          2、啟動shipper。執行如下命令:

          java -jar logstash-1.3.2-flatjar.jar agent -f shipper.conf  

          終端窗口將出現如下提示信息:

          Using milestone 2 output plugin 'redis'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.3.2/plugin-milestones {:level=>:warn} 

          然后在終端窗口直接按回車,將出現如下信息:

          {        "message" => "",       "@version" => "1",     "@timestamp" => "2014-01-16T08:15:19.400Z",           "type" => "example",           "host" => "redhat" } 

          這個json信息將發送給redis, 同時redis的終端窗口將出現類似下面的提示信息:

          [32470] 16 Jan 17:09:23.604 - Accepted 127.0.0.1:44640 [32470] 16 Jan 17:09:27.127 - DB 0: 1 keys (0 volatile) in 4 slots HT. [32470] 16 Jan 17:09:27.127 - 1 clients connected (0 slaves), 304752 bytes in use 

          logstash日志索引器(indexer)

          1、新建一個配置文件:indexer.conf,其內容如下:

          input {   redis {     host => "127.0.0.1"     # these settings should match the output of the agent     data_type => "list"     key => "logstash"      # We use the 'json' codec here because we expect to read     # json events from redis.     codec => json   } }  output {   stdout { debug => true debug_format => "json"}    elasticsearch {     host => "127.0.0.1"   } } 

          2、啟動日志索引器。執行如下命令:

          java -jar logstash-1.3.2-flatjar.jar agent -f indexer.conf  

          終端窗口將出現如下提示信息:

          Using milestone 2 input plugin 'redis'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.3.2/plugin-milestones {:level=>:warn} You are using a deprecated config setting "debug_format" set in stdout. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future.  If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"debug_format", :plugin=>, :level=>:warn} 

          索引器從Redis接收到信息,在終端窗口會顯示類似如下的信息:

          {"message":"","@version":"1","@timestamp":"2014-01-16T17:10:03.831+08:00","type":"example","host":"redhat"}{"message":"","@version":"1","@timestamp":"2014-01-16T17:13:20.545+08:00","type":"example","host":"redhat"}{ 

          logstash WEB界面(kibana)

          1、啟動kibana。執行如下命令:

          java -jar logstash-1.3.2-flatjar.jar web 

          2、打開瀏覽器(須支持HTML5),輸入地址:http://127.0.0.1:9292/index.html#/dashboard/file/logstash.json。界面效果如下: 

          安裝logstash+kibana+elasticsearch+redis搭建日志分析平臺 - 傲風 - 0與1構筑世界,程序員創造時代
           

          參考資料

          posted on 2015-06-12 11:30 Eric_jiang 閱讀(486) 評論(0)  編輯  收藏 所屬分類: 日志管理
          主站蜘蛛池模板: 武清区| 郧西县| 岳西县| 洱源县| 昌江| 垦利县| 长治市| 梧州市| 江山市| 新和县| 河南省| 嫩江县| 益阳市| 梁山县| 桑日县| 尉犁县| 扎赉特旗| 乐至县| 犍为县| 藁城市| 忻州市| 江津市| 宜宾县| 天全县| 阳江市| 安龙县| 鹿邑县| 乐安县| 宜兴市| 呼玛县| 墨竹工卡县| 吴忠市| 凤台县| 怀柔区| 通海县| 新蔡县| 体育| 修文县| 泊头市| 大冶市| 巍山|