聶永的博客

開源的消息隊(duì)列已經(jīng)很多了，但大部分很重，實(shí)際環(huán)境下，很多可能只是使用到了一點(diǎn)功能而已，殺雞使用牛刀，著實(shí)有些浪費(fèi)了。很多時(shí)候，我們只想要一片綠葉，但它們給了我們整個(gè)的春天，很難消化。本著DIR精神， 也琢磨了一個(gè)超級(jí)簡單的隊(duì)列實(shí)現(xiàn)。

說是超級(jí)簡單，嗯，絕對是超級(jí)簡單，隊(duì)列的存儲(chǔ)采用Redis進(jìn)行持久化存儲(chǔ)，采用Netty提供HTTP方式的隊(duì)列的出/入。Redis的客戶端采用的Jedis。然后呢，然后就沒了啊。

Redis內(nèi)置訂閱發(fā)布模型(Publish/Subscribe),其缺陷是，不存儲(chǔ)，一旦訂閱者斷線，將無法接收到消息，再次連接上之后，在斷線期間發(fā)布者發(fā)布的消息都是無法獲取到的。只能采用list數(shù)組實(shí)現(xiàn)，采用rpush/lpop組合命令來實(shí)現(xiàn)先進(jìn)先出的隊(duì)列模型，當(dāng)然redis也提供了阻塞版本的blpush/brpush/blpop/brpop等，就看我們實(shí)際環(huán)境下如何使用了。

JAVA客戶端使用Jedis，提供接口也很豐富。但要注意的是，需要使用連接池，否則在大數(shù)量的情況下，有可能jedis的連接不夠用。

private static JedisPool pool;
 static {
  ResourceBundle bundle = ResourceBundle.getBundle("redis");
  if (bundle == null) {
   throw new IllegalArgumentException(
     "cannot find the SignVerProp.properties");
  }
  JedisPoolConfig config = new JedisPoolConfig();
  config.setMaxActive(Integer.valueOf(bundle
    .getString("redis.pool.maxActive")));
  config.setMaxIdle(Integer.valueOf(bundle
    .getString("redis.pool.maxIdle")));
  config.setMaxWait(Integer.valueOf(bundle
    .getString("redis.pool.maxWait")));
  pool = new JedisPool(config, bundle.getString("redis.server"),
    Integer.valueOf(bundle.getString("redis.port")));
 }

很成熟的NIO框架，用它來提供HTTP方式的隊(duì)列的出入。嗯，目前只提供HTTP方式的入隊(duì)列，出隊(duì)列等。
HTTP形式為：http://服務(wù)器名稱:端口/隊(duì)列操作原語/隊(duì)列名稱?msg=消息內(nèi)容
隊(duì)列操作原語，只有g(shù)et和put；get為出隊(duì)列，put為入隊(duì)列。
返回為json： {s:0, m:'錯(cuò)誤消息/消息內(nèi)容'}
s為success的縮寫，值為0，意味著false，1對應(yīng)成功。 m為message的縮寫，錯(cuò)誤消息/消息內(nèi)容，添加消息時(shí)，會(huì)返回插入消息對應(yīng)的數(shù)目 默認(rèn)采用UTF-8。

入隊(duì)列：http://localhost:8080/put/demo?msg=消息內(nèi)容
出隊(duì)列：http://localhost:8080/get/demo HTTP方式，特別適合局域網(wǎng)之間，消息數(shù)據(jù)的推送。

采用了QueueDaemon封裝了netty代碼： 調(diào)用很簡單，程序啟動(dòng)的入口： 而HTTP方式隊(duì)列請求處理器為HttpRequestHandler：

這個(gè)很簡單，直接采用Jedis客戶端即可，這里一直占用一個(gè)連接，不釋放。

本機(jī)配置：Pentium(R) Dual-Core CPU E5200 2.50GHz，2.00 GB內(nèi)存，Windows XP系統(tǒng)，redis-2.4.5-win32 32版本（非linux版本），都在一臺(tái)機(jī)器上運(yùn)行。 插入 512 bytes 文本消息隊(duì)列: 1781.24 requests/sec

D:\Apache2.2\bin>ab -c 10 -n 100000 "http://localhost:8080/put/demo?msg=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
Server Software:
Server Hostname:        localhost
Server Port:            8080
Document Path:          /put/demo?msg=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Document Length:        12 bytes
Concurrency Level:      10
Time taken for tests:   56.141 seconds
Complete requests:      100000
Failed requests:        99991
   (Connect: 0, Receive: 0, Length: 99991, Exceptions: 0)
Write errors:           0
Total transferred:      8188895 bytes
HTML transferred:       1588895 bytes
Requests per second:    1781.24 [#/sec] (mean)
Time per request:       5.614 [ms] (mean)
Time per request:       0.561 [ms] (mean, across all concurrent requests)
Transfer rate:          142.45 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.7      0      16
Processing:     0    5  10.7      0     781
Waiting:        0    5  10.4      0     766
Total:          0    6  10.8      0     781
Percentage of the requests served within a certain time (ms)
  50%      0
  66%     16
  75%     16
  80%     16
  90%     16
  95%     16
  98%     16
  99%     16
 100%    781 (longest request)

插入 512 bytes 文本消息隊(duì)列(添加Keep-Alive支持): 1875.18 requests/sec

D:\Apache2.2\bin>ab -k -c 10 -n 100000 "http://localhost:8080/put/demo?msg=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
Server Software:
Server Hostname:        localhost
Server Port:            8080
Document Path:          /put/demo?msg=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Document Length:        17 bytes
Concurrency Level:      10
Time taken for tests:   53.328 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    0
Total transferred:      8300000 bytes
HTML transferred:       1700000 bytes
Requests per second:    1875.18 [#/sec] (mean)
Time per request:       5.333 [ms] (mean)
Time per request:       0.533 [ms] (mean, across all concurrent requests)
Transfer rate:          151.99 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.5      0      16
Processing:     0    5   7.8      0     203
Waiting:        0    5   7.8      0     203
Total:          0    5   7.9      0     203
Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%     16
  80%     16
  90%     16
  95%     16
  98%     16
  99%     16
 100%    203 (longest request)

獲取 512 bytes 消息(With Keep-Alive): 1875.73 requests/sec

D:\Apache2.2\bin>ab -k -c 10 -n 100000 "http://localhost:8080/get/demo"
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
Server Software:
Server Hostname:        localhost
Server Port:            8080
Document Path:          /get/demo
Document Length:        523 bytes
Concurrency Level:      10
Time taken for tests:   53.313 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    0
Total transferred:      58900000 bytes
HTML transferred:       52300000 bytes
Requests per second:    1875.73 [#/sec] (mean)
Time per request:       5.331 [ms] (mean)
Time per request:       0.533 [ms] (mean, across all concurrent requests)
Transfer rate:          1078.91 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.9      0      16
Processing:     0    5   7.5      0      94
Waiting:        0    4   6.9      0      94
Total:          0    5   7.6      0      94
Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%     16
  80%     16
  90%     16
  95%     16
  98%     16
  99%     16
 100%     94 (longest request)

1. 暫時(shí)對安全/授權(quán)沒有支持，這個(gè)其實(shí)很容易實(shí)現(xiàn)
2. 隊(duì)列處理器很簡單，直接使用jedis即可
3. 對隊(duì)列數(shù)據(jù)進(jìn)行分片(sharding)，寫一個(gè)QueueService實(shí)現(xiàn)即可
4. 對Redis的內(nèi)存持久化不放心，采用diskstore存儲(chǔ)模式好了，在Redis中配置，不涉及到程序
5. 對隊(duì)列分布式存儲(chǔ)，寫一個(gè)QueueService實(shí)現(xiàn)，其它可不用變化
6. 不適合嚴(yán)格意義上的訂閱/發(fā)布模型，這里適合多個(gè)發(fā)布者/單個(gè)訂閱者環(huán)境
7. HTTP請求返回內(nèi)容為json格式，xml格式，暫時(shí)不需要
8. 局域網(wǎng)環(huán)境下，系統(tǒng)之間進(jìn)行消息的推送/通知，

項(xiàng)目下載地址：http://code.google.com/p/nettyqueue/ 參考資料：

1. httpsqs