聶永的博客

開源的消息隊列已經很多了，但大部分很重，實際環境下，很多可能只是使用到了一點功能而已，殺雞使用牛刀，著實有些浪費了。很多時候，我們只想要一片綠葉，但它們給了我們整個的春天，很難消化。本著DIR精神， 也琢磨了一個超級簡單的隊列實現。

說是超級簡單，嗯，絕對是超級簡單，隊列的存儲采用Redis進行持久化存儲，采用Netty提供HTTP方式的隊列的出/入。Redis的客戶端采用的Jedis。然后呢，然后就沒了啊。

Redis內置訂閱發布模型(Publish/Subscribe),其缺陷是，不存儲，一旦訂閱者斷線，將無法接收到消息，再次連接上之后，在斷線期間發布者發布的消息都是無法獲取到的。只能采用list數組實現，采用rpush/lpop組合命令來實現先進先出的隊列模型，當然redis也提供了阻塞版本的blpush/brpush/blpop/brpop等，就看我們實際環境下如何使用了。

JAVA客戶端使用Jedis，提供接口也很豐富。但要注意的是，需要使用連接池，否則在大數量的情況下，有可能jedis的連接不夠用。

private static JedisPool pool;
 static {
  ResourceBundle bundle = ResourceBundle.getBundle("redis");
  if (bundle == null) {
   throw new IllegalArgumentException(
     "cannot find the SignVerProp.properties");
  }
  JedisPoolConfig config = new JedisPoolConfig();
  config.setMaxActive(Integer.valueOf(bundle
    .getString("redis.pool.maxActive")));
  config.setMaxIdle(Integer.valueOf(bundle
    .getString("redis.pool.maxIdle")));
  config.setMaxWait(Integer.valueOf(bundle
    .getString("redis.pool.maxWait")));
  pool = new JedisPool(config, bundle.getString("redis.server"),
    Integer.valueOf(bundle.getString("redis.port")));
 }

很成熟的NIO框架，用它來提供HTTP方式的隊列的出入。嗯，目前只提供HTTP方式的入隊列，出隊列等。
HTTP形式為：http://服務器名稱:端口/隊列操作原語/隊列名稱?msg=消息內容
隊列操作原語，只有get和put；get為出隊列，put為入隊列。
返回為json： {s:0, m:'錯誤消息/消息內容'}
s為success的縮寫，值為0，意味著false，1對應成功。 m為message的縮寫，錯誤消息/消息內容，添加消息時，會返回插入消息對應的數目 默認采用UTF-8。

入隊列：http://localhost:8080/put/demo?msg=消息內容
出隊列：http://localhost:8080/get/demo HTTP方式，特別適合局域網之間，消息數據的推送。

采用了QueueDaemon封裝了netty代碼： 調用很簡單，程序啟動的入口： 而HTTP方式隊列請求處理器為HttpRequestHandler：

這個很簡單，直接采用Jedis客戶端即可，這里一直占用一個連接，不釋放。

本機配置：Pentium(R) Dual-Core CPU E5200 2.50GHz，2.00 GB內存，Windows XP系統，redis-2.4.5-win32 32版本（非linux版本），都在一臺機器上運行。 插入 512 bytes 文本消息隊列: 1781.24 requests/sec

D:\Apache2.2\bin>ab -c 10 -n 100000 "http://localhost:8080/put/demo?msg=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
Server Software:
Server Hostname:        localhost
Server Port:            8080
Document Path:          /put/demo?msg=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Document Length:        12 bytes
Concurrency Level:      10
Time taken for tests:   56.141 seconds
Complete requests:      100000
Failed requests:        99991
   (Connect: 0, Receive: 0, Length: 99991, Exceptions: 0)
Write errors:           0
Total transferred:      8188895 bytes
HTML transferred:       1588895 bytes
Requests per second:    1781.24 [#/sec] (mean)
Time per request:       5.614 [ms] (mean)
Time per request:       0.561 [ms] (mean, across all concurrent requests)
Transfer rate:          142.45 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.7      0      16
Processing:     0    5  10.7      0     781
Waiting:        0    5  10.4      0     766
Total:          0    6  10.8      0     781
Percentage of the requests served within a certain time (ms)
  50%      0
  66%     16
  75%     16
  80%     16
  90%     16
  95%     16
  98%     16
  99%     16
 100%    781 (longest request)

插入 512 bytes 文本消息隊列(添加Keep-Alive支持): 1875.18 requests/sec

D:\Apache2.2\bin>ab -k -c 10 -n 100000 "http://localhost:8080/put/demo?msg=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
Server Software:
Server Hostname:        localhost
Server Port:            8080
Document Path:          /put/demo?msg=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Document Length:        17 bytes
Concurrency Level:      10
Time taken for tests:   53.328 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    0
Total transferred:      8300000 bytes
HTML transferred:       1700000 bytes
Requests per second:    1875.18 [#/sec] (mean)
Time per request:       5.333 [ms] (mean)
Time per request:       0.533 [ms] (mean, across all concurrent requests)
Transfer rate:          151.99 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.5      0      16
Processing:     0    5   7.8      0     203
Waiting:        0    5   7.8      0     203
Total:          0    5   7.9      0     203
Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%     16
  80%     16
  90%     16
  95%     16
  98%     16
  99%     16
 100%    203 (longest request)

獲取 512 bytes 消息(With Keep-Alive): 1875.73 requests/sec

D:\Apache2.2\bin>ab -k -c 10 -n 100000 "http://localhost:8080/get/demo"
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
Server Software:
Server Hostname:        localhost
Server Port:            8080
Document Path:          /get/demo
Document Length:        523 bytes
Concurrency Level:      10
Time taken for tests:   53.313 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    0
Total transferred:      58900000 bytes
HTML transferred:       52300000 bytes
Requests per second:    1875.73 [#/sec] (mean)
Time per request:       5.331 [ms] (mean)
Time per request:       0.533 [ms] (mean, across all concurrent requests)
Transfer rate:          1078.91 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.9      0      16
Processing:     0    5   7.5      0      94
Waiting:        0    4   6.9      0      94
Total:          0    5   7.6      0      94
Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%     16
  80%     16
  90%     16
  95%     16
  98%     16
  99%     16
 100%     94 (longest request)

1. 暫時對安全/授權沒有支持，這個其實很容易實現
2. 隊列處理器很簡單，直接使用jedis即可
3. 對隊列數據進行分片(sharding)，寫一個QueueService實現即可
4. 對Redis的內存持久化不放心，采用diskstore存儲模式好了，在Redis中配置，不涉及到程序
5. 對隊列分布式存儲，寫一個QueueService實現，其它可不用變化
6. 不適合嚴格意義上的訂閱/發布模型，這里適合多個發布者/單個訂閱者環境
7. HTTP請求返回內容為json格式，xml格式，暫時不需要
8. 局域網環境下，系統之間進行消息的推送/通知，

項目下載地址：http://code.google.com/p/nettyqueue/ 參考資料：

1. httpsqs