鷹翔宇空

          學習和生活

          BlogJava 首頁 新隨筆 聯系 聚合 管理
            110 Posts :: 141 Stories :: 315 Comments :: 1 Trackbacks
          引自:http://www.3doing.net/forums/dispbbs.asp?boardID=57&ID=432&page=2

          作者:鐘華

          (一)斷點續傳的原理

          其實斷點續傳的原理很簡單,就是在Http的請求上和一般的下載有所不同而已。
          打個比方,瀏覽器請求服務器上的一個文時,所發出的請求如下:
          假設服務器域名為wwww.sjtu.edu.cn,文件名為down.zip。
          GET /down.zip HTTP/1.1
          Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-
          excel, application/msword, application/vnd.ms-powerpoint, */*
          Accept-Language: zh-cn
          Accept-Encoding: gzip, deflate
          User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)
          Connection: Keep-Alive

          服務器收到請求后,按要求尋找請求的文件,提取文件的信息,然后返回給瀏覽器,返回信息如下:

          200
          Content-Length=106786028
          Accept-Ranges=bytes
          Date=Mon, 30 Apr 2001 12:56:11 GMT
          ETag=W/"02ca57e173c11:95b"
          Content-Type=application/octet-stream
          Server=Microsoft-IIS/5.0
          Last-Modified=Mon, 30 Apr 2001 12:56:11 GMT



          所謂斷點續傳,也就是要從文件已經下載的地方開始繼續下載。所以在客戶端瀏覽器傳給
          Web服務器的時候要多加一條信息--從哪里開始。
          下面是用自己編的一個"瀏覽器"來傳遞請求信息給Web服務器,要求從2000070字節開始。
          GET /down.zip HTTP/1.0
          User-Agent: NetFox
          RANGE: bytes=2000070-
          Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2

          仔細看一下就會發現多了一行RANGE: bytes=2000070-
          這一行的意思就是告訴服務器down.zip這個文件從2000070字節開始傳,前面的字節不用傳了。
          服務器收到這個請求以后,返回的信息如下:
          206
          Content-Length=106786028
          Content-Range=bytes 2000070-106786027/106786028
          Date=Mon, 30 Apr 2001 12:55:20 GMT
          ETag=W/"02ca57e173c11:95b"
          Content-Type=application/octet-stream
          Server=Microsoft-IIS/5.0
          Last-Modified=Mon, 30 Apr 2001 12:55:20 GMT

          和前面服務器返回的信息比較一下,就會發現增加了一行:
          Content-Range=bytes 2000070-106786027/106786028
          返回的代碼也改為206了,而不再是200了。

          知道了以上原理,就可以進行斷點續傳的編程了。

          (二)Java實現斷點續傳的關鍵幾點

          1. (1)用什么方法實現提交RANGE: bytes=2000070-。
            當然用最原始的Socket是肯定能完成的,不過那樣太費事了,其實Java的net包中提供了這種功能。代碼如下:

            URL url = new URL("http://www.sjtu.edu.cn/down.zip");
            HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection();


            //設置User-Agent
            httpConnection.setRequestProperty("User-Agent","NetFox");
            //設置斷點續傳的開始位置
            httpConnection.setRequestProperty("RANGE","bytes=2000070");
            //獲得輸入流
            InputStream input = httpConnection.getInputStream();



            從輸入流中取出的字節流就是down.zip文件從2000070開始的字節流。
            大家看,其實斷點續傳用Java實現起來還是很簡單的吧。
            接下來要做的事就是怎么保存獲得的流到文件中去了。

          2. 保存文件采用的方法。
            我采用的是IO包中的RandAccessFile類。
            操作相當簡單,假設從2000070處開始保存文件,代碼如下:
            RandomAccess oSavedFile = new RandomAccessFile("down.zip","rw");
            long nPos = 2000070;
            //定位文件指針到nPos位置
            oSavedFile.seek(nPos);
            byte[] b = new byte[1024];
            int nRead;
            //從輸入流中讀入字節流,然后寫到文件中
            while((nRead=input.read(b,0,1024)) > 0)
            {
            oSavedFile.write(b,0,nRead);
            }

          怎么樣,也很簡單吧。
          接下來要做的就是整合成一個完整的程序了。包括一系列的線程控制等等。

          (三)斷點續傳內核的實現
          主要用了6個類,包括一個測試類。
          SiteFileFetch.java負責整個文件的抓取,控制內部線程(FileSplitterFetch類)。
          FileSplitterFetch.java負責部分文件的抓取。
          FileAccess.java負責文件的存儲。
          SiteInfoBean.java要抓取的文件的信息,如文件保存的目錄,名字,抓取文件的URL等。
          Utility.java工具類,放一些簡單的方法。
          TestMethod.java測試類。

          下面是源程序:

          /*
          **SiteFileFetch.java
          */
          package NetFox;
          import java.io.*;
          import java.net.*;


          public class SiteFileFetch extends Thread {


          SiteInfoBean siteInfoBean = null; //文件信息Bean
          long[] nStartPos; //開始位置
          long[] nEndPos; //結束位置
          FileSplitterFetch[] fileSplitterFetch; //子線程對象
          long nFileLength; //文件長度
          boolean bFirst = true; //是否第一次取文件
          boolean bStop = false; //停止標志
          File tmpFile; //文件下載的臨時信息
          DataOutputStream output; //輸出到文件的輸出流


          public SiteFileFetch(SiteInfoBean bean) throws IOException
          {
          siteInfoBean = bean;
          //tmpFile = File.createTempFile ("zhong","1111",new File(bean.getSFilePath()));
          tmpFile = new File(bean.getSFilePath()+File.separator + bean.getSFileName()+".info");
          if(tmpFile.exists ())
          {
          bFirst = false;
          read_nPos();
          }
          else
          {
          nStartPos = new long[bean.getNSplitter()];
          nEndPos = new long[bean.getNSplitter()];
          }



          }


          public void run()
          {
          //獲得文件長度
          //分割文件
          //實例FileSplitterFetch
          //啟動FileSplitterFetch線程
          //等待子線程返回
          try{
          if(bFirst)
          {
          nFileLength = getFileSize();
          if(nFileLength == -1)
          {
          System.err.println("File Length is not known!");
          }
          else if(nFileLength == -2)
          {
          System.err.println("File is not access!");
          }
          else
          {
          for(int i=0;i<nStartPos.length;i++)
          {
          nStartPos[i] = (long)(i*(nFileLength/nStartPos.length));
          }
          for(int i=0;i<nEndPos.length-1;i++)
          {
          nEndPos[i] = nStartPos[i+1];
          }
          nEndPos[nEndPos.length-1] = nFileLength;
          }
          }


          //啟動子線程
          fileSplitterFetch = new FileSplitterFetch[nStartPos.length];
          for(int i=0;i<nStartPos.length;i++)
          {
          fileSplitterFetch[i] = new FileSplitterFetch(siteInfoBean.getSSiteURL(),
          siteInfoBean.getSFilePath() + File.separator + siteInfoBean.getSFileName(),
          nStartPos[i],nEndPos[i],i);
          Utility.log("Thread " + i + " , nStartPos = " + nStartPos[i] + ", nEndPos = " + nEndPos[i]);
          fileSplitterFetch[i].start();
          }
          // fileSplitterFetch[nPos.length-1] = new FileSplitterFetch(siteInfoBean.getSSiteURL(),
          siteInfoBean.getSFilePath() + File.separator + siteInfoBean.getSFileName(),nPos[nPos.length-1],nFileLength,nPos.length-1);
          // Utility.log("Thread " + (nPos.length-1) + " , nStartPos = " + nPos[nPos.length-1] + ",
          nEndPos = " + nFileLength);
          // fileSplitterFetch[nPos.length-1].start();


          //等待子線程結束
          //int count = 0;
          //是否結束while循環
          boolean breakWhile = false;


          while(!bStop)
          {
          write_nPos();
          Utility.sleep(500);
          breakWhile = true;


          for(int i=0;i<nStartPos.length;i++)
          {
          if(!fileSplitterFetch[i].bDownOver)
          {
          breakWhile = false;
          break;
          }
          }
          if(breakWhile)
          break;


          //count++;
          //if(count>4)
          // siteStop();
          }


          System.err.println("文件下載結束!");
          }
          catch(Exception e){e.printStackTrace ();}
          }


          //獲得文件長度
          public long getFileSize()
          {
          int nFileLength = -1;
          try{
          URL url = new URL(siteInfoBean.getSSiteURL());
          HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection ();
          httpConnection.setRequestProperty("User-Agent","NetFox");


          int responseCode=httpConnection.getResponseCode();
          if(responseCode>=400)
          {
          processErrorCode(responseCode);
          return -2; //-2 represent access is error
          }


          String sHeader;


          for(int i=1;;i++)
          {
          //DataInputStream in = new DataInputStream(httpConnection.getInputStream ());
          //Utility.log(in.readLine());
          sHeader=httpConnection.getHeaderFieldKey(i);
          if(sHeader!=null)
          {
          if(sHeader.equals("Content-Length"))
          {
          nFileLength = Integer.parseInt(httpConnection.getHeaderField(sHeader));
          break;
          }
          }
          else
          break;
          }
          }
          catch(IOException e){e.printStackTrace ();}
          catch(Exception e){e.printStackTrace ();}


          Utility.log(nFileLength);


          return nFileLength;
          }


          //保存下載信息(文件指針位置)
          private void write_nPos()
          {
          try{
          output = new DataOutputStream(new FileOutputStream(tmpFile));
          output.writeInt(nStartPos.length);
          for(int i=0;i<nStartPos.length;i++)
          {
          // output.writeLong(nPos[i]);
          output.writeLong(fileSplitterFetch[i].nStartPos);
          output.writeLong(fileSplitterFetch[i].nEndPos);
          }
          output.close();
          }
          catch(IOException e){e.printStackTrace ();}
          catch(Exception e){e.printStackTrace ();}
          }


          //讀取保存的下載信息(文件指針位置)
          private void read_nPos()
          {
          try{
          DataInputStream input = new DataInputStream(new FileInputStream(tmpFile));
          int nCount = input.readInt();
          nStartPos = new long[nCount];
          nEndPos = new long[nCount];
          for(int i=0;i<nStartPos.length;i++)
          {
          nStartPos[i] = input.readLong();
          nEndPos[i] = input.readLong();
          }
          input.close();
          }
          catch(IOException e){e.printStackTrace ();}
          catch(Exception e){e.printStackTrace ();}
          }


          private void processErrorCode(int nErrorCode)
          {
          System.err.println("Error Code : " + nErrorCode);
          }


          //停止文件下載
          public void siteStop()
          {
          bStop = true;
          for(int i=0;i<nStartPos.length;i++)
          fileSplitterFetch[i].splitterStop();


          }
          }
          /*
          **FileSplitterFetch.java
          */
          package NetFox;


          import java.io.*;
          import java.net.*;


          public class FileSplitterFetch extends Thread {


          String sURL; //File URL
          long nStartPos; //File Snippet Start Position
          long nEndPos; //File Snippet End Position
          int nThreadID; //Thread's ID
          boolean bDownOver = false; //Downing is over
          boolean bStop = false; //Stop identical
          FileAccessI fileAccessI = null; //File Access interface


          public FileSplitterFetch(String sURL,String sName,long nStart,long nEnd,int id) throws IOException
          {
          this.sURL = sURL;
          this.nStartPos = nStart;
          this.nEndPos = nEnd;
          nThreadID = id;
          fileAccessI = new FileAccessI(sName,nStartPos);
          }


          public void run()
          {
          while(nStartPos < nEndPos && !bStop)
          {


          try{
          URL url = new URL(sURL);
          HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection ();
          httpConnection.setRequestProperty("User-Agent","NetFox");
          String sProperty = "bytes="+nStartPos+"-";
          httpConnection.setRequestProperty("RANGE",sProperty);
          Utility.log(sProperty);


          InputStream input = httpConnection.getInputStream();
          //logResponseHead(httpConnection);


          byte[] b = new byte[1024];
          int nRead;
          while((nRead=input.read(b,0,1024)) > 0 && nStartPos < nEndPos && !bStop)
          {
          nStartPos += fileAccessI.write(b,0,nRead);
          //if(nThreadID == 1)
          // Utility.log("nStartPos = " + nStartPos + ", nEndPos = " + nEndPos);
          }


          Utility.log("Thread " + nThreadID + " is over!");
          bDownOver = true;
          //nPos = fileAccessI.write (b,0,nRead);
          }
          catch(Exception e){e.printStackTrace ();}
          }
          }


          //打印回應的頭信息
          public void logResponseHead(HttpURLConnection con)
          {
          for(int i=1;;i++)
          {
          String header=con.getHeaderFieldKey(i);
          if(header!=null)
          //responseHeaders.put(header,httpConnection.getHeaderField(header));
          Utility.log(header+" : "+con.getHeaderField(header));
          else
          break;
          }
          }


          public void splitterStop()
          {
          bStop = true;
          }


          }


          /*
          **FileAccess.java
          */
          package NetFox;
          import java.io.*;


          public class FileAccessI implements Serializable{


          RandomAccessFile oSavedFile;
          long nPos;


          public FileAccessI() throws IOException
          {
          this("",0);
          }


          public FileAccessI(String sName,long nPos) throws IOException
          {
          oSavedFile = new RandomAccessFile(sName,"rw");
          this.nPos = nPos;
          oSavedFile.seek(nPos);
          }


          public synchronized int write(byte[] b,int nStart,int nLen)
          {
          int n = -1;
          try{
          oSavedFile.write(b,nStart,nLen);
          n = nLen;
          }
          catch(IOException e)
          {
          e.printStackTrace ();
          }


          return n;
          }


          }


          /*
          **SiteInfoBean.java
          */
          package NetFox;


          public class SiteInfoBean {


          private String sSiteURL; //Site's URL
          private String sFilePath; //Saved File's Path
          private String sFileName; //Saved File's Name
          private int nSplitter; //Count of Splited Downloading File


          public SiteInfoBean()
          {
          //default value of nSplitter is 5
          this("","","",5);
          }


          public SiteInfoBean(String sURL,String sPath,String sName,int nSpiltter)
          {
          sSiteURL= sURL;
          sFilePath = sPath;
          sFileName = sName;
          this.nSplitter = nSpiltter;


          }


          public String getSSiteURL()
          {
          return sSiteURL;
          }


          public void setSSiteURL(String value)
          {
          sSiteURL = value;
          }


          public String getSFilePath()
          {
          return sFilePath;
          }


          public void setSFilePath(String value)
          {
          sFilePath = value;
          }


          public String getSFileName()
          {
          return sFileName;
          }


          public void setSFileName(String value)
          {
          sFileName = value;
          }


          public int getNSplitter()
          {
          return nSplitter;
          }


          public void setNSplitter(int nCount)
          {
          nSplitter = nCount;
          }
          }


          /*
          **Utility.java
          */
          package NetFox;


          public class Utility {


          public Utility()
          {


          }


          public static void sleep(int nSecond)
          {
          try{
          Thread.sleep(nSecond);
          }
          catch(Exception e)
          {
          e.printStackTrace ();
          }
          }


          public static void log(String sMsg)
          {
          System.err.println(sMsg);
          }


          public static void log(int sMsg)
          {
          System.err.println(sMsg);
          }
          }


          /*
          **TestMethod.java
          */
          package NetFox;


          public class TestMethod {


          public TestMethod()
          { ///xx/weblogic60b2_win.exe
          try{
          SiteInfoBean bean = new SiteInfoBean("http://localhost/xx/weblogic60b2_win.exe","L:\\temp","weblogic60b2_win.exe",5);
          //SiteInfoBean bean = new SiteInfoBean("http://localhost:8080/down.zip","L:\\temp","weblogic60b2_win.exe",5);
          SiteFileFetch fileFetch = new SiteFileFetch(bean);
          fileFetch.start();
          }
          catch(Exception e){e.printStackTrace ();}


          }


          public static void main(String[] args)
          {
          new TestMethod();
          }
          }
          posted on 2006-05-30 14:20 TrampEagle 閱讀(309) 評論(0)  編輯  收藏 所屬分類: java
          主站蜘蛛池模板: 新建县| 越西县| 赤水市| 祁连县| 赤壁市| 万州区| 宁远县| 大丰市| 屯昌县| 怀化市| 光泽县| 榆社县| 高台县| 珠海市| 新巴尔虎左旗| 金华市| 柳州市| 原阳县| 柞水县| 林西县| 曲阜市| 海淀区| 绍兴县| 宜章县| 象州县| 洮南市| 南陵县| 漳浦县| 婺源县| 汉寿县| 寿光市| 南康市| 长兴县| 龙泉市| 依安县| 时尚| 台江县| 大丰市| 宽城| 中山市| 阳曲县|