鷹翔宇空

          學(xué)習(xí)和生活

          BlogJava 首頁(yè) 新隨筆 聯(lián)系 聚合 管理
            110 Posts :: 141 Stories :: 315 Comments :: 1 Trackbacks
          引自:http://www.3doing.net/forums/dispbbs.asp?boardID=57&ID=432&page=2

          作者:鐘華

          (一)斷點(diǎn)續(xù)傳的原理

          其實(shí)斷點(diǎn)續(xù)傳的原理很簡(jiǎn)單,就是在Http的請(qǐng)求上和一般的下載有所不同而已。
          打個(gè)比方,瀏覽器請(qǐng)求服務(wù)器上的一個(gè)文時(shí),所發(fā)出的請(qǐng)求如下:
          假設(shè)服務(wù)器域名為wwww.sjtu.edu.cn,文件名為down.zip。
          GET /down.zip HTTP/1.1
          Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-
          excel, application/msword, application/vnd.ms-powerpoint, */*
          Accept-Language: zh-cn
          Accept-Encoding: gzip, deflate
          User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)
          Connection: Keep-Alive

          服務(wù)器收到請(qǐng)求后,按要求尋找請(qǐng)求的文件,提取文件的信息,然后返回給瀏覽器,返回信息如下:

          200
          Content-Length=106786028
          Accept-Ranges=bytes
          Date=Mon, 30 Apr 2001 12:56:11 GMT
          ETag=W/"02ca57e173c11:95b"
          Content-Type=application/octet-stream
          Server=Microsoft-IIS/5.0
          Last-Modified=Mon, 30 Apr 2001 12:56:11 GMT



          所謂斷點(diǎn)續(xù)傳,也就是要從文件已經(jīng)下載的地方開始繼續(xù)下載。所以在客戶端瀏覽器傳給
          Web服務(wù)器的時(shí)候要多加一條信息--從哪里開始。
          下面是用自己編的一個(gè)"瀏覽器"來(lái)傳遞請(qǐng)求信息給Web服務(wù)器,要求從2000070字節(jié)開始。
          GET /down.zip HTTP/1.0
          User-Agent: NetFox
          RANGE: bytes=2000070-
          Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2

          仔細(xì)看一下就會(huì)發(fā)現(xiàn)多了一行RANGE: bytes=2000070-
          這一行的意思就是告訴服務(wù)器down.zip這個(gè)文件從2000070字節(jié)開始傳,前面的字節(jié)不用傳了。
          服務(wù)器收到這個(gè)請(qǐng)求以后,返回的信息如下:
          206
          Content-Length=106786028
          Content-Range=bytes 2000070-106786027/106786028
          Date=Mon, 30 Apr 2001 12:55:20 GMT
          ETag=W/"02ca57e173c11:95b"
          Content-Type=application/octet-stream
          Server=Microsoft-IIS/5.0
          Last-Modified=Mon, 30 Apr 2001 12:55:20 GMT

          和前面服務(wù)器返回的信息比較一下,就會(huì)發(fā)現(xiàn)增加了一行:
          Content-Range=bytes 2000070-106786027/106786028
          返回的代碼也改為206了,而不再是200了。

          知道了以上原理,就可以進(jìn)行斷點(diǎn)續(xù)傳的編程了。

          (二)Java實(shí)現(xiàn)斷點(diǎn)續(xù)傳的關(guān)鍵幾點(diǎn)

          1. (1)用什么方法實(shí)現(xiàn)提交RANGE: bytes=2000070-。
            當(dāng)然用最原始的Socket是肯定能完成的,不過那樣太費(fèi)事了,其實(shí)Java的net包中提供了這種功能。代碼如下:

            URL url = new URL("http://www.sjtu.edu.cn/down.zip");
            HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection();


            //設(shè)置User-Agent
            httpConnection.setRequestProperty("User-Agent","NetFox");
            //設(shè)置斷點(diǎn)續(xù)傳的開始位置
            httpConnection.setRequestProperty("RANGE","bytes=2000070");
            //獲得輸入流
            InputStream input = httpConnection.getInputStream();



            從輸入流中取出的字節(jié)流就是down.zip文件從2000070開始的字節(jié)流。
            大家看,其實(shí)斷點(diǎn)續(xù)傳用Java實(shí)現(xiàn)起來(lái)還是很簡(jiǎn)單的吧。
            接下來(lái)要做的事就是怎么保存獲得的流到文件中去了。

          2. 保存文件采用的方法。
            我采用的是IO包中的RandAccessFile類。
            操作相當(dāng)簡(jiǎn)單,假設(shè)從2000070處開始保存文件,代碼如下:
            RandomAccess oSavedFile = new RandomAccessFile("down.zip","rw");
            long nPos = 2000070;
            //定位文件指針到nPos位置
            oSavedFile.seek(nPos);
            byte[] b = new byte[1024];
            int nRead;
            //從輸入流中讀入字節(jié)流,然后寫到文件中
            while((nRead=input.read(b,0,1024)) > 0)
            {
            oSavedFile.write(b,0,nRead);
            }

          怎么樣,也很簡(jiǎn)單吧。
          接下來(lái)要做的就是整合成一個(gè)完整的程序了。包括一系列的線程控制等等。

          (三)斷點(diǎn)續(xù)傳內(nèi)核的實(shí)現(xiàn)
          主要用了6個(gè)類,包括一個(gè)測(cè)試類。
          SiteFileFetch.java負(fù)責(zé)整個(gè)文件的抓取,控制內(nèi)部線程(FileSplitterFetch類)。
          FileSplitterFetch.java負(fù)責(zé)部分文件的抓取。
          FileAccess.java負(fù)責(zé)文件的存儲(chǔ)。
          SiteInfoBean.java要抓取的文件的信息,如文件保存的目錄,名字,抓取文件的URL等。
          Utility.java工具類,放一些簡(jiǎn)單的方法。
          TestMethod.java測(cè)試類。

          下面是源程序:

          /*
          **SiteFileFetch.java
          */
          package NetFox;
          import java.io.*;
          import java.net.*;


          public class SiteFileFetch extends Thread {


          SiteInfoBean siteInfoBean = null; //文件信息Bean
          long[] nStartPos; //開始位置
          long[] nEndPos; //結(jié)束位置
          FileSplitterFetch[] fileSplitterFetch; //子線程對(duì)象
          long nFileLength; //文件長(zhǎng)度
          boolean bFirst = true; //是否第一次取文件
          boolean bStop = false; //停止標(biāo)志
          File tmpFile; //文件下載的臨時(shí)信息
          DataOutputStream output; //輸出到文件的輸出流


          public SiteFileFetch(SiteInfoBean bean) throws IOException
          {
          siteInfoBean = bean;
          //tmpFile = File.createTempFile ("zhong","1111",new File(bean.getSFilePath()));
          tmpFile = new File(bean.getSFilePath()+File.separator + bean.getSFileName()+".info");
          if(tmpFile.exists ())
          {
          bFirst = false;
          read_nPos();
          }
          else
          {
          nStartPos = new long[bean.getNSplitter()];
          nEndPos = new long[bean.getNSplitter()];
          }



          }


          public void run()
          {
          //獲得文件長(zhǎng)度
          //分割文件
          //實(shí)例FileSplitterFetch
          //啟動(dòng)FileSplitterFetch線程
          //等待子線程返回
          try{
          if(bFirst)
          {
          nFileLength = getFileSize();
          if(nFileLength == -1)
          {
          System.err.println("File Length is not known!");
          }
          else if(nFileLength == -2)
          {
          System.err.println("File is not access!");
          }
          else
          {
          for(int i=0;i<nStartPos.length;i++)
          {
          nStartPos[i] = (long)(i*(nFileLength/nStartPos.length));
          }
          for(int i=0;i<nEndPos.length-1;i++)
          {
          nEndPos[i] = nStartPos[i+1];
          }
          nEndPos[nEndPos.length-1] = nFileLength;
          }
          }


          //啟動(dòng)子線程
          fileSplitterFetch = new FileSplitterFetch[nStartPos.length];
          for(int i=0;i<nStartPos.length;i++)
          {
          fileSplitterFetch[i] = new FileSplitterFetch(siteInfoBean.getSSiteURL(),
          siteInfoBean.getSFilePath() + File.separator + siteInfoBean.getSFileName(),
          nStartPos[i],nEndPos[i],i);
          Utility.log("Thread " + i + " , nStartPos = " + nStartPos[i] + ", nEndPos = " + nEndPos[i]);
          fileSplitterFetch[i].start();
          }
          // fileSplitterFetch[nPos.length-1] = new FileSplitterFetch(siteInfoBean.getSSiteURL(),
          siteInfoBean.getSFilePath() + File.separator + siteInfoBean.getSFileName(),nPos[nPos.length-1],nFileLength,nPos.length-1);
          // Utility.log("Thread " + (nPos.length-1) + " , nStartPos = " + nPos[nPos.length-1] + ",
          nEndPos = " + nFileLength);
          // fileSplitterFetch[nPos.length-1].start();


          //等待子線程結(jié)束
          //int count = 0;
          //是否結(jié)束while循環(huán)
          boolean breakWhile = false;


          while(!bStop)
          {
          write_nPos();
          Utility.sleep(500);
          breakWhile = true;


          for(int i=0;i<nStartPos.length;i++)
          {
          if(!fileSplitterFetch[i].bDownOver)
          {
          breakWhile = false;
          break;
          }
          }
          if(breakWhile)
          break;


          //count++;
          //if(count>4)
          // siteStop();
          }


          System.err.println("文件下載結(jié)束!");
          }
          catch(Exception e){e.printStackTrace ();}
          }


          //獲得文件長(zhǎng)度
          public long getFileSize()
          {
          int nFileLength = -1;
          try{
          URL url = new URL(siteInfoBean.getSSiteURL());
          HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection ();
          httpConnection.setRequestProperty("User-Agent","NetFox");


          int responseCode=httpConnection.getResponseCode();
          if(responseCode>=400)
          {
          processErrorCode(responseCode);
          return -2; //-2 represent access is error
          }


          String sHeader;


          for(int i=1;;i++)
          {
          //DataInputStream in = new DataInputStream(httpConnection.getInputStream ());
          //Utility.log(in.readLine());
          sHeader=httpConnection.getHeaderFieldKey(i);
          if(sHeader!=null)
          {
          if(sHeader.equals("Content-Length"))
          {
          nFileLength = Integer.parseInt(httpConnection.getHeaderField(sHeader));
          break;
          }
          }
          else
          break;
          }
          }
          catch(IOException e){e.printStackTrace ();}
          catch(Exception e){e.printStackTrace ();}


          Utility.log(nFileLength);


          return nFileLength;
          }


          //保存下載信息(文件指針位置)
          private void write_nPos()
          {
          try{
          output = new DataOutputStream(new FileOutputStream(tmpFile));
          output.writeInt(nStartPos.length);
          for(int i=0;i<nStartPos.length;i++)
          {
          // output.writeLong(nPos[i]);
          output.writeLong(fileSplitterFetch[i].nStartPos);
          output.writeLong(fileSplitterFetch[i].nEndPos);
          }
          output.close();
          }
          catch(IOException e){e.printStackTrace ();}
          catch(Exception e){e.printStackTrace ();}
          }


          //讀取保存的下載信息(文件指針位置)
          private void read_nPos()
          {
          try{
          DataInputStream input = new DataInputStream(new FileInputStream(tmpFile));
          int nCount = input.readInt();
          nStartPos = new long[nCount];
          nEndPos = new long[nCount];
          for(int i=0;i<nStartPos.length;i++)
          {
          nStartPos[i] = input.readLong();
          nEndPos[i] = input.readLong();
          }
          input.close();
          }
          catch(IOException e){e.printStackTrace ();}
          catch(Exception e){e.printStackTrace ();}
          }


          private void processErrorCode(int nErrorCode)
          {
          System.err.println("Error Code : " + nErrorCode);
          }


          //停止文件下載
          public void siteStop()
          {
          bStop = true;
          for(int i=0;i<nStartPos.length;i++)
          fileSplitterFetch[i].splitterStop();


          }
          }
          /*
          **FileSplitterFetch.java
          */
          package NetFox;


          import java.io.*;
          import java.net.*;


          public class FileSplitterFetch extends Thread {


          String sURL; //File URL
          long nStartPos; //File Snippet Start Position
          long nEndPos; //File Snippet End Position
          int nThreadID; //Thread's ID
          boolean bDownOver = false; //Downing is over
          boolean bStop = false; //Stop identical
          FileAccessI fileAccessI = null; //File Access interface


          public FileSplitterFetch(String sURL,String sName,long nStart,long nEnd,int id) throws IOException
          {
          this.sURL = sURL;
          this.nStartPos = nStart;
          this.nEndPos = nEnd;
          nThreadID = id;
          fileAccessI = new FileAccessI(sName,nStartPos);
          }


          public void run()
          {
          while(nStartPos < nEndPos && !bStop)
          {


          try{
          URL url = new URL(sURL);
          HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection ();
          httpConnection.setRequestProperty("User-Agent","NetFox");
          String sProperty = "bytes="+nStartPos+"-";
          httpConnection.setRequestProperty("RANGE",sProperty);
          Utility.log(sProperty);


          InputStream input = httpConnection.getInputStream();
          //logResponseHead(httpConnection);


          byte[] b = new byte[1024];
          int nRead;
          while((nRead=input.read(b,0,1024)) > 0 && nStartPos < nEndPos && !bStop)
          {
          nStartPos += fileAccessI.write(b,0,nRead);
          //if(nThreadID == 1)
          // Utility.log("nStartPos = " + nStartPos + ", nEndPos = " + nEndPos);
          }


          Utility.log("Thread " + nThreadID + " is over!");
          bDownOver = true;
          //nPos = fileAccessI.write (b,0,nRead);
          }
          catch(Exception e){e.printStackTrace ();}
          }
          }


          //打印回應(yīng)的頭信息
          public void logResponseHead(HttpURLConnection con)
          {
          for(int i=1;;i++)
          {
          String header=con.getHeaderFieldKey(i);
          if(header!=null)
          //responseHeaders.put(header,httpConnection.getHeaderField(header));
          Utility.log(header+" : "+con.getHeaderField(header));
          else
          break;
          }
          }


          public void splitterStop()
          {
          bStop = true;
          }


          }


          /*
          **FileAccess.java
          */
          package NetFox;
          import java.io.*;


          public class FileAccessI implements Serializable{


          RandomAccessFile oSavedFile;
          long nPos;


          public FileAccessI() throws IOException
          {
          this("",0);
          }


          public FileAccessI(String sName,long nPos) throws IOException
          {
          oSavedFile = new RandomAccessFile(sName,"rw");
          this.nPos = nPos;
          oSavedFile.seek(nPos);
          }


          public synchronized int write(byte[] b,int nStart,int nLen)
          {
          int n = -1;
          try{
          oSavedFile.write(b,nStart,nLen);
          n = nLen;
          }
          catch(IOException e)
          {
          e.printStackTrace ();
          }


          return n;
          }


          }


          /*
          **SiteInfoBean.java
          */
          package NetFox;


          public class SiteInfoBean {


          private String sSiteURL; //Site's URL
          private String sFilePath; //Saved File's Path
          private String sFileName; //Saved File's Name
          private int nSplitter; //Count of Splited Downloading File


          public SiteInfoBean()
          {
          //default value of nSplitter is 5
          this("","","",5);
          }


          public SiteInfoBean(String sURL,String sPath,String sName,int nSpiltter)
          {
          sSiteURL= sURL;
          sFilePath = sPath;
          sFileName = sName;
          this.nSplitter = nSpiltter;


          }


          public String getSSiteURL()
          {
          return sSiteURL;
          }


          public void setSSiteURL(String value)
          {
          sSiteURL = value;
          }


          public String getSFilePath()
          {
          return sFilePath;
          }


          public void setSFilePath(String value)
          {
          sFilePath = value;
          }


          public String getSFileName()
          {
          return sFileName;
          }


          public void setSFileName(String value)
          {
          sFileName = value;
          }


          public int getNSplitter()
          {
          return nSplitter;
          }


          public void setNSplitter(int nCount)
          {
          nSplitter = nCount;
          }
          }


          /*
          **Utility.java
          */
          package NetFox;


          public class Utility {


          public Utility()
          {


          }


          public static void sleep(int nSecond)
          {
          try{
          Thread.sleep(nSecond);
          }
          catch(Exception e)
          {
          e.printStackTrace ();
          }
          }


          public static void log(String sMsg)
          {
          System.err.println(sMsg);
          }


          public static void log(int sMsg)
          {
          System.err.println(sMsg);
          }
          }


          /*
          **TestMethod.java
          */
          package NetFox;


          public class TestMethod {


          public TestMethod()
          { ///xx/weblogic60b2_win.exe
          try{
          SiteInfoBean bean = new SiteInfoBean("http://localhost/xx/weblogic60b2_win.exe","L:\\temp","weblogic60b2_win.exe",5);
          //SiteInfoBean bean = new SiteInfoBean("http://localhost:8080/down.zip","L:\\temp","weblogic60b2_win.exe",5);
          SiteFileFetch fileFetch = new SiteFileFetch(bean);
          fileFetch.start();
          }
          catch(Exception e){e.printStackTrace ();}


          }


          public static void main(String[] args)
          {
          new TestMethod();
          }
          }
          posted on 2006-05-30 14:20 TrampEagle 閱讀(309) 評(píng)論(0)  編輯  收藏 所屬分類: java
          主站蜘蛛池模板: 乌海市| 崇文区| 河曲县| 彰化县| 霍城县| 龙泉市| 奉贤区| 泌阳县| 上饶市| 磐石市| 武邑县| 固安县| 门源| 句容市| 肃北| 肥城市| 简阳市| 五莲县| 金川县| 西贡区| 鄂伦春自治旗| 靖远县| 济宁市| 乐业县| 西充县| 千阳县| 陕西省| 独山县| 工布江达县| 丽水市| 阿拉善左旗| 绩溪县| 胶州市| 凌源市| 高平市| 乐都县| 昭觉县| 梨树县| 青海省| 德清县| 山阳县|