Java网络爬虫源码

上传者: qy1989525 | 上传时间: 2019-12-21 20:58:55 | 文件大小: 11.52MB | 文件类型: zip
由于项目需要,特研究了一段时间关于java爬虫的相关技术,发现一个比较好用的爬虫框架--WebMagic,只需少量代码即可实现一个爬虫,本项目就是基于它的一个简单实现,导入项目即可运行,项目只有两个类,一个用于抓取,一个用于处理抓取到的数据,存入数据库或导出到excel等(只打印到控制台,后续自己发挥),简单吧,代码真的很少

文件下载

资源详情

[{"title":"( 48 个子文件 11.52MB ) Java网络爬虫源码","children":[{"title":"spider","children":[{"title":"bin","children":[{"title":"pipeline","children":[{"title":"HdPipeline.class <span style='color:#111;'> 1.01KB </span>","children":null,"spread":false}],"spread":true},{"title":"processor","children":[{"title":"HdProcessor.class <span style='color:#111;'> 8.09KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":".settings","children":[{"title":"org.eclipse.core.resources.prefs <span style='color:#111;'> 103B </span>","children":null,"spread":false},{"title":"org.eclipse.jdt.core.prefs <span style='color:#111;'> 598B </span>","children":null,"spread":false}],"spread":true},{"title":"src","children":[{"title":"pipeline","children":[{"title":"HdPipeline.java <span style='color:#111;'> 587B </span>","children":null,"spread":false}],"spread":true},{"title":"processor","children":[{"title":"HdProcessor.java <span style='color:#111;'> 7.64KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":".project <span style='color:#111;'> 382B </span>","children":null,"spread":false},{"title":".classpath <span style='color:#111;'> 2.76KB </span>","children":null,"spread":false},{"title":"lib","children":[{"title":"commons-collections-3.1.jar <span style='color:#111;'> 546.26KB </span>","children":null,"spread":false},{"title":"webmagic-extension-0.5.2.jar <span style='color:#111;'> 91.69KB </span>","children":null,"spread":false},{"title":"commons-collections-3.2.1.jar <span style='color:#111;'> 561.90KB </span>","children":null,"spread":false},{"title":"commons-beanutils-1.7.0.jar <span style='color:#111;'> 184.25KB </span>","children":null,"spread":false},{"title":"httpcore-4.3.2.jar <span style='color:#111;'> 275.65KB </span>","children":null,"spread":false},{"title":"guava-15.0.jar <span style='color:#111;'> 2.07MB </span>","children":null,"spread":false},{"title":"assertj-core-1.5.0.jar <span style='color:#111;'> 562.83KB </span>","children":null,"spread":false},{"title":"commons-pool-1.5.4.jar <span style='color:#111;'> 93.97KB </span>","children":null,"spread":false},{"title":"jedis-2.1.0.jar <span style='color:#111;'> 136.16KB </span>","children":null,"spread":false},{"title":"httpcore-4.4.5.jar <span style='color:#111;'> 319.70KB </span>","children":null,"spread":false},{"title":"httpclient-4.3.3.jar <span style='color:#111;'> 575.70KB </span>","children":null,"spread":false},{"title":"fastjson-1.1.37.jar <span style='color:#111;'> 348.29KB </span>","children":null,"spread":false},{"title":"commons-pool-1.5.5.jar <span style='color:#111;'> 97.84KB </span>","children":null,"spread":false},{"title":"commons-io-2.0.1.jar <span style='color:#111;'> 155.77KB </span>","children":null,"spread":false},{"title":"httpclient-4.5.2.jar <span style='color:#111;'> 719.39KB </span>","children":null,"spread":false},{"title":"json-path-0.8.1.jar <span style='color:#111;'> 65.20KB </span>","children":null,"spread":false},{"title":"ezmorph-1.0.3.jar <span style='color:#111;'> 76.00KB </span>","children":null,"spread":false},{"title":"slf4j-api-1.6.1.jar <span style='color:#111;'> 24.90KB </span>","children":null,"spread":false},{"title":"junit-4.11.jar <span style='color:#111;'> 239.30KB </span>","children":null,"spread":false},{"title":"json-smart-1.1.1.jar <span style='color:#111;'> 50.28KB </span>","children":null,"spread":false},{"title":"slf4j-log4j12-1.7.6.jar <span style='color:#111;'> 8.66KB </span>","children":null,"spread":false},{"title":"slf4j-api-1.7.6.jar <span style='color:#111;'> 28.02KB </span>","children":null,"spread":false},{"title":"hamcrest-core-1.3.jar <span style='color:#111;'> 43.97KB </span>","children":null,"spread":false},{"title":"commons-lang-2.6.jar <span style='color:#111;'> 277.56KB </span>","children":null,"spread":false},{"title":"webmagic-core-0.5.2.jar <span style='color:#111;'> 93.15KB </span>","children":null,"spread":false},{"title":"xsoup-0.2.4.jar <span style='color:#111;'> 38.83KB </span>","children":null,"spread":false},{"title":"commons-logging.jar <span style='color:#111;'> 44.34KB </span>","children":null,"spread":false},{"title":"commons-logging-1.1.3.jar <span style='color:#111;'> 60.60KB </span>","children":null,"spread":false},{"title":"jedis-2.0.0.jar <span style='color:#111;'> 122.89KB </span>","children":null,"spread":false},{"title":"commons-codec-1.7.jar <span style='color:#111;'> 253.52KB </span>","children":null,"spread":false},{"title":"commons-codec-1.6.jar <span style='color:#111;'> 227.32KB </span>","children":null,"spread":false},{"title":"bcprov-jdk15on-1.52.jar <span style='color:#111;'> 2.77MB </span>","children":null,"spread":false},{"title":"commons-logging-1.2.jar <span style='color:#111;'> 60.38KB </span>","children":null,"spread":false},{"title":"log4j-1.2.15.jar <span style='color:#111;'> 382.65KB </span>","children":null,"spread":false},{"title":"commons-lang-2.5.jar <span style='color:#111;'> 272.65KB </span>","children":null,"spread":false},{"title":"commons-io-1.3.2.jar <span style='color:#111;'> 85.72KB </span>","children":null,"spread":false},{"title":"log4j-1.2.17.jar <span style='color:#111;'> 478.40KB </span>","children":null,"spread":false},{"title":"json-lib-2.4-jdk15.jar <span style='color:#111;'> 155.39KB </span>","children":null,"spread":false},{"title":"commons-lang3-3.1.jar <span style='color:#111;'> 308.40KB </span>","children":null,"spread":false},{"title":"jsoup-1.7.2.jar <span style='color:#111;'> 286.79KB </span>","children":null,"spread":false}],"spread":false}],"spread":true}],"spread":true}]

评论信息

  • 灵动领域 :
    网页地址打不开
    2019-11-27
  • qq_23900647 :
    没找到入口,再试一次,
    2019-08-28
  • wx19870619 :
    还可以不错不错
    2019-04-02
  • lin1888 :
    不错不错不错不错
    2019-03-16
  • ntzcm :
    很好上手,简单实用。
    2018-07-25

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明