JarCasting
JarCasting JarCasting

SpiderLib

爬取 html json jsonp的工具

License

License
Categories

Categories

IDE Development Tools
GroupId

GroupId

cn.miw
ArtifactId

ArtifactId

spiderlib
Last Version

Last Version

0.0.2
Release Date

Release Date
Type

Type

jar
Description

Description

SpiderLib
爬取 html json jsonp的工具
Project URL

Project URL

http://www.miw.cn/
Source Code Management

Source Code Management

https://github.com/askmiw/spiderlib.git

Download spiderlib

Filename Size
spiderlib-0.0.2.pom
spiderlib-0.0.2.jar 19 KB
spiderlib-0.0.2-sources.jar 9 KB
spiderlib-0.0.2-javadoc.jar 61 KB
Browse

How to add to project

<!-- https://jarcasting.com/artifacts/cn.miw/spiderlib/ -->
<dependency>
    <groupId>cn.miw</groupId>
    <artifactId>spiderlib</artifactId>
    <version>0.0.2</version>
</dependency>
// https://jarcasting.com/artifacts/cn.miw/spiderlib/
implementation 'cn.miw:spiderlib:0.0.2'
// https://jarcasting.com/artifacts/cn.miw/spiderlib/
implementation ("cn.miw:spiderlib:0.0.2")
'cn.miw:spiderlib:jar:0.0.2'
<dependency org="cn.miw" name="spiderlib" rev="0.0.2">
  <artifact name="spiderlib" type="jar" />
</dependency>
@Grapes(
@Grab(group='cn.miw', module='spiderlib', version='0.0.2')
)
libraryDependencies += "cn.miw" % "spiderlib" % "0.0.2"
[cn.miw/spiderlib "0.0.2"]

Dependencies

compile (4)

Group / Artifact Type Version
net.sourceforge.htmlunit : htmlunit jar 2.31
org.jsoup : jsoup jar 1.10.3
net.sourceforge.htmlunit : htmlunit-core-js jar 2.31
com.alibaba : fastjson jar 1.2.47

Project Modules

There are no modules declared in this project.

spiderlib

使用样例

// type=0 为爬取HTML数据；type=1 派去API 数据
ICatTask task;
if (taskinfo.getType() == 0) {
	task = new CatHTML(taskinfo.getFlag(), taskinfo.getPage(), taskinfo.getFin(), taskinfo.getStep(),
			taskinfo.getDelay(), taskinfo.getUrl(), taskinfo.getListSelector(), aDefs);
	if (taskinfo.getEnaleJS() == 1) {
		((CatHTML) task).enaleJS();
	}
	// 爬取API 数据
} else {
	task = new CatJSON(taskinfo.getFlag(), taskinfo.getPage(), taskinfo.getFin(), taskinfo.getStep(),
			taskinfo.getDelay(), taskinfo.getUrl(), taskinfo.getListSelector(), def);
}
SaveData saveAction = new SaveData();
task.start(saveAction);

String url = "https://c.y.qq.com/v8/fcg-bin/fcg_v8_toplist_cp.fcg?tpl=3&page=detail&date=2018-08-24&topid=4&type=top&song_begin= {{page}}&song_num=30&g_tk=5381&jsonpCallback=MusicJsonCallbacktoplist&loginUin=0&hostUin=0&format=jsonp&inCharset=utf8&outCharset=utf-8&notice=0&platform=yqq&needNewCode=0";
ICatTask task = new CatJSONP("QQ音乐", 0, 1000, 30, 2000, url, "songlist", null, "MusicJsonCallbacktoplist(", ")");
		
SaveData saveAction = new SaveData();
task.start(saveAction);
cn.miw

蜜友工作室

蜜友工作室

Versions

Version
0.0.2
0.0.1