com.github.zhangyingwei:cockroach

Sonatype helps open source projects to set up Maven repositories on https://oss.sonatype.org/

License

License

GroupId

GroupId

com.github.zhangyingwei
ArtifactId

ArtifactId

cockroach
Last Version

Last Version

1.0.6-Beta
Release Date

Release Date

Type

Type

jar
Description

Description

Sonatype helps open source projects to set up Maven repositories on https://oss.sonatype.org/
Source Code Management

Source Code Management

https://github.com/zhangyingwei/cockroach

Download cockroach

How to add to project

<!-- https://jarcasting.com/artifacts/com.github.zhangyingwei/cockroach/ -->
<dependency>
    <groupId>com.github.zhangyingwei</groupId>
    <artifactId>cockroach</artifactId>
    <version>1.0.6-Beta</version>
</dependency>
// https://jarcasting.com/artifacts/com.github.zhangyingwei/cockroach/
implementation 'com.github.zhangyingwei:cockroach:1.0.6-Beta'
// https://jarcasting.com/artifacts/com.github.zhangyingwei/cockroach/
implementation ("com.github.zhangyingwei:cockroach:1.0.6-Beta")
'com.github.zhangyingwei:cockroach:jar:1.0.6-Beta'
<dependency org="com.github.zhangyingwei" name="cockroach" rev="1.0.6-Beta">
  <artifact name="cockroach" type="jar" />
</dependency>
@Grapes(
@Grab(group='com.github.zhangyingwei', module='cockroach', version='1.0.6-Beta')
)
libraryDependencies += "com.github.zhangyingwei" % "cockroach" % "1.0.6-Beta"
[com.github.zhangyingwei/cockroach "1.0.6-Beta"]

Dependencies

compile (1)

Group / Artifact Type Version
log4j : log4j jar 1.2.17

Project Modules

  • cockroach-core
  • cockroach-annotation
  • cockroach-queue-redis

cockroach 爬虫:又一个 java 爬虫实现

License

重构了 cockroach2

简介

cockroach[小强] 当时不知道为啥选了这么个名字,又长又难记,导致编码的过程中因为单词的拼写问题耽误了好长时间。

这个项目算是我的又一个坑吧,算起来挖的坑多了去了,多一个不多少一个不少。

一个小巧、灵活、健壮的内容(pa)获取(chong)框架,暂且叫做框架吧。

简单到什么程度呢,几句话就可以创建一个内容(pa)获取(chong)程序。

依赖部分

<dependency>
  <groupId>com.github.zhangyingwei</groupId>
  <artifactId>cockroach-core</artifactId>
  <version>1.0.6-Beta</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.github.zhangyingwei/cockroach-annotation -->
<dependency>
    <groupId>com.github.zhangyingwei</groupId>
    <artifactId>cockroach-annotation</artifactId>
    <version>1.0.6-Beta</version>
</dependency>

代码部分:

@EnableAutoConfiguration
public class CockroachApplicationTest {
    public static void main(String[] args) throws Exception {
        TaskQueue queue = TaskQueue.of();
        queue.push(new Task("http://blog.zhangyingwei.com"));
        CockroachApplication.run(CockroachApplicationTest.class,queue);
    }
}

没错,就是这么简单。这个内容(pa)获取(chong)程序就是获(pa)取 http://blog.zhangyingwei.com 这个页面的内容并将结果打印出来。 在结果处理这个问题上,程序中默认使用 PringStore 这个类将所有结果打印出来。

scala & kotlin

作为目前使用的 jvm 系语言几大巨头,scala 与 kotlin 这里基本上对跟 java 的互调做的很好,但是这里还是给几个 demo。

scala

/**
  * Created by zhangyw on 2017/12/25.
  */
class TTTStore extends IStore{
    override def store(taskResponse: TaskResponse): Unit = {
        println("ttt store")
    }
}

object TTTStore{}
/**
  * Created by zhangyw on 2017/12/25.
  */
@EnableAutoConfiguration
@ThreadConfig(num = 1)
@Store(classOf[TTTStore])
object MainApplication {
    def main(args: Array[String]): Unit = {
        println("hello scala spider")
        val queue = TaskQueue.of()
        queue.push(new Task("http://blog.zhangyingwei.com"))
        CockroachApplication.run(MainApplication.getClass(),queue)
    }
}

kotlin

class TTTStore :IStore{
    override fun store(response: TaskResponse) {
        print("ttt store")
    }
}
/**
 * Created by zhangyw on 2017/12/25.
 */
@EnableAutoConfiguration
@ThreadConfig(num = 1)
@Store(TTTStore::class)
object MainApplication {
    @JvmStatic
    fun main(args: Array<String>) {
        print("hello kotlin spider")
        val queue = TaskQueue.of()
        queue.push(Task("http://blog.zhangyingwei.com"))
        CockroachApplication.run(MainApplication::class.java, queue)
    }
}

联系方式

Lisence

Lisenced under Apache 2.0 lisence

Versions

Version
1.0.6-Beta
1.0.5.04-Beta
1.0.5.03-Beta
1.0.5.03-Alpha
1.0.5.02-Beta
1.0.5.01-Beta
1.0.5-Beta
1.0.5-Alpha
1.0.4-Alpha
1.0.3-Alpha
1.0.2-Alpha
1.0-Alpha