Solr OSGi HTTP

Solr OSGi HTTP endpoints.

License

License

GroupId

GroupId

hu.blackbelt
ArtifactId

ArtifactId

solr-osgi-http
Last Version

Last Version

1.0.2
Release Date

Release Date

Type

Type

bundle
Description

Description

Solr OSGi HTTP
Solr OSGi HTTP endpoints.

Download solr-osgi-http

Dependencies

compile (2)

Group / Artifact Type Version
org.apache.solr : solr-core jar 7.0.1
org.slf4j : slf4j-api jar 1.7.25

provided (8)

Group / Artifact Type Version
org.osgi : osgi.core jar 6.0.0
org.osgi : osgi.cmpn jar 6.0.0
org.osgi : org.osgi.service.component.annotations jar 1.3.0
org.osgi : org.osgi.annotation jar 6.0.0
org.osgi : org.osgi.service.metatype.annotations jar 1.3.0
org.projectlombok : lombok jar 1.16.16
com.google.code.findbugs : annotations jar 3.0.1
com.google.code.findbugs : jsr305 jar 3.0.1

test (2)

Group / Artifact Type Version
org.slf4j : slf4j-simple jar 1.7.25
junit : junit jar 4.12

Project Modules

There are no modules declared in this project.

Solr OSGi

This project goal is to be able to run Apache Solr in OSGi environment and bring the OSGi dynamic lifecycle to the Solr world.

Note

There is no all of the Solr features supported yet, clustering and sharding haven’t been tested. In theory every piece of feature can be implemented in the future. Cellar based SolCloud implementation is planned. Now only configSet based cores can be used.

How it works?

There are a declarative component called SolrCoreContainerManager which monitors the following items:

  • solr.core or solr.core-<pid> configuration PID-s in ConfigurationAdmin service. This configuration defines a core cofigurations. The properties are very same as can be used in CoreAdmin API. Custom properties can be defined in admin-requests, 'property' prefix can be used on custom properties. Two mandatory properties are: name and configSet.

  • All bundles which have the Solr-Configset tag in META-INF/MANIFEST.MF. It loades configSets to CoreContainer. The cores can use thats configSets as a template. The configSets are stored to an in-Memory filesystem which is used to load files by solr.

The manager creates SolrClient instances when the configSet loaded which referenced by solr.core confugurations. In the future the core definitions which does not have configSet can be created as normal cores, but for that some file based resource management implementation is required.

When a core config or configSet bundle removed the related solrClients are stopping. The data directory is not removed, so it can be reused later.

Installation and usage

Any OSGi 6.0 container canbe used for deployment. It is tested with Apache Karaf 4.1.2.

To install in karaf console:

feature:repo-add cxf
feature:repo-add mvn:hu.blackbelt/solr-osgi-feature/<version>/xml/features
feature:install -v solr-http

To be able to use the SolrCoreContainerManager’s functionality have to create a configuration with pid solr.corecontainer

Create a file solr.corecontainer.cfg in karaf/deploy with the following content:

solrHome=/tmp/solr
karaf@root()> log:tail
2018-08-22 00:33:01,207 | INFO  | tures-1-thread-1 | HttpServiceFactoryImpl           | 181 - org.ops4j.pax.web.pax-web-runtime - 6.0.6 | Binding bundle: [org.restlet.ext.servlet [185]] to http service
2018-08-22 00:33:01,208 | DEBUG | tures-1-thread-1 | HttpServiceStarted               | 181 - org.ops4j.pax.web.pax-web-runtime - 6.0.6 | Creating http service for: org.restlet.ext.servlet [185]
2018-08-22 00:33:01,210 | DEBUG | tures-1-thread-1 | HttpServiceProxy                 | 181 - org.ops4j.pax.web.pax-web-runtime - 6.0.6 | HttpServiceProxy created for HttpService org.ops4j.pax.web.service.internal.HttpServiceStarted@6de4a397 for bundle org.restlet.ext.servlet [185]
2018-08-22 00:33:01,214 | INFO  | tures-1-thread-1 | FeaturesServiceImpl              | 9 - org.apache.karaf.features.core - 4.1.2 |   hu.blackbelt.solr-osgi-services/1.0.0.SNAPSHOT
2018-08-22 00:33:01,342 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT |  ___      _       Welcome to Apache Solr? version 7.0.1
2018-08-22 00:33:01,344 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | / __| ___| |_ _   Starting in standalone mode
2018-08-22 00:33:01,344 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | \__ \/ _ \ | '_|
2018-08-22 00:33:01,358 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | |___/\___/_|_|    Start time: 2018-08-21T22:33:01.345Z
2018-08-22 00:33:01,459 | INFO  | tures-1-thread-1 | SolrResourceLoader               | 147 - org.apache.solr - 7.0.1.1 | solr home defaulted to 'solr/' (could not find system property or JNDI)
2018-08-22 00:33:01,867 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: /
2018-08-22 00:33:01,869 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: /
2018-08-22 00:33:01,882 | INFO  | tures-1-thread-1 | SolrXmlConfig                    | 147 - org.apache.solr - 7.0.1.1 | MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44, but no JMX reporters were configured - adding default JMX reporter.
2018-08-22 00:33:02,120 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: /
2018-08-22 00:33:02,120 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: /
2018-08-22 00:33:02,121 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: /
2018-08-22 00:33:02,382 | WARN  | tures-1-thread-1 | LogWatcher                       | 147 - org.apache.solr - 7.0.1.1 | Unable to read SLF4J version.  LogWatcher will be disabled: java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder
2018-08-22 00:33:02,390 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.ZookeeperInfoHandler Expected tyoe: org.apache.solr.handler.admin.ZookeeperInfoHandler
2018-08-22 00:33:02,402 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.CollectionsHandler Expected tyoe: org.apache.solr.handler.admin.CollectionsHandler
2018-08-22 00:33:04,662 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.InfoHandler Expected tyoe: org.apache.solr.handler.admin.InfoHandler
2018-08-22 00:33:04,672 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.CoreAdminHandler Expected tyoe: org.apache.solr.handler.admin.CoreAdminHandler
2018-08-22 00:33:04,690 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.ConfigSetsHandler Expected tyoe: org.apache.solr.handler.admin.ConfigSetsHandler
2018-08-22 00:33:04,696 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.MetricsHandler Expected tyoe: org.apache.solr.handler.admin.MetricsHandler
2018-08-22 00:33:04,697 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.MetricsCollectorHandler Expected tyoe: org.apache.solr.handler.admin.MetricsCollectorHandler
2018-08-22 00:33:04,706 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.cloud.autoscaling.AutoScalingHandler Expected tyoe: org.apache.solr.cloud.autoscaling.AutoScalingHandler
2018-08-22 00:33:04,718 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.metrics.reporters.SolrJmxReporter Expected tyoe: org.apache.solr.metrics.SolrMetricReporter
2018-08-22 00:33:04,751 | INFO  | tures-1-thread-1 | SolrJmxReporter                  | 147 - org.apache.solr - 7.0.1.1 | JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44
2018-08-22 00:33:04,752 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.metrics.reporters.SolrJmxReporter Expected tyoe: org.apache.solr.metrics.SolrMetricReporter
2018-08-22 00:33:04,753 | INFO  | tures-1-thread-1 | SolrJmxReporter                  | 147 - org.apache.solr - 7.0.1.1 | JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44
2018-08-22 00:33:04,753 | INFO  | tures-1-thread-1 | OsgiSolrResourceLoader           | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.metrics.reporters.SolrJmxReporter Expected tyoe: org.apache.solr.metrics.SolrMetricReporter
2018-08-22 00:33:04,754 | INFO  | tures-1-thread-1 | SolrJmxReporter                  | 147 - org.apache.solr - 7.0.1.1 | JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44
2018-08-22 00:33:04,793 | INFO  | tures-1-thread-1 | CorePropertiesLocator            | 147 - org.apache.solr - 7.0.1.1 | Found 0 core definitions underneath /

To add a configSet template, copy the solr-osgi-example-configsets-exampleCollection projects jar to karaf/deploy

2018-08-22 00:33:04,850 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Solr content found in bundle hu.blackbelt.solr-osgi-example-configsets-exampleCollection Registering
2018-08-22 00:33:04,852 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/lang/stopwords_en.txt to configsets/exampleCollection/conf/lang/stopwords_en.txt
2018-08-22 00:33:04,858 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/params.json to configsets/exampleCollection/conf/params.json
2018-08-22 00:33:04,859 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/protwords.txt to configsets/exampleCollection/conf/protwords.txt
2018-08-22 00:33:04,859 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/schema.xml to configsets/exampleCollection/conf/schema.xml
2018-08-22 00:33:04,860 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/solrconfig.xml to configsets/exampleCollection/conf/solrconfig.xml
2018-08-22 00:33:04,861 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/stopwords.txt to configsets/exampleCollection/conf/stopwords.txt
2018-08-22 00:33:04,861 | INFO  | tures-1-thread-1 | SolrCoreContainerManager         | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/synonyms.txt to configsets/exampleCollection/conf/synonyms.txt
2018-08-22 00:33:04,869 | INFO  | tures-1-thread-1 | SolrCoreContainerMan

The configSet files processed and copied to in-memory store.

To create a core based on a configSet, create a configuration with PID solr.core-<pid>

In karaf create a file solr.core-test1.cfg in karaf/deploy with the following content:

name=test1
configSet=exampleCollection

HTTP access

To install HTTP service create configuration with PID solr.http

In karaf create a file solr.http.cfg in karaf/deploy with the following content:

contextRoot=/solr

After the Solr Admin can be accessed http://localhost:8181/solr/

Core container parameters

Core containers have very same parameters as the original ones, but not all of them can be used now.

hu.blackbelt

BlackBelt Technology

Versions

Version
1.0.2
1.0.1
1.0.0