简介
Solr是一种开放源码的、底层的核心技术是使用Lucene 来实现的搜索引擎。
OK,这里提到了search engine,《solr in action》中,详细说明了search engine的适用场景,以及和 DB的区别,对我收获很大,摘抄一段:
1. Search engine
Search engines like Solr are optimized to handle data exhibiting four main characteristics:
- Text-centric
- Read- dominant
- Document- oriented
- Flexible schema
1.1 Text-centric
A search engine supports non text data such as dates and numbers, but its primary strength is handling text data based on natural language
.
If users aren’t interested in the information in the text, a search engine may not be the best solution for your problem.
Think about whether your data is text-centric. The main consideration is whether or not the text fields in your data contain information that users will want to query.
Solr等搜索引擎为搜索包含自然语言的文本内容做了优化,比如电子邮件,网页,简历,PDF文档,或是推特、微博、博客这些社交内容等等,都适合用Solr来处理。
1.2 Read- dominant
Think of read-dominant as meaning that documents are read far more often than they’re created or updated.
If you must update existing data in a search engine often, that could be an indication that a search engine might not be the best solution for your needs. Another NoSQL technology, like Cassandra, might be a better choice when you need fast random writes to existing data.
1.3 Document-oriented
In a search engine, a document is a self-contained collection of fields, in which each field only holds data(can have multiple values) and doesn’t contain subfields.
A search engine isn’t the place to store data unless it’s useful for search or displaying results
In general, you should store the minimal set of information for each document needed to satisfy search requirements.
1.4 Flexible schema
In a relational database, every row in a table has the same structure. In Solr, documents can have different fields.
2. Solr vs Lucene
两者的区别有:
- Lucene本质上是搜索库,不是独立的应用程序,而Solr是
- Lucene专注于搜索底层的建设,而Solr专注于企业应用
- Lucene不负责支撑搜索服务所必须的管理,而Solr负责
所以说,一句话概括: Solr是Lucene面向企业搜索应用的扩展
。
Solr 提供了层面搜索、命中醒目显示并且支持多种输出格式(包括XML/XSLT 和JSON等格式),它附带了一个基于HTTP 的管理界面。Solr的特性包括:
- 高级的全文搜索功能
- 一个真正的拥有动态字段(Dynamic Field)和唯一键(Unique Key)的数据模式(Data Schema)
- 专为高通量的网络流量进行的优化
- 基于开放接口(XML和HTTP)的标准
- 综合的HTML管理界面
- 可伸缩性-能够有效地复制到另外一个Solr搜索服务器
- 使用XML配置达到灵活性和适配性
- 可扩展的插件体系
- 支持对结果进行动态的分组和过滤
- 高度可配置和可扩展的缓存机制
因为 Solr 包装并扩展了Lucene,所以它们使用很多相同的术语。
2. solr 配置
2.1 solrconfig.xml
定义solr的处理程序(handler)和一些扩展程序。其中的配置很多,其实很多都可以保持默认。
- dataDir:索引存放位置
- autoCommit:solr在建索引的时候收到请求并没用立即写入文件,而是先放到缓存中,等收到commit命令时才将缓存中得数据写入索引文件。
- maxDocs:
Maximum number of documents to add since the last commit before automatically triggering a new commit.
- maxTime:
Maximum amount of time in ms that is allowed to pass since a document was added before automatically triggering a new commit.
- openSearcher:
if false, the commit causes recent index changes to be flushed to stable storage, but does not cause a new searcher to be opened to make those changes visible.
- autoSoftCommit:
softAutoCommit is like autoCommit except it causes a 'soft' commit which only ensures that changes are visible but does not ensure that data is synced to disk. This is faster and more near-realtime friendly than a hard commit.
2.2 manage-schema
用于定义索引的字段和字段类型
2.2.1 fieldType:字段类型(int、float、string、ik...)
<fieldType name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
<fieldType name="boolean" class="solr.BoolField" sortMissingLast="true"/>
<fieldType name="int" class="solr.TrieIntField" precisionStep="0" positionIncrementGap="0"/>
<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
</analyzer>
</fieldType>
<fieldType name="text_ik" class="solr.TextField" sortMissingLast="true" omitNorms="true" autoGeneratePhraseQueries="false">
<analyzer type="index" isMaxWordLength="false" class="org.wltea.analyzer.lucene.IKAnalyzer"/>
<analyzer type="query" isMaxWordLength="true" class="org.wltea.analyzer.lucene.IKAnalyzer"/>
</fieldType>
2.2.2 field:字段,定义需要的字段名和它的类型
- name 字段名
- type 字段类型
- indexed 是否进行索引
- stored 是否进行保存,如不保存,可以进行搜索,但不能显示该字段的内容
- required 是否是必须字段,如若是,该字段必须有值,否则索引报错
- multiValued 是否允许多值
- docValues
- sortMissingLast
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
2.2.3 dynamicFields
动态字段表示,如果字段的定义没有在配置中找到,就在动态字段类型中进行查找
<dynamicField name="*_txt" type="text_general" indexed="true" stored="true" multiValued="true"/>
2.2.4 copyField
复制源字段到目标字段,maxchars 限制复制的最大长度
<copyField source="body" dest="teaser" maxChars="300"/>
2.2.5 uniqueKey
相当于数据库中得主键,如建索引时遇到重复的,则会覆盖掉以前的记录
<uniqueKey>id</uniqueKey>
2.2.6 defaultSearchField
如果搜索参数中没有指定具体的field,那么这是默认的域
<defaultSearchField>text</defaultSearchField>
2.2.7 solrQueryParser
配置搜索参数短语间的逻辑,可以是"AND | OR"。
<solrQueryParser defaultOperator="OR" />