18深度探秘搜索技术_在案例实战中掌握phrase matching搜索技术
作者:互联网
1、什么是近似匹配
两个句子
java is my favourite programming language, and I also think spark is a very good big data system.
java spark are very related, because scala is spark's programming language and scala is also based on jvm like java.
match query,搜索java spark
{
"match": {
"content": "java spark"
}
}
match query,只能搜索到包含java和spark的document,但是不知道java和spark是不是离的很近
包含java或包含spark,或包含java和spark的doc,都会被返回回来。我们其实并不知道哪个doc,java和spark距离的比较近。如果我们就是希望搜索java spark,中间不能插入任何其他的字符,那这个时候match去做全文检索,就不行了。
如果我们要尽量让java和spark离的很近的document优先返回,要给它一个更高的relevance score,这就涉及到了proximity match,近似匹配
如果说,要实现两个需求:
1、java spark,就靠在一起,中间不能插入任何其他字符,就要搜索出来这种doc
2、java spark,但是要求,java和spark两个单词靠的越近,doc的分数越高,排名越靠前
phrase match:短语匹配 ,proximity match:,近似匹配
这一讲,要学习的是phrase match,下一讲会讲proximity match,
phrase match就是仅仅搜索出java和spark靠在一起的那些doc,比如有个doc,是java use’d spark,不行。必须是比如java spark are very good friends,是可以搜索出来的。
就是要去将多个term作为一个短语,一起去搜索,只有包含这个短语的doc才会作为结果返回。不像是match,java spark,java的doc也会返回,spark的doc也会返回。
2、match_phrase
修改doc5的content,包含 java spark
POST /forum/article/5/_update
{
"doc": {
"content": "spark is best big data solution based on scala ,an programming language similar to java spark"
}
}
响应结果
{
"_index": "forum",
"_type": "article",
"_id": "5",
"_version": 9,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
}
}
将一个doc的content设置为恰巧包含java spark这个短语
使用match
GET /forum/article/_search
{
"query": {
"match": {
"content": "java spark"
}
}
}
响应结果
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 0.68640786,
"hits": [
{
"_index": "forum",
"_type": "article",
"_id": "2",
"_score": 0.68640786,
"_source": {
"articleID": "KDKE-B-9947-#kL5",
"userID": 1,
"hidden": false,
"postDate": "2017-01-02",
"tag": [
"java"
],
"tag_cnt": 1,
"view_cnt": 50,
"title": "this is java blog",
"content": "i think java is the best programming language",
"sub_title": "learned a lot of course",
"author_first_name": "Smith",
"author_last_name": "Williams",
"new_author_last_name": "Williams",
"new_author_first_name": "Smith"
}
},
{
"_index": "forum",
"_type": "article",
"_id": "5",
"_score": 0.68324494,
"_source": {
"articleID": "DHJK-B-1395-#Ky5",
"userID": 3,
"hidden": false,
"postDate": "2021-11-11",
"tag": [
"elasticsearch"
],
"tag_cnt": 1,
"view_cnt": 10,
"title": "this is spark blog",
"content": "spark is best big data solution based on scala ,an programming language similar to java spark",
"sub_title": "haha, hello world",
"author_first_name": "Tonny",
"author_last_name": "Peter Smith",
"new_author_last_name": "Peter Smith",
"new_author_first_name": "Tonny"
}
}
]
}
}
单单包含java的doc也返回了,不是我们想要的结果
使用match_phrase语法
GET /forum/article/_search
{
"query": {
"match_phrase": {
"content": "java spark"
}
}
}
响应结果
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.5753642,
"hits": [
{
"_index": "forum",
"_type": "article",
"_id": "5",
"_score": 0.5753642,
"_source": {
"articleID": "DHJK-B-1395-#Ky5",
"userID": 3,
"hidden": false,
"postDate": "2021-11-11",
"tag": [
"elasticsearch"
],
"tag_cnt": 1,
"view_cnt": 10,
"title": "this is spark blog",
"content": "spark is best big data solution based on scala ,an programming language similar to java spark",
"sub_title": "haha, hello world",
"author_first_name": "Tonny",
"author_last_name": "Peter Smith",
"new_author_last_name": "Peter Smith",
"new_author_first_name": "Tonny"
}
}
]
}
}
成功了,只有包含java spark这个短语的doc才返回了,只包含java的doc不会返回
3、term position
doc1:hello world, java spark
doc2:hi, spark java
建立倒排索引并会记录这个关键字所在的position
hello doc1(0)
wolrd doc1(1)
java doc1(2) doc2(2)
spark doc1(3) doc2(1)
了解什么是分词后的position
GET _analyze
{
"text": "hello world, java spark",
"analyzer": "standard"
}
响应结果
{
"tokens": [
{
"token": "hello",
"start_offset": 0,
"end_offset": 5,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "world",
"start_offset": 6,
"end_offset": 11,
"type": "<ALPHANUM>",
"position": 1
},
{
"token": "java",
"start_offset": 13,
"end_offset": 17,
"type": "<ALPHANUM>",
"position": 2
},
{
"token": "spark",
"start_offset": 18,
"end_offset": 23,
"type": "<ALPHANUM>",
"position": 3
}
]
}
4、match_phrase的基本原理
索引中的position,match_phrase
doc1:hello world, java spark
doc2:hi, spark java
建立倒排索引并会记录这个关键字所在的position
hello doc1(0)
wolrd doc1(1)
java doc1(2) doc2(2)
spark doc1(3) doc2(1)
java spark 使用 match phrase搜索
java spark -->相当于搜索 java和spark
java --> doc1(2) doc2(2)
spark --> doc1(3) doc2(1)
要找到每个term都在的一个共有的那些doc,就是要求一个doc,必须包含每个term,并且java 的position比spark 的position小1,才能拿出来继续计算
doc1 --> java和spark --> spark position恰巧比java大1 --> java的position是2,spark的position是3,恰好满足条件
doc1符合条件
doc2 --> java和spark --> java position是2,spark position是1,spark position比java position小1,而不是大1 --> 光是position就不满足,那么doc2不匹配
必须理解这块原理!!!!
因为后面的proximity match就是原理跟这个一模一样!!!
标签:java,18,doc1,doc,搜索,position,phrase,spark,match 来源: https://blog.csdn.net/m0_37450089/article/details/121442993