Springboot + ElasticSearch 构建博客检索系统
作者:互联网
项目效果预览:
安装成功的效果图:
kibana安装:
ES使用的是倒排索引
参考:https://www.yuque.com/gaohanghang/vx5cb2/aa576g#HuZ1N
《Springboot + ElasticSearch 构建博客检索系统》
视频地址:https://www.imooc.com/learn/1161
代码地址:https://github.com/gaohanghang/springboot-blog-es
我所使用的 elasticsearch、logstash、kibana的版本均为 7.5.0 最新版
简介:从实际需求分析开始,打造个人博客检索系统。内容涵盖 ES安装、ES基本概念和数据类型、Mysql 到 ES 数据同步、SpringBoot 操作 ES。通过本课,让学员对ES有一个初步认识,理解ES的一些适用场景,以及如何使用springboot来同ES进行交互。
第1章 课程介绍
1-1 课程导学
- 可以反复看
- 上手做
- 学会应用
第2章 初识 ElassticSearch
2-1 ElasticSearch 概念和适用场景
2-2 ElasticSearch 数据类型,和关系型数据库对比
2-3 安装 ES、postman、Kibana
2-4 演示 postman、kibana对ES的交互
PostMan
Get 查看所有索引 localhost:9200/_all PUT 创建索引-test localhost:9200/test DEL 删除索引-test localhost:9200/test PUT 创建索引-person-1 localhost:9200/person PUT 新增数据-person-1 localhost:9200/person/_doc/1 { "first_name" : "John", "last_name" : "Smith", "age" : 25, "about" : "I love to go rock climbing", "interests" : [ "sports", "music" ] } PUT 新增数据-person-2 localhost:9200/person/_doc/2 { "first_name" : "Eric", "last_name" : "Smith", "age" : 23, "about" : "I love basketball", "interests" : [ "sports", "reading" ] } GET 搜索数据-person-id localhost:9200/person/_doc/1 GET 搜索数据-person-name localhost:9200/person/_doc/_search?q=first_name:john { "took": 56, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 1, "relation": "eq" }, "max_score": 0.6931472, "hits": [ { "_index": "person", "_type": "_doc", "_id": "1", "_score": 0.6931472, "_source": { "first_name": "John", "last_name": "Smith", "age": 25, "about": "I love to go rock climbing", "interests": [ "sports", "music" ] } } ] } }
Kibana
http://localhost:5601/app/kibana
GET _search { "query": { "match_all": {} } } GET _all GET /person/_doc/1 POST /person/_search { "query": { "bool": { "should": [ {"match": { "first_name": "Eric" } } ] } } POST /person/_search { "query": { "bool": { "should": [ {"match": { "last_name": "Smith" } }, { "match": { "about": "basketball" } } ] } } } POST /person/_search { "query": { "bool": { "must": [ {"match": { "last_name": "Smith" } }, { "match": { "about": "basketball" } } ] } } }
第3章 博客网站全文检索
3-1 基于 Mysql 实现
CREATE DATABASE blog; USE blog; CREATE TABLE `t_blog` ( `id` int(11) NOT NULL AUTO_INCREMENT COMMENT '自增id', `title` varchar(60) DEFAULT NULL COMMENT '博客标题', `author` varchar(60) DEFAULT NULL COMMENT '博客作者', `content` mediumtext COMMENT '博客内容', `create_time` datetime DEFAULT NULL COMMENT '创建时间', `update_time` datetime DEFAULT NULL COMMENT '更新时间', PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=9 DEFAULT CHARSET=utf8mb4 # 自己造的几条数据 INSERT INTO `blog`.`t_blog`(`id`, `title`, `author`, `content`, `create_time`, `update_time`) VALUES (1, 'Springboot 为什么这', 'bywind', '没错 Springboot ', '2019-12-08 01:44:29', '2019-12-08 01:44:34'); INSERT INTO `blog`.`t_blog`(`id`, `title`, `author`, `content`, `create_time`, `update_time`) VALUES (3, 'Springboot 中 Redis', 'bywind', 'Spring Boot', '2019-12-08 01:44:29', '2019-12-08 01:44:29'); INSERT INTO `blog`.`t_blog`(`id`, `title`, `author`, `content`, `create_time`, `update_time`) VALUES (4, 'Springboot 中如何优化', 'bywind', NULL, '2019-12-08 01:44:29', '2019-12-08 01:44:29'); INSERT INTO `blog`.`t_blog`(`id`, `title`, `author`, `content`, `create_time`, `update_time`) VALUES (5, 'Springboot 消息队列', 'bywind', NULL, '2019-12-08 01:44:29', '2019-12-08 01:44:29'); INSERT INTO `blog`.`t_blog`(`id`, `title`, `author`, `content`, `create_time`, `update_time`) VALUES (6, 'Docker Compose + Springboot', 'bywind', NULL, '2019-12-08 01:44:29', '2019-12-08 01:44:29');
SELECT * FROM t_blog WHERE title LIKE "%spring%" or content LIKE "%spring%"
3-2 基于ES实现
第4章 Mysql、ES 数据同步
4-1 数据同步中间件
不足:不支持 ES6.X 以上、Mysql 8.X 以上
time 标识最大时间
4-2 logstash全量、增量同步解决方案
https://www.elastic.co/cn/downloads/logstash
jar 包下载地址
https://mvnrepository.com/artifact/mysql/mysql-connector-java/5.1.31
mysql.conf
input{ jdbc{ # jdbc驱动包位置 jdbc_driver_library => "/Users/gaohanghang/software/0ELK7.5/logstash-7.5.0/mysql-connector-java-5.1.31.jar" # 要使用的驱动包类 jdbc_driver_class => "com.mysql.jdbc.Driver" # mysql数据库的连接信息 jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/blog" # mysql用户 jdbc_user => "root" # mysql密码 jdbc_password => "root" # 定时任务,多久执行一次查询,默认一分钟,如果想要没有延迟,可以使用 schedule => "* * * * * *" schedule => "* * * * *" # 清空上传的sql_last_value记录 clean_run => true # 你要执行的语句 statement => "select * FROM t_blog WHERE update_time > :sql_last_value AND update_time < NOW() ORDER BY update_time desc" } } output { elasticsearch{ # es host : port hosts => ["127.0.0.1:9200"] # 索引 index => "blog" # _id document_id => "%{id}" } }
bin/logstash -f config/mysql.conf
会报错,错误信息:
Unable to find driver class via URLClassLoader in given driver jars: com.mysql.jdbc.Driver and com.mysql.jdbc.Driver
我用的 logstash7.5.0
这里只是一个解决方法。只需将驱动程序Jar文件复制到<logstash_install_dir>/logstash-core/lib/jars/目录。
然后删除conf里的jdbc驱动包配置
mysql.conf
input{ jdbc{ # 要使用的驱动包类 jdbc_driver_class => "com.mysql.jdbc.Driver" # mysql数据库的连接信息 jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/blog" # mysql用户 jdbc_user => "root" # mysql密码 jdbc_password => "root" # 定时任务,多久执行一次查询,默认一分钟,如果想要没有延迟,可以使用 schedule => "* * * * * *" schedule => "* * * * *" # 清空上传的sql_last_value记录 clean_run => true # 你要执行的语句 statement => "select * FROM t_blog WHERE update_time > :sql_last_value AND update_time < NOW() ORDER BY update_time desc" } } output { elasticsearch{ # es host : port hosts => ["127.0.0.1:9200"] # 索引 index => "blog" # _id document_id => "%{id}" } }
成功
GET /blog/_stats GET /blog/_search
第 5 章
5-1 分词器介绍
5-2 IK分词器的安装和使用
POST _analyze { "analyzer": "standard", "text" : "hello imooc" }
POST _analyze { "analyzer": "standard", "text" : "我是中国人" }
ik分词器下载地址:
https://github.com/medcl/elasticsearch-analysis-ik/releases
启动报错
https://github.com/medcl/elasticsearch-analysis-ik/issues/384
ES存放文件路径中不能带有空格啊!!!!
POST _analyze { "analyzer": "ik_smart", "text" : "我是中国人" }
POST _analyze { "analyzer": "ik_max_word", "text" : "我是中国人" }
POST _analyze { "analyzer": "ik_max_word", "text" : "我是慕课网" }
字典添加慕课网后
5-3 springboot 项目搭建
server: port: 8081 spring: #数据库配置 datasource: driver-class-name: com.mysql.cj.jdbc.Driver url: jdbc:mysql://localhost:3306/blog?useUnicode=true&characterEncoding=utf-8 username: root password: root # hikari 数据源专用配置 hikari: maximum-pool-size: 20 minimum-idle: 5 # jpa 相关配置 jpa: hibernate: ddl-auto: update show-sql: true # 数据库方言 database-platform: org.hibernate.dialect.MySQLDialect # es 配置 # mvc 静态资源映射 mvc: static-path-pattern: /** # 静态资源热部署 devtools: livereload: enabled: true restart: additional-paths: static/** # 日期格式化 jackson: date-format: yyyy-MM-dd HH:mm:ss
代码地址:https://github.com/gaohanghang/springboot-blog-es
5-4 项目结构和JPA演示
5-5 springboot 集成ES
5-6 项目后端 REST API 实现
第 6 章
6-1 课程回顾与总结
标签:jdbc,Springboot,time,检索系统,blog,person,ElasticSearch,mysql,id 来源: https://www.cnblogs.com/JasonPeng1/p/12238489.html