其他分享
首页 > 其他分享> > 搬运工

搬运工

作者:互联网

本文作者:hukey

本文链接:https://www.cnblogs.com/hukey/p/16469999.html

手把手系列 - 使用docker容器搭建efk8

 

目录


  1. Docker容器最佳实践
  2. 手把手系列 - 搭建 EFK 7 收集 docker 容器日志
  3. 手把手系列 - 搭建 EFK 8 收集 docker 容器日志
  4. 手把手系列 - 使用docker容器搭建efk8

前言

使用 docker 容器部署 EFK(elasticsearch+filebeat+kibana) 日志系统。

 

root@efk-node(192.168.1.101)/opt/efk> tree -L 2
.
├── elasticsearch # elasticsearch:8.2.2
│   ├── data # es数据存储目录
│   ├── logs # es日志目录
│   ├── plugins # 插件目录
│   └── start.sh # 容器启动脚本
├── filebeat # filebeat:8.2.2
│   ├── filebeat.docker.yml # 收集 docker 容器日志
│   └── start.sh # 容器启动脚本
├── images # 整体打包的基础镜像(elasticsearch:8.2.2 | filebeat:8.2.2 | kibana:8.2.2)
│   └── efk-images-8.2.2.tar.gz
└── kibana # kibana:8.2.2
└── start.sh # 容器启动脚本

7 directories, 5 files

 

下载链接:

链接:https://pan.baidu.com/s/1HfT6_S_52fxXXBjXRp2Faw?pwd=hkey
提取码:hkey
获取文件目录:docker / efk8.2.2 / efk-8.2.2.zip

系统版本

System: CentOS Linux release 7.9.2009 (Core)
Kernel: 3.10.0-1160.el7.x86_64

docker 版本

 

Docker-CE
* Server Version: 20.10.7
* Storage Driver: overlay2

实现过程

主机名ip地址
efk-node 192.168.1.101

系统初始化

系统初始化分为以下几步:

  1. 修改主机名
  2. 关闭selinux 和 firewalld
  3. 配置国内yum源
  4. 校对时间

修改主机名

 

>hostnamectl set-hostname efk-node
>hostname efk-node
>echo "192.168.1.101 efk-node" >> /etc/hosts

#断开会话重新连接
root@efk-node(192.168.1.101)/root>hostname
efk-node

关闭selinux 和 firewalld

 

### 关闭 selinux
>sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
>systemctl disable firewalld
>reboot

配置国内yum源

 

>cd /etc/yum.repos.d/
#centos-7源
>curl http://mirrors.aliyun.com/repo/Centos-7.repo -o ./Centos-7.repo
>sed -i '/aliyuncs/d' Centos-7.repo

#docker-ce源
>curl http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -o ./docker-ce.repo

#epel-7源
>curl http://mirrors.aliyun.com/repo/epel-7.repo -o ./epel-7.repo

#efk源
>cat << EOF > elasticstack.repo
[elasticstack]
name = elasticstack
gpgcheck = 0
baseurl = https://mirrors.tuna.tsinghua.edu.cn/elasticstack/yum/elastic-8.x/
EOF

校对时间

 

>yum install -y ntpdate
>ntpdate tiger.sina.com.cn

Docker-ce

  1. 安装docker-ce

>yum install -y docker-ce

  1. 添加docker-ce 配置
 

>mkdir /etc/docker/
>cat << 'EOF' > /etc/docker/daemon.json
{
"log-driver": "json-file",
"log-opts": {
"max-size": "100m",
"max-file": "3"
},
"exec-opts": ["native.cgroupdriver=systemd"],
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"registry-mirrors": [
"https://docker.mirrors.ustc.edu.cn",
"https://hub-mirror.c.163.com"
]
}
EOF

  1. 启动 docker
systemctl enable docker; systemctl start docker

查看docker 信息

 
>cat << 'EOF' >> /etc/sysctl.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
>sysctl --system

>docker info

Elasticsearch

将下载好的压缩包上传到服务器解压

> unzip efk-8.2.2.zip -d /opt/

赋予权限(否则后面容器启动时,会报权限问题。)

 
> chown -R 1000:1000 /opt/efk/

导入镜像

> cd /opt/efk/images/
> docker load < efk-images-8.2.2.tar.gz

启动 elasticsearch

> cd /opt/efk/elasticsearch/
> ./start.sh

第一次启动 elasticsearch 容器时,控制台会打印很多日志信息,主要获取了下面信息,就可以多次使用 Ctrl+C 停止容器。

image-20220712140929332

图里打印的信息就是 elasticsearch的口令及 kibana token信息。需要记录下来。

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-> Elasticsearch security features have been automatically configured!
-> Authentication is enabled and cluster connections are encrypted.

->  Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`):
  R+80iMZAShBNrOfFW2*k

->  HTTP CA certificate SHA-256 fingerprint:
  56bb845e700c9a57161c707f90c946491a46c42bfdae218ccee7c71a17e13ff7

->  Configure Kibana to use this cluster:
* Run Kibana and click the configuration link in the terminal when Kibana starts.
* Copy the following enrollment token and paste it into Kibana in your browser (valid for the next 30 minutes):
  eyJ2ZXIiOiI4LjIuMiIsImFkciI6WyIxNzIuMTguMC4yOjkyMDAiXSwiZmdyIjoiNTZiYjg0NWU3MDBjOWE1NzE2MWM3MDdmOTBjOTQ2NDkxYTQ2YzQyYmZkYWUyMThjY2VlN2M3MWExN2UxM2ZmNyIsImtleSI6IkhBXzc4SUVCZ0tWT0R6X081cHM5OnlJU0I0MlBKUlVXd3JOb3AyNWFWelEifQ==

-> Configure other nodes to join this cluster:
* Copy the following enrollment token and start new Elasticsearch nodes with `bin/elasticsearch --enrollment-token <token>` (valid for the next 30 minutes):
  eyJ2ZXIiOiI4LjIuMiIsImFkciI6WyIxNzIuMTguMC4yOjkyMDAiXSwiZmdyIjoiNTZiYjg0NWU3MDBjOWE1NzE2MWM3MDdmOTBjOTQ2NDkxYTQ2YzQyYmZkYWUyMThjY2VlN2M3MWExN2UxM2ZmNyIsImtleSI6IkhnXzc4SUVCZ0tWT0R6X081cHRLOlZKR0l3VC1lUkE2N1VWYVlyU1dCaGcifQ==

  If you're running in Docker, copy the enrollment token and run:
  `docker run -e "ENROLLMENT_TOKEN=<token>" docker.elastic.co/elasticsearch/elasticsearch:8.2.2`
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

如果这里没有记录,后面也可通过指令来修改。上面时效时间为 30分钟。

当获取到上面信息后,再次启动 elasticsearch 容器。

> docker start elasticsearch

通过浏览器访问 https://192.168.1.101:9200

用户名:elastic
密码:R+80iMZAShBNrOfFW2*k

image-20220712141417029

Kibana

 
> cd /opt/efk/kibana
> ./start.sh

浏览器访问 http://192.168.1.101:5601

kibana token:
eyJ2ZXIiOiI4LjIuMiIsImFkciI6WyIxNzIuMTguMC4yOjkyMDAiXSwiZmdyIjoiNTZiYjg0NWU3MDBjOWE1NzE2MWM3MDdmOTBjOTQ2NDkxYTQ2YzQyYmZkYWUyMThjY2VlN2M3MWExN2UxM2ZmNyIsImtleSI6IkhBXzc4SUVCZ0tWT0R6X081cHM5OnlJU0I0MlBKUlVXd3JOb3AyNWFWelEifQ==

image-20220712141725211

image-20220712141738252

查看验证码:

> docker logs -f kibana
...
Your verification code is:  257 354

image-20220712141830768

用户名:elastic
密码:R+80iMZAShBNrOfFW2*k

image-20220712141905110

Filebeat

这里采用了一个收集容器日志的示例来做演示,具体更多使用,参考官方文档及目录中其他文章。

> cd /opt/efk/filebeat/


### 这里采用了收集容器一个实例来做演示
> cat filebeat.docker.yml
filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false

filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true

processors:
- add_cloud_metadata: ~
- drop_fields:
    fields: ["log","docker","agent","ecs","host","log.offset","agent.hostname","container.id","agent.id"]

output.elasticsearch:
  hosts: '${ELASTICSEARCH_HOSTS:elasticsearch:9200}'
  ssl.verification_mode: "none"
  username: '${ELASTICSEARCH_USERNAME:}'
  password: '${ELASTICSEARCH_PASSWORD:}'

修改 启动脚本中 elasticsearch 密码:

> vim start.sh

image-20220712142614014

启动容器

 
> ./start.sh

配置索引

启动三个容器后,filebeat 会收集日志并存入 elasticsearch 中, 查看索引信息

image-20220712142845993

然后通过,kibana 配置展示索引

image-20220712142947162

image-20220712143024711

   

标签:容器,8.2,repo,efk,elasticsearch,搬运工,docker
来源: https://www.cnblogs.com/sheepships/p/16470078.html