ELK + Filebeat 手动部署指南（tar.gz 包方式）

基于 Linux 环境CentOS/Ubuntu 通用。前提条件安装 Java 11ELK/Logstash 必需bash运行# CentOS yum install -y java-11-openjdk-devel # Ubuntu apt install -y openjdk-11-jdk # 验证Java java -version # 输出openjdk version 11.x.x即可关闭防火墙 / 放行端口5601-Kibana、9200-ES、5044-Filebeat→Logstashbash运行# CentOS systemctl stop firewalld systemctl disable firewalld setenforce 0 sed -i s/^SELINUX.*/SELINUXdisabled/ /etc/selinux/config # Ubuntu ufw disable创建统一部署目录方便管理bash运行mkdir -p /usr/local/elk cd /usr/local/elk第一步下载 ELK Filebeat 安装包推荐下载8.x 稳定版所有组件版本保持一致避免兼容性问题bash运行# 下载地址可替换为国内镜像如阿里云https://mirrors.aliyun.com/elasticstack/8.x/yum/ # Elasticsearch wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.14.0-linux-x86_64.tar.gz # Logstash wget https://artifacts.elastic.co/downloads/logstash/logstash-8.14.0-linux-x86_64.tar.gz # Kibana wget https://artifacts.elastic.co/downloads/kibana/kibana-8.14.0-linux-x86_64.tar.gz # Filebeat wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.14.0-linux-x86_64.tar.gz # 解压所有包 tar -zxvf elasticsearch-8.14.0-linux-x86_64.tar.gz tar -zxvf logstash-8.14.0-linux-x86_64.tar.gz tar -zxvf kibana-8.14.0-linux-x86_64.tar.gz tar -zxvf filebeat-8.14.0-linux-x86_64.tar.gz # 简化目录名方便后续操作 mv elasticsearch-8.14.0 elasticsearch mv logstash-8.14.0 logstash mv kibana-8.14.0-linux-x86_64 kibana mv filebeat-8.14.0-linux-x86_64 filebeat第二步配置 Elasticsearch2.1 创建 ES 专用用户ES 不允许 root 启动bash运行useradd -r -s /sbin/nologin elasticsearch chown -R elasticsearch:elasticsearch /usr/local/elk/elasticsearch2.2 修改 ES 配置文件编辑/usr/local/elk/elasticsearch/config/elasticsearch.ymlyaml# 核心配置 cluster.name: elk-cluster node.name: node-1 path.data: /usr/local/elk/elasticsearch/data # 数据存储目录 path.logs: /usr/local/elk/elasticsearch/logs # 日志目录 network.host: 0.0.0.0 http.port: 9200 discovery.type: single-node # 单节点部署 xpack.security.enabled: false # 关闭安全认证新手简化 xpack.security.enrollment.enabled: false # 解决内核参数限制 bootstrap.memory_lock: false bootstrap.system_call_filter: false2.3 修改系统内核参数解决 ES 启动报错编辑/etc/security/limits.conf添加inielasticsearch soft nofile 65535 elasticsearch hard nofile 65535 elasticsearch soft nproc 4096 elasticsearch hard nproc 4096编辑/etc/sysctl.conf添加inivm.max_map_count262144 fs.file-max655350生效配置bash运行sysctl -p ulimit -n 655352.4 启动 ES 设置开机自启1. 启动 ESbash运行# 切换到elasticsearch用户启动 su - elasticsearch -c /usr/local/elk/elasticsearch/bin/elasticsearch -d # 验证启动等待30秒ES启动较慢 curl http://localhost:9200 # 输出{name:node-1,cluster_name:elk-cluster,...}即成功2. 创建 systemd 服务实现开机自启编辑/usr/lib/systemd/system/elasticsearch.serviceini[Unit] DescriptionElasticsearch Afternetwork.target [Service] Typesimple Userelasticsearch Groupelasticsearch ExecStart/usr/local/elk/elasticsearch/bin/elasticsearch -d ExecStop/usr/local/elk/elasticsearch/bin/elasticsearch-stop.sh Restarton-failure LimitNOFILE65535 LimitNPROC4096 [Install] WantedBymulti-user.target设置开机自启bash运行systemctl daemon-reload systemctl enable elasticsearch第三步配置 Logstash3.1 创建 Logstash 配置文件bash运行mkdir -p /usr/local/elk/logstash/conf.d编辑/usr/local/elk/logstash/conf.d/nginx-log.confrubyinput { beats { port 5044 host 0.0.0.0 } } filter { grok { match { message %{COMBINEDAPACHELOG} } # 匹配Nginx默认日志格式 remove_field [message] } date { match [ timestamp, dd/MMM/yyyy:HH:mm:ss Z ] target timestamp } mutate { add_field { log_source nginx_access } } } output { elasticsearch { hosts [http://localhost:9200] index nginx-logs-%{YYYY.MM.dd} } # stdout { codec rubydebug } # 调试用可选开启 }3.2 启动 Logstash 设置开机自启1. 启动 Logstashroot 用户即可bash运行/usr/local/elk/logstash/bin/logstash -f /usr/local/elk/logstash/conf.d/nginx-log.conf # 验证启动查看日志 tail -f /usr/local/elk/logstash/logs/logstash-plain.log # 无ERROR即正常2. 创建 systemd 服务开机自启编辑/usr/lib/systemd/system/logstash.serviceini[Unit] DescriptionLogstash Afternetwork.target elasticsearch.service [Service] Typesimple Userroot Grouproot ExecStart/usr/local/elk/logstash/bin/logstash -f /usr/local/elk/logstash/conf.d/nginx-log.conf ExecStop/bin/kill -9 ps -ef | grep logstash | grep -v grep | awk {print $2} Restarton-failure [Install] WantedBymulti-user.target设置开机自启bash运行systemctl daemon-reload systemctl enable logstash第四步配置 Kibana4.1 修改 Kibana 配置文件编辑/usr/local/elk/kibana/config/kibana.ymlyamlserver.port: 5601 server.host: 0.0.0.0 elasticsearch.hosts: [http://localhost:9200] kibana.index: .kibana i18n.locale: zh-CN # 中文界面4.2 启动 Kibana 设置开机自启1. 启动 Kibanabash运行/usr/local/elk/kibana/bin/kibana --daemonize # 后台启动 # 验证启动访问http://服务器IP:5601首次加载约1分钟2. 创建 systemd 服务开机自启编辑/usr/lib/systemd/system/kibana.serviceini[Unit] DescriptionKibana Afternetwork.target elasticsearch.service [Service] Typesimple Userroot Grouproot ExecStart/usr/local/elk/kibana/bin/kibana --daemonize ExecStop/bin/kill -9 ps -ef | grep kibana | grep -v grep | awk {print $2} Restarton-failure [Install] WantedBymulti-user.target设置开机自启bash运行systemctl daemon-reload systemctl enable kibana第五步配置 Filebeat采集 Nginx 日志5.1 修改 Filebeat 配置文件编辑/usr/local/elk/filebeat/filebeat.ymlyamlfilebeat.inputs: - type: filestream enabled: true paths: - /elk/nginx/logs/access.log - /elk/nginx/logs/error.log # 自定义标签方便后续筛选 tags: [nginx-logs] # 关闭直接发送ES output.elasticsearch: enabled: false # 发送到Logstash output.logstash: hosts: [localhost:5044] # 启用Nginx模块可选 filebeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: false - module: nginx access: enabled: true var.paths: [/var/log/nginx/access.log] error: enabled: true var.paths: [/var/log/nginx/error.log]5.2 启动 Filebeat 设置开机自启1. 启动 Filebeatbash运行/usr/local/elk/filebeat/filebeat -c /usr/local/elk/filebeat/filebeat.yml # 验证状态 ps -ef | grep filebeat2. 创建 systemd 服务开机自启编辑/usr/lib/systemd/system/filebeat.serviceini[Unit] DescriptionFilebeat Afternetwork.target logstash.service [Service] Typesimple Userroot Grouproot ExecStart/usr/local/elk/filebeat/filebeat -c /usr/local/elk/filebeat/filebeat.yml ExecStop/bin/kill -9 ps -ef | grep filebeat | grep -v grep | awk {print $2} Restarton-failure [Install] WantedBymulti-user.target设置开机自启bash运行systemctl daemon-reload systemctl enable filebeat第六步验证全流程启动所有组件按顺序bash运行systemctl start elasticsearch systemctl start logstash systemctl start kibana systemctl start filebeat访问 Nginx 生成日志curl http://localhost。检查 ES 是否有数据bash运行curl http://localhost:9200/nginx-logs-$(date %Y.%m.%d)/_count输出count:1或更大表示数据写入成功。浏览器访问http://服务器IP:5601创建索引模式nginx-logs-*即可在 Discover 查看实时日志。常见问题解决ES 启动失败检查/usr/local/elk/elasticsearch/logs下的日志确保内核参数vm.max_map_count262144已生效。Filebeat 无权限读取 Nginx 日志bash运行chmod 644 /var/log/nginx/access.log chown -R root:root /var/log/nginx # 确保root能读取Logstash 端口 5044 被占用bash运行netstat -tlnp | grep 5044 # 查看占用进程 kill -9 进程ID # 杀死占用进程后重启Logstash总结手动部署核心是下载解压→配置文件→创建 systemd 服务实现开机自启所有组件需版本一致8.14.0。ES 必须用非 root 用户启动需提前配置系统内核参数和权限其他组件Logstash/Kibana/Filebeat可直接用 root 启动。核心流程不变Filebeat 采集 Nginx 日志→Logstash 解析→ES 存储→Kibana 可视化关键是确保 Logstash 的 grok 规则匹配 Nginx 日志格式。