一、Loki简介

1、简介

Loki Loki是一个开源、分布式的日志聚合系统，由Grafana Labs推出。Loki的设计目标是为了高效地处理大规模的日志数据，并具有良好的可扩展性。Loki的最大优点是它具有低资源占用和高效的查询速度。这是因为Loki不需要在处理日志数据时进行索引，而是将数据存储在类似于散列表的内存数据结构中，可以快速地定位和检索日志数据。此外，Loki可以与Prometheus集成，实现更强大的监控功能。

2、组件

• distributor：是负责获取日志数据并将其转发给ingester的无状态组件。Distributor对数据进行预处理，检查其有效性，并确保其来自已配置的租户，这有助于系统扩展并保护其免受潜在的Dos攻击。这里Grafana很好地解释了Promtail(推荐的分发代理)如何处理数据。
• ingester：是Loki架构的关键组件。从distributors接收到的数据由ingester写入云云原生存储服务。ingesters还与查询协作，返回内存中的数据以响应读请求。
• Queriers：负责解析LogQL查询请求，并从ingesters或持久化存储中获取数据。
• query fronted：可选组件-提供API接口，可用于加速读处理。该组件通过将读请求排队、将大请求拆分为多个小请求以及缓存数据来优化读处理。

二、实验环境

1、k8s环境

2、Prometheus+Grafana环境

三、Loki安装

1、下载镜像

[root@jenkins ~]# docker pull grafana/loki:2.8.2
[root@jenkins ~]# docker pull grafana/promtail:2.8.2

push到本地harbor：

[root@jenkins ~]# docker tag grafana/loki:2.8.2 harbor.wielun.com/library/grafana/loki:2.8.2
[root@jenkins ~]# docker tag grafana/loki:2.8.2 harbor.wielun.com/library/grafana/promtail:2.8.2
[root@jenkins ~]# docker push harbor.wielun.com/library/grafana/loki:2.8.2
[root@jenkins ~]# docker push harbor.wielun.com/library/grafana/promtail:2.8.2

2、启动loki

[root@jenkins ~]# mkdir -p /etc/loki/conf
[root@jenkins ~]# wget https://raw.githubusercontent.com/grafana/loki/v2.8.2/cmd/loki/loki-local-config.yaml -O loki-config.yaml
[root@jenkins ~]# mv loki-config.yaml /etc/loki/conf/
[root@jenkins ~]# docker run -d --name loki \
--privileged=true \
--restart always -p 3100:3100 \
-v /etc/loki/conf:/mnt/config \
grafana/loki:2.8.2 \
-config.file=/mnt/config/loki-config.yaml

[root@jenkins ~]# cat /etc/loki/conf/loki-config.yaml
auth_enabled: falseserver:http_listen_port: 3100grpc_listen_port: 9096common:instance_addr: 127.0.0.1path_prefix: /tmp/lokistorage:filesystem:chunks_directory: /tmp/loki/chunksrules_directory: /tmp/loki/rulesreplication_factor: 1ring:kvstore:store: inmemoryquery_range:results_cache:cache:embedded_cache:enabled: truemax_size_mb: 100schema_config:configs:- from: 2020-10-24store: boltdb-shipperobject_store: filesystemschema: v11index:prefix: index_period: 24hruler:alertmanager_url: http://localhost:9093# By default, Loki will send anonymous, but uniquely-identifiable usage and configuration
# analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/
#
# Statistics help us better understand how Loki is used, and they show us performance
# levels for most users. This helps us prioritize features and documentation.
# For more information on what's sent, look at
# https://github.com/grafana/loki/blob/main/pkg/usagestats/stats.go
# Refer to the buildReport method to see what goes into a report.
#
# If you would like to disable reporting, uncomment the following lines:
#analytics:
#  reporting_enabled: false

3、启动promtail

[root@jenkins ~]# wget https://raw.githubusercontent.com/grafana/loki/v2.8.2/clients/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml
[root@jenkins ~]# mv promtail-config.yaml /etc/loki/conf/
[root@jenkins ~]# docker run -d --name promtail \--privileged=true \--restart always \-v /etc/loki/conf/:/mnt/config \-v /var/log:/var/log \grafana/promtail:2.8.2 \-config.file=/mnt/config/promtail-config.yaml

[root@jenkins ~]# cat /etc/loki/conf/promtail-config.yaml
server:http_listen_port: 9080grpc_listen_port: 0positions:filename: /tmp/positions.yamlclients:- url: http://10.10.10.10:3100/loki/api/v1/pushscrape_configs:
- job_name: systemstatic_configs:- targets:- localhostlabels:job: varlogs__path__: /var/log/*log

4、测试结果

http://10.10.10.10:3100/metrics
http://10.10.10.10:3100/ready

四、grafana查看

1、添加Loki数据源

2、查看本机日志

五、监控K8s日志

1、上传镜像

[root@jenkins ~]# docker pull grafana/promtail:2.5.0
[root@jenkins ~]# docker tag grafana/promtail:2.5.0 harbor.wielun.com/library/grafana/promtail:2.5.0
[root@jenkins ~]# docker push harbor.wielun.com/library/grafana/promtail:2.5.0

2、创建Endpoints

[root@master01 ~]# kubectl create ns logging
[root@master01 ~]# cat loki.yaml
apiVersion: v1
kind: Endpoints
metadata:name: lokinamespace: logging
subsets:- addresses:- ip: 10.10.10.10ports:- port: 3100---
apiVersion: v1
kind: Service
metadata:name: lokinamespace: logging
spec:clusterIP: Noneports:- name: mysqlport: 3100protocol: TCPtargetPort: 3100type: ClusterIP[root@master01 ~]# kubectl apply -f loki.yaml

3、创建promtail

[root@master01 ~]# cat promtail.yaml
--- # Daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:name: promtail-daemonsetnamespace: logginglabels:app: promtail
spec:selector:matchLabels:name: promtailtemplate:metadata:labels:name: promtailspec:serviceAccount: promtail-serviceaccountcontainers:- name: promtail-containerimage: harbor.wielun.com/library/grafana/promtail:2.5.0imagePullPolicy: IfNotPresentargs:- -config.file=/etc/promtail/promtail.yamlenv:- name: 'HOSTNAME' # needed when using kubernetes_sd_configsvalueFrom:fieldRef:fieldPath: 'spec.nodeName'volumeMounts:- name: logsmountPath: /var/log- name: promtail-configmountPath: /etc/promtail- mountPath: /var/lib/docker/containersname: varlibdockercontainersreadOnly: truevolumes:- name: logshostPath:path: /var/log- name: varlibdockercontainershostPath:path: /var/lib/docker/containers- name: promtail-configconfigMap:name: promtail-config
--- # configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:name: promtail-confignamespace: logginglabels:app: promtail
data:promtail.yaml: |server:http_listen_port: 9080grpc_listen_port: 0clients:- url: http://loki.logging.svc.cluster.local:3100/loki/api/v1/pushpositions:filename: /tmp/positions.yamltarget_config:sync_period: 10sscrape_configs:- job_name: pod-logskubernetes_sd_configs:- role: podpipeline_stages:- docker: {}relabel_configs:- source_labels:- __meta_kubernetes_pod_node_nametarget_label: __host__- action: labelmapregex: __meta_kubernetes_pod_label_(.+)- action: replacereplacement: $1separator: /source_labels:- __meta_kubernetes_namespace- __meta_kubernetes_pod_nametarget_label: job- action: replacesource_labels:- __meta_kubernetes_namespacetarget_label: namespace- action: replacesource_labels:- __meta_kubernetes_pod_nametarget_label: pod- action: replacesource_labels:- __meta_kubernetes_pod_container_nametarget_label: container- replacement: /var/log/pods/*$1/*.logseparator: /source_labels:- __meta_kubernetes_pod_uid- __meta_kubernetes_pod_container_nametarget_label: __path__--- # Clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: promtail-clusterrolelabels:app: promtailnamespace: logging
rules:- apiGroups: [""]resources:- nodes- services- podsverbs:- get- watch- list--- # ServiceAccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:name: promtail-serviceaccountlabels:app: promtailnamespace: logging--- # Rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: promtail-clusterrolebindinglabels:app: promtailnamespace: logging
subjects:- kind: ServiceAccountname: promtail-serviceaccountnamespace: default
roleRef:kind: ClusterRolename: promtail-clusterroleapiGroup: rbac.authorization.k8s.io[root@master01 ~]# kubectl apply -f promtail.yaml

4、查看结果

[root@master01 ~]# kubectl get pod -n logging
NAME                       READY   STATUS    RESTARTS   AGE
promtail-daemonset-2r8sv   1/1     Running   0          56s
promtail-daemonset-f4p8q   1/1     Running   0          56s
promtail-daemonset-tddsf   1/1     Running   0          56s
promtail-daemonset-wdn47   1/1     Running   0          56s

5、dashboard使用

本文链接：https://my.lmcjl.com/post/1477.html