$ sudo td-agent-gem install fluent-plugin-prometheus --version='~>1.6.1'
# source
<source>
@type forward
bind 0.0.0.0
port 24224
</source>
# count number of incoming records per tag
<filter company.*>
@type prometheus
<metric>
name fluentd_input_status_num_records_total
type counter
desc The total number of incoming records
<labels>
tag ${tag}
hostname ${hostname}
</labels>
</metric>
</filter>
# count number of outgoing records per tag
<match company.*>
@type copy
<store>
@type forward
<server>
name myserver1
hostname 192.168.1.3
port 24224
weight 60
</server>
</store>
<store>
@type prometheus
<metric>
name fluentd_output_status_num_records_total
type counter
desc The total number of outgoing records
<labels>
tag ${tag}
hostname ${hostname}
</labels>
</metric>
</store>
</match>
# expose metrics in prometheus format
<source>
@type prometheus
bind 0.0.0.0
port 24231
metrics_path /metrics
</source>
<source>
@type prometheus_output_monitor
interval 10
<labels>
hostname ${hostname}
</labels>
</source>
配置检查
完成以上3步配置之后,重启fluentd。
# For stand-alone Fluentd installations
$ fluentd -c fluentd.conf
# For td-agent users
$ sudo systemctl restart td-agent
向fluentd发送测试数据
$ echo '{"message":"hello"}' | bundle exec fluent-cat company.test1
$ echo '{"message":"hello"}' | bundle exec fluent-cat company.test1
$ echo '{"message":"hello"}' | bundle exec fluent-cat company.test1
$ echo '{"message":"hello"}' | bundle exec fluent-cat company.test2
访问http://localhost:24231/metrics,可以看到Prometheus格式的Fluentd相关统计指标。
curl http://localhost:24231/metrics
# TYPE fluentd_input_status_num_records_total counter
# HELP fluentd_input_status_num_records_total The total number of incoming records
fluentd_input_status_num_records_total{tag="company.test",host="KZK.local"} 3.0
fluentd_input_status_num_records_total{tag="company.test2",host="KZK.local"} 1.0
# TYPE fluentd_output_status_num_records_total counter
# HELP fluentd_output_status_num_records_total The total number of outgoing records
fluentd_output_status_num_records_total{tag="company.test",host="KZK.local"} 3.0
fluentd_output_status_num_records_total{tag="company.test2",host="KZK.local"} 1.0
# TYPE fluentd_output_status_buffer_queue_length gauge
# HELP fluentd_output_status_buffer_queue_length Current buffer queue length.
fluentd_output_status_buffer_queue_length{hostname="KZK.local",plugin_id="object:3fcbccc6d388",type="forward"} 1.0
....
global:
scrape_interval: 10s # Set the scrape interval to every 10 seconds. Default is every 1 minute.
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
- job_name: 'fluentd'
static_configs:
- targets: ['localhost:24231']
$ ./prometheus --config.file="prometheus.yml"
打开浏览器,访问http://localhost:9090/
。
可以看到如下指标:
fluentd_input_status_num_records_total
fluentd_output_status_buffer_queue_length
fluentd_output_status_buffer_total_bytes
fluentd_output_status_emit_count
fluentd_output_status_num_errors
fluentd_output_status_num_records_total
fluentd_output_status_retry_count
fluentd_output_status_retry_wait
这里,我们只对Prometheus的用法做简单介绍,感兴趣的话可以去官网做进一步了解。
联系客服