Total Pageviews

Sunday, 4 June 2023

小米开源的监控系统open-falcon



小米运维团队开源了自家的监控系统:open-falcon。整个系统的后端,全部用golang编写;前端dashboard使用python编写。小米运维称随着业务的快速发展,以及互联网公司特有的一些需求,现有的开源的监控系统如zabbix,在性能、扩展性、和用户的使用效率方面,已无法支撑。

open-falcon特色:

    强大灵活的数据采集 :自动发现,支持falcon-agent、snmp、支持用户主动push、用户自定义插件支持、opentsdb data model like(timestamp、endpoint、metric、key-value tags)
    水平扩展能力 :支持每个周期上亿次的数据采集、告警判定、历史数据存储和查询
    高效率的告警策略管理 :高效的portal、支持策略模板、模板继承和覆盖、多种告警方式、支持callback调用
    人性化的告警设置 :最大告警次数、告警级别、告警恢复通知、告警暂停、不同时段不同阈值、支持维护周期
    高效率的graph组件 :单机支撑200万metric的上报、归档、存储(周期为1分钟)
    高效的历史数据query组件 :采用rrdtool的数据归档策略,秒级返回上百个metric一年的历史数据
    dashboard :多维度的数据展示,用户自定义Screen
    高可用 :整个系统无核心单点,易运维,易部署,可水平扩展

下载:http://github.com/open-falcon
--------------------------------------------------------------------

An open-source and enterprise-level monitoring system. 

Notice

If you are heavily using k8s and adopting microservices architecture, we recommend you to upgrade to use Nightingale building your modern monitoring system.

Nightingale is an enterprise-level cloud-native monitoring tool, which can be used as drop-in replacement of Prometheus for alerting and management.

Documentations

Prerequisite

  • Git >= 1.7.5
  • Go >= 1.6

Getting Started

Build from source

before start, please make sure you prepared this:

yum install -y redis
yum install -y mysql-server

NOTE: be sure to check redis and mysql-server have successfully started.

And then

# Please make sure that you have set `$GOPATH` and `$GOROOT` correctly.
# If you have not golang in your host, please follow [https://golang.org/doc/install] to install golang.

mkdir -p $GOPATH/src/github.com/open-falcon
cd $GOPATH/src/github.com/open-falcon
git clone https://github.com/open-falcon/falcon-plus.git

And do not forget to init the database first (if you have not loaded the database schema before)

cd $GOPATH/src/github.com/open-falcon/falcon-plus/scripts/mysql/db_schema/
mysql -h 127.0.0.1 -u root -p < 1_uic-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 2_portal-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 3_dashboard-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 4_graph-db-schema.sql
mysql -h 127.0.0.1 -u root -p < 5_alarms-db-schema.sql

NOTE: if you are upgrading from v0.1 to v0.2.0(or above),then. More upgrading instruction

mysql -h 127.0.0.1 -u root -p < 5_alarms-db-schema.sql

Compilation

cd $GOPATH/src/github.com/open-falcon/falcon-plus/

# make all modules
make all

# make specified module
make agent

# pack all modules
make pack
  • after make pack you will got open-falcon-vx.x.x.tar.gz
  • if you want to edit configure file for each module, you can edit config/xxx.json before you do make pack

Unpack and Decompose

export WorkDir="$HOME/open-falcon"
mkdir -p $WorkDir
tar -xzvf open-falcon-vx.x.x.tar.gz -C $WorkDir
cd $WorkDir

Start all modules in single host

cd $WorkDir
./open-falcon start

# check modules status
./open-falcon check

Run More Open-Falcon Commands

for example:

# ./open-falcon [start|stop|restart|check|monitor|reload] module
./open-falcon start agent

./open-falcon check
        falcon-graph         UP           53007
          falcon-hbs         UP           53014
        falcon-judge         UP           53020
     falcon-transfer         UP           53026
       falcon-nodata         UP           53032
   falcon-aggregator         UP           53038
        falcon-agent         UP           53044
      falcon-gateway         UP           53050
          falcon-api         UP           53056
        falcon-alarm         UP           53063
  • For debugging , You can check $WorkDir/$moduleName/logs/xxx.log

Install Frontend Dashboard

  • Follow this:https://github.com/open-falcon/dashboard

NOTE: if you want to use grafana as the dashboard, please check this.

Package Release

make clean all pack

API Standard

Q&A

from  https://github.com/open-falcon/falcon-plus

No comments:

Post a Comment