包详细信息

spm-agent

sematext1.9kApache-2.02.2.5

Node.js agent framework for SPM by Sematext

SPM, APM, application performance monitoring, Sematext

自述文件

Sematext Agent Framework for Node.js

A framework for monitoring agents written in Node.js for sending metrics to Sematext Cloud or InfluxDB. The Sematext Node.js monitoring agent is built on top of this framework.

Functionality

  • Sender interface to Sematext backend receivers
  • Buffering metrics in case of network outages
  • Reconnect after failures
  • Logging functions
  • Configuration handling
  • Pluggable agents

Example to Implement a Monitoring Agent

Sematext Cloud supports the Influx Line Protocol for metric ingestion.

Agent modules must provide a start and stopfunction.

The function agent.addMetrics(metric) collects a metric, which is shipped in bulk requests via Influx Line Protocol to the metrics receiver.

All metric objects, must have the mandatory fields measurement, tags and fields:

  • measurement - the name of the measurement e.g. 'process.memry'
  • tags - key/value pairs, useful for filtering or aggregation of data
  • fields - numeric fields with the measurement values

const SpmAgent = require('spm-agent')
const client = new SpmAgent()
const os = require('os')

// configure client, with non-default values
// or use the file ./.spmagentrc in YAML format
// the default configuration contains values for Sematext Cloud US
// SpmAgent.Config.tokens.spm = process.env.MONITORING_TOKEN
SpmAgent.Config.influx = {
  dbName: 'metrics',
  // change receiver to Sematext Cloud EU
  // default is spm-receiver.sematext.com for US region
  host: 'spm-receiver.eu.sematext.com',
  port: 443,
  protocol: 'https'
}

// a monitoring agent needs a start an stop function
class MemoryMonitor {
  start (agent) {
    this.agent = agent
    // initialize your metrics collector ...
    // Typically agents collect metrics periodically, every N seconds. The time between // two collection activities is the collectionInterval, specified in milliseconds.
    this.tid = setInterval(function () {
      const measurement = {
        // measurment namespace
        measurement: 'process.memory',
        timestap: new Date(),
        tags: {
          role: 'frontend',
          'os.host': os.hostname()
          // The monitoring token can be set as 'tags.token' value
          // Routing a metrics to a different monitoring app
          // requires setting the `token` tag
          // ,token = process.env.OTHER_APP_TOKEN
        },
        // metrics names and values
        fields: {
          rss: process.memoryUsage().rss
        }
      }
      // pass metrics to sender schedule
      agent.addMetrics(measurement)
    }, SpmAgent.Config.collectionInterval)
  }

  stop () {
    if (this.tid) {
      clearInterval(this.tid)
    }
  }
}

client.createAgent(new SpmAgent.Agent(new MemoryMonitor()))

// log collected metrics to console
// observe all metrics for testing/debuggin
client.on('metrics', console.log)
// observe a specific metric using the measurment name
client.on('process.memory', console.log)

Metric Tags

The agent is able to use environment variables as metrics tags. Define a list of environment variables to be used as metric tags.

Let's assume this is the enviroment of the monitored application:

> env
TERM_PROGRAM=Apple_Terminal
TERM=xterm-256color
SHELL=/bin/bash
TMPDIR=/var/folders/v6/bh2q1xsn27g4d2g54z2ylv100000gn/T/
TERM_PROGRAM_VERSION=404.1
TERM_SESSION_ID=F12E0DBD-BF40-466D-85EC-08E89EDC3440
USER=stefan
PWD=/Users/stefan
HOME=/Users/stefan
LOGNAME=stefan
LC_CTYPE=UTF-8
DISPLAY=/private/tmp/com.apple.launchd.J60ybGpban/org.macosforge.xquartz:0
_=/usr/bin/env

You want to see later the performance metrics aggregated or filtered for specific user. In this case the agent should collect the user name as tag. You can find the user name in the process environment variable USER. Let's also assume the workingdirectory of the application PWD is another relevant identifier for your monitored application.

To configure the agent collecting USER and PWD as tags you can specify a commaseparated list of environment variables for the the agent configuration:

# add the values of env. variables USER, PWD as tags to metrics
export MONITORING_TAGS_FROM_ENV="USER, PWD"

The generated metrics include then the environment variable name and its value as tags. The result in Influx Line Protocol (ILP) format, produced by the agent for metric called swap:

swap, USER=stefan,PWD=/Users/stefan out=0i,in=0i 1576765680000000000

Define static tags in key:value format

export MONITORING_TAGS_FROM_ENV="organisation:sematext, service:api"

The result in in ILP format:

swap, organisation=sematext,service=api out=0i,in=0i 1576765680000000000

Both types of tags can be mixed:

export MONITORING_TAGS_FROM_ENV="organisation:sematext, service:api, USER, PWD"

The result in in ILP format:

swap, organisation=sematext,service=api,USER=stefan,PWD=/Users/stefan,os.host=imac.local out=0i,in=0i 1576765680000000000

The config file entry influx.tagsFromEnv in .spmagenrc works as well:

tokens: 
  spm: 'YOUR_MONITORING_TOKEN'
  infra: 'YOUR_INFRA_MONITROING_TOKEN'
influx:
  tagsFromEnv: 'organisation:sematext, USER, PWD' 
  dbName: 'metrics'
  host: spm-receiver.sematext.com
  protocol: https
  port: 443

Contribute

Let us know about monitoring agents you need, maybe you like to contribute with your domain expertise!

Related Modules

更新日志

Changelog

All notable changes to this project will be documented in this file. Dates are displayed in UTC.

Generated by auto-changelog.

2.2.5

7 August 2023

  • fix: upgrade release-it library #38

2.2.4

15 June 2023

  • fix: influxdb sender issue with emitter #36
  • Release 2.2.4 5dc40c3

2.2.3

15 June 2023

  • Remove is docker library #35
  • Release 2.2.3 8924c34

2.2.2

14 June 2023

  • chore: housekeeping #34
  • refactor: upgrade and replace libraries with vulnerabilities #33
  • Bump qs from 6.5.2 to 6.5.3 #30
  • Bump minimatch from 3.0.4 to 3.1.2 #29
  • Bump vm2 from 3.9.10 to 3.9.11 #28
  • refactor: remove libraries with vulnerabilities 947261d
  • refactor: replace 'request' library with 'node-fetch' 650dda3
  • Release 2.2.2 d95f2ef

2.2.1

20 September 2022

  • Update dependencies #27
  • Bump jose from 1.28.1 to 1.28.2 #26
  • Update dependencies #25
  • Release 2.2.1 d2c7e24

2.2.0

7 July 2022

  • Bump moment from 2.29.2 to 2.29.4 #24
  • SC-13033 Update dependencies #23
  • Bump moment from 2.29.1 to 2.29.2 #22
  • Bump node-fetch from 2.6.1 to 2.6.7 #21
  • Bump ansi-regex from 3.0.0 to 3.0.1 #20
  • Bump minimist from 1.2.5 to 1.2.6 #19
  • Upgrade critical dependencies c75a57f
  • Upgrade all dependencies and drop support for Node.js versions <12 for dev environments efa7a55
  • Revert downgrade to kubernetes-client package 19a0e0b

2.1.4

1 July 2021

  • Bump ws from 6.2.1 to 6.2.2 #17
  • Bump glob-parent from 5.1.1 to 5.1.2 #18
  • Bump hosted-git-info from 2.8.8 to 2.8.9 #16
  • Bump handlebars from 4.7.6 to 4.7.7 #15
  • Bump jose from 1.28.0 to 1.28.1 #14
  • change nedb package 9715137
  • updated deps 71efca9
  • Release 2.1.4 ce7688d

2.1.3

7 April 2021

2.1.2

15 February 2021

2.1.1

13 January 2021

2.1.0

22 December 2020

2.0.9

27 February 2020

2.0.8

22 January 2020

  • Backward compatible usage of tokens.monitoring and tokens.spm #9
  • ensure tokens.spm = tokens.monitoring when set via config file d86e714
  • Release 2.0.8 df85a89

2.0.7

22 January 2020

  • Set missing pid, ppid tags only for nodejs & process metrics #7
  • set missing pid,ppid tags only for nodejs & process metrics 0cdb250
  • add process prefix to pid, ppid tags fa16427
  • Release 2.0.7 2df5f9e

2.0.6

16 January 2020

  • add infra-token to config #6
  • don't overwrite token tag, add unit test 7a5d4f7
  • mention individual token setting in readme 77434a0
  • Release 2.0.6 d7c2d6d

2.0.4

22 December 2019

  • SC-6450 add custom tags #5
  • add tags from ENV and static tags 2acf5c0
  • resolve comments from review ec285ec
  • update README, Monitoring Tags section a576662

2.0.3

10 December 2019

2.0.2

10 December 2019

  • handle http status 400 correctly, limit bulk size aad9aa9
  • add test for http status 400 handling 96da9d2
  • ensure metric objects have a timestamp c16a38a

2.0.1

10 December 2019

2.0.0

5 December 2019

  • add support for influx line protocol #4
  • restrcuture, remove old metrics format from readme, enhanced example ce99cd0
  • remove st-events in spmsender / avoid duplicated events via influx-sender dde8f10
  • make url parsing compatible with Node 8 01513bd

1.31.20

14 August 2019

  • Wording and link tweaks 7b98be0
  • adjust maxMetricsAge to 12 hrs 4b3cac0
  • limit bulk req size to config.maxDataPoints || 100 f0109bd

1.31.18

23 April 2019

1.31.17

17 January 2019

  • fix unique key for NeDB + update dependencies 9886aa7
  • Release 1.31.17 70d876a

1.31.16

13 August 2018

  • adjust error handling for spm-receiver status codes f38a0b7
  • Release 1.31.16 d549558

1.31.15

17 May 2018

  • adjust logging dir for windows/metor apps 266881b
  • Release 1.31.15 02e20a7

1.31.14

19 January 2018

  • add exit event logging to node agents 8d840e4
  • Release 1.31.14 5f3c594

1.31.13

11 January 2018

  • Keep metrics buffer up to 12 hours, when DB reaches more than 100 MB 52ebb39
  • Update travis.yml, remove node 0.12 added v8 & v9 db6b485
  • Release 1.31.13 a2e7230

1.31.12

27 November 2017

  • fix unhadled excpetion when log directory is not writeable c28a2dc
  • Release 1.31.12 929b63d

1.31.11

16 May 2017

  • load env vars from /etc/sematext/receivers.config for onprem/multi-region 7871cab
  • add config.set(flatKey) method f769b4b
  • Release 1.31.11 b49e40c

1.31.10

19 April 2017

1.31.9

26 March 2017

1.31.8

17 February 2017

  • remove c++ tools from travis 42e9c0d
  • move to rc-yaml-2 until PR for rc-yaml is accepted 9b907cb
  • Release 1.31.8 4edca76

1.31.7

9 February 2017

  • changed SPM event field content and version info cfe2ee1
  • Release 1.31.7 445bca7

1.31.6

9 February 2017

1.31.5

2 February 2017

1.31.4

27 January 2017

  • Release 1.31.4 c4110a6
  • fix url parameters, when dockerHostName is not available 4dca309

1.31.3

25 January 2017

  • add dynamic url parameters, via global setting a3a40d3
  • Release 1.31.3 8664d62
  • add dynamic url parameters, via global setting 12fddeb

1.31.2

24 January 2017

  • read spm client setting for docker hostname 2b89348
  • add. parameters for docker setups 0d63e3d
  • add support for YAML config files, updated dependencies ec6c242

1.31.1

29 November 2016

  • change event type, use calling module version for SPM agent version in URL 15e42ff
  • Release 1.31.1 35d0c13

1.30.5

5 October 2016

  • don't send data to SPM when SPM_TOKEN is not set, change exit handler to beforeExit event 931796e
  • suppress error message in SDA with Logsene storage 8c3b7eb
  • Release 1.30.5 82c0370

1.30.4

4 August 2016