其他分享
首页 > 其他分享> > Skywalking

Skywalking

作者:互联网

目录

OpenTracing规范

OpenTracing是一种分布式系统链路跟踪的设计原则、规范、标准。类似JDBC的规范,主要为了提供一套标准的JDBC API。OpenTracing也是一样,是为了统一提供一套链路追踪的标准API,所制定的一种规范。OpenTracing通过提供平台无关、厂商无关的API,使得开发人员能够方便的添加(或更换)追踪系统的实现。

为什么需要OpenTracing

OpenTracing通过提供平台无关、厂商无关的API,使得开发人员能够方便的添加(或更换)追踪系统的实现。 OpenTracing提供了用于运营支撑系统的和针对特定平台的辅助程序库。

什么是一个Trace

在广义上,一个trace代表了一个事务或者流程在(分布式)系统中的执行过程。在OpenTracing标准中,trace是多个span组成的一个有向无环图(DAG),每一个span代表trace中被命名并计时的连续性的执行片段。

什么是一个Trace

分布式追踪中的每个组件都包含自己的一个或者多个span。例如,在一个常规的RPC调用过程中,OpenTracing推荐在RPC的客户端和服务端,至少各有一个span,用于记录RPC调用的客户端和服务端信息。

什么是一个Trace2

一个父级的span会显示的并行或者串行启动多个子span。在OpenTracing标准中,甚至允许一个子span有个多父span(例如:并行写入的缓存,可能通过一次刷新操作写入动作)。

一个典型的Trace案例

一个典型的Trace案例

在一个分布式系统中,追踪一个事务或者调用流一般如上图所示。虽然这种图对于看清各组件的组合关系是很有用的,但是,它不能很好显示组件的调用时间,是串行调用还是并行调用,如果展现更复杂的调用关系,会更加复杂,甚至无法画出这样的图。另外,这种图也无法显示调用间的时间间隔以及是否通过定时调用来启动调用。一种更有效的展现一个典型的trace过程,如下图所示:

一个典型的Trace案例2

这种展现方式增加显示了执行时间的上下文,相关服务间的层次关系,进程或者任务的串行或并行调用关系。这样的视图有助于发现系统调用的关键路径。通过关注关键路径的执行过程,项目团队可能专注于优化路径中的关键位置,最大幅度的提升系统性能。例如:可以通过追踪一个资源定位的调用情况,明确底层的调用情况,发现哪些操作有阻塞的情况。

Skywalking

Skywalking是一款APM(Application Performance Management & Monitoring)系统。Skywalking是分布式系统应用程序性能监视工具,专为微服务、云原生架构和基于容器(Docker、K8s、Mesos)架构而设计。提供分布式追踪、服务网格遥测分析、度量聚合和可视化一体化解决方案。

功能介绍

整体架构

Skywalking

整个架构,分成上、下、左、右四部分:

Tracing、Logging和Metrics

在微服务领域,很早以来就形成了Tracing、Logging和Metrics相辅相成,合力支撑多维度、多形态的监控体系,三类监控各有侧重:

Tracing:它在单次请求的范围内,处理信息。 任何的数据、元数据信息都被绑定到系统中的单个事务上。例如:一次调用远程服务的RPC执行过程;一次实际的SQL查询语句;一次HTTP请求的业务性ID;

Logging:日志,不知道大家有没有想过它的定义或者边界。Logging即是记录处理的离散事件,比如我们应用的调试信息或者错误信息等发送到ES;审计跟踪时间信息通过Kafka处理送到BigTable等数据仓储等等,大多数情况下记录的数据很分散,并且相互独立,也许是错误信息,也许仅仅只是记录当前的事件状态,或者是警告信息等等。

Metrics:当我们想知道我们服务的请求QPS是多少,或者当天的用户登录次数等等,这时我们可能需要将一部分事件进行聚合或计数,也就是我们说的Metrics。可聚合性即是Metrics的特征,它们是一段时间内某个度量(计数器或者直方图)的原子或者是元数据。例如接收的HTTP数量可以被建模为计数器,每次的HTTP请求即是我们的度量元数据,可以进行简单的加法聚合,当持续了一段时间我们又可以建模为直方图。

关系图

.NET6 对接 Skywalking

部署Skywalking环境

version: '3.3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.5.0
    container_name: elasticsearch
    restart: always
    ports:
      - 9200:9200
    environment:
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms256m -Xmx256m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
  oap:
    image: apache/skywalking-oap-server:8.8.0
    container_name: oap
    depends_on:
      - elasticsearch
    links:
      - elasticsearch
    restart: always
    ports:
      - 11800:11800
      - 12800:12800
    environment:
      SW_STORAGE: elasticsearch
      SW_STORAGE_ES_CLUSTER_NODES: elasticsearch:9200
  ui:
    image: apache/skywalking-ui
    container_name: ui
    depends_on:
      - oap
    links:
      - oap
    restart: always
    ports:
      - 8080:8080
    environment:
      SW_OAP_ADDRESS: http://oap:12800

对接.NET6 程序

添加依赖

<ItemGroup>
    <PackageReference Include="SkyAPM.Agent.AspNetCore" Version="1.3.0" />
</ItemGroup>

编辑Skywalking配置文件skyapm.json

手动编写skyapm.json

{
  "SkyWalking": {
    "ServiceName": "MySkyWalkingDemoTest",
    "Namespace": "",
    "HeaderVersions": [
      "sw8"
    ],
    "Sampling": {
      "SamplePer3Secs": -1,
      "Percentage": -1.0
    },
    "Logging": {
      "Level": "Information",
      "FilePath": "logs\\skyapm-{Date}.log"
    },
    "Transport": {
      "Interval": 3000,
      "ProtocolVersion": "v8",
      "QueueSize": 30000,
      "BatchSize": 3000,
      "gRPC": {
        "Servers": "192.168.3.245:11800",
        "Timeout": 10000,
        "ConnectTimeout": 10000,
        "ReportTimeout": 600000,
        "Authentication": ""
      }
    }
  }
}

自动生成Skyapm.json

安装CLI(SkyAPM.DotNet.CLI)
 dotnet tool install -g SkyAPM.DotNet.CLI
自动生成skyapm.json文件

server name指的就是您刚才配置的SKYWALKING__SERVICENAME,server指的是您Skywalking的ip地址。执行命令后,会自动生成一个skywalking.json 。

 dotnet skyapm config [service name] [server]:11800
 eg: dotnet skyapm config MySkyWalking_OrderService 192.168.3.245:11800

SkyAPM Config 配置说明

ServiceName
服务名称

Sampling
采样配置节点

  • SamplePer3Secs 每3秒采样数
  • Percentage 采样百分比,例如10%采样则配置为10

Logging
日志配置节点

  • Level 日志级别
  • FilePath 日志保存路径

Transport

传输配置节点

  • Interval 每多少毫秒刷新
  • gRPC
    gRPC配置节点
  • Servers gRPC地址,多个用逗号“,”
  • Timeout 创建gRPC链接的超时时间,毫秒
  • ConnectTimeout gRPC最长链接时间,毫秒

launchSettings.json文件配置SK

"profiles": { // 项目
    "IIS Express": { // IIS部署项
      "commandName": "IISExpress",
      "launchBrowser": true,
      "launchUrl": "weatherforecast",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development",
        "ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "SkyAPM.Agent.AspNetCore",
        "SKYWALKING__SERVICENAME": "MySkyWalkingDemoTest"
      }
    },
    "SkyWalkingDemo": { // castrol部署项
      "commandName": "Project",
      "launchBrowser": true,
      "launchUrl": "weatherforecast",
      "applicationUrl": "http://localhost:5000",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development",
        "ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "SkyAPM.Agent.AspNetCore", // 必须配置
        "SKYWALKING__SERVICENAME": "MySkyWalkingDemoTest" // 必须配置,在skywalking做标识
      }
    }
  }

startup.cs文件中添加

public void ConfigureServices(IServiceCollection services)
{
    services.AddSkyApmExtensions(); // 添加Skywalking相关配置
    services.AddControllers();
    services.AddHttpClient();
}

获取traceId

private readonly IEntrySegmentContextAccessor segContext;

public SkywalkingController(IEntrySegmentContextAccessor segContext)
{
    this.segContext = segContext;
}

/// <summary>
/// 获取链接追踪ID
/// </summary>
/// <returns></returns>
[HttpGet("traceId")]
public string GetSkywalkingTraceId()
{
    return segContext.Context.TraceId;
}

自定义调用链路的信息

[HttpGet]
public async Task<IActionResult> SkywalkingTest()
{
    //获取全局的skywalking的TracId
    var TraceId = _segContext.Context.TraceId;
    Console.WriteLine($"TraceId={TraceId}");
    _segContext.Context.Span.AddLog(LogEvent.Message($"SkywalkingTest---Worker running at: {DateTime.Now}"));

    System.Threading.Thread.Sleep(1000);

    _segContext.Context.Span.AddLog(LogEvent.Message($"SkywalkingTest---Worker running at--end: {DateTime.Now}"));

    return Ok($"Ok,SkywalkingTest-TraceId={TraceId} ");
}

接入微服务网关+后台微服务

网关接入

添加依赖

<ItemGroup>
    <PackageReference Include="SkyAPM.Agent.AspNetCore" Version="1.3.0" />
</ItemGroup>

拷贝配置文件并简单修改

{
  "SkyWalking": {
    "ServiceName": "MySkyWalking_Gateway", #修改名称就OK
    "Namespace": "",
    "HeaderVersions": [
      "sw8"
    ],
    "Sampling": {
      "SamplePer3Secs": -1,
      "Percentage": -1.0
    },
    "Logging": {
      "Level": "Debug",
      "FilePath": "logs\\skyapm-{Date}.log"
    },
    "Transport": {
      "Interval": 3000,
      "ProtocolVersion": "v8",
      "QueueSize": 30000,
      "BatchSize": 3000,
      "gRPC": {
        "Servers": "192.168.3.245:11800",
        "Timeout": 10000,
        "ConnectTimeout": 10000,
        "ReportTimeout": 600000,
        "Authentication": ""
      }
    }
  }
}

launchsettings.json添加环境变量

"profiles": {
    "Zhaoxi.MicroService.GatewayCenter": {
      "commandName": "Project",
      "dotnetRunMessages": true,
      "launchBrowser": true,
      "launchUrl": "swagger",
      "applicationUrl": "https://localhost:7141;http://localhost:5141",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development",
        "ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "SkyAPM.Agent.AspNetCore", #添加HOST变量
        "SKYWALKING__SERVICENAME": "MySkyWalking_Gateway" #添加服务名称
      }
    },
    "IIS Express": {
      "commandName": "IISExpress",
      "launchBrowser": true,
      "launchUrl": "swagger",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development",
        "ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "SkyAPM.Agent.AspNetCore",
        "SKYWALKING__SERVICENAME": "MySkyWalking_Gateway"
      }
    }
  }

修改网关配置文件,添加OrderServiceInstance微服务的路由

{
    "DownstreamPathTemplate": "/api/{url}", //服务地址--url变量
    "DownstreamScheme": "http",
    "UpstreamPathTemplate": "/microservice/{url}", //网关地址--url变量
    "UpstreamHttpMethod": [ "Get", "Post" ],
    "UseServiceDiscovery": true,
    "ServiceName": "OrderService", //consul服务名称
    "LoadBalancerOptions": {
        "Type": "RoundRobin" //轮询
}

启动网关

dotnet run --urls=http://*:6299

订单微服务接入

添加依赖

<ItemGroup>
    <PackageReference Include="SkyAPM.Agent.AspNetCore" Version="1.3.0" />
</ItemGroup>

拷贝配置文件并简单修改

{
  "SkyWalking": {
    "ServiceName": "MySkyWalking_OrderService",
    "Namespace": "",
    "HeaderVersions": [
      "sw8"
    ],
    "Sampling": {
      "SamplePer3Secs": -1,
      "Percentage": -1.0
    },
    "Logging": {
      "Level": "Debug",
      "FilePath": "logs\\skyapm-{Date}.log"
    },
    "Transport": {
      "Interval": 3000,
      "ProtocolVersion": "v8",
      "QueueSize": 30000,
      "BatchSize": 3000,
      "gRPC": {
        "Servers": "192.168.3.245:11800",
        "Timeout": 10000,
        "ConnectTimeout": 10000,
        "ReportTimeout": 600000,
        "Authentication": ""
      }
    }
  }
}

launchsettings.json添加环境变量

"profiles": {
    "Zhaoxi.MicroService.OrderServiceInstance": {
      "commandName": "Project",
      "dotnetRunMessages": true,
      "launchBrowser": true,
      "launchUrl": "swagger",
      "applicationUrl": "http://192.168.3.105:7900",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development",
        "ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "SkyAPM.Agent.AspNetCore",
        "SKYWALKING__SERVICENAME": "MySkyWalking_OrderService"
      }
    },
    "IIS Express": {
      "commandName": "IISExpress",
      "launchBrowser": true,
      "launchUrl": "swagger",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      }
    }
  }

启动订单微服务

dotnet run

用户微服务接入

步骤和订单微服务一样

配置Skywalking告警

配置告警规则

docker exec -it 12f053748e85 /bin/sh
ls -l

Skywalking告警

查阅配置规则文件及配置规则解读

通过cat alarm-settings.yml可以查阅文件内容,如下:

docker cp 12f053748e85:/skywalking/config/alarm-settings.yml .
# Sample alarm rules.
rules:
  # Rule unique name, must be ended with `_rule`.
  service_resp_time_rule:
    metrics-name: service_resp_time
    op: ">"
    threshold: 1000
    period: 10
    count: 3
    silence-period: 5
    message: Response time of service {name} is more than 1000ms in 3 minutes of last 10 minutes.
  service_sla_rule:
    # Metrics value need to be long, double or int
    metrics-name: service_sla
    op: "<"
    threshold: 8000
    # The length of time to evaluate the metrics
    period: 10
    # How many times after the metrics match the condition, will trigger alarm
    count: 2
    # How many times of checks, the alarm keeps silence after alarm triggered, default as same as period.
    silence-period: 3
    message: Successful rate of service {name} is lower than 80% in 2 minutes of last 10 minutes
  service_resp_time_percentile_rule:
    # Metrics value need to be long, double or int
    metrics-name: service_percentile
    op: ">"
    threshold: 1000,1000,1000,1000,1000
    period: 10
    count: 3
    silence-period: 5
    message: Percentile response time of service {name} alarm in 3 minutes of last 10 minutes, due to more than one condition of p50 > 1000, p75 > 1000, p90 > 1000, p95 > 1000, p99 > 1000
  service_instance_resp_time_rule:
    metrics-name: service_instance_resp_time
    op: ">"
    threshold: 1000
    period: 10
    count: 2
    silence-period: 5
    message: Response time of service instance {name} is more than 1000ms in 2 minutes of last 10 minutes
  database_access_resp_time_rule:
    metrics-name: database_access_resp_time
    threshold: 1000
    op: ">"
    period: 10
    count: 2
    message: Response time of database access {name} is more than 1000ms in 2 minutes of last 10 minutes
  endpoint_relation_resp_time_rule:
    metrics-name: endpoint_relation_resp_time
    threshold: 1000
    op: ">"
    period: 10
    count: 2
    message: Response time of endpoint relation {name} is more than 1000ms in 2 minutes of last 10 minutes
#  Active endpoint related metrics alarm will cost more memory than service and service instance metrics alarm.
#  Because the number of endpoint is much more than service and instance.
#
#  endpoint_avg_rule:
#    metrics-name: endpoint_avg
#    op: ">"
#    threshold: 1000
#    period: 10
#    count: 2
#    silence-period: 5
#    message: Response time of endpoint {name} is more than 1000ms in 2 minutes of last 10 minutes

webhooks:
#  - http://127.0.0.1/notify/
#  - http://127.0.0.1/go-wechat/

规则常用指标解读
rule name: 规则名称,必须唯一,必须以 _rule结尾;
metrics name: oal(Observability Analysis Language)脚本中的度量名;名称在SkyWalking后端服务中已经定义,进入容器skywalking-oap之后,进入如下目录就可以找到。

include names: 本规则告警生效的实体名称,如服务名,终端名;
exclude-names:将此规则作用于不匹配的实体名称上,如服务名,终端名;
threshold: 阈值,可以是一个数组,即可以配置多个值;
op: 操作符, 可以设定 >, <, =;
period: 多久检查一次当前的指标数据是否符合告警规则;以分钟为单位
count: 超过阈值条件,达到count次数,触发告警;
silence period:在同一个周期,指定的silence period时间内,忽略相同的告警消息;
更多告警规则详情,请参照这个地址:https://github.com/apache/skywalking/blob/master/docs/en/setup/backend/backend-alarm.md

修改告警规则

rules:
	service_test_sal_rule:
		# 指定指标名称
    	metrics-name: service_test_sal
    	# 小于
    	op: "<"
    	# 指定阈值
    	threshold: 8000
    	# 每2分钟检测告警该规则
    	period: 2
    	# 触发1次规则就告警
    	count: 1
    	# 设置三分钟内容相同告警,不重复告警
    	silence-period: 3
    	# 配置告警信息
    	message: Successful rate of service {name} is lower than 80% in 2 minutes of last 10 minutes

概要:服务成功率在过去2分钟内低于80%

告警API编写

本质还是SkyWalking根据规则进行检查,如果符合规则条件,就通过WebHook、gRPCHook、WeChat Hook、Dingtalk Hook等方式进行消息通知;接收到告警数据信息之后,可以自行处理消息。这里为了方便,就采用WebHook的方式进行演示,即触发告警条件之后,SkyWalking会调用配置的WebHook 接口,并传递对应的告警信息;

定义数据模型

public class AlarmMsg
{
    public int scopeId { get; set; }
    public string? scope { get; set; }
    public string? name { get; set; }
    public string? id0 { get; set; }
    public string? id1 { get; set; }
    public string? ruleName { get; set; }
    public string? alarmMessage { get; set; }
}

定义WebHook调用API

/// <summary>
/// 告警API
/// </summary>
/// <param name="msgs"></param>
/// <returns></returns>
[HttpPost("AlarmMsg")]
public void AlarmMsg(List<AlarmMsg> msgs)
{
    string msg = "触发告警:";
    msg += msgs.FirstOrDefault()?.alarmMessage;
    Console.WriteLine(msg);
    SendMail(msg);
}

配置webHook

http://192.168.3.105:7900/api/Skywalking/AlarmMsg
# Sample alarm rules.
rules:
  # Rule unique name, must be ended with `_rule`.
  service_resp_time_rule:
    metrics-name: service_resp_time
    op: ">"
    threshold: 1000
    period: 10
    count: 3
    silence-period: 5
    message: Response time of service {name} is more than 1000ms in 3 minutes of last 10 minutes.
  service_sla_rule:
    # Metrics value need to be long, double or int
    metrics-name: service_sla
    op: "<"
    threshold: 8000
    # The length of time to evaluate the metrics
    period: 10
    # How many times after the metrics match the condition, will trigger alarm
    count: 2
    # How many times of checks, the alarm keeps silence after alarm triggered, default as same as period.
    silence-period: 3
    message: Successful rate of service {name} is lower than 80% in 2 minutes of last 10 minutes
  service_resp_time_percentile_rule:
    # Metrics value need to be long, double or int
    metrics-name: service_percentile
    op: ">"
    threshold: 1000,1000,1000,1000,1000
    period: 10
    count: 3
    silence-period: 5
    message: Percentile response time of service {name} alarm in 3 minutes of last 10 minutes, due to more than one condition of p50 > 1000, p75 > 1000, p90 > 1000, p95 > 1000, p99 > 1000
  service_instance_resp_time_rule:
    metrics-name: service_instance_resp_time
    op: ">"
    threshold: 1000
    period: 10
    count: 2
    silence-period: 5
    message: Response time of service instance {name} is more than 1000ms in 2 minutes of last 10 minutes
  database_access_resp_time_rule:
    metrics-name: database_access_resp_time
    threshold: 1000
    op: ">"
    period: 10
    count: 2
    message: Response time of database access {name} is more than 1000ms in 2 minutes of last 10 minutes
  endpoint_relation_resp_time_rule:
    metrics-name: endpoint_relation_resp_time
    threshold: 1000
    op: ">"
    period: 10
    count: 2
    message: Response time of endpoint relation {name} is more than 1000ms in 2 minutes of last 10 minutes
#  Active endpoint related metrics alarm will cost more memory than service and service instance metrics alarm.
#  Because the number of endpoint is much more than service and instance.
#
#  endpoint_avg_rule:
#    metrics-name: endpoint_avg
#    op: ">"
#    threshold: 1000
#    period: 10
#    count: 2
#    silence-period: 5
#    message: Response time of endpoint {name} is more than 1000ms in 2 minutes of last 10 minutes

webhooks:
  - http://192.168.3.105:7900/api/Skywalking/AlarmMsg
#  - http://127.0.0.1/go-wechat/
rules:
  # 告警规则名称,必须唯一,以_rule结尾
  service_sla_rule:
     # 指定metrics-name
     metrics-name: service_sla
     # 小于
     op: "<" 
     # 指定阈值
     threshold: 8000                                                                           
     # 10分钟检测一次告警规则                                                 
     period: 10                                                                                 
     # 触发2次告警规则就告警                   
     count: 2                                                                                   
     # 设置的3分钟时间段有相同的告警,不重复告警.
     silence-period: 3 
      # 配置告警消息
     message: Successful rate of service {name} is lower than 80% in 2 minutes of last 10 minutes
webhooks:
- http://192.168.3.105:7900/api/Skywalking/AlarmMsg

标签:10,name,service,time,Skywalking,minutes,1000
来源: https://www.cnblogs.com/AJun816/p/16212441.html