其他分享
首页 > 其他分享> > Sysbench 使用总结

Sysbench 使用总结

作者:互联网

Sysbench使用总结

实践环境

CentOS 7.8

Sysbench 1.0.20

下载地址:https://github.com/akopytov/sysbench/archive/refs/tags/1.0.20.tar.gz

用法介绍

sysbench [options]... [testname] [command]

具体说明:

应用实践

mysql数据库性能基准测试

创建数据库

CREATE DATABASE `sbtest` DEFAULT CHARACTER SET utf8;

注意:如果不事先创建数据库,则运行一下测试脚本时,会报错:FATAL: error 1049: Unknown database 'sbtest'

插入数据测试

运行压测准备命令

# sysbench --oltp-tables-count=5 --oltp_table_size=0 --mysql-host=10.118.80.88 --mysql-user=testacc --mysql-password=test1234 sysbench-1.0.20/tests/include/oltp_legacy/insert.lua prepare

说明:

1、执行以上命令后,会自动创建5张空表

2、sysbench-1.0.20/tests/include/oltp_legacy/insert.luasysbench工具压缩包中自带脚本,下文所用到的脚本皆非自定义脚本,不再赘述

运行插入数据测试

# sysbench --threads=80 --time=600 --events=50000000 --oltp-tables-count=5 --oltp_table_size=10000000 --mysql-host=10.118.80.88 --mysql-user=testacc --mysql-password=test1234 sysbench-1.0.20/tests/include/oltp_legacy/insert.lua run > result.txt

查看运行结果

混合操作测试

# sysbench  --threads=300 --time=1200 --oltp-tables-count=5 --oltp_table_size=10000000 mysql-host=10.118.80.88 --mysql-user=testacc --mysql-password=test1234 sysbench-1.0.20/tests/include/oltp_legacy/oltp.lua run > result.txt

运行清理命令

# sysbench --oltp-tables-count=5 --mysql-host=10.118.80.88 --mysql-user=testacc --mysql-password=test1234 sysbench-1.0.20/tests/include/oltp_legacy/insert.lua cleanup

注意:执行preparecleanup命令时的--oltp-tables-count选项参数值要保持一致

lua脚本分析

sysbench-1.0.20/tests/include/oltp_legacy/oltp.lua

具体分析参见中文注释

pathtest = string.match(test, "(.*/)") # 首先,正则匹配当前测试名称(sysbench命令行提供的testname)

if pathtest then # 如果测试名称正则匹配 .*/
   # 说明测试名称为脚本,形如sysbench-1.0.20/tests/include/oltp_legacy/oltp.lua,因为测试脚本和common.lua位于同一层级目录,所以,拼接脚本所在目录路径和"common.lua"得到common.lua文件路径,然后,加载文件
   dofile(pathtest .. "common.lua") 
else # 否则,加载工具捆绑的common脚本,即sysbench_installation_home/share/sysbench/common.lua
   require("common")
end


function thread_init() # 线程初始化函数定义
   set_vars() # 调用common中的set_vars函数设置变量

   # 如果数据库驱动为mysql、attachsql,并且mysql表引擎为myisam 
   if (((db_driver == "mysql") or (db_driver == "attachsql")) and mysql_table_engine == "myisam") then
      # 则走下述逻辑
      local i
      local tables = {}
      # 根据命令行oltp_tables_count参数值,初始化tables数组元素为表名,格式形如sbtestN WRITE,其中N从1开始,最大值等于oltp_tables_count,即表数量
      for i=1, oltp_tables_count do 
         tables[i] = string.format("sbtest%i WRITE", i)
      end
      begin_query = "LOCK TABLES " .. table.concat(tables, " ,")
      commit_query = "UNLOCK TABLES"
   else # 否则定义事务起始查询语句为 begin_query为BEGIN,结束查询语句为 COMMIT
      begin_query = "BEGIN"
      commit_query = "COMMIT"
   end

end

# 定义函数 用于获取查询条件
function get_range_str()
   local start = sb_rand(1, oltp_table_size) # 随机获取 [1, 单表记录总数] 范围内的整数,作为范围查询条件起始值
   return string.format(" WHERE id BETWEEN %u AND %u",
                        start, start + oltp_range_size - 1) # 返回查询条件WHERE id BETWEEN start AND (start + 99) (oltp_range_size默认为100
end

定义待执行的“事件”函数
function event()
   local rs # 查询返回结果
   local i   
   local table_name
   # 以下变量用于存放字段值和查询
   local c_val
   local pad_val
   local query

   table_name = "sbtest".. sb_rand_uniform(1, oltp_tables_count) # 随机获取表名,格式形如 sbtestN,其中N为从[1,表数量]范围内的随机整数
   if not oltp_skip_trx then # 如果不跳过事务,
      db_query(begin_query) # 则执行do_query(oltp_skip_trx默认为false,所以这里会执行do_query),即标记事务开始
   end
   
    
   if not oltp_write_only then # 如果not oltp_write_only为真(oltp_write_only默认为false)
   # 则执行以下for循环(oltp_point_selects默认为10次,所以,会执行10次for循环查询,查询语句为:
   # SELECT c FROM table_name WHERE id=id_value
   # 说明:
   #  table_name   上述获取的随机表名,下同,不再赘述
   #  c            非索引列
   #  id           主键列
   #  id_value     目标id值,随机获取已存在记录ID
   for i=1, oltp_point_selects do
      rs = db_query("SELECT c FROM ".. table_name .." WHERE id=" ..
                       sb_rand(1, oltp_table_size))
   end

   if oltp_range_selects then # 如果oltp_range_selects为true,(oltp_range_selects默认为true)
   # 执行按范围简单查询 1次(oltp_simple_ranges默认为1次,所以只会执行1次for循环查询)
   # SQL语句为:
   # SELECT c FROM table_name WHERE id BETWEEN id_value1 AND id_value2 
   # 说明:
   #  id_value1   随机获取已存在记录ID,下同,不再赘述
   #  id_value2   id_value1 + 99,下同,不再赘述
   for i=1, oltp_simple_ranges do 
      rs = db_query("SELECT c FROM ".. table_name .. get_range_str())
   end


   # 执行 按范围sum查询  1次 (oltp_sum_ranges默认值为 1,所以只会执行1次for循环查询)
   # SQL语句为:
   # SELECT sum(c) FROM table_name WHERE id BETWEEN id_value1 AND id_value2 
   for i=1, oltp_sum_ranges do 
      rs = db_query("SELECT SUM(K) FROM ".. table_name .. get_range_str())
   end

   # 执行 按范围排序查询  1次 (oltp_order_ranges默认值为 1,所以只会执行1次for循环查询)
   # SQL语句为:
   # SELECT c FROM table_name WHERE id BETWEEN id_value1 AND id_value2 ORDER BY c
   for i=1, oltp_order_ranges do
      rs = db_query("SELECT c FROM ".. table_name .. get_range_str() ..
                    " ORDER BY c")
   end

   # 执行 DISTINCT 排序查询 1次 (oltp_distinct_ranges默认值为 1,所以只会执行1次for循环查询)
   # SQL语句为:
   # SELECT DISTINCT c FROM table_name WHERE id BETWEEN id_value1 AND id_value2 ORDER BY c
   for i=1, oltp_distinct_ranges do
      rs = db_query("SELECT DISTINCT c FROM ".. table_name .. get_range_str() ..
                    " ORDER BY c")
   end

   end

   end
   
   if not oltp_read_only then # not oltp_read_only为真(oltp_read_only默认为false)
   # 执行 更新索引列字段操作 1次 (oltp_index_updates默认值为 1,所以只会执行1次for循环查询)
   # SQL语句为:
   # UPDATE table_name SET k=k+1  WHERE id=id_value
   # 说明:
   #   k为索引列字段
   for i=1, oltp_index_updates do
      rs = db_query("UPDATE " .. table_name .. " SET k=k+1 WHERE id=" .. sb_rand(1, oltp_table_size))
   end

   # 执行 更新非索引列字段操作 1次 (oltp_index_updates默认值为 1,所以只会执行1次for循环查询)
   # SQL语句为:
   # UPDATE table_name SET c="c_val"  WHERE id=id_value
   # 说明:
   #  c_val  长度为119位字符的随机字符串
   for i=1, oltp_non_index_updates do
      c_val = sb_rand_str("###########-###########-###########-###########-###########-###########-###########-###########-###########-###########") # 长度为119位字符的随机字符串
      query = "UPDATE " .. table_name .. " SET c='" .. c_val .. "' WHERE id=" .. sb_rand(1, oltp_table_size)
      rs = db_query(query)
      if rs then
        print(query) # 打印查询结果
      end
   end
   
   # 执行 删除指定记录操作 1次 (oltp_delete_inserts默认值为 1,所以只会执行1次for循环查询)
   # SQL语句为:
   # DELETE FROM table_name WHERE id=id_value1
   for i=1, oltp_delete_inserts do

   i = sb_rand(1, oltp_table_size)

   rs = db_query("DELETE FROM " .. table_name .. " WHERE id=" .. i)
   
   c_val = sb_rand_str([[
###########-###########-###########-###########-###########-###########-###########-###########-###########-###########]]) 
   pad_val = sb_rand_str([[
###########-###########-###########-###########-###########]]) # 64位随机字符串 
   
   # 执行 插入数据操作 1次
   # SQL语句为:
   # INSERT INTO table_name(id, K, c, pad)VALUES(id_value, num, field_value1, field_value2 )
   # 说明:
   #   num   [1,表记录数] 范围内的随机整数
   #   field_value1  119位随机字符串 
   #   field_value2  64位随机字符串 
    rs = db_query("INSERT INTO " .. table_name ..  " (id, k, c, pad) VALUES " .. string.format("(%d, %d, '%s', '%s')",i, sb_rand(1, oltp_table_size) , c_val, pad_val))

   end

   end -- oltp_read_only

   if not oltp_skip_trx then
      db_query(commit_query)
   end

end
sysbench-1.0.20/tests/include/oltp_legacy/common.lua

具体分析参见中文注释

-- Input parameters
-- oltp-tables-count - number of tables to create
-- oltp-secondary - use secondary key instead PRIMARY key for id column
--
--

# 定义插入函数
function create_insert(table_id)

   local index_name
   local i
   local j
   local query

   if (oltp_secondary) then
     index_name = "KEY xid"
   else
     index_name = "PRIMARY KEY"
   end

   if (pgsql_variant == 'redshift') then
      auto_inc_type = "INTEGER IDENTITY(1,1)"
   else
      auto_inc_type = "SERIAL"
   end

   i = table_id

   print("Creating table 'sbtest" .. i .. "'...")
   # 根据不同驱动,执行对应建表语句,表名格式 sbtestN,其中N从1开始,最大值为oltp_tables_count
   if ((db_driver == "mysql") or (db_driver == "attachsql")) then
      query = [[
CREATE TABLE sbtest]] .. i .. [[ (
id INTEGER UNSIGNED NOT NULL ]] ..
((oltp_auto_inc and "AUTO_INCREMENT") or "") .. [[,
k INTEGER UNSIGNED DEFAULT '0' NOT NULL,
c CHAR(120) DEFAULT '' NOT NULL,
pad CHAR(60) DEFAULT '' NOT NULL,
]] .. index_name .. [[ (id)
) /*! ENGINE = ]] .. mysql_table_engine ..
" MAX_ROWS = " .. myisam_max_rows .. " */ " ..
   (mysql_table_options or "")

   elseif (db_driver == "pgsql") then
      query = [[
CREATE TABLE sbtest]] .. i .. [[ (
id ]] .. auto_inc_type .. [[ NOT NULL,
k INTEGER DEFAULT '0' NOT NULL,
c CHAR(120) DEFAULT '' NOT NULL,
pad CHAR(60) DEFAULT '' NOT NULL,
]] .. index_name .. [[ (id)
) ]]

   elseif (db_driver == "drizzle") then
      query = [[
CREATE TABLE sbtest (
id INTEGER NOT NULL ]] .. ((oltp_auto_inc and "AUTO_INCREMENT") or "") .. [[,
k INTEGER DEFAULT '0' NOT NULL,
c CHAR(120) DEFAULT '' NOT NULL,
pad CHAR(60) DEFAULT '' NOT NULL,
]] .. index_name .. [[ (id)
) ]]
   else
      print("Unknown database driver: " .. db_driver)
      return 1
   end

   db_query(query)

   print("Inserting " .. oltp_table_size .. " records into 'sbtest" .. i .. "'")

   if (oltp_auto_inc) then
      db_bulk_insert_init("INSERT INTO sbtest" .. i .. "(k, c, pad) VALUES")
   else
      db_bulk_insert_init("INSERT INTO sbtest" .. i .. "(id, k, c, pad) VALUES")
   end

   local c_val
   local pad_val


   # for循环,往表插入数据,直到插入oltp_table_size条记录
   for j = 1,oltp_table_size do

   c_val = sb_rand_str([[
###########-###########-###########-###########-###########-###########-###########-###########-###########-###########]])
   pad_val = sb_rand_str([[
###########-###########-###########-###########-###########]])

      if (oltp_auto_inc) then
	 db_bulk_insert_next("(" .. sb_rand(1, oltp_table_size) .. ", '".. c_val .."', '" .. pad_val .. "')")
      else
	 db_bulk_insert_next("("..j.."," .. sb_rand(1, oltp_table_size) .. ",'".. c_val .."', '" .. pad_val .. "'  )")
      end
   end

   db_bulk_insert_done()

   if oltp_create_secondary then
     print("Creating secondary indexes on 'sbtest" .. i .. "'...")
     db_query("CREATE INDEX k_" .. i .. " on sbtest" .. i .. "(k)")
   end

end

# 定义执行prepare命令时调用的函数
function prepare()
   local query
   local i
   local j

   set_vars() # 设置变量

   db_connect() # 连接数据库


   # for循环插入数据,循环次数为 oltp_tables_count,
   for i = 1,oltp_tables_count do
     create_insert(i) 
   end

   return 0
end

# 定义执行clean命令时调用的函数
function cleanup()
   local i

   set_vars() # 调用set_vars函数,设置变量

   for i = 1,oltp_tables_count do
   print("Dropping table 'sbtest" .. i .. "'...") 
   db_query("DROP TABLE IF EXISTS sbtest".. i ) # 删除表sbtestN,其中N从1~ oltp_tables_count,即表数量
   end
end

function set_vars()
   # 以下非本地变量的默认值都为 nil
   oltp_table_size = tonumber(oltp_table_size) or 10000  # 单表记录总数 默认值10000
   oltp_range_size = tonumber(oltp_range_size) or 100    # between and 范围查询 默认值100
   oltp_tables_count = tonumber(oltp_tables_count) or 1    # 表数量 默认值 1
   oltp_point_selects = tonumber(oltp_point_selects) or 10 # 按点查询次数(where id=) 默认值 10
   oltp_simple_ranges = tonumber(oltp_simple_ranges) or 1  # 简单查询次数 默认值 1
   oltp_sum_ranges = tonumber(oltp_sum_ranges) or 1      # sum查询次数 默认值 1
   oltp_order_ranges = tonumber(oltp_order_ranges) or 1  # order by排序查询次数 默认值 1
   oltp_distinct_ranges = tonumber(oltp_distinct_ranges) or 1 # distinct范围查询次数 默认值 1
   oltp_index_updates = tonumber(oltp_index_updates) or 1     # 更新索引列操作次数 默认值 1
   oltp_non_index_updates = tonumber(oltp_non_index_updates) or 1 # 更新索引列字段操作次数 默认值 1
   oltp_delete_inserts = tonumber(oltp_delete_inserts) or 1   # 执行删除记录操作次数 默认值 1

   if (oltp_range_selects == 'off') then
      oltp_range_selects = false
   else 
      oltp_range_selects = true
   end

   if (oltp_auto_inc == 'off') then
      oltp_auto_inc = false
   else
      oltp_auto_inc = true
   end

   if (oltp_read_only == 'on') then
      oltp_read_only = true
   else
      oltp_read_only = false
   end

   if (oltp_write_only == 'on') then
      oltp_write_only = true
   else
      oltp_write_only = false
   end

   if (oltp_read_only and oltp_write_only) then
      error("--oltp-read-only and --oltp-write-only are mutually exclusive")
   end

   if (oltp_skip_trx == 'on') then
      oltp_skip_trx = true
   else
      oltp_skip_trx = false
   end

   if (oltp_create_secondary == 'off') then
      oltp_create_secondary = false
   else
      oltp_create_secondary = true
   end

   if (pgsql_variant == 'redshift') then
      oltp_create_secondary = false
      oltp_delete_inserts = 0
   end

end

运行结果说明

混合操作压测结果为例

# cat result.txt
sysbench 1.0.20 (using bundled LuaJIT 2.1.0-beta2)

Running the test with following options:
Number of threads: 300
Initializing random number generator from current time


Initializing worker threads...

Threads started!

SQL statistics:
    queries performed:
        read:                            34507228
        write:                           9859208
        other:                           4929604
        total:                           49296040
    transactions:                        2464802 (2053.74 per sec.)
    queries:                             49296040 (41074.74 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          1200.1525s
    total number of events:              2464802

Latency (ms):
         min:                                  129.86
         avg:                                  146.06
         max:                                  806.70
         95th percentile:                      155.80
         sum:                            360014005.42

Threads fairness:
    events (avg/stddev):           8216.0067/329.90
    execution time (avg/stddev):   1200.0467/0.04

结果项说明:

SQL statistics:
    queries performed:                   执行的SQL查询:
        read:                            读请求次数
        write:                           写请求次数
        other:                           其它请求次数
        total:                           总请求次数
    transactions:                        执行的事务总数   (每秒事务数.)
    queries:                             执行的查询总次数   (每秒查询次数.)
    ignored errors:                      忽略错误数      (每秒忽略错误次数.)
    reconnects:                          数据库重连次数      (每秒重连次数.)

General statistics:
    total time:                          总运行时间,以秒为单位
    total number of events:              总执行事件数

Latency (毫秒为单位):
         min:                                  最小请求响应时间
         avg:                                  平均请求响应时间
         max:                                  最大请求响应时间
         95th percentile:                      95%的请求响应时间
         sum:                                  总请求响应时间

Threads fairness:
    events (avg/stddev):           执行的事件总数(平均每个线程执行的事件总数/标准差)
    execution time (avg/stddev):   执行耗时(平均每个线程执行事件耗时/标准差)

参考连接

https://github.com/akopytov/sysbench#general-command-line-options

标签:总结,end,..,--,oltp,使用,table,id,Sysbench
来源: https://www.cnblogs.com/shouke/p/14618738.html