其他分享
首页 > 其他分享> > YCSB项目学习

YCSB项目学习

作者：互联网

主要总结Yahoo的数据库测试项目YCSB的使用（针对redis）。

github网址：https://github.com/brianfrankcooper/YCSB

概述

详细说明：https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload

运行workload一共有六个部分

设置数据库

简要的说，就是创建名为usertable的表，因为ycsb默认的是对usertable进行相关操作。

而对于redis，则不需要相关的操作。

选择合适的DB interface

YCSB的操作是通过DB interface来实现的。最基本的DB interface是com.yahoo.ycsb.BasicDB，会将输出输出到System.out里。可以通过继承DB interface来自定义DB interface，也可以使用原有的DB interface。

选择合适的workload

当前版本提供了六种workload

Workload A: Update heavy workload

This workload has a mix of 50/50 reads and writes. An application example is a session store recording recent actions.

Workload B: Read mostly workload

This workload has a 95/5 reads/write mix. Application example: photo tagging; add a tag is an update, but most operations are to read tags.

Workload C: Read only

This workload is 100% read. Application example: user profile cache, where profiles are constructed elsewhere (e.g., Hadoop).

Workload D: Read latest workload

In this workload, new records are inserted, and the most recently inserted records are the most popular. Application example: user status updates; people want to read the latest.

Workload E: Short ranges

In this workload, short ranges of records are queried, instead of individual records. Application example: threaded conversations, where each scan is for the posts in a given thread (assumed to be clustered by thread id).

Workload F: Read-modify-write

In this workload, the client will read a record, modify it, and write back the changes. Application example: user database, where user records are read and modified by the user or to record user activity.

自定义Workload

当然也可以自定义workload：https://github.com/brianfrankcooper/YCSB/wiki/Implementing-New-Workloads

一般常用的workload property如fieldcount，fieldlength，requestdistribution等。

全部properties如下：

The property files used with the core workload generator can specify values for the following properties:

选择合适的runtime parameter

主要是

load data

需要指定redis.host，redis.port。（可以指定redis.password和redis.cluster）

以上参数也可以在命令中指定，比如

1
./bin/ycsb load redis -s -P workloads/workloada -p "redis.host=127.0.0.1" -p "redis.port=6379"

run data

同理

1
./bin/ycsb run redis -s -P workloads/workloada -p "redis.host=127.0.0.1" -p "redis.port=6379"

一些注意事项

ycsb不支持对key的长度的修改

issue网址：https://github.com/brianfrankcooper/YCSB/issues/587

ycsb对key命名规则是两种，hashed模式会生成user加固定长度的一串hash值，而ordered模式会按照user加顺序的方式来命名。

1
2
insertorder=hashed      # user6284781860667377211, user8517097267634966620, user1820151046732198393
insertorder=ordered     # user1, user2, user3032

ycsb运行时错误：Read time out

错误类似于

1
2
3
4
5
6
redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out

at redis.clients.jedis.Protocol.process(Protocol.java:79)
at redis.clients.jedis.Protocol.read(Protocol.java:131)
at redis.clients.jedis.Connection.getIntegerReply(Connection.java:188)
at redis.clients.jedis.Jedis.sismember(Jedis.java:1266)

当数据量变大之后，延时可能会变成几秒，而Jedis默认的是2秒的超时限制。

修改redis的src文件夹的RedisClient.java

将

jedis = new Jedis(host, port);

修改为

jedis = new Jedis(host, port, 10000);

即从默认的2秒上升为10秒。

参考资料

标签：workload,项目,default,YCSB,redis,ycsb,学习,com
来源： https://www.cnblogs.com/lijianming180/p/12251327.html