dubbo泛化调用踩坑精度问题
作者：互联网
问题背景
开放平台对接外部服务暴露http接口，然后http接口请求根据参数将请求分发至内部dubbo服务，分发动作使用的dubbo泛化调用。计费接口测试时发现cost字段（BigDecimal）出现精度问题
测试数据的cost值为0.01
http网关服务日志，可以看到入参的cost=0.01，没有问题
2021-04-22 10:40:18,597 INFO [4fb5b4087d2c462eab8b9ede87b8d272] [DubboServerHandler-10.200.25.210:16976-thread-477] com.dianwoda.open.toolbox.dubbo.filter.DubboInvokeLogFilter:invoke:67 Invoke com.alibaba.dubbo.rpc.service.GenericService.$invoke(java.lang.String,[Ljava.lang.String;,[Ljava.lang.Object;):1.0.0 cost 21ms , 10.200.25.210:0=>10.200.36.21:16815 , arguments=["addBalanceLogCheckTradeNoStr",["com.dianwoda.billing.settle.provider.outmoded.dto.BalanceDTO"],[{"cityId":1,"riderId":5150213,"riderType":0,"type":95,"cost":0.01,"tradeNoStr":"6899423540852876801619059218000"}]] , result=true
dubbo服务端日志，可以看到如惨已经出现问题：“cost”:0.009999999776482582
2021-04-22 10:40:18,577 INFO [4fb5b4087d2c462eab8b9ede87b8d272] [DubboServerHandler-10.200.36.21:16815-thread-498] dubbo.accesslog.com.....billing.settle.provider.outmoded.RiderTradeCostSettleOutmodedProvider:info:42 [DUBBO] [2021-04-22 10:40:18] 10.200.25.210:42688 -> 10.200.36.21:16815 - com.....billing.settle.provider.outmoded.RiderTradeCostSettleOutmodedProvider:1.0.0 addBalanceLogCheckTradeNoStr(com.....billing.settle.provider.outmoded.dto.BalanceDTO) [{"reason":null,"bankCard":null,"withdrawType":null,"bankName":null,"cityId":1,"type":95,"riderId":5150213,"riskChecked":null,"payType":null,"feature":null,"blocked":null,"id":null,"riderType":0,"batchRecordId":null,"batchNo":null,"cost":0.009999999776482582,"tradeNo":null,"bankCardType":null,"currentServiceType":null,"effectiveBalance":null,"tradeWay":null,"sourceTradeNo":null,"sourceBalanceType":null,"factorage":null,"verifyTm":null,"finishTm":null,"tradeNoStr":"6899423540852876801619059218000","paid":null,"name":null,"insTm":null,"withdrawTm":null,"account":null}], dubbo version: 2.5.3, current host: 10.200.36.21
问题排查
泛化调用的锅？
网关侧封装的泛化调用代码，代码看到此处，精度问题有两种可能原因
- dubbo泛化调用中类型转换问题？
- 泛化调用前参数转换问题？
public ResponseDTO<String> dock(DockRequest request) {
Object retObj;
try {
CtURL url = CtURL.parseURL(request.getRequestType());
GenericService genericService = getDubboGenericService(url);
...
Class clazz = getClass(url);
// 上下文类加载器中存在类则尝试通过上下文获取bean进行调用
if (null != clazz) {
Map<String, Object> beanMap = applicationContext.getBeansOfType(clazz);
if (beanMap.size() == 1) {
retObj = invokeMethod(clazz, beanMap.values().iterator().next(), url,
request.getRequestBody());
} else {
// 上下文同类型bean存在多个走泛化调用
retObj = genericService
.$invoke(url.getMethod(), parameterTypes, convertArray(jsonArray));
}
} else {
// 类加载器中不存在class直接泛化调用
retObj = genericService
.$invoke(url.getMethod(), parameterTypes, convertArray(jsonArray));
}
}...
}
// 注册dubbo泛化调用消费者至spring上下文
private GenericService getDubboGenericService(CtURL url) {
String group = url.getGroup();
String version = url.getVersion();
String beanName =
StringUtils.firstNonBlank(url.getGroup(), "DEFAULT") + "/" + url.getService() + (
StringUtils.isNotBlank(version) ? "" : "/" + version);
if (applicationContext.containsBean(beanName)) {
return (GenericService) applicationContext.getBean(beanName);
}
synchronized (applicationContext) {
if (applicationContext.containsBean(beanName)) {
return (GenericService) applicationContext.getBean(beanName);
}
AbstractBeanDefinition definition = BeanDefinitionBuilder
.genericBeanDefinition(ReferenceBean.class)
.addPropertyValue("application", new ApplicationConfig("dock-common"))
.addPropertyValue("registries", registryConfigs)
.addPropertyValue("interface", url.getService())
.addPropertyValue("group", group).addPropertyValue("version", version)
.addPropertyValue("generic", "true").addPropertyValue("retries", 0)
.addPropertyValue("timeout", 3000).getBeanDefinition();
BeanFactory beanFactory = applicationContext.getAutowireCapableBeanFactory();
BeanDefinitionRegistry registry = (BeanDefinitionRegistry) beanFactory;
registry.registerBeanDefinition(beanName, definition);
return (GenericService) beanFactory.getBean(beanName);
}
}
泛化调用原理
dubbo消费者调用服务提供者过程不多说，消费者与提供者端invoker都会经历对应group的filter，泛化调用则由其中一个filter实现：com.alibaba.dubbo.rpc.filter.GenericImplFilter，将invocation方法设置为com.alibaba.dubbo.common.Constants#$INVOKE，提供者端则由com.alibaba.dubbo.rpc.filter.GenericFilter责任链节点识别泛化调用，消费者端GenericImplFilter源码如下
public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException {
String generic = invoker.getUrl().getParameter(Constants.GENERIC_KEY);
// 确定是泛化调用
if (ProtocolUtils.isGeneric(generic)
&& !Constants.$INVOKE.equals(invocation.getMethodName())
&& invocation instanceof RpcInvocation) {
RpcInvocation invocation2 = (RpcInvocation) invocation;
String methodName = invocation2.getMethodName();
Class<?>[] parameterTypes = invocation2.getParameterTypes();
Object[] arguments = invocation2.getArguments();
String[] types = new String[parameterTypes.length];
for (int i = 0; i < parameterTypes.length; i++) {
types[i] = ReflectUtils.getName(parameterTypes[i]);
}
Object[] args;
// generic参数为bean，则以bean的方式序列化参数
if (ProtocolUtils.isBeanGenericSerialization(generic)) {
args = new Object[arguments.length];
for (int i = 0; i < arguments.length; i++) {
args[i] = JavaBeanSerializeUtil.serialize(arguments[i], JavaBeanAccessor.METHOD);
}
} else {
// 否则使用工具类序列化
args = PojoUtils.generalize(arguments);
}
invocation2.setMethodName(Constants.$INVOKE);
invocation2.setParameterTypes(GENERIC_PARAMETER_TYPES);
invocation2.setArguments(new Object[]{methodName, types, args});
Result result = invoker.invoke(invocation2);
...
}
dubbo对参数进行了序列化，继续看下工具类序列化代码，序列化其实是将Java类序列化为一个map对象，我们只需要查看primitive类型的部分即可，可以看到对于primitive类型，dubbo序列化工具类什么也没做直接返回原类型数据
private static Object generalize(Object pojo, Map<Object, Object> history) {
...
if (ReflectUtils.isPrimitives(pojo.getClass())) {
return pojo;
}
...
}
小结
dubbo表示这锅我不背-_-!!!
泛化调用前参数转换问题？
网关侧参数转换代码
// 参数原JSON字符串来自JsonArray.get(0)：[{\"cityId\":1,\"riderId\":5150213,\"riderType\":0,\"type\":95,\"cost\":0.01,\"tradeNoStr\":\"6899423540852876801618889049000\"}]
private Object convert(JsonElement element) {
if (element == null || element.isJsonNull()) {
return null;
} else if (element.isJsonPrimitive()) {
JsonPrimitive primitive = (JsonPrimitive) element;
if (primitive.isNumber()) {
// commons-lang3-3.8.1.jar
// org.apache.commons.lang3.math.NumberUtils#createNumber
return NumberUtils.createNumber(primitive.getAsString());
} else if (primitive.isBoolean()) {
return element.getAsBoolean();
} else {
return element.getAsString();
}
} else if (element.isJsonArray()) {
return convertArray(element.getAsJsonArray());
} else {
JsonObject jsonObject = element.getAsJsonObject();
Map<String, Object> map = new LinkedHashMap<>();
jsonObject.entrySet().forEach(entry -> {
map.put(entry.getKey(), convert(entry.getValue()));
});
return map;
}
}
尝试复现问题
public static void main(String[] args) {
Gson gson = new GsonBuilder().setDateFormat("yyyy-MM-dd HH:mm:ss,SSS").create();
JsonArray je = gson.fromJson("[{\"cityId\":1,\"riderId\":5150213,\"riderType\":0,\"type\":95,\"cost\":0.01,\"tradeNoStr\":\"6899423540852876801618889049000\"}]", JsonArray.class);
System.out.println("333="+je);
Object[] objs = convertArray(je);
for (Object data : objs) {
System.out.println(data.getClass());
if (data instanceof Map) {
Map<?, ?> datas = (Map<?, ?>) data;
datas.forEach((k, v) -> System.out
.printf("k=%s,v=%s%n", k, v));
}
}
}
数据结果
333=[{"cityId":1,"riderId":5150213,"riderType":0,"type":95,"cost":0.01,"tradeNoStr":"6899423540852876801618889049000"}]
class java.util.LinkedHashMap
k=cityId,v=1
k=riderId,v=5150213
k=riderType,v=0
k=type,v=95
k=cost,v=0.01
k=tradeNoStr,v=6899423540852876801618889049000
？？？cost值没有问题，为毛到了服务端出现了问题？服务端有内鬼？查看服务端泛化调用责任链逻辑：com.alibaba.dubbo.rpc.filter.GenericFilter
public Result invoke(Invoker<?> invoker, Invocation inv) throws RpcException {
if (inv.getMethodName().equals(Constants.$INVOKE)
&& inv.getArguments() != null
&& inv.getArguments().length == 3
&& !ProtocolUtils.isGeneric(invoker.getUrl().getParameter(Constants.GENERIC_KEY))) {
String name = ((String) inv.getArguments()[0]).trim();
String[] types = (String[]) inv.getArguments()[1];
Object[] args = (Object[]) inv.getArguments()[2];
try {
Method method = ReflectUtils.findMethodByMethodSignature(invoker.getInterface(), name, types);
Class<?>[] params = method.getParameterTypes();
if (args == null) {
args = new Object[params.length];
}
String generic = inv.getAttachment(Constants.GENERIC_KEY);
if (StringUtils.isEmpty(generic)
|| ProtocolUtils.isDefaultGenericSerialization(generic)) {
// 反序列化对象
args = PojoUtils.realize(args, params, method.getGenericParameterTypes());
} ...
}
查看反序列化逻辑，我们客户端泛化调用时将入参序列化为一个map对象，此时服务端将其反序列化为接口的实际入参类型：com.alibaba.dubbo.common.utils.PojoUtils#realize0，源码可以看到通过反射调用目标方法参数类型的set方法或filed字段写入map的value值
private static Object realize0(Object pojo, Class<?> type, Type genericType, final Map<Object, Object> history) {
...
if (pojo instanceof Map<?, ?> && type != null) {
...
} else {
Object dest = newInstance(type);
history.put(pojo, dest);
for (Map.Entry<Object, Object> entry : map.entrySet()) {
Object key = entry.getKey();
if (key instanceof String) {
String name = (String) key;
Object value = entry.getValue();
if (value != null) {
Method method = getSetterMethod(dest.getClass(), name, value.getClass());
Field field = getField(dest.getClass(), name);
if (method != null) {
if (!method.isAccessible())
method.setAccessible(true);
Type ptype = method.getGenericParameterTypes()[0];
value = realize0(value, method.getParameterTypes()[0], ptype, history);
try {
method.invoke(dest, value);
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException("Failed to set pojo " + dest.getClass().getSimpleName() + " property " + name
+ " value " + value + "(" + value.getClass() + "), cause: " + e.getMessage(), e);
}
} else if (field != null) {
value = realize0(value, field.getType(), field.getGenericType(), history);
try {
field.set(dest, value);
} catch (IllegalAccessException e) {
throw new RuntimeException("Failed to set filed " + name + " of pojo " + dest.getClass().getName() + " : " + e.getMessage(), e);
}
}
}
}
}
...
}
这里有一个内部类型转换Object-》BigDecimal，如果Map中的value值不是BigDecimal类型则可能会出现问题，回过头修改下复现代码查看org.apache.commons.lang3.math.NumberUtils#createNumber方法将字符“0.01”转换为Number类型的实际类型
public static void main(String[] args) {
Gson gson = new GsonBuilder().setDateFormat("yyyy-MM-dd HH:mm:ss,SSS").create();
JsonArray je = gson.fromJson("[{\"cityId\":1,\"riderId\":5150213,\"riderType\":0,\"type\":95,\"cost\":0.01,\"tradeNoStr\":\"6899423540852876801618889049000\"}]", JsonArray.class);
System.out.println("333="+je);
Object[] objs = convertArray(je);
for (Object data : objs) {
System.out.println(data.getClass());
if (data instanceof Map) {
Map<?, ?> datas = (Map<?, ?>) data;
datas.forEach((k, v) -> System.out
.printf("k=%s,v=%s,k.class=%s,v.class=%s%n", k, v, k.getClass(), v.getClass()));
}
}
}
输出结果
333=[{"cityId":1,"riderId":5150213,"riderType":0,"type":95,"cost":0.01,"tradeNoStr":"6899423540852876801618889049000"}]
class java.util.LinkedHashMap
k=cityId,v=1,k.class=class java.lang.String,v.class=class java.lang.Integer
k=riderId,v=5150213,k.class=class java.lang.String,v.class=class java.lang.Integer
k=riderType,v=0,k.class=class java.lang.String,v.class=class java.lang.Integer
k=type,v=95,k.class=class java.lang.String,v.class=class java.lang.Integer
k=cost,v=0.01,k.class=class java.lang.String,v.class=class java.lang.Float
k=tradeNoStr,v=6899423540852876801618889049000,k.class=class java.lang.String,v.class=class java.lang.String
小结
问题已经确认正是String类型转Number类型过程中对于小数类型，由于程序无法识别你的小数类型是哪种浮点类型，默认按照最小满足方式转换，0.01转换为Float类型，服务端实际字段类型为BigDecimal类型，Float至BigDecimal类型隐式转换出现了精度问题。验证代码如下
float f = 0.01f; System.out.println(new BigDecimal(f)); // 输出结果 0.00999999977648258209228515625
总结
问题原因：字符串转浮点型导致，因为程序无法感知你是哪种浮点型，浮点型float转BigDecimal存在精度问题
问题解法：较为粗暴的将所有Number类型转为BigDecimal，Gson当然也早已考虑到这些场景咯，针对primitive类型提供了转换方法=com.google.gson.JsonPrimitive#getAsBigDecimal
———————————————— 版权声明：本文为CSDN博主「会灰翔的灰机」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。 原文链接：https://blog.csdn.net/u010597819/article/details/116095242
标签：dubbo,调用,泛化,Object,null,class,String 来源： https://www.cnblogs.com/Not-If/p/16336870.html