其他分享
首页 > 其他分享> > dubbo泛化调用踩坑精度问题

dubbo泛化调用踩坑精度问题

作者:互联网

问题背景 开放平台对接外部服务暴露http接口,然后http接口请求根据参数将请求分发至内部dubbo服务,分发动作使用的dubbo泛化调用。计费接口测试时发现cost字段(BigDecimal)出现精度问题 测试数据的cost值为0.01 http网关服务日志,可以看到入参的cost=0.01,没有问题
2021-04-22 10:40:18,597 INFO [4fb5b4087d2c462eab8b9ede87b8d272] [DubboServerHandler-10.200.25.210:16976-thread-477] com.dianwoda.open.toolbox.dubbo.filter.DubboInvokeLogFilter:invoke:67 Invoke com.alibaba.dubbo.rpc.service.GenericService.$invoke(java.lang.String,[Ljava.lang.String;,[Ljava.lang.Object;):1.0.0 cost 21ms , 10.200.25.210:0=>10.200.36.21:16815 , arguments=["addBalanceLogCheckTradeNoStr",["com.dianwoda.billing.settle.provider.outmoded.dto.BalanceDTO"],[{"cityId":1,"riderId":5150213,"riderType":0,"type":95,"cost":0.01,"tradeNoStr":"6899423540852876801619059218000"}]] , result=true

dubbo服务端日志,可以看到如惨已经出现问题:“cost”:0.009999999776482582

2021-04-22 10:40:18,577 INFO [4fb5b4087d2c462eab8b9ede87b8d272] [DubboServerHandler-10.200.36.21:16815-thread-498] dubbo.accesslog.com.....billing.settle.provider.outmoded.RiderTradeCostSettleOutmodedProvider:info:42  [DUBBO] [2021-04-22 10:40:18] 10.200.25.210:42688 -> 10.200.36.21:16815 - com.....billing.settle.provider.outmoded.RiderTradeCostSettleOutmodedProvider:1.0.0 addBalanceLogCheckTradeNoStr(com.....billing.settle.provider.outmoded.dto.BalanceDTO) [{"reason":null,"bankCard":null,"withdrawType":null,"bankName":null,"cityId":1,"type":95,"riderId":5150213,"riskChecked":null,"payType":null,"feature":null,"blocked":null,"id":null,"riderType":0,"batchRecordId":null,"batchNo":null,"cost":0.009999999776482582,"tradeNo":null,"bankCardType":null,"currentServiceType":null,"effectiveBalance":null,"tradeWay":null,"sourceTradeNo":null,"sourceBalanceType":null,"factorage":null,"verifyTm":null,"finishTm":null,"tradeNoStr":"6899423540852876801619059218000","paid":null,"name":null,"insTm":null,"withdrawTm":null,"account":null}], dubbo version: 2.5.3, current host: 10.200.36.21

问题排查

泛化调用的锅?

网关侧封装的泛化调用代码,代码看到此处,精度问题有两种可能原因

  1. dubbo泛化调用中类型转换问题?
  2. 泛化调用前参数转换问题?
public ResponseDTO<String> dock(DockRequest request) {
    Object retObj;
    try {
        CtURL url = CtURL.parseURL(request.getRequestType());
        GenericService genericService = getDubboGenericService(url);
        ...
        Class clazz = getClass(url);
        // 上下文类加载器中存在类则尝试通过上下文获取bean进行调用
        if (null != clazz) {
            Map<String, Object> beanMap = applicationContext.getBeansOfType(clazz);
            if (beanMap.size() == 1) {
                retObj = invokeMethod(clazz, beanMap.values().iterator().next(), url,
                        request.getRequestBody());
            } else {
                // 上下文同类型bean存在多个走泛化调用
                retObj = genericService
                        .$invoke(url.getMethod(), parameterTypes, convertArray(jsonArray));
            }
        } else {
            // 类加载器中不存在class直接泛化调用
            retObj = genericService
                    .$invoke(url.getMethod(), parameterTypes, convertArray(jsonArray));
        }
    }...
}
// 注册dubbo泛化调用消费者至spring上下文
private GenericService getDubboGenericService(CtURL url) {
    String group = url.getGroup();
    String version = url.getVersion();
    String beanName =
            StringUtils.firstNonBlank(url.getGroup(), "DEFAULT") + "/" + url.getService() + (
                    StringUtils.isNotBlank(version) ? "" : "/" + version);

    if (applicationContext.containsBean(beanName)) {
        return (GenericService) applicationContext.getBean(beanName);
    }
    synchronized (applicationContext) {
        if (applicationContext.containsBean(beanName)) {
            return (GenericService) applicationContext.getBean(beanName);
        }

        AbstractBeanDefinition definition = BeanDefinitionBuilder
                .genericBeanDefinition(ReferenceBean.class)
                .addPropertyValue("application", new ApplicationConfig("dock-common"))
                .addPropertyValue("registries", registryConfigs)
                .addPropertyValue("interface", url.getService())
                .addPropertyValue("group", group).addPropertyValue("version", version)
                .addPropertyValue("generic", "true").addPropertyValue("retries", 0)
                .addPropertyValue("timeout", 3000).getBeanDefinition();

        BeanFactory beanFactory = applicationContext.getAutowireCapableBeanFactory();
        BeanDefinitionRegistry registry = (BeanDefinitionRegistry) beanFactory;
        registry.registerBeanDefinition(beanName, definition);
        return (GenericService) beanFactory.getBean(beanName);
    }
}

泛化调用原理

dubbo消费者调用服务提供者过程不多说,消费者与提供者端invoker都会经历对应group的filter,泛化调用则由其中一个filter实现:com.alibaba.dubbo.rpc.filter.GenericImplFilter,将invocation方法设置为com.alibaba.dubbo.common.Constants#$INVOKE,提供者端则由com.alibaba.dubbo.rpc.filter.GenericFilter责任链节点识别泛化调用,消费者端GenericImplFilter源码如下

public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException {
    String generic = invoker.getUrl().getParameter(Constants.GENERIC_KEY);
    // 确定是泛化调用
    if (ProtocolUtils.isGeneric(generic)
            && !Constants.$INVOKE.equals(invocation.getMethodName())
            && invocation instanceof RpcInvocation) {
        RpcInvocation invocation2 = (RpcInvocation) invocation;
        String methodName = invocation2.getMethodName();
        Class<?>[] parameterTypes = invocation2.getParameterTypes();
        Object[] arguments = invocation2.getArguments();
        String[] types = new String[parameterTypes.length];
        for (int i = 0; i < parameterTypes.length; i++) {
            types[i] = ReflectUtils.getName(parameterTypes[i]);
        }
        Object[] args;
        // generic参数为bean,则以bean的方式序列化参数
        if (ProtocolUtils.isBeanGenericSerialization(generic)) {
            args = new Object[arguments.length];
            for (int i = 0; i < arguments.length; i++) {
                args[i] = JavaBeanSerializeUtil.serialize(arguments[i], JavaBeanAccessor.METHOD);
            }
        } else {
            // 否则使用工具类序列化
            args = PojoUtils.generalize(arguments);
        }
        invocation2.setMethodName(Constants.$INVOKE);
        invocation2.setParameterTypes(GENERIC_PARAMETER_TYPES);
        invocation2.setArguments(new Object[]{methodName, types, args});
        Result result = invoker.invoke(invocation2);
... 
}

dubbo对参数进行了序列化,继续看下工具类序列化代码,序列化其实是将Java类序列化为一个map对象,我们只需要查看primitive类型的部分即可,可以看到对于primitive类型,dubbo序列化工具类什么也没做直接返回原类型数据

private static Object generalize(Object pojo, Map<Object, Object> history) {
    ...
    if (ReflectUtils.isPrimitives(pojo.getClass())) {
        return pojo;
    }
...
}

小结

dubbo表示这锅我不背-_-!!!

泛化调用前参数转换问题?

网关侧参数转换代码

// 参数原JSON字符串来自JsonArray.get(0):[{\"cityId\":1,\"riderId\":5150213,\"riderType\":0,\"type\":95,\"cost\":0.01,\"tradeNoStr\":\"6899423540852876801618889049000\"}]
private Object convert(JsonElement element) {
    if (element == null || element.isJsonNull()) {
        return null;
    } else if (element.isJsonPrimitive()) {
        JsonPrimitive primitive = (JsonPrimitive) element;
        if (primitive.isNumber()) {
            // commons-lang3-3.8.1.jar
            // org.apache.commons.lang3.math.NumberUtils#createNumber
            return NumberUtils.createNumber(primitive.getAsString());
        } else if (primitive.isBoolean()) {
            return element.getAsBoolean();
        } else {
            return element.getAsString();
        }
    } else if (element.isJsonArray()) {
        return convertArray(element.getAsJsonArray());
    } else {
        JsonObject jsonObject = element.getAsJsonObject();
        Map<String, Object> map = new LinkedHashMap<>();
        jsonObject.entrySet().forEach(entry -> {
            map.put(entry.getKey(), convert(entry.getValue()));
        });
        return map;
    }
}

尝试复现问题

public static void main(String[] args) {
    Gson gson = new GsonBuilder().setDateFormat("yyyy-MM-dd HH:mm:ss,SSS").create();
    JsonArray je = gson.fromJson("[{\"cityId\":1,\"riderId\":5150213,\"riderType\":0,\"type\":95,\"cost\":0.01,\"tradeNoStr\":\"6899423540852876801618889049000\"}]", JsonArray.class);
    System.out.println("333="+je);
    Object[] objs = convertArray(je);
    for (Object data : objs) {
        System.out.println(data.getClass());
        if (data instanceof Map) {
            Map<?, ?> datas = (Map<?, ?>) data;
            datas.forEach((k, v) -> System.out
                .printf("k=%s,v=%s%n", k, v));
        }
    }
}

数据结果

333=[{"cityId":1,"riderId":5150213,"riderType":0,"type":95,"cost":0.01,"tradeNoStr":"6899423540852876801618889049000"}]
class java.util.LinkedHashMap
k=cityId,v=1
k=riderId,v=5150213
k=riderType,v=0
k=type,v=95
k=cost,v=0.01
k=tradeNoStr,v=6899423540852876801618889049000

???cost值没有问题,为毛到了服务端出现了问题?服务端有内鬼?查看服务端泛化调用责任链逻辑:com.alibaba.dubbo.rpc.filter.GenericFilter

public Result invoke(Invoker<?> invoker, Invocation inv) throws RpcException {
    if (inv.getMethodName().equals(Constants.$INVOKE)
            && inv.getArguments() != null
            && inv.getArguments().length == 3
            && !ProtocolUtils.isGeneric(invoker.getUrl().getParameter(Constants.GENERIC_KEY))) {
        String name = ((String) inv.getArguments()[0]).trim();
        String[] types = (String[]) inv.getArguments()[1];
        Object[] args = (Object[]) inv.getArguments()[2];
        try {
            Method method = ReflectUtils.findMethodByMethodSignature(invoker.getInterface(), name, types);
            Class<?>[] params = method.getParameterTypes();
            if (args == null) {
                args = new Object[params.length];
            }
            String generic = inv.getAttachment(Constants.GENERIC_KEY);
            if (StringUtils.isEmpty(generic)
                    || ProtocolUtils.isDefaultGenericSerialization(generic)) {
                // 反序列化对象
                args = PojoUtils.realize(args, params, method.getGenericParameterTypes());
            } ...
}

查看反序列化逻辑,我们客户端泛化调用时将入参序列化为一个map对象,此时服务端将其反序列化为接口的实际入参类型:com.alibaba.dubbo.common.utils.PojoUtils#realize0,源码可以看到通过反射调用目标方法参数类型的set方法或filed字段写入map的value值

private static Object realize0(Object pojo, Class<?> type, Type genericType, final Map<Object, Object> history) {
    ...
    if (pojo instanceof Map<?, ?> && type != null) {
        ...
        } else {
            Object dest = newInstance(type);
            history.put(pojo, dest);
            for (Map.Entry<Object, Object> entry : map.entrySet()) {
                Object key = entry.getKey();
                if (key instanceof String) {
                    String name = (String) key;
                    Object value = entry.getValue();
                    if (value != null) {
                        Method method = getSetterMethod(dest.getClass(), name, value.getClass());
                        Field field = getField(dest.getClass(), name);
                        if (method != null) {
                            if (!method.isAccessible())
                                method.setAccessible(true);
                            Type ptype = method.getGenericParameterTypes()[0];
                            value = realize0(value, method.getParameterTypes()[0], ptype, history);
                            try {
                                method.invoke(dest, value);
                            } catch (Exception e) {
                                e.printStackTrace();
                                throw new RuntimeException("Failed to set pojo " + dest.getClass().getSimpleName() + " property " + name
                                        + " value " + value + "(" + value.getClass() + "), cause: " + e.getMessage(), e);
                            }
                        } else if (field != null) {
                            value = realize0(value, field.getType(), field.getGenericType(), history);
                            try {
                                field.set(dest, value);
                            } catch (IllegalAccessException e) {
                                throw new RuntimeException("Failed to set filed " + name + " of pojo " + dest.getClass().getName() + " : " + e.getMessage(), e);
                            }
                        }
                    }
                }
            }
            ...
}

这里有一个内部类型转换Object-》BigDecimal,如果Map中的value值不是BigDecimal类型则可能会出现问题,回过头修改下复现代码查看org.apache.commons.lang3.math.NumberUtils#createNumber方法将字符“0.01”转换为Number类型的实际类型

public static void main(String[] args) {
    Gson gson = new GsonBuilder().setDateFormat("yyyy-MM-dd HH:mm:ss,SSS").create();
    JsonArray je = gson.fromJson("[{\"cityId\":1,\"riderId\":5150213,\"riderType\":0,\"type\":95,\"cost\":0.01,\"tradeNoStr\":\"6899423540852876801618889049000\"}]", JsonArray.class);
    System.out.println("333="+je);
    Object[] objs = convertArray(je);
    for (Object data : objs) {
        System.out.println(data.getClass());
        if (data instanceof Map) {
            Map<?, ?> datas = (Map<?, ?>) data;
            datas.forEach((k, v) -> System.out
                .printf("k=%s,v=%s,k.class=%s,v.class=%s%n", k, v, k.getClass(), v.getClass()));
        }
    }
}

输出结果

333=[{"cityId":1,"riderId":5150213,"riderType":0,"type":95,"cost":0.01,"tradeNoStr":"6899423540852876801618889049000"}]
class java.util.LinkedHashMap
k=cityId,v=1,k.class=class java.lang.String,v.class=class java.lang.Integer
k=riderId,v=5150213,k.class=class java.lang.String,v.class=class java.lang.Integer
k=riderType,v=0,k.class=class java.lang.String,v.class=class java.lang.Integer
k=type,v=95,k.class=class java.lang.String,v.class=class java.lang.Integer
k=cost,v=0.01,k.class=class java.lang.String,v.class=class java.lang.Float
k=tradeNoStr,v=6899423540852876801618889049000,k.class=class java.lang.String,v.class=class java.lang.String

小结

问题已经确认正是String类型转Number类型过程中对于小数类型,由于程序无法识别你的小数类型是哪种浮点类型,默认按照最小满足方式转换,0.01转换为Float类型,服务端实际字段类型为BigDecimal类型,Float至BigDecimal类型隐式转换出现了精度问题。验证代码如下

float f = 0.01f;
System.out.println(new BigDecimal(f));
// 输出结果
0.00999999977648258209228515625

总结

问题原因:字符串转浮点型导致,因为程序无法感知你是哪种浮点型,浮点型float转BigDecimal存在精度问题

问题解法:较为粗暴的将所有Number类型转为BigDecimal,Gson当然也早已考虑到这些场景咯,针对primitive类型提供了转换方法=com.google.gson.JsonPrimitive#getAsBigDecimal

 

———————————————— 版权声明:本文为CSDN博主「会灰翔的灰机」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。 原文链接:https://blog.csdn.net/u010597819/article/details/116095242

标签:dubbo,调用,泛化,Object,null,class,String
来源: https://www.cnblogs.com/Not-If/p/16336870.html