JAVA—HTTP客户端警告:Going to buffer response body of large or unknown size.
作者:互联网
关注微信公众号:CodingTechWork,一起学习进步。
原编码问题
在修复原有代码bug时,发现日志里经常抛出Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended
,这种提示。查看了原有代码中的逻辑如下:
... ...
HttpClient httpclient = new HttpClient();
GetMethod getMethod = new GetMethod(url);
int statusCode = httpclient.executeMethod(getMethod);
String respContent = getMethod.getResponseBodyAsString();
... ...
原因就在于使用了getResponseBodyAsString()
方法
源码分析
getResponseBodyAsString()源码
//getResponseBodyAsString()方法源码
public String getResponseBodyAsString() throws IOException {
byte[] rawdata = null;
if (this.responseAvailable()) {
//调用了getResponseBody(),容易消耗内存
rawdata = this.getResponseBody();
}
return rawdata != null ? EncodingUtil.getString(rawdata, this.getResponseCharSet()) : null;
}
//responseAvailable()方法源码
private boolean responseAvailable() {
return this.responseBody != null || this.responseStream != null;
}
//getResponseBody()方法源码
public byte[] getResponseBody() throws IOException {
if (this.responseBody == null) {
InputStream instream = this.getResponseBodyAsStream();
if (instream != null) {
long contentLength = this.getResponseContentLength();
if (contentLength > 2147483647L) {
throw new IOException("Content too large to be buffered: " + contentLength + " bytes");
}
int limit = this.getParams().getIntParameter("http.method.response.buffer.warnlimit", 1048576);
if (contentLength == -1L || contentLength > (long)limit) {
//这里是warn的原文
LOG.warn("Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.");
}
LOG.debug("Buffering response body");
ByteArrayOutputStream outstream = new ByteArrayOutputStream(contentLength > 0L ? (int)contentLength : 4096);
byte[] buffer = new byte[4096];
int len;
while((len = instream.read(buffer)) > 0) {
outstream.write(buffer, 0, len);
}
outstream.close();
this.setResponseStream((InputStream)null);
this.responseBody = outstream.toByteArray();
}
}
return this.responseBody;
}
从源码中可以看出,warn的条件是(contentLength == -1L || contentLength > (long)limit)
,如果http头没有指定contentLength
或大于上限值(默认1M),就会抛异常。其实,如果返回的结果比较确定,对程序没有太大影响。而对于返回结果不确定时,源码也建议我们使用下面的getResponseBodyAsStream()
方法。
getResponseBodyAsStream()源码
public InputStream getResponseBodyAsStream() throws IOException {
if (this.responseStream != null) {
return this.responseStream;
} else if (this.responseBody != null) {
InputStream byteResponseStream = new ByteArrayInputStream(this.responseBody);
LOG.debug("re-creating response stream from byte array");
return byteResponseStream;
} else {
return null;
}
}
从源码中可以看出,getResponseBodyAsStream()
内部没有使用getResponseBody()
方法,避免了内存耗尽问题,而是使用了InputStream
流方式处理。
优化原编码
... ...
HttpClient httpclient = new HttpClient();
GetMethod getMethod = new GetMethod(url);
int statusCode = httpclient.executeMethod(getMethod);
//String respContent = getMethod.getResponseBodyAsString();
//使用getResponseBodyAsStream()
InputStream inputStream = getMethod.getResponseBodyAsStream();
BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
StringBuffer stringBuffer = new StringBuffer();
String str = "";
while ((str = br.readLine()) != null) {
stringBuffer.append(str);
}
LOGGER.info("respContent: {}", stringBuffer.toString());
... ...
测了一把,哦豁,不再报这个异常了。完美。
标签:body,...,HTTP,buffer,contentLength,getResponseBodyAsStream,源码,new,null 来源: https://www.cnblogs.com/Andya/p/14493614.html