编程语言
首页 > 编程语言> > 【JavaWeb】HttpClient

【JavaWeb】HttpClient

作者:互联网

 

需要的依赖:

        <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.13</version>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>

 

参考视频:

https://www.bilibili.com/video/BV1W54y1s7BZ

 

1、原生JDK实现的网络请求

这个在狂神的爬虫上面有看到过原生的方式

当时还不明白这个技术其实就是后台的Ajax

    @Test
    public void quickStart() throws IOException {
        InputStream inputStream = null;
        InputStreamReader inputStreamReader = null;
        BufferedReader bufferedReader = null;
        try {

            // 原生JDK API 发送请求
            String urlString = "https://www.baidu.com/";
            URL url = new URL(urlString);
            URLConnection urlConnection = url.openConnection();
            HttpURLConnection httpURLConnection = (HttpURLConnection) urlConnection;

            httpURLConnection.setRequestMethod("GET");
            httpURLConnection.setRequestProperty("aaa", "123");

            inputStream = httpURLConnection.getInputStream();
            inputStreamReader = new InputStreamReader(inputStream, StandardCharsets.UTF_8);
            bufferedReader = new BufferedReader(inputStreamReader);

            String line = null;
            while (null != ( line = bufferedReader.readLine())) {
                System.out.println(line);
            }
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            bufferedReader.close();
            inputStreamReader.close();
            inputStream.close();
        }
    }

 

2、使用ApacheHttpClient发送请求:

    @Test
    public void useHttpClient() throws IOException {
        // 使用 HttpClient Get 无参请求实现

        CloseableHttpClient closeableHttpClient = null;
        CloseableHttpResponse closeableHttpResponse = null;
        HttpEntity httpEntity = null;
        try {
            // 创建一个可关闭的Http客户端对象
            closeableHttpClient = HttpClients.createDefault();

            // 要请求的地址
            String urlString = "https://www.baidu.com/";

            // GET参数需要进行URL编码处理
            String param1 = "orderId=21343000123324";
            String param2 = "orderRemark=大三大四1王企鹅1 哇多久啊是巴西 &%……¥%";
            param1 = URLEncoder.encode(param1, StandardCharsets.UTF_8.name());
            param2 = URLEncoder.encode(param2, StandardCharsets.UTF_8.name());

            // 根据地址创建一个Http请求对象 这里使用的是GET请求对象
            // HttpGet httpGet = new HttpGet(urlString);
            HttpGet httpGet = new HttpGet(urlString + "?" + param1 + "&" + param2);

            // 浏览器伪装信息
            httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36 Edg/94.0.992.38");

            // 防盗链设置 https://www.jianshu.com/p/0a1338db6cab
            httpGet.addHeader("Referer", "https://ntp.msn.cn/");

            // 客户端对象带着请求对象 执行请求发送, 返回响应对象
            closeableHttpResponse = closeableHttpClient.execute(httpGet);

            // 可以从响应对象获取对应的响应信息
            StatusLine statusLine = closeableHttpResponse.getStatusLine();
            System.out.println(statusLine); // HTTP/1.1 200 OK

            // 响应不成功状态直接结束后续逻辑
            if (HttpStatus.SC_OK != statusLine.getStatusCode()) return;

            ProtocolVersion protocolVersion = statusLine.getProtocolVersion(); // HTTP/1.1
            int major = protocolVersion.getMajor(); // 1 主版本协议号
            int minor = protocolVersion.getMinor(); // 1 附属小版本协议号
            String protocol = protocolVersion.getProtocol(); // HTTP

            int statusCode = statusLine.getStatusCode(); // 200
            String reasonPhrase = statusLine.getReasonPhrase(); // OK

            Header[] allHeaders = closeableHttpResponse.getAllHeaders();
            for (Header header : allHeaders) {
                System.out.println("Response Header -> " + header.getName()  + " : " + header.getValue());
            }
            
            // 从响应对象中获取响应实体对象
            httpEntity = closeableHttpResponse.getEntity();

            Header contentType = httpEntity.getContentType();
            String contentTypeName = contentType.getName(); // Content-Type
            String contentTypeValue = contentType.getValue(); // Content-Type: text/html;charset=utf-8

            // 这个响应头不常见
            // Header contentEncoding = httpEntity.getContentEncoding(); // null
            // String contentEncodingName = contentEncoding.getName();
            // String contentEncodingValue = contentEncoding.getValue();

            // 使用实体工具类转换成字符结果
            String httpEntityResult = EntityUtils.toString(httpEntity, StandardCharsets.UTF_8);
            // System.out.println(httpEntityResult);

        } catch (Exception exception) {
            exception.printStackTrace();
        } finally {
            // 最后调用此方法确保资源释放
            EntityUtils.consume(httpEntity);
            closeableHttpResponse.close();
            closeableHttpClient.close();
        }
    }

 

3、下载图片资源:

    @Test
    public void downloadWebPicture() throws Exception {

        // 请求发送
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        String resource = "https://img.zcool.cn/community/01088b5a052431a801204a0e253198.jpg@1280w_1l_2o_100sh.jpg";
        HttpGet httpGet = new HttpGet(resource);
        CloseableHttpResponse closeableHttpResponse = closeableHttpClient.execute(httpGet);
        HttpEntity httpEntity = closeableHttpResponse.getEntity();

        // 类型解析
        Header contentType = httpEntity.getContentType();
        String contentTypeValue = contentType.getValue(); // image/jpeg
        String fileTypeSuffix = contentTypeValue.split("/")[1]; // jpeg

        // 文件下载
        byte[] bytes = EntityUtils.toByteArray(httpEntity);
        String localPath = "C:\\Users\\Administrator\\Desktop\\test." + fileTypeSuffix;
        OutputStream outputStream = new FileOutputStream(localPath);
        outputStream.write(bytes);

        // 资源释放
        outputStream.close();
        EntityUtils.consume(httpEntity);
        closeableHttpResponse.close();
        closeableHttpClient.close();
    }

 

4、配置代理主机:

    @Test
    public void proxySettings() throws Exception {
        
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        String target = "https://www.baidu.com/";
        HttpGet httpGet = new HttpGet(target);

        // 代理主机的信息 http://www.66ip.cn/
        String ip = "176.121.1.81";
        int port = 8181;
        HttpHost httpHost = new HttpHost(ip, port);

        // 创建请求配置对象
        RequestConfig requestConfig = RequestConfig
                .custom()
                .setProxy(httpHost) // 设置代理主机的信息
                .build();

        // 设置请求配置
        httpGet.setConfig(requestConfig);

        CloseableHttpResponse closeableHttpResponse = closeableHttpClient.execute(httpGet);
        HttpEntity httpEntity = closeableHttpResponse.getEntity();

        Header[] allHeaders = closeableHttpResponse.getAllHeaders();
        for (Header header : allHeaders) {
            System.out.println("Response Header -> " + header.getName()  + " : " + header.getValue());
        }

        String httpEntityResult = EntityUtils.toString(httpEntity, StandardCharsets.UTF_8);
        System.out.println(httpEntityResult);

        // 资源释放
        EntityUtils.consume(httpEntity);
        closeableHttpResponse.close();
        closeableHttpClient.close();
    }

 

5、设置超时相关的配置:

    @Test
    public void proxySettings() throws Exception {
        // 请求发送
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        String target = "https://www.baidu.com/";
        HttpGet httpGet = new HttpGet(target);

        // 代理主机的信息 http://www.66ip.cn/
        String ip = "176.121.1.81";
        int port = 8181;
        HttpHost httpHost = new HttpHost(ip, port);

        // 创建请求配置对象
        RequestConfig requestConfig = RequestConfig
                .custom()
                .setProxy(httpHost)
                .setConnectTimeout(5000) // 设置连接超时的上限 TCP3次握手的时限
                .setSocketTimeout(3000) // 设置读取超时上限  从请求的网址中获取响应数据的间隔时限(因为并不是一次请求就完成了加载)
                .setConnectionRequestTimeout(3000) // 从HttpClient连接池中获取connection对象的时限
                .build();

        // 设置请求配置
        httpGet.setConfig(requestConfig);

        CloseableHttpResponse closeableHttpResponse = closeableHttpClient.execute(httpGet);
        HttpEntity httpEntity = closeableHttpResponse.getEntity();

        Header[] allHeaders = closeableHttpResponse.getAllHeaders();
        for (Header header : allHeaders) {
            System.out.println("Response Header -> " + header.getName()  + " : " + header.getValue());
        }

        String httpEntityResult = EntityUtils.toString(httpEntity, StandardCharsets.UTF_8);
        System.out.println(httpEntityResult);

        // 资源释放
        EntityUtils.consume(httpEntity);
        closeableHttpResponse.close();
        closeableHttpClient.close();
    }

 

6、MIME-TYPE 邮件扩展类型 与POST请求

mime-type 就是具体文件类型的前面的所属规范类型

在Tomcat里面配置的web.xml信息就可以看到所有的规范类型了

E:\apache-tomcat-8.5.70\conf\web.xml

片段:

    <mime-mapping>
        <extension>zirz</extension>
        <mime-type>application/vnd.zul</mime-type>
    </mime-mapping>
    <mime-mapping>
        <extension>zmm</extension>
        <mime-type>application/vnd.handheld-entertainment+xml</mime-type>
    </mime-mapping>

 

常见Content-type:

# 一般html表单提交 发送的类型
application/x-www-form-urlencoded

# html上传文件规范的类型
multipart/form-data

# 目前主流规范的类型
application/json

 

POST表单类型提交案例:

1、客户端请求代码

    @Test
    public void postWithFormType() throws Exception {
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        String target = "http://localhost:8080/mvc-framework/test/testMethod2";

        // 首先创建POST请求对象
        HttpPost httpPost = new HttpPost(target);


        // 设置表单需要提交的参数
        List<NameValuePair> nvpList = new ArrayList<>();
        nvpList.add(new BasicNameValuePair("username", "张三"));
        nvpList.add(new BasicNameValuePair("password", "w123e21"));

        // 表单类型实体对象 装填参数信息 application/x-www-form-urlencoded
        UrlEncodedFormEntity urlEncodedFormEntity = new UrlEncodedFormEntity(nvpList, Consts.UTF_8);

// 设置表单类型实体对象
        httpPost.setEntity(urlEncodedFormEntity);

        // 客户端执行请求发送
        CloseableHttpResponse closeableHttpResponse = closeableHttpClient.execute(httpPost);
        HttpEntity httpEntity = closeableHttpResponse.getEntity();
        System.out.println(EntityUtils.toString(httpEntity, StandardCharsets.UTF_8));

        EntityUtils.consume(httpEntity);
        closeableHttpResponse.close();
        closeableHttpClient.close();
    }

2、服务器处理代码:

    /**
     *
     *  POST请求测试
     *  http://localhost:8080/mvc-framework/test/testMethod2
     * @param request
     * @param response
     */
    // @RequestMethod(MethodType.POST)
    @RequestMapping(value = "/testMethod2", methodType = MethodType.POST)
    public void testMethod2(HttpServletRequest request, HttpServletResponse response) {
        System.out.println("testMethod2");

        ServletUtil.printAllParamByRequestMap(request);

        Map<String, Object> requestAttribute = (Map<String, Object>) request.getAttribute(ServletConstants.POST_PARAM_KEY);

        for (String s : requestAttribute.keySet()) {
            System.out.println("postParam " + s + ": " + requestAttribute.get(s));
        }
    }

服务这里没有对请求做出响应,就是打印看看没有收到参数信息:

doPost detected
doGet detected
testMethod2
username: [张三]
password: [w123e21]
postParam password: w123e21
postParam username: 张三

做了两次打印的原因是第一个打印是直接调用ServletAPI实现:

基本的POST表单请求Servlet有做识别处理

    public static void printAllParamByRequestMap(HttpServletRequest request) {
        Map<String, String[]> parameterMap = request.getParameterMap();
        for (String s : parameterMap.keySet()) {
            System.out.println(s + ": " + Arrays.toString(parameterMap.get(s)));
        }
    }

第二次打印是从输入流中获取识别的

    /**
     * 从POST请求中获取参数
     * @param request
     * @return
     * @throws Exception
     */
    public static Map<String, Object> getPostParam(HttpServletRequest request) throws Exception {
        // 返回参数
        Map<String, Object> params = new HashMap<>();

        // 获取内容格式
        String contentType = request.getContentType();

        if (null == contentType || "".equals(contentType)) throw new ServletException("没有设置请求头项Content-Type!!!");
        else contentType = contentType.split(";")[0];

        // form表单格式  表单形式可以从 ParameterMap中获取
        if (ServletConstants.CONTENT_TYPE_VALUE_URL_ENCODED2.equalsIgnoreCase(contentType)) {
            // 获取参数
            Map<String, String[]> parameterMap = request.getParameterMap();
            if (parameterMap != null) {
                for (Map.Entry<String, String[]> entry : parameterMap.entrySet()) {
                    params.put(entry.getKey(), entry.getValue()[0]);
                }
            }
        }

        // json格式 json格式需要从request的输入流中解析获取
        if (ServletConstants.CONTENT_TYPE_VALUE_JSON2.equalsIgnoreCase(contentType)) {
            // 使用 commons-io中 IOUtils 类快速获取输入流内容
            String paramJson = IOUtils.toString(request.getInputStream(), StandardCharsets.UTF_8);
            Map parseObject = JSON.parseObject(paramJson, Map.class);
            params.putAll(parseObject);
        }

        return params ;
    }

 

也可以不使用  UrlEncodedFormEntity  ,使用请求头设置即可

这个意思在视频里说错了,应该是这个Entity已经在Header这么处理了

        // 配置Http请求头
        httpPost.addHeader("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8");

因为表单的参数还是需要放到Entity里面发送过去的

 

这里特意看了下Entity的实现结构:

搜寻Entity相关的时候发现这个博客写的也很好,涉及到Cookie相关的操作

https://www.cnblogs.com/licl11092/p/9075677.html

 

POST + JSON类型案例:

SpringMVC接受Post JSON数据时要带上 @RequestBody给方法

普通Get参数则是@RequestParam

类注解为@RestController就可以不注解@RequestBody方法

在这里我使用的是封装的一套MVC,Servlet对JSON参数是不支持的,只能从输入流自行获取

 

这里JSON参数采用的是StringEntity实现存储,看了源码发现默认是text/plain类型,也允许构造器自行设置类型

    @Test
    public void postWithFormJson() throws Exception {
        CloseableHttpClient closeableHttpClient = HttpClients.createDefault();
        java.lang.String target = "http://localhost:8080/mvc-framework/test/testMethod2";

        // 首先创建POST请求对象
        HttpPost httpPost = new HttpPost(target);

        // 设置需要提交的JSON参数 这里做简单案例,就不去下载Fastjson来转换了,自己手写一个
        String jsonParam = "{ \"username\": \"张三\", \"password\": \"w123e21\" }";

// 表单类型实体对象 装填参数信息 application/x-www-form-urlencoded
        StringEntity jsonEntity = new StringEntity(jsonParam , Consts.UTF_8);

        // 配置Http请求头
        // httpPost.addHeader("Content-Type", "application/json; charset=UTF-8");
        jsonEntity.setContentType(new BasicHeader("Content-Type", "application/json; charset=UTF-8"));
        // StringEntity 设置编码
        jsonEntity.setContentEncoding(Consts.UTF_8.name());

        // 设置JSON实体对象
        httpPost.setEntity(jsonEntity);

        // 客户端执行请求发送
        CloseableHttpResponse closeableHttpResponse = closeableHttpClient.execute(httpPost);
        HttpEntity httpEntity = closeableHttpResponse.getEntity();
        System.out.println(EntityUtils.toString(httpEntity, StandardCharsets.UTF_8));

        EntityUtils.consume(httpEntity);
        closeableHttpResponse.close();
        closeableHttpClient.close();
    }

 

POST + 文件类型案例:

 

标签:httpEntity,String,closeableHttpClient,请求,new,HttpClient,closeableHttpResponse,Ja
来源: https://www.cnblogs.com/mindzone/p/15367993.html