首页 > 其他分享> > 爬取微信公众号

爬取微信公众号

2022-08-03 15:33:17 作者：互联网

技术准备

HttpClient

Java比较常用的发起请求的工具，功能有：

方便的发起get、post等请求
可以设置连接池（类似线程池），使用池化思想降低频繁创建连接的开销
可以自己编写代码设置多线程爬取
...

代码模板

 public static void main(String[] args) throws Exception {
        CloseableHttpClient httpClient = HttpClients.createDefault();
        URIBuilder uriBuilder = new URIBuilder("https://mp.weixin.qq.com/cgi-bin/bizlogin");
        HttpPost httpPost = new HttpPost(uriBuilder.build());

        httpPost.addHeader("User-Agent","Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36");

        List<NameValuePair> params = new ArrayList<>();
        params.add(new BasicNameValuePair("action","startlogin"));

        UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(params,"utf8");
        httpPost.setEntity(formEntity);

        CloseableHttpResponse response = null;
        try {
            response = httpClient.execute(httpPost);
            if(response.getStatusLine().getStatusCode() == 200){
                String content = EntityUtils.toString(response.getEntity(),"utf8");
                System.out.println(content);
            }
        }catch (IOException e){
            e.printStackTrace();
        }finally {
            try {
                response.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
            try {
                httpClient.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

以上是post请求的代码模板，get等其他请求也差不太多，将HttpPost换成HttpGet就行了