HttpClient爬虫登录类模板
作者:互联网
因为最近编写需要登录类的爬虫比较多,所以想封装一个使用的模板,方便以后编写
继承结构
interface mypachong
----abstract mypachongimpl
介绍
mypachong接口,描述了一个爬虫程序需要一个login的方法实现登录操作,并封装get,post请求(未将方法设置未static是考虑到后续多用户爬取时,其实现类可以封装成员变量描述,不同用户信息)
public interface mypachong {
boolean login();//登录
HttpResponse get(String url,Map<String,String> map) ;//get
HttpResponse post(String url,Map<String,String> map);//post
}
mypachong的抽象子类,封装默认的post和get.后续使用只需继承并实现login,即可完成浏览网站的基本操作(post,get),防止出现代码繁多
public abstract class mypachongimpl implements mypachong {
public CookieStore cookieStore = null;
public HttpResponse get(String url, Map<String, String> map) {
//未登录时跑去执行login
while(cookieStore == null)
login();
//根据登录获取的cookieStore创建Htttpclient
HttpClient httpclient = HttpClientBuilder.create().setDefaultCookieStore(cookieStore).build();
String geturl = url;
//添加参数
if (map != null)
for (Map.Entry<String, String> entry : map.entrySet()) {
geturl += "&" + entry.getKey() + "=" + entry.getValue();
}
HttpGet get = new HttpGet(geturl);
HttpResponse response = null;
try {
response = httpclient.execute(get);
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return response;
}
public HttpResponse post(String url, Map<String, String> map) {
//未登录时跑去执行login
while(cookieStore == null)
login();
//根据登录获取的cookieStore创建Htttpclient
HttpClient httpclient = HttpClientBuilder.create().setDefaultCookieStore(cookieStore).build();
String posturl = url;
HttpPost post = new HttpPost(posturl);
//添加参数
if (map != null) {
List<NameValuePair> formlist = new ArrayList<NameValuePair>();
for (Map.Entry<String, String> entry : map.entrySet()) {
formlist.add(new BasicNameValuePair(entry.getKey(), entry.getValue()));
}
try {
UrlEncodedFormEntity urlEncodedFormEntity = new UrlEncodedFormEntity(formlist, "UTF-8");
post.setEntity(urlEncodedFormEntity);
} catch (UnsupportedEncodingException e1) {
e1.printStackTrace();
}
}
HttpResponse response = null;
try {
response = httpclient.execute(post);
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return response;
}
}
标签:map,get,爬虫,cookieStore,post,null,login,模板,HttpClient 来源: https://blog.csdn.net/qq_47503001/article/details/120191146