- 2.3.3. Pooling connection manager
- 2.4. Multithreaded request execution
- 2.5. Connection eviction policy
- 2.6. Connection keep alive strategy
- 2.7.1. Secure socket layering
- 2.7.4. Hostname verification
- 2.8. HttpClient proxy configuration
- Managing a pool of client connections and is able to service connection requests from multiple execution threads.
- Connections are pooled on a per route basis.
- A request for a route for which the manager already has a persistent connection available in the pool will be serviced by leasing a connection from the pool rather than creating a brand new connection.
HttpClient
can be used to execute multiple requests simultaneously using multiple threads of execution.- If all connections for a given route have already been leased, a request for a connection will block until a connection is released back to the pool. One can ensure the connection manager does not block indefinitely in the connection request operation by setting
http.conn-manager.timeout
to a positive value. - While
HttpClient
instances are thread safe and can be shared between multiple threads of execution, it is highly recommended that each thread maintains its own dedicated instance ofHttpContext
.
- If the connection gets closed on the server side, the client side connection is unable to detect the change in the connection state (and react appropriately by closing the socket on its end).
HttpClient
tries to mitigate the problem by testing whether the connection is stale, that is no longer valid because it was closed on the server side, prior to using the connection for executing an HTTP request.- The stale connection check is not 100% reliable.
- The only feasible solution that does not involve a one thread per socket model for idle connections, but is a dedicated monitor thread used to evict connections that are considered expired due to a long period of inactivity.
- The monitor thread can periodically call
ClientConnectionManager#closeExpiredConnections()
method to close all expired connections and evict closed connections from the pool. - It can also optionally call
ClientConnectionManager#closeIdleConnections()
method to close all connections that have been idle over a given period of time.
public static class IdleConnectionMonitorThread extends Thread {
private final HttpClientConnectionManager connMgr;
private volatile boolean shutdown;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
super();
this.connMgr = connMgr;
}
@Override
public void run() {
try {
while (!shutdown) {
synchronized (this) {
wait(5000);
// Close expired connections
connMgr.closeExpiredConnections();
// Optionally, close connections
// that have been idle longer than 30 sec
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
} catch (InterruptedException ex) {
// terminate
}
}
public void shutdown() {
shutdown = true;
synchronized (this) {
notifyAll();
}
}
}
- If the Keep-Alive header is not present in the response,
HttpClient
assumes the connection can be kept alive indefinitely. - However, many HTTP servers in general use are configured to drop persistent connections after a certain period of inactivity in order to conserve system resources, quite often without informing the client.
- In case the default strategy turns out to be too optimistic, one may want to provide a custom keep-alive strategy.
ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
// Honor 'keep-alive' header
HeaderElementIterator it = new BasicHeaderElementIterator(
response.headerIterator(HTTP.CONN_KEEP_ALIVE));
while (it.hasNext()) {
HeaderElement he = it.nextElement();
String param = he.getName();
String value = he.getValue();
if (value != null && param.equalsIgnoreCase("timeout")) {
try {
return Long.parseLong(value) * 1000;
} catch(NumberFormatException ignore) {
}
}
}
HttpHost target = (HttpHost) context.getAttribute(
HttpClientContext.HTTP_TARGET_HOST);
if ("www.naughty-server.com".equalsIgnoreCase(target.getHostName())) {
// Keep alive for 5 seconds only
return 5 * 1000;
} else {
// otherwise keep alive for 30 seconds
return 30 * 1000;
}
}
};
CloseableHttpClient client = HttpClients.custom()
.setKeepAliveStrategy(myStrategy)
.build();
HttpClient
ships withSSLSocketFactory
that implements SSL/TLS layering.- Please note
HttpClient
does not use any custom encryption functionality. - It is fully reliant on standard Java Cryptography (JCE) and Secure Sockets (JSEE) extensions.
-
Important: hostname verification should not be confused with SSL trust verification.
- DefaultHostnameVerifier: The default implementation used by
HttpClient
is expected to be compliant with RFC 2818. The hostname must match any of alternative names specified by the certificate, or in case no alternative names are given the most specific CN of the certificate subject. A wildcard can occur in the CN, and in any of the subject-alts. - NoopHostnameVerifier: This hostname verifier essentially turns hostname verification off. It accepts any SSL session as valid and matching the target host.
- DefaultHostnameVerifier: The default implementation used by
SSLContext sslContext = SSLContexts.createSystemDefault();
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
sslContext,
NoopHostnameVerifier.INSTANCE);
- As of version 4.4
HttpClient
uses the public suffix list kindly maintained by Mozilla Foundation to make sure that wildcards in SSL certificates cannot be misused to apply to multiple domains with a common top-level domain. HttpClient
ships with a copy of the list retrieved at the time of the release.- The latest revision of the list can found at https://publicsuffix.org/list/.
- It is highly adviseable to make a local copy of the list and download the list no more than once per day from its original location.
- Even though
HttpClient
is aware of complex routing schemes and proxy chaining, it supports only simple direct or one hop proxy connections out of the box. - The simplest way to tell
HttpClient
to connect to the target host via a proxy is by setting the default proxy parameter:
HttpHost proxy = new HttpHost("someproxy", 8080);
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();
// One can also instruct HttpClient to use the standard JRE proxy selector to obtain proxy information:
SystemDefaultRoutePlanner routePlanner = new SystemDefaultRoutePlanner(
ProxySelector.getDefault());
CloseableHttpClient httpclient = HttpClients.custom()
.setRoutePlanner(routePlanner)
.build();
- proxy request example:
/* https://stackoverflow.com/questions/44821561/how-to-set-proxy-host-on-httpclient-request-in-java */
HttpClient client = new HttpClient();
HostConfiguration config = client.getHostConfiguration();
config.setProxy("someProxyURL", "someProxyPort");
Credentials credentials = new UsernamePasswordCredentials("username", "password");
AuthScope authScope = new AuthScope("someProxyURL", "someProxyPort");
client.getState().setProxyCredentials(authScope, credentials);
EntityEnclosingMethod method = new PostMethod(url);
method.setRequestEntity(new StringRequestEntity(requestXML, "text/xml", "utf-8"));