Tech Blogs

perspectives for IT decision makers

Introduction to HttpURLConnection – HTTP Client for Performing Efficient Network Operations

by (November 21, 2014)

HttlurlConnector from RapidValue

HttpURLConnection is a HTTP client which helps perform efficient network operations. This article reveals the features and various advantages of HttpURLConnection over other HTTP clients. Also, we delve deeper and provide meticulous detail on how to implement the response caching mechanism in Android, using the HttpResponseCache library, available in HttpURLConnection.

Using HttpURLConnection as REST Client & HTTP response caching, Android mainly provides two HTTP clients to perform operations related to network. They are:

  1. Apache HTTP client
  2. HttpUrlConnection

Apache HTTP client

  • Large and extensive APIs
  • Supports cookie handling, authentication and connection management
  • More suitable for web browser and other web applications, because of its large size
  • Does not support HttpResponseCache mechanism, hence leading to increased network usage and battery consumption

HttpURLConnection

HttpUrlConnection(java.net.HttpUrlConnection) is a light weight HTTP client, used to access network resources, and hence, suitable for mobile applications. Google also recommends the usage of HttpUrlConnection, instead of the other HTTP clients, available in standard Java and Apache libraries.

As per the Google blog:

“Apache HTTP client has fewer bugs on Eclair and Froyo. It is the best choice for these releases. For Gingerbread , HttpURLConnection is the best choice. Its simple API and small size makes it great fit for Android.

Transparent compression and response caching reduce network use, improve speed and save battery. New applications should use HttpURLConnection; it is where we will be spending our energy going forward.”

1. Cookie Storage and Management

a) Managing the Cookies

To manage the cookies, use java.net.CookieManager. The CookieManager class is an implementation of java.net.CookieHandler. The CookieHandler contains different ways to manage the cookies. CookieManager separates the storage of cookies, from the policy, which decides either to accept the cookies or to deny them. The constructor has two arguments: a CookieStore and a CookiePolicy. The former is in charge of cookie storage and the latter makes decision on acceptance/rejection.

Below is an example on how to set the cookie manager.

CookieManager cmrCookieMan = new CookieManager(cookieStore,
CookiePolicy.ACCEPT_ALL); // CookieStore object and the Policy to accept the cookies as parameters.
CookieHandler.setDefault(cmrCookieMan); // Set this CookieManager as the default CookieHandler.

Note: If CookieStore object is not specified when the CookieManager is created, a CookieManager instance will use an internal in-memory implementation. This implementation is not considered to be persistent and it only survives till the lifetime of the Java Virtual Machine.

b) Persistence of Cookies

Cookies can be persisted by implementing the abstract class org.apache.http.client.CookieStore.

As of Android API level 9, the CookieStore has the method: add (URI uri, HttpCookiecookie)

As per the documentation:

It saves a HTTP cookie to this store. This is the case for every incoming HTTP response. A cookie may or may not have an associated URI. If it does not have one, then the cookie’s domain and path attribute will show cradle land.

If there is an associated URI, and no domain, and path attribute is specified for the cookie, the given URI will indicate from where the cookie has come. If a cookie, which corresponds to the given URI, already exists, then it is replaced with the new one.

In Android, you could use the SQLite to create a database to store the cookies. All you need to do is add a database insert operation, when overriding the above method.

2. HTTP Response caching

As of Android 4.0(Ice Cream Sandwich), response caching is supported via the android.net.http.HttpResponseCache.http.HttpResponseCache library. The HttpResponseCache library provides transparent and automatic caching of HTTP and HTTPS requests that use the java.net.HttpUrlConnection classes.

For supporting the android versions below 4.0, you could use the third party library com.integralblue.httpresponsecache.HttpResponseCache.

How to Enable HttpResponseCache?

Enabling HttpResponseCache is pretty straight forward. You need to create a folder in the cache directory to store the cache files of each request.

Find below the code for enabling cache, using both the platform HttpResponseCache library, as well as the third party HttpResponseCache library, for previous versions.

try {
      Object existingResponseCache = Class.forName("android.net.http.HttpResponseCache").getMethod("getInstalled").invoke(null);
if(null == existingResponseCache){
long httpCacheSize = 10 * 1024 * 1024; // 10 MiB
            File httpCacheDir = new File(dir, "http");
            Class.forName("android.net.http.HttpResponseCache")
            .getMethod("install", File.class, long.class)
            .invoke(null, httpCacheDir, httpCacheSize);
       }
} catch (Exception httpResponseCacheNotAvailable) {
      // For versions before Android 4.0, the exception will be thrown.
try{
            com.integralblue.httpresponsecache.HttpResponseCache existingResponseCache = com.integralblue.httpresponsecache.HttpResponseCache.getInstalled();
            if(null == existingResponseCache){
                  long cacheSize = 10 * 1024 * 1024; // 10 MiB
                  File httpCacheDir = new File(dir, "http");
com.integralblue.httpresponsecache.HttpResponseCache.install(httpCacheDir,cacheSize);
            }
      }catch(Exception e){
      }
 }

Clear existing HttpResponseCache :

HttpResponseCache provides a delete method in order to clear the stored cache.

try {
      Object existingResponseCache = Class.forName("android.net.http.HttpResponseCache")
                  .getMethod("getInstalled").invoke(null);
      if(null != existingResponseCache){
                   Class.forName("android.net.http.HttpResponseCache")
                        .getMethod("delete").invoke(existingResponseCache);
 }
 } catch (Exception httpResponseCacheNotAvailable) {
      try{
                  com.integralblue.httpresponsecache.HttpResponseCache existingResponseCache  =  com.integralblue.httpresponsecache.HttpResponseCache.getInstalled();
            if(null != existingResponseCache){
                        existingResponseCache.delete();
            }
      }catch(Exception e){
      }
}

Get CacheResponse of a URL from cache :

You could fetch the cached response of a particular URL from the cache using the “get” method of HttpResponseCache.  You need to pass the URI of a URL, the request method (“GET”,”PUT” etc.) and also the request headers. With the above three parameters, the httpResponseCache matches and returns a CacheResponse object.  A CacheResponse object provides an InputStream to access the response body and a Map for the response headers.

Below is the code to fetch the CacheResponse of a GET request.

CacheResponse cache = null;
URI uri = new URI(url);
try {
      Object existingResponseCache = Class.forName("android.net.http.HttpResponseCache")
                  .getMethod("getInstalled").invoke(null);
      if(null != existingResponseCache){
            cache = (CacheResponse) Class.forName("android.net.http.HttpResponseCache")
            .getMethod("get",URI.class, String.class, Map.class)
      .invoke(existingResponseCache,uri,”GET”, httpUrlConnection.getRequestProperties());
       }
} catch (Exception httpResponseCacheNotAvailable) {
      com.integralblue.httpresponsecache.HttpResponseCache existingResponseCache = null;
      try{
            existingResponseCache  =  com.integralblue.httpresponsecache.HttpResponseCache.getInstalled();
            if(null != existingResponseCache){
cache = existingResponseCache.get(uri, ”GET”, httpUrlConnection.getRequestProperties());
            }
      }catch(Exception e){
      }
}

usesCache flag in HttpUrlConnection :

By default, for an HttpUrlConnection, the usesCache flag is set to be true. i.e., caching of the responses is enabled. If you do not need to cache a particular HTTP request, even if you have an HttpResponseCache enabled, then set the usesCache to false with the method setUseCaches (boolean newValue).

As per the documentation for setUseCaches method :

It sets the flag, indicating whether this connection allows you to use caches or not. This method can only be called prior to establishing the connection.

Advantages of HttpUrlConnection in Mobile Environment

  • Light weight apis helps in easier management and reduces compatibility issues.
  • Automatic handling of the caching mechanisms, with the help of HttpResponseCache.
  • Reduces the network usage and also, the battery consumption.

NOTE : RestClient java class is available here.

Below are Certain Scenarios we came across during Implementation:   

Scenario 1:

Send an HTTP request to server. If the server returns response code HTTP_NOT_MODIFIED(304), the HttpUrlConnection will fetch the response for the same URL in the cache, and return a HTTP_OK(200) and the response.

Problem: To identify whether the response is from cache or server.

Solution:

If the response is fetched from the cache, then the inputStream will be libcore.net.http.HttpResponseCache or com.android.okhttp.HttpResponseCache.

Below is the code to find the source of the response.

httpURLConnection.connect();
inputStream = httpURLConnection.getInputStream();
String classname = inputStream.getClass().toString();
if(classname.contains("com.android.okhttp.HttpResponseCache") || classname.contains("libcore.net.http.HttpResponseCache")){
      isFromCache = true;
}

Scenario 2:

For a particular api request, the server returns Last-Modified header in the response header. When the same api is requested, Last-Modified header value present in cache, will be added as the If-Modified-Since header by HttpUrlConnection. HttpUrlConnection converts the Last-Modified header value to GMT format. Last-Modified header, sent from the server, was in the format “yyyy-MM-dd’T’HH:mm:ss.SSSZ” and the server would only accept If-Modified-Since headers in the same format.

Problem: To send the If-Modified-Since header as per the server’s format.

Solution

Fetch CacheResponse, for the URL from cache, and find the Last-Modified header.  Add the Last-Modified header value as the If-Modified-Since request header for the new request.

Find below the code for the same.

CacheResponse cache =null;
URI uri = new URI(url);
HttpURLConnection httpURLConnection  = (HttpURLConnection) url.openConnection();
httpURLConnection.setRequestMethod("GET");
Object existingResponseCache = Class.forName("android.net.http.HttpResponseCache")
                  .getMethod("getInstalled").invoke(null);
if(null != existingResponseCache){
                cache = (CacheResponse) Class.forName("android.net.http.HttpResponseCache")
                        .getMethod("get",URI.class, String.class, Map.class)
      .invoke(existingResponseCache,uri,"GET",
httpURLConnection.getRequestProperties());
}
if(null != cache){
Map<String,List<String>> cachedResponseHeaders = cache.getHeaders();
      if(null != cachedResponseHeaders){
            if(cachedResponseHeaders.containsKey("Last-Modified")){
List<String> lastModifiedList = cachedResponseHeaders.get("Last-Modified");
                  if(null != lastModifiedList && !lastModifiedList.isEmpty()){
                        String lastModified = lastModifiedList.get(0);
                        if(null != lastModified){
httpURLConnection.addRequestProperty("If-Modified-Since", lastModified);
                        }
}
            }
      }
}

Coding Source:

  1. http://developer.android.com/reference/android/net/http/HttpResponseCache.html
  2. https://github.com/candrews/HttpResponseCache

By,

Deepak Unnikrishnan,

Sr. Software Engineer, RapidValue Solutions

ONE COMMENT

  1. INOC | Network Operations Center /  April 22, 2015 AT 7:25 am / Reply

    This is a great introduction for using HTTP client for network operations. It may seem pretty daunting at first, but once you know all the necessary codes that should be added to the HttpURLConnection, everything becomes easy. Thanks for sharing, now everyone will know how properly use the program easily.

LEAVE A COMMENT

24/7 Toll Free(877)-643-1850(US)

Would you like to know more about us?

  • This field is for validation purposes and should be left unchanged.
Scroll Top