Internal NullReferenceException in HttpWebRequest when using a CachePolicy

Setting the CachePolicy to the default HttpRequestCacheLevel causes an exception internal to HttpWebRequest when a web application is running with an application pool configured to use a custom identity. For example:

HttpWebRequest request = …;  
request.CachePolicy =   
  new System.Net.Cache.HttpRequestCachePolicy (  
    System.Net.Cache.HttpRequestCacheLevel.Default);  
request.GetResponse ();

GetResponse () throws a WebException: “The request was aborted: The request was canceled.” Inner exception shows that the private method CheckCacheUpdateOnResponse () of System.Net.HttpWebRequest encounters a NullReferenceException.

A “workaround” would involve loading the user’s profile. For example, if the application pool custom identity is DOMAIN\MyWebApp, log in to the server as MyWebApp. The error goes away – until the profile is unloaded.

This problem does not occur when the web application is using the default application pool identity, NetworkService, because I believe that profile is already loaded by default.

Another bug report is somewhat similar to this but only manifests after a few thousand requests. These are probably symptoms of a much larger bug (or design flaw – http request caching is currently implemented through IE’s cache).

Update: Microsoft has reproduced this bug and will be addressing it in a future release.