Faceted Search Intermittent Error - How We Found and Fixed It!
Picture the scene; Faceted Search is giving us an intermittent failure in our integration environment. Search is fine then all of a sudden upon a new search we get a page saying;
"An unexpected error has occurred.
Web Parts Maintenance Page: If you have permission, you can use this page to temporarily close Web Parts or remove personal settings. For more information, contact your site administrator."
At first this was puzzling. We figured there might be an environment problem stopping the facet parts querying the database or some such.
As we have several web front ends, someone had the idea of editing the masterpages to display which WFE was serving the request. This would allow us to rule out that the intermittent failure was caused by a single machine having problems. So, I added some code like this <%Response.Write("SERVERNAME");%> to simple.master and our custom master page.
We then discovered that Faceted Search only failed on one of the WFE machines. Closer inspection showed that the event log on that machine had a recurrent error like this;
A configuration failure occurred while creating policy ‘Faceted Search Policy’.
The exception that occurred was: System.Configuration.ConfigurationErrorsException: The entry 'Logging Handler' has already been added. (C:\Inetpub\wwwroot\wss\VirtualDirectories\devsearch.mydomain.com\web.config line 75)
Looking at the web.config I immediately found a duplicate entry for Logging Handler which differed from the original entry (that was still there). The extra entry was this;
<add logCategory="Information" eventId="100" severity="Error" title="Faceted Search Exception Handling" formatterType="Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.TextExceptionFormatter, Microsoft.Practices.EnterpriseLibrary.ExceptionHandling, Version=4.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" priority="0" useDefaultLogger="false" type="Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging.LoggingExceptionHandler, Microsoft.Practices.EnterpriseLibrary.ExceptionHandling.Logging, Version=4.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" name="Logging Handler" />
Now, this is puzzling as this wasn't added by anyone on the team, and must have been added by some process.
We have another strange issue where web.config modifications are occurring when we push out certain solutions. Perhaps that had something to do with this. When I get to the bottom of it I guess I can judge whether the two issues are related.
Anyway, Faceted Search now works without any intermittent errors, which is nice.