URL Rewrite and URLs with invalid characters

Topics: Troubleshooting
Apr 10, 2014 at 2:37 PM
I often use the URL rewrite module when deploying a website that is replacing an old website to ensure that all old website URLs redirect with 301 to the corresponding URL on the new website.

I had the situation today where a new website that I deployed was failing to redirect some of the URLs from the old website. The old website it appears contained + (plus) and & (ampersand) characters in some of its URLs and I'm running Orchard on IIS 7.5 which considers these characters in the URL to be invalid.

Therefore any of the old URLs (still in Google) that contained these characters would fail to redirect to the new pages. URLs with '+' result in a blank page. URLs containing '&' would throw a 501 error back to the client.

To allow '+' in the URL so that redirection would work I could add the allowDoubleEscaping attribute to to web.config as follows...
<system.webServer>
    <security>
        <requestFiltering allowDoubleEscaping="true" />
    </security>
</system.webServer>
And to allow the '&' in old URLs to be redirected I overrode the default value for requestPathInvalidCharacters in the httpRuntime parameter as follows...
<httpRuntime requestValidationMode="2.0" requestPathInvalidCharacters="&lt;,&gt;,*,%,:,\"/>
by default the requestPathInvalidCharacters attribute (when not specified at all) contains the ampersand character as follows...
requestPathInvalidCharacters="&lt;,&gt;,*,%,:,&amp;,\"
The following article was my reference for these changes...

http://www.hanselman.com/blog/ExperimentsInWackinessAllowingPercentsAnglebracketsAndOtherNaughtyThingsInTheASPNETIISRequestURL.aspx

I understand (from Scott's article) that this is potentially risky, but I really want to ensure that these old URLs are redirected nicely, at least in the short term. I'm thinking I will leave these enabled only for a limited period.

Would anyone with more in-depth knowledge care to comment further on this approach? Ie. Are there any obvious problems or security issues that this may open up specifically in relation to using this approach with Orchard?

On a side note, I find it interesting that Scott recommends against even using the requestValidationMode="2.0" attribute on the httpRuntime parameter. In the default Orchard web.config this appears to be in place. Is this something we should be looking at removing, or are there reasons why we still need this in place?