Another day, another issue – this time with an on-premises environment with a ridiculously complicated architecture and an absolute ton of pipelines.
There are plenty of ways to fix this particular issue (because there are plenty of reasons), and after ferocious googling (and very brief binging just to see if Microsoft had found something Google hadn’t), the trail ran cold and I felt like nobody had run into the same issue before. This is probably not true – just that the documentation wasn’t there.
So, once again, let’s fix the documentation for one particular, very specific issue, that’s currently not properly covered!
This particular case was a result of a site being hosted as a virtual application (or “virtual directory”) under another site in IIS – but failing to start. The only error message would be the following:
The error – for me – happened with any request that should’ve been served by the virtual application. Something would apparently route them to another application pool, and that would cause a 403.18 – Forbidden error message to be thrown.
You can observe the whole error message by expanding the area below.HTTP Error 403.18 - Forbidden
HTTP Error 403.18 - Forbidden The specified request cannot be processed in the application pool that is configured for this resource on the Web server. Most likely causes: An ISAPI filter or custom module changed the URL to run in a different application pool than the original URL. An ISAPI extension (or custom module) used ExecuteURL (or ExecuteRequest) to run in a different application pool than the original URL. You have a custom error page that is located in one application pool but is referenced by a Web site in another application pool. When the URL is processed, it is determined by IIS that that it should have been processed in the first application pool, not the other pool. The Web site has multiple applications configured. The application this request is configured to run in is set to run in an application pool that does not exist. Things you can try: If you have an application that is trying to process a URL in another application pool (such as trying to process a custom error), ensure that they both run in the same application pool if appropriate. If you are trying to process a custom error URL that is located in another application pool, enable the custom errors Redirect feature. Verify that the application pool for the application exists. Create a tracing rule to track failed requests for this HTTP status code and see if ExecuteURL is being called. For more information about creating a tracing rule for failed requests, click here. Detailed Error Information: Module IIS Web Core Notification BeginRequest Handler StaticFile Error Code 0x00000000 Requested URL https://[URL]:443/ Physical Path C:\inetpub\[path] Logon Method Not yet determined Logon User Not yet determined More Information: This error occurs if the application pool for the request does not exist, or if an ISAPI filter, ISAPI extension or HTTP module calls the ExecuteURL server support function (or ExecuteRequest) with a URL that is configured in a different application pool. Due to security reasons, a Web site in one application pool cannot make ExecuteURL requests against a URL in another application pool. If you have an application that is trying to process a URL in another application pool, ensure that they both run in the same application pool if appropriate. View more information »
But what causes this?
It turns out that Url Rewrite rules were being inherited from
I disabled the rule – and during the next deployment, it was reactivated and everything broke again.
The weird part was that nothing in our pipeline had even the slightest hint of rewrite rules being applied. Nothing in the logs either. What’s creating these rules?
Then it hit me. Oh, how simple it was – and completely our fault! 😂
The reason? The web.config of the “parent” site – the one that’s deployed to the IIS site directly, not a virtual application – trickles down to the virtual application. And somehow, both updates to the parent and “child” would re-enable the routes from the parent. And that parent web.config had some unfortunate routes set up for a React app hosted there…
But this is luckily easy to fix!
It’s worth mentioning extremely briefly, that if you can change the virtual directories to run under the same application pool as the parent site, that’s an easy way to fix the issue as well. In our configuration, that was not allowed – so we needed to find another fix.
That particular fix is described below!
Time needed: 1 hour.
How to make sure URL Rewrite isn’t breaking your website hosted in a virtual application in IIS?
- Fire up Internet Information Services (IIS) Manager
You could probably do this in PowerShell as well, but I’m not going to google how to do that since it’s easy with IIS Manager.
- Select URL Rewrite section
Find your website (blurred but highlighted in the picture below) and select the URL Rewrite option from there.
- Compare the URL Rewrite configuration between parent and virtual application
For us, the parent had rules that were inherited by the virtual application and then caused the request to be routed outside the virtual directory – that’s not safe, and causes the exception to be thrown.
To be more precise, it was caused by the web.config changes React requires for routing.
- Disable the URL Rewrite option in the virtual application
No worries, this is temporary.
- Now re-try accessing the site
If it works now, we know the reason!
Disabling the rule might work for a while, but at least for us, the next deployment (to the virtual application or the parent site) would break it. So we had to apply an actual fix to the deployment process as well!
- Fix the web.config for the parent site
Yes, you read that right – we need to modify the parent site’s web.config instead of the virtual application’s.
For me, it’s a version-controlled file, so I’m going through the whole fork-edit-commit-PR-merge-deploy -cycle. And here’s the change required:
Wrap your <system.webserver> in a <location path=”.” inheritInChildApplications=”false”> directive.
It’ll look something like this:
- Redeploy and test
Run your pipelines or actions or whatever you need to do to deploy your web.config changes to the parent site, and see if it worked!
That’s all I had today! There are plenty of ways to fix this particular issue (because there are plenty of reasons), and this is just one. Did it work for you? Or did it fail miserably? Let me know in the comments -section below!
- System.Text.Json.JsonPropertyName not working for CosmosDb in .NET Core/5? - October 19, 2021
- Errors loading an assembly that’s using Microsoft Graph API - October 12, 2021
- ILogger binding suddely failing for Azure Functions – what to do? - October 5, 2021