Server Error when bringing online a new tenant

Topics: Administration, Troubleshooting
Sep 13, 2012 at 8:30 PM

I have only experienced this issue since upgrading the site from 1.4.x to 1.5.1.

When I bring online a new tenant the tenant throws a 404 error on the home page. While attemtping to go into the admin I get a YSOD with the following message:

System.InvalidOperatonException: A 'script' named 'jQuery' could not be found.

I was too anxious to get the site up and running to bother copying out the entire error message.

The work-around I have had to use is to delete the site folder under App_Data/Sites, then go back to the hosting site and recreate the tenant and then restart IIS or at least the app pool since the second time I create the tenant it brings all my sites down without the restart of the app pool.

I know this isn't much to go on as far as diagnosing the problem. I will update when I do another tenant.

Nov 28, 2012 at 2:14 AM

For what it's worth I'm also getting this problem in Orchard v1.6 when creating a new tenant. After creating the tenant the new site doesn't display. A resource not found exception is instead displayed. And if I login to the admin for the tenant, the the script not found exception is displayed. Full error stack trace below.

 

[TargetInvocationException: Exception has been thrown by the target of an invocation.]   System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) +0   System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) +76   System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) +193   Orchard.DisplayManagement.Descriptors.ShapeAttributeStrategy.ShapeAttributeBindingStrategy.PerformInvoke(DisplayContext displayContext, MethodInfo methodInfo, Object serviceInstance) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Descriptors\ShapeAttributeStrategy\ShapeAttributeBindingStrategy.cs:65   Orchard.DisplayManagement.Descriptors.<>c__DisplayClass5.<BoundAs>b__2(DisplayContext displayContext) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Descriptors\ShapeAlterationBuilder.cs:55   Orchard.DisplayManagement.Implementation.DefaultDisplayManager.Process(ShapeBinding shapeBinding, IShape shape, DisplayContext context) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Implementation\DefaultDisplayManager.cs:179   Orchard.DisplayManagement.Implementation.DefaultDisplayManager.Execute(DisplayContext context) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Implementation\DefaultDisplayManager.cs:88   Orchard.DisplayManagement.Implementation.DisplayHelper.ShapeExecute(Object shape) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Implementation\DisplayHelper.cs:71   ClaySharp.<>c__DisplayClass16.<InvokeMember>b__13() in C:\Users\sebros\My Projects\Clay\src\ClaySharp\ClayBehaviorCollection.cs:29   ClaySharp.ClayBehaviorCollection.Execute(Func`1 proceed, Func`3 linker) in C:\Users\sebros\My Projects\Clay\src\ClaySharp\ClayBehaviorCollection.cs:13   Orchard.Core.Shapes.CoreShapes.DocumentZone(Object Display, Object Shape, TextWriter Output) in c:\Users\Michael Yates\Documents\Visual Studio 2012\Projects\Orchard.Source.1.6\src\Orchard.Web\Core\Shapes\CoreShapes.cs:264
[TargetInvocationException: Exception has been thrown by the target of an invocation.]   System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) +0   System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) +76   System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) +193   Orchard.DisplayManagement.Descriptors.ShapeAttributeStrategy.ShapeAttributeBindingStrategy.PerformInvoke(DisplayContext displayContext, MethodInfo methodInfo, Object serviceInstance) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Descriptors\ShapeAttributeStrategy\ShapeAttributeBindingStrategy.cs:65   Orchard.DisplayManagement.Descriptors.<>c__DisplayClass5.<BoundAs>b__2(DisplayContext displayContext) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Descriptors\ShapeAlterationBuilder.cs:55   Orchard.DisplayManagement.Implementation.DefaultDisplayManager.Process(ShapeBinding shapeBinding, IShape shape, DisplayContext context) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Implementation\DefaultDisplayManager.cs:179   Orchard.DisplayManagement.Implementation.DefaultDisplayManager.Execute(DisplayContext context) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Implementation\DefaultDisplayManager.cs:88   Orchard.DisplayManagement.Implementation.DisplayHelper.ShapeExecute(Object shape) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Implementation\DisplayHelper.cs:71   ClaySharp.<>c__DisplayClass16.<InvokeMember>b__13() in C:\Users\sebros\My Projects\Clay\src\ClaySharp\ClayBehaviorCollection.cs:29   ClaySharp.ClayBehaviorCollection.Execute(Func`1 proceed, Func`3 linker) in C:\Users\sebros\My Projects\Clay\src\ClaySharp\ClayBehaviorCollection.cs:13   ASP._Page_Core_Shapes_Views_Document_cshtml.Execute() in c:\inetpub\orchard.1.6\Core\Shapes\Views\Document.cshtml:15   System.Web.WebPages.WebPageBase.ExecutePageHierarchy() +280   System.Web.Mvc.WebViewPage.ExecutePageHierarchy() +125   System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage) +196   System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData) +147   Orchard.DisplayManagement.Descriptors.ShapeTemplateStrategy.ShapeTemplateBindingStrategy.Render(ShapeDescriptor shapeDescriptor, DisplayContext displayContext, HarvestShapeInfo harvestShapeInfo, HarvestShapeHit harvestShapeHit) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Descriptors\ShapeTemplateStrategy\ShapeTemplateBindingStrategy.cs:137   Orchard.DisplayManagement.Descriptors.ShapeTemplateStrategy.<>c__DisplayClass28.<Discover>b__15(DisplayContext displayContext) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Descriptors\ShapeTemplateStrategy\ShapeTemplateBindingStrategy.cs:118   Orchard.DisplayManagement.Descriptors.<>c__DisplayClass5.<BoundAs>b__2(DisplayContext displayContext) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Descriptors\ShapeAlterationBuilder.cs:55   Orchard.DisplayManagement.Implementation.DefaultDisplayManager.Process(ShapeBinding shapeBinding, IShape shape, DisplayContext context) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Implementation\DefaultDisplayManager.cs:179   Orchard.DisplayManagement.Implementation.DefaultDisplayManager.Execute(DisplayContext context) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Implementation\DefaultDisplayManager.cs:98   Orchard.DisplayManagement.Implementation.DisplayHelper.ShapeExecute(Object shape) in c:\Users\sebros\My Projects\Orchard\src\Orchard\DisplayManagement\Implementation\DisplayHelper.cs:71   ClaySharp.<>c__DisplayClass16.<InvokeMember>b__13() in C:\Users\sebros\My Projects\Clay\src\ClaySharp\ClayBehaviorCollection.cs:29   ClaySharp.ClayBehaviorCollection.Execute(Func`1 proceed, Func`3 linker) in C:\Users\sebros\My Projects\Clay\src\ClaySharp\ClayBehaviorCollection.cs:13   Orchard.Mvc.ViewEngines.ThemeAwareness.<>c__DisplayClass7.<FindView>b__5(ViewContext viewContext, TextWriter writer, IViewDataContainer viewDataContainer) in c:\Users\sebros\My Projects\Orchard\src\Orchard\Mvc\ViewEngines\ThemeAwareness\LayoutAwareViewEngine.cs:63   System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context) +380   System.Web.Mvc.<>c__DisplayClass1a.<InvokeActionResultWithFilters>b__17() +33   System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) +613   System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) +613   System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) +613   System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) +613   System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) +613   System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) +613   System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) +613   System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) +613   System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation) +613   System.Web.Mvc.ControllerActionInvoker.InvokeActionResultWithFilters(ControllerContext controllerContext, IList`1 filters, ActionResult actionResult) +263   System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName) +720   System.Web.Mvc.<>c__DisplayClass1d.<BeginExecuteCore>b__19() +40   System.Web.Mvc.Async.<>c__DisplayClass1.<MakeVoidDelegate>b__0() +15   System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult) +53   System.Web.Mvc.Async.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar) +15   System.Web.Mvc.<>c__DisplayClass8.<BeginProcessRequest>b__3(IAsyncResult asyncResult) +42   System.Web.Mvc.Async.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar) +15   Orchard.Mvc.Routes.HttpAsyncHandler.EndProcessRequest(IAsyncResult result) in c:\Users\sebros\My Projects\Orchard\src\Orchard\Mvc\Routes\ShellRoute.cs:162   System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +606   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +288

Nov 28, 2012 at 2:37 AM

I deleted and re-created the tenant/db a few times and had the same issue. I was logged into the default orchard instance remotely from my workstation when creating the tenant and initially received an exception (when the recipe was cooking) but the exception details were not displayed since the browser was remote.

So I tried again via a browser in an RDP session on the web server running the orchard instance (expecting to see the exception details thrown during my recipe bake-off) but in this case the tenant creation went through without error.

In short, in my case, I believe the errors I received above in the admin pages and the homepage were a result of the recipe not completing successfully.

 

Jan 25, 2013 at 2:02 AM

I'm getting this error pretty much every time I attempt to setup a new tenant now. This occurs when setting up the tenant (after clicking "Set Up" and specifying the tenant information such as website name, administrator username and password, initial recipe) not when defining the tenant from the master Orchard instance.

The error is "Sequence contains no matching element" coming out of the Orchard.Environment.State.DefaultProcessingEngine.Execute method...

Here's a link to a screenshot of the full error... http://i48.tinypic.com/34yo1uv.jpg

Any tips into what areas I could look into to investigate further?

 

 

 

 

Jan 30, 2013 at 11:08 AM
Edited Jan 30, 2013 at 12:17 PM

Tried restarting your server? Also, check you have space in your database

Jan 30, 2013 at 2:41 PM
Edited Jan 30, 2013 at 9:27 PM
I delved into this further. I think the error is coming from the call to Orchard\Environment\DefaultOrchardHost.GetShellContext() when it calls .Single() on the list of shell contexts. It only expects to receive one shell context back for a given shell settings name, but for whatever reason, there are multiple stored with the same name (or none at all).

FYI. While previously above I got the error "Sequence contains no matching element" ...today I am getting the error "Sequence contains more than one matching element".

From looking into the DefaultOrchardHost.cs class it seems that it may be possible that the _shellContexts list could end up with the same context (or at least contexts with the same name) multiple times.

Looking at the method below (around line 240), it seems to me that if a shell context is found in _shellContexts with a matching setting name AND the context is yet to be initialised (I presume possible when a tenant has just been created), then the context can be added into the _shellContexts array a second time (via the call to ActivateShell(context).
void ActivateShell(ShellSettings settings) {
    // look for the associated shell context
    var shellContext = _shellContexts.FirstOrDefault(c => c.Settings.Name == settings.Name);

     // is this is a new tenant ? or is it a tenant waiting for setup ?
    if (shellContext == null || settings.State.CurrentState == TenantState.State.Uninitialized) {
        // create the Shell
        var context = CreateShellContext(settings);

        // activate the Shell
        ActivateShell(context);
   }
Because if you look at the code for ActivateShell(ShellContext context) around line 154, it just unions whatever context you pass it to the _shellContexts list without even checking first if it's already in the list.

The method at line 240 (ActivateShell - that takes a ShellSettings object as parameter) is called from the method StartUpdatedShells(). This method cycles through each tenant in the _tenantsToRestart array and calls ActivateShell() for each. _tenantsToRestart is populated when a new tenant is saved (presumably with an unitialised state). And StartUpdatedShells() is called from BeginRequest().

So I'm thinking what may be happening is that when I define a new tenant (from the default/master instance) the _tenantsToRestart list is populated with this, then I go to setup the new tenant and the act of calling BeginRequest() results in the _shellContexts list containing the new tenant twice. Then as the setup runs through (cooking the recipe) at some point the call to GetShellContext is made which calls linq .Single() on the _shellContexts list and I get the exception.

I'm going to add a check the code in ActivateShell(ShellContext context) (around line 158) to only add the context if it's not already in the list and see what happens...will report back...

Any other thoughts or comments appreciated...
Jan 30, 2013 at 9:46 PM
I changed the ActivateShell(ShellContext context) method in DefaultOrchardHost.cs from this...
        /// <summary>
        /// Start a Shell and register its settings in RunningShellTable
        /// </summary>
        private void ActivateShell(ShellContext context) {
            Logger.Debug("Activating context for tenant {0}", context.Settings.Name); 
            context.Shell.Activate();

            _shellContexts = (_shellContexts ?? Enumerable.Empty<ShellContext>()).Union(new [] {context});
            _runningShellTable.Add(context.Settings);
        }
to this...
        /// <summary>
        /// Start a Shell and register its settings in RunningShellTable
        /// </summary>
        private void ActivateShell(ShellContext context) {
            Logger.Debug("Activating context for tenant {0}", context.Settings.Name); 
            context.Shell.Activate();

            _shellContexts = (_shellContexts ?? Enumerable.Empty<ShellContext>())
                .Where(c => c.Settings.Name != context.Settings.Name)
                .Concat(new [] {context})
                .ToArray();

            _runningShellTable.Add(context.Settings);
        }
...and I was able to initialise the tenant (ie. create tables, cook initial recipe, etc) without any error. That said, it was still occasionally working in the past, so it could have just been the fact that I restarted IIS or something else.

I'll keep an eye on it as I create new tenants, and if it seems to resolve the issue I'll create a pull request for the change.
Jun 13, 2013 at 12:20 PM
I'm using Orchard v1.6, and experience the same issue.

I have done a script which create a database, create a tenant and do en setup on tenant successivly.

Frequently, "Sequence contains more than one matching element" in Orchard.Environment.State.DefaultProcessingEngine.Execute method is fired by the script.
Thanks to this discussion, applying alternate ActivateShell fonction resolve the issue for the duplicated context. If you may request for change, it will be helpfull.

Otherwise it didn't resolve the "Sequence contains no matching element", which continue to fire, seems randomly.

After test, i realize the error is fired if call is done (page request) on Orchard during the script execution (In my case, that was my load balancer that firing a ping on Orchard Default tenant, who stop and generate the error in the script).

I have done a new test with script containing only a create database and a create tenant (without tenant setup). It doesn't generate error.
Resuming, it seems that if a page call (like ping) is done during the setup of a new tenant, a error "Sequence contains no matching element" is fired in the web request running the setup.
Probably, the ping change the stat of the new tenant by the BeginRequest function, and with stat altered, the setup fail on something.

Did you reproduce the issue ?
I do not master how the setup is done and i have no idea how to ensure the complet setup execution. Add a lock somewhere in the setup ?
Jun 13, 2013 at 7:20 PM
I was having the same issue with the "Sequence contains more than one matching element" issue intermittently and I was able to diagnose when it happens. It happens when one of the other tenants is having any activity on their site (which would seem to be common in a production environment). The fix from mjy78 definitely fixes the issue so I definitely think a pull request is necessary since this seems like it would be a common issue.

However, I did run into another issue with multitenancy that throws a stackoverflow issue on every tenant that is created which I posted in a separate ticket here: https://orchard.codeplex.com/discussions/446985.