Some questions about Dynamic and Precompiled extensions..

Topics: General, Writing modules
May 24, 2011 at 3:53 PM
Edited May 24, 2011 at 4:05 PM

I took a quick dive into the ExtensionLoaders. As far as i figured out a precompiled extension (module) is an extension that contains the bin directory. Could it be (memory wise) better to have pre-compiled extensions? In that case i think i would "extend the extensionmanager" to save the dynamicly compiled assembly to the bin directory.

- Is a precompiled extension handled differendly? For example not being monitored etc?
- In the case of 100 modules would it be better to have 100 precompiled or 100 code-only (dynamic) modules?
- Can or will a precompiled module with source be recompiled when the source is modified?

 

Coordinator
May 24, 2011 at 4:41 PM

Answering your questions is not easy. Because there is the current situation (i.e. Orchard 1.1) and what we could do in the future to improve. Let me answer with the "current situation" first.

As far as i figured out a precompiled extension (module) is an extension that contains the bin directory.

Correct: If a module has an assembly in "~/Modules/<MyModule>/bin", then that assembly will be loaded by the "precompiled extension loader"

Could it be (memory wise) better to have pre-compiled extensions? In that case i think i would "extend the extensionmanager" to save the dynamicly compiled assembly to the bin directory.

It could be. However, we are limited by Medium Trust. If you look at "dynamic extension loader", we use asp.net BuildManager (and have our custom build provider) to build .csproj files. This is the only way to generate code that is compatible with Medium Trust. One of the limitiation is that we don't have access to the assembly file (it's stored somewhere in temp asp.net folder), we are given back the assembly already loaded in memory.

- Is a precompiled extension handled differendly? For example not being monitored etc?

It is handled differently in a sense that the module assembly is copied to "~/App_Data/Dependencies" before being loaded. The original assembly file is monitored too.

- Can or will a precompiled module with source be recompiled when the source is modified?

Currently, sources files are monitored even when a module is pre-compiled. This is so that, if a source file is modified and the module is not recompiled (e.g. WebMatrix or notepad editing scenario), the module will be re-compiled dynamically.

I took a quick dive into the ExtensionLoaders. As far as i figured out a precompiled extension (module) is an extension that contains the bin directory. Could it be (memory wise) better to have pre-compiled extensions? In that case i think i would "extend the extensionmanager" to save the dynamicly compiled assembly to the bin directory.

- In the case of 100 modules would it be better to have 100 precompiled or 100 code-only (dynamic) modules?

Currently, if you are looking for the best possible performance, I think (even though I haven't measured) your best bet is to deploy all the module binaires to ASP.NET "~/bin", and remove all source code for modules and assemblies from "~/Modules/<MyModule>/bin". The second best option would be to remove all source code and leave assemblies in "~/Modules/<MyModule>/bin". This would prevent Orchard from monitoring source code files and also trying to re-compile modules dynamically.

However, it is very *unlikely* that 100s of modules will scale well even if you deploy to "~/bin" (option above). In .NET, loading hundreds of small assemblies is slower than loading a few big ones. This is why, in the future, we are considering having an "advanced" dynamic compilation loader/manager which would be able to compile all source code into one big assembly (or maybe a few big ones). To be able to do that though, Orchard would need to have the source code of all modules available, which is different than the recommendation above. Finally, if you have an Orchard installation with 100s of modules, please share your exeperience with us (e.g. on this forum). We would like to understand the pain points.

Another option we are looking at (shorter term) is to offer "settings" to the Orchard compilation system so that a deployed site which is "stable" (i.e. modules are not edited or updated, but at the same idled out very often) can have better startup performance. Stay tuned!

HTH,

Renaud

May 24, 2011 at 7:07 PM

Thanks for your fast reply. I see what u mean and i hope that the future isn't that far away. We're generating modules foreach customer generated "statemachine - pageflow with views in a wizard".
What about when the site is in "Full trust" to register another ExtensionLoader? Saving the dynamic assembly into the bin folder?

The best performance as u suggest won't be that easy for us since the code is generated and then we also need to use the codedom ourself :) ... and deploy them into the /bin or/Modules/..../bin 

 

Coordinator
May 24, 2011 at 8:12 PM

Could you explain your scenario a little bit further? It sounds like you are taking advantage of an Orchard feature (dynamic modules) for a scenario we hadn't thought about. Maybe there are other ways of adressing that scenario...

For example, adding/removing a module will result in ASP.NET recompiling all the views (cshtml files) in your theme. We have to do that because a view in a them can potentially reference classes in any module. If the modules you generate won't be referenced by views, this is another overhead that could be overkill. (Note that this is *no* different than adding/removing an assembly from the ~/bin folder, it's how ASP.NET works).

As for your question about behaving differently under FullTrust, it is an option of course, but we have to consider the impact. Unfortunately for us, the type of sites where we are seeing the worst startup performance are sites hosted in environments where Medium Trust is enforced. Basically, it would be better if we could find an option which is compatible with Medium Trust (easier said than done, of course :)).

Renaud

May 25, 2011 at 9:45 AM
Edited May 25, 2011 at 10:06 AM

At the moment we have an xml / xsd combination where views are defined and a pageflow is defined. A generator generates a project, a pageflowdefinition, a controller, parts, records, drivers, viewmodel, views, etc when the xml is published. Orchard takes care of the rest. We've a own framework layer where all those generated projects depend on. The framework takes care of pageflow routes and based on a state the drivers know when to generate a specific shape or none.

Using webmatrix or something a developer can alter the generated code a bit with stuff that our xml doesn't support.
Or they can override views in a theme, all the stuff Orchard is good in.

In the future the tooling that will create the xml definition will be smart and easy enough for the endcustomer, even reusing an existing view within a new flow might be possible.

A scenario
1) the endcustomer uses our visual tool to define views and arange them in a pageflow (and lots more);
2) xml is accepted and published;
3) the T4 generator gets notified and generates the module source (a .xamlx, a library with a datacontract and workflow activity's)
4) command gets executed to enable the new feature.

Mar 13, 2012 at 10:30 AM

Is there any status update on a dynamic compilation loader/manager for performance when having lots of modules?