Azure Cloud Services -- New Module to Synchronize Installed Modules

Topics: Core, Customizing Orchard, Writing modules
Jun 26, 2013 at 3:34 PM
Edited Jun 26, 2013 at 3:34 PM
With regard to the following concept:

"Deploying Orchard to Azure with Optional Modules

"The only constraint is that the modules cannot be installed dynamically from the gallery as you would do with a regular deployment of Orchard, because of the distributed nature of Azure. The local file system is not automatically replicated across instances; instances might get out of sync if this were allowed. "

Has anyone considered changing Orchard's behavior to accommodate running in a distributed cloud environment? I am considering writing a module which persists installed modules to Azure blob storage for synchronized retrieval on web role startup. There are many numerous reasons for preferring Azure Cloud Service over Azure Websites for a large Orchard-based enterprise application.

Would anyone like to weigh in on the suitability of such a module? Are there any other potential pitfalls besides gallery modules? (themes, caches in app_data)? My time frame for developing this functionality is the next 2-3 months.

Thanks,
Shaun
Jul 2, 2013 at 7:39 PM
I didn't hear back from anyone in the community. I think this is a real problem with deploying Orchard to Cloud Services. The "scale out" story for Orchard is up in the air until we have clarity around how to synchronize state across server instances. I'd like to be a contributor to Orchard and some feedback around this functionality is welcome.

Thanks,
Shaun
Jul 2, 2013 at 7:51 PM
What prevents you from scaling out using cloud services? Unless I misunderstand, you can scale in/out as long as you don't install modules directly in production (and installing modules in production is not supported or recommended). If you want to install new modules, update your code, and re-deploy the cloud service.

Enabling/disabling modules is also something you can't do (I think), but you can workaround that by doing the enabling/disabling via migrations + re-deploy.
Coordinator
Jul 2, 2013 at 7:53 PM
When you deploy on Azure Cloud Services, the package contains all the modules your application requires. If you then scale out to several VMs, then every VM will have the same modules. The database, configuration and media are store in Azure Services like SQL Azure and blob storage hence they will be shared across all the VMs.
I will assume you understood these concepts.

I don't see the point of getting dynamic modules into Cloud Services. You must have a good business reason though and I would be glad if you could share it.

In theory such a module would be doable, but it might also be less painful to just update your Azure Cloud Package whenever you want to add a new module. I can easily see how a module could look for a specific container in the blos and copy all available modules locally, because you would have to do it after each restart as the file system would be reseted to its initial state.

Using full Azure VMs is also another solution to your problem as they have persistent storage.
Jul 2, 2013 at 8:51 PM
Edited Jul 2, 2013 at 8:52 PM
Hi Sebastien. Dynamic module loading is an incredibly important feature in Orchard, and in our world this feature is what differentiates Orchard from other .NET-based CMS systems. I am using Orchard successfully as a client-consulting platform for line of business software. My company builds websites for Fortune 100 companies.

Many of our clients request similar functionality (i.e. Active Directory integration, Inventory Management, Billing). Instead of reinventing the wheel we are using the modular architecture of Orchard to make features developed for these clients reusable. Orchard is a terrific CMS! We love it. And it is also a terrific architecture for loosely coupled business applications.

I agree with the explanation that it is possible to recompile and redeploy a site to surface new modules. Our goal is to deliver to clients websites that are easily to configure and easy to extend. With regard to the business needs, we have multiple 10+ person teams comprised of business analysts, support staff, and developers. With teams of this size communication can be a challenge. It is much much easier for staff to upload a module to a free-standing site than for a developer to modify source code and redeploy. The value of Orchard to companies like ours is that it is extensible and it has an open architecture. Recompiling and redeploying source code is undesirable.

Azure Websites do not support custom SSL certificates. They also lack staging and production capabilities and instantaneous VIP swaps. Our cloud business has been very successful leveraging Azure Cloud Services and we find many advantages to using them.

I may be short-sighted but my thinking here is that Orchard could be modified to support this scenario. I am considering writing a module which persists installed modules to Azure blob storage for synchronized retrieval on web role startup. When modules are added or updated they are written to Blob storage. A flag is set in the database to indicate availability of a new module. My understanding is that we would need to do this for theme CSS as well.

Thanks,
Shaun
Coordinator
Jul 2, 2013 at 9:06 PM
They support SSL now.
And there are solutions for Staging/Production, based on a reverse proxy, also hosted on Azure, which would BTW make your website really fast.
Jul 3, 2013 at 7:33 PM
Hey Sebastien. I appreciate this feedback. I am still thinking that we wouldn't be able to add and remove modules dynamically across multiple instances -- and this is the particular challenge we are hoping to solve. It sounds like the Orchard team isn't interested in this capability? If this is the case I could implement the changes in a fork or module.
Coordinator
Jul 3, 2013 at 8:33 PM
Why would you want to do that on a production site?
Jul 3, 2013 at 8:36 PM
@stonstad, it sounds like you might need a better workflow / automation around building and recompiling your Orchard sites. It shouldn't be a big deal to redeploy if you script things out. Maybe then having to re-deploy to add/remove modules wouldn't be a drawback for you?
Jul 8, 2013 at 6:23 PM
Edited Jul 8, 2013 at 6:26 PM
Hey guys -- I have considered your responses very carefully and this is why I have waited a few days to respond. I appreciate your insight and suggestions. From what I am able to understand, the Orchard team has a fundamentally different view of Modules than I do. You are the creators of the software and your viewpoint is logically correct. I would like to explain my confusion for your feedback and consideration.

Many CMS systems are moving to a plug-in based open architecture. WordPress is a notable example and they support over 20,000 plug-ins, many of which can be enabled through a series of simple clicks. There are many open CMS systems to choose from. With the advent of the app store approach to software, many clients now expect this level of functionality. Orchard appears to support this concept from a technical viewpoint, but what I am hearing is that this is not the preferred workflow for assembling a site. My question is, "Why not?"

You wouldn't redeploy your phone's operating system image each time you installed a new app, would you? Why would users of Orchard want to redeploy their entire site from source code in Visual Studio to enable a module? This raises the question -- Who is your target audience? Is your target audience a CMS administrator who is not a developer? Is your target audience a software developer?

At the end of the day the goal of a site implementation is to turn it over to an end user or IT staff who use and maintain the system. These individuals may possess a high level of technical skill but they are not developers. There is no reason to believe that an administrator of a CMS would desire to learn Visual Studio. If new features cannot be added to an Orchard site without developer intervention this calls into question the viability of Orchard as a plugin-based CMS. I love Orchard and I see it as a powerful CMS, and it speaks to a future architecture of powerful line-of-business HTML5/MVC4 applications.

I manage a sizable team of .NET developers -- many of whom are training to learn Orchard. I would like to know what the long-term vision is for module integration. If you are concerned about the work effort involved in getting Orchard to support multiple Cloud Service WebRoles -- my team and I can give you added bandwidth.

Thanks,
Shaun
Coordinator
Jul 8, 2013 at 6:42 PM
I would not let my customers the permission to enable/disable a module on a production website, just because it can break it, or alter the initial intent, but that is a personal choice. Not that you can't do it if you know what module you are enabling and for what reason, it's totally right. On my own website I do it personally. If you think your customers should be able to do it, maybe with your own curated list of modules, it's fine.

As for having modules work in Cloud Service, if you want to do it it's great. But I think the audience is very limited as people might tend to use Azure Websites, or if they use the Cloud Service then might now try to let their customers install modules dynamically. But if you are in the need for it and are willing to implement it I can give you some guidance, if I have any knowledge you might use.
Coordinator
Jul 8, 2013 at 8:53 PM
I see your point, but there is of course a world of difference between installing an app on a phone and installing a WordPress plugin. Case in point, WordPress.com won't let you install your own modules, because you could bring down your site and others with you. We're pretty much on the same page as they are: if you own your Orchard instance, in principle you can do anything, install modules, etc. But that doesn't mean that you should. Modules in any CMS don't run in sandboxes, and thus have enormous potential for destruction. I do not personally install anything directly in production, even though I could. It's just proper engineering practice: I proceed carefully and try to minimize downtime on my production sites. Everything is tested first on dev/staging boxes, then only deployed. If anything, I would like deployment of modules and data to be facilitated, but I would still make the recommended architecture have a staging and a production site, always. A dev machine is very easily set-up after all.
Now I do have a customer that is in the process of setting up a private gallery of modules and themes, with the goal of letting their customers install those themselves. That is similar or a little more advanced than what WP is doing on wordpress.com, and something that is completely reasonable, because the set of available modules is fully tested, and we're in a very controlled environment. I hope that what they are doing can benefit the community at large.
Of course, we welcome the participation of your team, and would be happy to chat, maybe over Skype, to better understand what you are trying to achieve.