Azure + Orchard

Topics: Administration, Installing Orchard
Oct 18, 2012 at 1:29 AM
Edited Oct 18, 2012 at 1:32 AM

I understand there is quite a bit of content out there regard this, but I still feel like some best practices and discussion is missing in the space.

A little bit of brief history of how I got here:

I noticed that my site, particularly the admin pages, was running quite slow on my easymode Azure Website + WebMatrix deploy. After digging a little bit the whole dynamic compilation and modules aspect of orchard came front and center. Unfortunately it seemed to me that the only way to get around this was to build from the source and maintain the source project with the modules and themes you wanted compiled and built so that you could disable dynamic compilation.

My process for doing this was just "ClickToBuild.cmd" and then using WebMatrix to deploy the Stage folder (the stage folder appeared to be the only one with all the bin folders and dlls for the modules). I also made sure to delete app_data after the first build (right?).

After doing this for a bit, everything seemed to work well and my linked SQL database resource was easy to connect to. Unfortunately I stumbled upon another limitation of Azure Websites not working with SSL certificates.

Back to the drawing board.

Having worked with SSL certificates and Azure in the past before, I was aware that dedicated "web-roles" had the ability to manage them along with some neat visual studio packaging integration. After more tinkering I loaded up the Orchard.Azure.sln solution and configured it to deploy properly with some caveats.

Currently I'm not really sure how ClickToBuildAzurePackage.cmd should be used. I get that it creates a package that you can deploy to Azure. Does that package work well with web roles, instead of websites? Does that package contain the dlls to prevent dynamic compilation? I kind of avoided the whole ClickToBuildAzurePackage bit and went in favor of deploying directly from the Azure visual studio solution.

I'm now in another situation where I'm not sure of the differences between Azure Storage and Azure SQL. It looks like in the past, based on the documentation, that Azure Storage contained the SQL databases. Now they appear to be separate entities with different ideas of connection strings. Also, where does local storage (such as app_data) go in the case of web roles?

After spending nearly two days on this, I'm kind of burnt out with all the options for development, deployment, and configuration. Any suggestions on how to do this best in the cloud with the following requirements:

  • No dynamic compilation of modules or themes in production.
  • Ability to use SSL certificates for https.
  • Ability to keep track of themes and modules in source control.
  • Ability to develop themes and modules and deploy them easily.
  • Ability to sanely manage application data in one or two places (Azure SQL vs Azure tables? App_Data? Local Storage? etc?)
  • Ability to quickly deploy with minimal downtime. (Production / Staging swap of web roles maybe?)

I really don't know where to take all of this and its becoming a little overwhelming figuring out a way to do this that will last.

Any takers to help correct my mistakes and guide me?

Oct 18, 2012 at 3:23 PM
Edited Oct 19, 2012 at 2:00 PM

Yes, clicktobuildAzurePackage.cmd produces packages that you can deploy into web roles in Azure. Those packages that it outputs should already be set up to be precompiled, and have dynamic compilation disabled (i'm using pretty recent changesets from the 1.x branch, i'm not sure how long ago this change took place). 

I'm working on figuring out the story behind a more typical developer building versioning, and deploying an Orchard site into Azure. I'm planning on either making some blog posts or updating the Orchard docs or both once I have explored further. 

Azure Storage is the underlying storage tech that is used across all of Azure. It is redundant (3 copies of data are saved any time to write to storage). On top of Azure storage, you have blob storage, table storage and Azure SQL. Azure SQL is a subset of of SQL Server. Azure Tables is a key-value storage system, and Blob storage, you can think of as storing files. This is grossly oversimplified so you probably want to read the Azure website to get informed on this. I believe you need a storage account to use any of the techs (blob/table/sql azure). 

Orchard, when deployed to Azure, typically uses SQL Azure, which is similar enough to SQL Server that you can choose the "SQL Server" option from the Orchard setup wizard when you first deploy to Azure, and just provide the connection string for your SQL Azure database. 

For settings.txt, and media (uploaded files, Piedone's combinator module, etc) in Azure, Orchard uses blob storage, which is shared across all web role instances. Blob stored items also get a public URL so you can link to them from your web app. I'm just learning about this now, but there are apps you can use, such as one by Cloud Berry, to browse the contents of your blob storage account.

There is also a 3rd party app called "Azure Diagnostics Manager 2", which is similar, but instead allows you to browse the contents of your Azure Table Storage account. The Orchard logs are written to table storage in Azure. 

Update with info Sebastien provided in his post below: 

Orchard stores App_Data/ stuff in local storage, which is volatile storage that is private to each web role. App_Data is used for things like indexing, and warmup files, as well as some cached data about your orchard config (I think Orchard grabs this from the db when it starts up). 

 

Oct 18, 2012 at 3:33 PM
Edited Oct 19, 2012 at 2:04 PM

 

  • No dynamic compilation of modules or themes in production.
    • Just use clicktobuildazurepackage.cmd (assuming 1.x branch, see prev. post) 
  • Ability to use SSL certificates for https.
    • Azure web role, and configure that? haven't touched this yet. 
  • Ability to keep track of themes and modules in source control.
    • What I do right now is keep a private fork of Orchard. Define an "upstream" repo in your .hg/hgrc file, and use that to pull commits from the main Orchard repo. Add your main custom modules and theme directly to this, and pull in 3rd party modules as subrepo's. 
  • Ability to develop themes and modules and deploy them easily.
    • Not sure what you're asking with this one. 
  • Ability to sanely manage application data in one or two places (Azure SQL vs Azure tables? App_Data? Local Storage? etc?)
    • Your content is in SQL Azure, "media" folder is in blob storage, and logs are in Azure Tables, nothing else to worry about. You shouldn't need to touch app_data, although I'd like to find out how to blow away app_data so I can restart my Orchard instance from scratch. I think I can just browse my blob storage with Cloud Berry App, and delete the App_data folder, but haven't tried it yet. 
  • Ability to quickly deploy with minimal downtime. (Production / Staging swap of web roles maybe?)
    • That works. I tried it a few times. But I'm worried about more complex scenarios where you have site in production with  multiple web servers(or roles), and you also have corresponding db changes. Deploying a web role in azure so far takes quite a long time, like 10-15 minutes at least. But if you do staging/prod, you can do it without downtime. If you have db changes, for example altering or deleting schema, you have to come up with a more complex strategy. I'm not sure how to do it without downtime. 
    • as

Update: 

Confirmed that if I delete sites/settings.txt from blob storage I am able to hit the setup wizard again. This is useful for blowing away an Orchard instance that's already been deployed to Azure, and starting from scratch. I just used a different db table prefix for now, but I could have also run a script to drop all the tables from the db. 

Developer
Oct 18, 2012 at 6:18 PM

@TheMonarch: so are you storing App_Data in blob storage?

Oct 18, 2012 at 6:33 PM
Edited Oct 18, 2012 at 6:34 PM

No, I was probably wrong though. I was trying to explain how Orchard stores things in Azure. Is app_data stored locally, and not in blob? 

Developer
Oct 18, 2012 at 6:38 PM

I've never used Azure, that's why I was curious :-). But AFAIK App_Data is generally not stored in blob.

Oct 18, 2012 at 6:45 PM

i think sites/settings.txt is stored in blob, which is why i assumed all of app_data was there. But i don't know for sure what the full picture is. Someone else should be able to answer this. 

Coordinator
Oct 18, 2012 at 7:06 PM

Settings are save using ShellSettingsManager. On Azure it's in the blobs as it must be common for all nodes.

App_Data is by definition a local folder to the web application. App_data will then be always stored locally. The question is actually "what from a local App_data folder will be stored on Blobs when running on Azure".

For instance, search indexes will still be saved in App_data, but all nodes will have their own updated search index. Same thing for the Warmup files.

Oct 18, 2012 at 7:18 PM
Edited Oct 18, 2012 at 7:24 PM

Thanks for clearing that up. If search index is stored locally, do you lose the local storage whenever you redeploy a web role? How often does the local storage get "lost" usually? 

Oct 19, 2012 at 5:49 AM

Thank you TheMonarch for so much valuable information. It helps clarify a lot of points I was unsure about.

One more question. How do you manage ClickToBuildAzurePackage with different service configurations? I can't seem to get that to work with the AzurePackage.proj file correctly. I get the following error.

C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Windows Azure Tools\1.7\Microsoft.WindowsAzure.targets(352,5): error : No default service configuration "ServiceConfiguration.cscfg" could be found in the project. [X:\Code\git\src\Orchard.Azure\Orchard.Azure.CloudService\Orchard.Azure.CloudService.ccproj]

Even though I believe I'm pointing to the right configurations in the AzurePackage.proj. Is there somewhere else I need to define a target configuration?

I'm also interested in some of the localstorage finer details regarding instance recycling/failure and app_data.

Oct 19, 2012 at 1:54 PM
Edited Oct 21, 2012 at 3:27 AM

Are you asking about transformations for the ServiceConfiguration.cscfg? 

I haven't tried transforms yet, but it sounds like you have and found that they aren't. I know I will have to soon, it's getting annoying to have to change the settings back and forth for local debugging vs deploying to Azure. This link might be useful: http://blog.alexlambert.com/

Could you verify that the transforms aren't being applied, and if so, create a work item for the task?  Even better, if you have time to work on it, it would be awesome if you could submit a patch to add the transforms :) 

Update: I patched the azure build package to allow config "transforms" for ServiceConfiguration.cscfg. They aren't really transforms, they are different copies for each configuration. ServiceConfiguration doesn't change nearly as often as web.config, and the amount of settings in there is usually pretty small, meaning overhead for maintaining copies rather than applying transforms is low. Here is the work item: http://orchard.codeplex.com/workitem/19172

If you want to specify different Cloud service configs with this patch (other than the default, which is "Production") you have to call msbuild by hand rather than use the ClickToBuildAzurePackage.cmd. You could also modify the .cmd to take in parameters for property names, or make separate copy of the .cmd for Production/Deployment/etc. 

Oct 23, 2012 at 7:22 AM
Edited Oct 23, 2012 at 7:39 AM

Awesome patch! I downloaded it and started playing with it and it appears to work but I get the following error on my deploy.

None of the constructors found with 'Public binding flags' on type 'Orchard.Environment.DefaultOrchardShell' can be invoked with the available services and parameters:
Cannot resolve parameter 'Orchard.Mvc.Routes.IRoutePublisher routePublisher' of constructor 'Void .ctor(System.Func`1[Autofac.Features.OwnedInstances.Owned`1[Orchard.Environment.IOrchardShellEvents]], System.Collections.Generic.IEnumerable`1[Orchard.Mvc.Routes.IRouteProvider], Orchard.Mvc.Routes.IRoutePublisher, System.Collections.Generic.IEnumerable`1[Orchard.Mvc.ModelBinders.IModelBinderProvider], Orchard.Mvc.ModelBinders.IModelBinderPublisher, Orchard.Tasks.ISweepGenerator)'.

I'm not sure how related it is to service configuration management, it looks like other people have encountered this before but the answers aren't entirely clear. Looks to be something with the way orchard is configured on app start.

Any suggestions or should I open another thread? (I can provide a full stack trace if you like, but its pretty long)

related:

http://orchard.codeplex.com/discussions/347495

http://orchard.codeplex.com/discussions/396085

Oct 23, 2012 at 12:28 PM

Not sure, but it doesn't seem related to service def. You mentioned in that 2nd thread you linked to that you had this error before applying my patch, so i don't think it's the patch.

How did you build and deploy? Did you run ClickToBuildAzurePackage.cmd from the command line (from a Visual Studio Command Line at that)? And you uploaded the package and the .csdef to your staging environment in Azure? 

What non-default modules do you have in this build? What recipe, and what version of the code (1.6 RC?)? 

Oct 23, 2012 at 5:57 PM

I've just been clicking the cmd file in windows explorer. No non-default modules, I haven't initialized any recipes yet if that makes a difference, and I believe this is the 1.5.1 branch (tag?). I'm still new to hg.

Oct 23, 2012 at 6:15 PM

How are you running the build? You are uploading through the Azure portal to your Azure staging env? 

Oct 23, 2012 at 6:37 PM

I've tried both: uploading through the portal and publishing from visual studio. The result is the same. Its worth noting that it works when debugging on the local cloud emulator.

Oct 23, 2012 at 6:48 PM

This one's beyond my abilities. The only suggestion I have is to try the 1.6 RC.

Oct 24, 2012 at 10:32 PM

Yeah, I don't know what the issue was, but it seems to be gone once I switched to the RC.

Oct 30, 2012 at 3:59 PM

Tested Azure Trial for 90 days and started a website with orchard 1.5 cms through app gallery.

It worked like a dream to publish it through webmatrix 2!

So when will the 1.6 update come to azure app gallery? Using VS 2010 Pro.

Coordinator
Oct 30, 2012 at 5:18 PM

It has been submitted for the gallery, should not take so long. It's in our hands anymore.

You can still download it from codeplex and push with Webmatrix.

Nov 1, 2012 at 4:44 PM

Thank you!

Installed 1.5 on Azuresite, started to use it and updated it through Dashboard, then downloaded it to keep working on it locally. Same result right? :)