Workflow for Multiple Developers on a Website Project

Topics: Administration
Jun 3, 2013 at 8:03 AM
Greetings,

Does anyone have a good set of best practices for keeping multiple developers from stepping on each others toes? Any Azure methods even better. The use case is multiple roles for creating a new website:
  • Developer/PM
  • Developer
  • CSS/JavaScript/HTML
  • Content author
We're creating the theme and content parts using code, not from the Admin UI, so that we can quickly set-up similar sites (or re-configure the existing one if needed). We can't wait until the content types, HTML and JavaScript are all perfect before entering content, but with multiple people deploying changes, it seems there's a high likelihood of either wiping out the database, corrupting it, or introducing incompatible changes requiring the content to be re-entered or modified.

This seems to be a very common situation for new website development, but I can't quite put my finger on a winning formula with Orchard, despite quite a bit of reading. My thinking was to make regular backups, but this won't solve the corruption problem.

Anyone else got a good workflow for multiple people building a website?

Cheers,
- SteveN
Jun 3, 2013 at 1:13 PM
Yes, have everyone work off a local copy of the database. If you use Migrations.cs to manage the database schema, and also automate any static data/content (via Recipes), everyone should be able to check out the source and build the current version of the site from scratch. The build should include step to create the database, populate the data, and run the setup for the Orchard site.

You can use a small batch file or powershell script to handle most of this, and if you create recipes to manage your content, and keep the recipes in your source control, the setup script will run Orchard setup and then execute the recipes. If there are minor changes from version to version, Orchard itself will handle bringing each development machine up to the current version by automatically detecting which migrations in Migrations.cs need to run.

I have posted similar replies before, but possibly with more detail. Try looking up some of my old posts on the same subject and feel free to ask more questions if you have any.
Jun 4, 2013 at 1:39 AM
Searching for your name and 'recipe' shows two links, this one appears most relevant:

https://orchard.codeplex.com/discussions/431926

Is this the one you're thinking of? The way I read this, content is handled by the import/export ('recipe'), which is checked in as part of the source. It makes sense that this all be part of the build process to catch any errors early. Being able to rebuild the entire site from scratch, including content, is exactly what we want.

Do you have an example of a batch file or powershell script for this? Having a build target for the content author to deploy the recipe/content would be a very helpful thing as well, since it's his time that's most difficult to get.

This would be a great thing to get written up someone where in a 'blog post or on a website. Happy to do this if someone has a friendly 'blog (and assuming I get it worked out).
Jun 13, 2013 at 7:47 AM
So far, no luck with a working multi-developer environment/workflow, so thought I'd bump this up with a few comments and questions

TheMonarch wrote:
Yes, have everyone work off a local copy of the database. If you use Migrations.cs to manage the database schema,
OK, easy enough to manage content-types by Migrations.cs. None are, yet, custom, just assemblies of existing content-parts.
and also automate any static data/content (via Recipes), everyone should be able to check out the source and build the current version of the site from scratch. The build should include step to create the database, populate the data, and run the setup for the Orchard site.
After investigating, this looks to be a rather uphill battle. I've reported many errors in import/export over the last week. In our experience so far, it's just too broken to be used in this manner. Did you mange the recipes with any custom code? Did the site content types include any third-party content-parts or types? If import/export is reliable with the bog-standard content-types, we could add the third-party module content afterward. Painful, but possible.
You can use a small batch file or powershell script to handle most of this, and if you create recipes to manage your content, and keep the recipes in your source control, the setup script will run Orchard setup and then execute the recipes. If there are minor changes from version to version, Orchard itself will handle bringing each development machine up to the current version by automatically detecting which migrations in Migrations.cs need to run.
So do you split the recipes into multiple parts and run them a bit like makefiles? What sort of split have you found works well?
I have posted similar replies before, but possibly with more detail. Try looking up some of my old posts on the same subject and feel free to ask more questions if you have any.
I would really, really love to see some working example of this set-up. I want to use Orchard, but we're rapidly running out of the time allocated for the set-up phase, with little progress to show for it. If we don't have a breakthrough soon I'm going to have some tough questions to answer regarding progress and choices.
Jun 13, 2013 at 12:24 PM
How are you running the recipes? I used batch/powershell scripts to run orchard.exe, and broke out the static data recipes into their own files. In cases where performance was an issue, due to a recipe being too big, I simply broke the recipes into multiple files. The recipes I made were mostly for custom parts, but included some default types/parts. I didn't have any problem there. In some cases there were very minor bugs, like a particular property of a contentPart wasn't mapped in the driver, and I either reported it and it got fixed, or submitted a patch for it myself.

Maybe if you list more specifics of what problems you're having I can help (point me to the work items you filed).

I had some content types where there were dozens of thousands of items to import. I just broke them up in to four or five files. I experimented until I found a split that worked. Try cutting the files in half until they work. You should be doing the imports into a local copy of sql server.

I'm busy right now with other work but I'll try posting my scripts by Saturday afternoon. Here is my powershell script, which I name Orchard\src\Orchard.Web\rehydrate.ps1. It deletes the sql db, recreates it, and then runs the Orchard setup from the command line, targeting the "gemini" recipe. The gemini recipe is located at Orchard\src\Orchard.Web\Modules\Orchard.Setup\Recipes\geminiSetup.xml, so that the setup process can find it (I don't think the setup process will find recipes located elsewhere, when it is first setting up the Orchard site).
Write-Host "
==================================================
gemini Rehydrate Powershell script started."
$StopWatch = [System.Diagnostics.Stopwatch]::StartNew()

## 
## Kill Visual Studio development web server, to free lock on App_Data folder: 
Import-Module .\..\..\scripts\KillCassini.ps1
Kill-Cassini 
# Sleep for half a second to give the Cassini process time to die: 
Start-Sleep -m 500

if (Test-Path "App_Data") {
    Write-Host "Deleting App_Data folder."
    Remove-Item -Path .\App_Data -Recurse -Force
}

## 
## Drop and re-create the SQL database
Write-Host "Running Recreate_db.sql."
osql.exe -E -S localhost -i "..\..\scripts\Recreate_db.sql"

##
## Run Orchard setup
Write-Host "Running Orchard setup."
bin\Orchard.exe setup /SiteName:gemini /AdminUsername:admin /AdminPassword:adminPassword /DatabaseProvider:SQLServer /DatabaseConnectionString:"Data Source=localhost;Initial Catalog=geminiDB;Persist Security Info=True;User ID=sql_user;Password=sql_password" /DatabaseTablePrefix:geminiDB /Recipe:"geminiSetup"

Write-Host "
gemini Rehydrate Powershell script ended.
Elapsed time: $($StopWatch.Elapsed.ToString()).

"
The geminiSetup recipe has the following section:
    <Command>
        layer create Default /LayerRule:"true"
        layer create Authenticated /LayerRule:"authenticated"
        layer create Anonymous /LayerRule:"not authenticated"
        layer create Disabled /LayerRule:"false"
        layer create TheHomepage /LayerRule:"url '~/'"
        site setting set baseurl
        theme activate "geminiTheme"
        recipes execute gemini ImportNavigation
        recipes execute gemini ImportPages
        recipes execute gemini ImportCustomTypes1
        recipes execute gemini ImportCustomTypes2
        recipes execute gemini ImportCustomTypes...
        recipes execute gemini ImportCustomTypesN
     </Command>
The recipes have names according to the actual custom types being imported in each one. All those recipes being executed inside the <Command> section are located in a folder called "recipes" inside my gemini module.
If you haven't already done so, I recommend you take one day to just get the rehydrate process working. Start simple and improve the process one step at a time:
  1. Start with just getting the rehydrate to drop and recreate the database
  2. Then get it able to run orchard setup with one of the default recipes (Default, Core, Blog, etc)
  3. Then get it to run with your custom module's setup recipe that you drop into Orchard.Web/Modules/Orchard.Setup/Recipes/
  4. Then start plugging in your static data recipes into your module's setup recipe.
  5. Any recipes that timeout, break them up into smaller files.
I define all my custom parts and types programmatically, inside Migrations.cs, as opposed to doing so from recipe files. That has just worked better for me.

Hope this helps, let me know if you need more info.
I