Using SpecFlow to test custom modules

Topics: Writing modules
May 26, 2011 at 2:20 AM


I've created a SpecFlow feature in the Orchard.Specs directory for a simple Customer module I'm using to test SpecFlow with.

However when I run the feature in NUnit it fails with the message:

Expected: String matching "]*href="/Admin/Contents/Create/Customer"[^>]*>Create New Customer"
  But was:  "

this was for the following feature file:

Feature: Customer
	In order to provide more useful services to customers of the site
	As a site administrator
	I want to be able to create, edit, view and delete customers

Scenario: In the admin (ContentTypes menu) there is a link to create a Customer
	Given I have installed Orchard
	When I go to "admin/contenttypes"
	Then I should see "<a[^>]*href="/Admin/Content/Create/Customer"[^>]*>Create New Customer</a>"

I also added "Customer" string to the OrchardSiteFactory.cs file to the TableData in the GivenIHaveInstalledOrchard(string directory) method.

May 26, 2011 at 2:22 AM

The string after "But was: " got cut off. It is meant to say:

"<!DOCTYPE html>

May 26, 2011 at 6:36 PM

Yeah, as the proverb says: if you have a pattern matching problem, you can solve it with regular expressions. And then you have two problems.

May 26, 2011 at 9:21 PM

There is nothing wrong with my syntax or assumption that there will be a string matching on that particular page (at least not as yet and as far as I know), as

I copied the first scenario from the Blogs feature file into my Customer feature file and it also fails to run, resulting in the same error. Yet running the Blogs feature

the same scenario runs fine?

Am I missing something in my setup? Maybe something I need to add to some of the Hosting or Binding classes in Orchard.Specs or to some kind of SpecFlow/NUnit config somewhere?

May 26, 2011 at 9:23 PM

Missing the step definition maybe? The error message you gave seems incomplete.

May 27, 2011 at 12:09 AM

OK, I just noticed when I loaded the Orchard.Specs project in NUnit my Customer.feature file did not load with it.  Any ideas on how to set up Orchard.Specs/SpecFlow/NUnit to work with custom modules?

May 27, 2011 at 12:10 AM

You should have your own project rather than attempt to piggyback on the existing one.

May 27, 2011 at 2:16 AM

It just seemed easier to use your existing project as it already has all of the setup of a clean instance of Orchard for running the SpecFlow tests, which I didn't want to implement myself as I would have to learn about all the inner workings of Orchard and copying all the Orchard.Specs test scaffolding classes over to my own project was a pain in the butt when it came to adding in all the references (unless there's some easy way of doing this with Visual Studio that I don't know about?)

Thing is I'm finding it difficult to test my modules without having subsystems/components of Orchard running, since things like handlers and drivers don't seem to make sense outside of an Orchard context.

Some documentation of how to do testing for module development with Orchard would be really helpful if someone could spare time to do this, or maybe I am just misunderstanding how to do testing in a web context as I am new to web development, C# and mvc, having just come from a fairly basic Java development background at university.

Anyway, I discovered the solution for the regression in functionality for my forum module anyway - setting higher priorities in my custom routes, which I hit upon after seeing route priorities mentioned in a recent forum discussion here.

Links that had previously worked had stopped working after I implemented new functionality and whenever I fixed one link by trial and error editing of my routes, views and controller actions another link would break. I was confused as clicking each broken link would result in a completely different error such as 'object reference not set to an instance of an object' and http 404 not found errors. Now that I realize it wasn't finding my correct controller actions this behaviour makes perfect sense. If I knew how mvc routing worked maybe I wouldn't have run into all of this trouble.  So sorry for wasting your time.

Still any tips or directions to information on how to test for Orchard/Web development would be handy, thanks.

May 27, 2011 at 4:39 PM

I would like to second PiechartPirate's comment.

If someone were to do a blog post and or a video walking through in detail how to setup a unit test and submit it to the core team I would greatly appreciate it.

In particular whne I uncover an "issue" I would love to be able to submit a unit test that fails rather than just posting to the issue tracker. The only reson I don't do it currently is that I just don't know how ;-)

An added benefit would be that if the unit test were accebpted by the core team I could be assured that the "issue" would not resurface in a future build without being caught.


May 27, 2011 at 5:05 PM

There are two things here: testing your own module, and adding tests for Orchard itself. Both are clearly in need of more documentation contributions.

May 27, 2011 at 5:43 PM
Perhaps a convention can be developed that defines hooks and channels between Orchard and 'Orchard-specific modules' - by that I mean, mvc3 modules built from the ground up to exist within Orchard. Wouldn't something like that lead to significantly better behaved interactions between Orchard and future Orchard-specific modules?

Are there known locations of Orchard Core that need more SpecFlow tests?
Sep 29, 2011 at 2:46 PM

I've set up a new test project for testing our own modules using SpecFlow. I've referenced Orchard.Specs to be able to use all the implementation goodies that have already been provided, and I've subclassed OrchardSiteFactory and WebAppHosting to add some custom code.

Now, trying to do something fairly simply, I've encountered the following problem: after loading a custom module, a table that I've defined in that module's Migrations.cs cannot be found and thus my custom Part could not be loaded. The code responsible for installing a module in the Orchard test instance is the following from OrchardSiteFactory.cs:


[Given(@"I have installed ""(.*)\""")]
public void GivenIHaveInstalled(string name) {
    var webApp = Binding<WebAppHosting>();



I went to search for some usages of this method and found the Media.feature and MultiTenancy.feature which run just fine (where Orchard.MultiTenancy does not contain a Migrations.cs).

When I run my own scenario


Scenario: I visit the search page for the first time
	Given I have installed Orchard
		And I have installed "Teamaton.Discoverize"
		And I have installed "Discoverize.GoogleMap"
		And I have installed "Teamaton.Test"
		And I have installed "Teamaton.Search"
	When I go to "TeamSearch"
	Then I should see "..."

even doing this inside the Orchard.Specs project, I'm getting the following error:

The specified table does not exist. [ Teamaton_Search_SearchSettingsPartRecord ]

To me this looks like somehow my custom module's Migration.cs is not being executed. How do I get my custom module's Migration.cs to be run when installing that module within a SpecFlow test?