Client Side

Topics: Customizing Orchard
Dec 1, 2011 at 12:19 AM

Hello Orchard Developers,

I am looking for some advice on how to solve a problem I am up against. I have a backend database that I have built a sync tool (coded in the Orchard framework - it is it's own module) to bring items over into Orchard. The sync tool works great and the Content Items are created successfully.  Once I have the inital sync done, Orchard will handle the daily sync on it's own with no problem.

Here is the problem, I have to sync 3 years of data over to Orchard.  I started the process by pressing the sync button one day at a time in Orchard... and am not a big fan of doing this and it is also not replicable if I have to repeat this process. The problem I run into is that the transaction times out on the "Save" if I try to do multiple days (makes sense to me that it would and I don't really want to mess around with trying to change the timeout anywhere).

Here is the question, is there a way to break this down into multiple transactions easily.  :

  1. Could I make a client side script that will emulate what I am doing using Orchard.exe (i.e. are there any code examples out there)? 
    1. I have not tried integrating Orchard.exe into a solution yet... but maybe there is a way to have it launch this off one transaction at a time
  2. Could a single HTTP Post be broken into multiple transactions?
    1. This would allow me in my code to close out a transaction for every day but seems impossible since Orchard Explicitly wraps the Http post event in a transaction rollback process
      1. Does it seem like I could wrap the whole thing in a TransactionScope like this (I have done this once before when using an external database SQL command but not sure if it works within Orchard when I am trying to use the Orchard framework):
      2. // Need to use a new TransactionScope to not be confused with one Orchard already has open
        using (TransactionScope scope = new TransactionScope(TransactionScopeOption.Suppress)) {
             // Create my content for the particular day
        }
I just thought of the TransactionScope while writing this and maybe it will work.  I value your feedback and thoughts on this question though as there is probably an "easy" way that I am not aware of.
Thanks...

 

I have to call it a night - I am posting a request on codeplex to see if someone out there has a good suggestion.  I have an issue in that the sync should really occur over a single day and then move to the next day.  The way that “Transaction” management in Orchard is done is such that all the transactions are committed at one time.  This is not a big deal on a single day sync, but it is a big deal if you are trying to “catch up”.  We need to have this ability anyway in case we do an overhaul later on.  Here is what it boils down to, I need a tool that opens a new transaction,  runs the sync tool for a one day period, commits the transaction, and opens a new transaction for the next day.  I suspect this will be some type of client side tool that initializes the system and that the web-side will take over from there.  This is NOT proposing a new coding strategy, the strategy works fine. I just need a way to have it automate (as I cannot keep going to the site and pressing submit).  Please throw that in the notes and we will see what other users have to say.

 

Dec 1, 2011 at 12:22 AM

You can use an IBackgroundTask - it'll run approx. once every minute where you can check (with an ISingletonDependency) if a sync is still running, and if not start a new one. So you'll have a fresh TransactionScope each time the IBackgroundTask runs.

Dec 1, 2011 at 12:45 AM
Edited Dec 2, 2011 at 2:01 AM

Thanks randompete, that is a great idea.

I looked for all the instances of IBackgroundTask I could find in the source code and I saw there is also a “Scheduling” module called Orchard.Core.Scheduling that implements this interface. I see there is a “ScheduledTaskExecutor” service in there that maybe I can utilize for scheduling my daily syncs. While I think there are better examples for the current problem I posted, perhaps you have answered a second question I had not asked yet in the best way to schedule the sync for a daily occurrence. Well, I will have to look more into this and I will post back to this discussion with what I find in case someone else is also looking to do something similar. Thanks for your speedy reply.

Dec 2, 2011 at 2:11 AM

Hello randompete and anyone else who can assist.  I have gone ahead and tried to work down the road of IBackgroundTask and am encountering an error with Transaction Scope.  It seems that the IBackgroundTask is not enough to get the process to run in it's own Transaction Scope.  Possibly I am missing something but as of right now, I get an issue with the server telling me I need to enable MSDTC (which I don't want to do) and no results.

I have gone ahead and tried to wrap the process in it's own transaction scope (see code below) and while this throws no errors, the data is not actually written back to the database.  Interestingly enough, the Content Items are actually created (in other words, the actual Content Id's are created but no data is populated/updated in the database.  It almost seems as though there is no persistence in how my content is being added into the database.  Here are some code snippets:

This first one is for the implemented IBackgroundTask (implements the Sweep method):

public void Sweep() {
            // Check to see if we are already in an initialize sync process - wait until done to continue
            //   This may be done by ISingletonDependency
            // Check to see if we are in initialize sync mode
            // If we are in initialize sync mode, grab the StartDate from cache
            // If we are in initialize sync mode, grab the SyncMode from cache
            // If we are in initialize sync mode, grab the SyncType from cache
            // We will always set the Maximum Days to Pull equal to 1
            bool isInitializeSync = _syncService.IsInitializeSync();

            if (isInitializeSync) {
                using (TransactionScope scope = new TransactionScope(TransactionScopeOption.Suppress)) {
                    // Retrieve latest Sync Date from the database
                    DateTime startDate = _syncService.GetLastSyncDate();
                    SyncMode syncMode = (SyncMode)HttpRuntime.Cache["SyncMode"];
                    SyncType syncType = (SyncType)HttpRuntime.Cache["SyncType"];
                    int maximumDaysToPull = 1;

                    _syncService.RunSync(startDate, syncMode, syncType, maximumDaysToPull);

                    if (startDate >= DateTime.Now) {
                        // We are done - go ahead and end this process
                        _syncService.ResetInitializeSync();
                    }

                } // End of using statement

            } // End of if statment

        }   

Here is the actual snippet under "RunSync" method above that creates the new ContentItem and then updates it (well, the update does not actually get recorded to the database - going back to my belief that it is not persisted/committed back to the database even though the content item itself is created):

contentItem_Item = Services.ContentManager.Create("Item");  // Create the new content item (which will bring in all the connected parts)
UpdateItem(item, contentItem_Item, UpdateMode.Create, syncType);

I am not sure where to go from here, it seems like from the earlier post I would not need to utilize my own transaction scope (as this was handled by IBackgroundTask).  If I do need to manage it, am I disposing something wrong such that the data for the ContentItem is not going back to the database.  Thanks everyone for your feedback and support.

Dec 2, 2011 at 3:06 AM

You shouldn't need to create your own transaction scope, and it's likely causing problems.

What was the original code and error you encountered? I suspect the problem lies in your SyncService. If it's an ISingletonDependency then its own instance of ContentManager will have an expired transaction scope.

Dec 2, 2011 at 3:16 AM

Thanks for the prompt reply randompete. More than likely I don't understand the ISingletonDependency. When I run without the Transaction Scope I receive the following error on the first datbase call I make (made through IRepository):

MSDTC on server '#mycomputername#' is unavailable.

My research on this indicates a transaction scope issue (i.e. that I am trying to make a transaction while one is already open... something like that) and I don't want to enable MSDTC as I don't think that is what I need to be doing. 

Regarding ISingletonDependency, all I am doing right now is the fowllowing code block (this is the whole class in full:

public class InitializeSyncService : IInitializeSyncService, IBackgroundTask {
        private readonly ISyncService _syncService;

        public InitializeSyncService(
            ISyncService syncService
            ) {
            _syncService = syncService;
        }

        // This is from the IBackgroundTask and seems to be checked every minute
        public void Sweep() {
            // Check to see if we are already in an initialize sync process - wait until done to continue
            //   This may be done by ISingletonDependency
            // Check to see if we are in initialize sync mode
            // If we are in initialize sync mode, grab the StartDate from cache
            // If we are in initialize sync mode, grab the SyncMode from cache
            // If we are in initialize sync mode, grab the SyncType from cache
            // We will always set the Maximum Days to Pull equal to 1
            bool isInitializeSync = _syncService.IsInitializeSync();

            if (isInitializeSync) {
                //using (TransactionScope scope = new TransactionScope(TransactionScopeOption.Suppress)) {
                    // Retrieve latest Sync Date from the database
                    DateTime startDate = _syncService.GetLastSyncDate();
                    SyncMode syncMode = (SyncMode)HttpRuntime.Cache["SyncMode"];
                    SyncType syncType = (SyncType)HttpRuntime.Cache["SyncType"];
                    int maximumDaysToPull = 1;

                    _syncService.RunSync(startDate, syncMode, syncType, maximumDaysToPull);

                    if (startDate >= DateTime.Now) {
                        // We are done - go ahead and end this process
                        _syncService.ResetInitializeSync();
                    }

                //} // End of using statement

            } // End of if statment

        }   
    }

So ISingletonDependency is implemented on the IInitializeSyncService as seen here:

public interface IInitializeSyncService : IDependency, ISingletonDependency {
        
    }

Unfortunately, this is likely where I am not understanding how ISingletonDependency can be used.  I tried looking through existing Orchard source code but cont not fully understand it's value. Thanks again for helping me out.

Dec 2, 2011 at 3:51 AM

Well, something can't be both IDependency and ISingletonDependency. The difference is as follows:

IDependency: An instance will be created for each and every web request, or background process, etc.

ISingletonDependency: An instance will be created per *shell*. This means one instance is created for the entire lifetime of your application.

So if you inject any database accessors into your ISingletonDependency, you will find they very soon lose their connection, and you won't get a new version.

Dec 2, 2011 at 12:51 PM

Hello again, thanks for your final comments - they helped me realize a couple of things and I have now resolved my problem. 

I ended up removing the ISingletonDependency and retained the IDependency and then everything worked as I desired. I did try it using just the ISingletonDependency and this caused the same error I was seeing earlier.  I suspect that using IRepository is NOT the same as using ContentManager when it comes to Transaction Scope.  I know they are not the same for data access, but I suspect that IRepository access is more of a straight SQL call (and I also do not believe that roll backs are in effect with IRepository as they operate immediately on the database level - might be wrong here and will need to do some testing).  Therefore, they would need to operate within an instance whereas the ContentManager approach would set up it's own Transaction Scope and thus complete with no issue under ISingletonDependency (and would also have rollback as the commands are processed in one transaction scope).

I am not sure my understanding is perfect (so if anyone else reads this and can elighten me further it is appreciated) though at this time it is a working knowledge and I am sure I will understand more the longer I am in Orchard.

Thanks again randompete for all your help.

Dec 2, 2011 at 4:10 PM

As far as I know, Content Manager will also have problems in an ISingletonDependency, it is also affected by transaction scope.