Bulk Import of Data Failing on Second Insert

Topics: Writing modules
Nov 4, 2011 at 11:31 AM

Creating a module that looks to a WCF service which exports a list of items from a third party database, the module uses the following process:

  1. Load the data from the WCF Service [Working]
  2. Go through the data row by row and compare with the AdvertId field for a custom content part [Working]
  3. If it exists in both Orchard and the WCF Data - add it to a list of content parts to update [Working]
  4. If it exists in Orchard and not the WCF Data - add it to a list of content parts to delete [Working]
  5. If it does not exist in Orchard and does exist in the WCF Data - add it to a list of items to add to Orchard [Working]
  6. Perform Additions [Failing]
  7. Perform Removals [Working]
  8. Perform Updates [Don't know]

The Perform additions step is failing with the message and stack trace below, I don't know how to debug this to identify why the key is duplicated.

Violation of PRIMARY KEY constraint 'PK__Common_C__3214EC0744FF419A'. Cannot insert duplicate key in object 'dbo.Common_CommonPartRecord'. The duplicate key value is (7015).
The statement has been terminated.

[SqlException (0x80131904): Violation of PRIMARY KEY constraint 'PK__Common_C__3214EC0744FF419A'. Cannot insert duplicate key in object 'dbo.Common_CommonPartRecord'. The duplicate key value is (7015).
The statement has been terminated.]
   System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection) +2073502
   System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection) +5064460
   System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning() +234
   System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj) +2275
   System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString) +215
   System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async) +987
   System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result) +162
   System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe) +178
   System.Data.SqlClient.SqlCommand.ExecuteNonQuery() +137
   NHibernate.AdoNet.AbstractBatcher.ExecuteNonQuery(IDbCommand cmd) +297
   NHibernate.AdoNet.NonBatchingBatcher.AddToBatch(IExpectation expectation) +17
   NHibernate.Persister.Entity.AbstractEntityPersister.Insert(Object id, Object[] fields, Boolean[] notNull, Int32 j, SqlCommandInfo sql, Object obj, ISessionImplementor session) +497

[GenericADOException: could not insert: [Orchard.Core.Common.Models.CommonPartRecord#7015][SQL: INSERT INTO Common_CommonPartRecord (OwnerId, CreatedUtc, PublishedUtc, ModifiedUtc, Container_id, Id) VALUES (?, ?, ?, ?, ?, ?)]]
   NHibernate.Persister.Entity.AbstractEntityPersister.Insert(Object id, Object[] fields, Boolean[] notNull, Int32 j, SqlCommandInfo sql, Object obj, ISessionImplementor session) +729
   NHibernate.Persister.Entity.AbstractEntityPersister.Insert(Object id, Object[] fields, Object obj, ISessionImplementor session) +166
   NHibernate.Action.EntityInsertAction.Execute() +181
   NHibernate.Engine.ActionQueue.Execute(IExecutable executable) +117
   NHibernate.Engine.ActionQueue.ExecuteActions(IList list) +92
   NHibernate.Event.Default.AbstractSaveEventListener.PerformSaveOrReplicate(Object entity, EntityKey key, IEntityPersister persister, Boolean useIdentityColumn, Object anything, IEventSource source, Boolean requiresImmediateIdAccess) +167
   NHibernate.Event.Default.AbstractSaveEventListener.PerformSave(Object entity, Object id, IEntityPersister persister, Boolean useIdentityColumn, Object anything, IEventSource source, Boolean requiresImmediateIdAccess) +323
   NHibernate.Event.Default.AbstractSaveEventListener.SaveWithGeneratedId(Object entity, String entityName, Object anything, IEventSource source, Boolean requiresImmediateIdAccess) +130
   NHibernate.Event.Default.DefaultSaveOrUpdateEventListener.SaveWithGeneratedOrRequestedId(SaveOrUpdateEvent event) +27
   NHibernate.Event.Default.DefaultSaveEventListener.SaveWithGeneratedOrRequestedId(SaveOrUpdateEvent event) +20
   NHibernate.Event.Default.DefaultSaveOrUpdateEventListener.EntityIsTransient(SaveOrUpdateEvent event) +63
   NHibernate.Event.Default.DefaultSaveEventListener.PerformSaveOrUpdate(SaveOrUpdateEvent event) +65
   NHibernate.Event.Default.DefaultSaveOrUpdateEventListener.OnSaveOrUpdate(SaveOrUpdateEvent event) +191
   NHibernate.Impl.SessionImpl.FireSave(SaveOrUpdateEvent event) +263
   NHibernate.Impl.SessionImpl.Save(Object obj) +255
   Orchard.Data.Repository`1.Create(T entity) in C:\Project\orchard-rtm\src\Orchard\Data\Repository.cs:96
   Orchard.Data.Repository`1.Orchard.Data.IRepository<T>.Create(T entity) in C:\Project\orchard-rtm\src\Orchard\Data\Repository.cs:36
   Orchard.ContentManagement.DefaultContentManager.Create(ContentItem contentItem, VersionOptions options) in C:\Project\orchard-rtm\src\Orchard\ContentManagement\DefaultContentManager.cs:421
   Orchard.ContentManagement.ContentCreateExtensions.Create(IContentManager manager, String contentType, VersionOptions options, Action`1 initialize) in C:\Project\orchard-rtm\src\Orchard\ContentManagement\ContentExtensions.cs:64
   Orchard.ContentManagement.ContentCreateExtensions.Create(IContentManager manager, String contentType, VersionOptions options) in C:\Project\orchard-rtm\src\Orchard\ContentManagement\ContentExtensions.cs:51
   Makemedia.Classifieds.Services.ClassifiedsListingService.Import(PlatformClassifiedsListing listing) +73
   Makemedia.Classifieds.Services.SyncronizationService.PerformAdditions(Queue`1 toAdd) +76
   Makemedia.Classifieds.Services.SyncronizationService.PerformSyncronization(IEnumerable`1 currentLiveListings) +1127
   Makemedia.Classifieds.Services.SyncronizationService.SyncronizeWithPlatform() +258
   Makemedia.Classifieds.Controllers.AdminController.Syncronize() +45
   lambda_method(Closure , ControllerBase , Object[] ) +62
   System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters) +17
   System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary`2 parameters) +208
   System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary`2 parameters) +27
   System.Web.Mvc.<>c__DisplayClass15.<InvokeActionMethodWithFilters>b__12() +55
   System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodFilter(IActionFilter filter, ActionExecutingContext preContext, Func`1 continuation) +263
   System.Web.Mvc.<>c__DisplayClass17.<InvokeActionMethodWithFilters>b__14() +19
   System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodFilter(IActionFilter filter, ActionExecutingContext preContext, Func`1 continuation) +263
   System.Web.Mvc.<>c__DisplayClass17.<InvokeActionMethodWithFilters>b__14() +19
   System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodFilter(IActionFilter filter, ActionExecutingContext preContext, Func`1 continuation) +263
   System.Web.Mvc.<>c__DisplayClass17.<InvokeActionMethodWithFilters>b__14() +19
   System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodWithFilters(ControllerContext controllerContext, IList`1 filters, ActionDescriptor actionDescriptor, IDictionary`2 parameters) +191
   System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName) +343
   System.Web.Mvc.Controller.ExecuteCore() +116
   System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext) +97
   System.Web.Mvc.ControllerBase.System.Web.Mvc.IController.Execute(RequestContext requestContext) +10
   System.Web.Mvc.<>c__DisplayClassb.<BeginProcessRequest>b__5() +37
   System.Web.Mvc.Async.<>c__DisplayClass1.<MakeVoidDelegate>b__0() +21
   System.Web.Mvc.Async.<>c__DisplayClass8`1.<BeginSynchronous>b__7(IAsyncResult _) +12
   System.Web.Mvc.Async.WrappedAsyncResult`1.End() +62
   System.Web.Mvc.<>c__DisplayClasse.<EndProcessRequest>b__d() +50
   System.Web.Mvc.SecurityUtil.<GetCallInAppTrustThunk>b__0(Action f) +7
   System.Web.Mvc.SecurityUtil.ProcessInApplicationTrust(Action action) +22
   System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult) +60
   System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result) +9
   Orchard.Mvc.Routes.HttpAsyncHandler.EndProcessRequest(IAsyncResult result) in C:\Project\orchard-rtm\src\Orchard\Mvc\Routes\ShellRoute.cs:148
   System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +8963149
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +184

Coordinator
Nov 4, 2011 at 4:53 PM

Not seeing any of your code, it's impossible to guess what could have gone wrong.

Nov 18, 2011 at 4:56 PM

Indeed, Sebastien has the code.