Large Content Item Import Problems

Topics: General
Aug 30, 2012 at 6:18 PM

I have been trying to import about 18,000 records into the Orchard DB to be used for city information in longtail SEO search results. After trying to import multiple times and having the website timeout many times, I started to batch the imports. Unfortunately, the more I import into the DB the smaller the batches need to be. I'm down to 100 records per batch with over 8,000 records left to import. I have another 22,000 records after this for zip related data. Unfortunately, this is not going to be possible with Orchard and it's causing me to have to look at other options.

I have also tried to use the Orchard CLI using this command "import file /Filename:<file>" and this gets the timeout as well.

Does anyone have a workaround for me so that I can import a large number of content items into the DB? This is a required function for us as we really need the data for longtail SEO purposes.

Aug 30, 2012 at 8:10 PM

Although probably not the nicest solution, as a last resort you could try and insert records into the database directly, without using the content manager. You could try using IRepository, or grabbing the NHibernate session directly. Or even do it from an external process and use something like EF, or ADO.NET for that matter.

Aug 30, 2012 at 9:54 PM

I wonder if there's something in Web.Config you can do to increase page timeout values.  And/Or modifying settings.txt and increase your timeout in your DB connection string.

Aug 30, 2012 at 10:42 PM

Good point. If the timeout is caused by ASP.NET, then it could help to increase the executionTimeOut in web.config.