NHibernate "randomly" exceptions and DB corrupted

Topics: Core, Troubleshooting
Jul 12, 2012 at 4:32 PM
Edited Nov 19, 2012 at 4:08 PM

Sometimes (0.5% of cases), I get NHibernate exceptions such as:

NHibernate.Util.ADOExceptionReporter - System.Data.SqlClient.SqlException (0x80131904): A transport-level error has occurred when receiving results from the server. (provider: Session Provider, error: 19 - Physical connection is not usable)

NHibernate.Util.ADOExceptionReporter - System.InvalidOperationException: Invalid attempt to call Read when reader is closed.

NHibernate.Util.ADOExceptionReporter - System.InvalidOperationException: Invalid operation. The connection is closed.

And after running around 6 hours, my DB gets corrupted because of the following lines:

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (1, 1, 1, 1, NULL, NULL)

instead of

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (1, 1, 1, 1, NULL, 1)

and new lines appeared:

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (24, 1, 1, 1, NULL, 19)

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (25, 1, 1, 1, NULL, 20)

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (26, 1, 1, 1, NULL, 18)

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (27, 1, 1, 1, NULL, 17)

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (28, 1, 1, 1, NULL, 22)

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (29, 1, 1, 1, NULL, 23)

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (30, 1, 1, 1, NULL, 21)

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (31, 1, 1, 1, NULL, 26)

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (32, 1, 1, 1, NULL, 25)

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (33, 1, 1, 1, NULL, 24)

INSERT [dbo].[Orchard_Framework_ContentItemVersionRecord] ([Id], [Number], [Published], [Latest], [Data], [ContentItemRecord_id]) VALUES (34, 1, 1, 1, NULL, 27)

and:

INSERT [dbo].[Orchard_Framework_ContentItemRecord] ([Id], [Data], [ContentType_id]) VALUES (17, NULL, 1)INSERT [dbo].[Orchard_Framework_ContentItemRecord] ([Id], [Data], [ContentType_id]) VALUES (18, NULL, 1)INSERT [dbo].[Orchard_Framework_ContentItemRecord] ([Id], [Data], [ContentType_id]) VALUES (19, NULL, 1)

INSERT [dbo].[Orchard_Framework_ContentItemRecord] ([Id], [Data], [ContentType_id]) VALUES (20, NULL, 1)INSERT [dbo].[Orchard_Framework_ContentItemRecord] ([Id], [Data], [ContentType_id]) VALUES (21, NULL, 1)INSERT [dbo].[Orchard_Framework_ContentItemRecord] ([Id], [Data], [ContentType_id]) VALUES (22, NULL, 1)

INSERT [dbo].[Orchard_Framework_ContentItemRecord] ([Id], [Data], [ContentType_id]) VALUES (23, NULL, 1)INSERT [dbo].[Orchard_Framework_ContentItemRecord] ([Id], [Data], [ContentType_id]) VALUES (24, NULL, 1)INSERT [dbo].[Orchard_Framework_ContentItemRecord] ([Id], [Data], [ContentType_id]) VALUES (25, NULL, 1)

INSERT [dbo].[Orchard_Framework_ContentItemRecord] ([Id], [Data], [ContentType_id]) VALUES (26, NULL, 1)INSERT [dbo].[Orchard_Framework_ContentItemRecord] ([Id], [Data], [ContentType_id]) VALUES (27, NULL, 1)

Maybe you'll find a clue if I specify that I changed the connection string to:

Data Source=PC-DE-GWENA;Initial Catalog=Theia;Integrated Security=True;Pooling=false;Enlist=false;MultipleActiveResultSets=True

before that I used:

Data Source=PC-DE-GWENA;Initial Catalog=Theia;Integrated Security=True

and I got the following errors several times before the server crashed :( and I had to restart it:

The transaction associated with the current connection has completed but has not been disposed. The transaction must be disposed before the connection can be used to execute SQL statements.

Thanks for helping because DB corruption is a real issue for us.

Coordinator
Jul 12, 2012 at 6:25 PM

Please file a bug with repro steps.

Developer
Jul 13, 2012 at 12:35 PM
Edited Jul 14, 2012 at 2:49 AM

In addtition to providing repro steps as Bertrand asked, please describe your environment in more detail (what version of Orchard do you use, what custom modules do you have, is the db on the same machine or not, do you use multiple tenants, run in webfarm and so on). And of course - try to repro the problem using vanilla Orchard instance - usually the custom modules are the cause of such problems.

Nov 19, 2012 at 3:58 PM
Edited Nov 19, 2012 at 4:00 PM

I had some time for investigating on my issue and I found a fix BUT I'm not fully happy with it.

Indeed, it requires to create a new session each time ISessionLocator::For(Type entityType) is called and I feel like it's not the good way to do things. I also had to change ISessionLocator dependency to  ISingletonDependency. Thus, with the following changes, I don't get errors anymore:

in ISessionLocator.cs, replace 

public interface ISessionLocator : IDependency
by
public interface ISessionLocator : ISingletonDependency
in SessionLocator.cs, replace

 

public ISession For(Type entityType) {
            Logger.Debug("Acquiring session for {0}", entityType);

            if (_session == null) {

                var sessionFactory = _sessionFactoryHolder.GetSessionFactory();

                _transactionManager.Demand();

                Logger.Information("Openning database session");
                _session = sessionFactory.OpenSession(new SessionInterceptor(this));
            }
            return _session;
        }

 

by

 

public ISession For(Type entityType) {
            Logger.Debug("Acquiring session for {0}", entityType);

            var sessionFactory = _sessionFactoryHolder.GetSessionFactory();

            _transactionManager.Demand();

            Logger.Information("Openning database session");
            _session = sessionFactory.OpenSession(new SessionInterceptor(this)); // always open a new session

            return _session;
        }

 

I guess, the issue was because of session sharing through different threads. Do you have any suggestions about avoiding to create a new session for each ISessionLocator::For(Type entityType) call? How can I make a thread-safe session or a session per thread? That would be better than using so many sessions.

Thanks.

Coordinator
Nov 23, 2012 at 3:26 AM

Why do you think Session Locator should be a singleton? Why do you think it's a problem to create a session per request? Or did I misunderstand?

Nov 23, 2012 at 8:53 AM

Oh no, it's not what I meant. I totally agree with Session Locator and session being one per request respectively but I didn't find another way to fix my issue than what I described above.

Actually, meanwhile, I noticed that my fix caused another bug. If in a same request I update a record and then I query it, I get the non updated version.

Now, I try to understand the logic that you put in place in ShellContainerFactory. I know nothing about Autofac. I'm learning.

I would like to have one session locator and one session per request but I don't understand why when the server receives multiple requests from the same client at the same time I get exceptions (those in my first post of this thread).

 

Coordinator
Nov 23, 2012 at 9:01 AM

I don't know. Did you file a bug?

Jul 22, 2013 at 10:35 PM
I have a similar error. I am using 1.5.1. Has anybody found the fix?