NHibernate 2nd level caching with AzureCacheProvider desynchronization issues

Topics: Core, Customizing Orchard, General, Writing modules
Dec 2, 2014 at 11:49 AM
Edited Dec 2, 2014 at 12:30 PM
We activated the feature "Orchard.Azure.DatabaseCache" to be able to use multiple instances for our website (hosted on an Azure cloud service). Since then (still using only 1 instance), we have synchronization issues when updating records.

Here is a description of the records and the scenario to understand what's happening. To summarize, we have teachers and students, each of them are associated with many of the other type. Accessing a student's teachers (and vice versa) is done lazily.
public class TeacherPart : ContentPart<TeacherPartRecord>
{
    internal LazyField<IList<StudentPart>> _students = new LazyField<IList<StudentPart>>();
    public IList<StudentPart> Students { get { return this._students.Value; } }
}

public class StudentPart : ContentPart<StudentPartRecord>
{
    internal LazyField<IList<TeacherPart>> _teachers = new LazyField<IList<TeacherPart>>();
    public IList<TeacherPart> Teachers { get { return this._teachers.Value; } }
}

public class TeacherPartRecord : ContentPartRecord
{
    [CascadeAllDeleteOrphan]
    public virtual IList<StudentTeacherAssociationRecord> StudentTeacherAssociations { get; set; }
}

public class StudentPartRecord : ContentPartRecord
{
    [CascadeAllDeleteOrphan]
    public virtual IList<StudentTeacherAssociationRecord> StudentTeacherAssociations { get; set; }
}

public class StudentTeacherAssociationRecord
{
    public virtual int Id { get; set; }
    public virtual StudentPartRecord StudentPartRecord { get; set; }
    public virtual TeacherPartRecord TeacherPartRecord { get; set; }
}
We also have a handler which initializes the lazy fields:
this.OnLoading<TeacherPart>((context, part) => {
    part._students.Loader(() => {
        var studentIds = part.Record.StudentTeacherAssociations.Select(x => x.StudentPartRecord.Id);
        return _contentManager.GetMany<StudentPart>(studentIds, VersionOptions.Published, QueryHints.Empty);
    })
});
The problem: when we delete an association between a teacher and a student with:
StudentTeacherAssociationRecord record = GetAssociationRecord(teacherId, studentId);
_repository.Delete(record);
Then, for a few minutes, any access to teacher.Students fails with an nhibernate "not found" exception. After a few minutes, the error goes away and the teacher gets its students back and up to date. If on the contrary we add a new association, there is no error but the association does not appear before a long while (longer than when removing the association). Note that when the feature "Orchard.Azure.DatabaseCache" is disabled, we don't have any problem with this code.

It seems the NHibernate 2nd cache level (when using the azure provider) takes a while to update itself, even though we only have 1 instance! What could be the problem?