[en] Zyan and parallel working (solved)

Topics: Technical Support
Dec 20, 2013 at 7:56 AM
hello,

I need a solution for simultanus Access on some Functions. We are using much EF Routines on server-side. But EF 4.4 (ObjectContext) is not thread-safe. So we have no Problems with data Transfer, if all commands come Serial.

But come 2 or more commands at the time where the EF is running, then come up Transactions Errors and big deadlocks on MS-SQL

How can I the incomming commands use a Queue?
Coordinator
Dec 20, 2013 at 9:55 AM
Edited Dec 20, 2013 at 10:20 AM
Hi Michael,

Concurrent (not serialized!) data access is essential in any application server.
If you serialize all requests to your database using a queue, you'll get a bottleneck.
Consider an example of a request queue:

Query1 (100ms) -> Query2 (10ms) -> Query3 (40000ms) -> Query4 (20ms) ->...

You see, any long-running query degrades performance of your application badly, affecting all users.
Fast and efficient Query4 will freeze until the long-running Query3 is processed, which is simply unacceptable.
So, I strongly advise against serializing the data access using queues.

To enable parallel data access in your application, do the following:
  1. Don't use singletons. All your services should be single-call (instantiated per request).
  2. Instead of serializing the incoming commands, create separate ObjectContexts instances for all concurrent requests. Every incoming remote call to your application should create its own ObjectContext and release it when the call ends. To hook up the incoming calls, use ZyanHost events: BeforeInvoke, AfterInvoke and InvokeCanceled.
Here is an example (pseudocode):
// create component host, register components, hook up events
ZyanHost = new ZyanComponentHost(...);
ZyanHost.BeforeInvoke += BeforeInvokeHandler;
ZyanHost.AfterInvoke += AfterInvokeHandler;
ZyanHost.InvokeCanceled += InvokeCanceledHandler;

...
// Thread-static field to hold a separate ObjectContext instance for every incoming request
[ThreadStatic]
private static ObjectContext currentObjectContext; 

// Event handlers to handle incoming calls
private void ZyanHost_BeforeInvoke(object sender, BeforeInvokeEventArgs args)
{
     currentObjectContext = new ObjectContext(...);
     // TODO: start a new transaction for the current request
     // In my application, I use TransactionScope to manage transactions
}

// Don't forget to close the database connection! Otherwise, you’ll get a leak
private void ZyanHost_AfterInvoke(object sender, AfterInvokeEventArgs args)
{
    // TODO: commit the transaction for the current request
    currentObjectContext.Dispose(); // don’t know how it’s done in EF, feel free to fix this code
}

// Even if your server call failed with an exception, you need to close your database connection
private void ZyanHost_ClientLoggedOn(object sender, LoginEventArgs args)
{
    // TODO: rollback the current transaction for the current request
    currentObjectContext.Dispose(); // don’t know how it’s done in EF, feel free to fix this code
}
In your EF queries, never use shared ObjectContext.
All queries must use currentObjectContext (which is stored in thread-local storage) to access the data.

UPD. Of course, you can use other techniques to use separate database connections for concurrent calls. For example, you can create and dispose of the connection every time you need to access the database:
using (var db = new ObjectContext()) // I'm not sure EF supports this, but it's quite likely
{
    // do the query and get yout ObjectContext disposed of when you leave the current "using" block
}
Dec 20, 2013 at 3:41 PM
Thank you,

I think, some of your Suggestion can help me. I will try out now.

Have you a working sample-application (like your Chat or so) that Show the first mentioned way? I can read and translate your idea - but I not realy understand it (what is on client-side , what is on server-side)

Thanks for all
Coordinator
Dec 20, 2013 at 4:34 PM
Edited Dec 20, 2013 at 4:34 PM
Hi,

I don't currently have a working application (we use BLToolkit ORM instead of EF).
But I'll try to prepare a small sample for you.
 
I not realy understand it (what is on client-side , what is on server-side)
 
Everything I mentioned is on the server-side.
Coordinator
Dec 20, 2013 at 11:21 PM
Edited Dec 20, 2013 at 11:43 PM
Hi Michael,

I've just made an example demonstrating the concurrent data access with Zyan and Entity Framework 6.
Please download it here: https://zyan.codeplex.com/downloads/get/770458

The archive contains two small executables, eftest1.exe and eftest2.exe.
The first one simply checks whether EF6 with SqlCe4 provider works on your machine.
It creates a database file (if it doesn't exist) and inserts a row into a table.

The second executable is a client-server application built with Zyan.
Run it for the first time to start the database server and wait until it prints that it's ready.
Run it for the second time to check the concurrent data access.

The client connects to the server and spawns multiple threads to access the database.
It creates multiple rows in a table in parallel. Then it selects all rows from the table to make sure it worked.
The program demonstrates using thread-static data contexts and transaction scopes.

Hope it helps.
Regards, Alexey.

P.S. I've created one more example that demonstrates exception handling.
When your server method throws an exception, the transaction is automatically rolled back.
Using this technique you basically get data consistency for free.
Here is the third sample application: the source code and the configuration file.
Marked as answer by yallie on 12/20/2013 at 3:21 PM
Dec 21, 2013 at 9:25 AM
Hey Alexey,

I have look at your sample. Now I think I understand the right way. I think that I have not do much changes in my code. Most of them Is right - but I will try it and inform you

In my application we have the request to notify all other Clients that current on tabel of the changing in this tabel from other Client. That is the reason why were are using shared ObjectContext.

But I think with your sample-code i can modify my current code to a parallel-working app ;-)

Thanks very much for your explanations and effort

Regards Michael
Coordinator
Dec 21, 2013 at 12:14 PM
In my application we have the request to notify all other Clients that current on tabel of the changing in this tabel from other Client. That is the reason why were are using shared ObjectContext.
 
Shared object context is simply not an option, you can't build scalable application using a single database connection. For any practical purposes, you'll need a pool of connections (you get this automatically when you create separate object contexts and your database provider supports pooling).

How do you get your clients notified? Does ObjectСontext fire events when saving changes? If it has such events, you can subscribe to them when creating ObjectContext (in BeforeInvoke event handler) and retranslate to all connected clients. Of course, when ObjectContext is disposed, you need to unsubscribe from its events (in AfterInvoke and InvokeCanceled handlers). The exact details may vary (I'm not an expert in EF, you know) but I hope you've got the idea.

Regards, Alexey.
Dec 22, 2013 at 10:21 PM
Hey Alexey,

sorry for my delay. The reason is: the implemetation of the notification is from a colleage. I must wait until he give me the wanted infos to answer your questions. I hope that he answer on monday (or lately on Tuesday).

Reagards
Michael
Coordinator
Dec 22, 2013 at 10:47 PM
No worries, Michael!

I'm sure there are plenty of options for implementing such notifications without using shared ObjectContext instance.
Feel free to ask me if you have any further questiong, I'll be glad to help.

Regards, Alexey.
Dec 30, 2013 at 10:01 AM
Edited Dec 30, 2013 at 1:52 PM
Hello Alexey,

today I have the Response of my college. He told me following:

we using the a single instance of ObjectContext to registering the same Server-Events to every Client for notify the changing of related data.

All Serverside-Classes implement the Method
Public Event ServerCalled(args As DataExtensions.DataSource.ClientCallerArgs) Implements IClientNotificator.ServerCalled

    ''' <summary>
    ''' Informiert über Änderungen an der DB.
    ''' </summary>
    ''' <param name="entity">Betroffene Entität</param>
    ''' <param name="action">Art der Änderung</param>
    ''' <remarks></remarks>
Public Function NotifyDbChanged(entity As IEntity, action As ClientAction) As UpdateTicket
        entity.ChangeTracker.AcceptChanges()
        Dim args As New ClientCallerArgs
        args.Action = action
        args.EntityGuid = entity.GUID

        RaiseEvent ServerCalled(args)
        Return args.Ticket

End Function

Private EntityContext As ERP2folioXT.Inventures.Domain.Inventures.Entities

Private Sub Initialize()
       Me.EntityContext = New ERP2folioXT.Inventures.Domain.Inventures.Entities(EntityConnectionData.EntityConnectionString)

     [....]

End Sub
Following Sample is for serverside implementation of Add new Entry-functions
   ''' <summary>
    ''' Diese Methode erstellt einen neuen Inventur.
    ''' </summary>
    ''' <param name="InventureEntity"></param>
    ''' <returns>Als Result wird die erstellte GUID zurückgegeben</returns>
    ''' <remarks></remarks>
    Public Function CreateNewInventure(InventureEntity As Entities.Inventures.Inventure) As ServerResult Implements Common.Application.IInventuresManagement.CreateNewInventure
        Dim _result As New ServerResult

        Try
            NLOGAPP.Info("New Inventure will create in database [IP=" & SessionClientAddress & ":" & SessionIdentityName & "]")

            ' Grundlegende Validierung
            InventureEntity.ValidateObjectGraph()

            EntityContext.Inventures.AddObject(InventureEntity)
            EntityContext.SaveChanges()

            _result.Ticket = NotifyDbChanged(InventureEntity, ClientAction.InsertEntity)

            _result.ReturnValue = InventureEntity.Guid.ToString
            NLOGAPP.Info("'" & _result.ReturnValue & "'-Inventure is successfull create in database")

        Catch ex As Exception

            Dim _exception As Exception = ex
            While _exception.InnerException IsNot Nothing
                _exception = _exception.InnerException
            End While

            NLOGLOGGER.Fatal(ex.Message)
            _result.ErrorMessages.Add(_exception.Message)

        End Try

        Return _result

End Function
On Client-Side we have in every modul following code:
Public Sub New()

       _ZyanProxy = ConnectClient.Connection.CreateProxy(Of IInventuresManagement)()
 
       ' Datenbankaktualisierungen empfangen
        AddHandler _ZyanProxy.ServerCalled, AddressOf _VDataManager.ServerCalled

       [...]

End Sub
If you Need more informations then notify me again

lg
Michael

BTW: Can I contact you directly via Skype/IRC or another Messenger?
Coordinator
Jan 1, 2014 at 7:56 PM
Hi Michael, Happy New Year!

I don't see anything in your code that needs shared ObjectContext instances or shared service instances.
Try registering all your services in single-call activation mode, and use thread-local storage for ObjectContexts.

My Skype nickname is "yalgol".
Feel free to contact me in case of emergency, but please note I cannot guarantee any realtime support.
I have a full-time job, so I can only work on Zyan project on my spare time.
For new questions, please use this forum so that other users can benefit from our discussion.

I'm on vacation till Jan 10, 2014.
Kind regards, Alexey.
Jan 2, 2014 at 6:57 AM
Edited Jan 2, 2014 at 7:15 AM
Thank you.

I think, i Need some help from again. Beginning with the first Statement of you:
Try registering all your services in single-call activation mode
I define one module with direct Registering and the other modules with MEF

I can change
 HostInstance.RegisterComponent(Of IUpdateComponent)(Function() New UpdateComponent(ServerData), ActivationType.SingleCall)
But what is the right Syntax for MEF?
            Dim catalog As New AggregateCatalog(New DirectoryCatalog("."))
            Dim container As CompositionContainer = New CompositionContainer(catalog)
 
            If catalog.Parts.Count > 0 Then
                 ' 'MEF-Module' registrieren
                HostInstance.RegisterComponents(catalog)
           End If
Is it right, that I must Change the class attribute to
PartCreationPolicy(ComponentModel.Composition.CreationPolicy.NonShared)>
Coordinator
Jan 2, 2014 at 1:03 PM
Edited Jan 2, 2014 at 1:23 PM
Hi,
PartCreationPolicy(ComponentModel.Composition.CreationPolicy.NonShared)
Yes, that's correct.
But, as far as I remember, Zyan-MEF integration uses NonShared policy by default, so the attribute is not required.
When using MEF catalogs in Zyan, you get single-call components by default.