[en] Deterministic cleanup

Topics: Announcements
Coordinator
Jun 7, 2011 at 1:32 PM

What about the component cleanup (including, but not limited to IDisposable support)?

As far as I know, Zyan doesn't handle component cleanup explicitly.
It's up to GC to clean up unused components.
But consider the following scenarios:

  • Single-call component uses some limited resource and implements IDisposable — it should be cleaned up upon method invocation
  • Singleton component does the same — it should be disposed of together with its owner, ZyanComponentHost
  • Some component is registered with the use of factory and requires explicit deinitialization, i. e.

// local usage scenario
var svc = myFancyContainer.Resolve<IService>();
try
{
	svc.DoTheTrick();
}
finally
{
	// allow container to do the cleanup according to its instantiation policy
	myFancyContainer.Release(svc);
}

//------------------------------------

// Zyan server scenario
using (var host = new ZyanComponentHost(...))
{
   host.RegisterComponent<IService>(() => 
   	myFancyContainer.Resolve<IService>()); // what about the cleanup?
}

// Zyan client scenario
using (var conn = new ZyanConnection(...))
{
	var svc = conn.CreateProxy<IService>(); // factory is called at the server-side
	svc.DoTheTrick(); // if IService is single-call, cleanup should be done at this very moment 
}

Coordinator
Jun 7, 2011 at 10:08 PM
Edited Jun 7, 2011 at 10:14 PM

> Single-call component uses some limited resource and implements IDisposable — it should be cleaned up upon method invocation

SingleCall activated components should be designed as stateless components. So the invoked method itself should clean up limited resources. SingleCall activated components should not have any fields, so the Dispose method has nothing to clean up. So in most cases there is no need for clean up support.

Only if you publish a component that is not stateless by design, you may run into clean up issues. But if you want to use dependecy injection stuff for your server components, the components must be stateful. Injections are commonly made via constructor or properties. Injected object references are stored in fields. If we have fields, we may have the need to run clean up code for this fields.

That´s reason enough for implementing clean up support for SingleCall components into Zyan.

I suggest to check if the invoked component implements IDisposable. If it does, the ZyanDispatcher could call Dispose after processing the remote method invocation. If you agree with this solution for SingleCall, I would create a work item for this development task.

> Singleton component does the same — it should be disposed of together with its owner, ZyanComponentHost

ZyanComponentHost is not the owner. Components are registered in a component catalog. One catalog may be published by any number of ZyanComponentHost instances. This is very important, if you want to publish components to different entpoints (e.G. TCP and HTTP). The ZyanDispatcher obtains the component instance - which should be used for method invocation - from it´s hosts component catalog. So the component catalog is the owner of the component.

You can even change the component catalog of a ZyanComponentHost in runtime:

// Create a new component catalog
ComponentCatalog catalog = new ComponentCatalog();

// Register a component in the new catalog
catalog.RegisterComponent<ICoolComponent, CoolComponent>();

// Publish the new catalog using an existing host 
// (Current component catalog of the host is not published then anymore!)
host.ComponentCatalog = catalog;

You can also create a new ZyanComponentHost with an existing ComponentCatalog instance, which has already registered components:

// Create a new component catalog
ComponentCatalog catalog = new ComponentCatalog();

// Register a component in the new catalog
catalog.RegisterComponent<ICoolComponent, CoolComponent>();

// Publish the entire component catalog via TCP
ZyanComponentHost host = new ZyanComponentHost("CoolHost", 4711, catalog);

And finally you can use Zyan ComponentCatalog instances as local mangement containers without any remote or network stuff:

// Create a new component catalog
ComponentCatalog catalog = new ComponentCatalog();

// Register a component in the new catalog
catalog.RegisterComponent<ICoolComponent, CoolComponent>();

// Get a instance if the registered component for the ICoolComponent interface
var instance = catalog.GetComponent<ICoolComponent>();

I should write a documentation page about component catalogs in Zyan ;-)

But now back to the cleaning up problems. You´re right. Dispose should be called for disposable components, when the owner (the component catalog) is disposed. To accomplish this, the following tasks must be done:

  • ComponentCatalog must implement the IDisposable interface (Actually it does not).
  • Logic for cleaning up registered components must be implemented in ComponentCatalog.Dispose. 

> Some component is registered with the use of factory and requires explicit deinitialization, i. e.

If not otherwise specified, factory generated component instances are SingleCall activated. In this case the SingleCall scenario from above matches. The ZyanDispatches checks (via Reflection) if the component type implements IDisposable and calls Dispose. All is fine.

What happens, when a factory generated component is registered with Singleton activation? Zyan calls the factory delegate when the component is invoked first time after registration. The created instance is stored within the component catalog (See code line 355 in ComponentCatalog class). From this point on, it behaves like any other Singleton activated component. This means that the Singleton clean up solution from above will work for factory generated components too.

 

 

Coordinator
Jun 8, 2011 at 2:45 PM

Hi, Rainbird!

Thank you for the detailed analysis.

Any hosted component could be designed by a third-party.
So, we cannot assume that our own design rules are satisfied in that case.
You may take any .NET class and host it in Zyan (which is a big advantage over both WCF and Remoting).
If some class implements IDisposable, it's most likely because it may need explicit cleanup.
Therefore, it's a good idea to support IDisposable in Zyan.

>SingleCall activated components should be designed as stateless components.
>So the invoked method itself should clean up limited resources.

This usage pattern will force you to insert try/finally/using stuff in every method call.
So, instead of

class Service : IService, IDisposable
{
	IDisposable resource = new Resource();
	void IDisposable.Dispose()
	{
		resource.Dispose();
	}
	public void Method1()
	{
		resource.Use();
	}
	public void Method2()
	{
		resource.Use();
	}	
}

you will be forced to write something like this:

class Service : IService
{
	public void Method1()
	{
		using (var resource = new Resource())
		{
			resource.Use();
		}
	}
	public void Method2()
	{
		using (var resource = new Resource())
		{
			resource.Use();
		}
	}
}

>I suggest to check if the invoked component implements IDisposable.

That would be fine for most cases, but what about externally-owned components?
I'd suggest to add custom cleanup lambda-function for that scenario:

host.RegisterComponent<IService>(
	() => container.Resolve<IService>(uniqueName),
	service => container.Release(service)
);

>ZyanComponentHost is not the owner.
>This is very important, if you want to publish components to different entpoints.

That's great, thanks for the clarification.
I wasn't aware of this feature.

>Dispose should be called for disposable components, when the owner (the component catalog) is disposed.

Yes, I agree, with the same exception: some components might be externally-owned.
Singleton component may also be registered with the use of factory, just like the example above.
So they may also need custom cleanup code.

>The ZyanDispatches checks (via Reflection) if the component type implements IDisposable and calls Dispose.

There is no need to use Reflection for that purpose:

if (instance is IDisposable)
{
	((IDisposable)instance).Dispose();
}

So, to summarize, I would suggest to implement:

  1. Built-in IDisposable support
    1. Single-call components are disposed after every method call
    2. Singleton components are disposed together with their owning ComponentCatalog
  2. Custom cleanup code via lambda-functions
    1. Single-call components: lambda is called after method invocation
    2. Singleton components: lambda is called in ComponentCatalog.Dispose()
    3. If some cleanup lambda-function is specified, Zyan doesn't use IDisposable interface.

Regards, yallie.

Coordinator
Jun 9, 2011 at 10:12 PM

I fully agree. Explicitly ceaning up via an optional delegate, provided on component registration is  a very good idea. And will be easy to integrate. The optional clean up delegate can be stored in the ComponentRegistration class.

 

Coordinator
Jun 10, 2011 at 8:52 AM

One more thing to mention.

Instances registered with RegisterComponent<I, T>(T instance) overload should be treated as externally-owned. Zyan shouldn't try to dispose them.

Consider the following example:

public partial class MainForm : Form, IMyApplication, IDisposable
{
	ZyanComponentHost Host { get; set; }

	void StartServer()
	{
		if (Host == null)
		{
			Host = new ZyanComponentHost("MyApplication", 12345);
			Host.RegisterComponent<IMyApplication, MainForm>(this);
		}
	}

	void StopServer
	{
		if (Host != null)
		{
			// MainForm shouldn't be disposed here
			// even if host's own ComponentCatalog may be freed
			Host.Dispose();
			Host = null;
		}
	}
}

Coordinator
Jun 11, 2011 at 10:16 AM

I think we need a more general and predictible rule for automatic call of dispose. How sounds that:

Zyan automaticly disposes all instances which are created by Zyan (Zyan is owner). This includes the component catalog.

Wenn a component instance is created by a external factory or a already created instance is registered in component catalog, then Zyan doesn´t call Dispose automaticly.
But for all Singleton/SingleCall activated components, which are only registered with a type, Zyan calls Dispose.

Wenn the component catalog is created internally inside the ZyanComponentHost, then it will be disposed automaticly.
Otherwise, if the component catalog is created externally.

Coordinator
Jun 11, 2011 at 10:55 AM

Yes, exactly!

Coordinator
Jun 13, 2011 at 4:27 PM

Support for deterministic clean up is now integrated, as discueed. I´ve created a feature for easy tracking of this topic: http://zyan.codeplex.com/workitem/1042.

I´ve go no time to test all clean up scenarios, yet. There also no unit tests written so far.
But I should work.

Coordinator
Jun 13, 2011 at 10:52 PM
Edited Jun 14, 2011 at 10:31 AM

That's great!

I've checked in unit tests for cleanup routines.

We're seemingly getting too much method overloads... Let's upgrade to C# 4.0 to utilize default and named parameters :)

UPD. Here is a summary table:

Registration Cleanup
catalog.RegisterComponent<IService>(() => new Service(), ActivationType.SingleCall);
Disposed
catalog.RegisterComponent<IService, Service>(ActivationType.SingleCall);
Disposed
catalog.RegisterComponent<IService>(() => new Service(), ActivationType.Singleton);
Disposed
catalog.RegisterComponent<IService, Service>(ActivationType.Singleton);
Disposed
catalog.RegisterComponent<IService, Service>(serviceInstance);
Not disposed
catalog.RegisterComponent<IService>(() => new Service(), 
	ActivationType.SingleCall, v => ((Service)v).Release());
Cleanup delegate is called
catalog.RegisterComponent<IService, Service>(
	ActivationType.SingleCall, v => ((Service)v).Release());
Cleanup delegate is called
catalog.RegisterComponent<IService>(() => new Service(), 
	ActivationType.Singleton, v => ((Service)v).Release());
Cleanup delegate is called
catalog.RegisterComponent<IService, Service>(
	ActivationType.Singleton, v => ((Service)v).Release());
Cleanup delegate is called
catalog.RegisterComponent<IService, Service>(
	serviceInstance, v => ((Service)v).Release());
Cleanup delegate is called
Coordinator
Jun 14, 2011 at 9:54 AM
Edited Jun 14, 2011 at 4:22 PM

I suggest to make ZyanComponentHost.ComponentCatalog property write-only :)
Consider the following code:

using (var host1 = new ZyanComponentHost(...))
using (var host2 = new ZyanComponentHost(..., host1.ComponentCatalog)) // create second endpoint
{
	...
	// ComponentCatalog is disposed twice
}

Write-only property will force user to rewrite the snippet like this:

using (var catalog = new ComponentCatalog())
using (var host1 = new ZyanComponentHost(..., catalog))
using (var host2 = new ZyanComponentHost(..., catalog)) // create second endpoint
{
	...
	// ComponentCatalog is disposed once
}

Alternatively, we can replace ComponentCatalog.DisposeWithHost property with ZyanComponentHost.DisposeCatalog to prevent duplicate disposal. 

Coordinator
Jun 14, 2011 at 9:59 PM

I think., the write-only solution is the better on, because the user has no chance to do something wrong.

Thanks for fixing the bugs and writing the unit tests. Great work!

Zyan 2.x should support ,NET 3.5. In Zyan 3.x and later we can get rid of the endless overloads.

Coordinator
Jun 15, 2011 at 10:05 PM

I've just figured out that optional parameters have nothing to do with .NET framework version :)
[Optional] and [DefaultParameterValue] attributes are supported since Fx 2.0 (they were used by VB compiler, I think).
VS2010 solution can be targeted to Fx 3.5 while using C# 4.0 compiler (see screenshot below).

So the question is: do we really need to support C# 3.0 and VS2008?
The only drawback of removing extra overloads would be the inability to use shorter overloads from C# 3.0/VS2008.
Complete methods (with all parameters) will still be available.

I think, there's no need to wait for Zyan 3.0 release :)

Coordinator
Jun 16, 2011 at 7:53 PM

> So the question is: do we really need to support C# 3.0 and VS2008?
> The only drawback of removing extra overloads would be the inability to use shorter overloads from C# 3.0/VS2008.
> Complete methods (with all parameters) will still be available.

Please have a look at the current Zyan 2.0 binary download counters (http://zyan.codeplex.com/releases/view/62104). Nearly 50% of the Zyan users are downloading the Fx3.5 binaries. So I assume, that they still work with VS 2008 and .NET 3.5. If we remove the overloads, the compatibility is broken (when using Vs 2008). All the VS 2008 users will get compiler errors in thier own projects, after updating to the next Zyan release.
When they have to provide all parameters, this will feel terrible (like Excel automation with default COM Wrappers in C# 3.0).
Removing the overloads will force VS2008 users to deal mit features they don´t use.

> I think, there's no need to wait for Zyan 3.0 release :)

I don´t like that confusing number of overloads too, but I don´t want to leave the VS2008 users out in the rain. So I think the best solution is to create a Zyan 2.x branch (in version control) for Fx3.5 support. The main development then continues with .NET 4.0 support only and dont´t have to regard old Fx3.5 limits. But we can still provide bugfixes and important features to the Fx3.5 branch.

What do you think about that compromise?

Coordinator
Jun 17, 2011 at 12:27 AM
Edited Jun 17, 2011 at 12:27 AM

>Nearly 50% of the Zyan users are downloading the Fx3.5 binaries.
>So I assume, that they still work with VS 2008 and .NET 3.5. 

You're most probably right.
The fact that VS2010 supports older frameworks doesn't mean that folks use it.
Most Fx3.5 developers are working with VS2008, for sure.

>So I think the best solution is to create a Zyan 2.x branch (in version control)
>for Fx3.5 support. The main development then continues with .NET 4.0 support...

Most of planned features don't need Fx4 (generics, linq, reflection, ref/out args).
Freezing 2.x development will mean that Fx3.5 users won't ever get these.

I would rather create two separate project files with different target frameworks.
Say, Zyan.Communication.csproj and Zyan.Communication.Fx3.csproj.
Language version can be specified per project (see .csproj properties -> Build -> Advanced).

Features which require Fx4 (like MEF integration) will probably be located
in separate classes, so we just exclude those extra files from Fx3-project.
Similarly, Fx3-specific method overloads can be placed into separate files
(using partial classes) and not included in Fx4-targeted project.
Here is an example solution: http://sharp-shooter.ru/misc/MultiTargeting.zip

>What do you think about that compromise?

Branches are fine when no further maintenance is planned, except for occasional bugfixes.
If that is the case, we can fork 2.x.

But if you prefer to continue active 2.x development for Fx3.5 as long as possible,
having two project files is much easier than merging changesets between SVN branches.

Anyway, I'll be fine with either of these setups :)
I don't use Fx3.5 assembly, I'd just prefer less overloads for Fx4 version.

Coordinator
Jun 17, 2011 at 8:13 PM

> But if you prefer to continue active 2.x development for Fx3.5 as long as possible,
> having two project files is much easier than merging changesets between SVN branches.

Very good idea. This is indeed much more comfortable than a branch.

Coordinator
Jun 20, 2011 at 8:30 AM
Edited Jun 20, 2011 at 8:41 AM

I've found that C# 4.0 optional parameters can't emulate most RegisterComponent() overloads, for example,

  void RegisterComponent<I>(string uniqueName = "", Func<object> factoryMethod, Action<object> cleanUpHandler = null) // syntax error

(optional parameters must be defined at the end of the parameter list, after any required parameters).

So I decided to take a different approach. I added IComponentCatalog interface (it was also needed for MEF integration) and implemented all overloads as extension methods to this interface. Then I implemented IComponenCatalog interface by both ComponentCatalog and ZyanComponentHost and removed all duplicate overloads. This solution is compatible with C# 3.0 and above:

// This interface has only full versions of RegisterComponent methods
public interface IComponentCatalog
{
  void RegisterComponent<I, T>(string uniqueName, ActivationType activationType, Action<object> cleanUpHandler);
  void RegisterComponent<I>(string uniqueName, Func<object> factoryMethod, ActivationType activationType, Action<object> cleanUpHandler);
  void RegisterComponent<I, T>(string uniqueName, T instance, Action<object> cleanUpHandler);
  ...
}

// All extra overloads went here
public static class IComponentCatalogExtensions
{
  public static void RegisterComponent<I, T>(this IComponentCatalog catalog) { ... }
  public static void RegisterComponent<I, T>(this IComponentCatalog catalog, Action<object> cleanUpHandler)  { ... }
  public static void RegisterComponent<I, T>(this IComponentCatalog catalog, string uniqueName)  { ... }
  public static void RegisterComponent<I, T>(this IComponentCatalog catalog, ActivationType activationType)  { ... }
  ...
}

public class ComponentCatalog : IComponentCatalog, IDisposable
{
  ...
}

public class ZyanComponentHost : IComponentCatalog, IDisposable
{
  ...
}

//----------------------------------
// write as usual:
host.RegisterComponent<IService, Service>(name);

// C# 3.0 and above translates this call into:
IComponentCatalogExtensions.RegisterComponent<IService, Service>((IComponentCatalog)host, name);
Coordinator
Jun 21, 2011 at 6:14 AM

Great. This is a clear and easy to understand approach. And the behavoir ist predictible.
Many thanks for explanation and the example.