[en] Any performance views probably?

Topics: Technical Support
Sep 3, 2011 at 12:32 PM

Hello !

I am just regarding to extend some of my management agent software with RPC.

I am in the monent very interested to see some perf numbers. Though Zyan is based
on .Net Remoting, this disallows usually to have full control over the internals, like
how many threads are just waiting [ready to process new incoming "packages" really
in parallel]. Do you probably have such numbers [on your workbench]? It would be
really interesting to see RTT[round trip time], where a source sends a package to
the server and it then travels back to the source.

It is already inside a usual LAN, that this numbers cannot be really predicted, but
the thread handling will make this even more worse.

Thanks so far and
best regards!

++mabra

Coordinator
Sep 4, 2011 at 2:56 PM

Hi mabra,

There are already plans to provide Zyan with a monitoring and statistic feature, as you can see under http://zyan.codeplex.com/workitem/1092.
But work on this feature hasn´t started yet.

> I am in the monent very interested to see some perf numbers.

Performance numbers always depends on many factors: network protocol, size of messages, number and frequency of messages sent, number of concurrent client, network bandwidth, hardware, and so on.
I currently have no performance charts. Performance was never a problem, with one exception. Zyan need long time to register distributed events and delegates to a server. This issue is caused by dynamic code generation and compiling. Since Changeset 12851 the issue is resolved. Zyan uses now lambda expressions instead of dynamic code generation to attach remote event handlers.

If don´t want to wait for completion of  the Zyan monitoring and statistic feature, you can collect numbers and facts yourself. To be able to do this, you need to know, how Zyan works.

> Though Zyan is based on .Net Remoting, this disallows usually to have full control over the internals, like
> how many threads are just waiting [ready to process new incoming "packages" really in parallel].

.NET Remoting uses the .NET ThreadPool and this applies also to Zyan. Changes you make on the ThreadPool configuration affects Zyan. You can increase the minimum and maximum number of parallel worker threads, for example (via ThreadPool.SetMinThreads and ThreadPool.SetMaxThreads). The ThreadPool class let you get the number of available threads in the thread pool (ThreadPool.GetAvailableThreads). Please remember that the ThreadPool is also used by other components of the .NET Framework. This means that not all used threads are Zyan worker threads!

Generally the .NET Framework ThreadPool implementation is robust and has good performance. The .NET ThreadPool is optimized for short running tasks. Receiving messages from a socket is such a short running taks in most cases. You can find some detailed information about the ThreadPool (and its improvements in .NET Framework 4.0) here: http://aviadezra.blogspot.com/2009/04/task-parallel-library-parallel.html.

Other interessting perf numbers depend on the sent bytes over the network and the amount of time thas was required to doing this. It is not difficult to get the raw byte[] to be transmitted. You just have to wirte a component called channel sink. Zyan uses this existing extension system from .NET Remoting (Read more about Remoting sinks: http://msdn.microsoft.com/en-us/library/tdzwhfy3%28VS.80%29.aspx). Every sink needs to have a sink provider. The provider is a factory class which receives configuration settings and manages creating instances of the sink and plugging them into the sink chain, when needed.

You can see a very simple implementation of such a channel sink inside the Zyan source code. The ClientAddressServerChannelSink is used inside Zyan to get the client´s IP address on server side.

ClientAddressServerChannelSinkProvider This is the provider. As you can see in the CreateSink method, it creates a instance of the sink and puts it into the sink chain.
ClientAddressServerChannelSink This is the channel sink implementation. The music plays inside the ProcessMessage method and the AsyncProcessResponse method.

You can take this code as a template to create your own sink for acquiring your perf numbers on server side (for a client side sink, you must implement the IClientChannelSink interface instead of IServerChannelSink and the IClientChannelSinkProvider interface for the provider instead of IServerChannelSinkprovider; Create always seperate sinks for server and client!). Interessting for you are the requestStream and responseStream parameters. They can be used to get the count of raw bytes to be transferred over the network.

Please have a look at a patch from the user blackdog at http://zyan.codeplex.com/SourceControl/list/patches (last entry). He wrote already a counter sink to get perf numbers. The patch was not applied to Zyan because the feature of easy sink integration wasn´t completed at this time. His code should give you a glue about writing your own performance monitoring channel sink.

The position of your sink inside the sink chain is very imporant. If you plug your sink in to early, the stream may not be created already, for example.

Your sink provider must be added to a protocol setup. This is very simple. On server side, you can use the following extension methods on your protocol setup instance:

AddServerSink (Adds your sink provider at the end of the sink chain)
AddServerSinkBeforeFormatter (Adds your sink provider before the formatter sink; The formatter sink handles (de)serialization of the data to be sent)
AddServerSinkAfterFormatter (Adds your sink provider after the formatter sink; Here the data should be serialized to the requestStream)

On client side you use the methods called AddClientSink, and so on.
I hope this helps you.

As soon as I have any news about the final Zyan monitoring and statistic feature, I´ll post it here on Zyan discussions.

Best regards,
Rainbird

Sep 8, 2011 at 7:28 PM

Hi Rainbird!

Much thanks for your information.

I am sure, I understandsome of the basics [have used Remoting once, in my
EventlogCollectorServices {which collects Windows Eventlog Events} as
a controller and this is a ServerActivatedObject], but performance was never
checked, it was just enough ;-)

I am working on a system management application, consisting of a chain of distributed functionality
I see these mostly as agents which communicate.

On of my requirement is to provide basic informations, like computer lists and other base data from a cache, instead
of a database [DB]. This will only make sense, if I can get the response within 5-10 ms, otherwise I could use a DB
directly. I just checked such a scenario with a space based computing tool, but I feel, this is too slow, because
it has too many outliers within the 15-100 ms range.

I would not like to use memcached, because it is a plain tuple store, where I want to be able to query the
cache with linq. Another task, what the caching machine should do, is build live statistics on performance data,
which will arrive in a big flood. The will cause surely a lot of concurrency and the threadpools will probably
not be able to handle that at all. Probably - will see - I have to use different computers [according to
separation of concern ...]. So, the ThreadPool may be not responsive enough in this scenario. Additionally,
in remoting, I can not set a request timeout, so far as I remember. I must have something like this [with
a timeout value with something like ~20 ms.

So, all my question really was, if you - in the moment - have some numbers, so that I can get a better
imgaination. I'll take one of your examples and will try it! It will be probably too much work for me - and
require more knowledge, than I have - to make my own transport. Plain TCP would be really fast, but there
is no top-level framework with serailization etc. One simple way for me could be, to use MSMQ.
But I must make another demo for this [my message times are down to 130 µs [one way] and serializing
is simple.

Will see. Thanks anyway, your Zyan looks well !

br++mabra



Sep 11, 2011 at 7:25 PM

Hi Rainbird !

I made some numbers ;-)

I really, really don't call this benchmarking or something like this.... It's just that simple, that I took the solution
named "DynamicECBResponses" and hacked it to have a message with an Id and a text and send/return 1000
of them. That gave me 1-3 ms for the local machine and something about 4-7 ms for a remote machine inside
the same subnet [100 MB/s eth]. I feel, this is really a good value!

Regard:I know, that this time contains already a bit of processing time on each side - good to see. I have not
really wanted to see plain net-io, but would give probably more insight.

I have similar numbers for other communication packages, but found nothing really socket-based, which might
be even faster ... !?? This question arised to me, because my test using MSMQ are really a lot faster! I can
read ~7.500 Message/s [~133 µs] on a local machine [and write even more - but this is async] and these are
comparable objects [some DateTime and a text up to 1000 B]. Even reading/writing remote queues is ultra-fast.

I think, Zyan may come to my RPC usage. I really do not want to write the plumbing code for sockets [and for
a similar reason, I do not want to use 0MQ].

Any thoughts??

Thanks so far and
best regards,

++mabra

 

Coordinator
Sep 12, 2011 at 6:21 AM

Hi mabra,

I´m glad you managed to get numbers, very good numbers.

> I have similar numbers for other communication packages, but found nothing really socket-based, which might
> be even faster ... !??

Zyan uses .NET Remoting (TcpChannel as default) and send the data as byte[] (so serialization is done by Zyan, not by .NET Remoting!).
.NET Remoting is fast. What other communication packages did you test? WCF?

> This question arised to me, because my test using MSMQ are really a lot faster! I can
> read ~7.500 Message/s [~133 µs] on a local machine [and write even more - but this is async] and these are
> comparable objects [some DateTime and a text up to 1000 B]. Even reading/writing remote queues is ultra-fast.

I´m no MSMQ expert. I´ve no glue, why MSMQ is so fast. The fastest way to communicate on a local machine with Zyan should be
the IpcBinaryServerProtocolSetrup/IpcBinaryClientProtocolSetup. This protocol setup uses local named pipes.

> I think, Zyan may come to my RPC usage. I really do not want to write the plumbing code for sockets [and for
> a similar reason, I do not want to use 0MQ].

That´s great.

Best Regards,
Rainbird

Sep 15, 2011 at 11:28 PM

Hello Rainbird !

Thanks for your reply.

I thought a bit about my post. I am not sure, if it is a wanted race to answer your question about other packages
and so please sorry, that I posted numbers;I feel, this was not too polite.

BTW, compared to perf numbers of 0MQ [ZeroMQ], everythings is increditable slow
and even MSMQ is a bicycle ;-)
This was one of my reasons to ask. The send 4.5 Mio/s, this is a latency of 12.5 µs in
a 10 GBE network. I do not understand, how this can be done [ok, C/C++ is a bit faster then the framework].
For my - some other - networking programs, it would be really great to come to this greater details.

Best regards,

++mabra