This forum has been archived. Please start a new discussion on GitHub.

How to scale for many unknown services?

I am designing a platform (in .net) that is based on a plug-in architecture. The idea being, that plug-in authors can write plug-ins to the platform that provide functionality in form of some sort of web-service like service.

The platform is supposed to load the assemblies of each plug-in and host the services implemented in the plug-ins. Versioning is done by deploying a new version of the .net assembly (never modifying a once deployed version), so it is very possible, that between different versions of a plug-in, most of the slices would be the same.

I see two main challenges; the number of services is very large, easily in the hundreds, maybe thousands. Though it is also very possible, that only a fraction of the services would actually be called most of the time.

And the second challenge is isolation, so that in case of one service failing, the whole application should not break.

My idea was to dynamically recognize the services provided through plug-ins in .net, then create an ice object for each at run-time and somehow dynamically register the ice object.

Does this sound like a reasonable plan of action, does ICE scale well in such a scenario and does ICE maybe provide some other, better solution? This is still very much in the conceptual phase, so a very different approach is also very welcome. Also any ideas about isolation would be interesting.