Archived

This forum has been archived. Please start a new discussion on GitHub.

Implementing DAO Pattern using Ice

I am thinking about an architecture for a system where all data access is de-coupled from clients/servers into a seperate process using ICE. It's kind of like the DAO pattern but at the process level so every data acess method would be an ICE Method.

For example, if I had an application server design which retreived a list of files from a database, I would implement this using two seperate processes, one for the server and another for database access ONLY.

I am worried this may be considered overkill and I am wondering how, generally speaking, architectures developed with Ice handle the Database Access Layer.

Also, assuming I go ahead with this approach, would it be possible to "merge" the two processes together effectively compiling the server into the client to make a single process?

Comments

  • Appologies for stupid post but lets be fair, it was 8:30 in the morning and I wrote it before my morning coffee.
  • bernard
    bernard Jupiter, FL
    Actually, there are some situations where writing dedicated data-access servers make sense. I can think of two:
    - your database-client library is not thread-safe and you provide a pool of single-threaded servers for the "real" multi-threaded server(s).
    - you need to access the same database from several servers and your database does not support concurrent access from several processes (a good example is Freeze). In that case, only one server access the database directly. Other servers will use Ice operations to access this data.

    However I would not make this a general pattern: having two servers instead of one is more complicated to develop, configure, and at runtime, it can only be slower.

    Cheers,
    Bernard
  • Would depend on what features your DB clients need that aren't natively supplied by the DBMS. You could get real sophisticated :) :-
    - connection pooling from multiple processes/clients
    - result caching within and across connections
    - implementation of ACID semantics if your DB's are nonexistent
    - fault tolerance if your DB client API seems to crash/hang and you want to isolate this from your client processes
    - aggregation of multiple physical sources, presented as a single logical source
    - language portability and to a lesser extent platform portability
    - multi-threading as bernard mentions
    - related to multithreading is obviously asynchronous invocation, if clients require this but DB client APIs are all synchronous

    Note that you don't need a separate process space for some of these. If none of these are required, then a thin ICE facade is virtually pointless from an architecture pov.

    My typical practice is to design data tier classes that represent the logical data model (ie, no embedded SQL within bus logic, etc) but run these within the same process space as business services/logic. As I typically use SQL Server (2005 Native Client now), the only reason I'd have an ICE database facade would be platform and language portability. The latency costs for a process middleman are not cheap if you need high performance.

    HTH
  • joe wrote:
    The latency costs for a process middleman are not cheap if you need high performance.

    Yeah, that's what I'm struggling with. The problem is more general really, to do with the fact that we need to seperate database access from business logic. In the past, this has been acheived by creating DAOs and just creating methods to contain the SQL rather than write the SQL inline. These solutions have never really worked - they become very bloated and poluted with many methods to handle slightly different SQL for different applications.

    We are looking for a different solution and I was considering using ICE to seperate the data access from the business logic out-of-process, the main benefit being language & platform independence but I realise the performance hit is probably too great.
  • Just to report my experience on this topic.
    Long time ago I implemented what you propose with CORBA in Java.
    I had my domain classes defined as Java interfaces with concrete implementations as:
    1. Normal Java classes performing OR mapping running on the server side
    2. client side implementation based on IDL definitions much like Java interfaces

    Performances ?
    Well, rather bad.

    A different approach that I have not completely implemented was to slightly modify the IDL specs allowing bulk transfer of attributes together with normal method invocation.
    It was not so bad.
    It was in 1997.

    Many years later, at least in Java world we have seen Entity Beans that had the same first approach, IMHO even worse.

    Now, again in Java world, JDO allows "easier" and more efficient implementations of the second approach. See, for example, Solarmetric/BEA KODO implementation.

    Now, if your architecture doesn't prevent the access to the datastore, I would say let the client use domain classes and delegates to ICE servers high level processing related to business processing.
    I mean, define your layers not with technology in mind (i.e., presentation, data access) but much more business process oriented.

    My 0.5 cent.

    Guido
  • joe wrote:
    Would depend on what features your DB clients need that aren't natively supplied by the DBMS.

    That's the point - we are trying to come up with an architecture that is DBMS idepdentant.

    ganzuoni wrote:
    Now, if your architecture doesn't prevent the access to the datastore, I would say let the client use domain classes and delegates to ICE servers high level processing related to business processing.
    I mean, define your layers not with technology in mind (i.e., presentation, data access) but much more business process oriented.

    Sorry to be a simpleton, but I can't help it (because I am one ;) ) As I understand it, you are saying that everything talks to servers via ICE leaving the responsiliby of data access to the server.

    It's how the server talks to the database that's the issue - we need that to be database independant. If the server is soley responsible for data access then fine but it may also perform business logic processing. Ideally, I'd like to seperate the business processing and the data access into two seperate server processes which communicate via ICE.

    Doing so would be extremely flexible but would the overhead be too great? From the responses so far it seems so.
  • Doing so would be extremely flexible but would the overhead be too great? From the responses so far it seems so.

    Well, it depends.
    If you follow an EJB-like approach, i.e. accessing an object attribute on the client means remote invocation, the answer is yes, even if it is a simple solution.
    If you follow a much more JDO-like approach, i.e. bulk transfer of attributes, the results might not be that bad, but the implementation isn't that trivial....

    Guido.
  • Paul,
    That's the point - we are trying to come up with an architecture that is DBMS idepdentant.

    I didn't get this from your first post. DBMS independence is a different beast to process-space decisions. If you're saying you want your business logic to access a "logical" data model that abstracts away the underlying DBMS, then you're looking for a facade library - they're a dime a dozen - ODBC, JDBC, yada yada. If you want to hand-roll one of these because a pre-existing facade doesn't suit (ie, you want to facade BerkeleyDB, SQL-Lite and Oracle) then you have some work cut out for you. Regardless, this design could operate both in-process or out-of-process so you need other factors to decide that question.
    It's how the server talks to the database that's the issue - we need that to be database independant. If the server is soley responsible for data access then fine but it may also perform business logic processing. Ideally, I'd like to seperate the business processing and the data access into two seperate server processes which communicate via ICE.

    There is only limited value that a general architecture discussion will add to your decision here. The textbook says that separate processes give you different isolation, coupling and deployment characteristics, and more scalability options (ie, moving the DB process to a dedicated server at some point, etc). You also lose slightly on performance with marshalling and IPC burdens... The decision should be based on a real-live production scenario you're architecting, because more "general" discussion might not help.
    Doing so would be extremely flexible but would the overhead be too great? From the responses so far it seems so.

    What's "good" performance? What's "too great" an overhead? Build a testbox and profile it using two prototypes with realistic throughput. If your transaction rate goes from 2000/sec to 1500/sec does it really matter? Especially if your ICE servants can only process 400 requests/seq? Especially if your estimated real peak is 100/sec? Raw performance is academic - if you prefer designing a separate process (and it has merits) and it won't ever impact your real performance, go ahead.

    HTH