Archived

This forum has been archived. Please start a new discussion on GitHub.

Question for send large data

my slice file definition
/** A sequence of bytes. **/
sequence<byte> ByteSeq;

["amd"]
interface DataTransfer {
string send(int offset, ByteSeq bytes);
};
In the Client, I have a large XML (maybe 10MB); I want send the XML to Server.
if I use synchronous call send, my client must block. if I use asynchronous call send, when the server shutdown, myclient continue to send that it doesn't know the server shutdown. And, if the server get the XML clips, and the server how can ensure the integrity of the large XML.
use synchronous or use asynchronous ?
How can I do with the large data?
somebody has the send large data demo ? I'm a java programmer, I want java demo, thanks very much.

Comments

  • xdm
    xdm La Coruña, Spain
    myclient continue to send that it doesn't know the server shutdown

    If i not wrong you will receive an exception in the AMI callback.
    if the server get the XML clips, and the server how can ensure the integrity of the large XML.

    I think the correct is to compute a hash (SHA1) of the total contents in the client and send it first to the server, once the server ends receiving data it compute the hash (SHA1) and it should match.
    /** A sequence of bytes. **/
    sequence<byte> ByteSeq;
    
    interface Writer
    {
      string send(int offset, ByteSeq bytes);
    };
    
    interface DataTransfer {
      Writer* createWriter(string path, // Where to write the file
                           string hash, // The cliente computed hash
                           long length); // How much bytes length
    };
    
    

    I will do something like the above, so the program first call createWriter, and pass in the path, hash and length of the file, the server then create a writer to write that file, and returns a Writer proxy, as the Writer knows the hash and length, is easy for it to know if the transfer is complete and compute the data integrity.

    See also Optimizing Performance of File Transfers in
    http://www.zeroc.com/newsletter/issue20.pdf
    synchronous or use asynchronous ?
    That depends, if your program should care about not blocking, you have not option but using asynchronous, if your program is not concerning with blocking, synchronous is some how a bit simple.

    Hope that helps,
    Jose
  • thanks for your answer

    hi, XDM, these questions I thought for a long time, I will try it, thank you for the information!
  • benoit
    benoit Rennes, France
    Hi,

    You don't really need to compute a checksum to guarantee data integrity. Tcp guarantees that already.

    If the client successfully sends the request (either using a synchronous or asynchronous call) and the server is still running, the data received by the server will be valid. If the server is going down while receiving the request, the client will get an exception indicating that the request failed to be dispatched by the server (unless you use oneway requests) and the server won't dispatch the request to the servant.

    Checksums are useful if you want to make sure the data isn't tampered with while in transit over the network. If you are concerned about this, I recommend to use IceSSL instead of explicit checksums.

    Cheers,
    Benoit.
  • xdm
    xdm La Coruña, Spain
    You don't really need to compute a checksum to guarantee data integrity. Tcp guarantees that already.

    That is totally truth, but by compute a separate checksum you can have a extra security, TCP checksums, doesn't protect you about application errors (your application doing wrong things with the correct data), nor about disk io errors (bytes messed when write to the disks), if you have a checksum of the total data, you can ensure that what end write to the disk/db is correct.