Archived

This forum has been archived. Please start a new discussion on GitHub.

[High frequency/Low latency] Datagram / BatchDatagram : how to set TTL and Interface?

Hi,

I need a quick help to set up TTL and interface for UDP Endpoints, I don't find any example and I am a little bit disapointed with doc for that point.

On the other hand, I would like to share here some good results I have with ICE on Linux RHEL 6.3 + Solarflare cards (bidir oneway or datagram) : 20 - 30 usecs as round trip delay (sending ticks and client sending back info). :eek:

It was not simple to do a multicast pub/sub (demo example is somehow confusing) without any discovery phasis. The main issue I have is the fact that, in terms of market data publication, the publisher has to be the client with datagram whereas, in TCP, I am using bidir callbacks.

Here is my Demo example : :cool:

Hello.ice
pragma once

module Demo
{

interface Hello
{
    idempotent void say(string message);
};

};

Server.cpp
#include <Ice/Ice.h>

#include <Hello.h>

using namespace std;
using namespace Demo;

class HelloI : public Hello
{
public:

    virtual void
    say(const std::string& message, const Ice::Current&)
    {
        cout << "Received " << message.c_str() << endl;
    }
};

class HelloServer : public Ice::Application
{
public:

    virtual int run(int, char*[]);
};

int
main(int argc, char* argv[])
{
    HelloServer app;
    return app.main(argc, argv, "config.server");
}

int
HelloServer::run(int argc, char* argv[])
{
    Ice::StringSeq args = Ice::argsToStringSeq(argc, argv);
    args = communicator()->getProperties()->parseCommandLineOptions("Hello", args);

    Ice::ObjectAdapterPtr adapter = communicator()->createObjectAdapter("Hello");

    Ice::ObjectPrx hello = adapter->add(new HelloI, communicator()->stringToIdentity("Hello"));
    adapter->activate();

    communicator()->waitForShutdown();
    return EXIT_SUCCESS;
}

config.server
Hello.Endpoints=udp -h 239.1.1.1 -p 10000

Ice.IPv6=0
Ice.Warn.Connections=1
Ice.Trace.Network=3

Client.cpp
#include <Ice/Ice.h>
#include <IceUtil/IceUtil.h>

#include <Hello.h>

using namespace std;
using namespace Demo;

class HelloClient : public Ice::Application
{
public:

    virtual int run(int, char*[]);

private:

    void menu();
};

int
main(int argc, char* argv[])
{
    HelloClient app;
    return app.main(argc, argv, "config.client");
}

int
HelloClient::run(int argc, char* argv[])
{
    Ice::StringSeq args = Ice::argsToStringSeq(argc, argv);
    args = communicator()->getProperties()->parseCommandLineOptions("Hello", args);

    HelloPrx hello = HelloPrx::uncheckedCast(
        communicator()->propertyToProxy("Hello.Proxy")->ice_datagram());
    if(!hello)
    {
        cerr << argv[0] << ": invalid proxy" << endl;
        return EXIT_FAILURE;
    }

    while (1)
    {
        std::cout << "try to send Hello" << std::endl;
        hello->say("coucou c'est moi!");
        sleep(1);
    }

    return EXIT_SUCCESS;
}

config.client
Hello.Proxy=Hello:udp -h 239.1.1.1 -p 10000
Ice.IPv6=0
Ice.Warn.Connections=1
Ice.Trace.Network=1

Comments

  • I tried to add --ttl 10 into my Endpoint string and I checked like that :
    Ice::EndpointSeq endPointSeq = pushMessageCBPrx_->ice_getEndpoints();
    Ice::EndpointPtr endpoint = endPointSeq[0];
    std::cout << "Endpoint definition :" << endpoint->toString() << std::endl;
    

    It works fine even if it is not clear in the doc ;)
  • benoit
    benoit Rennes, France
    Hi Philippe,

    Great to hear it's performing well!

    The UDP endpoint syntax is described here in the Ice manual: UDP Endpoint Syntax. For the interface, you can pass --interface.

    Cheers,
    Benoit.
  • Indeed :)
    I am not uising Zero Copy (because I didn't really understand how to use it).

    But before, could you give me some advises regarding network MTU limitation and Ice.MessageSizeMax setting?
    By default, Ice.MessageSizeMax=1024 (1MB) and minimum 1 (1KB).
    But when sending udp packets with BatchDatagram, how to avoid network MTU threshold/cut?

    EDIT: some packets are missed if I publish more than MTU.
    If I set Ice.MessageSizeMax=1, no missed packet anymore but the size is 1024 bytes and I could use up to 1500.
    What can I do to maxmimize packets size?
  • benoit
    benoit Rennes, France
    Hi Philippe,

    You can use the Ice.UDP.SndSize property to configure the maximum datagram size Ice will send. If you make a request that is larger than this size, Ice will throw a DatagramLimitException exception (you will therefore have to be careful to send requests which aren't larger than this size).

    For batch requests, Ice provides a feature to automatically flush the batch when it reaches the message size limit of the transport, it can be enabled with the Ice.BatchAutoFlush property. Note that for UDP, the message size limit can be specified with both Ice.MessageSizeMax and Ice.UDP.SndSize. I recommend to only set Ice.UDP.SndSize. It's in bytes so you should be able to specify 1500.

    Cheers,
    Benoit.
  • Hi Benoît,
    Thank you for your help, I will continue my tests tomorrow.

    I succeeded to publish 570 000 upd/sec (600 Mb/sec) continuously with no packet missed or other issues :D