Archived
This forum has been archived. Please start a new discussion on GitHub.
Glacier2's "DoS protection" (Feature proposal)
Hi,
I have read the newsletters that introduce Glacier2 and state its ability to sleep 500ms as "DoS protection" or so. I also have read old forum entries where you say that it not really is one.
This is kind of a follow up to my last post where I wanted to implement DoS protection in my server w/o using glacier2 and the main answer was "use Glacier2", I can understand this as Glacier2 seems useful, then I made some tests.
(As always, I can't seem to keep my posts short
)
my proposal:
like you have perfectly modularized-out the session manager and the session verifier in glacier2, allow the user to give it a spam protector module. If the user doesn't create one such object, just don't use it. If he does, call a method from it upon every message reception along with the IP the message comes from and let the spam protector tell you whether it is spam or not. If it is, somehow deny the message right away.
If the user either compiles the spam protector right into glacier2 (best solution) or puts it into another server on localhost, the main server would not be affected by this, as no spam-message would ever come to him, leaving him his CPU cycles for his precious work. What do you think? Is it possible? Is it worth it? Is it naive?
Probably you'll say there is no commercial demand on this, so no money, so no time to implement. If so, if I'd try an implementation sometime in the future and it works, would you be interested by a patch?
So, here comes my reasoning/test:
I made my simple tests, here is how to reproduce it in case you're interested:
cpp/demo/Glacier2/chat/client.cpp, around line 184, change the code to the following:
Add this to the glacier2 config file:
Start glacier, the server and five clients on localhost.
"login" with 3 of the clients.
on the first client, type "/spam 10000000", wait 1 sec, analyze: it seems we can spam 500ms, then it sleeps 500ms, then we spam 500ms, ... OK, fine.
try to "login" with the two last clients --> TimeOut
check if the others see the spammed messages --> sometimes, sometimes not.
run: top: oh, it's the server that has full load, followed by glacier2! just because of one misbehaving client! The server trusted Glacier2 to get protection (you call Glacier2 a "firewall", stupid naive programmers like me trust it to protect them from everything
)
Definitely, this "feature" is not really a spam protection. I did the following now:
In the server, ChatSessionI.h, add a private int member _iBal to ChatSessionI,
ChatSessionI.cpp: put iBal(200) into the ChatSessionI's constructor's init list.
Line 115 (at the beginning of ChatSessionI::say):
Run the same tests as described above again.
You will now notice that the two last "logins" succeed with only a little delay (!), the other clients won't get spammed and last but not least the other clients can "talk" together with only little delay.
I started "top" on my dual core notebook, it shows me that now glacier2 uses 70%, the server 30, the client 15 and the Konsole 30.
Result: the other clients are no more disturbed by the one misbehaving, the server no more uses full CPU. The high CPU of glacier2 probably comes because of enabled tracing, the 30% of Konsole support this hypothesis.
I attached the client.cpp and ChatSessionI.cpp in case you're too lazy but want to try it out;)
Thanks for reading.
PS: Even if you think it's completely stupid, nonsense and overkill, I will be more then happy to have discussed this with experts :cool:
I have read the newsletters that introduce Glacier2 and state its ability to sleep 500ms as "DoS protection" or so. I also have read old forum entries where you say that it not really is one.
This is kind of a follow up to my last post where I wanted to implement DoS protection in my server w/o using glacier2 and the main answer was "use Glacier2", I can understand this as Glacier2 seems useful, then I made some tests.
(As always, I can't seem to keep my posts short
)my proposal:
like you have perfectly modularized-out the session manager and the session verifier in glacier2, allow the user to give it a spam protector module. If the user doesn't create one such object, just don't use it. If he does, call a method from it upon every message reception along with the IP the message comes from and let the spam protector tell you whether it is spam or not. If it is, somehow deny the message right away.
If the user either compiles the spam protector right into glacier2 (best solution) or puts it into another server on localhost, the main server would not be affected by this, as no spam-message would ever come to him, leaving him his CPU cycles for his precious work. What do you think? Is it possible? Is it worth it? Is it naive?
Probably you'll say there is no commercial demand on this, so no money, so no time to implement. If so, if I'd try an implementation sometime in the future and it works, would you be interested by a patch?
So, here comes my reasoning/test:
I made my simple tests, here is how to reproduce it in case you're interested:
cpp/demo/Glacier2/chat/client.cpp, around line 184, change the code to the following:
if(s == "/quit") {
break;
} else if(s.substr(0,5) == "/spam") {
std::stringstream ss(s);
std::string sCmd;
size_t n = 1000;
ss >> sCmd >> n;
std::cout << n << std::endl;
ChatSessionPrx oneway;
try {
oneway = session->ice_oneway();
for(size_t i = 0 ; i < n ; i++) {
oneway->say("BlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBlaBla");
}
} catch (const Ice::NoEndpointException&) {
cerr << "No endpoint for oneway invocations" << endl;
}
}
Add this to the glacier2 config file:
Glacier2.Client.SleepTime=500 Glacier2.Server.SleepTime=500
Start glacier, the server and five clients on localhost.
"login" with 3 of the clients.
on the first client, type "/spam 10000000", wait 1 sec, analyze: it seems we can spam 500ms, then it sleeps 500ms, then we spam 500ms, ... OK, fine.
try to "login" with the two last clients --> TimeOut
check if the others see the spammed messages --> sometimes, sometimes not.
run: top: oh, it's the server that has full load, followed by glacier2! just because of one misbehaving client! The server trusted Glacier2 to get protection (you call Glacier2 a "firewall", stupid naive programmers like me trust it to protect them from everything
)Definitely, this "feature" is not really a spam protection. I did the following now:
In the server, ChatSessionI.h, add a private int member _iBal to ChatSessionI,
ChatSessionI.cpp: put iBal(200) into the ChatSessionI's constructor's init list.
Line 115 (at the beginning of ChatSessionI::say):
if(_iBal >= 0)
--_iBal;
if(_iBal < 0)
throw(Ice::ConnectionRefusedException("FU SPAMMER", 1));
(I picked a random exception and implemented really stupid spam protection, this is just a quick proof of concept).Run the same tests as described above again.
You will now notice that the two last "logins" succeed with only a little delay (!), the other clients won't get spammed and last but not least the other clients can "talk" together with only little delay.
I started "top" on my dual core notebook, it shows me that now glacier2 uses 70%, the server 30, the client 15 and the Konsole 30.
Result: the other clients are no more disturbed by the one misbehaving, the server no more uses full CPU. The high CPU of glacier2 probably comes because of enabled tracing, the 30% of Konsole support this hypothesis.
I attached the client.cpp and ChatSessionI.cpp in case you're too lazy but want to try it out;)
Thanks for reading.
PS: Even if you think it's completely stupid, nonsense and overkill, I will be more then happy to have discussed this with experts :cool:
0
Comments
-
Hi,
As you discovered, The SleepTime properties have nothing to do with throttling client or server requests. Their main purpose is to allow the batching of client or server requests to improve throughput at the expense of latency. There are currently no ways to throttle client requests at the Glacier2 level, supporting this is on our TODO list. In any case, this is something you can easily implement in your backend servers.
In general, Glacier2 trust clients that successfully authenticated and established a session and at this point it's the responsibility of your application to ensure that the client uses the backend services in a sensible way. The easiest way for your application to ensure this is to restrict the client to only talk to few Ice objects: facade objects such as the session Ice object created by the session manager (incarnated by the ChatSessionI servant in your case). This way, you can do all the checks here and eventually add throttling a bit like you described.
I wouldn't throw ConnectionRefusedException though as this won't cause the client session to be closed. Instead, once your chat session implementation detects that it's dealing with a mis-behaving client the best is to destroy its session. You can do this through the Glacier2::SessionControl object provided with the session manager create method. So in your small proof of concept, I would do something like the following instead:if(_iBal >= 0) --_iBal; if(_iBal < 0) _sessionControl->destroy();
This will ensure that the client can't continue its spamming and consume un-necessary CPU for Glacier2 and your session server. You can then add a mechanism to temporarily blacklist this given client and prevent it from re-connecting right away (i.e.: the implementation of your session manager would lookup the list of blacklisted client and throw CannotCreateSessionException if the client is blacklisted).
Cheers,
Benoit.0
