Re: [Xen-devel] xenconsoled CPU denial of service problem

[Keir Fraser <Keir.Fraser@xxxxxxxxxxxx>]

On 29/8/06 4:59 pm, "Daniel P. Berrange" <berrange@xxxxxxxxxx> wrote:

> Ah, that's very handy. Looking at the xenconsoled code I think I've worked
> out how to slow things down - although I'm not entirely sure I'm correctly
> activating the throttling - basically I'm delaying the calls to the methods
> xc_evtchn_notify & xc_evtchn_unmask after consuming data - is this the
> correct approach ?

Yes, the essence of your approach is exactly right. Delaying the unmask is
the critical thing -- there's not really any need to delay the notification
(in fact I don't think you should bother). But by delaying the unmask you
avoid any possibility of a DoS attack on that interdomain link -- without
this the domU could potentially hose the dom0 with lots of pointless
event-channel notifications. This is theoretically a problem on every one of
our split drivers (xenstore, blk, net) so it'd be good to think of the most
general possible solution we can think of here that we can apply to every
backend.

> The patch sets
>   - data size 5 kb
>   - period 200 ms
>   - delay 200 ms

A few comments:
 * I think the 'delay' parameter is not really useful. Think of this simply
as a simple credit-based scheduler that replenishes credit every 'period'.
So in this case you get 5kB every 200ms. If the domU offers more data in a
period, it must wait until its credit is replenished at the end of the
current 200ms period.
 * I'm not sure bytes of data is the right thing to limit here. The main
thing that hoses domain0 is the repeated rescheduling of the console daemon,
and that is fundamentally caused by event-channel notifications. So it might
make sense to rate limit the number of times we read the event-channel port
from xc_evtchn_pending -- e.g., no more than 10 times a second (should be
plenty). This has a few advantages: 1. Looking just at data transferred
doesn't stop a malicious domain from hosing you with no-op event-channel
notifications; 2. This is a fundamental metric that can be measured and
rate-limited on all backend interfaces, so perhaps we can come up with some
common library of code that we apply to all backends/daemons.

It may turn out we need to rate limit on data *as well*, if it turns out
that sinking many kilobytes of data a second is prohibitively expensive, but
I doubt this will happen. For a start, the strict limiting of notifications
will encourage data to get queued up and improve batching of console data,
and batches of data should be processed quite efficiently. This same
argument extends to other backends (e.g., batching of requests in xenstored,
netback, blkback, etc).

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel