[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] [PATCH V4] X86/vMCE: handle broken page with regard to migration
On 29/11/12 10:02, Ian Campbell wrote: On Wed, 2012-11-28 at 14:37 +0000, Liu, Jinsong wrote:Ping?Sorry I've been meaning to reply but didn't manage to yet. Also you replied to V4 saying to ignore it, so I was half waiting for V5 but I see this should actually be labelled V5 anyway. I'm afraid I still don't fully grok the reason for the loop that goes with: + /* + * At the last iter, count the number of broken pages after sending, + * and if there are more than before sending, do one or more iter + * to make sure the pages are marked broken on the receiving side. + */ Can we go through it one more time? Sorry. Let me outline the sequence of events and you can point out where I'm going wrong. I'm afraid this has turned out to be rather long, again I'm sorry for that. First we do some number of iterations with the guest live. If an MCE occurs during this phase then the page will be marked dirty and we will pick this up on the next iteration and resend the page with the dirty type etc and all is fine. This all looks good to me, so we don't need to worry about anything at this stage. Eventually we get to the last iteration, at which point we pause the guest. From here on in the guest itself is not going to cause an MCE (e.g. by touching its RAM) because it is not running but we must still account for the possibility of an asynchronous MCE of some sort e.g. triggered by the error detection mechanisms in the hardware, cosmic rays and such like. The final iteration proceeds roughly as follows. 1. The domain is paused 2. We scan the dirty bitmap and add dirty pages to the batch of pages to process (there may be several batches in the last iteration, we only need to concern ourselves with any one batch here). 3. We map all of the pages in the resulting batch with xc_map_foreign_bulk 4. We query the types of all the pages in the batch with xc_get_pfn_type_batch 5. We iterate over the batch, checking the type of each page, in some cases we do some incidental processing. 6. We send the types of the pages in the batch over the wire. 7. We iterate over the batch again, and send any normal page (not broken, xtab etc) over the wire. Actually we do this as runs of normal pages, but the key point is we avoid touching any special page (including ones marked as broken by #4) Is this sequence of events accurate? Now lets consider the consequences of an MCE occurring at various stages here. Any MCE which happens before #4 is fine, we will pick that up in #4 and the following steps will do the right thing. Note that I am assuming that the mapping step in #3 is safe even for a broken page, so long as we don't actually try and use the mapping (more on that later), is this true? If an MCE occurs after #4 then the page will be marked as dirty in the bitmap and Xen will internally mark it as broken, but we won't see either of those with the current algorithm. There are two cases to think about here AFAICT, A. The page was not already dirty at #2. In this case we know that the guest hasn't dirtied the page since the previous iteration and therefore the target has a good copy of this page from that time. The page isn't even in the batch we are processing So we don't particularly care about the MCE here and can, from the PoV of migrating this guest, ignore it. B. The page was already dirty (but not broken, we handled that case above in "Any MCE which happens before #4...") at #2 which means we have do not have an up to date copy on the target. This has two subcases: I. The MCE occurs before (or during) #6 (sending the page) and therefore we do not have a good up to date copy of that data at either end. II. The MCE occurs after #6, in which case we already have a good copy at the target end. To fix B you have added an 8th step to the above: 8. Query the types of the pages again, using xc_get_pfn_type_batch, and if there are more pages dirty now than we say at #4 (actually #5 when we scanned the array, but that distinction doesn't matter) then a new MCE must have occurred. Go back to #2 and try again. This won't do anything for A since the page wasn't in the batch to start with and so neither #4 or #8 will look at its type, this is good and proper. So now we consider the two subcases of B. Lets consider B.II first since it seems to be the more obvious case. In case B.II the target end already has a good copy of the data page, there is no need to mark the page as broken on the far end, nor to arrange for a vMCE to be injected. I don't know if/how we arrange for vMCEs to be injected under these circumstances, however even if a vMCE does get injected into the guest when it eventually gets unpaused on the target then all that will happen is that it will needlessly throw away a good page. However this is a rare corner case which is not worth concerning ourselves with (it's largely indistinguishable from case A). If the MCE had happened even a single cycle earlier then this would have been a B.I event instead of a B.II one. In any case there is no need to return to #2 and try again, everything will be just fine if we complete the migration at this point. In case B.I the MCE occurs before (or while) we send the page onto the wire. We will therefore try to read from this page because we haven't looked at the type since #4 and have no idea that it is now broken. Reading from the broken page will cause a fault, perhaps causing a vMCE to be delivered to dom0, which causes the kernel to kill the process doing the migration. Or maybe it kills dom0 or the host entirely. Either way the idea of looping again is rather moot. Have I missed a case which needs thinking about? I suspect B.I is the case where you are most likely to find a flaw in my argument. Is there something else which is done in this case which would allow us to continue? I think your analysis is correct -- the only question is whether B.I will 100% cause the migration to crash, or whether there's a chance of not crashing on the read. I had tried to ask that question before, and understood Jinsong's response to be that it's not 100% sure that the read would cause an error. However, looking back at the thread, I think I may have understood something that was not there. :-) So I guess the question for Jinsong is this:The only time this extra loop could help is if there is a page broken after being paused but before being sent the last time (B.I in Ian's analysis) -- in which case, the migration code is 100% guaranteed to read a now-broken page. What are the chances that this read of a broken page will *not* cause a fault which will kill at least the migration process, if not dom0? If the chances are "slim-to-none", then there is no point for the extra check. -George _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |