Re: [Xen-devel] Merging xen/dom0/backend/blktap2 ..

[Daniel Stodden <daniel.stodden@xxxxxxxxxx>]

On Tue, 2011-03-15 at 23:02 -0400, Konrad Rzeszutek Wilk wrote: 
> > > Can you make a different branch (or just send me the git commit)
> > > for the one that touches mm/memory.c?
> > 
> > That's the initial one: 7ccd87f. You could strip that but it's not going
> > to build as a module before that ref is gone. It might go at some point.
> 
> For upstream we need that as a separate git commit. Otherwise we might
> not get MM maintainers Ack-ing the git commit b/c they would have to review
> the blktap driver too. Please do split it out in two commits.
> 

This piece is not ready for upstream, but you'd be right.

Proposing a zap_page_range export to mm maintainers will produce a
simple "you're doing it wrong", let's not split commits just for that.

My apologies, that detail dates back to the dawn of xen-blktap1.
Userspace is used to mapping a single large VMA, and the driver flips
ptes implicitly as I/O reqs leave and return again. That's machine-wise
not much of a difference, but for normal memory there's mmap/munmap
already, so module code is usually not encouraged to invent funny new VM
area models. Hence, zap_page_range remains private.

Can it just resort to mmap/munmap? Probably, even implicitly from ioctl
context I guess.

If it hopelessly got in your way, one could back it out, the loss is
TAP=m.

> > 
> > > > 
> > > > Does this sound acceptable?
> > > 
> > > I looked briefly at blktap/next-2.6.38 and it looks like an ongoing
> > > merge tree. Can you make a tree that is based off 2.6.38 (or some
> > > branch of my 'stable/' ones). Basically trying to have something that is
> > > easy to merge and is self-containted within one branch.
> > 
> > It is rooted on 2.6.32 and incrementally moves on up to currently
> > v2.6.38, no stray diffs except the forward porting where needed. There's
> > at least one conflict resolution in Kconfig around 36 or 37, iirc. Any
> > later kernel will look more or less the same.
> > 
> > $ git list xenbits/blktap/next-2.6.38 ^v2.6.38
> > eb8040c Merge commit 'v2.6.38' into blktap/next-2.6.38
> > bf926e6 blktap: Forward port to 2.6.37
> > 9f65e90 Merge commit 'v2.6.37' into blktap/next-2.6.37
> > 7ef4e35 blktap: Forward port to 2.6.36
> > a55f064 Merge commit 'v2.6.36' into blktap/next-2.6.36
> > 994a546 blktap: Add discard queue limits.
> > ea2d7e7 Merge blktap into blktap/next-2.6.33
> > e37d737 blktap: Add BLKTAP_OP_TRIM command option.
> > 42276df blktap: Add BLKTAP_OP_FLUSH command option.
> > 5d10876 blktap: Add physical sector device info.
> > 8169b25 blktap: Drop the ring message timestamp.
> > 34574a8 blktap: Support non-R/W requests
> > a765257 blktap: Fix reference to freed struct request.
> > 7ccd87f blktap: Blktap userspace devices.
> > 
> > You could rebase it on v2.6.38 if you want to reproduce the merging and
> > mangle commits, although I'm not sure why. For the foreseeable future,
> > I'd like to keep 2.6.32 and anything later going, so that repo will get
> > incoming stuff at 2.6.32 and, where possible, I'd expect the greater
> 
> Ah, ambitious. I tried that with xen-pcifront and had to resort
> to making trees for different versions. But this could work for you.

But it should work fine here. There's only a single source tree, which
is linux-2.6.git, and only a single component, which aims to yield the
same extensions where possible. Doesn't get much easier than that.

I'd agree that components interfacing with 4 moving xen subsystems, as
in blkback etc. is a bit of a different matter.

Still, killing history altogether can't be the answer.

> > kernels to typically just fast-forward from where they are now.
> 
> The issue is git bisection. The dates on those commits is quite early so
> I wonder if somebody did a git bisect whether they would hit those and
> end up with compilation issues. Perhaps the first patch should
> just introduce the driver, but not touch the Kconfig at all.
> 
> And then the last one actually enables the Kconfig. And as you
> add updates, you move the patch with Kconfig to be the last one?
> 
> > 
> > If you want topic branches: I'm not planning to do those. Mainly because
> > it's a bad idea. If that entire thing would be about to go upstream with
> > limited time to land that might a different thing, but it's clearly not
> > there in its present state.
> 
> What is a "bad" about it? What do you want to do when the driver is ready
> for upstream? You would have to do this topic branch at some point, wouldn't 
> you?
> What is a topic branch to you? When I think topic branch I think:
> devel/xen-pciback-0.5 for example.

When (iff) that driver gets ready for upstream I guess it'd have to be
folded down and optionally split, pretty much exactly as you describe.

This is my third attempt to write a proper response to your concerns,
because so far I'm not super-experienced with git either.

I was going to disagree because keeping separate queues is a bit of a
workflow killer.

Then I was going to agree because commit atomicity for proper bisection
is a actually pretty good point, imho.

The thing I hate about rebasing/cherry-picking is not even the extra
work, it's that the dupes mean one loses any kind of traction wrt what's
in a particular derived tree and not. So in order to be able to compare
two such branches one has to diff sources, git-blame, and understanding
a whole lot too much about the content.

The commit atomicity issue could be solved by making sure stay in a
branch, and only get merged, not pulled through by fast-forward. That
git-merge --no-ff option looks like it was meant to ensure that.

There's another way to make stuff atomic: Amending merges with the
following port. A git-commit --amend can do that. Took me ages to find
that out, because for reasons unclear, git-rebase -i alone (as 'fixup'
changes) can't.

I tried whether that's better. Can't recommend it. Git appears to have a
really hard time turning such ports/merges back into diffs, and having
those diffs is quite useful. It's not impossible, but format-patch alone
can't, which gets a bit counterintuitive.

> The reason I am asking about this is b/c of my inept git skills. Doing
> "git pull daniel/blktap" and just having it fetch patches that are relevant
> to blktap makes it soooo much easier to get an idea what is happening.

Well, it's not like it's going to pull garbage. These are all blktap
exclusively, rooted at v2.6.38. Just not ported individually, but in
piles.

Let's assume one would go through the pain to port those one after one,
by cherry-picking and careful rebuild. With 2.6.39 one had to start over
again, porting every commit once again, and making sure it compiles to
stay consistent, again for bisection purposes. And so on, every couple
months at least.

That continues, until this thing gets either dropped or upstreamed, at
which time an O(n*m) disaster gets folded down and discarded, one way or
the other. That not very efficient.

The reality is it looks like git can't deal with it any better that what
I got there.

So I'd rather ask you open a konrad/blktap-2.6.38, tracking that branch,
and carefully merge that in, in a way not risking bisection.

If that's okay, next question would probably be if one shouldn't try do
the same with blkback >:)

Cheers,
Daniel 


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel