Out of curiosity, how much memory does your host have? Are you oversubscribing?
Also, what kind of performance degradation are you seeing on B? Are you possibly being bitten by the spectre/meltdown cpu "fixes" masquerading as other problems?
Regardless of how much memory you allocate to a linux machine, it will always use as much memory as it can. So if you gave your host 100GB of memory, it would try to find a way to use up pretty much all of it, even if that means caching a file that you used one time in memory. So every time you increase the memory on a domu, it will look like it's just sucking it up, when in reality, it's just taking advantage of the resources that you provided it.
FWIW, on my Debian Stretch Xen host at home, it has 4GB of memory total. I've allocated a fixed 512MB to dom0, 2GB to one domu, and 384MB to three other domu's, and it runs circles around my previous vmware host, even when running intensive stuff like my Minecraft server that I host for my friends and I.
Chris