It's not the direction in which the comments thread went, and so here rather than there, but my understanding has always been that the civil rights movement--and associated movements for recognition of racial and ethnic identities--have had a feminism problem, not the other way around. My history's a little rusty, but I recall that white women were among the strongest supporters of abolition and full civil rights for slaves and former slaves, with the understanding that pushing for black voting rights would also include their pushing for women's voting rights. This did not happen, obviously, with varying degrees of blame apportioned depending on which party you're listening to. But that seems to me to be the explanation for the whiteness of the feminist movement: an institutional legacy of things that happened 150 years ago.
On Schwyzer himself, I'm in agreement with Russell Arben Fox: he was a figure most interesting to me just before he became famous, when he was working in a more explicitly Christian frame of reference, and when he had a lot of sensible things to say about how men and women should interact. He seems to me now not unlike what one might imagine Augustine to have been like in the middle-early chapters of the Confessions, knowing that the things he does are wrong, completely unable to stop doing them, and tearing himself to pieces as a result, obvious disanalogies notwithstanding, including the fact that Augustine didn't do this process in public. He's a man with problems, and it'd probably be better for him if people stopped paying attention to him--not maliciously, not wishing him ill, but with the understanding that straightening his life out is by far the most important thing he has to do, not make amends to strangers over the internet.