The insistence that Chesterton's anti-Semitism needs to be understood "in the context of his time" defines the problem, because his time--from the end of the Great War to the mid-thirties--was the time that led to the extermination of the European Jews. In that context, his jocose stuff is even more sinister than his serious stuff. He claims that he can tolerate Jews in England, but only if they are compelled to wear "Arab" clothing, to show that they are an alien nation. Hitler made a simpler demand for Jewish dress, but the idea was the same. Of course, there were, tragically and ironically, points of contact between Chesterton and Zionism. He went to Jerusalem in 1920 and reported back on what he found among the nascent Zionists, whom he liked: he wanted them out of Europe and so did they; he wanted Jews to be turned from rootless cosmopolitans into rooted yeomen, and so did they.
Chesterton wasn't a fascist, and he certainly wasn't in favor of genocide, but that is about the best that can be said for him--and is surely less of a moral accomplishment than his admirers would like. He did speak out, toward the end of his life, against the persecution in Nazi Germany, writing that he was "appalled by the Hitlerite atrocities," that "they have absolutely no reason or logic behind them," that "I am quite ready to believe now that Belloc and I will die defending the last Jew in Europe." Yet he insisted, "I still think there is a Jewish problem," and he denounced Hitler in the context of a wacky argument that Nazism is really a form of "Prussianism," which is really a form of Judaism; that is, a belief in a chosen, specially exalted people. (For what it's worth, although he mistrusts Judaism, he detests Islam; Judaism is merely pre-Christian but Islam is a kind of parody Christianity. All the favorite historical arguments for Jesus--that he had to be either crazy or right, and he doesn't seem crazy; that he changed the world with a suddenness not plausible in an ordinary human; that the scale of the edifice he inspired is proof of divine inspiration--apply just as well to Muhammad, and they can't both be the guy.)
The trouble for those of us who love Chesterton's writing is that the anti-Semitism is not incidental: it rises from the logic of his poetic position. The anti-Semitism is easy to excise from his arguments when it's explicit. It's harder to excise the spirit that leads to it--the suspicion of the alien, the extreme localism, the favoring of national instinct over rational argument, the distaste for "parasitic" middlemen, and the preference for the simple organ-grinding music of the folk.
If there's a reason my sympathies are ultimately cosmopolitan, it's for precisely this reason. Love of place implies a dislike for other places. Love of a people implies a lack of love for peoples other than one's own. It's an ugly truth, but one that probably needs to be said, that a love for the local is often accompanied by an extreme discomfort for the new or unusual, which often appears as racial or ethnic denigration, dislike of the city, etc.
Gopnik has very interesting things to say on Chesterton: how luminous and interesting his early work was; how his conversion affected his work (by making it more humorless and at times insufferable; one puts up with certain aspects of his writing until he gets to his talents); and how the radical change in English prose style after World War I made him a curiosity even before he died.