I can't say I feel a lot of sympathy for Schwartz. Setting aside the vexing questions of whether the world is getting more homogenous, and how that relates to peace and standards of living, the issue is romantic primitivism and the Great White (or in this case Jewish) Anthrolopologist.
When Schwartz talks about reading Margaret Mead's Coming of Age in Samoa as a young man, I could tell right away he had latched onto the heroic image of the anthropologist, a noble hero who travels into an alien, distant environment, has adventures (sexual, psychedelic and otherwise) among the "primitive" peoples there, and returns to tell the tale and show off souvenirs. It's more PC than the Great White Hunter with his gazelle head mounted on the wall, but the underlying assumptions and fantasies are the same.
It isn't the lack of cultural diversity in the world that Schwartz mourns, but the lack of a space for him to be heroic, as opposed to just another scholar. His fantasy is untenable, and all that's left for him to do is evangelize the romantic primitivist fantasy to people who feel the same kind of "colonial nostalgia," mourning for what you yourself destroyed.
Silverberg hints a bit at this, but doesn't really go far enough. In both frames of reference, Schwartz's "Others", Dawn and the not-male, are standard male sexual fantasies, beautiful, supportive, uncritical and available. The not-male even lets itself be taken as "female" by Schwartz with nary a complaint.
That's my chief complaint of this story, that it doesn't get post-colonial enough, though I guess for a story written in the 1970s that's to be expected.