As with "The Leviathan", I think it's a mistake to fault this story for not measuring up to Lovecraft when it doesn't seem to be trying to emulate that style. "Stranger arrives in town of oddballs" is a fairly venerable start to many a horror story in many different styles (Blackwood's "Ancient Sorceries", etc.). This story's Robert Aickman touches felt very much like his "The House of the Russians" to me, and Aickman's style is slow psychological boil, not pulp front-loading. There are many ways to skin a cat.
Thanks for listening.
“The best thing to do is to loosen my grip on my pen and let it go wandering about until it finds an entrance. There must be one – everything depends on the circumstances, a rule applicable as much to literary style as to life. Each word tugs another one along, one idea another, and that is how books, governments and revolutions are made – some even say that is how Nature created her species.”
Machado de Assis, “Those Cousins From Sapucaia”