I didn't think that he mistook baby mice for marzipan, but rather that he picked up chunks of marzipan with baby mice in it. Presumably, the mice taste would be covered by the stronger flavors of the cake.
That said, the story still doesn't make sense because if this was the case, he would have eaten the entire mice. In order to leave half-eaten baby mice behind, he would have had to pick up half-mice to begin with.
On the other other hand (side note - I refuse to say gripping hand, as I utterly hated that novel), my initial reaction to the story was amused disgust, and it was only a while later that I started to second-guess the premise. Yeah, this isn't the strongest tale ever, but as far as horror flash goes, I'd much, much rather than more stories like this than incomprehesible semi-surrealist mishmashes of imagery like "Hunan Fare".