I think the author of this article doesn’t really make his case with the dress example as immediately after saying “none of this explains why the visual systems of different people would automatically infer different ambient light” he adds that “one predictive factor seems to be a person’s typical wake-up time: night owls have more exposure to warmer, indoor light” which I would take as a perfectly good possible explanation of why different people make different inferences without any need for attributing that to innate differences in their visual systems per se.
Another point in the article that interests me is the reference to ‘aphantasia’. I’ve seen quite a few references to this recently, but am not convinced that it is real as opposed to being just a difference in how different people interpret the difference between an imagined scene and one that is actually seen. I have never had the experience of an imagined object being anything near so vivid as to be indistinguishable from reality, and so could perhaps think of myself as aphantastic; but on the other hand I have no reason to assume that others see their imaginings as more real than I do. Does anyone really “see” what they imagine with anything like the intensity of what they actually see in the real world? And if so, how would one quantify the difference so as to decide whether or not I am relatively aphantastic?
Source: The moral imperative to learn from diverse phenomenal experiences | Aeon Essays