We don’t know where our first impressions come from or precisely what they mean, so we don’t always appreciate their fragility