That happened to me when "The undead guy" (don't want to spoil it) asked for once last favour, my character wasn't interested and looked somewhere else :P
Once in the middle of the FMV in Iron Mines in front of the seer, my character backed away. So when the FMV was over, I was standing 20 meters away from them.