No, whose doesn’t always refer to a person but it could refer to things also and this is depending on its position in the sentence, if it comes as interrogative pronoun, it refers to a person, but if it comes as a relative pronoun, it refers to things and objects.
Examples: Whose mother are you?(interrogative pronoun).
The house, whose color is red is mine. (relative pronoun).