Yes, queen should be capitalized if it comes at the beginning of the sentences, if it is a part of a title or if it replaces the name of the person, but if it comes as a general noun, there’s no need to be capitalized.
Examples: Queen of the United Kingdom visited this country last week.
At your service, my Queen!
this picture belongs to Queen Elizabeth II.
She is the queen of this country.