What is a windowed room above the roof?

It is a Belvedere.

Definition according to Buffalo Architecture and History website:

A small lookout tower (usually square) on the roof of a house

Commonly found on Italianate Octagon, Second Empire, Greek Revival styles

The term cupola is a broader term that encompasses "belvedere."