The prime minister was from not royal blood so he ruled from behind the throne by marrying three successive queens. His house, which is necessarily lower on the hill than the royal palaces and smaller, is now the museum.
AiE in Madagascar I: Fleeced at the Palace of the Mad Queen & Other (Poignantly) Colonial Experiences in Madagascar