The Gem Mine is the last building you get to unlock for your kingdom, becoming available at Castle level 7. It produces the most valuable resource in the game, Gems 
, and limits you to one per kingdom.
Because of its value, the Gem Mine takes longer to upgrade than all of the other buildings in your kingdom. Its daily production stays in the double-digits, even in the higher levels of your kingdom, so upgrading the Gem Mine should be lower on your priority list.