[Keith Webb]

Wasn't it a brass frame used to keep a pyramid of canon balls in place which contracted with cold causing the balls to be displaced...

But then again, a little research reveals this may be false (although it is common useage now).

enough to freeze the balls off a brass monkey" is nonsense because:


Not even the venerable Oxford English Dictionary, records a usage of "brass monkey" like the one presented here.

When references to "brass monkeys" started appearing in print in the mid-19th century, they did not always mention balls or cold temperatures. It was sometimes cold enough to freeze the ears, tail, nose, or whiskers off a brass monkey. Likewise, it was sometimes hot enough to "scald the throat" or "singe the hair" of a brass monkey. These usages are inconsistent with the putative origins offered here.

Warships didn't store cannonballs (or "round shot") on deck around the clock, day after day, on the slight chance that they might go into battle. Space was a precious commodity on sailing ships, and decks were kept as clear as possible in order to allow room for hundreds of men to perform all the tasks necessary for ordinary ship's functions. (Stacking round shot on deck would also create the danger of their breaking free and rolling around loose on deck whenever the ship encountered rough seas.) Cannonballs were stored elsewhere and only brought out when the decks had been cleared for action.

Particularly diligent gunners (not "masters," who were in charge of navigation, sailing and pilotage, not ordnance) would have their crews chip away at imperfections on the surface of cannonballs to make them as smooth as possible, in the hopes that this would cause them fly truer. They did not leave shot on deck, exposed to the elements, where it would rust.
Nobody really knows where the phrase "cold enough to freeze the balls off a brass monkey" came from, but the explanation offered here certainly isn't the answer.

Source