In the simplest terms, regardless of frame size, the back strap on the square butt stays straight, while the back strap on a round butt curves inward.
This refers to the grip frame itself - not the shape of the stocks, which can extend significantly in front of the grip frame and can also extend behind the grip frame, creating a square butt shape on round butt frame.
The picture below contains a 6" square butt Model 66, and a 2 1/2" round butt Model 66. The primary different is the sharply inward curving back strap on the round butt, which comes to a point where the angle is greater than 90 degrees. In comparison, the square butt comes to a point where the angle between back strap and bottom is less than 90 degrees.
There is also less curve to the front strap on the round butt to keep it from flaring are far forward, but that's not the definitive trait across all frames.
The critical dimensions and internals shape however are identical on these K-frame revolvers, and if some one wants to do so, they can remove metal from the square butt grip frame to create a round butt grip frame.
IMHO, the square butt revolvers are a dying breed. Ruger for example no longer uses a grip frame at all but rather uses a grip stub. This allows much more latitude in grip sizes for smaller or larger hands and provides for a better fit for customers, particularly on large frame revolvers.
Other companies, including S&W are leaning heavily toward round butt models in their current production as you can add a grip that provides the square butt shape, but not the reverse.