My "day job" is occupational therapy (OT), and as a Certified Hand Therapist (CHT) I treat conditions like yours for a living.
Three years is a long time to be living with the level of loss of function that you describe. Did you have therapy following your surgery? If you did, what kind of shape were you in when you were discharged?
Following a fracture of the distal radius bone (the most common type of wrist fracture) the last two things to recover are grip strength and the ability to bear weight on the palm of the affected hand. While people tend to recover most of their hand and wrist function in the first 90 days following surgical correction, strength and full weight bearing can take many months. I tell my post-surgical wrist fracture patients that 13 months or so is quite common before feeling completely normal again, provided that things went well. Adding other fractured bones into the mix, such as the distal ulna bone or one of the small bones of the wrist, can complicate things greatly. There are also ligament injuries that can be very significant which do not appear on X-rays.
By all means consult with an orthopedist who specializes in the hand. While waiting for that appointment (in my area the wait for a non-emergency orthopedist appointment can be as much as 3 months) get an appointment with either an occupational therapist or physical therapist who has advanced credentials as a Certified Hand Therapist (CHT). Don't wait; you have lived with this long enough.
Contrary to popular understanding, it tends to be occupational therapists rather than physical therapists who do hand and wrist rehabilitation. About 85% of CHT qualified therapists are occupational (OT) rather than physical (PT) therapists. To the public, it all seems to be physical therapy, but it ain't.
You can find a CHT on the Hand Therapy Certification Commission website, HTCC.org. Click on "Find a Therapist."
In the mean time PM me your address or fax number and I will send you a sheet of exercises that you can try when I am back in the office tomorrow.