There is a nerve that runs along that part of your elbow called the ulnar nerve. It gives sensation to your pinkie finger and a part of your ring finger. It runs along the "backside" of your elbow. So bashing your elbow in that area sometimes compresses your nerve there and causes pain/weird sensations in that area. You're directly smashing/compressing the nerve when that happens.
The sensations to the rest of your fingers are from different nerves. One of the other main ones is called the median nerve. It does run along your arm as well, but in a pathway that is not as exposed at the elbow like your ulnar nerve is. It runs along your "elbow pit" area, so bashing your elbow doesn't affect it at all.
But the median nerve is commonly compressed at the wrist. This happens in carpal tunnel syndrome, where you can get pain/tingling/weird sensations in your remaining fingers.
Other nerves can be compressed in other ways, but the above two are probably among the most common.
Check out this image of what nerve supplies sensations in any given area of the arm/hand:
https://m.instantanatomy.net/diagrams/arm061.png
Here is an image that shows where the main nerves run along the elbow. This view is looking at the "elbow pit" head on. The backside of the elbow is essentially behind this image if that makes sense.
https://lemmy.world/pictrs/image/93de5fe2-a798-435e-beff-fbc42897d839.png
Anatomy stuff is pretty cool, imo. I guess I'm a bit of a dork for stuff like that.