Sounds like the set screw or the collar may be stripped out. My guess would be the collar because it's made out of aluminum.
If working properly, it doesn't need to be tightened that much to hold. Just good and snug.
To replace the collar, the shaft will need to be removed.
If you, or someone you know is "good" with a drill and tap(as in don't drill through the shaft), you can drill and tap the collar to the next size up.
You may even be able to run a metric tap in it without drilling.