Are there any programs that let me assign custom action such as changing the volume on mouse gestures (right click+scroll wheel), similar to Volumouse on Windows?
I haven't heard of a new tool for some time, but SteerMouse might be worth a look, but it depends on your current OS version. I'd verify it would work with you machine before you download/purchase.
Using SteerMouse (thanks @CrashBurner), click "Chords", then the "+" on the bottom left, add shortcut and assign action (under the Music Control category).
Note in the image below: Roll Down = Volume Up is because the "natural scrolling" nonsense (for standard mouse wheels) was fixed using "Standard wheel direction" in SmoothScroll.