This filter uses a description file which follows this format :
x1,y1,x2,y2,level,downspeed,upspeed,mrl
...
where x1,y1,x2,y2 describe a sensible rectangle, level is the minimum difference for an active pixel, downspeed an upspeed are to integers.
mrl is a mrl to go when the filter has detected movement in the rectangle.

This is fun with vlm. For example, you should try to launch vlc normaly with vlm and then type :
new camera broadcast
setup camera enabled
setup camera input mrl to your camera
setup camera option filter=motiondetect,transform
#setup camera option transform-type=vflip
#indeed, I did not reach to make this working
control camera play

Have fun ;)