Yea, I risked it all and decided to send the object flying around and move a bit faster when standing on it, no problem with it! The only problems I had was falling through it from the server not updating the object placement as fast as the client (Slightly slower on the server)
Movement is 100% done on the server. The client says "I want this object here" and the server says "Ok, now this object is here." Client doesn't update without server as there is no prediction for object movement.
More likely what you are seeing are little lag jumps where you receive a burst of movement messages, thus the object ends up moving faster than your character does for a second. Same results: latency = falling.