If it's been awhile since you've had the battery replaced it's probably it.
Quartz time pieces (this includes clocks, too) do funny stuff when the battery is dying. The second hands dance around.
I'd have it checked out.
Or it could be dirt or debris in the movement. Especially if it's a new watch.
Or, sometimes if it gets dropped it will dislodge crap in the case and gum up the works.
This can happen even in a quartz watch.
Hope this helps.