Pretty simple. Run rvm implode.

After that's done, check you .bash_profile (or similar shell startup script) for the line that loads the rvm environment. It looks something like

[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm"

Delete that line.

Finally, delete ~/.rvmrc, if it exists.

Close out all your terminals, and you're now free of rvm.