You can check here for getting the latest version. Change the wget url to download newer versions.
LAMP stack is a group of open source software used to get web servers up and running. The acronym stands for Linux, Apache, MySQL, and PHP. Since the virtual private server is already running Ubuntu, the linux part is taken care of. Here is how to install the rest.
If any existing lamp installation & you need to upgrade your old server, uninstall all things & backup first all databases & files.
sudo apt-get purge php5*