Every server is different. You run different software for a different audience. So why should your VPS provider force you into a one-size-fits-all OS distribution or architecture?
We give you a choice. You choose the architecture: 32-bit or 64-bit. You choose the OS: Debian, Ubuntu, CentOS, Fedora, or Windows. We give you a choice of recent versions of each too.
Different workloads benefit from 32-bit or 64-bit environments.
In most situations, the memory footprint of a 64-bit environment is only slightly more than that of a 32-bit environment. The 64-bit environment will often perform better too.
However, there is one broad class of exceptions to that rule—memory-hungry applications built in dynamic script languages. PHP, Ruby, Perl, Python, and the like all tend to use up a lot of extra memory in 64-bit environments. For this reason, if you're building an application server that will be running one of these languages, you may be better off with a 32-bit server.
On the other hand, memory-intensive applications, like databases and other similar data-stores, usually perform better in 64-bit environments.
In general, unless a scripting language-based application is a significant part of what runs on your server, a 64-bit server is probably your best bet.