A development virtual machine can be really handy. It gives you a sandbox of sorts where you can feel free to test and experiment knowing that in a worst-case scenario you can just delete the VM and start over. It can also be a great way to practice server configurations and sketch out "real-world" server setups. Here's the process I follow to setup my Ubuntu 10.04 LTS Drupal development VM in VirtualBox. Aside from the VM-specific steps, these instructions should work for a regular Ubuntu server (VM or not).
This is a collection of notes I took while setting up a Drupal-ready Apache / MySQL / PHP environment on OS X using Homebrew and PEAR / PECL. I believe it should work to recreate an environment from scratch, but I've not run through the notes from start to finish on a fresh environment to validate. Hopefully this will help you get your own environment up and running easily!
Note: Mark Sonnabaum's Megalodon might be of interest to you, as well. For me, it involved learning a few too many things, and I also wanted as much as possible in my environment to be handled via a package manager like Homebrew or PEAR.
Update: Per Mark, Megalodon does entirely use Homebrew and Chef, I just misunderstood it. I look forward to seeing what I can do with it, once I dig in and understand it a bit more.
Update: Thanks, @stevepurkiss, for pointing out the typo on the first line for
brew tap... That's been fixed!
Update: Thanks, @cashwilliams, for pointing out some issues with the my.cnf and order of packages being installed. I've updated a couple of the instructions below to reflect his findings.
Update: I've added instructions for initial setup of Drupal coding standards for PHP Code Sniffer.