64MB won't fly. An empty site will take 200MB.
Are you talking about the memory allocation/limits being set on the app pool?
I run Orchard on a VM with 1GB memory for my test server and it's usable. I would recommend a VM with 2GB memory in the system. I have no experience with setting max physical limits on app pools.