Because virtual machines have to be mapped on to the widest range of hardware possible, they have to make as few assumptions as they can (such as number of CPU registers in particular). This is the main reason why most virtual machines are stack based designs - almost all processors can implement one or two stacks fairly easilly.
The inverse isn't true. Some programming languages are based entirely on stack operations (Forth), but most are based on stack frames (C, Pascal, and their common ancestor ALGOL), or patternless memory access (FORTRAN, Smalltalk). Forth processors are effective because of the simplicity which comes from eliminating non-Forth features, but implementing a stack frame can be a real headache.
The Forth virtual machine contains two stacks. The first is the data stack, which consists of 16 bit entries (double entries can hold 32 bit values). The second is the return stack, used to hold PC values during subroutines.
The Forth equivalent to an instruction is a 'word', and can either be a predefined operation, or a programmer defined word made up of a sequence of executable words (the Forth version of subroutines, similar to Smalltalk). Forth also allows a word to be deleted with the "forget" word, normally only used for interactive Forth development (the language INTERCAL also includes a FORGET statement, but it is used for more evil purposes). Operations typically pull operands from the stack and push the results back onto it, which reduces instruction size since operands don't need to be specified. A subroutine is called by pushing the operands and executing the subroutine word, which leaves the results in the stack.
Operations can be either 16 bit or 32 bit, but there are two cases where types can be mixed - mixed multiplication will multiply two 16 bit numbers and leave a 32 bit result on the stack, while mixed division will divide a 16 bit (top of stack) number into a 32 bit number, producing a 16 bit quotient and 16 bit remainder (note that these two operations are directly supported by the PDP-11 architecture). There are I/O instructions as well.
The Forth two-stack machine has been implemented in the M17 CPU, among many others. The Transputer is stack oriented to a lesser extent (single evaluation stack only), and provides direct memory access abilities (for stack frames and other structures) without penalty.
As for Forth, although it has dedicated advocates, it's explicit stack orientation and its lack of modularity limit the scale of Forth programs. One of the largest Forth efforts was an integrated operating system called Valdocs (Valuable Document System) on the Epson QX-10. The software remained buggy couldn't be updated quickly enough for the machine to remain competitive - although you could just as easilly blame the computer's Z-80 processor (since at the time the 8088 based IBM PC and 68000 based Apple Macintosh were being introduced) and difficulty in finding experienced Forth programmers. Whatever the cause, this soured the acceptance of Forth for large scale projects.
Pascal, like Algol and C, is a stack frame oriented language, and so the p-Machine is a stack oriented machine. Memory is arranged from the top down as follows: p-System operating system code, system stack (growing down), relocatable p-Code pool, system heap (growing up), a series of process stacks as needed (growing down), a series of global data segments, and the p-Machine interpreter. The code pool contains compiled procedure segments in a linked list. Segments can be swapped into and out of memory, and relocated - if the stack needs more space to grow, the highest code segment can be relocated below the code pool. Similarly if the heap needs more space, code segments can be relocated upwards, and if both stack and heap need memory, code segments can be swapped out of memory altogether.
The UCSD p_System used a 64K memory map (standard for microcomputers of the time), but could also keep code in a separate 64K bank, freeing up data memory. The p-System also defined terminal I/O, a simple file system, serial and printer I/O, and allowed other device drivers to be added like any other operating system. It included an interactive program development system (all written in Pascal).
Western Digital implemented the p-Machine in the WD9000 Pascal Microengine (1980), based on the WD MCP-1600 programmable processor.
The JVM contains a stack stack used for parameters and instruction operands as in Forth, and a 'vars' register which points to the memory segment containing any number of local variables (like the workspace register in the Transputers).
Data typing is strongly enforced - while in Forth pushing two integers on the stack and treating them as a double is allowed, the JVM prohibits this. Object oriented support is also defined in the JVM, but not the architectual mechanisms, so implementation can vary. Objects are dynamically linked and can be swapped in or out (similar to the UCSD p-Machine, but the p-Machine segments are not grouped like objects and methods, and must be part of the program being executed, while JVM objects can be linked from external sources at run time). The other main difference between the JVM and the p-Machine is that the JVM memory segments (heap (data) and method area (code)) are not tied to a memory map, but may be allocated any way the operating or run-time system supports. Apart from that, the concept and implementation are quite similar (including multitasking support).
The Java language relies heavily on garbage collection, which is accomplished using a background thread and is not part of the JVM itself.
One other thing about the Java Virtual Machine is that some versions need to run code of unknown reliability which has been transferred over networks, and so includes security features (a "sandbox") to prevent a program from unauthorised access to the computer that it's running on.
Sun produced Java processors (starting with the picoJava CPU) to execute Java bytecode directly, faster than a virtual machine or recompiled code. Like most language-specific processors, it did not catch on.