The idea of virtual machines (VMs) and the use of bytecode abstraction is central to modern computing, serving as an important link between programming languages and the actual hardware. The operational layer created by using virtual machines (VMs) is portable, secure, and consistent regardless of the number of different platforms used, because compilers will always have a stable instruction set to compile for. This architectural characteristic not only enables applications to run across different platforms but also influences how the program is executed, managed, and controlled. Knowledge of the fundamental issues surrounding virtual machine abstraction is essential to appreciating its capabilities.
Virtual Machines As Claims About Operational Equivalence
A virtual machine (VM) creates an intentionally designed environment that lies between the source code and the physical systems where it actually runs. Such an environment is usually characterized by a well-defined instruction set, a well-defined state model, principles for memory and control, and an execution agreement that can be repeated across various hosts. First, a compiler focuses on a single virtual instruction set, and each system then provides an interpreter, an ahead-of-time (AOT) translator, or a just-in-time (JIT) compiler for that set.
This style of intervention has produced some of the most useful forms of portability in computing. The use of bytecode (the encoded instructions of the VM) also enables the creation of a secure boundary, the coordination of different languages around a single object model, and the provision of services such as memory management, exception handling, verification, instrumentation, and dynamic loading, all using a singularly consistent vocabulary. There is nothing in the idea of a virtual machine that is in any way suspect, and any criticism that starts from the assumption that ‘virtual’ means ‘unreal’ has already mistaken a deliberately designed level of description for deception.
The relevant issue for this series arises from the functional definition of translation developed in part 1, Translation As Transfer Of Semantic Authority: translation can be defined as the mediation between regimes, in which one object, model, interface, or machine acquires the legitimacy to represent, describe, constrain, approximate, and implement another.
A bytecode-based virtual machine performs operational translation, together with representational translation, since virtual instructions and virtual states are authorized to represent computations ultimately realized through host instructions, runtime data structures, operating-system services, and physical state transitions. The virtual instruction set can thus be internally exact while the relationship by which it results in actual behavior remains outside the normal semantic account available to the program and its observer (the tooling or the programmer).
Bytecode-based virtual machine abstraction should not be considered to be nothing more than decoding. In fact, a bytecode operation like invokevirtual, newobj, or monitorenter can end up resulting in a chain of events that depends on class metadata, inline caches, allocator status, collector barrier operations, scheduling strategy, architecture-specific call conventions, and exceptions, all of which cannot be fully expressed semantically by saying that one opcode has been converted to many opcodes. What has crossed the boundary is authority over the meaning of an action: the virtual operation tells you what you did while you were running the program, while a changing complex of supporting operations on the host machine tells you what you really did with the computer.
The Preservation Contract Implicit In Virtual Execution
Assume that a small managed language routine updates the account balance and writes an audit entry.
credit(account, amount):
account.balance = checked_add(account.balance, amount)
audit.append((account.id, amount))
A compiler for the language could generate virtual instructions that load an object reference, read a field, call a checked addition primitive, write the field, allocate a tuple, and then invoke an append method.
At the virtual-machine level (the Dominant level), the relevant state consists of managed references, object fields, entries on the operand stack, method frames, an exception state, and possibly a memory-model relation which determines which writes other threads may observe. With respect to the host program, the object reference might be a compressed offset in a movable heap, field updates may need a card-marking barrier, and allocation could begin from a thread-local buffer followed by a stop-the-world garbage collection. Method invocation may select the compiled code via the versioned dispatch cell. Append could be blocked due to a lock, allocate storage, execute native code, or fail due to exhaustion of a resource outside the original source’s entire framework of reference.
To call these two accounts equivalent requires a declared set of observations.
The semantic obligations of a standard language-level contract could maintain the order of values and exceptions in the case of a single thread under an ignorance of allocation addresses, garbage collection pauses, cache access, and writing times that remain visible from another process’s point of view.
A stronger, and thus broader semantic contract could include the language memory model, failure modes, termination modes, changes to the security domain, and foreign function invocation.
However, a constant-time security claim would still require another observational boundary due to possible nondeterminism, since the equality of values alone is not sufficient to rule out timing, allocation, and memory access.
The reader can see that if I were to say that the “bytecode means the same thing on every conforming virtual machine,” it would be an incomplete statement unless I also defined the observed properties, the environment, and the permitted nondeterminism.
The virtual-machine abstraction is based on a number of standard assumptions. It is necessary for the bytecode to conform to a given format and version; if verification is relied upon, then it must be capable of establishing the properties that are later ascribed to it; the implementation must maintain the virtual transition relation for all pertinent states; the runtime services must fulfill their contracts; foreign code must not alter the protected state outside the scope of those contracts; and the host must offer the atomicity, ordering, isolation, and resource behavior which the implementation assumes.
These conditions are not embarrassing, since all implementations involve some assumptions, but they become epistemically significant when the VM is presented as if it were entirely self-sufficient; it is precisely these assumptions that determine the outcomes important to the argument being made.
Four Conditional Forms Of Fracture
Elsewhere in this series, these forms of fracture are referred to as consequences. The reader may learn more about these consequences in the first entry in the series, beyond the introduction: Translation As The Transfer Of Semantic Authority.
The initial kind of failure is what I’ve been referring to as Semantic Drift, that is, a situation in which the operational relationship changes even though the official Dominant description does not reveal this change.
For example, a change at runtime could entail replacing one collector with another, changing the scheduling of exceptions relative to allocations, tightening or loosening the memory barrier, or changing the definition of how a managed reference is represented and its lifetime. Where a lack of specification of a particular aspect of the language contract was intentional, such a change would not violate the Dominant semantics; but if we look in the opposite direction as well, it does change the relationship non-trivially between Dominance and Submission. Drift arises when an application, debugger, security argument, or performance contract depends on a feature that, semantically speaking, appears to be guaranteed by the VM’s public specification but that, in reality, the VM regards as accidental.
Another kind of failure is what I call Opaque Responsibility. Suppose that in the example above the audit append is observed twice, is omitted, or is reordered following a failure; in that case responsibility could easily be shared between the compiler’s emissions, the bytecode verification, virtual dispatch, the runtime’s exception handling, a native library, storage buffering, and the application’s recovery mechanism.
It might surprise the reader, after so much in this series, to learn that modular decomposition is not itself a refusal of responsibility: these duties can be allocated effectively when the decomposition is accompanied by proper logs and specifications. The problem of opacity rears its ugly head only when there are no conventional artifact records of what implemented the virtual operation and what Submachine effects were employed. Was it the provider, the runtime version, the generated implementation, or a recovery decision?
The Dominant trace indicates that an append took place; the operating system indicates that the bytes were written; and the gap between these two statements remains nothing more than an investigation carried out afterward in audits and debugs that rely on intuitive, incomplete, and unnecessary reconstructions.
The third type of discontinuity is epistemic, also called Epistemic Discontinuity: the programmer has both explanations available to him—the one from the program and the one from the substrate—but remains unable to integrate them into a single coherent narrative.
A source debugger can display the accounted-for object and the current line. A native profiler can display allocator frames and page faults. And yet, despite this embarrassment of riches, the identity relations connecting a specific virtual allocation, managed object, native region, collector relocation, and source expression may be partial, transient, or available only through privileged runtime internals.
Each tool provides valid knowledge within its regime, but the causal question spans regimes and currently requires reconstructions that could be rendered unnecessary given the appropriate insight and care.
The final failure, one of Conceptual Dishonesty, should be approached cautiously, as it doesn’t follow merely because a VM hides an implementation detail. Of course the VM hides an implementation detail. That’s its fucking job. Conceptual Dishonesty is justified when the virtual concept claims properties that are contingent on the denial of materially important Submachine truths:
- describing an operation as isolated while unreported native callbacks escape the sandbox,
- presenting an object identity as stable while tooling silently conflates distinct reincarnations (you have to bring its childhood toys to tell them apart), or
- describing an action as deterministic while provider selection and runtime policy introduce unacknowledged variation.
The dishonesty lies in the excess of the claim over its preservation contract, rather than in the act of virtualization.
The Bounded Epistemological Analogy
The Objectivist diagnostic used by the source essay, contained within Ayn Rand’s epistemological treatment Introduction to Objectivist Epistemology, treats a valid concept as an integration of differentiated referents whose particular measurements may be omitted without being denied, and treats higher concepts as dependent upon a retraceable conceptual hierarchy.
If this analogy is employed with care, which I stand for no less, it shows that a virtual operation can achieve both legitimate cognitive and engineering savings by combining multiple Submissive execution paths into a single operational unit, just as a concept can bring together a variety of particulars without needing to list all their individual measurements. There is no need for a managed operation to show off every register transfer when programming normally, since an abstraction that demanded constant attention to all the lower details would not succeed in abstracting.
The analogy sets out a clear boundary, however: whenever an operation can still be used as a symbol even though no officially recognized path allows one to determine which of the lower actions count as its referents, which distinctions have been deliberately left out, and which contextual factors justify their omission, the virtual instruction is similar to a floating abstraction. Once again, this statement serves as an analogy about accountability architecture, not as a claim that some VM is broken.
It should be stressed that the human capacity to form concepts is not dependent on machine-style provenance records, though any realistically useful hierarchy must be able to trace its lineage to objective reality. Instead, my analogy implicitly poses a question: what specific lower-level details are sufficient to justify the higher-level unit?
The engineering solution to that question must be derived from the specifications, implementations, evidence, and tests.
A Monotonic Redesign Of The Virtual Boundary
Bottom-Driven Semantic Monotonicity, the subject of this series, treats a Dominant machine (higher machine) as an extension of a chosen Submachine (lower machine), whose identity and actions remain authoritative when the Dominant machine is Projected onto the Submachine, a prerequisite known as the Dominance Condition of the Submachine.
In the case under consideration, the chosen bottom need not consist of highly excitable transistor states; it could instead be an operating-system process model together with a versioned native execution substrate, provided that the selection is made explicitly and no assertions are made that go beyond the depth of that bottom, capably speaking.
Let the lower machine be , comprising executable mappings, memory regions, threads, system-call events, and native transitions, and let the Dominant virtual machine include managed heaps, virtual frames, method identities, bytecode positions, exceptions, and runtime policies.
The redesign starts, as all things Aristotelian do, by assigning stable identities. A stable identity is assigned to each bytecode module, method body, instruction, runtime build, generated native region, managed object incarnation, and the relevant service provider. It is stable because the assigned identity is not reused, implicitly, explicitly, or complicitly if the item’s content or function changes. When a module is updated, or the code is recompiled, a successor is created rather than overwriting the provenance of previous executions. Although an object can be relocated in a way that preserves its managed identity and records of its previous locations, recovery and subsequent reuse of an address must not suggest that two incarnations were the same object.
The redesign then explains the support relationships in nauseating detail: virtual instructions cite their module and the corresponding range of bytes; when a method body gets activated, the verified format, the loader’s choice, and the runtime version on which it has been accepted are mentioned; generated regions refer to the virtual instructions, the assumptions behind it, and the compiler strategy used by it; and finally, the managed write refers to the lower region and the barrier operations by which it is performed.
A Projection provides an honest view of the relevant Submachine state without including any information not present within the Submachine. Since the Projection need not be one-to-one, multiple virtual facts may be based on overlapping lower regions, and a single virtual action may project into a trace rather than a single transition.
For inherited-action preservation, any Submission that remains available inside the Dominance’s context must retain its original, Submissive meaning. There shall be no change in the result of loading the native word merely because it is used as part of the managed field, and Dominant observations can be added within the run-time instrumentation without changing the definition of the Submachine event. The Dominant and Submachine transitions should commute under the given Projection for inherited actions, subject to explicit environmental nondeterminism.
Virtual actions need interpretive closure to be obtained from Submachine traces or from declared resolvers. When a method is invoked, the invocation resolver keeps track of the method’s identity, the dispatch inputs, the selected target, the cache state, and the Submachine code region; likewise, the allocation resolver records the allocator path, the identity that is produced, the Submachine writes, and any collector transition; and the monitor resolver records the scheduling and synchronization events whose collective total supports the virtual action. This proposed level of excruciating detail is not meant to imply the need to maintain all micro-architectural events indefinitely, but apparently the person who writes down his own intention isn’t the person who decides whether it’s valid these days, or so it seems, so who the fuck knows?
Instead, if his abusers permit it, it may appear as an inspection of the execution trace, a proof object that is provably correct, a cryptographic commitment, or a reproducible resolution by the resolver, all depending on the authority claimed and the retention policy in question.
Or maybe it’s just nonsense, because anyone but the author will decide instead of him, no matter if he writes the very same thing 127 times over the course of two years. Who cares about him, right?
The resulting Machine Tower derived from this technique also requires versioned activation and invalidation. Bytecode version, verification judgment, generated plan, and runtime assumption can go into an inactivated state without being deleted. The active view shows the current versions of the mechanisms, while previous views are available only to the extent necessary for audits, debugging, or accountability. Access is granted rather than universally public: the application developer might be able to see virtual-to-source lineage, the runtime engineer would enjoy access to lower-level code and allocator information, and the security auditor would have access to native call evidence under a given capability; secrets and other personal data would remain subject to bespoke access and deletion policies.
What This Architecture Costs, And When It Is Unnecessary
The redesign unquestionably increases costs for identity management, storage, synchronization, and instrumentation, and keeping cross-layer relations in place can prevent code from being moved, reveal sensitive details about the implementation, or increase the attack surface of diagnostic facilities. Fine-grained low-level traces could affect timing and generate so much data that analysis would be rendered unfeasible; stable identity would hinder recovery; runtime checks on projections could incur high overhead; and provenance would become a falsehood, since an erroneous or malicious resolver could generate data that is valid but incorrect.
Bottom-Driven Semantic Monotonicity defines responsibilities and identifies where evidence can and even should be attached, not necessarily when and if it should be collected, examined, used, or even if it’s true.
As we can see, there are a great many instances in which conventional methods are sufficient. In a controlled environment, a small bytecode interpreter can be justified by a mechanically carried out simulation proof of the transitions between its virtual and concrete systems, together with versioned binaries and standard debug metadata.
A transportable application that fails to take into account the history of retained allocation does not require any history to be recorded. Reproducible builds, signed runtime elements, test suites, verifiable bytecode, limited observability, and event logs are sufficient to ensure continuity without requiring that the production runtime serve as a history database.
These observations don’t serve as ad hoc proof that an approach consisting solely of monotonic extension is useless or unproductive. The monotonic approach is most appropriate in cases where the system makes cross-layer claims while executing dynamically, where versions and providers are changing, or where it is necessary to localize responsibility later across the virtual boundary.
Bottom-Driven Semantic Monotonicity is rendered useless and unproductive when it is erroneously seen as some sort of “proofing” routine that replaces an adequate, smaller proof for a sufficiently narrow question.
The virtual machine is an abstraction that is doomed only in specific conditions that, so far, haven’t been fully explored by the mainstream software trade, and instead have been relegated to the quirks and caveats of maintenance programmers gluing systems together. The abstraction fails when its universe of operation gains power that exceeds the limits set by the observations contained within it, while the physical connection that allows such power to be exerted remains inaccessible or irrelevant.
On the other hand, as outlined in this entry, when the virtual layer claims equivalence, shows enough ancestry, reveals its resolvers, and keeps its claims limited to the contents contained within the observations, it can be used as a means of reaching the intended goals of portability and safety without requiring the pretense that virtual execution must be unsupported execution.