I think maybe we’re misunderstanding each other and I don’t want to go too far down a rabbit hole on this, but let me bring up the timesharing example I mentioned earlier as a real-life, practical way to describe what I’m talking about. On the DECsystem10 timesharing system, specifically, commonly used system programs were designed with distinctly separate instruction and data segments. The “instruction” or “code” part of the program was called the shareable high segment, and only one instance needed to be in memory to service all users, because it was always the same code. The other part of the program, the low segment, contained all the data the user was creating or working with, including all the tables that governed the settings of how the user’s particular instance of the program behaved. But the high segment was always identical for all users.
To this day this remains good programming practice. A program’s behaviour should be governed by control tables, not by munging the code. This is reflected in the Windows registry paradigm, where incredible numbers of things that govern how Windows and its applications behave are just values in the registry tables.
So is the code that you’re talking about more like a PDP-10 “pure code” high segment, or like Windows behaviour being strongly governed by registry settings, or is the code literally modifying its own instructions? If the latter, I’d suggest that it’s really bad software design.
ETA: To be clear, modifying a program’s instructions and modifying a program’s execution state are very different things!
The architecture is basically: a virtual machine is spun up using a fixed memory “image” that’s been developed by me; that machine reads data from a particular spot in the cloud; the machine writes and runs some functions to handle the data (and tries different things if it does poorly); the machine writes out data to a particular spot in the cloud; then the machine disappears forever (all changes are lost).
The whole thing is very abstracted away from the hardware, so it’s hard to compare, but I’d say the entire instance is “code”, while the cloud storage buckets are the “data”.
Well, I was talking about the lack of index registers on ENIAC. I suppose it could be called a Harvard architecture, but is code on a plug board really in an address space?
As for von Neumann, for my research I read a book about the UNIVAC patent trial, that invalidated E & Ms patent. The book was written by a historian who was married to someone who worked on ENIAC, and she was very pro von Neumann. von Neumann’s name is associated with his architecture because he issued a report about it not crediting anyone else. Who came up with the idea is a bit unclear, and in the book I just said it was controversial. A shame since I’m von Neumann’s academic grandson.
That’s cool but not self-modifying in the definition I was using. I once had a program that wrote a shell script to run ftp to get data, used that data to build another ftp script to collect new files, and then stopped. Generated and ran all sorts of code in one run, but I was more doing code generation than self modification. It sounds like that’s what you are doing also.
True, but it does match the definition suggested by @LSLGuy.
An instance spun up in a cloud is analogous to a program loaded by an operating system. Both an instance and a program are a chunk of code that is executed, separate from their input and output data. An instance can alter its own instructions and those changes don’t persist after the instance is deleted.
The IBM 360 (and therefore 370) had an Execute instruction with assembler mnemonic EX. It was sort of a virtual self-modifying code feature.
(After some research to refresh my aging now 50yo memory of all things 360…)
The EX instruction itself referenced one data register and one memory address via their typical [base register + index register + hard-coded offset] addressing scheme. The idea was to suck the instruction at the target address into the CPU, alter it there by ORing part of it with part of the referenced data register, execute that, and continue with the instruction below the EX. If the subkect instruction was a successful branch, the branch was taken in the normal way. None of which diddling altered the actual instruction in RAM that had been referenced. hence my “virtual” qualifier.
Depending on what kind of instruction was targeted, the twiddled bits of the target instruction could affect all kinds of wacky things that the instruction did.
I never quite got why they bothered with it, but that’s what it did.
The obvious use case is to create a template instruction with a skeleton, but lacking a couple of key fields, such as which register(s) to affect, or which branching condition to use, etc. Then use the data in the EX instruction’s data register to fill out the skeleton & do whatever. You might easily replace a vast if/else tree (or moral equivalent of a giant switch/case block) with a single table lookup, then stuff the magic bits from the table into the instruction & fire it.
Although the 360 instruction set was a fresh effort not deriving directly from prior IBM processors, ISTM they probably gave some thought to ease of migration and this was added since older machines had something similar.
I did all that research & writing, then I looked at wiki. There’s an article on the general idea. Of course there is.
Turns out the PDP-10 had an execute instruction too. It was privileged though so only the OS writers got to use it.
That’s pretty much what it was, which is why I question the wisdom of having such a thing. One potential use I read about was placing “execute” instructions within code for debugging purposes, where the targets in fixed locations would normally be a NOPs but could be patched with calls to debugging routines.
The PDP-10 XCT instruction was not privileged, it worked fine in user mode. The Wiki article is confusing because it describes a couple of privileged execute-type instructions, XCTR which was due to special paging hardware developed at MIT, and PXCT, which allowed the operating system to execute an instruction in a user process in the virtual address space of that process rather than in the kernel physical address space. But anyone could use the ordinary XCT. Most never did. I’d forgotten it even existed.
I think it’s significant that arguably the most important computers in DEC’s history, the PDP-11 and its successor, the VAX, had no such instruction despite the especially rich instruction set of the latter. The VAX had 244 instructions at introduction and more were added later, including such obscure privileged instructions as “Save Process Context” (SVPCTX) and “Load Process Context” (LDPCTX) which were specifically designed to facilitate context switching by the OS for multi-tasking/timesharing. In a real sense, the core job of the OS in switching process context was done by hardware, reduced to two instructions. But no “execute” instruction.
I ran out of time editing my earlier post. The current model 360-derivative IBM mainframe, the Z/13. still has EX. And a more advanced 64-bit version of EX (EXR) as well. Apparently somebody thinks its important.
I suspect I’m not far off in my idea of replacing a wall of comparison and branch instructions implementing a switch /case with computing the decision then using EX to inject the computed answer into an otherwise unreachable dummy / template instruction to implement the computed decision.
Thinking about this “Execute” instruction stuff, a late-night puzzle occured to me last night: how would you efficiently kludge this on a machine that didn’t have such an instruction?
The most immediately obvious solution is to just use a single “MOV” instruction to emulate “XCT addr” by moving the contents of “addr” to the next word in the instruction sequence, which would contain a placeholder like “NOP” at assembly time. The instruction would then behave just like “XCT addr” including the behaviour of “skip if true” types of test instructions.
Unfortunately this is seriously confounded by the fact that many machines like the PDP-11 have variable-length instructions, which turns out to very considerably complicate the problem. You could end up incurring a lot of overhead and on some architectures it may not even be possible at all (for example, because you might lose status code information that might not be recoverable, causing test instructions to perform incorrectly).
I prefer to console myself with the belief that it’s a pretty useless instruction anyway! The DEC VAX, the successor to the PDP-11 that was DEC’s last and most modern computer architecture, made by a company renowned for its technical excellence, specifically and deliberately did not have such an instruction. I like to think it was a repudiation of self-modifying code. “Here, execute this data that I just cobbled together” is just not a beloved concept in the realm of disciplined software development.
I don’t remember many details of VAX machine level code but I think the their JUMP instruction did nothing but change the program counter register so it was easy to direct execution to a memory location (possibly needed to be word aligned). That makes an effective XECUTE type by storing the custom op codes in memory. However, the custom opcode you store in memory has to be followed by a branch back to the next instruction from original code stream. This could be compiled into static code but there would be additional overhead to handle re-entrancy.
I agree with your general conclusion, “self-modifying code is just not a beloved concept in the realm of disciplined software development”. The problem potential is off the charts and it’s entirely unnecessary.
The IBM folks had the same variable length instruction issue. For your halfassed emulation, just compile a string of NOPs as long as the longest single instruction. Then twiddle and branch to that.
The key thing about the IBM 360+ EX opcode was that it couldn’t do anything. What it could do was alter most of the options, but only the options, of the existing hard-coded opcode and target address.
E.g. when ADD was EXecuted it still added. It could not possibly multiply or move or compare.
Whether it used register 2 or 12 as its indexer was modifiable. If it was written to use a base address of e.g. segment origin plus 0x12A3, it still did. Tweaking that stuff was beyond EX’s power.
Not really the same thing, though. Self-modifying code is dynamic within its own tenure. It sounds to me like your program composes functions and then jumps into them, but the functions are not modifying their code within their own tenure: the function code itself is non-dynamic.
I may not have been clear enough on what the potential problems are, but I don’t think it’s that simple. My idea is a cute solution for architectures with simple single-word instructions, but consider the following problems when using this method for variable-length instructions.
Sure, you can easily pad out enough NOPs to contain the longest possible instruction. But then your emulation has to examine the opcode and addressing modes of the target instruction to figure out how many bytes to copy and where to place it; since the instruction might be a “test-and-skip” type of instruction, it needs to be located at the end of NOP padding so it will appropriately skip (or not) when executed. But here’s the kicker. In order to figure this stuff out, the emulation probably needs to execute some test or compare instructions itself, which will set condition codes in the PSW. Most condition codes can (usually) be programmatically set or cleared but they can’t necessarily be directly saved. You’d have to test each one to determine its state, then set or clear appropriately just before your emulation code was done so the target instruction – which might be a test instruction – yielded the correct result, as it would with a hardware “XCT”. What a mess!
In short, emulating XCT when non-trivial code is required to do it may mess up the machine’s states in ways that a hardware or microcode based implementation would not.
“Execute” instructions don’t intrinsically modify anything. There’s nothing inherently wrong with them except that they could be considered enablers of potentially bad programming practices like dynamically modifying code. In terms of best practices, they’re kinda like the unconditional branch – the unfairly much-maligned “GO TO” statement in high level languages – which some (falsely) thought was the work of Satan because it led to “unstructured” code. The thing that actually leads to unstructured code is programmers with unstructured brains.
I now see and agree with your point. Writing a full-fidelity emulator to run on the machine you’re emulating has a lot of pitfalls.
And IMO you win the thread with this gem:
If you want to tear even more of your hair out, begin visiting The Daily WTF: Curious Perversions in Information Technology on Mon-Thu (IMO their Fri -Sun stuff is weak sauce). The part I enjoy most is the guys pontificating on what’s wrong with a 3-line hunk of admittedly bad code and they’ve utterly misunderstood it themselves.
Unstructured brains indeed. And untrained. The bane of modern IT is the number of utterly untrained untrainable monkeys pecking at keyboards for a living.
One of the questions on my orals was about a problem where you need some kind of loop exit, which is easy with a go to but a real pain if you have to put it into the conditional for the while statement, or, even worse, hack the loop variable.
I didn’t fall into the trap.
But the real problem with go to statements is that they lead to chaos if you code bottom up instead of top down.
And I can’t help remembering the ACM Computing Surveys special issue on structured programming, where the paper by Knuth began by mentioning a Professor Goto from Japan who wondered why everyone thought he was so harmful.
What trap? A “GO TO” is a perfectly fine way to exit a loop, and I can’t imagine any legitimate circumstance where it wouldn’t be part of a conditional anyway, like
IF ( {*condition*} ) GO TO {*statement*}
IOW, if some condition is true, forget this loop, I’m outta here! It’s indeed probably bad practice to have a bunch of unconditional "GO TO"s in a program, but its existence in original FORTRAN implementations (for example) harks back to the semantics of the “IF” statement which allows for any single valid statement to be executed if the condition is true.
Anyway, my vocation, before I retired and became a drain on society, wasn’t programming but software design, and specifically, the architecture of large software systems. From that perspective, I’ve never placed particularly high priority on coding disciplines compared to the importance of structured design and consistent, disciplined development methodologies. To me, the stricture of “never use a ‘GO TO’ statement” is exactly the equivalent of telling a writer to never end a sentence with a preposition. It’s the sort of thing that pedantic schoolteachers teach children and great writers ignore, and likewise the sort of thing that might be taught in CS101 but actual software designers couldn’t care less about.
The DEC VAX, the successor to the PDP-11 that was DEC’s last and most modern computer architecture, made by a company renowned for its technical excellence, specifically and deliberately did not have such an instruction. I like to think it was a repudiation of self-modifying code. “Here, execute this data that I just cobbled together” is just not a beloved concept in the realm of disciplined software development.