I'm not sure I'd call X86 segmentation "elaborate", at least in the context of X86 programming (sure, it's very elaborate compared to MIPS).
I don't think I've heard the word "call gate" used with the definite article before, as if there was just one of them... but I'm an X86 autodidact and that could be my mistake. My understanding is that a call gate is anything that vectors a program from one context to another. In most X86 operating systems, there are 2-3 basic call gates that will get you from userland to kernel: the INT instruction (the interrupt handler will check your program state and dispatch the right system call) and the SYSCALL instruction (which does the same thing without the interrupt overhead).
NaCl disallows both of these instructions, along with the FAR CALL opcode that would let you jump between segments and the segment override prefix that does the same (note this was the epic fail Dowd found in the contest).
The trampoline mechanism that NaCl uses is not at all dissimilar from how Win32 and BSD libc issue system calls; the library exports a stub interface and hides the mechanics of actually issuing a system call.
Note: not trying to be pedantic here. Just love geeking out on this stuff.
The x86 instruction set has a mechanism called "call gates" for system calls. Basically, the OS puts the entry point of the system call handler into a segment descriptor with the call gate bits set. The unprivileged user program then performs a far call to an address consisting of a segment selector for that descriptor and an offset which does not matter. Execution resumes at the system call handler, with a privilege level as encoded in the call gate descriptor.
That way, you could have thousands of system call entry points and avoid the overhead of an int instruction and the syscall-number dispatch. I believe OS/2 used that mechanism extensively (and all the other elaborate segmentation stuff).
Some other notes:
I'm not sure I'd call X86 segmentation "elaborate", at least in the context of X86 programming (sure, it's very elaborate compared to MIPS).
I don't think I've heard the word "call gate" used with the definite article before, as if there was just one of them... but I'm an X86 autodidact and that could be my mistake. My understanding is that a call gate is anything that vectors a program from one context to another. In most X86 operating systems, there are 2-3 basic call gates that will get you from userland to kernel: the INT instruction (the interrupt handler will check your program state and dispatch the right system call) and the SYSCALL instruction (which does the same thing without the interrupt overhead).
NaCl disallows both of these instructions, along with the FAR CALL opcode that would let you jump between segments and the segment override prefix that does the same (note this was the epic fail Dowd found in the contest).
The trampoline mechanism that NaCl uses is not at all dissimilar from how Win32 and BSD libc issue system calls; the library exports a stub interface and hides the mechanics of actually issuing a system call.
Note: not trying to be pedantic here. Just love geeking out on this stuff.