How is system call executed?

Answer 1

For a user most system calls are simply wrapped in simple command utilities such as chroot and mkdir and the like.

But system calls proper are made by software, plain and simple. Usually how it works on the low level is dependent on the architecture being used. For "simplicity" I'll refer to x86(_64).

When a system call is made, it's done by putting the parameters of the call in a series of general purpose registers in the CPU, then either signaling an interrupt or, in the case of 64-bit x86, actually using the syscall instruction.

What happens then is a context switch occurs, and the CPU is switched over to kernel mode and the kernel itself is actually running now and has your system call. Meanwhile, the process that made the system call is put in the "blocked" list until the needs of the system call are met.

Remember that a system call is a simple request for a service from the operating system by something in user space. Things in user space can't do a whole lot themselves in reality, and have to ask the operating system for a lot of support.

While an application can do a lot with the CPU, as soon as it needs any sort of hardware support, or more memory, or less memory, it has to ask the operating system for it. For example:

I wrote a text document in vi. To save, vi has to make a system call (Actually, a LOT of system calls.) to the kernel to actually write the buffer to disk.

Think of system calls as being among the building blocks of the actual execution of software on *nix systems. The process will happily use CPU instructions without operating system help, but applications make constant use of hardware in some way, and to do that, they have to use instructions for the kernel, who may or may not actually GRANT the requests based on permissions or sanity of the system call, as opposed to a CPU which will MOSTLY do whatever it is told even in user mode.

The only time a CPU will actually reject an instruction is if its bonkers. Either it's an opcode that makes no sense to the CPU or, more commonly, it is an opcode that requests memory be used in some way that is flat out disallowed in virtual memory. In those cases the CPU will usually fire off an exception and force a context switch to the kernel to handle the error. It'll be up to the operating system how to proceed, but usually, except in cases of page faults, this means death to the offending process. The cases where the CPU will reject an instruction because it's plain invalid or violates memory access rules is called a "fault."