Tarik Hireche

Shutdown after SIGTERM

2.5 to 3.1 ms

Seven runs, worker supervised by sentinel on PID 1 of a PID namespace, timed through wait() so there is no polling error. The same worker alone was still alive at the ten second deadline and needed SIGKILL.

Child that ignores SIGTERM

5.003 s

Same harness, worker in stubborn mode, three runs inside 1 ms of each other. That is the SIGKILL escalation firing on schedule.

Restart backoff

1, 2, 4, 8, 16 s

Timed from the supervisor's own log against a child that dies at once. Five restarts in forty seconds, and the pause caps at sixteen.

Test suite

13 of 13

ctest on a Release build, 33 s. CI runs build, test, shellcheck, docker and compose as separate jobs.

Supervisor footprint

3.9 MB RSS

Read from /proc while supervising, on two threads. The binary is 36 KB, or 31 KB stripped.

The kernel will not deliver a signal to PID 1 unless PID 1 installed a handler for it. So the container sits through the whole grace period and then dies by SIGKILL. sentinel takes the PID 1 slot and lets your program run as an ordinary child. It exits with the child's status code, or 128 plus the signal number if the child was killed. Exit 137 out of a container is 128 plus 9. That is the OOM killer.

The handlers go in without SA_RESTART. That matters: with the flag, the kernel restarts waitpid after a signal and the reap loop never gets a turn. Without it, waitpid returns EINTR and the loop keeps control. The handler calls kill() and nothing else. Very little is safe to call from signal context. Escalation lives under the same constraint. The SIGTERM handler arms alarm(5), and a SIGALRM handler does the killing.

There used to be a window between the fork and the line that installed the handlers. A SIGTERM landing in it killed sentinel and left the child running as an orphan. That is the exact failure sentinel exists to prevent. I marked it as a TODO and lived with it for a while. The fix is sigprocmask around the fork. It blocks SIGINT and SIGTERM before, and unblocks once the handlers are up. The child restores the original mask before exec. Otherwise everything sentinel launches starts out deaf to those signals.

The metrics server needs its own thread, because the main one sits blocked in waitpid and cannot also wait in accept. That thread blocks every signal for itself. SIGTERM has to keep landing on the main thread where the handler lives, or waitpid never gets its EINTR. The counters are atomic since both threads touch them.

One compose command brings up sentinel, a Prometheus scraping it every 5 s, and a Grafana with the dashboard already provisioned. The stack runs a child that crashes on purpose, so the graphs have something real on them. Two alert rules ship with it, one for a crashloop and one for a child that has been down two minutes. CI stands the whole thing up on every push and fails unless Prometheus reports the target up, the restart counter has moved, both rules loaded, and Grafana is serving the dashboard.

I had never seen the gate fail, so I did not know it worked. I broke a test to push the score from 92% to 91%. The job went red. The drop signal reached the next step and fired a custom action that Rickrolls whoever caused it. That path now runs end to end.

The suite needed tests that run without loading a real routing graph. I wrote three with Mockito, in their own package. They mock PointList, DistanceCalcEarth and EdgeIteratorState. A test can then pin exact coordinates and edge attributes and check the code that consumes them.

It started as three-tier coursework. I built it as three services, each in its own container. In an interview, I would want to be asked about the deploy. There is no README of manual steps and no start-up order to memorise. Anyone with Docker gets the same stack, in the same state, from one command.

I found this one because I switched machines. A version of the tagged allocator computed alignment from self.next, an offset into the buffer. The buffer's real address in memory never entered the calculation. It passed every test on x86-64 Linux. On macOS and ARM it hit panic: incorrect alignment. A stack-allocated [N]u8 is not guaranteed to start on an 8-byte boundary. Aligning an offset only aligns you relative to wherever the buffer begins, so an odd base address throws off every address derived from it. Aligning the absolute address fixed it.

The three build on each other. The first is a byte buffer and an index that moves forward. It has no per-block free at all, because a stack allocator releases everything or nothing. The second puts a header in front of each block, which makes free() possible as bookkeeping. A freed block gets marked and then sits there. The third walks those headers for a block big enough before it allocates at the end. It takes the first block that fits. That costs O(blocks) and wastes the tail of an oversized block. It also needs no free list and no coalescing pass.

When the model gets an example badly wrong, the MSE gradient is scaled by the output derivative. That derivative is near zero exactly when the error is largest. The network then learns most slowly on the examples it has most to learn from. Cross-entropy cancels the term. That is why it is the default for classification.