But also big props to Andreas for speaking so candidly about the back story of Serenity. I feel like there are certain things we don't talk about in tech and that can change.
I love the idea that programming, esp side projects, can be a therapeutic thing.
Guest here. Thanks for having me on your show, Adam!
I'm definitely a fan of talking more about things that we've been hiding from each other.
On that topic, one of the most common questions I get personally is "which drugs did you use in the past?" and I recently made a video[1] answering that question. It was a difficult thing to get through, but the overwhelmingly supportive response has been amazing.
For anyone who's struggling and is afraid to open up about it, know that there's a community of millions around the world who will welcome you with open arms.
I wanted to watch this video for a minute or so (very shallow of me) but ended up being stuck in it for the whole 29 minutes. Very interesting and hard story to tell but also extremely inspiring to see what hell have you been through and was able to mature to the point to where you became an inspiration for many others. Well done.
Now more technical questions ... for my own sake of understanding and learning.
What would you say that were most important aspects of re-implementing (subset of?) C++ runtime that must be appropriate for in-kernel usage?
Where would I look in the source code to learn about SerenityOS paging algorithms, and virtual memory management in general?
> What would you say that were most important aspects of re-implementing (subset of?) C++ runtime that must be appropriate for in-kernel usage?
Something that I messed up originally and we're now correcting incrementally is the handling of allocation failures. There are many places in a kernel where errors don't have a natural propagation path (IRQ handling for example), so operations that may fail (like allocation) must be avoided.
In general, heavy use of C++ features like templates and RAII has been very helpful for making kernel development productive.
> Where would I look in the source code to learn about SerenityOS paging algorithms, and virtual memory management in general?
> Something that I messed up originally and we're now correcting incrementally is the handling of allocation failures.
What in particular? Can you use exceptions reliably in the kernel land? I presume not unless you pre-allocate some hard-to-exhaust space for them.
Have you maybe thought of using LSBs to encode an allocation failure within the pointer itself? It's an entertaining idea I had for some time now. In 64-bit byte-addressable architectures one can take advantage of suitable alignments and have 3 least significant bits used to encode 8 different states. In a similar vein one could also take advantage of MSBs given the virtual address memory space limit of nowadays CPUs.
We don't use C++ exceptions anywhere in SerenityOS. Instead we use an ErrorOr<T>[1] return type for functions that may fail. These are then automatically propagated by use of a TRY() macro[2].
I haven't thought about encoding allocation failure as a low pointer value. I suppose you could, but it'd only work for pointers with sufficient alignment. Character/byte pointers often start at unaligned offsets when not coming directly from an allocator. Either way, I'm not sure compressing this information would solve any real problem we're having.
Then I might have misunderstood you with the issue of propagating errors. In IRQ, as you mentioned, it wouldn't be very practical to blow out the system just because your error propagating mechanism depends on yet another thing that can miserably fail. That's why I thought of tagged pointers, given the sufficiently aligned pointer operation of marking the action as failed cannot be unsuccessful. Nice thing about it is that you also get atomicity basically for free.
> Character/byte pointers often start at unaligned offsets
Yes, but the address of that pointer is still 8 bytes long and idea of using 3 LSB of that address to encode the state is still applicable, isn't it?
> > Character/byte pointers often start at unaligned offsets
> How's that possible? Isn't that an UB?
"Unaligned" was not the best word to use. What I really meant was byte-aligned:
char* s = strdup("Hello world!");
char* p = &s[1];
Here, `p` is byte-aligned and will have a 1 in the least significant position. So a scheme that uses low bits to encode allocation failure would have to work hard to avoid confusing byte-aligned pointers with allocation-failure-signalling pointers.
> Unaligned" was not the best word to use. What I really meant was byte-aligned
Yeah, I figured just few moments after I replied what you actually meant. That's why I edited the post above.
Well, to make it more robust one would have to wrap it in it's own pointer type but I think it's doable. I don't know whether or not it is a wise choice to make it a dependency in the OS though :)
But also big props to Andreas for speaking so candidly about the back story of Serenity. I feel like there are certain things we don't talk about in tech and that can change.
I love the idea that programming, esp side projects, can be a therapeutic thing.