Building a kernel par 1

I've decided to detail my adventures in building my own kernel here. First I should detail why, and the answer is two fold: 1 - I'm sick of the "source only no binary" approach in Linux, and 2 - it's a fun way of learning how this stuff works. Looking at Linux, it strikes me it's success is not so much in the amount of innovation, but the amount of functionality. The Linux kernel can do just about anything you need it to, scale how you need it to, even slice fries and juice fruit. Now, there is a lot of innovation going on now, but it's more coming out of commercial code dumps and work from the likes of Sun, SGI, and IBM.

My own kernel finally passed the huge milestone of having the full boot process and transition completed. That means at this point, the kernel loads itself, relocates to a good place in virtual address space, sets up the segment descriptor tables, and enables paging. Further, there is a full boot loader I'm using now that is tried and tested and works. Getting this to work involved a bit of pain and work with AS, and GCC. Now that I have that I'm working on the memory management subsystem, and ELF relocating/loading. The basic design is something that I might call a true "hybrid-kernel". The same service/driver can be implemented in user space or kernel space depending on need. To keep things simple, there will be a select few types of kernel space driver types. The intention is that if you REALLY need it, you can write a kernel mode portions of a user space service driver. The vast majority of user space services will use an L4 type approach. Only things that deal directly with hardware will be allowed in pure kernel space.

Unlike Unix/Plan 9, I'm taking the approach that instead of everything is a file, everything is a service, including files. Services are opened via the kernel. A user space program asks the kernel for a service using a URI, and the kernel then brokers the request to the proper device driver/service manager/etc... Every request has associated with it a context that is given to both the broker and the service. Once the service is opened, the user can contact the kernel to look up capabilities (or RPCs), and then send requests to the service (like ioctls). At this point, even files are services, and modifying the context can change how they are opened and accessed.

So, kernel drivers are implemented as services, and the kernel predefines operations that certain services are required to have.

I hate AT&T

I *HAD* fully intended on putting in a deposit and application fee for a really nice apartment on Saturday. I liked the floor plan, I liked the people, I liked the pool.... That decision is being thrown into a large amount of doubt by the utility company - AT&T. I have a Cingular/AT&T phone, it's been good so far. Of course, it was good before AT&T bought them.

Here's what I have now from Time Warner Cable -
- High Definition Digital Cable with Showtime
- High definition DVR w/ dual tuner, high def digital output
- High bandwidth internet access (measured at 6.4mbps)
- Digital phone service w/ Unlimited long distance and call waiting
- 1 Bill
TOTAL - about $130 before taxes

Here's what AT&T is offering if I move -
- Digital Satellite, no premium channels
- Non-HD DVR (no HD available), single tuner, requires phoneline, AV output only
- High bandwidth internet access (3.0 mbps rated)
- Phone service w/ 120 minutes long distance included
- Separate bills for each service
TOTAL - $54.95 + $5.99 + $59.99 = $120.93

So, the prices are roughly the same, and I get to loose high definition TV, 1/2 my internet access speeds for remoting to work, and the ability to watch lost and record one of my other shows at the same time. Where do I sign up?

AT&T - YOU SUCK - and you lost a very apartment complex a lease.

Anyway, the apartment is nice enough to make me ponder whether or not I want to nuke them on this issue, but I'm *really* leaning toward it. It's sad to, I *really* liked what the seemed to offer.

Hash Tables....

It's a fairly constant amusement to me the lack of forethought put into certain details when other details are obsessed over. A great example can be found in the realm of the hash table. Now, I don't want to rant about basic programming practice, but it's my blog so I can do what I want.

Here is the scenario: our intrepid programmer friend has spent much time developing a 32bit hash table key. Now, the hash key is in reality 4 characters, the last of which is least important and often duplicated. Satisfied that the hash key works well, our friend creates the hash class, and decides that 256 entries is a nice round number.

Thus happens the line of code: arrayindex = hashkey % 256;

In this case, the modulo has left nothing more than the last byte of the key as significant. The last byte of our hash key is the *least* important and most duplicated, so our friend has managed nothing more than to remove a few entries from a linear searched list.

That, however, pales in comparison to this blunder done by using a word-aligned pointer as a hash key. arrayindex = hashkey % 100;

Now, this code looks harmless enough, until you consider the small fact that 100 equals 25 * 4. The 4 factor cancels out nicely because the integer value of the pointer must be a multiple of 4, and at this point, the programmer has essentially created a 25 bucket hash table using 100 buckets worth of space.

The funny thing is - I found BOTH of these in real code. The point? If you are going to spend hours working on one portion of a data structure, keep in mind how the rest works to know if you are wasting that time or not...

Object Serialization....

AKA - how to do something very useful and possibly shoot yourself very badly in the foot at the same time. Now imagine this scenario:

class Foo
{
...
int i;
float fp;
};

That is serialized as such:
fwrite( &foo, sizeof(Foo), 1, fp);

Then later on in life, reclaimed via:
fread( &foo, sizeof(Foo), 1, fp);

Yes, this is very fast. Yes, it allows for interesting things like mapping variables to memory and so forth. But it really makes life difficult if a couple weeks down the road, after you have serialized objects floating around, you have the need to do something like changing "float fp" to "double fp". Let's not even consider the effects of having a virtual function inside our class...

If there is a need to read a large binary glob from a file, then the glob itself should be saved and read by the class - which would make my life MUCH easier here. Just imagine if I could do something nice and sane like:

Foo bar(stream_source);

Or

bar->ReadSerialize(stream_source);

Anyway, if you're going to work with a large amount of objects serialized to files, then please please please please PLEASE, consider that someone might need to CHANGE those objects later.

Grrrrr......

And there was much rejoicing!

I'll save the photos for when they are "preapproved", but I thought I'd take a few seconds to mention the wedding I went to yesterday. In my time, I have been to several weddings, but the one I went to yesterday will hold a very dear place in my heart. This was the first "worldly" wedding I went to. It was also the first wedding I've been to where I could just be myself.

After canceled flights and a 500 mile drive, Steve stood in front of Stacy and stated in clear, slightly tearful words, his devotion to a woman that "made his dreams come true". People from around the world stood in witness of the event, traveling anywhere from hundreds to thousands of miles. And as I stood watching the candlelit ceremony, the power and strength of the love reflected in that room overcame me.

This was nothing of the sterile void of the Witness weddings I saw before, where a pre-outlined talk is given talking about the basics of marriage. There was no need to glorify a man-made organization or it's "arrangement". No thoughts given about who could not attend because they were "not in good standing". No whispering behind the back of the bride because she had children. The children here were not an embarrassment, but an active and valuable part of the ceremony. Instead of removing the significance of the individual, we celebrated two people and their love for each-other.

And we rejoiced in song, dance, story, drink, hugs, and togetherness. I did not sit and reflect upon the meaning of marriage, but instead I found joy in a marriage that had already happened. The wedding was not an opportunity to preach about God, but instead a milestone to remember the point at which Steve and Stacy were united.

Yesterday, for the first time in my life, I witnessed a marriage.

Some Pictures: