Memory Manager preliminar Design

Me and Ponniah has started the coding for memory manager preliminary code few weeks ago. The sequence of events after memory manager starts are:
1. Counts the number of pages in the Main Memory:
This is done by writing a value to a start of a page and then reading it again. If the written and read value are the same then the page exists else page doesnot exsits. A counter is incremented if page exists. The value of counter is the number of pages.

2. Create a bit map for the Occupied Pages:
Initally grub loads some modules in the memory those pages are to be found from the Kernel's command line arguments and those pages are marked as used. A page is allocated for creating this bit map.

3. Allocating a Page and Deallocating a Page are to be coded.

Preparing a design document

Guys, Will one of you take the responsibility of preparing a design document. Please use TexInfo format. That ways we can convert it to a number of other formats. The draft posted by sap is a good start. Please put it into a document and lets get this thing rolling..

Discussion on base goatee design - Minutes

Place : Joe's house
People attended : Joe, SAP, Bala, Mani, Keyan, Bara

The different tasks we have decided to write now are

(0) Task 0 (root task, still unnamed!!)
(1) Memory manager
(2) Pager and Physical memory management
(3) Process Manager

The detailed functionalities of different tasks are below.

Task 0 :
- Has to maintain a table whose entries are the different modules to be loaded. The base addresses of these modules can be obtained from the resource table that is provided by the kernel.
- Has to load (start) all the modules.
- Has to put an entry in the task table after a process has been scheduled to run by the Process Manager.

Memory Manager :
- Has to process the requests for memory from the Process Manager while creating a new process.
- Has to assign (not allocate) virtual memory pages for the requested process.
- Has to maintain a structure that maps the different pages allocated to different processes.
- Has to define policies on which processes can access which pages and how many number of pages can be allocated to the different processes.
- Has to maintain the memory map (forgot whats this, someone please explain).
- Has to free the entries in the structure defined above, when a process is done (completed).

Pager :
- Has to handle the page faults.
- Has to maintain information of which pages should not be swapped out (for examples, the pages of this Pager, pages of the swap device, etc).
- Has to maintain the information abt which sections in the secondary storage belongs to which process. This is used when a new process is started to run (the next point says abt this).
- Has to allocate pages when the page fault occurs (when the process is loaded to run for the first time).
- Has deallocate pages when a process is done.

Process Manager :
- Maintains the process table.
- Scheduling of processes is done here.
- Has to make an ITC (to make an entry in the task table), when a process is scheduled to run.
- Has to receive fork requests and process them.
- Setting the priorities for different processes.
- A process table entry caotains the following fields
pid
uid, gid
euid, eguid
status
registers
stack
priority
is this process has an entry in the task table

PS : I have written all those that have come to to my mind. Its now like a scrap. You need to make changes so that the final copy is proper

Need for code examples in OS course

Recently TCE had some syllabus revisions done for the computer science dept. I heard that the people in charge unanimously rejected the idea of including portions of MINIX into the curriculum.

A course in OS usually contains topics such as process, IPC, deadlocks, etc.. However these theoretical topics are inadequate. The student does not usually get the full picture or atleast a gross idea of how the thing works. Often students tend to think that an OS is something that belongs to a completely different world. They dont think of applying those concepts in their day to day programming. This is a case seen widely in CSE courses across TamilNadu and probably all over India.

I think a course on OS should contain some code example. An ideal choice would be the Linux kernel. However, the Linux kernel is a production system and hence would not be that helpful to a new learner. MINIX is a good choice. The book "OS implementation and design" by Prof. A.S.Tanenbaum explains MINIX in a way that would be easily understood by the students.

Such a approach would bring the students to a higher level. They'll get trained to understand code written by other people. They'll be encouraged to read other's code and ideas before implementing their own.

KernelAnalysis-HOWTO: Linux Multitasking

Need this to fine tune the timer.

KernelAnalysis-HOWTO: Linux Multitasking

Interesting OS projects

Kernel architecture is one thing.. OS architecture is another..

The Plan 9 OS: http://www.cs.bell-labs.com/plan9dist/
The Inferno OS: http://c2.com/cgi/wiki?InfernoOs

Google Search: elf 32 specification

Some references on ELF specification. This is needed for the relocation of the first task.

http://www.skyfree.org/linux/references/ELF_Format.pdf
http://developers.sun.com/solaris/articles/elf.html
http://www.cs.princeton.edu/courses/archive/fall04/cos217/reading/elf.pdf
http://refspecs.freestandards.org/LSB_2.0.0/LSB-Core-S390X/LSB-Core-S390X/s390x-elf.html
http://refspecs.freestandards.org/LSB_2.0.0/LSB-Core/LSB-Core/generic-elf.html

Patent Prescription

An interesting article on the increasing irrelevance of the patent system

http://www.spectrum.ieee.org/careers/careerstemplate.jsp?ArticleId=i120204

Race while modifying the task structures??

Well, this is a concern that has hit my head lately. When exposing the kernel structures to user tasks, I did not think about the race conditions that might arise. For example, when the task_table is being updated., (may be in the process of creation of a new driver task to handle a interrupt), if a spurious interrupt turns up, then it is possible that the task_table might be in a inconsistent state. Now, this calls for measures of avoiding race. Well, should cook up some hack to come about this.

I dont think that this problem would arise if the number of tasks actually manipulating these tables stays to a handful. However, this issue needs to be investigated.

Striking similarity between L4 and our Mexokernel

I just happened to notice a similarity between the L4 microkernel and the design that I've been working on., our dear Mexokernel. The L4 has a region of memory called the KIP (Kernel interface page). This page has some details which may be used by user programs. The L4 supports threads and address spaces. Each address space should have atleast one thread. Also, the creation of address space is done by a user land program. It creates a address space and configures it to run the first thread in it. Each address space can have multiple threads (there is a hard limit on this). Now, this multiple threads can modify their information table (user thread control block). These are in the user region., but the kernel uses these blocks to do the multitasking. Hence, a per address space, thread control is done completely in the code running in the address space.

There is a transcript of an interesting conversation about the L4 and hurd-l4 on #hurd@irc.freenode.net in here.

Now, in our case we dont have address spaces or threads. We have execution-states. We also export a set of pages containing data structures. But in our case, these data structures can be modified. Now, this is the beauty. I hope to avoid a number of task switches this way. Morover, this way we can actually give a lot of liberty to the userland code. However we should develop on this model to see more of this,