last change: Apr 26,2000

OS/2 Aurora KEE32 API

IBM is usually good for hiding easter eggs in every new (seldom enough) version of OS/2. While, with the OS/2 toolkit 4.5 (available through DEVCON), they have documented some treasures from older releases, the new WSeB contains, apparently for JFS, two new APIs, the KEE and the IFSM32.

The KEE is the Kernel Execution Environment, basically the 32 bit version of what is known as the 16 bit Device Helper API known to every driver programmer. The following describes this API.

Unfortunately, it is not yet complete enough to allow writing pure 32 bit physical device drivers, as several device helpers don't have a KEE equivalent yet (e.g. DevHlp_SetIRQ), and even worse, there is no 32 bit driver strategy/IDC support yet, so you need at least a 16 bit header and some small thunk that converts the 16 bit strategy call into a call of a 32 bit routine in a flat segment of the driver. This is possible and has already been done. The KEE will supposedly greatly reduce the effort of writing (almost) 32 bit drivers. Currently, IBM apparently uses KEE in OS2DASD.DMD and JFS.IFS which are both not exactly PDDs, so won't need a real strategy entry point at all. See PDD, ADD, and IFS docs how they actually interface with the rest of the system.

Header and Lib files

To use the API, you need a 32 bit C compiler. Given the calling conventions, which for basically all functions require CS, DS, ES, and SS to point to flat kernel segments, it seems to me that finally even EMX/gcc is suitable for writing the 32 bit part of the driver.

In addition, you need a header file and a forwarder library. Use the direct save to disk feature of your browser to get both files - at least the library link may severely confuse your browser.

The KEE API

KernAllocSpinLock

Synopsis

APIRET APIENTRY KernAllocSpinLock(KEESpinLock* sl);

Description

Initializes a spinlock to synchronize SMP processor access to a resource.

Parameters

sl: Pointer to a spinlock structure in 32 bit memory.

KernFreeSpinLock

Synopsis

APIRET APIENTRY KernFreeSpinLock(KEESpinLock* sl);

Description

Destroys a spinlock structure that is no longer in use.

Parameters

sl: Pointer to a spinlock structure.

Comments

Actually, this routine is supposed to clear the internal "in-use" flag. But in 14.039SMP, it does - nothing. I think this is a bug.

KernAcquireSpinLock

Synopsis

VOID APIENTRY KernAcquireSpinLock(KEESpinLock* sl);

Description

Try to acquire the spinlock. If another processor already owns the spinlock, the caller will busy wait until the other owner releases the lock, then acquires the lock immediately.

Parameters

sl: Pointer to an initialized spinlock structure-

KernReleaseSpinLock

Synopsis

VOID APIENTRY KernReleaseSpinLock(KEESpinLock* sl);

Description

Release the spinlock by the current owner after use.

Parameters

sl: Pointer to an acquired spinlock structure.

KernAllocMutexLock

Synopsis

APIRET APIENTRY KernAllocMutexLock(KEEMutexLock* ml);

Description

Initialize a mutexlock. This is basically the semaphore equivalent of a spinlock. Likewise, it is usable to protect critical regions. The difference to a spinlock is that waiting on a mutex will send the thread that processor is executing to sleep. This makes sense - in contrast to a spinlock - for longer critical regions.

Parameters

ml: Pointer to a mutexlock structure in 32 bit memory.

KernFreeMutexLock

Synopsis

APIRET APIENTRY KernFreeMutexLock(KEEMutexLock* ml);

Description

Destroy a mutexlock.

Parameters

ml: Pointer to a mutexlock.

Comments

Likewise to KernFreeSpinLock, this is supposed to do something, but doesn't. Again a sloppiness in code.

KernRequestSharedMutex

Synopsis

VOID APIENTRY KernRequestSharedMutex(KEEMutexLock* ml);

Description

Request a mutex in shared mode. Multiple threads may own the lock. If a thread already owns the lock, then the calling thread will block uninteruptibly. The block id is the address of this mutex. Use KernTryRequestSharedMutex to attempt to acquire the mutexlock without blocking.

Parameters

ml: Pointer to a mutexlock structure.

Comments

KernReleaseSharedMutex

Synopsis

VOID APIENTRY KernReleaseSharedMutex(KEEMutexLock* ml);

Description

Give up an acquired shared mutex.

Parameters

Comments

KernTryRequestSharedMutex

Synopsis

INT APIENTRY KernTryRequestSharedMutex(KEEMutexLock* ml);

Description

Try to acquire a shared mutexlock. If this fails, 0 is returned, if it succeeds, the mutexlock is acquired, and 1 is returned.

Parameters

ml: Pointer to a mutexlock structure.

KernRequestExclusiveMutex

Synopsis

VOID APIENTRY KernRequestExclusiveMutex(KEEMutexLock* ml);

Description

Request an exclusively owned mutex. Thread will block with the mutexlock is already owned.

Parameters

ml: Pointer to exclusively owned mutexlock structure.

KernReleaseExclusiveMutex

Synopsis

VOID APIENTRY KernReleaseExclusiveMutex(KEEMutexLock* ml);

Description

Give up an owned exclusively locked structure. If there are threads waiting, they will be unblocked.

Parameters

ml: Pointer to an exclusive owned mutexlock structure.

KernTryRequestExclusiveMutex

Synopsis

INT APIENTRY KernTryRequestExclusiveMutex(KEEMutexLock* ml);

Description

Attempt to acquire an exclusive owned mutex lock. Returns 0 if the request failed, else 1, and the mutexlock is acquired successfully.

Parameters

ml: Pointer to an exclusive mutexlock.

KernBlock

Synopsis

APIRET APIENTRY KernBlock(ULONG eventid, ULONG timeout, ULONG flags,
			  PVOID lockptr, PULONG retdata);

Description

This is an extended version of the DevHlp_ProcBlock function. Basically, you specify an arbitrary eventid and a timeout. The block may be coordinated with either an acquired spinlock or a mutexlock, as declared by the flags. The lock is released immediately before blocking, and reacquired after returning from the routine, unless the flag bit KEE_BLOCK_NOACQUIRE prevents re-acquiration. The blocked thread will hang in this routine, and can be resurrected by a signal (unless turned off with KEE_BLOCK_NOSIGNALS) or a KernWakeup with the same eventid.

Parameters

eventid

Some unique value, which is to be used by KernWakeup to unblock the thread again.

timeout

a timeout value in milliseconds. 0xffffffff is supposedly infinite wait.

flags

One or more of the following flags:

KEE_BLOCK_NOSIGNALS: Ignore signals, i.e. the block can be released only by end of timeout or an explicit wakeup.
KEE_BLOCK_SPINLOCK: The lockptr points to an acquired KEESpinLock structure.
KEE_BLOCK_EXMUTEXLOCK: The lockptr points to an acquired exclusive mutexlock structure.
KEE_BLOCK_SHMUTEXLOCK: The lockptr points to an acquired shared mutexlock structure.
KEE_BLOCK_NOACQUIRE: Don't reacquire the lock (if lockptr points to a lock at all) when the KernBlock routine returns.

lockptr

Used when one of the three *LOCK bits is set: points to the specified lock structure

retdata

points to a long variable that receives the value passed by the KernWakeup routine. May be NULL, in this case the value is ignored.

Comments

The KEE_NOSIGNALS bit looks dangerous to me - you can this way easily produce unkillable processes: Block in an ioctl with this bit set, and a bug preventing the wakeup, and you won't get rid of this process anymore. The three *LOCK bits are exclusive since there is only one pointer.
I guess the retdata value us irrelevant for a timeout or an interrupt by a signal (return values ERROR_TIMEOUT and ERROR_INTERRUPT).

KernWakeup

Synopsis

APIRET APIENTRY KernWakeup(ULONG eventid, ULONG flags, PULONG tcnt, ULONG retdata);

Description

This wakes up threads that are blocking in KernBlock. The eventid to be specified has to match the eventid used in KernBlock. It is possible to submit a 32 bit return value to the unblocked thread. The number of threads that are unblocked is returned. must match

Parameters

eventid

Event id of the blocked thread(s).

flags

One or more of the following flag bits:

KEE_WAKE_SINGLE: Only wakeup the first matching thread; by default, if multiple threads are waiting, all will be awaking.
KEE_WAKE_PRIOINCR: Run the woke threads with higher priority.
KEE_WAKE_RETDATA: return data value specified to the KernBlock retdata variable. By default, nothing is returned.

tcnt

pointer to an ULONG to return the actual number of threads that were woke. Ignored when NULL.

retdata

Data to be returned to the unblocked threads. Ignored when KEE_WAKE_RETDATA is not set.

Comments

I don't know whether the priority increase issued by KEE_WAKE_PRIOINCR is permanent, or only for the current timeslice. It shouldn't be permanent.

KernThunkStackTo16

Synopsis

VOID APIENTRY KernThunkStackTo16(VOID);

Description

This will convert a 16:32 SS:ESP value to a 16:16 SS:SP value.

Parameters

None.

Comments

I think this is required for correct calling of 16 bit code from the 32 bit driver, as 16 bit drivers assume a 16:16 stackpointer.

KernThunkStackTo32

Synopsis

VOID APIENTRY KernThunkStackTo32(VOID);

Description

This will convert a 16:16 SS:SP back to the 16:32 SS:ESP convention.

Parameters

None.

Comments

This is supposedly to be called after returning from 16 bit code.

KernSerialize16BitDD

Synopsis

VOID APIENTRY KernSerialize16BitDD(VOID);

Description

This will acquire a special kernel spinlock.

Parameters

None.

Comments

I think, according to the name, this has to be called before accessing a 16 bit driver. I wonder though, since you supposedly also have to adjust the stack pointer (KernThunkStackTo16), why both functions haven't been joined in a single call.

KernUnserialize16BitDD

Synopsis

VOID APIENTRY KernUnserialize16BitDD(VOID);

Description

This releases a special kernel spinlock.

Parameters

None.

Comments

See comments for KernSerialize16BitDD. This apparently needs to be called after returning from a 16 bit driver.

KernArmHook

Synopsis

VOID APIENTRY KernArmHook(ULONG hook,ULONG data);

Description

This is the 32 bit equivalent of DevHlp_ArmCtxHook.

Parameters

hook: Hook as returned from KernAllocateContextHook.
data: Data that is passed to the hook when called.

Comments

There is no such thing as a DevHlp_FreeCtxHook. I am unsure whether the hook handles are portable between 16 and 32 bit code. At least, this demonstrates that the KEE API has been tailored straight to its use in JFS and DASD, where one probably never needs to release a hook again.

KernAllocateContextHook

APIRET APIENTRY KernAllocateContextHook(PVOID pfHandler,ULONG dummy,
 					PULONG pHook);

Synopsis

Description

Allocate a new Context hook for use with KernArmHook.

Parameters

pfHandler: This is the address of the hook handler. This is likely a pointer to a 32 bit function.
dummy: I don't know why this is there, it is never accessed in the code.
pHook: The pointer to an ULONG where the hook handle is returned.

Comments

See comments for KernArmHook.

KernCopyIn

Synopsis

APIRET APIENTRY KernCopyIn(PVOID trgt, PVOID src, ULONG size);

Description

Copy a buffer of user space data into a kernel buffer.

Parameters

trgt: Target buffer address in kernel space.
src: Source buffer address in user space.
size: number of bytes to copy.

Comments

Finally, the routine I have waited for for eons...

KernCopyOut

Synopsis

APIRET APIENTRY KernCopyOut(PVOID trgt, PVOID src, ULONG size);

Description

Copy a buffer of kernel data into a user space buffer.

Parameters

trgt: Target buffer in user space.
src: Source buffer in kernel space.
size: number of bytes to copy.

KernVMAlloc

Synopsis

APIRET APIENTRY KernVMAlloc(ULONG size, ULONG flags, PVOID* linaddr,
			    PVOID* physaddr, PSHORT sel);

Description

This is the equivalent of the DevHlp_VMAlloc function.

Parameters

size

Number of bytes to allocate.

flags

One or more of the following flags. See DevHlp_VMAlloc docs for details.

KEE_VMA_16MB: allocate memory from below 16MB
KEE_VMA_FIXED: allocate memory that won't be swapped out
KEE_VMA_SWAP: allocate memory that is swappable
KEE_VMA_CONTIG: allocate contiguous physical memory pages
KEE_VMA_PHYS: allocate physical memory (pass address in physaddr argument)
KEE_VMA_PROCESS: allocate memory in the process address space.
KEE_VMA_SGSCONT: register memory under screen group control
KEE_VMA_GETSEL: return a GDT selector for the allocated memory. This option exists also in DevHlp_VMAlloc (undocumented).
KEE_VMA_RESERVE: Don't commit the allocated memory.
KEE_VMA_SHARED: allocate memory that is shareable
KEE_VMA_USEHIGHMEM: allocate memory from above 16MB.

linaddr

address of a variable to receive the linear address of the memory.

physaddr

address of a variable that contains the physical address of the memory.

sel

address of a variable to receive the allocated GDT selector, when KEE_VMA_GETSEL is set.

KernVMFree

Synopsis

APIRET APIENTRY KernVMFree(PVOID linaddr);

Description

This frees dynamic memory allocated with KernVMAlloc.

Parameters

linaddr: Address of allocated memory.

KernVMLock

Synopsis

APIRET APIENTRY KernVMLock(ULONG flags,PVOID linaddr,ULONG size,
			   KEEVMLOCK* lock, KEEVMPageList* pglist,
			   PULONG pgsize);

Description

This is the equivalent of the DevHlp_VMLock function. See the PDD docs for details.

Parameters

flags

One or more of the following bits:

KEE_VML_NOTBLOCK: Don't block the call, if the pages can't be locked.
KEE_VML_CONTIG: Pages locked must be contiguous.
KEE_VML_16M: Pages must be below 16MB.
KEE_VML_WRITEABLE: Pages will be writeable.
KEE_VML_LONGLOCK: The lock is for long time.
KEE_VML_VERIFYONLY: Verify the accessibility of the region. If KEE_VML_WRITEABLE is set, check for R/W access, else R/O.

linaddr

linear address of the memory region.

size

Size of the memory region.

lock

Pointer to a lock data structure that contains the lock information.

pglist

Pointer to a pagelist structure, provided by the user

pgsize

Pointer to an ULONG where the size of the pagelist structure is returned.

KernVMUnlock

Synopsis

APIRET APIENTRY KernVMUnlock(KEEVMLOCK* lock);

Description

Unlock a memory region.

Parameters

lock: Pointer to the lock structure used by KernVMLock.

KernLinToPageList

Synopsis

APIRET APIENTRY KernLinToPageList(PVOID linaddr,ULONG size,KEEVMPageList* list,PULONG pgsize);

Description

Convert a linear address region to a pagelist. This is the equivalent to DevHlp_LinToPageList.

Parameters

linaddr: Address of the linear region
size: Length of the linear region
list: Address of an array of pagelist elements long enough to receive the pagelist entries.
pgsize>: Pointer to an ULONG that contains the number of pagelist entries that were actually stored.

KernVMSetMem

Synopsis

APIRET APIENTRY KernVMSetMem(ULONG flags, PVOID linaddr, ULONG size);

Description

This is the equivalent of the DevHlp_VMSetMem function.

Parameters

flags

One or more of the following flags:

KEE_VMS_UNCOMMIT: Uncommit the region
KEE_VMS_RESIDENT: make the region resident
KEE_VMS_SWAPPABLE: make the region swappable

linaddr

Linear address of region

size

Size of region

KernSelToFlat

Synopsis

ULONG KernSelToFlat(ULONG addr16);

Description

Convert a 16:16 address to a flat address. Returns the linear flat address.

Parameters

addr16: 16:16 pointer

KernDynamicAPI

Synopsis

APIRET APIENTRY KernDynamicAPI(PVOID addr, ULONG cnt, ULONG dummy, PUSHORT sel);

Description

This is the equivalent of the DevHlp_DynamicAPI function.

Parameters

addr: Address of the routine to be called through this gate.
cnt: Number of arguments to be copied from user stack to kernel stack
dummy: I don't know what this parameter is for. It is not used in the code.
sel: Pointer to a variable where the generated GDT gate selector is returned

KernRASSysTrace

Synopsis

APIRET APIENTRY KernRASSysTrace(ULONG major,ULONG minor,PVOID buf, ULONG size);

Description

This is the equivalence of the DevHlp_RASTrace function.

Parameters

major: Major number of RAS event
minor: Minor number of RAS event
buf: buffer to be added to the RAS log
size: size of the buffer

Comments

This is not explained in the PDD reference manual, but in the debug handbook IIRC.

KernPerfSysTrace

Synopsis

APIRET APIENTRY KernPerfSysTrace(ULONG major,ULONG minor,PVOID buf, ULONG size);

Description

This is the equivalent of the DevHlp_Perftrace function used by PerfView.

Parameters

major: Major number of Perf event
minor: Minor number of Perf event
buf: buffer to be added to the Perf log
size: size of the buffer

KernLockFile

Synopsis

APIRET APIENTRY KernLockFile(HFILE hfd,KEEhfile* khfd);

Description

This function converts a user space file handle to a system file table pointer.

Parameters

hfd: User space file handle
khfd: Pointer to an variable to return the SFT entry pointer.

Comments

This is some sort of DosOpen to access user files. This seems to be a kludge required for some feature of JFS.

KernUnLockFile

Synopsis

APIRET APIENTRY KernUnLockFile(KEEhfile khfd);

Description

This will return the specified SFT pointer to the system.

Parameters

khfd: SFT pointer.

Comments

This is some sort of DosClose after a file has been opened with KernLockFile.

KernGetFileSize

Synopsis

APIRET APIENTRY KernGetFileSize(KEEhfile khfd,KEEfilesize* sz);

Description

This returns the size of the file designated by KernLockFile.

Parameters

khfd: SFT pointer
Address of a double long variable to hold the size of the file.

Comments

Like the other SFT related functions, this seems to be a hack for JFS.

KernTestFileCache

Synopsis

APIRET APIENTRY KernTestFileCache(KEEhfile khfd);

Description

Check whether the associated IFS for the specified file allows access to the file cache. Return 1 if yes, 0 if not. This seems to be again a special feature for JFS.

Parameters

khfd: SFT pointer

KernReadFileAt

APIRET APIENTRY KernReadFileAt(KEEhfile khfd,PVOID buf, QWORD off,
			       ULONG nbytes, PULONG nread);

Synopsis

Description

This reads the content of a file into a buffer.

Parameters

khfd: SFT pointer
buf: buffer to read data into
off: Offset in file to start reading
nbytes: Number of bytes to read
nread: Pointer to an ULONG to receive the number of bytes actually read.

Comments

KernReadFileAtCache

APIRET APIENTRY KernReadFileAtCache(KEEhfile khfd,KEECachePages** ptr,
				    QWORD off, ULONG nbytes, PULONG nread);

Synopsis

Description

This seems to do the same as KernReadAtFile, but reads from the cache and returns the corresponding pages of the cache data structure.

Parameters

khfd: SFT pointer
ptr: Pointer to a variable the receives the cache data structure for the specified data blocks.
off: Offset in file to start reading
nbytes: Number of bytes to read
nread: Pointer to an ULONG to receive the number of bytes actually read.

Comments

Yet another JFS hack. Meanwhile I understand why IBM is reluctant to provide JFS support for Warp 4.

KernReturnFileCache

Synopsis

APIRET APIENTRY KernReturnFileCache(KEEhfile khfd,KEECachePages* ptr);

Description

This appears to be the "free" function to return the ptr structure from KernReadFileAtCache back to the system.

Parameters

khfd: SFT pointer
ptr: Pointer to a previously received cache data structure.

KernCreateUconvObject

Synopsis

APIRET APIENTRY KernCreateUconvObject(USHORT codepage, KEEUnicodeStruct* ucs);

Description

This initializes a data structure related to Unicode conversion.

Parameters

codepage: Codepage for the data structure.
ucs: Pointer to the unicode conversion structure created.

Comments

Finally, this seems to be Unicode support in the kernel...

KernStrFromUcs

Synopsis

APIRET APIENTRY KernStrFromUcs(KEEUnicodeStruct* ucs, PCHAR trgt,
			       PCHAR usrc, ULONG trgtsize, ULONG srcsize);

Description

This converts a string from Unicode to ASCII.

Parameters

ucs: Pointer to the Unicode conversion object created with KernCreateUconvObject.
trgt: Pointer to the ASCII buffer
usrc: Pointer to the Unicode buffer
trgtsize: length of the target buffer
srcsize: length of the source buffer

KernStrToUcs

Synopsis

APIRET APIENTRY KernStrToUcs(KEEUnicodeStruct* ucs, PCHAR utrgt, PCHAR src,
			     ULONG trgtsize,ULONG srcsize);

Description

This converts a string from ASCII to Unicode.

Parameters

ucs: Pointer to the Unicode conversion object created with KernCreateUconvObject.
utrgt: Pointer to the Unicode buffer
src: Pointer to the ASCII buffer
trgtsize: length of the target buffer
srcsize: length of the source buffer