changeset 103:d00b5e91cba8

Changes that have accumulated over the past couple years.
author Rob Landley <rob@landley.net>
date Fri, 01 Jan 2010 04:51:43 -0600
parents 2df5e5719a48
children 7bcba6e2acc3
files master.idx
diffstat 1 files changed, 335 insertions(+), 25 deletions(-) [+]
line wrap: on
line diff
--- a/master.idx	Sat Aug 23 19:07:22 2008 -0500
+++ b/master.idx	Fri Jan 01 04:51:43 2010 -0600
@@ -32,17 +32,19 @@
 <ul>
 <li><a href=Documentation>Text files in the kernel's Documentation directory.</a></li>
 <li><a href=htmldocs>Output of kernel's "make htmldocs".</a></li>
-<li><a href=menuconfig>Menuconfig help</a></li>
+<li><a href=makehelp.txt>Output of kernel's "make help".</a></li>
+<li><a href=menuconfig>Menuconfig/kconfig help for each configuration option.</a></li>
 <li><a href=readme>Linux kernel README files</a></li>
 <li><a href=rfc-linux.html>IETF RFCs referred to by kernel source files.</a></li>
 </ul>
 </span>
 <span id="Out on the web">
 <ul>
-<li><a href=xmlman>html version of man-pages package</a></li>
+<li><a href=http://kernel.org/doc/man-pages>Linux man-pages website, includes HTML versions of man pages</a></li>
 <li><a href=http://lwn.net/Kernel/Index/>Linux Weekly News kernel articles</a></li>
 <li>Linux Device Drivers book (<a href=http://lwn.net/Kernel/LDD3/>third edition</a>) (<a href=http://www.xml.com/ldd/chapter/book/>second edition</a>)</li>
-<li><a href=ols>Ottawa Linux Symposium papers</li>
+<li><a href=ols>Ottawa Linux Symposium papers</a></li>
+<li><a href=als1999>Atlanta Linux Showcase CD (1999)</a></li>
 <li><a href=http://www.linuxjournal.com/xstatic/magazine/archives>Linux Journal archives</a></li>
 <li><a href=http://www.ibm.com/developerworks/views/linux/library.jsp>IBM Developerworks Linux Library</a> (also <a href=http://www.ibm.com/developerworks/linux/library/l-linux-kernel/>here</a>)
 </li>
@@ -55,8 +57,8 @@
 
 <span id="Standards">
 <ul>
-<li><a href=http://www.opengroup.org/onlinepubs/009695399/>Single Unix Specification v3</a> (Also known as Open Group Base Specifications issue 6, and closely overlapping with Posix.  See especially <a href=http://www.opengroup.org/onlinepubs/009695399/idx/xsh.html>system interfaces</a>)</li>
-<li>C99 standard (defining the C programming language): <a href=http://www.open-std.org/jtc1/sc22/wg14/www/standards>ISO/IEC C9899 PDF</a> or <a href=http://c0x.coding-guidelines.com/>searchable website</a>.</li>
+<li><a href=http://www.opengroup.org/onlinepubs/9699919799/>Single Unix Specification v4</a> (Also known as Open Group Base Specifications issue 7, and POSIX 2008.  See especially <a href=http://www.opengroup.org/onlinepubs/9699919799/idx/xsh.html>system interfaces</a>)</li>
+<li>C99 standard (defining the C programming language): <a href=http://www.open-std.org/jtc1/sc22/wg14/www/standards>ISO/IEC C9899 PDF</a>, <a href=http://busybox.net/~landley/c99-draft.html>html</a>, or <a href=http://c0x.coding-guidelines.com/>searchable website</a>.</li>
 <li><a href=http://www.linux-foundation.org/spec/refspecs/>Linux Foundation's specs page</a> (ELF, Dwarf, ABI...)</li>
 </ul>
 </span id="Standards">
@@ -579,6 +581,30 @@
           <span id="proc">
           </span>
           <span id="sys">
+<p>Although the sysfs filesystem probably wasn't intentionally named after the
+greek myth about pushing a rock to the top of a hill only to see it forever
+roll back down again, this is a remarkably accurate analogy for the
+task of documenting sysfs.</p>
+
+<p>The maintainers of sysfs do not believe in a stable API, and change
+userspace-visible elements from release to release.  The rationale is that
+sysfs exports information from inside the kernel to outside the kernel
+(what API doesn't?) and the kernel internals change, thus sysfs changes to
+reflect it.  This doesn't explain why sysfs regularly changes things that aren't
+dictated by kernel internals, such as moving partition directories under block
+device directories after initially exporting them at the same level, moving
+/sys/block into /sys/devices, removing the "devices" symlink, and so on.<p>
+
+<p>In reality, sysfs is treated as a private API exported for the use of the
+"udev" program, which is maintained by the same developers as sysfs.  Any
+attempt to use sysfs directly from other programs is condemned by sysfs'
+authors as an abuse of sysfs, and attemps to document it are actively resisted
+and ridiculed.  (Yes, you must often update udev when you update the kernel.)</p>
+
+<p>The following documentation reflects the current state of sysfs.  This is
+likely to change in future, as its maintainers break compatability with
+existing userspace programs they didn't personally write.</p>
+
           </span>
           <span id="internal (pipefs)">
           </span>
@@ -651,6 +677,7 @@
       <span id="null">
       </span>
       <span id="random/urandom">
+<p><a href=http://eprint.iacr.org/2006/086.pdf>Analysis of the Linux Random Number Generator</a> - Zvi Gutterman, Benny Pinkas, Tzachy Reinman (iacr 2006)</p>
       </span>
       <span id="zero">
       </span>
@@ -742,40 +769,325 @@
 
 <span id="Hardware">
   <span id="Architectures">
+<p>Linux supports many more hardware platforms than its original PC.
+The first modern port of Linux was to the DEC Alpha processor
+[TODO: open sources]</p>
+
+<p>The most widely used modern ports are i386, x86-64, ARM, mips, and powerpc,
+all of which are supported by the emulator <a href=http://qemu.org>QEMU</a>.
+Bootable kernel and filesystem images for those platforms (bootable under
+QEMU) are available <a href=http://landley.net/code/firmware>here</a>.</p>
+
+<p>Alpha, sparc, parisc, itanium are primarily of historical interest.
+Each of those platforms used to have a bigger developer community than it
+does now, but has peaked and gone into a pronounced decline.</p>
+
+<p>Most of the other platforms have special-purpose niches.  For example,
+super-hitachi is widely used in the Japanese auto industry.</p>
+
 <pre>
-alpha
-arm
 avr32
 blackfin
 cris
 frv
 h8300
 i386
-ia64
 m32r
 m68k
-m68knommu
-  <a href=ols/2002/ols2002-pages-130-145.pdf>Running Linux on a DSP: Exploiting the Computational Resources of a programmable DSP Micro-Processor with uClinux</a> (OLS 2002)
-mips
-parisc
-  <a href=ols/2002/ols2002-pages-183-190.pdf>Porting Drivers to HP ZX1</a>
-powerpc
-  <a href=ols/2001/iseries.pdf>The Linux Kernel on iSeries</a> (OLS 2001)
-  <a href=ols/2001/ppc64.pdf>PowerPC 64-bit Kernel Internals</a> (OLS 2001)
-  <a href=http://perso.magic.fr/l_indien/qemu-ppc/PowerPC_ref/PowerPC_ref.html>PowerPC implementation reference for QEMU</a>
-ppc
 s390
 sh
 sh64
 sparc
 sparc64
-um
-  <a href=ols/2001/uml.pdf>User-Mode Linux</a> (OLS 2001)
-  <a href=ols/2002/ols2002-pages-107-116.pdf>Making Linux Safe for Virtual Machines</a> (OLS 2002)
 v850
-x86_64
+xtensa
+
+</pre>
+<span id="alpha">
+<p>The now-obsolete
+<a href=http://en.wikipedia.org/wiki/DEC_Alpha>DEC Alpha</a> was one of the
+first 64-bit processors, one of the fastest and cleanest processor
+designs of its time, and still has fans to this day.  Despite excellent
+performance and widespread use in supercomputers, manufacturing of
+Alpha was
+<a href=http://news.zdnet.co.uk/hardware/0,1000000091,2127122,00.htm>repeatedly
+disrupted</a> and a series of acquisitions by PC vendors uninterested in any
+non-PC architecture.  Despite pressure from users and even
+<a href=http://www.ftc.gov/opa/1998/04/digitala.shtm>government intervention</a>
+to preserve the Alpha processor, new development of the hardware ceased
+towards the end of the 1990's.</p>
+
+<p>The legacy of Alpha lives on in the x86-64 architecture.  When
+<a href=http://www.chron.com/content/chronicle/page1/98/01/27/com.html>Compaq
+bought DEC</a> it acquired the rights to the Alpha processor, but not the
+chip design team.  Many ex-Alpha chip designers wound up at AMD, where they
+<a href=http://arstechnica.com/cpu/3q99/athlon/athlon-1.html>designed
+the Athlon</a> (x86) and Opteron (x86-64) processors.  Intel also
+<a href=http://www.news.com/Compaq-Life-after-Alpha/2100-1001_3-268986.html?tag=st.ref.goo>licensed</a> and <a href=http://www.news.com/Intel-hyperthreading-shows-Digital-roots/2100-1001_3-961495.html?tag=st.ref.goo>incorporated</a>
+Alpha technology into all its processor lines.  Internally, modern PC
+processors owe more to the Alpha than to the original 8086 processor.</p>
+
+<p>Alpha is of great historical importance to Linux as the
+<a href=http://www.oreilly.com/catalog/opensources/book/linus.html>first non-PC
+port incorporated into Linus's tree</a>, as well as the first 64-bit port.
+<a href=http://qemu.org>QEMU</a> recently grew preliminary support for
+emulating Alpha processors.</a>
+
+<ul>
+<li><a href=http://www.alphalinux.org/>AlphaLinux.org</a></li>
+</ul>
+
+</span>
+<span id="arm">
+<p>The ARM processor is the most popular embedded processor, powering 80-90% of
+the cell phone market and most battery powered handheld devices.  The iPod,
+iPhone, Nokia N800, and Nintendo DS are all ARM-based.</p>
+
+<p>While the x86 family has the world's leading price/performance ratio,
+the ARM processor family has the best ratio power consumption to performance.
+By delivering the best bang for the watt, ARM has become overwhelmingly popular
+in embedded devices.</p>
+
+<p>ARM originally for "Acorn RISC Machine", a processor designed by a British
+company in the early 80's to replace the 8-bit 6502 in Acorn's successful BBC
+Micro.  Unlike most RISC design efforts which focused on using RISC techniques
+to increase performance, Acorn focused on creating a small, simple processor
+design, initially with under 25,000 transistors (these days with about 43,000
+transistors worth of core logic, before adding a cache and memory controller).
+In 1990, the processor design team moved to a new company,
+<a href=http://www.arm.com>ARM Ltd</a>, which doesn't manufacture chips but
+instead licenses its designs to other companies interested in fabricating
+chips.  This also allows ARM designs to be easily customized, and
+embedded in things like network cards or system-on-chip designs.</p>
+
+<p>Arm processor generations are divided by "architecture", which among other
+things indicates the instruction set the processor can run:</p>
+<ul>
+<li>ARMv3 - The oldest 32-bit ARM architecture, now considered obsolete.</li>
+<li>ARMv4 - The oldest architecture still in widespread use.</li>
+<li>ARMv5 - The oldest architecture still in production.  The baseline
+modern" architecture.</li>
+<li>ARMv6,v7 - An architecture ARM inc has used NDA terms to prevent QEMU
+developers from releasing support for (apparently because it wants to sell
+proprietary emulators, and considers a GPLed emulator a threat).  These
+processors run ARMv4/v5 code.</li>
+</ul>
+
+<p>The newest archtecture that can be emulated by
+<a href=http://qemu.org>QEMU</a> is ARMv5TEJ (I.E. ARMv5
+with the Thumb, Enhanced DSP, and Java extensions).  Unfortunately, ARM Ltd.
+has leveraged its NDAs with prominent open source developers to
+<a href=http://lists.gnu.org/archive/html/qemu-devel/2007-02/msg00426.html>explicitly
+forbid</a> them from contributing ARMv6 support to QEMU, apparently because
+it's trying to sell a competing proprietary emulation product.</p>
+
+<p>Newer ARM processors run older instruction sets, and are thus backwards
+compatible.  The advantage of newer instruction sets is that they execute
+faster (and are thus more energy efficient), and some produce smaller binary
+sizes (the "Thumb" extensions are designed specifically for small code size,
+but may exchange performance to get it).  Recompiling an ARMv4 program as ARMv5
+usually results in a 25% performance improvement.</p>
+
+<ul>
+<li><a href="http://www.arm.com/products/CPUs/architecture.html">The ARM instruction set architecture</a>
+<li><a href="http://www.elinux.org/ARM_Processor">List of ARM processors</a></li>
+<li><a href="http://www.arm.linux.org.uk">The ARM Linux web page</a></li>
+<li><a href=http://www.arm.linux.org.uk/developer/machines/>List of
+over 1500 known arm systems</a>.</li>
+<li><a href=http://www.ot1.com/arm/armchap1.html>History of the ARM CPU</a></li>
+</ul>
+</span>
+
+<span id="ia64">
+<p>The Itanium was a failed attempt to create a 64-bit successor to the x86, a
+role that went to AMD's x86-64 design instead.  In 1994, Intel partnered with
+HP to produce a successor to both x86 and HP's PA-RISC, with a new instruction
+set ("ia64") fundamentally different from both.  To support software written
+for the older processors, the designers included a complete implementation of
+each, because the new chip was already so big and complex that including _two_
+entire previous processors wasn't a significant increase to either.  (If this
+sounds unlikely to end well...)</p>
+
+<p>The result was a late, slow, inefficient chip that was difficult to
+manufacture, more expensive than available alternatives, difficult to write
+efficient compilers for, quickly nicknamed "Itanic" and essentially ignored by
+the market.  (This was remarkably similar to Intel's earlier
+<a href=http://en.wikipedia.org/wiki/Intel_iAPX_432>i432 project</a>,
+a 1970's attempt to jump straight from the 8 bit 8080 to a
+32-bit processor which also resulted resulted in a slow, late, overcomplicated
+and overpriced design which the industry ignored.  The i432 was finally killed
+off by the arrival of the 80286, which outperformed it by a factor of four.
+History does repeat itself.)</p>
+
+<p>For comparison purposes, the
+<a href=http://www.failuremag.com/arch_history_edsel.html>Ford Edsel</a> sold
+64,000 units in its first year.  Itanium took over four years to sell that
+many:
+only <a href=http://news.com.com/2100-1001-276880.html?tag=nl>500 units in
+2001</a>, <a href=http://www.theinquirer.net/?article=7983>3,500 in 2002</a>,
+and around
+<a href=http://www.theregister.co.uk/2005/02/28/itanium_04_sales/>19,000 in
+2003 and 30,000 in 2004</a>.  In 2005, x86-64 systems emerged as the new
+64-bit PC standard, at which point Dell and IBM discontinued their Itanium
+servers and HP discontinued its Itanium workstations.</p>
+
+<p>To give a <a href=http://media.corporate-ir.net/media_files/irol/19/197211/press/Q12007EarningsRelease.pdf>sense of perspective</a>, in the first quarter of
+2007, the licensees of ARM Inc. shipped 724 million ARM processors.  (In one
+quarter, not a full year.) In the third quarter of 2007, the PC market shipped
+<a href=http://www.news.com/8301-10784_3-9825843-7.html>http://www.news.com/8301-10784_3-9825843-7.html>68.1 million</a> systems (mostly x86-64).
+Over in <a href=http://www.wikinvest.com/concept/Game_Consoles_Wars:_Xbox_360_vs._PS3_vs._Wii>PowerPC land</a>,
+from their launch through August 2007 the Wii had sold 9 million units, Xbox
+360 8.9 million, and Playstation 3 3.7 million (all three PowerPC based).
+Shipments of
+<a href=http://www.mdronline.com/publications/epw/issues/epw_31.html>many other
+interesting processor families</a> each number in the millions of units
+annually</a>.  The Itanium's cumulative total of 0.05 million
+in its first four years combined doesn't even show up on the same graph.</p>
+
+<p>The history of Itanium through 2003 was extensively detailed
+<a href=http://www.catb.org/~esr/halloween/halloween9.html#itanium>here</a>.
+A more recent obituary for the chip is zdnet's
+<a href=http://news.zdnet.com/2100-9584_22-5984747.html>Itanium: A cautionary
+tale</a>.</p>
+
+<p>Despite the Itanium's failure to gain any marketplace traction
+(and <a href=http://www.theinquirer.net/en/inquirer/news/2003/02/24/linus-torvalds-itanium-threw-out-all-the-good-parts-of-the-x86>Linus Torvalds'
+personal disdain for the chip</a>, the billions
+of dollars poured into Itanium resulted in lots of corporate engineers assigned
+to developing extensive Linux support for this virtually nonexistent hardware.
+But despite a documented instruction set, no open source emulators run Itanium
+code due to lack of interest.  (HP does offer a binary-only Itanium
+emulator called
+<a href=http://www.hpl.hp.com/research/linux/ski/download.php>SKI</a>, last
+updated in 2004.)</p>
+
+<p>Silicon Graphics still produces Itanium systems.  HP no longer produces
+Itanium workstations, but offers some Itanium servers.  Intel still spends
+money on it.</p>
+
+</span>
+<span id="m68knommu">
+<p>The most popular nommu 68k variant is Coldfire, which uses a subset of the
+68k instruction set and has no memory management unit.  Coldfire is currently
+used in a small number of high volume devices.  (I.E. Coldfire isn't used in
+many different products, but the products it's used in are produced in high
+volume.)</p>
+
+<p><a href=ols/2002/ols2002-pages-130-145.pdf>Running Linux on a DSP: Exploiting the Computational Resources of a programmable DSP Micro-Processor with uClinux</a> (OLS 2002)</p>
+</span>
+<span id="mips">
+<p>Mips is probably the main competitor to ARM.  One advantage of MIPS is its
+availability as a FPGA program, allowing easy prototyping of custom
+hardware.</p>
+
+<p>SGI produced primarily MIPS systems back in the Irix days.  Sony's
+Playstation 2, and PSP are MIPS based, as are some Tivo and Linksys devices.</p>
+
+<p><a href=http://en.wikipedia.org/wiki/MIPS_architecture>MIPS architecture</a></a>
+<p><a href=http://linux-mips.org>The Linux/MIPS web page</a></p>
+
+</span>
+<span id="parisc">
+<p>The PA-RISC is from Hewlett Packard.  It was scheduled to be discontinued in
+favor of the Itanium, but the failure of ia64 led to a restart of
+PA-RISC development.</p>
+
+  <a href=ols/2002/ols2002-pages-183-190.pdf>Porting Drivers to HP ZX1</a>
+</span>
+<span id="powerpc">
+<p>The PowerPC was created in the early 90's by a parnership between IBM,
+Apple, and Motorola.  Apple switched to x86-64 in 2005 and Motorola spun off
+its processor division as Freescale (which now also manufactures Coldfire and
+ARM processors).  But IBM is still strongly behind PowerPC, and the various
+users of PowerPC formed a <a href=http://powerpc.org>consortium</a> to
+promote and develop it.</p>
+
+<p>PowerPC is commonly used in high volume set-top boxes and game consoles
+such as the PlayStation 3, Xbox and Xbox 360, and Nintendo Wii.  PowerPC
+is the third most common processor type in the
+<a href=http://top500.org>Top 500</a> supercomputers list, and was used in
+older cell phones (before Motorola spun off Freescale).</p>
+
+<p>The most interesting recent PowerPC development is the <a href=http://en.wikipedia.org/wiki/Cell_microprocessor>Cell processor</a>,
+which combines a PowerPC core with 8 DSP-like "synergistic processing
+units" which can offload compute-intensive tasks like 3D acceleration,
+compression, encryption, and so on.</p>
+
+<p>The PowerPC 7xx is the "386" of PowerPC systems, meaing most modern PowerPC
+processors can run code compiled fro PowerPC 7xx (although such older code
+may not take full advantage of the new chip's capabilities, especially
+with regard to performance).  The PowerPC family also has
+<a href=http://en.wikipedia.org/wiki/PowerPC_970>64-bit variants</a> (an
+early version of which Apple marketed as the "G5") that can still run 32-bit
+PowerPC code.</p>
+
+<p>The main exceptions to 7xx compatability are two embedded subsets of the
+PowerPC, which were separately developed by IBM (the 4xx series) and Motorola
+(the 8xx series) for use in low power devices.  These are stripped down PowerPC
+processors in roughly the same way Coldfire was a stripped down 68k:
+instructions were removed from the architecture to get the transistor count
+down, and thus code must be recompiled to avoid using those instructions.
+Unfortunately, the two vendors chose a different subset of the PowerPC
+instruction set, so code compiled for 4xx won't run on 8xx, and vice versa.</p>
+
+<p>The 4xx line was purchased by <a href=http://www.amcc.com/Embedded/>AMCC</a>
+(which has the most annoying website design ever, click one of the tabs to
+get it to STOP MOVING).  Freescale mostly seems to have lost interest in the
+8xx now that Motorola has switched its' cell phones to arm, but information
+is <a href=http://www.freescale.com/webapp/sps/site/overview.jsp?nodeId=0162468rH3bTdGJk194204>still available</a>.</p>
+
+<p>The Linux PowerPC developers hang out on the #mklinux channel on
+irc.freenode.net.</p>
+
+  <a href=ols/2001/iseries.pdf>The Linux Kernel on iSeries</a> (OLS 2001)
+  <a href=ols/2001/ppc64.pdf>PowerPC 64-bit Kernel Internals</a> (OLS 2001)
+  <a href=http://perso.magic.fr/l_indien/qemu-ppc/PowerPC_ref/PowerPC_ref.html>PowerPC implementation reference for QEMU</a>
+</span>
+
+<span id="ppc">
+<p>The "ppc" architecture is obsolete, and
+<a href="Documentation/feature-removal-schedule.txt">scheduled for removal
+in June 2008</a>.</p>
+
+<p>Once upon a time, ARCH=ppc was for 32-bit PowerPC processors (7xx and up),
+and ARCH=powerpc was for 64-bit (970/G5 and up), but the two architectures were
+merged together and support for most boards has since been ported over to
+powerpc.  If you care about any of the remaining boards, bug the powerpc
+maintainers.</p>
+
+<p>Note that ARCH=ppc does not support newer features like "make
+headers_install", but ARCH=powerpc does.</p>
+</span>
+<span id="um">
+<p>User Mode Linux is a port of Linux to run as a userspace program.  Instead
+of talking to the hardware, it makes system calls to the C library.  Instead
+of using a memory managment unit it makes clever use of mmap.</p>
+
+<p>UML is sort of like an emulator: it can run Linux programs under itself
+(its processes show up as threads to the host system).  It's sometimes used
+as a superior "fakeroot", and sometimes used to provide an emulated system
+for honeypots or shared hosting services.  It's an excellent tool for
+learning and debugging the Linux kernel, because you can use all the normal
+userspace debugging techniques, up to and including putting "printf()"
+statements into the source code to see what it's doing.  (It's great for
+developing things like filesystems, not so good for device drivers.)</p>
+
+<ul>
+<li><a href=ols/2001/uml.pdf>User-Mode Linux</a> (OLS 2001)</li>
+<li><a href=ols/2002/ols2002-pages-107-116.pdf>Making Linux Safe for Virtual Machines</a> (OLS 2002)</li>
+<li><a href=http://landley.net/writing/docs/UML.html>User Mode Linux HOWTO</a></li>
+</ul>
+</span>
+<span id="x86_64">
+<p><a href=http://x86-64.org>x86-64</a> is the 64-bit successor to x86, and
+the new dominant PC processor.  Essentially all current PCs are now shipping
+with x86-64 processors, including traditionally non-x86 architectures such
+as Apple's Macintosh and Sun's servers.</p>
+
+
   <a href=ols/2001/x86-64.pdf>Porting Linux to x86-64</a> (OLS 2001)
-xtensa
+</span>
+
 
 include/asm-generic
 uml
@@ -859,8 +1171,6 @@
 for developing git, in the Google Video tech talk
 <a href=http://video.google.com/videoplay?docid=-2199332044603874737>Linus Torvalds on git</a>.</p>
 
-<p>To get started with git, see <a href=local/git-quick.html>git quickstart</a>.</p>
-
 <p>The linux kernel source is also available as a
 <a href=http://kernel.org/hg/linux-2.6>mercurial repository</a>, another
 popular open source distributed source control system.</p>