annotate www/presentation.html @ 958:0581a13cadb3

Revert patch that breaks sh4 target in 2.6.32 kernel.
author Rob Landley <rob@landley.net>
date Sat, 23 Jan 2010 06:16:59 -0600
parents 343fb4f8ffda
children 333c8f799302
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
2 <html>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
3 <head>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
4 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
5 <title>Developing for non-x86 targets using QEMU</title>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
6 <style type="text/css">
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
7 #docheader h1, #docheader h2, #docheader h3 {
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
8 text-align: center;
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
9 }
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
10 </style>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
11 </head>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
12 <body>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
13
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
14 <div id="docheader">
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
15 <h1>Developing for non-x86 targets using QEMU</h1>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
16 <h2>Rob Landley and Mark Miller</h2>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
17 <h2>Impact Linux, LLC</h2>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
18 <h2><a href="http://impactlinux.com">http://impactlinux.com</a></h2>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
19 <h3>Saturday, September 26, 2009</h3>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
20 </div>
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
21
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
22 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
23 <li><a href="#intro">Introduction</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
24 <li><p>Cross compiling, native compiling, and the third option
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
25 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
26 <li><a href="#cross_advantages">Advantages of cross compiling</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
27 <li><a href="#cross_disadvantages">Disadvantages of cross compiling</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
28 <li><a href="#native_advantages">Advantages of native compiling on real hardware</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
29 <li><a href="#native_disadvantages">Disadvantages of native compiling on real hardware</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
30 <li><a href="#what_do_we_want" >What do we want?</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
31 </ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
32 </p></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
33
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
34 <li><p><a href="#compiling_under_emulation">Compiling under emulation</a>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
35 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
36 <li><a href="#why_qemu">Why QEMU?</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
37 <li><a href="#other_emulators">Other emulators</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
38 <li><a href="#application_emulation">QEMU Application Emulation</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
39 <li><a href="#system_emulation">QEMU System Emulation</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
40 </ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
41 </p></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
42
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
43 <li><p><a href="#dev_environment">Obtaining a development environment for QEMU</a>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
44 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
45 <li>Prebuilt binaries, or build from source?</li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
46 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
47 <li><a href="#prebuilt_binaries">Using prebuilt binaries</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
48 <li><a href="#build_from_source">Building a development environment from source</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
49 </ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
50 </ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
51 </p></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
52
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
53 <li><p><a href="#understanding_build_environment">Understanding your build environment</a>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
54 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
55 <li><p>Things that can go wrong
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
56 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
57 <li><a href="#reinventing_wheel">Reinventing the wheel</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
58 <li><a href="#accidental_distros">Package management and accidental distros</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
59 <li><a href="#buildroot">Buildroot example</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
60 </ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
61 </p></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
62 <li><p><a href="#howto_dev_environment">How to put together a development environment</a>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
63 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
64 <li><a href="#download_source">Download source</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
65 <li><a href="#host_tools">Setup host tools</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
66 <li><a href="#cross_compiler">Create a cross compiler</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
67 <li><a href="#root_filesystem">Cross compile a root filesystem</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
68 <li><a href="#system_image">Package a system image</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
69 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
70 <li><a href="#alt_packaging">Alternatives to packaging</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
71 </ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
72 <li><a href="#booting_image">Booting a system image under QEMU</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
73 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
74 <li><a href="#qemu_options">QEMU command line options</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
75 <li><a href="#troubleshooting">Troubleshooting</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
76 </ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
77 </ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
78 </p></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
79 </ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
80 </p></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
81
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
82 <li><p><a href="#using_emulated">Using your emulated development environment</a>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
83 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
84 <li><a href="#getting_data_in_out">Getting data in/out of the emulator</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
85 <li><a href="#debugging">Debugging software under the emulator</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
86 <li><a href="#package_maintainers">Interacting with upstream package maintainers</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
87 </ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
88 </p></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
89
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
90 <li><p><a href="#performance">Performance considerations</a>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
91 <ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
92 <li><a href="#benchmarks">Benchmarks, bottlenecks, and context</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
93 <li><a href="#distcc">Accelerating compiles with distcc</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
94 <li><a href="#low_hanging_fruit">Other low hanging fruit</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
95 <li><a href="#future">Future approaches</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
96 <li><a href="#hardware">Throwing hardware at the problem</a></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
97 </p></li>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
98 </ul>
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
99
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
100 <a name="intro" /><h2>Introduction</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
101
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
102 <p>Emulation allows even casual hobbyist developers to build and test the software they write on multiple hardware platforms from the comfort of their own laptop.</p>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
103 <p>QEMU is rapidly becoming a category killer in open source emulation software, capable of not only booting a Knoppix CD in a window but booting Linux systems built for ARM, MIPS, PowerPC, SPARC, sh4, and more.</p>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
104 <p>This talk covers application vs system emulation, native vs cross compiling (and combining the two with distcc), using QEMU, setting up an emulated development environment, real world scalability issues, using the Amazon EC2 Cloud, and building a monster server for under $3k.</p>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
105
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
106 <a name="cross_advantages" /><h2>Advantages of cross-compiling</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
107
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
108 <ul> <li>Why would you want to cross-compile?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
109 <li>Fast
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
110 <ul> <li>About as fast as native compiling on the host</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
111 <li>Same SMP Scalability</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
112 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
113 <li>Prerequisites are common
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
114 <ul> <li>Just software, common PC Hardware and Linux OS</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
115 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
116 <li>Unavoidable
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
117 <ul> <li>Someone somewhere has to do a certain amount of cross-compiling in order to get a new target up and running</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
118 <li>Minimal subset must be supported for each target</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
119 <li>Can you build a current Linux 2.6 Kernel on an Alpha Machine running a 2.2 Kernel with GCC 2.95? If not, you need to cross-compile from a newer system.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
120 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
121 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
122
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
123 <a name="cross_disadvantages" /><h2>Disadvantages of cross-compiling</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
124
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
125 <ul> <li>Keeping Multiple Build Contexts Straight
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
126 <ul> <li>Information leaks between the target and the host</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
127 <li>Each toolchain has its own headers and libraries</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
128 <li>Different things in <code>$PATH</code> depending on target or host</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
129 <li>Hard to remove things the host has, but the target doesn't
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
130 <ul> <li>Install <code>gzip</code> on the host, but not in the target</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
131 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
132 <li><code>uname -m</code>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
133 <ul> <li>Machine type given by the host may not equal target</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
134 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
135 <li>Environment Variables</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
136 <li>Looking at <code>/proc</code></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
137 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
138 <li>Lying to Configure
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
139 <ul> <li>The design of configure is fundamentally wrong for cross-compiling</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
140 <li>Asking questions about the host to build programs for the target is crazy when the host and the target are not the same</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
141 <li><code>libtool</code>, <code>pkg-config</code>, finding Python, wrong signal numbers for Perl...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
142 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
143 <li>Hard to test the result
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
144 <ul> <li>The test suite is often part of the source. You didn't build the source on the target, so you have to copy the source tarball to the target hardware. Then to run its test suite you need to cross-compile <code>tcl</code>/<code>expect</code> and install it to the target's root filesystem.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
145 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
146 <li>Do you want to ship that?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
147 <ul> <li>Will you bother?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
148 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
149 <li>Bad Error Reporting
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
150 <ul> <li>Using the wrong "<code>strip</code>" mostly works... until you build for sh4 target.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
151 <li>The Perl thing mentioned earlier shipped to a customer. It built, it ran, it just didn't work in this one area.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
152 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
153 <li>Not all packages cross-compile
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
154 <ul> <li>Most developers barely care about native compiling on non-x86, let alone cross-compiling
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
155 <ul> <li>Implementing cross compiling complicates the heck out of the build system for everybody, to serve less than 10% of the userbase</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
156 <li>Less than 10% of the userbase will ever test it, so it'll break a lot</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
157 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
158 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
159 <li>The package developer can't reproduce your bug.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
160 <ul> <li>Developers haven't got a cross compiling build environment set up for every possible target. Installing a binary-only toolchain requiring a root login on their development machine isn't appealing</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
161 <li>Developers haven't got a test environment to run the result. Building a binary they can't run proves nothing and isn't very interesting.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
162 <li>The packages that do cross compile don't do it the same way, it's not just "<code>./configure; make; make install</code>".</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
163 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
164 <li>Maintenance
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
165 <ul> <li>Each new release breaks</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
166 <li>Discourages upgrading</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
167 <li>MIPS was broken in the 2.6.30 kernel</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
168 <li>sh4 is broken in 2.6.31</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
169 <li>The kernel is <strong>the</strong> poster child for many eyeballs</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
170 <li>Non-x86 gets much less attention than x86 at the best of times</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
171 <li>Now add 100 different build environments you could be cross compiling <strong>from</strong>. Add enough variables (which toolchain, which build scripts, which C library) and you may be the <strong>only</strong> person with your particular setup.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
172 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
173 <li>Little or no regression testing means bit-rot.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
174 <li>Open source community only cares if you're using current version
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
175 <ul> <li>Debugging old versions doesn't help the project</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
176 <li>Fixes only go upstream if applied to current</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
177 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
178 <li>Upshot
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
179 <ul> <li>Most embedded projects get stuck using old packages, even when you can get all the source.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
180 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
181 <li>Hairball Builds
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
182 <ul> <li>everything has to know about everything else</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
183 <li>buildroot wasn't initially intended to be a distro (embedded variant of Zawinski's law)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
184 <li>non-orthogonal, hard to mix and match toolchain, build scripts, packaging (rpm, dpkg, ipkg/opkg, portage, tarballs), system imaging
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
185 <ul> <li>Build system generally picks one of each and requires you to use its choices together</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
186 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
187 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
188 <li>Hidden dependencies
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
189 <ul> <li>The <strong>host</strong> has to be set up specially
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
190 <ul> <li>This is never properly documented, and the docs bit-rot if they exist</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
191 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
192 <li>It builds for this engineer but not that one</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
193 <li>Try to reproduce 6 months later and something's changed</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
194 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
195 <li>Previous mention of installing a target version of gzip into your toolchain, not just into your target system, so later packages can build against it
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
196 <ul> <li>Version in toolchain and target must match</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
197 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
198 <li>Pain to set up
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
199 <ul> <li>Knowing how to build a working cross compiler toolchain yourself is a black art
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
200 <ul> <li>Even using an existing toolchain build system ala crosstool-ng requires extensive configuration</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
201 <li>Lots of toolchain build systems aren't standalone, using their toolchain for other purposes is an afterthought</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
202 <li>Slightest mistake results in leaky toolchain</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
203 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
204 <li>Prebuilt binaries may or may not work for you unless you use the recommended version of the recommended distro
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
205 <ul> <li>Linux has never been big into binary portability</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
206 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
207 <li>Lots of absolute paths hardwired into gcc. Can't just extract into your home directory, need root access to install it as a package</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
208 <li>Most developers get a prebuilt BSP (Board Support Package, I.E. build system) and use that as a black box. (See hairball, previously)
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
209 <ul> <li>Can't report bugs upstream because package developers won't set up 15 different BSPs. Which leads into...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
210 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
211 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
212 <li>Hard to get package bugs fixed
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
213 <ul> <li>Can't report bugs upstream because package developers don't have your cross compiling environment set up and don't have the hardware to test the result anyway</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
214 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
215 <li>Restricted package selection
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
216 <ul> <li>About 200-600 packages cross compile, depending on which platform you're targeting, how much effort you're willing to put in, and your definition of success</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
217 <li>Debian has &gt;30,000 packages. That's less than 2% of the total</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
218 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
219 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
220
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
221 <a name="native_advantages" /><h2>Advantages of native compiling on real hardware</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
222
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
223 <ul> <li>Native compiling is simpler
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
224 <ul> <li>No special support required in build</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
225 <li>One context, no host/target split to leak information from</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
226 <li><code>./configure</code> works as designed</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
227 <li>Pushing bugfixes upstream takes less explanation</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
228 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
229 <li>For something like an iPhone, doing this is almost feasible.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
230 <ul> <li>There are also high-end PPC and ARM workstations.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
231 <ul> <li>This is how Fedora for ARM is built, using a build cluster of high-end ARM systems.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
232 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
233 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
234 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
235
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
236 <a name="native_disadvantages" /><h2>Disadvantages of native compiling on real hardware</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
237
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
238 <ul> <li>Need to obtain and install a bootable development environment
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
239 <ul> <li>Chicken and egg problem</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
240 <li>Either cross compile from host or use prebuilt binaries</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
241 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
242 <li>Is the device powerful enough?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
243 <ul> <li>Fast CPU, enough memory, enough disk, console access
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
244 <ul> <li>200 Mhz CPU with 32 Megs of RAM and a jffs2 flash root filesystem is not an ideal modern build environment
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
245 <ul> <li>Flash has limited write cycles, building on it shortens device life.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
246 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
247 <li>Building on NFS is painful
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
248 <ul> <li>Timestamp granularity and skew due to cacheing</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
249 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
250 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
251 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
252 <li>Did you bring enough for everybody?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
253 <ul> <li>These systems can be expensive and hard to share.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
254 <li>Can you afford to buy one for each engineer?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
255 <li>Lots of money into rapidly depreciating non-general-purpose asset.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
256 <li>Is it really cheaper to manage resource contention when engineers try to share 'em?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
257 <li>Even when you can ssh into it and it's something high-end like an iPhone or a game console, does it have enough resources to run two builds at once?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
258 <ul> <li>If not, sharing it between multiple developers is cumbersome</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
259 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
260 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
261 <li>Headless box UI issues
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
262 <ul> <li>Need lots of infrastructure before you can run "hello world"
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
263 <ul> <li>Appropriately configured/packaged/installed: toolchain, hardware installer (JTAG?), bootloader, kernel, console (serial port? Network?), root filesystem, C library, init program</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
264 <li>If any of that fails, you may not get <strong>any</strong> output. Good luck.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
265 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
266 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
267 <li>Easy to brick it
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
268 <ul> <li>Need to reboot, re-flash, configure serial console. (See "sharing" above. If it's in the server room, you walk in there a lot or set up expensive infrastructure to automate power cycling, which has its own layer of control infrastructure.)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
269 <li>Can't necessarily just stick in a boot CD and start over</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
270 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
271 <li>Reduces flexibility
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
272 <ul> <li>You have to make your hardware decisions early in your development cycle, you can't do much software development until you have the hardware</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
273 <li>You can always get it running first on x86 and then port it, which has its own issues
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
274 <ul> <li>Since x86 is the default platform everything gets developed on, "it works on x86" doesn't prove much</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
275 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
276 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
277 <li>Still tricky to send stuff upstream.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
278 <ul> <li>Package maintainers are less likely to have your hardware than they are to have/install your BSP. They can't build/test your issues.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
279 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
280 <li>Less portable
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
281 <ul> <li>A laptop can come with you on a plane, or work from home</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
282 <li>Not all targets can be powered from USB or battery</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
283 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
284 <li>Not easily replaced
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
285 <ul> <li>In case of a coffee spill you can get another PC and set it up same day</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
286 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
287 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
288
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
289 <a name="what_do_we_want" /><h2>What do we want?</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
290
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
291 <ul> <li>It's possible to cope with the above, and a lot of people do. None of these problems are insurmountable, with enough engineering time and effort.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
292 <ul> <li>But it's not ideal</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
293 <li>What have we identified so far that would bring us closer to an ideal build system?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
294 <ul> <li>Scalable
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
295 <ul> <li>Scale to large development teams without significant resource contention</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
296 <li>Run on cheap personal machines like hobbyists have, so open source hobbyists can use/develop with it. (Minimally usable on a $250 Thinkpad.)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
297 <li>Reasonably fast
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
298 <ul> <li>Take advantage of SMP (possibly even clustering for build servers) so you have the option to throw money at it to make it faster</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
299 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
300 <li>Capable of building complicated projects without collapsing into a tangle or requiring days for each build cycle</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
301 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
302 <li>Reproducible
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
303 <ul> <li>Easy to modify and debug</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
304 <li>Breaking pieces you can't rebuild from source sucks
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
305 <ul> <li>When something does break, you can make it happen again, take it apart and examine it, add logging, attach gdb to it, etc.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
306 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
307 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
308 <li>Maintainable
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
309 <ul> <li>Something you can archive and retry years from now without having to dig up a copy of Red Hat 7.2 and try to get it to work on current hardware behind enough of a firewall it won't immediately get pwned by the ambient flora on the department's Exchange server.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
310 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
311 <li>Simple
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
312 <ul> <li>Understandable, easy to learn, debug, extend.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
313 <li>If it doesn't exist, it can't break.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
314 <ul> <li>Well, almost. Piggy's zero byte file story</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
315 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
316 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
317 <li>Orthogonal
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
318 <ul> <li>Don't tie pieces together. Pick and choose.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
319 <ul> <li>Toolchain from here, build scripts from here, package management system from here...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
320 <li>Too many "accidental distros", like buildroot. Hairball is the natural state of cross compiling.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
321 <li>If you wind up writing your own in-house (or forking an upstream BSP), maintenance snowballs. You'll wind up rewriting it from scratch every 5 years or so, without improving much.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
322 <li>Something you can re-use for your <strong>next</strong> embedded project.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
323 <li>Don't let the tool take over your project</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
324 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
325 <li>Not stuck to a single implementation.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
326 <li>Upgradeable
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
327 <ul> <li>Drop in new upstream components as they release.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
328 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
329 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
330 <li>Portable
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
331 <ul> <li>Move build to new machines</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
332 <li>Take build home, to coffee shop, on a bus, on a plane</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
333 <li>What if your developer's laptop is a Mac or Windows machine?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
334 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
335 <li>Self-contained
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
336 <ul> <li>Something you can download/run without root access would be nice
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
337 <ul> <li>Dear hobbyist developer, I found a bug in your code. To reproduce it, log into your laptop as root and go...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
338 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
339 <li>Easy enough to point upstream package managers at so they can reproduce your bug, diagnose the problem themselves, and merge a fix.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
340 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
341 <li>Scriptable
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
342 <ul> <li>Automated regression testing is nice.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
343 <li>Cron job, nightly build and test.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
344 <ul> <li>Including upstream component projects.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
345 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
346 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
347 <li>Cheap
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
348 <ul> <li>Free software running on cheap commodity x86-64 hardware.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
349 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
350 <li>Easy transition to your deployment environment (final hardware).</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
351 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
352 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
353 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
354
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
355 <a name="compiling_under_emulation" /><h2>Compiling under emulation</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
356
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
357 <ul> <li>Native compiling under emulation can give us this.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
358 <li>Moore's law is on your side.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
359 <ul> <li>It's already given us ridiculously fast SMP x86-64 hardware.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
360 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
361 <li>Others have better ratio of power consumption to performance, giving longer battery life, fanless operation, less weight, physically smaller, integrated peripherals...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
362 <li>A few like the DEC Alpha, Sparc, and Itanic aimed for better absolute performance.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
363 <ul> <li>They got steamrollered.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
364 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
365 <li>Often cheaper to throw hardware at the problem instead of engineering time. Moore's law does not make programmers 50% cheaper every 18 months.</li>
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
366 <li><a name="why_qemu" />QEMU is your friend.
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
367 <ul> <li>QEMU conceptually forked off of Fabrice Bellard's Tinycc project in 2003, originally as a way to run Wine on non-x86 hosts. In six years it's become a category killer in open source emulation projects.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
368 <li>QEMU does "dynamic recompilation" on a per-page basis. As each page of executable code is faulted in, QEMU translates it to an equivalent chunk of host code. It keeps around a cache (~16 megabytes) of translated pages, so that running the same code multiple times (loops, function calls) doesn't re-incur the translation overhead.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
369 <li>The reasons this is slower than native performance are:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
370 <ul> <li>Translation overhead
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
371 <ul> <li>(essentially increased page fault latency)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
372 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
373 <li>The translated code is less efficient because any serious optimizer would slow down the translation too much</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
374 <li>That said, QEMU's ARM emulation on a 2Ghz x86_64 is likely to be faster than a real 400mhz ARM920T.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
375 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
376 <li>Good rule of thumb is 20% of native speed, but it varies target, per QEMU version, per load...
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
377 <ul> <li>Bench it yourself.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
378 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
379 </ul></li>
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
380 <li><a name="other_emulators" />There are several other open source emulation projects but:
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
381 <ul> <li>They don't emulate different targets like qemu does, just host-on-host (generally x86 on x86).</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
382 <li>Most of them use QEMU code to emulate I/O devices because QEMU has the best device support (not counting 3D acceleration hardware).</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
383 <li>Mostly survive in special purpose niches:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
384 <ul> <li>Virtualbox (which was sold to Sun which was sold to Oracle) and Xen (which was sold to Citrix) are kept alive by for-profit corporate owners, and are mostly targeting non-Linux hosts since KVM was (recently) integrated into QEMU and the Linux kernel.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
385 <li>Valgrind is really a debugger</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
386 <li>Bochs is largely an x86 bios project these days</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
387 <li>User Mode Linux and lguest are the Linux kernel emulating itself.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
388 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
389 <li>There are other non-x86 emulators (aranym for m68k, Hercules for S/390), and the concepts here apply to them too, but QEMU will probably eat them eventually.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
390 <ul> <li>They generally focus on a single target. QEMU emulates many targets, so you can apply what you've learned to future projects.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
391 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
392 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
393 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
394
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
395 <a name="application_emulation" /><h2>QEMU Application Emulation</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
396
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
397 <ul> <li>QEMU has two modes:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
398 <ul> <li>Application and System Emulation</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
399 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
400 <li>This mode runs a userspace program, intercepting system calls and translating them to/from the host OS.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
401 <li>Good for "smoke testing" your toolchain and C library: run a statically linked "hello world" program to make sure your toolchain (compiler, linker, C library, kernel headers) is set up correctly for your target.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
402 <li>Some projects (such as some versions of scratchbox) set up the kernel's "misc binary support" to call QEMU application emulation to run target binaries.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
403 <ul> <li>This is to placate <code>./configure</code>, and builds that aren't cross compile aware.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
404 <li>This is only a partial fix, leaving many problems unaddressed. The resulting build is still fairly brittle, very complicated, and requires root access on the host to set up, but it's a viable approach.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
405 <ul> <li>Also needs a version of uname that lies about the host, and some fiddling with shared library paths to run non-static target binaries, and remember to export <code>HOSTTYPE</code> and <code>MACHTYPE</code> environment variables...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
406 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
407 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
408 <li>QEMU Application Emulation is nice but limited</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
409 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
410
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
411 <a name="system_emulation" /><h2>QEMU System Emulation</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
412
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
413 <ul> <li>Standalone build system isolated from the host.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
414 <ul> <li>The rest of this talk describes this approach.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
415 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
416 <li>Advantages
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
417 <ul> <li>See "What We Want" previously. All that applies here</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
418 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
419 <li>Disadvantages
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
420 <ul> <li>As with native hardware build, need to find/create a development system
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
421 <ul> <li>Install's a lot easier though</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
422 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
423 <li>Emulation overhead, slower than native compiling
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
424 <ul> <li>Needs extra work to take advantage of SMP</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
425 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
426 <li>Sometimes building on native hardware wins (faster, more convenient, more accurate, etc.)
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
427 <ul> <li>There exists hardware with no good emulation</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
428 <li>Running on the emulator doesn't always prove it'll work properly on the real hardware</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
429 <li>You can always do both. (See "orthogonal" above, even QEMU can be swapped out.)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
430 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
431 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
432 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
433
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
434 <a name="dev_environment" /><h2>Obtaining a development environment for QEMU</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
435
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
436 <ul> <li>Root filesystem and native toolchain for target hardware, plus kernel configured specifically for one of QEMU's system emulations.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
437 <li>First question: Use prebuilt binaries or build from source?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
438 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
439
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
440 <a name="prebuilt_binaries" /><h2>Using Prebuilt Binaries</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
441
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
442 <ul> <li>Prebuilt binaries are easier but less flexible.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
443 <ul> <li>They save time up-front at the risk of costing you a <strong>lot</strong> of time if anything goes wrong</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
444 <li>If your intention is to put together a new embedded system, putting together your own development environment is good practice</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
445 <li>Tailoring a system to save space/power/latency means eliminating unnecessary packages/libraries/ daemons</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
446 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
447 <li>There are lots of prebuilt distros to choose from.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
448 <ul> <li>Your hardware vendor probably has a BSP you can repackage for QEMU.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
449 <ul> <li>They may not offer a native toolchain, though.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
450 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
451 <li>Angstrom Linux, Gentoo Embedded, Emdebian, Openwrt, Openmoko, Fedora for ARM, Buildroot, and many many more.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
452 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
453 <li>No guarantee any of them will meet your needs out of the box.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
454 <ul> <li>May need to repackage them for QEMU if they don't explicitly target it.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
455 <ul> <li>New kernel, drivers</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
456 <li>ext3 filesystem image (with native toolchain and prereq packages)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
457 <li>QEMU command line.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
458 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
459 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
460 <li>Even if you're not personally going to build a development environment from source, it's helpful to understand how one is put together.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
461 <ul> <li>Learning how it works can only help
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
462 <ul> <li>Unless you're using Windows, which stops working when you collapse its quantum state.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
463 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
464 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
465 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
466
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
467 <a name="build_from_source" /><h2>Building a development environment from source</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
468
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
469 <ul> <li>Using a build system without understanding how it works isn't that much better than just grabbing prebuilt binaries.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
470 <ul> <li>But it does give you a fall back position if something goes wrong, you can then start learning about the build system when you need to.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
471 <li>Thinking ahead, you might want to pick a build system that isn't too hard to learn if you <strong>do</strong> need to take it apart</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
472 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
473 <li>The simpler your build system is, the less there is to go wrong
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
474 <ul> <li>Less to keep up-to-date</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
475 <li>Easier to learn</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
476 <li>If cross compiling is inherently harder than native compiling, then the most maintainable system is probably one that does as little cross compiling as possible before switching over to a native build</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
477 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
478 <li>This is similar to the way tailoring an embedded system to save space/power/latency means eliminating unnecessary packages/libraries/daemons.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
479 <ul> <li>Less is more</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
480 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
481 <li>That said, the less the build system starts with, the more you'll have to add to it to meet your needs. <code>dropbear</code>, <code>strace</code>, <code>gdb</code>, <code>gzip</code>....</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
482 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
483
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
484 <a name="understanding_build_environment" /><h2>Understanding your build environment</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
485
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
486 <ul> <li>The best educational reference for this area is Linux From Scratch at <a href="http:/www.linuxfromscratch.com">http:/www.linuxfromscratch.com</a>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
487 <ul> <li>Excellent educational resource. It is explicitly <strong>not</strong> a build system. (Not automated, you cut and paste chunks to run them as you read.)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
488 <li>It describes how to create a fairly big system by embedded standards, around 100 megabytes.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
489 <li>Linux from Scratch
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
490 <ul> <li>Uses "standard" GNU bloatware. No busybox, uClibc, dropbear, full System V boot scripts</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
491 <li>Conservative approach, builds prerequisites for test suites (such as tcl/expect) as part of base system and runs each test suite during build.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
492 <li>Based on native compiling, building a new system of the same type as the host.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
493 <li>There's a Cross Linux From Scratch that covers cross compiling, but it's much newer, less thorough, less easily understandable, still a bit buggy in places, and not very actively developed (Last release was three years ago).
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
494 <ul> <li>Read (and understand) the original one first</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
495 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
496 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
497 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
498 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
499
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
500 <a name="reinventing_wheel" /><h2>Reinventing the wheel</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
501
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
502 <ul> <li>Many warnings (Danger Will Robinson)
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
503 <ul> <li>On reinventing the wheel, and the hairball problem
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
504 <ul> <li>Just about every new distro or build system in the past decade has started with somebody writing a shell script to automate the Linux From Scratch steps, and then tweaking it. You are not the first, and most of the others have a 10 year headstart on you.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
505 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
506 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
507 <li>If you do want to create your own embedded build system anyway, expect it to take about a year to get something reasonably usable. (It's a great learning experience.)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
508 <li>Maintaining your own build system is enormously time consuming.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
509 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
510
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
511 <a name="accidental_distros" /><h2>Package Management and Accidental Distros</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
512
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
513 <ul> <li>Linux From Scratch intentionally does not cover package management.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
514 <li>Package management two parts: Using a package manager (<code>rpm</code>, <code>dpkg</code>, <code>ipkg</code>/<code>opkg</code>, <code>portage</code>) to install/uninstall/track packages, and creating a repository of build scripts and package dependencies.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
515 <li>Creating a repository of package dependencies and build options is a huge undertaking and requires constant maintenance as new packages are released upstream.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
516 <li>Fedora, Debian, Gentoo, and others already have perfectly acceptable repositories you can leverage. Creating your own is a huge time sink.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
517 <li>Despite this, it's very easy to fall into the trap of creating your own Linux distro by mistake, at which point complexity explodes out of control.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
518 <ul> <li>Maintaining your own distro is enormously time consuming.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
519 <li>The more packages your build system builds, the more likely this is to happen.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
520 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
521 <li>Defending yourself from accidental distros.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
522 <ul> <li>Divide the hairball into orthogonal layers. Toolchain selection has nothing to do with package management</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
523 <li>Needing to change one is no excuse for accepting responsibility to maintain the other.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
524 <li>Delegate everything you can to existing projects.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
525 <ul> <li>Push your changes upstream, which makes them somebody else's problem.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
526 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
527 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
528 <li>Figure out what your goals are and what you're <strong>not</strong> going to do
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
529 <ul> <li>You can't stay focused if you can't say no</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
530 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
531 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
532
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
533 <a name="buildroot" /><h2>Buildroot Example</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
534
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
535 <ul> <li>The <strong>buildroot</strong> project was originally just a uClibc toolchain creator and test harness. The easy way to test that a package built against uClibc was to add it to buildroot, and since it never had a clear design boundary allowing it to say "no" to new features, this quickly grew out of hand.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
536 <li>The project maintainer (Erik Andersen) and several of his senior developers had so much of their time taken up by buildroot they stopped contributing to the original project, uClibc.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
537 <li>The uClibc mailing list was swamped by buildroot traffic for years until Rob created a new buildroot list and kicked the traffic over there, but uClibc development still hasn't fully recovered.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
538 <li>This is a significant contributing factor to uClibc's TLS/ NPTL support being feature complete for its first architecture in 2006 (there was an OLS paper on it) but still not having shown up in an actual uClibc release 3 years later.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
539 <li>They do the distro thing badly: no package manager, no repository.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
540 <ul> <li>Buildroot didn't use a package manager (like rpm, dpkg, ipkg/opkg, or portage), instead it encoded its dependency resolution in menuconfig's kconfig files, and the package build logic in a series of nested makefiles, then stored the whole thing in source control. Thus the builtroot develpers took a while to notice it was becoming a distro.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
541 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
542 <li>Lots of open source build systems forked off of buildroot, just as lots of in-house build systems forked off of board support packages. They quickly diverge enough to become independent projects capable of absorbing enormous amounts of time.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
543 <li>Moral: Original project stalled because its developers sucked away, wound up doing the distro thing badly with no package manager, no repository, no stable releases.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
544 <ul> <li>Don't let this happen to you.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
545 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
546 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
547
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
548 <a name="howto_dev_environment" /><h2>Orthogonal layers</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
549
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
550 <ul> <li>The following is based on what our Firmware Linux project does.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
551 <ul> <li>Other build systems need to do the same things, most of them just aren't as careful to separate the layers.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
552 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
553 <li>Each of these layers can be swapped out
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
554 <ul> <li>Use somebody else's cross compiler, use our cross compiler to build somebody else's root filesystem, package an arbitrary directory with our packaging script, etc.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
555 <li>We try to make it as easy as possible to NOT use our stuff, and show it here only as an example.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
556 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
557 <li>This build is intentionally implemented as a simple series of bash scripts, so you can read the scripts to see what it's doing.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
558 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
559
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
560 <a name="download_source" /><h2>Downloading Source</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
561
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
562 <ul> <li>Download source code (download.sh)
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
563 <ul> <li>Calls wget on a bunch of URLs to copy files into "packages" directory.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
564 <li>You can just <code>cp</code> 'em there yourself instead if you like</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
565 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
566 <li>Does security/integrity sha1sum checks on existing tarballs (if any)
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
567 <ul> <li>Some build systems (such as crosstool-ng) give the weirdest errors if you re-run the build after an interrupted download, and you have to figure out the problem and fix it by hand</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
568 <li>The GNU ftp server has been cracked before</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
569 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
570 <li>Cache tarballs so once you've run this step you no longer need net access
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
571 <ul> <li>Falls back to a couple mirrors if the source site is down
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
572 <ul> <li><code>PREFERRED_MIRROR</code> environment variable points to a mirror to try first if you have a local copy. Note that QEMU's 10.0.2.2 tunnels through to the host's 127.0.0.1, so if your build host runs a web server on loopback it can easily provide source tarballs to a build inside QEMU without bogging the public servers.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
573 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
574 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
575 <li>Track all the source URLs and versions in one place</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
576 <li>Only this file has version info in it, making drop-in upgrades a two-line change.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
577 <ul> <li>Non-drop-in upgrades require debugging, of course</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
578 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
579 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
580
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
581 <a name="host_tools" /><h2>Host Tools</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
582
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
583 <ul> <li>Entirely optional step, can be skipped if your host is set up right, but that's seldom a good idea.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
584 <li>Serves same purpose as LFS chapter 5: an airlock preventing leakage.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
585 <ul> <li>LFS chroots, FWL adjusts the <code>$PATH</code> to remove stuff.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
586 <li>Remember, adding stuff is easy. Removing stuff the host already has so cross compiling doesn't accidentally find and leak it into your build is one of the hard parts.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
587 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
588 <li>Provides prerequisites automatically, no need to install packages by hand as root
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
589 <ul> <li>Nothing the FWL build system does requires root access on the host. That's an explicit design goal.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
590 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
591 <li>Isolate your build from variations in the host distro
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
592 <ul> <li>Provide known versions of each tool</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
593 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
594 <li>Smoke test and regression test
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
595 <ul> <li>Building everything from here on with the same tools we're going to in the final system (busybox, etc) demonstrates that the build system we're putting together can reproduce itself.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
596 <li>And if it can't (regression) we find out early</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
597 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
598 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
599
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
600 <a name="cross_compiler" /><h2>Create a Cross-Compiler</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
601
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
602 <ul> <li>Four packages: <code>binutils</code>, <code>gcc</code>, linux kernel headers, <code>uclibc</code>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
603 <ul> <li>In that order
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
604 <ul> <li>gcc depends on binutils, building uClibc depends on all three previous packages.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
605 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
606 <li>Later, we add a fifth package: uClibc++, for c++ support.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
607 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
608 <li>Use a compiler wrapper (<code>ccrap.c</code>) based on the old uClibc wrapper to make it all work together
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
609 <ul> <li>The wrapper parses the gcc command line and then rewrites it to start with <code>--nostdinc</code> and <code>--nostdlib</code>, then explicitly adds in every library and header path gcc needs, pointing to a known location determined relative to where the wrapper binary currently is.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
610 <li>This prevents the host toolchain's files (in <code>/usr/include</code> and <code>/lib</code> and such) from leaking into the target toolchain's builds.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
611 <li>It also means the toolchain is "relocatable", I.E. it can be extracted into any directory and run from there, so it can live in a user's home directory and does not require root access to install.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
612 <li>Just add the toolchain's "bin" subdirectory to your <code>$PATH</code> and use the appropriately prefixed <code>$ARCH-gcc</code> name to build.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
613 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
614 <li>The simple compiler above can be created without prerequisites</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
615 <li>Thus creating a simple cross compiler is always the first step</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
616 <li>For FWL releases we build a second cross compiler that's statically linked against uClibc on the host, which is a two-stage process
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
617 <ul> <li>First build an i686-gcc compiler, which outputs 32-bit x86 binaries</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
618 <li>Then rebuild the cross compilers as <code>--static</code> binaries using that as the host compiler</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
619 <li>This increases portability and decreases size. You can run the resulting binaries on 32 bit Red Hat 9 and on 64-bit Ubuntu 9.04.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
620 <li>This one also builds uClibc++, an embedded replacement for libstdc++
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
621 <ul> <li>The first toolchain can build C++, but has no standard C++ library to link them against</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
622 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
623 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
624 <li><code>--disable-shared</code> vs <code>--enable-shared</code>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
625 <ul> <li>History of <code>ccwrap</code>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
626 <ul> <li>uClibc compiler wrapper</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
627 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
628 <li>evolution of buildroot</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
629 <li>Why building <code>libstdc++</code> is painful</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
630 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
631 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
632
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
633 <a name="root_filesystem" /><h2>Cross-compile a root filesystem</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
634
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
635 <ul> <li>Long story short: seven packages (<code>busybox</code>, <code>uClibc</code>, <code>linux</code>, <code>binutils</code>, <code>gcc</code>, <code>make</code>, <code>bash</code>), plus some empty directories and misc files like an init script.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
636 <ul> <li>This creates a directory full of files, suitable to <code>chroot</code> into.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
637 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
638 <li>Create directory layout
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
639 <ul> <li><code>tmp proc sys dev home usr</code></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
640 <li>Symlinks into <code>usr/{bin,sbin,lib,etc}</code> from top level.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
641 <ul> <li>Historical note where "<code>usr</code>" came from in 1971.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
642 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
643 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
644 <li>Install uClibc
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
645 <ul> <li>Build this first so everything else can build against it.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
646 <li>Cross compiler may not come with libc we're installing, so use <code>ccwrap</code> on existing cross compiler here.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
647 <ul> <li>This is redundant when using <strong>our</strong> compiler, but necessary for orthogonality.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
648 <ul> <li>Allows you to use arbitrary existing cross compiler</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
649 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
650 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
651 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
652 <li>Build busybox
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
653 <ul> <li>Provides almost all POSIX/SUSv4 command line utilities</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
654 <li>Switch off a few commands that don't build on non-x86</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
655 <li>Add a few supplemental commands (<code>getent</code>, <code>patch</code>, <code>netcat</code>, <code>oneit</code>) from "<code>toybox</code>" or as shell scripts. Mostly optional.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
656 <ul> <li>Biggest issue is that busybox <code>patch</code> can't apply hunks at an offset, which makes it too brittle to use in a development environment</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
657 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
658 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
659 <li>Install native toolchain
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
660 <ul> <li>Same packages as cross compiler, plus uClibc++</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
661 <li>Note, FWL builds a statically linked native compiler, which you can extract and run on an arbitrary target system.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
662 <ul> <li>Same theory as the statically linked cross compiler, only for native builds on systems that don't come with a uClibc toolchain.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
663 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
664 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
665 <li>Install <code>make</code>, <code>bash</code>, and (optionally) <code>distcc</code>.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
666 <ul> <li>You can't do <code>./configure; make; make install</code> without "<code>make</code>". (While cross compiling, used the host's version of make.)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
667 <li>Build <code>bash</code> because busybox <code>ash</code> isn't quite good enough yet
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
668 <ul> <li>Getting pretty close, need to retest and push fixes upstream</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
669 <li>Building bash 2 not bash 3, because it's 1/3 the size, and doesn't require <code>ncurses</code></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
670 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
671 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
672 <li>Install init script and a few misc files
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
673 <ul> <li>Copy <code>sources/native/*</code> to destination</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
674 <li>Includes an init script</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
675 <li>An <code>etc/resolf.conf</code> for QEMU</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
676 <li>Some "hello world" C source programs in <code>usr/src</code>...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
677 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
678 <li>The init script does the following
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
679 <ul> <li>Mount sysfs on <code>/sys</code>, and tmpfs on <code>/dev</code> and <code>/tmp</code>.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
680 <ul> <li>tmpfs is a swap-backed ramfs, writeable even if root filesystem is read-only.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
681 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
682 <li>populate <code>/dev</code> from <code>/sys</code> (using busybox <code>mdev</code> instead of <code>udev</code>)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
683 <li>Configure <code>eth0</code> with default values for QEMU's built-in vpn.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
684 <li>Set clock with <code>rdate</code> if unix time &lt; 1000 seconds.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
685 <ul> <li>This means this emulation has no battery backed up clock.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
686 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
687 <li>Uses host's <code>inetd</code> on 127.0.0.1 to get current time.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
688 <li>If we have <code>/dev/hdb</code> or <code>/dev/sdb</code>, mount it on <code>/home</code>.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
689 <li>It echos "<code>Type exit when done.</code>", which means the boot was successful and gives "<code>expect</code>" something to respond to.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
690 <li>Use "<code>oneit</code>" program to run a command shell.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
691 <ul> <li>This gives us a shell with a controlling tty (ctrl-c works), and the system shuts down cleanly when that process exits.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
692 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
693 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
694 <li>Optimizations
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
695 <ul> <li>Strip binaries, delete info and man pages, and gcc's "<code>install-tools</code>".</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
696 <li>Build without a native toolchain
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
697 <ul> <li>Means you can also delete <code>/usr/include</code> and <code>/usr/lib/*.a</code> from uClibc.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
698 <li>Don't need <code>make</code> or <code>bash</code></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
699 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
700 <li>Build static or dynamic
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
701 <ul> <li>When building static and not installing a native toolchain, we can skip this step entirely. Just use whichever libc the cross compiler has, and don't bother to install it on the target.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
702 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
703 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
704 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
705
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
706 <a name="system_image" /><h2>Package a system image</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
707
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
708 <ul> <li><code>qemu-system-*</code> needs a bootable kernel and a filesystem image.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
709 <ul> <li>So does real hardware.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
710 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
711 <li>Build a bootable Linux kernel
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
712 <ul> <li>Needs correct CPU and board layout (device tree or equivalent), drivers for serial console, hard drive, network card, clock, ext2/ext3/squashfs/tmpfs</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
713 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
714 <li>Which kernel <code>.config</code> do you build?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
715 <ul> <li>The kernel has default configuration files for a lot of boards.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
716 <li>Look in the Linux kernel source for an <code>arch/$ARCH/configs</code> directory
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
717 <ul> <li>Or a single <code>arch/$ARCH/defconfig</code> file for the less-diverse architectures</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
718 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
719 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
720 <li>We use the "miniconfig" technique
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
721 <ul> <li>Uses <code>miniconfig.sh</code> to create one</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
722 <li><code>make allnoconfig KCONFIG_ALLCONFIG=filename</code></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
723 <li>Portability, Readability</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
724 <li>Also works with uClibc and busybox <code>.config</code>s.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
725 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
726 <li>Use <code>mksquashfs</code>, <code>gene2fs</code>, or <code>cpio</code>/<code>gzip</code> to create standalone filesystem image.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
727 <ul> <li>Possibly rebuild kernel to include initramfs (gzipped cpio image).
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
728 <ul> <li>Note initramfs is size-limited by most kernels' memory mappings, and seldom has enough space for a dozen megabytes of toolchain.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
729 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
730 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
731 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
732
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
733 <a name="alt_packaging" /><h2>Alternatives to packaging</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
734
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
735 <ul> <li>All sadly inferior, but know your options</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
736 <li>QEMU Application Emulation
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
737 <ul> <li>It can run programs out of root-filesystem.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
738 <ul> <li>Requires root access to use the <code>-chroot</code> option (and to populate <code>/dev</code>).</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
739 <li>Not much faster than full system emulation, but significantly more brittle.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
740 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
741 <li>Strangely, application emulation is a harder problem to solve than system emulation.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
742 <ul> <li>System emulation starts from scratch and fakes something simple, a CPU, some attached memory (with MMU), and a half-dozen well-documented device chips (clock, serial, hard drive, etc).</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
743 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
744 <li>QEMU application emulation must intercept literally thousands of syscalls and ioctl structures (each of which may have a dozen members) and translate every one its own special case, dealing with endianness and alignment and sometimes different meanings of the fields on different architectures.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
745 <li>Signal numbers differ.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
746 <li>Page sizes differ, which affects mmap behavior in several ways (padding the end of the file with zeroes for <strong>how</strong> long?)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
747 <li>It's <strong>fiddly</strong>.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
748 <li>QEMU application emulation is a nice smoke test, but you wouldn't want to develop there.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
749 <ul> <li>Run a static "hello world" to see if your compiler's working.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
750 <li>Fewer prerequisites between you and a simple shell prompt.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
751 <ul> <li>No need to build a kernel, let alone get serial console working.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
752 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
753 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
754 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
755 <li>QEMU can create virtual FAT disk images
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
756 <ul> <li>Option "<code>-hda fat:/path</code>"</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
757 <li>This exports a directory as a virtual block device containing a read-only vfat filesystem constructed from the files at path.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
758 <ul> <li>Remember to mount it read-only: if Linux tries to write to it lost interrupts and driver unhappiness ensue.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
759 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
760 <li>Standard "booting from vfat" issues
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
761 <ul> <li>Case insensitivity, no ownership, incomplete permissions...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
762 <li>Doable, but fiddly.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
763 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
764 <li>Do <strong>not</strong> change contents of host directory while QEMU is running, it'll get very confused.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
765 <li>Assembling a virtual FAT with lots of little files can take lots of time and memory
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
766 <ul> <li>Doesn't scale. QEMU may grind for quite a while before launch with even a relatively small filesystem.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
767 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
768 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
769 <li>Exporting a directory via a network filesystem is another way to provide a root filesystem to QEMU or to real hardware.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
770 <ul> <li>May be OK for read-only root filesystem, not so good for writeable development space you compile stuff in.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
771 <li>Login management unpleasant, make sure you do this behind a firewall or via loopback (10.0.2.2 in qemu is 127.0.0.1 on host).</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
772 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
773 <li>Linux's most commonly used network filesystem (NFS, Samba, TFTP) all <strong>suck</strong> in various ways.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
774 <ul> <li>NFS sucks rocks
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
775 <ul> <li>The words "stateless" and "filesystem" do not belong in the same sentence. (Thanks Sun!)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
776 <li>Requires root access on the host to export anything</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
777 <li>Fiddly and brittle to automate, so usually done by hand</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
778 <li>Building on non-readonly NFS is unreliable due to dentry cacheing issues
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
779 <ul> <li>Make gets very confused when timestamps go backwards.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
780 <li>General problem: either you have network round trip latency for each directory lookup or you have local dentry caches that can get out of sync with the server's timestamp info.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
781 <li>At least most things don't care about atime.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
782 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
783 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
784 <li>Samba sucks almost as many rocks
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
785 <ul> <li>It's a Windows protocol, constructed entirely out of weird non-POSIX behavior and corner cases.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
786 <ul> <li>Case insensitive, many builds don't like this.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
787 <li>Dentry info translated (can confuse make), doing things like mmap (and even symlinks) hacked on.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
788 <li>Share naming and account management are their own little world.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
789 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
790 <li>Finding the server can be a pain
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
791 <ul> <li>Domain server and Active Directory and such: there be dragons</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
792 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
793 <li>But at least it's got a userspace server that can export a filesystem running as a normal user, and a lot of people already understand it quite well.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
794 <li>QEMU has a built in <code>-smb</code> option, which launches samba on the host for you to export the appropriate directory, visible inside the emulator on 10.0.2.4 as "<code>\\smbserver \qemu</code>".</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
795 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
796 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
797 <li>TFTP
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
798 <ul> <li>Only mentioned because QEMU has a built in <code>-tftp</code> option, which exports a directory of files to the emulator via a built-in (emulated) tftp server.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
799 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
800 <li>Filesystem in Userspace (sshfs and friends)
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
801 <ul> <li>Can't boot from FUSE, need another root filesystem (generally initramfs) for setup, so doesn't solve this problem.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
802 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
803 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
804
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
805 <a name="booting_image" /><h2>Booting a system under QEMU</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
806
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
807 <ul> <li>The command is "<code>qemu-system-$ARCH</code>"
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
808 <ul> <li>Use tab completion to see the list</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
809 <li>"<code>qemu</code>" should be called "<code>qemu-system-i386</code>" and have a symlink from "<code>qemu</code>" to your host.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
810 <li>Most hosts these days are not <strong>32</strong> bit, so "<code>qemu</code>" on them should probably point to <code>qemu-system-x86_64</code>.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
811 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
812 <li>Not only many different binaries, but also sub-options within each.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
813 <ul> <li>Try "<code>-M ?</code>" to list board types and "<code>-cpu ?</code>" to list processor variants.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
814 <li>Which boards have what hardware is at <a href="http://www.qemu.org/qemu-doc.html">http://www.qemu.org/qemu-doc.html</a> (mostly in chapter 4).</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
815 <li>The <code>-cpu</code> option affects choice of cross compiler and root filesystem.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
816 <li>The <code>-M</code> options just affects kernel <code>.config</code>.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
817 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
818 <li><code>qemu-system-arm</code> handles both endiannesses, but <code>qemu-system-mips/mipsel</code> are separate. <code>qemu-system-x86_64</code> has all the 32-bit <code>-cpu</code> options, but there's a 32 bit qemu also.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
819 <li>Running a binary on a Pentium doesn't prove it'll run on a 486.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
820 <ul> <li>Unaligned access works just fine on an x86 host, but should throw an exception on things like m68k or ARM. QEMU may allow it anyway.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
821 <ul> <li>Newer versions of QEMU are a lot better about catching that sort of thing (they're implementing the fiddliest weird corner cases these days), but "runs in the emulator" never proves it'll run on the real hardware.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
822 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
823 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
824 <li>Emulation is great for development and initial testing, but there's never any real substitute for testing in situ.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
825 <li>Your board may have peripherals QEMU doesn't emulate anyway.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
826 <ul> <li>Note you can tunnel any USB device through to QEMU, check the QEMU documentation for details.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
827 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
828 <li>Sometimes you want a pickier <code>-cpu</code> setting than QEMU currently provides
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
829 <ul> <li>ppc440 code can often run on full ppc (it's mostly a subset of the instruction set), but not always, and not the other way around. But although <code>qemu-system-ppc</code> has an "<code>-M bamboo</code>" board, it doesn't have ppc440 <code>-cpu</code> option.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
830 <li>My first attempt at an "armv4lt-eabi" target worked on qemu's default CPU (an armv5l) but didn't work on real arm920t hardware.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
831 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
832 <li>QEMU development is rapid, and they're adding more and more granularity all the time. Check back, ask on the list, write it yourself, offer to sponsor the existing developers to work on your feature (several of them work for codesourcery.com)...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
833 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
834
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
835 <a name="qemu_options" /><h2>QEMU Command Line Options</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
836
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
837 <ul> <li>Command line options to <code>qemu-system-$ARCH</code>:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
838 <ul> <li>Use "<code>-nographic</code>" to make qemu scriptable from the host.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
839 <li>Disables video card emulation, and instead connects the first emulated serial port to the QEMU process's stdin/stdout.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
840 <li>This gives you an emulator shell prompt about like ssh would, in a normal terminal window where cut and paste work properly.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
841 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
842 <li>You might want to build and use "<code>screen</code>" inside the emulator, or set <code>$TERM</code>, <code>$ROWS</code> and <code>$COLUMNS</code> for things like busybox "<code>vi</code>".</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
843 <li>This means you can run QEMU as a child process and script it with "<code>expect</code>" (or similar).
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
844 <ul> <li>This is marvelous for automating builds, regression testing, etc.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
845 <li>Wrap a GUI (or emacs) around the QEMU invocation.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
846 <ul> <li>QEMU runs on non-Linux hosts, no reason your GUI couldn't run on a Mac</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
847 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
848 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
849 <li>Note, some versions of QEMU are built with a broken curses support that craps ANSI escape sequences randomly to stdout, which will confuse expect. If you encounter one of these, rebuild it from source with <code>--disable-curses</code>.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
850 <ul> <li>In theory this shouldn't happen if stdin is not a tty, but there are buggy versions out there that don't check.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
851 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
852 <li>You can also cat a quick and dirty shell script into QEMU's stdin for the emulated Linux's shell prompt to execute.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
853 <ul> <li>The reason this is "quick and dirty" is it isn't 100% reliable, because the input is presented to stdin before the kernel has a chance to boot.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
854 <li>Serial port initialization eats a variable amount of data, generally around the UART buffer size (16 bytes-ish for a 16550a, but there's some race conditions in there so it's not quite contstant).</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
855 <li>This is a Linux kernel serial/tty issue, not a QEMU issue.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
856 <li>You can work around this by starting your script with a line of space characters and a newline, and then it'll work ~99% of the time. But if you use it a lot (automated regression, nightly builds, etc) the lack of flow control can inject the occasional spurious failure because the script didn't read right.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
857 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
858 <li>Using "<code>expect</code>" (or similar) is more reliable. The kernel doesn't discard data when it's <strong>listening</strong> for data. The tty driver and various programs just get a bit careless with the buffer if you get too far ahead when they're <strong>not</strong> listening for data.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
859 <li>Use "<code>-no-reboot</code>" so QEMU exits when told to shut down by the kernel.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
860 <ul> <li>Helps with scriptability.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
861 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
862 <li>Use "<code>-kernel FILENAME</code>" to specify which kernel image to boot.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
863 <ul> <li>Avoids the need for grub or u-boot to launch emulated system.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
864 <li>Uses a very simple built-in bootloader to load kernel image into QEMU's emulated physical memory and start it.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
865 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
866 <li>QEMU has code to boot an ELF format linux kernel
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
867 <ul> <li><code>vmlinux</code> in top level kernel build
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
868 <ul> <li>Alas, not enabled for all architectures</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
869 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
870 <li>Can also handle bzImage and various target-specific variants</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
871 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
872 <li>"<code>-hda FILENAME</code>"
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
873 <ul> <li>Provides virtual ATA/SCSI block devices for board emulation
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
874 <ul> <li>Specify more with <code>-hda</code>, <code>-hdb</code>, <code>-hdc</code>...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
875 <li>First one can be a read-only root filesystem, second writeable scratch space to compile stuff in, third extra data such as source tarballs...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
876 <li>Yes the option is "<code>-hda</code>" even when the board is emulating SCSI and thus the device it emulates becomes "<code>/dev/sda</code>".</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
877 <li>No need to partition these, Linux can mount <code>/dev/hda</code> just fine</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
878 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
879 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
880 <li>"<code>-append 'OPTIONS'</code>"
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
881 <ul> <li>The Linux kernel command line
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
882 <ul> <li>In theory this is extra stuff added to the default kernel command line, but in practice it's generally the whole kernel command line.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
883 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
884 <li>Relevant options you probably want to specify:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
885 <ul> <li>"<code>console=ttyS0</code>" (or platform equivalent device)
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
886 <ul> <li>Tell the kernel to send boot messages (and <code>/dev/console</code>, thus the stdin/stdout of PID 1) to a serial device</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
887 <li>I.E. "serial console"</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
888 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
889 <li>"<code>root=/dev/hda</code>"
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
890 <ul> <li>Specify where the root filesystem lives</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
891 <li>Also specify "<code>rw</code>" so root filesystem is read/write</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
892 <li>Don't need either if booting from initramfs</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
893 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
894 <li>"<code>init=/usr/sbin/init.sh</code>"
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
895 <ul> <li>Specify what executable file to launch as the initial process. The "init task" (PID 1) is special. It has signals blocked, the kernel panics if it ever exits, it gets sigexit for orphanedchild processes and has to reap their zombies, etc</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
896 <li>Note that just using <code>init=/bin/sh</code> will give you a quick and dirty shell prompt, but you won't have a controlling tty, thus ctrl-c won't work.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
897 <li>Meaning you won't be able to break out of simple things like "<code>ping</code>". Very easy to lose control, gets old fast.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
898 <li>Also need to set up <code>/proc</code> yourself, populate <code>/dev</code>, <code>ifconfig</code>...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
899 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
900 <li>"<code>panic=1</code>"
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
901 <ul> <li>This tells the kernel to reboot one second after panicking
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
902 <ul> <li>Which <code>-no-reboot</code> above turns into QEMU exiting</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
903 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
904 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
905 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
906 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
907 <li>Put it all together and you have something like:</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
908 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
909
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
910 <p><strong><code>qemu -nographic -no-reboot -kernel zImage-i686 -hda image-i686.sqf -append "root=/dev/hda rw init=/usr/sbin/init.sh panic=1 PATH=/usr/bin console=ttyS0"</code></strong></p>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
911
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
912 <a name="troubleshooting" /><h2>Troubleshooting</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
913
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
914 <ul> <li>Do you see kernel boot messages on QEMU's stdout? If not:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
915 <ul> <li>Your serial console might not work
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
916 <ul> <li>Linux serial driver
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
917 <ul> <li>static driver, not a module. <code>*</code> instead of <code>m</code>.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
918 <li><code>console=</code> line on kernel command line</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
919 <li>Is QEMU's board emulation giving you the serial device you expect?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
920 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
921 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
922 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
923 <li>Is my toolchain even producing the right output?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
924 <ul> <li>Build a statically linked "hello world" and run that on the host using QEMU's application emulation.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
925 <ul> <li>If it won't run, use the "<code>file</code>" command on the binary (and the <code>.o</code> and <code>.a</code> files) and check that it the kind of binary you think it is.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
926 <li>If the toolchain you're using uses <code>ccwrap</code>, try setting the environment variable <code>WRAPPER_DEBUG=1</code> to see all the component files linked into the binary, and check them with "<code>file</code>".</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
927 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
928 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
929 <li>Kernel may not be booting up far enough to spit anything out
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
930 <ul> <li>Is it the right configuration?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
931 <ul> <li>Enable <code>EARLY_PRINTK</code> in the kernel</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
932 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
933 <li>Is it packaged the right way?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
934 <ul> <li><code>vmlinux</code>, bzImage, <code>chrp</code>...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
935 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
936 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
937 <li>Do you see boot messages but no shell prompt?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
938 <ul> <li>Ok, where does it stop?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
939 <li>Seeing some output is always better than seeing no output</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
940 <li>You can always stick more <code>printk()</code> calls into the kernel source code</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
941 <li>Is it hanging on a driver before it tries to launch init?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
942 <ul> <li>Make sure the board emulation and kernel <code>.config</code> agree.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
943 <li>If it's a target that needs a "device tree", what does that say?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
944 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
945 <li>Not finding the root filesystem?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
946 <ul> <li>Is the <code>root=</code> line correct?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
947 <li>Correct block driver?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
948 <li>Correct filesystem format driver?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
949 <li>Make sure those two drivers are static, not modules.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
950 <li>Try an initramfs?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
951 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
952 <li>Mounting but unable to run init?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
953 <ul> <li>Check your filesystem image (via loopback mount) to make sure the binary you expect really is there.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
954 <li>Note that an initramfs uses "<code>rdinit=</code>" instead of "<code>init=</code>".</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
955 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
956 <li>Try pointing init= at a statically linked "hello world" program.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
957 <ul> <li>FWL contains one at <code>/bin/hello-static</code></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
958 <li>If this works, your dynamic linker is probably at fault.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
959 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
960 <li>Then try a dynamically linked hello world. If that fails:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
961 <ul> <li>Is your library loader (<code>/lib/ld-uClibc.so.0</code> or similar) where you think it is?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
962 <ul> <li><code>ldd</code> is target specific but "<code>readelf</code>" is fairly generic.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
963 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
964 <li>Make sure all the libraries it tries to link to are there.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
965 <ul> <li>Remember: Shared libraries link to other shared libraries</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
966 <li>run <code>ldd</code>/<code>readelf</code> on all</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
967 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
968 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
969 <li>Run "<code>file</code>" on everything to make sure no host binaries have leaked into your root filesystem.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
970 <li>Maybe dynamic loading isn't supported with that libc on that target? (*cough* uClibc on Sparc *cough*)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
971 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
972 <li>If it launches <code>init</code> but you get no further output.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
973 <ul> <li>Is your init trying to redirect <code>/dev/console</code> and missing the serial console?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
974 <ul> <li>Work your way up from <code>init=/helloworld</code> through a shell prompt to whatever your init program is doing.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
975 <ul> <li>Add our prebuilt static <code>strace</code> binaries to your root filesystem and run your init under that.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
976 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
977 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
978 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
979 <li><code>strace</code> turns random hangs into "it made it <strong>this</strong> far, and was trying to do <strong>that</strong>"
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
980 <ul> <li>Only useful if you're getting output from "hello world".</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
981 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
982 <li>Target-specific issues
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
983 <ul> <li>ARM, MIPS, PowerPC, SPARC, sh4, x86, x86_64, M68k...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
984 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
985 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
986
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
987 <a name="using_emulated" /><h2>Using your emulated development environment</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
988
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
989 <ul> <li>The emulated development environment gives you two things
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
990 <ul> <li>A scriptable build environment, so you can run nightly automated regression testing cron jobs to build and test your software
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
991 <ul> <li>Not a substitute for testing on real hardware, but a good automated smoke test</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
992 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
993 <li>A shell prompt at which you can do fresh development</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
994 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
995 <li>As with all new development environments, you may need to build/install lots of prerequisite packages to get the development environment you really want
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
996 <ul> <li><code>gzip</code>, <code>ncurses</code>, <code>openssl</code>, <code>strace</code>, <code>dropbear</code>, <code>screen</code>...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
997 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
998 <li>Remember: what your development machine needs and what your deployment environment needs aren't necessarily the same thing.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
999 <ul> <li>Your development image has a toolchain. Probably won't ship that.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1000 <li>Quite possibly other stuff you'll develop with but not ship.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1001 <ul> <li>Needing to build it != needing to ship it.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1002 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1003 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1004 <li>Editing source on your host system is a lot more comfortable.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1005 <ul> <li>Trying to add too much UI to the emulators is probably a waste of time. Your host is pretty and user friendly already</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1006 <li>busybox has a "<code>vi</code>" implementation, so if you really need to you can edit source inside the emulator.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1007 <li>Using "<code>screen</code>" (which requires ncurses) makes things much less painful.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1008 <ul> <li>Probably need to export <code>$TERM</code>, <code>$LINES</code>, and <code>$COLUMNS</code> by hand to get it to work, ncurses can't query a serial console for tty info</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1009 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1010 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1011 <li>You probably still want to do most of your editing and source control and such on your host, in an IDE, with multiple xterms and source control and so on.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1012 <ul> <li>So the problem becomes getting source from your host into your build system (and updating it), and getting results back out.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1013 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1014 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1015
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
1016 <a name="getting_data_in_out" /><h2>Getting data in/out of the emulator</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1017
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1018 <ul> <li>Your three main options are:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1019 <ul> <li>stdin/stdout
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1020 <ul> <li>We mentioned the <code>cat</code>/<code>expect</code> scriptability.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1021 <li>Also cut and paste.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1022 <li>Good for small stuff, but doesn't scale to bulk data.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1023 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1024 <li>Filesystem images
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1025 <ul> <li>Previously discussed</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1026 <li>Awkward, but sometimes good for really large amounts of data.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1027 <li>Loopback mounting them on the host requires root access.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1028 <ul> <li>Can get data in without root access, harder to get it out.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1029 <li><code>genext2fs</code> doesn't come with a corresponding extract</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1030 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1031 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1032 <li>Virtual Network
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1033 <ul> <li>The most powerful/flexible mechanism. This is the one to focus on.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1034 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1035 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1036 <li>Understanding the QEMU "user" network.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1037 <ul> <li>The QEMU docs at <a href="http://www.qemu.org/qemu-doc.html">http://www.qemu.org/qemu-doc.html</a> describe this.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1038 <li>QEMU defaults to providing a virtual LAN behind a virtual masquerading gateway, using the 10.0.2.x address range.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1039 <ul> <li>If you supply no <code>-net</code> options to QEMU, it defaults to "<code>-net nic,model=e1000 -net user</code>"</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1040 <li>The first option plugs a virtual ethernet card into the virtual PCI bus, which becomes eth0.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1041 <ul> <li>Used to be ne2k_pci, switched to e1000 in 0.11.0 release.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1042 <li>The second option connects the most recent network card to the virtual LAN.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1043 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1044 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1045 <li>The virtual LAN provides:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1046 <ul> <li>A virtual masquerading gateway at 10.0.2.1. Works like a standard home router (ala Linksys, D-Link, etc): you can dial out but nobody can dial in.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1047 <li>A tunnel from 10.0.2.2 to the host's 127.0.0.1 (loopback interface)
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1048 <ul> <li>An emulated Linux system has its own loopback interface, so if you want to connect to services running on the host's loopback you need to use this alias for it.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1049 <li>This comes in handy all the time.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1050 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1051 <li>A virtual DNS server on 10.0.2.3.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1052 <ul> <li>Receives DNS requests and resolves them using the host's <code>libresolv.so</code> and <code>/etc/resolv.conf</code>.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1053 <ul> <li>Any cacheing is the host's responsibility</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1054 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1055 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1056 <li>A virtual DHCP server.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1057 <ul> <li>This responds to any DHCP requests by assigning it the address 10.0.2.15, with the above virtual gateway and DNS server addresses.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1058 <li>Since these are constant values, you might as well just assign them statically with <code>ifconfig</code>, <code>route</code>, and <code>/etc/resolv.conf</code>.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1059 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1060 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1061 <li>This is really useful, it means the emulated target system has a net connection by default.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1062 <ul> <li>And it can use 127.0.0.1 to dial out to the host and talk to private local network serves, such as ssh, web servers, samba...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1063 <li>But it's masqueraded. The host can't dial in.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1064 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1065 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1066 <li>The easy way to let the host dial in is port forwarding.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1067 <ul> <li>Again, just like a home router.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1068 <li>Use the "<code>-redir</code>" option
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1069 <ul> <li><strong><code>qemu -redir tcp:127.0.0.1:9876::1234</code></strong></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1070 <li>This means any attempt to connect to port 9876 on the host's loopback interface will be forwarded to port 1234 on the target's 10.0.2.15.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1071 <li>The :: is intentional, you could put 10.0.2.15 there if you wanted to, but that's the default value.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1072 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1073 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1074 <li>Using the virtual network to get data in/out of QEMU
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1075 <ul> <li>busybox contains <code>wget</code>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1076 <ul> <li>Helps to run a web server on the host's loopback interface.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1077 <li>busybox also contains one of those, no reason you can't build it onthe host. If you run it on a port &gt; 1024, it doesn't require root access.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1078 <li>There's always Apache, if you're up for configuring it</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1079 <li>A public web server or department server works too</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1080 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1081 <li>The <code>netcat</code>/tarball trick
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1082 <ul> <li>Build host and target versions of a netcat with a "server" mode (such as the one in toybox), and you can do tricks like the following shell script snippet:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1083 <pre><code># Remember the first line of couple dozen spaces; Linux kernel serial
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1084 # port initialization eats about a FIFO buffer's worth of input.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1085 ./run-qemu.sh << EOF
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1086 [ -z "$CPUS" ] && CPUS=1
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1087 rm -rf /home/temp
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1088 mkdir -p /home/temp
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1089 cd /home/temp || exit 1
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1090 # Copy source code from host to target system
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1091 netcat 10.0.2.2 $(netcat -s 127.0.0.1 -l tar c sources/trimconfig-busybox \
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1092 build/sources/{busybox,dropbear,strace}) | tar xv || exit 1
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1093 mv sources/* build/sources/* .
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1094 EOF</code></pre></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1095 <li>Note that <code>$(netcat -l blah blah blah)</code> must both launch the netcat server in the background and resolve to the port number on which the server is listening for an incoming connection for this to work. The "toybox" version of netcat does this.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1096 <li>You can have multiple "netcat dialing out to netcat" pairs and they'll trigger in sequence. The host version waits until the target makes a connection before running the appropriate command with stdin/stdout/stderr redirected to the network socket.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1097 <ul> <li>This does not require <code>-redir</code>, becaue the target is always dialing out to the host.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1098 <li>Might need "<code>killall netcat</code>" to clean up if the target exits early and doesn't consume all the waiting netcat servers.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1099 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1100 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1101 <li>Build/install <code>dropbear</code> inside the emulator
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1102 <ul> <li>A very nice self-contained embedded version of ssh/sshd.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1103 <ul> <li>Both programs in a 100k binary</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1104 <li>Does the busybox swiss-army-knife trick of behaving differently based on what name you used to run it</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1105 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1106 <li>Run <code>sshd</code> on host's loopback, then scp tarballs to/from 10.0.2.2.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1107 <ul> <li>Or pipe tarball through ssh, ala:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1108 <ul> <li><strong><code>tar cz dirname | ssh USER@HOSTNAME tar xvzC destination</code></strong></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1109 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1110 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1111 <li>Unlike Apache, sshd requires little configuration. Just install/run.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1112 <ul> <li>Can also be run without root access if you do it on a nonstandard port and explicitly tell it where to find its config files.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1113 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1114 <li>Install dropbear's sshd inside emulator and <code>-redir</code> a host loopback port to emulated system's port 22.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1115 <ul> <li>You can tunnel rsync over this using the "<code>-e</code>" option:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1116 <ul> <li><strong><code>rsync --delete -avHzP -e "ssh -C -p 9876" source/. 127.0.0.1:dest/.</code></strong></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1117 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1118 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1119 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1120 <li>Combining rsync with out-of-tree builds allows easy debugging
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1121 <ul> <li>Incremental rebuilds. (Assuming the project's make dependencies work.)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1122 <li>If your project's build doesn't already support out of tree builds, try using "<code>cp -sfR srcdir builddir</code>" to populate a directory of symlinks.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1123 <ul> <li>To trim dead symlinks after rsyncing the package source and potentially deleting files, "<code>find -follow -type l</code>", and pipe any it finds to "<code>xargs rm</code>".</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1124 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1125 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1126 <li>This technique also mitigates most of the problems with network filesystems
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1127 <ul> <li>The symlinks are case sensitive, even if the filesystem they point to isn't</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1128 <li>The local filesystem the build writes new files to has persistent timestamps with good granularity</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1129 <li>Builds almost never modify files in their distro tarballs, and deleting any would remove the symlink</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1130 <li>So you can export your working source via a network filesystem, but build in a temporary directory of symlinks on an ext3 image.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1131 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1132 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1133 <li>As always when working in multiple contexts, keep track of where your master copy is.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1134 <li>Personally, we always treat the contents of the emulators as temporary, and have "master" directories on the host which we copy into the emulators.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1135 <li>That way we can always garbage collect leftover images without worrying about losing too much.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1136 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1137
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
1138 <a name="debugging" /><h2>Debugging</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1139
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1140 <ul> <li><code>gdbserver</code> (part of the <code>gdb</code> package) allows you to debug things remotely.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1141 <ul> <li>Runs on the target to trace an executable</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1142 <li>Lets you use <code>gdb</code> without the overhead of running <code>gdb</code> itself on the target, which isn't always even possible (it's a big piece of GNU bloatware).</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1143 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1144 <li><code>gdbserver</code> runs on the target to trace an executable, talks to <code>gdb</code> via a serial protocol.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1145 <ul> <li>Serial protocols tunnel well over ssh or netcat
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1146 <ul> <li>Run QEMU to <code>-redirect</code> a port (such as 9876) to the host loopback</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1147 <li>Inside QEMU: <strong><code>gdbserver 10.0.2.15:9876 FILENAME</code></strong></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1148 <li>Run gdb on host using the "pipe to process" option:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1149 <ul> <li><code>file FILENAME</code></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1150 <li><code>target remote | netcat 127.0.0.1 9876</code></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1151 <li><code>set height 0</code></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1152 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1153 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1154 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1155 <li>Note that QEMU has a "<code>-s</code>" option that lets you attach gdb to the emulated <strong>hardware</strong>, the way a JTAG would
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1156 <ul> <li>Uses the same gdbserver protocol.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1157 <li>This debugs the OS, not the applications.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1158 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1159 <li>Either way requires a host version of <code>gdb</code> that understands your target's instruction set.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1160 <ul> <li>Most distro versions configured host-only.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1161 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1162 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1163
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
1164 <a name="package_maintainers" /><h2>Interacting with upstream package maintainers</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1165
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1166 <ul> <li>You have a bug you would like fixed
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1167 <ul> <li>Offering them a test environment in which they can reproduce the problem.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1168 <li>Offering a development environment in which they can test their fixes.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1169 <li>Self-contained QEMU development environment images small enough to download, running locally without root access are good for both.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1170 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1171 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1172
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
1173 <a name="performance" /><h2>Performance considerations</h2>
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1174
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1175 <ul> <li>Building under emulation is going to be slower than cross compiling. Accept that up front.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1176 <ul> <li>But how bad is it, and how can we make it suck less?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1177 </ul></li>
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
1178 <li><a name="benchmarks" />Some numbers we had lying around
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1179 <ul> <li>Using a cheapish early 2007 laptop and comparing the native build of "make 3.81" versus building it under the now ancient QEMU 9.0.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1180 <li>Native Build
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1181 <ul> <li>Extract tarball: Just over 1 second</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1182 <li><code>./configure</code>: 15 seconds</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1183 <li>Run make (<code>-j 2</code>): 4 seconds</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1184 <li>Total: 20 seconds.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1185 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1186 <li>Native build under (now ancient) QEMU 9.0 armv4l emulation
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1187 <ul> <li>Extract tarball: 5 seconds</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1188 <li><code>./configure</code>: 2 minutes and 40 seconds</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1189 <li>Run make: 2 minutes and 50 seconds</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1190 <li>Total: 335 seconds</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1191 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1192 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1193 <li>That's a big difference
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1194 <ul> <li>20%, 10%, and 2%</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1195 <li>Respectively totaling 6% of native performance.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1196 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1197 <li>How do we speed it up?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1198 <ul> <li>Problems
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1199 <ul> <li>Translated code runs somewhat slower than native code
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1200 <ul> <li>Looks like around 20% of native speed in this case</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1201 <li>Varies per target and per QEMU version</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1202 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1203 <li>Launching lots of short-lived programs is slow
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1204 <ul> <li>Translating the code takes time</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1205 <li>Essentially a latency spike in faulting in pages</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1206 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1207 <li>Host is SMP, QEMU isn't</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1208 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1209 <li>Options
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1210 <ul> <li>Wait for Moore's Law to improve hardware
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1211 <ul> <li>6% of native performance would be 4 doublings, I.E. 6 years of Moore's Law</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1212 <li>QEMU developers also improving the software</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1213 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1214 <li>Make it scale to SMP or clustering so we can throw money at the problem</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1215 <li>Be clever so it's faster even on a cheap laptop
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1216 <ul> <li>Let's do this one.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1217 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1218 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1219 </ul></li>
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
1220 <li><a name="distcc" />The <code>distcc</code> acceleration trick
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1221 <ul> <li>Use distcc to call out to the cross compiler through the virtual network.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1222 <ul> <li>Hybrid approach between native and emulated build</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1223 <li>We want to leverage the speed of the cross compiler without re-introducing the endless whack-a-mole of cross compiling</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1224 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1225 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1226 <li><code>distcc</code> is a compile wrapper</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1227 <li>When told to compile <code>.c</code> files to <code>.o</code> files
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1228 <ul> <li>calls the preprocessor on each <code>.c</code> file (<code>gcc -E</code>) to resolve the includes</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1229 <li>Sends the preprocessed <code>.c</code> file through the network to a server
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1230 <ul> <li>Server compiles preprocessed <code>.c</code> code into <code>.o</code> object code</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1231 <li>Sends the <code>.o</code> file back when done</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1232 <li>Copies <code>.o</code> file back through network to local filesystem</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1233 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1234 <li>When told to produce an executable
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1235 <ul> <li>calls the local compiler to link the <code>.o</code> files together</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1236 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1237 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1238 <li>How do we use it?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1239 <ul> <li>Run <code>distccd</code> on the host's loopback interface, with a <code>$PATH</code> pointing to cross compiler executables</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1240 <li>Install <code>distcc</code> in the target, with "<code>gcc</code>" and "<code>cc</code>" symlinks to the distcc wrapper at the start of <code>$PATH</code>, configured with <code>DISTCC_HOSTS=10.0.2.2</code>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1241 <ul> <li>And <code>g++</code>/<code>c++</code> symlinks for C++ support</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1242 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1243 <li>Run build as normal</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1244 <li>This moves the heavy lifting of compilation outside the emulator (where CPU is expensive) into the native context (where CPU is cheap)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1245 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1246 <li>Why doesn't this reintroduce all the problems of cross compiling.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1247 <ul> <li>No "two contexts" problem.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1248 <li>Header resolution and library linking done locally inside the emulator. Only one of each set of files.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1249 <li><code>make</code> runs locally</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1250 <li><code>./configure</code> runs locally
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1251 <ul> <li>Still asking questions the local (target) system can answer.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1252 <li>Able to run the code it builds.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1253 <li>All the cross compiler does is turn preprocessed <code>.c</code> code into object files.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1254 <li>That's pretty much the one thing cross compiling can't screw up. (If it couldn't do that, we'd never have gotten this far.)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1255 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1256 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1257 <li>The build doesn't have to be cross-compile aware, or configured for cross compiling
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1258 <ul> <li>No <code>CROSS_COMPILER=prefix-</code>, no <code>$HOSTCC</code></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1259 <li>As far as the build is concerned, it's building fully natively</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1260 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1261 <li>FWL sets this up for you automatically. To do it by hand:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1262 <ul> <li>on host<br>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1263 <strong><code>PATH=/path/to/cross-compiler-binaries /usr/bin/distccd --listen 127.0.0.1 --log-stderr --daemon -a 127.0.0.1 --no-detach --jobs $CPUS</code></strong>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1264 </li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1265 <li>on target
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1266 <pre><code>mkdir ~/distcc
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1267 ln -s /usr/bin/distcc ~/distcc/gcc
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1268 ln -s /usr/bin/distcc ~/distcc/cc
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1269 cd ~/firmware
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1270 export PATH=~/distcc:$PATH
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1271 export DISTCC_HOSTS=10.0.2.2/$CPUS
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1272 ./cross-compiler.sh armv4l</code></pre></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1273 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1274 <li>Using <code>distcc</code> to call out to the cross compiler turned the above 2:50 down to 24 seconds.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1275 <ul> <li>Factor of 7 speedup</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1276 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1277 <li>The above numbers are old, almost certainly no longer accurate, and may actually be tolerable as-is. But they can still be improved upon.</li>
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
1278 <li><a name="low_hanging_fruit" />Further speed improvements
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1279 <ul> <li>Low hanging fruit
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1280 <ul> <li>Installing <code>distcc</code> was cheap and easy
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1281 <ul> <li>They'd already written it, and it did exactly what we needed.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1282 <li>Any other low hanging fruit?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1283 <ul> <li>Throw hardware at the problem
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1284 <ul> <li>More parallelism is the way of the future
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1285 <ul> <li>Grab more computers to run <code>distccd</code> on (clustering).
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1286 <ul> <li>This is what <code>distcc</code> was originally designed for anyway</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1287 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1288 <li>Build a big SMP server, increase <code>$CPUS</code>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1289 <ul> <li>Even laptops are heading to 4-way, Moore's law isn't stopping.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1290 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1291 <li>We'll give examples later.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1292 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1293 <li>Problem: running <code>make</code> and preprocessing quickly become a bottleneck.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1294 <ul> <li>Varies per package, but an 8-way server might be a reasonable investment for building the Linux kernel.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1295 <ul> <li>32-way, not so much.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1296 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1297 <li>QEMU's built-in SMP is useless
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1298 <ul> <li>Fakes multiple CPUS using one host processor</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1299 <li>Doing proper SMP in QEMU would involve making QEMU threaded on the host.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1300 <ul> <li>Also emulating inter-processor communication, locking, cache page ownership...</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1301 <li>Generally considered a nightmare by the QEMU developers.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1302 <li>Don't hold your breath.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1303 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1304 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1305 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1306 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1307 <li>Tweak emulator
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1308 <ul> <li>Faster network
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1309 <ul> <li>The above benchmarks were done with a virtual 100baseT card (<code>rtl8139</code>) that maxed out at around 3 megabytes/second data transfer on the test system, and became a bottleneck.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1310 <li>The preprocessed <code>.c</code> files are very large, and must be sent both ways across the virtual network.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1311 <li>Modern default on x86 is gigabit ethernet (<code>e1000</code>) (as of QEMU 0.11.0). More efficient (faster even in emulator).
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1312 <ul> <li>Try to configure your target to use this if possible.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1313 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1314 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1315 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1316 <li>Re-nice the QEMU process to -20 and <code>distccd</code> to 20
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1317 <ul> <li>If QEMU is the bottleneck, make sure it gets dibs on CPU time</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1318 <li><code>distccd</code> nices its children down slightly by default</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1319 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1320 <li>If you're actually <strong>developing</strong> packages, copy your new source snapshots in via rsync and built out-of-tree.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1321 <ul> <li>Actually use <code>make</code>'s dependencies for once to avoid rebuilding code that didn't change.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1322 <li>Don't re-run <code>./configure</code> when you don't need it.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1323 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1324 <li>Strategic use of static linking?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1325 <ul> <li>The OS can cache translated pages if it keeps the file MMAPed.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1326 <ul> <li>It tries to do this to optimize the host.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1327 <li>QEMU's translation overhead exacerbates normal disk fetch and cache fetch behavior patterns. Existing optimization techniques already designed to mitigate this.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1328 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1329 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1330 <li>Dynamically linked pages count as self-modifying code, forcing QEMU to retranslate the page. Also OS may discard and re-loads the page in response to memory pressure.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1331 <ul> <li>Give emulator at least 256 megs of RAM.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1332 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1333 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1334 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1335 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1336 <li>Ok, so what's <strong>not</strong> low hanging fruit?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1337 <ul> <li>Two main approaches:
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1338 <ul> <li>Identify bottlenecks and make them less dominant.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1339 <li>Improving scalability, more parallelism</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1340 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1341 <li><code>distcc</code> only addressed the "<code>make</code>" portion of <code>configure</code>/<code>make</code>/<code>install</code>.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1342 <ul> <li><code>./configure</code> quickly comes to dominate
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1343 <ul> <li>Make already spent most of its time in <code>./configure</code> even on the host</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1344 <li>This is becoming true in general
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1345 <ul> <li><code>autoconf</code> doesn't easily parallelize</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1346 <li>Trying causes it to do extra work.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1347 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1348 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1349 <li>Exec-ing 8 gazillion dynamically linked executables has horrible cache behavior, even outside an emulator.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1350 <ul> <li>So this is going to stay slow. What do we do about it?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1351 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1352 </ul></li>
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
1353 <li><a name="future" />Parallelize at the package level.
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1354 <ul> <li>Fire up multiple instances of QEMU in parallel and have each one build a separate package.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1355 <li>This quickly becomes a package management issue.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1356 <ul> <li>Dealing with prerequisites involves installing and distributing prebuilt binary packages to the various QEMU nodes.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1357 <ul> <li>One "master node" drives builds, other QEMU nodes build a package at a time each, and then <code>distcc</code> daemons compile.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1358 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1359 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1360 <li>See "accidental distros" above.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1361 <ul> <li>Not easy to do this and stay orthogonal, but we can at least avoid reinventing the wheel.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1362 <li>We decided to leverage Gentoo's "portage" package management system for this.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1363 <ul> <li>It's got a big emphasis on building from source already.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1364 <li>Sketched out a clustering build extension to the Gentoo From Scratch project.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1365 <ul> <li>Nobody's stepped forward to fund it yet.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1366 <li>Advancing at hobbyist rate. Check back.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1367 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1368 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1369 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1370 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1371 <li>It's tempting to cache <code>./configure</code> output (save <code>config.cache</code>)
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1372 <ul> <li>But the cure's worse than the disease.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1373 <li>Just cache the generated binary packages</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1374 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1375 <li>Autoconf was the bane of cross compiling, and it's the bane of native compiling too. Just less so.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1376 <ul> <li>Rant about <code>configure</code> and <code>make</code> becoming obsolete
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1377 <ul> <li>Note that open source never really uses <code>make</code>'s dependency generation.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1378 <ul> <li>"<code>make all</code>" is the norm, anything else is debugging.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1379 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1380 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1381 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1382 <li>Use a faster preprocessor (<code>gcc -E</code> or <code>cpp stage</code>)
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1383 <ul> <li>The <code>tinycc</code> compiler has a very fast preprocessor which could probably be turned into a drop-in replacement for <code>gcc</code>'s with a little effort.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1384 <ul> <li>Unfortunately, tinycc project more or less moribund.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1385 <li>Last serious progress in mainstream was before Fabrice Bellard left, around 2005.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1386 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1387 <li>Supports a very limited range of targets, need target- specific default preprocessor symbols. (run "<code>gcc -dM -E - &lt; /dev/null</code>" to see the full list)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1388 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1389 <li>Add <code>ccache</code> package on top of <code>distcc</code> to cache preprocessor output?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1390 <ul> <li>Is it worth it?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1391 <li>Launching another layer of executable more expensive inside QEMU, need to benchmark.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1392 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1393 <li>Better wrapper
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1394 <ul> <li>Installing <code>distcc</code> was easy, but <code>distcc</code> isn't perfect.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1395 <ul> <li>It only understands some gcc command lines, and when it can't parse it conservatively falls back to the native compiler, even when there's work to distribute.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1396 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1397 <li><code>ccwrap</code> already has to parse <code>gcc</code> command lines more deeply than <code>distcc</code> does, to rewrite them for uClibc and relocation.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1398 <li>Possibly we could add <code>distcc</code> functionality to <code>ccwrap</code>?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1399 <ul> <li>Use existing daemons, or write our own?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1400 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1401 <li>Alternatively, upgrade <code>distcc</code> itself, but see next point</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1402 <li>Fewer layers of wrapper
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1403 <ul> <li><code>exec</code> is expensive due to translation overhead
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1404 <ul> <li>Remember the "slowing 20% to 2%" example, above.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1405 <li>Currently up to four layers: <code>distcc</code>&rarr;<code>ccwrap</code>&rarr;<code>gcc</code>&rarr;<code>cc1</code></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1406 <li>This is why adding <code>ccache</code> may not be a win.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1407 <ul> <li>Still need to bench it anyway.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1408 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1409 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1410 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1411 <li><code>ccwrap</code> is small and lightweight, only a page or two needs to be faulted in. Trying to fix "<code>gcc</code>" to eliminate it not a big win.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1412 <ul> <li>eliminating <code>gcc</code>'s own wrapper instead, and teaching <code>cpp</code>/<code>cc1</code>/<code>ld</code> directly might be a bigger win.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1413 <ul> <li>It does more work to accomplish less</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1414 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1415 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1416 <li>Teaching <code>ccwrap</code> to do <code>distcc</code> might also be a win for this reason.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1417 <ul> <li>Teaching <code>ccwrap</code> to do <code>ccache</code>?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1418 <li>Complexity vs benefit.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1419 <ul> <li>Is this wheel improvable enough to be worth reinventing?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1420 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1421 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1422 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1423 <li>Sometimes gzipping data before/after sending it is faster than just sending it. (gzip is really cheap.) Need to benchmark this.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1424 <ul> <li>Helps "speeding up network" bottleneck.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1425 <li>pre-gzipped <code>ccache</code> data could be sent across the network without being re-compressed. THAT might be worth doing.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1426 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1427 <li>Persistent processes to avoid retranslation overhead?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1428 <ul> <li>Leave a process running and pipe several files through it?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1429 <li>Strategic use of static linking?</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1430 <li>The OS can cache translated pages if it keeps the file MMAPed.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1431 <ul> <li>It tries to do this to optimize the host.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1432 <li>QEMU's translation overhead exacerbates normal disk fetch and cache fetch behavior patterns. Existing optimization techniques already designed to mitigate this.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1433 <li>Dynamically linked pages count as self-modifying code, forcing QEMU to retranslate the page. Also OS may discard and re-loads the page in response to memory pressure.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1434 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1435 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1436 <li>Feed data in larger chunks?
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1437 <ul> <li>Lots of projects go "<code>gcc one.c two.c three.c -c -o out.o</code>"</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1438 <li>distcc breaks 'em up into individual files, passes to separate daemons.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1439 <li>This has several downsides
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1440 <ul> <li>Launches more individual processes</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1441 <li>Gives the optimizer less to work with.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1442 <li><code>distcc</code> is looking to maximize parallelism, but moving compilation outside of the emulator is our biggest win.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1443 <ul> <li>Our communications overhead is higher than normal.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1444 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1445 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1446 </ul></li>
894
343fb4f8ffda Add index to HTML version of Ohio LinuxFest presentation.
Rob Landley <rob@landley.net>
parents: 892
diff changeset
1447 <li><a name="hardware" />Arranging for a fast host to run the emulator on.
892
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1448 <ul> <li>Having a fast context in which to run builds doesn't prevent you from <strong>also</strong> running builds on your local laptop.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1449 <ul> <li>Something to ssh to when a local build would be cumbersome.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1450 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1451 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1452 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1453 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1454 <li>Building a fire-breathing server<
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1455 <ul> <li>Current sweet spot seems to be about 8-way server.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1456 <ul> <li>Both in terms of price/performance and in terms of build scalability.</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1457 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1458 <li>We got a Dell server for ~3K in early 2009.
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1459 <ul> <li>Tower Configuration (Can rackmount as 5U)
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1460 <ul> <li>8x 2.5Ghz (Xeon E5420)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1461 <li>32GB RAM (<code>mount -t tmpfs</code> and build in that)</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1462 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1463 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1464 <li>Amazon Cloud
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1465 <ul> <li>Can rent an 8x server instance with 7GB of RAM for $.80/hour</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1466 <li>Latency is an issue</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1467 <li>Requires setup and teardown of instances</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1468 <li>Potentially highly scalable though</li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1469 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1470 </ul></li>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1471 </ul>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1472
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1473 </body>
a3ab17ecff3f Ben Zanin converted the Ohio LinuxFest presentation slides to html.
Rob Landley <rob@landley.net>
parents:
diff changeset
1474 </html>