Mercurial > hg > aboriginal

annotate more/buildall.sh @ 1391:42ec661e3916

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fix BINARY_PACKAGE_TARBALLS generation: some packages produce no output (zlib used to build dropbear for example), and thus check for an empty list before trying to make a tarball.
author Rob Landley <rob@landley.net>
date Fri, 17 Jun 2011 13:00:17 -0500
parents 7eabfa815c90
children 8d0567d46cd7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
610
a47844b73616 New simple buildall.sh that's a wrapper around build.sh. (Chop out half of build-all-targets, need to make a separate script to build static toolchains.)
Rob Landley <rob@landley.net>
parents:
diff changeset
1 #!/bin/bash
a47844b73616 New simple buildall.sh that's a wrapper around build.sh. (Chop out half of build-all-targets, need to make a separate script to build static toolchains.)
Rob Landley <rob@landley.net>
parents:
diff changeset
2
1182
c5ebf1e353c7 Apparently this never quite got updated for commit 1116.
Rob Landley <rob@landley.net>
parents: 1157
diff changeset
3 # Build every target architecture, saving log files to build/log.
610
a47844b73616 New simple buildall.sh that's a wrapper around build.sh. (Chop out half of build-all-targets, need to make a separate script to build static toolchains.)
Rob Landley <rob@landley.net>
parents:
diff changeset
4 # If $FORK is set, build them in parallel.
a47844b73616 New simple buildall.sh that's a wrapper around build.sh. (Chop out half of build-all-targets, need to make a separate script to build static toolchains.)
Rob Landley <rob@landley.net>
parents:
diff changeset
5
1273
dadbb8873a36 functions.sh no longer includes utility-functions.sh, so adjust includes.
Rob Landley <rob@landley.net>
parents: 1192
diff changeset
6 . sources/utility_functions.sh || exit 1
610
a47844b73616 New simple buildall.sh that's a wrapper around build.sh. (Chop out half of build-all-targets, need to make a separate script to build static toolchains.)
Rob Landley <rob@landley.net>
parents:
diff changeset
7
1350
7eabfa815c90 Split up system-image.sh into root-image.sh, linux-kernel.sh, and system-image.sh. Rename CROSS_HOST_ARCH to CROSS_COMPILER_HOST.
Rob Landley <rob@landley.net>
parents: 1344
diff changeset
8 [ -z "$CROSS_COMPILER_HOST" ] && export CROSS_COMPILER_HOST=i686
778
e373d2b6d0b8 Add killtree and make stuff use it. Update smoketest-all.sh to use doforklog, allowing FORK=1 to run stuff in parallel.
Rob Landley <rob@landley.net>
parents: 776
diff changeset
9
e373d2b6d0b8 Add killtree and make stuff use it. Update smoketest-all.sh to use doforklog, allowing FORK=1 to run stuff in parallel.
Rob Landley <rob@landley.net>
parents: 776
diff changeset
10 trap "killtree $$" EXIT
e373d2b6d0b8 Add killtree and make stuff use it. Update smoketest-all.sh to use doforklog, allowing FORK=1 to run stuff in parallel.
Rob Landley <rob@landley.net>
parents: 776
diff changeset
11
787
23bff7aa79eb Teach buildall.sh to use new build.sh infrastructure, and replace doforklog with orthogonal maybe_fork and maybe_quiet functions.
Rob Landley <rob@landley.net>
parents: 778
diff changeset
12 # Build the host architecture. This has to be built first so the other
23bff7aa79eb Teach buildall.sh to use new build.sh infrastructure, and replace doforklog with orthogonal maybe_fork and maybe_quiet functions.
Rob Landley <rob@landley.net>
parents: 778
diff changeset
13 # architectures can canadian cross static compilers to run on the host using
23bff7aa79eb Teach buildall.sh to use new build.sh infrastructure, and replace doforklog with orthogonal maybe_fork and maybe_quiet functions.
Rob Landley <rob@landley.net>
parents: 778
diff changeset
14 # this toolchain to link against a host version of uClibc.
23bff7aa79eb Teach buildall.sh to use new build.sh infrastructure, and replace doforklog with orthogonal maybe_fork and maybe_quiet functions.
Rob Landley <rob@landley.net>
parents: 778
diff changeset
15
23bff7aa79eb Teach buildall.sh to use new build.sh infrastructure, and replace doforklog with orthogonal maybe_fork and maybe_quiet functions.
Rob Landley <rob@landley.net>
parents: 778
diff changeset
16 # This also performs the download.sh and host-tools.sh steps, which don't
23bff7aa79eb Teach buildall.sh to use new build.sh infrastructure, and replace doforklog with orthogonal maybe_fork and maybe_quiet functions.
Rob Landley <rob@landley.net>
parents: 778
diff changeset
17 # parallelize well if many build.sh instances try to call them at once.
23bff7aa79eb Teach buildall.sh to use new build.sh infrastructure, and replace doforklog with orthogonal maybe_fork and maybe_quiet functions.
Rob Landley <rob@landley.net>
parents: 778
diff changeset
18
23bff7aa79eb Teach buildall.sh to use new build.sh infrastructure, and replace doforklog with orthogonal maybe_fork and maybe_quiet functions.
Rob Landley <rob@landley.net>
parents: 778
diff changeset
19 # If this fails, don't bother trying to build the other targets.
610
a47844b73616 New simple buildall.sh that's a wrapper around build.sh. (Chop out half of build-all-targets, need to make a separate script to build static toolchains.)
Rob Landley <rob@landley.net>
parents:
diff changeset
20
1344
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
21 if [ -z "$BUILD_NATIVE_ONLY" ]
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
22 then
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
23 [ -z "$NO_CLEAN" ] && blank_tempdir build
630
0e8a887f9d8b buildall.sh should create the README file in the output directory.
Rob Landley <rob@landley.net>
parents: 610
diff changeset
24
1344
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
25 mkdir -p build/logs &&
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
26 (EXTRACT_ALL=1 ./download.sh 2>&1 &&
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
27 ./host-tools.sh 2>&1 &&
1350
7eabfa815c90 Split up system-image.sh into root-image.sh, linux-kernel.sh, and system-image.sh. Rename CROSS_HOST_ARCH to CROSS_COMPILER_HOST.
Rob Landley <rob@landley.net>
parents: 1344
diff changeset
28 ./simple-cross-compiler.sh 2>&1 "$CROSS_COMPILER_HOST" ||
1344
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
29 dienow) | tee build/logs/build-host-cc.txt | maybe_quiet
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
30
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
31 cp packages/MANIFEST build || dienow
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
32
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
33 # Build all non-hw targets, possibly in parallel
885
06f80138cb90 Add MANIFEST file to buildall.sh output.
Rob Landley <rob@landley.net>
parents: 852
diff changeset
34
1344
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
35 more/for-each-target.sh \
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
36 './build.sh $TARGET 2>&1 | tee build/logs/build-${TARGET}.txt'
717
5a3ebc77340c Teach buildall.sh to canadian cross static cross compilers with --enable-shared and uClibc++ when you set "$CROSS_COMPILERS_EH".
Rob Landley <rob@landley.net>
parents: 669
diff changeset
37
1344
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
38 more/build-control-images.sh
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
39
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
40 # Run smoketest.sh for each non-hw target.
1114
be94bf38648b Break up buildall.sh so it's easier to run individual portions of it.
Rob Landley <rob@landley.net>
parents: 1103
diff changeset
41
1344
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
42 more/for-each-target.sh \
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
43 'more/smoketest.sh $TARGET 2>&1 | tee build/logs/smoketest-$TARGET.txt'
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
44
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
45 fi
1192
8d5a672cd293 build-control-images.sh got broken out a while ago, remove redundant code from buildall.
Rob Landley <rob@landley.net>
parents: 1182
diff changeset
46
1344
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
47 # Build the hdb images sequentially without timeout.sh, to avoid potential
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
48 # I/O storm triggering timeouts
1114
be94bf38648b Break up buildall.sh so it's easier to run individual portions of it.
Rob Landley <rob@landley.net>
parents: 1103
diff changeset
49
1344
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
50 FORK= more/for-each-target.sh \
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
51 '. sources/toys/make-hdb.sh; HDBMEGS=2048; HDB=build/system-image-$TARGET/hdb.img; echo "$HDB"; rm -f "$HDB"; make_hdb'
1114
be94bf38648b Break up buildall.sh so it's easier to run individual portions of it.
Rob Landley <rob@landley.net>
parents: 1103
diff changeset
52
be94bf38648b Break up buildall.sh so it's easier to run individual portions of it.
Rob Landley <rob@landley.net>
parents: 1103
diff changeset
53 # Build static-tools (dropbear and strace) for each target
744
759adf5a0fe9 Refactor so include.sh mostly just sets environment variables. Move read_arch_dir to function and call it explicitly (no more need for $NO_ARCH). Make blank_tempdir a function, called explicitly by stages when needed, with some sanity checks. Insert prerequisite tests to later stages so they can detect failure early and provide an explicit erro rmessage, and have those tests happen before blanking $WORK dir, to preserve debugging info. Make buildall.sh depend on prerequisite tests rather than trying to avoid calling later stages (and thus do flow control from asynchronous context). Add FAIL_QUIET option so buildall.sh doesn't spam the log with the new prerequisite error messages.
Rob Landley <rob@landley.net>
parents: 736
diff changeset
54
1192
8d5a672cd293 build-control-images.sh got broken out a while ago, remove redundant code from buildall.
Rob Landley <rob@landley.net>
parents: 1182
diff changeset
55 mkdir -p build/native-static || dienow
8d5a672cd293 build-control-images.sh got broken out a while ago, remove redundant code from buildall.
Rob Landley <rob@landley.net>
parents: 1182
diff changeset
56 more/for-each-target.sh \
8d5a672cd293 build-control-images.sh got broken out a while ago, remove redundant code from buildall.
Rob Landley <rob@landley.net>
parents: 1182
diff changeset
57 'ln -sf ../native-static build/system-image-$TARGET/upload'
8d5a672cd293 build-control-images.sh got broken out a while ago, remove redundant code from buildall.
Rob Landley <rob@landley.net>
parents: 1182
diff changeset
58
1157
300e6d919d86 Move "sources/more/" to just "more/", add a README explaining why those scripts are there, and adjust calls to them.
Rob Landley <rob@landley.net>
parents: 1124
diff changeset
59 more/for-each-target.sh \
1344
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
60 'more/timeout.sh 60 "HDB=hdb.img more/native-build-from-build.sh $TARGET build/control-images/static-tools.hdc | tee build/logs/native-$TARGET.txt"'
1192
8d5a672cd293 build-control-images.sh got broken out a while ago, remove redundant code from buildall.
Rob Landley <rob@landley.net>
parents: 1182
diff changeset
61
8d5a672cd293 build-control-images.sh got broken out a while ago, remove redundant code from buildall.
Rob Landley <rob@landley.net>
parents: 1182
diff changeset
62 # If using a test version of busybox, run busybox test suite.
8d5a672cd293 build-control-images.sh got broken out a while ago, remove redundant code from buildall.
Rob Landley <rob@landley.net>
parents: 1182
diff changeset
63
8d5a672cd293 build-control-images.sh got broken out a while ago, remove redundant code from buildall.
Rob Landley <rob@landley.net>
parents: 1182
diff changeset
64 is_in_list busybox "$USE_UNSTABLE" &&
8d5a672cd293 build-control-images.sh got broken out a while ago, remove redundant code from buildall.
Rob Landley <rob@landley.net>
parents: 1182
diff changeset
65 more/for-each-target.sh \
1344
6a04a89a0d19 Add BUILD_NATIVE_ONLY and NO_CLEAN controls to buildall.sh. Also break out the hdb.img creation and do that single threaded for all targets, then reuse the scratch drives. (Avoids triggering timeout.sh due to disk thrashing on large FORK=1 builds.)
Rob Landley <rob@landley.net>
parents: 1273
diff changeset
66 'more/timeout.sh 60 "HDB=hdb.img more/native-build-from-build.sh $TARGET build/control-images/busybox-test.hdc" | tee build/logs/busybox-test-$TARGET.txt'
974
51a8a0076c9d Oops. Wait for native build to finish before exiting.
Rob Landley <rob@landley.net>
parents: 972
diff changeset
67
852
6ecee01bffa6 Update paths and comments in cronjob.sh, rename "none" to "stable", minor cleanup to logging.
Rob Landley <rob@landley.net>
parents: 850
diff changeset
68 # Create a file containing simple pass/fail results for all architectures.
6ecee01bffa6 Update paths and comments in cronjob.sh, rename "none" to "stable", minor cleanup to logging.
Rob Landley <rob@landley.net>
parents: 850
diff changeset
69
1157
300e6d919d86 Move "sources/more/" to just "more/", add a README explaining why those scripts are there, and adjust calls to them.
Rob Landley <rob@landley.net>
parents: 1124
diff changeset
70 more/smoketest-all.sh --logs | tee build/logs/status.txt