Mercurial > hg > aboriginal

annotate sources/include.sh @ 1187:878dbfe76341

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Move $BUILD/logs creation from record-commands.sh to include.sh so it gets reliably recreated when a user blanks the logs and re-runs.
author Rob Landley <rob@landley.net>
date Sun, 01 Aug 2010 15:19:00 -0500
parents 886a2ea90bc1
children 4021fb1183d7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
840
96b276eda7f1 Comment tweaks.
Rob Landley <rob@landley.net>
parents: 798
diff changeset
1 #!/bin/echo "This file is sourced, not run"
96b276eda7f1 Comment tweaks.
Rob Landley <rob@landley.net>
parents: 798
diff changeset
2
96b276eda7f1 Comment tweaks.
Rob Landley <rob@landley.net>
parents: 798
diff changeset
3 # Set up all the environment variables and functions for a build stage.
96b276eda7f1 Comment tweaks.
Rob Landley <rob@landley.net>
parents: 798
diff changeset
4 # This file is sourced, not run.
138
7dffe9648a56 Installing bash but not using it. Yeah, there's a political agenda at work
Rob Landley <rob@landley.net>
parents: 125
diff changeset
5
897
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
6 # Include config and sources/functions.sh
351
7f529baf0b57 Reorganize include.sh to put config options at the top, minor cleanups.
Rob Landley <rob@landley.net>
parents: 340
diff changeset
7
399
4e11cfb255e2 Break config out from include.sh, so environment variables users can edit to affect the build are collected together in one place with nothing else in it.
Rob Landley <rob@landley.net>
parents: 398
diff changeset
8 [ -e config ] && source config
897
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
9
744
759adf5a0fe9 Refactor so include.sh mostly just sets environment variables. Move read_arch_dir to function and call it explicitly (no more need for $NO_ARCH). Make blank_tempdir a function, called explicitly by stages when needed, with some sanity checks. Insert prerequisite tests to later stages so they can detect failure early and provide an explicit erro rmessage, and have those tests happen before blanking $WORK dir, to preserve debugging info. Make buildall.sh depend on prerequisite tests rather than trying to avoid calling later stages (and thus do flow control from asynchronous context). Add FAIL_QUIET option so buildall.sh doesn't spam the log with the new prerequisite error messages.
Rob Landley <rob@landley.net>
parents: 743
diff changeset
10 source sources/functions.sh
398
a7daae215647 Implement unstable package download logic, with examples. (Try "USE_UNSTABLE=uClibc,linux ./download.sh" for example.)
Rob Landley <rob@landley.net>
parents: 396
diff changeset
11
997
491eec5512dd Slightly tweaked bugfix from Wolfgang Denk: clean out environment variables from the host that would screw up the build.
Rob Landley <rob@landley.net>
parents: 984
diff changeset
12 # Avoid trouble from unexpected environment settings
491eec5512dd Slightly tweaked bugfix from Wolfgang Denk: clean out environment variables from the host that would screw up the build.
Rob Landley <rob@landley.net>
parents: 984
diff changeset
13
1175
886a2ea90bc1 Add sanitize_environment to unset unrecognized environment variables.
Rob Landley <rob@landley.net>
parents: 1174
diff changeset
14 [ -z "$NO_SANITIZE_ENVIRONMENT" ] && sanitize_environment
997
491eec5512dd Slightly tweaked bugfix from Wolfgang Denk: clean out environment variables from the host that would screw up the build.
Rob Landley <rob@landley.net>
parents: 984
diff changeset
15
972
cfa6262528f3 Genericize native build.
Rob Landley <rob@landley.net>
parents: 910
diff changeset
16 # List of fallback mirrors to download package source from
cfa6262528f3 Genericize native build.
Rob Landley <rob@landley.net>
parents: 910
diff changeset
17
1077
910785c683d5 New zlib release, and fix fallback mirror since new domain names aren't quite working yet.
Rob Landley <rob@landley.net>
parents: 1051
diff changeset
18 MIRROR_LIST="http://aboriginal.impactlinux.com/mirror http://landley.net/code/aboriginal/mirror http://127.0.0.1/code/aboriginal/mirror"
972
cfa6262528f3 Genericize native build.
Rob Landley <rob@landley.net>
parents: 910
diff changeset
19
897
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
20 # Where are our working directories?
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
21
984
9840847885e8 Add export_if_blank and make lots of build paths overrideable.
Rob Landley <rob@landley.net>
parents: 973
diff changeset
22 export_if_blank TOP=`pwd`
9840847885e8 Add export_if_blank and make lots of build paths overrideable.
Rob Landley <rob@landley.net>
parents: 973
diff changeset
23 export_if_blank SOURCES="$TOP/sources"
9840847885e8 Add export_if_blank and make lots of build paths overrideable.
Rob Landley <rob@landley.net>
parents: 973
diff changeset
24 export_if_blank SRCDIR="$TOP/packages"
9840847885e8 Add export_if_blank and make lots of build paths overrideable.
Rob Landley <rob@landley.net>
parents: 973
diff changeset
25 export_if_blank PATCHDIR="$SOURCES/patches"
9840847885e8 Add export_if_blank and make lots of build paths overrideable.
Rob Landley <rob@landley.net>
parents: 973
diff changeset
26 export_if_blank BUILD="$TOP/build"
9840847885e8 Add export_if_blank and make lots of build paths overrideable.
Rob Landley <rob@landley.net>
parents: 973
diff changeset
27 export_if_blank SRCTREE="$BUILD/packages"
9840847885e8 Add export_if_blank and make lots of build paths overrideable.
Rob Landley <rob@landley.net>
parents: 973
diff changeset
28 export_if_blank HOSTTOOLS="$BUILD/host"
9840847885e8 Add export_if_blank and make lots of build paths overrideable.
Rob Landley <rob@landley.net>
parents: 973
diff changeset
29 export_if_blank WRAPDIR="$BUILD/wrapdir"
897
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
30
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
31 # Set a default non-arch
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
32
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
33 export WORK="${BUILD}/host-temp"
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
34 export ARCH_NAME=host
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
35
351
7f529baf0b57 Reorganize include.sh to put config options at the top, minor cleanups.
Rob Landley <rob@landley.net>
parents: 340
diff changeset
36 # What host compiler should we use?
7f529baf0b57 Reorganize include.sh to put config options at the top, minor cleanups.
Rob Landley <rob@landley.net>
parents: 340
diff changeset
37
984
9840847885e8 Add export_if_blank and make lots of build paths overrideable.
Rob Landley <rob@landley.net>
parents: 973
diff changeset
38 export_if_blank CC=cc
351
7f529baf0b57 Reorganize include.sh to put config options at the top, minor cleanups.
Rob Landley <rob@landley.net>
parents: 340
diff changeset
39
7f529baf0b57 Reorganize include.sh to put config options at the top, minor cleanups.
Rob Landley <rob@landley.net>
parents: 340
diff changeset
40 # How many processors should make -j use?
7f529baf0b57 Reorganize include.sh to put config options at the top, minor cleanups.
Rob Landley <rob@landley.net>
parents: 340
diff changeset
41
900
266dc7ea04c2 If there's enough memory (half a gig per processor), use 1.5x as many CPUs to try to keep 'em busy.
Rob Landley <rob@landley.net>
parents: 897
diff changeset
42 MEMTOTAL="$(awk '/MemTotal:/{print $2}' /proc/meminfo)"
351
7f529baf0b57 Reorganize include.sh to put config options at the top, minor cleanups.
Rob Landley <rob@landley.net>
parents: 340
diff changeset
43 if [ -z "$CPUS" ]
7f529baf0b57 Reorganize include.sh to put config options at the top, minor cleanups.
Rob Landley <rob@landley.net>
parents: 340
diff changeset
44 then
553
80f3356577fc Move kernel building from mini-native to package-mini-native in preparation for initramfs packaging option.
Rob Landley <rob@landley.net>
parents: 538
diff changeset
45 export CPUS=$(echo /sys/devices/system/cpu/cpu[0-9]* | wc -w)
351
7f529baf0b57 Reorganize include.sh to put config options at the top, minor cleanups.
Rob Landley <rob@landley.net>
parents: 340
diff changeset
46 [ "$CPUS" -lt 1 ] && CPUS=1
900
266dc7ea04c2 If there's enough memory (half a gig per processor), use 1.5x as many CPUs to try to keep 'em busy.
Rob Landley <rob@landley.net>
parents: 897
diff changeset
47
266dc7ea04c2 If there's enough memory (half a gig per processor), use 1.5x as many CPUs to try to keep 'em busy.
Rob Landley <rob@landley.net>
parents: 897
diff changeset
48 # If there's enough memory, try to make CPUs stay busy.
266dc7ea04c2 If there's enough memory (half a gig per processor), use 1.5x as many CPUs to try to keep 'em busy.
Rob Landley <rob@landley.net>
parents: 897
diff changeset
49
266dc7ea04c2 If there's enough memory (half a gig per processor), use 1.5x as many CPUs to try to keep 'em busy.
Rob Landley <rob@landley.net>
parents: 897
diff changeset
50 [ $(($CPUS*512*1024)) -le $MEMTOTAL ] && CPUS=$((($CPUS*3)/2))
351
7f529baf0b57 Reorganize include.sh to put config options at the top, minor cleanups.
Rob Landley <rob@landley.net>
parents: 340
diff changeset
51 fi
7f529baf0b57 Reorganize include.sh to put config options at the top, minor cleanups.
Rob Landley <rob@landley.net>
parents: 340
diff changeset
52
984
9840847885e8 Add export_if_blank and make lots of build paths overrideable.
Rob Landley <rob@landley.net>
parents: 973
diff changeset
53 export_if_blank STAGE_NAME=`echo $0 | sed 's@.*/\(.*\)\.sh@\1@'`
897
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
54 [ ! -z "$BUILD_VERBOSE" ] && VERBOSITY="V=1"
615
7523d1f3b818 Some slight paranoia against multiple host-tools.sh runs that get interrupted in the middle. (Yeah, in general that's not guaranteed to work, but still...)
Rob Landley <rob@landley.net>
parents: 608
diff changeset
55
1174
5061606337d0 Add "make" to the default static list, for native-compiler tarball portability. (If you haven't got make working, it's hard to build it.)
Rob Landley <rob@landley.net>
parents: 1083
diff changeset
56 export_if_blank BUILD_STATIC=busybox,binutils,gcc-core,gcc-g++,make
1083
cb4dbdb7f2cd Make BUILD_STATIC take comma separated list of packages, or "all" or "none". Default behavior should remain the same.
Rob Landley <rob@landley.net>
parents: 1077
diff changeset
57
352
1782b77fae15 Add command logging. Set RECORD_COMMANDS=1 to log every command line run
Rob Landley <rob@landley.net>
parents: 351
diff changeset
58 # Adjust $PATH
1782b77fae15 Add command logging. Set RECORD_COMMANDS=1 to log every command line run
Rob Landley <rob@landley.net>
parents: 351
diff changeset
59
897
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
60 export OLDPATH="$PATH"
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
61 PATH="$(hosttools_path)"
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
62
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
63 # If record-commands.sh set up a wrapper directory, adjust $PATH again.
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
64 if [ -f "$WRAPDIR/wrappy" ]
352
1782b77fae15 Add command logging. Set RECORD_COMMANDS=1 to log every command line run
Rob Landley <rob@landley.net>
parents: 351
diff changeset
65 then
1187
878dbfe76341 Move $BUILD/logs creation from record-commands.sh to include.sh so it gets reliably recreated when a user blanks the logs and re-runs.
Rob Landley <rob@landley.net>
parents: 1175
diff changeset
66 mkdir -p "$BUILD/logs"
897
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
67 export WRAPPY_LOGPATH="$BUILD/logs/cmdlines.$ARCH_NAME.early"
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
68 OLDPATH="$PATH:$OLDPATH"
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
69 PATH="$WRAPDIR"
910
f1671488c740 Split busybox and toybox builds, update toybox and linux package versions.
Rob Landley <rob@landley.net>
parents: 900
diff changeset
70 elif [ ! -f "$HOSTTOOLS/busybox" ] || [ ! -f "$HOSTTOOLS/toybox" ]
897
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
71 then
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
72 PATH="$PATH:$OLDPATH"
352
1782b77fae15 Add command logging. Set RECORD_COMMANDS=1 to log every command line run
Rob Landley <rob@landley.net>
parents: 351
diff changeset
73 fi
1782b77fae15 Add command logging. Set RECORD_COMMANDS=1 to log every command line run
Rob Landley <rob@landley.net>
parents: 351
diff changeset
74
897
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
75 # Create files with known permissions
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
76 umask 022
608
ab76d4cdd392 Jean Wolter pointed out that running RECORD_COMMANDS=1 twice could create circular symlinks. Hopefully fixed now, and some related cleanups/clarifications.
Rob Landley <rob@landley.net>
parents: 597
diff changeset
77
897
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
78 # Tell bash not to cache the $PATH because we modify it. (Without this, bash
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
79 # won't find new executables added after startup.)
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
80 set +h
352
1782b77fae15 Add command logging. Set RECORD_COMMANDS=1 to log every command line run
Rob Landley <rob@landley.net>
parents: 351
diff changeset
81
897
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
82 # Disable internationalization so sort and sed and such can cope with ASCII.
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
83
850da666acc6 Redo command recording: no longer RECORD_COMMMANDS config entry, now sources/more/record-commands.sh run to set up the wrapper (either before or after host-tools.sh). General cleanup and simplification of code this interacted with, plus clean up bit rot in the reporting.
Rob Landley <rob@landley.net>
parents: 854
diff changeset
84 export LC_ALL=C