Mercurial > hg > aboriginal
annotate more/cronjob.sh @ 1442:1039cd16dc92
Make timeout duration configurable.
author | Rob Landley <rob@landley.net> |
---|---|
date | Mon, 12 Sep 2011 10:54:25 -0500 |
parents | 6ea45e5e39f5 |
children | ad5303b6d35a |
rev | line source |
---|---|
771
1fbe6d6a5a6b
Cron job script to build various nightly snapshots.
Rob Landley <rob@landley.net>
parents:
diff
changeset
|
1 #!/bin/bash |
1fbe6d6a5a6b
Cron job script to build various nightly snapshots.
Rob Landley <rob@landley.net>
parents:
diff
changeset
|
2 |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
3 # Build stable versions of all packages with current scripts. |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
4 |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
5 # This gets run in the aboriginal top directory. |
852
6ecee01bffa6
Update paths and comments in cronjob.sh, rename "none" to "stable", minor cleanup to logging.
Rob Landley <rob@landley.net>
parents:
851
diff
changeset
|
6 |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
7 pull_repo() |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
8 { |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
9 # Clone working copy |
852
6ecee01bffa6
Update paths and comments in cronjob.sh, rename "none" to "stable", minor cleanup to logging.
Rob Landley <rob@landley.net>
parents:
851
diff
changeset
|
10 |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
11 rm -rf "packages/alt-$1-0.tar.bz2" build/packages/alt-$1 |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
12 mkdir -p build/packages/alt-$1 |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
13 pushd build/packages/alt-$1 && |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
14 ln -s ../../../repos/$1/.git .git && |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
15 git checkout -f master && |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
16 git pull |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
17 popd |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
18 } |
852
6ecee01bffa6
Update paths and comments in cronjob.sh, rename "none" to "stable", minor cleanup to logging.
Rob Landley <rob@landley.net>
parents:
851
diff
changeset
|
19 |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
20 # Expire snapshots directory |
771
1fbe6d6a5a6b
Cron job script to build various nightly snapshots.
Rob Landley <rob@landley.net>
parents:
diff
changeset
|
21 |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
22 SNAPSHOTS="$(find snapshots -mindepth 1 -maxdepth 1 -type d)" |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
23 COUNT=$(( $(echo "$SNAPSHOTS" | wc -l) - 30 )) |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
24 if [ "$COUNT" -gt 0 ] |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
25 then |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
26 # Delete appropriate number of oldest entries, then dead symlinks. |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
27 rm -rf $( echo "$SNAPSHOTS" | sort | head -n $COUNT ) |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
28 rm -rf $(find -L snapshots -type l) |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
29 fi |
771
1fbe6d6a5a6b
Cron job script to build various nightly snapshots.
Rob Landley <rob@landley.net>
parents:
diff
changeset
|
30 |
1146 | 31 echo === Begin cron job |
32 | |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
33 # Start a new snapshot |
780
7f0e808e3be0
Allow specifying package list for nightly.sh, add explicit "fwl" and "all" stages, better handling of triage directories, do smoketest-all in parallel.
Rob Landley <rob@landley.net>
parents:
774
diff
changeset
|
34 |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
35 export SNAPSHOT_DATE=$(date +"%Y-%m-%d") |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
36 mkdir -p snapshots/$SNAPSHOT_DATE/base && |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
37 rm snapshots/latest && |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
38 ln -sf $SNAPSHOT_DATE snapshots/latest || exit 1 |
789
dde700454fbe
Update nightly.sh to new buildall.sh variables, move specifying package list to wrapper script, add "none" stage.
Rob Landley <rob@landley.net>
parents:
788
diff
changeset
|
39 |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
40 # build base repo |
780
7f0e808e3be0
Allow specifying package list for nightly.sh, add explicit "fwl" and "all" stages, better handling of triage directories, do smoketest-all in parallel.
Rob Landley <rob@landley.net>
parents:
774
diff
changeset
|
41 |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
42 build_snapshot() |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
43 { |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
44 if [ -z "$USE_UNSTABLE" ] |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
45 then |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
46 SNAPNAME=base |
771
1fbe6d6a5a6b
Cron job script to build various nightly snapshots.
Rob Landley <rob@landley.net>
parents:
diff
changeset
|
47 else |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
48 pull_repo $USE_UNSTABLE |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
49 SNAPNAME=$USE_UNSTABLE |
771
1fbe6d6a5a6b
Cron job script to build various nightly snapshots.
Rob Landley <rob@landley.net>
parents:
diff
changeset
|
50 fi |
1fbe6d6a5a6b
Cron job script to build various nightly snapshots.
Rob Landley <rob@landley.net>
parents:
diff
changeset
|
51 |
1146 | 52 echo === Building snapshot $SNAPNAME |
53 | |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
54 [ "$USE_UNSTABLE" == linux ] && |
1157
300e6d919d86
Move "sources/more/" to just "more/", add a README explaining why those scripts are there, and adjust calls to them.
Rob Landley <rob@landley.net>
parents:
1147
diff
changeset
|
55 more/for-each-arch.sh 'more/migrate-kernel.sh $TARGET' |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
56 |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
57 # Update manifest |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
58 |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
59 ./download.sh |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
60 |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
61 # If it's unchanged, just hardlink the previous binaries instead of rebuilding |
771
1fbe6d6a5a6b
Cron job script to build various nightly snapshots.
Rob Landley <rob@landley.net>
parents:
diff
changeset
|
62 |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
63 if cmp -s snapshots/latest/$SNAPNAME/MANIFEST packages/MANIFEST |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
64 then |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
65 cp -rl snapshots/latest/$SNAPNAME/* snapshots/$SNAPSHOT_DATE/$SNAPNAME |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
66 return |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
67 fi |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
68 |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
69 # Build it |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
70 |
1157
300e6d919d86
Move "sources/more/" to just "more/", add a README explaining why those scripts are there, and adjust calls to them.
Rob Landley <rob@landley.net>
parents:
1147
diff
changeset
|
71 nice -n 20 more/buildall.sh |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
72 rm build/simple-cross-compiler-*.tar.bz2 |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
73 mv build/*.tar.bz2 build/logs build/MANIFEST snapshots/$SNAPSHOT_DATE/$SNAPNAME |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
74 } |
771
1fbe6d6a5a6b
Cron job script to build various nightly snapshots.
Rob Landley <rob@landley.net>
parents:
diff
changeset
|
75 |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
76 build_snapshot base |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
77 |
1146 | 78 echo === Building QEMU |
79 | |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
80 # build qemu-git |
771
1fbe6d6a5a6b
Cron job script to build various nightly snapshots.
Rob Landley <rob@landley.net>
parents:
diff
changeset
|
81 |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
82 QPATH="" |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
83 CPUS=$(echo /sys/devices/system/cpu/cpu[0-9]* | wc -w) |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
84 pull_repo qemu |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
85 pushd build/packages/alt-qemu |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
86 ./configure --disable-werror && |
1147
593392b3186b
cronjob didn't include functions.sh so doesn't have maybe_quiet, and the stderr redirect has to go before the pipe to tee anyway.
Rob Landley <rob@landley.net>
parents:
1146
diff
changeset
|
87 nice -n 20 make -j $CPUS 2>&1 | tee build/logs/alt-qemu.txt && |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
88 QPATH="$(for i in *-softmmu;do echo -n $(pwd)/$i:; done)" |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
89 popd |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
90 |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
91 # test all with qemu-git |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
92 |
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
93 [ -z "$QPATH" ] || |
1157
300e6d919d86
Move "sources/more/" to just "more/", add a README explaining why those scripts are there, and adjust calls to them.
Rob Landley <rob@landley.net>
parents:
1147
diff
changeset
|
94 PATH="$QPATH:$PATH" more/for-each-target.sh \ |
300e6d919d86
Move "sources/more/" to just "more/", add a README explaining why those scripts are there, and adjust calls to them.
Rob Landley <rob@landley.net>
parents:
1147
diff
changeset
|
95 'more/smoketest.sh $TARGET | tee snapshots/$SNAPSHOT_DATE/base/logs/newqemu-smoketest-$TARGET.txt' |
1142
f2cd0b4f3936
New cron job to populate snapshots directory.
Rob Landley <rob@landley.net>
parents:
885
diff
changeset
|
96 |
1146 | 97 #USE_UNSTABLE=linux build_snapshot |
98 #USE_UNSTABLE=uClibc build_snapshot | |
99 #USE_UNSTABLE=busybox build_snapshot |