aboriginal: sources/download_functions.sh annotate

annotate sources/download_functions.sh @ 1622:ca0cf2dc37c2

Add more documentation.

author	Rob Landley <rob@landley.net>
date	Sat, 24 Aug 2013 05:56:17 -0500
parents	e8d846fb56eb
children	73651ad7ad13

rev	line source
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	1 #!/bin/echo "This file is sourced, not run"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	2
1622 ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	3 # Set of functions to download, extract, and patch source tarballs.
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	4 #
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	5 # Tools to populate and verify a directory of package source tarballs
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	6 # (saved in $SRCDIR which defaults to $TOP/packages). Used by download.sh.
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	7 #
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	8 # You can supply your own tarball in $SRCDIR to avoid downloading it.
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	9 #
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	10 # You can also provide an expanded directory (same filename as $URL
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	11 # but no version or extension) to be used instead of a tarball. This is
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	12 # usually a source control checkout.
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	13 #
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	14 # $IGNORE_REPOS - comma separated list of package names (or "all") to
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	15 # download tarballs for anyway, ignoring the directory version if present.
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	16 #
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	17 # Functions to call from here:
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	18 #
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	19 # download
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	20 # - fetch a file (with wget) if it doesn't already exist, or doesn't match
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	21 # checksum.
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	22 #
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	23 # It expects you to set:
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	24 # $URL - Default location of file, including filename
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	25 # $SHA1 - sha1sum of good file. (Blank means accept any file.)
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	26 #
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	27 # You can also set these (which use filename from $URL):
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	28 # $PREFERRED_MIRROR - Check here first (appending filename from $URL)
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	29 # $MIRROR_LIST - Space separated list of fallback locations (appending
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	30 # filename from $URL) to check if default location didn't have it.
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	31 #
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	32 # Does not re-download existing tarballs if the $SHA1 matches/is blank.
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	33 # Does not download tarball if expanded directory present.
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	34 #
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	35 # cleanup_oldfiles
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	36 # - remove stale files from $SRCDIR
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	37 #
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	38 # Stale means not referenced by a download call since start of script.
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	39 # Only affects files, not subdirectories.
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	40
ca0cf2dc37c2 Add more documentation. Rob Landley <rob@landley.net> parents: 1599 diff changeset	41
1257 1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	42 # Remove version information and extension tarball name "$1".
1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	43 # If "$2", add that version number back, keeping original extension.
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	44
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	45 noversion()
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	46 {
1257 1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	47 LOGRUS='s/-$\([0-9\.]$$[_-]rc$$-pre$$[0-9][a-zA-Z]$\)$\.tar\(\..z2$\)$'
1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	48 [ -z "$2" ] && LOGRUS="$LOGRUS//" \|\| LOGRUS="$LOGRUS/$2\\6/"
1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	49
1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	50 echo "$1" \| sed -e "$LOGRUS"
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	51 }
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	52
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	53 # Apply any patches to this package
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	54 patch_package()
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	55 {
1257 1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	56 ls "$PATCHDIR/${PACKAGE}"-*.patch 2> /dev/null \| sort \| while read i
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	57 do
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	58 if [ -f "$i" ]
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	59 then
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	60 echo "Applying $i"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	61 (cd "${SRCTREE}/${PACKAGE}" &&
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	62 patch -p1 -i "$i" &&
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	63 sha1file "$i" >> "$SHA1FILE") \|\|
1418 f2f1238d25e2 Actually stop after the first failed patch. Rob Landley <rob@landley.net> parents: 1416 diff changeset	64 if [ -z "$ALLOW_PATCH_FAILURE" ]
f2f1238d25e2 Actually stop after the first failed patch. Rob Landley <rob@landley.net> parents: 1416 diff changeset	65 then
f2f1238d25e2 Actually stop after the first failed patch. Rob Landley <rob@landley.net> parents: 1416 diff changeset	66 dienow
f2f1238d25e2 Actually stop after the first failed patch. Rob Landley <rob@landley.net> parents: 1416 diff changeset	67 fi
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	68 fi
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	69 done
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	70 }
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	71
1300 a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	72 # Get the tarball for this package
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	73
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	74 find_package_tarball()
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	75 {
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	76 # If there are multiple similar files we want the newest timestamp, in case
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	77 # the URL just got upgraded but cleanup_oldfiles hasn't run yet. Be able to
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	78 # distinguish "package-123.tar.bz2" from "package-tests-123.tar.bz2" and
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	79 # return the shorter one reliably.
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	80
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	81 ls -tc "$SRCDIR/$1-".tar 2>/dev/null \| while read i
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	82 do
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	83 if [ "$(noversion "${i/*\//}")" == "$1" ]
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	84 then
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	85 echo "$i"
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	86 break
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	87 fi
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	88 done
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	89 }
a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	90
1582 6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	91 package_cache()
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	92 {
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	93 SNAPFROM="$SRCDIR/$1"
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	94 (is_in_list "$1" "$IGNORE_REPOS" \|\| [ ! -d "$SNAPFROM" ]) &&
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	95 SNAPFROM="$SRCTREE/$1"
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	96
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	97 if [ ! -d "$SNAPFROM" ]
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	98 then
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	99 echo "$1 not found. Did you run download.sh?" >&2
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	100 dienow
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	101 fi
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	102
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	103 echo "$SNAPFROM"
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	104 }
6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	105
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	106 # Extract tarball named in $1 and apply all relevant patches into
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	107 # "$BUILD/packages/$1". Record sha1sum of tarball and patch files in
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	108 # sha1-for-source.txt. Re-extract if tarball or patches change.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	109
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	110 extract_package()
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	111 {
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	112 mkdir -p "$SRCTREE" \|\| dienow
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	113
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	114 PACKAGE="$1"
1553 da21ff27331e Add ability to check out repositories in packages, IGNORE_REPOS to build from tarballs anyway, a FAQ entry on it, and more/repo.sh to set up/update some of the common ones. Rob Landley <rob@landley.net> parents: 1552 diff changeset	115 ! is_in_list "PACKAGE" "$IGNORE_REPOS" && [ -d "$SRCDIR/$PACKAGE" ] &&
da21ff27331e Add ability to check out repositories in packages, IGNORE_REPOS to build from tarballs anyway, a FAQ entry on it, and more/repo.sh to set up/update some of the common ones. Rob Landley <rob@landley.net> parents: 1552 diff changeset	116 return 0
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	117
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	118 # Announce to the world that we're cracking open a new package
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	119
1358 9252453c40d0 Combine set_titlebar and "echo ===" stuff into announce() function. Consistently output target/stage info, and yank redundant instances. Rob Landley <rob@landley.net> parents: 1335 diff changeset	120 announce "$PACKAGE"
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	121
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	122 # Find tarball, and determine type
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	123
1300 a80d7f256ff9 Handle package tarballs with overlapping names (thingy and thingy-test for example). Rob Landley <rob@landley.net> parents: 1257 diff changeset	124 FILENAME="$(find_package_tarball "$PACKAGE")"
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	125 DECOMPRESS=""
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	126 [ "$FILENAME" != "${FILENAME/%\.tar\.bz2/}" ] && DECOMPRESS="j"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	127 [ "$FILENAME" != "${FILENAME/%\.tar\.gz/}" ] && DECOMPRESS="z"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	128
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	129 # If the source tarball doesn't exist, but the extracted directory is there,
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	130 # assume everything's ok.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	131
1515 63d3f491b92f Make tarball extract failure detection not rely on "kill" command. Rob Landley <rob@landley.net> parents: 1486 diff changeset	132 SHA1NAME="sha1-for-source.txt"
63d3f491b92f Make tarball extract failure detection not rely on "kill" command. Rob Landley <rob@landley.net> parents: 1486 diff changeset	133 SHA1FILE="$SRCTREE/$PACKAGE/$SHA1NAME"
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	134 if [ -z "$FILENAME" ]
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	135 then
1257 1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	136 if [ ! -e "$SRCTREE/$PACKAGE" ]
1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	137 then
1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	138 echo "No tarball for $PACKAGE" >&2
1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	139 dienow
1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	140 fi
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	141
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	142 # If the sha1sum file isn't there, re-patch the package.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	143 [ ! -e "$SHA1FILE" ] && patch_package
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	144 return 0
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	145 fi
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	146
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	147 # Check the sha1 list from the previous extract. If the source is already
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	148 # up to date (including patches), keep it.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	149
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	150 SHA1TAR="$(sha1file "$FILENAME")"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	151 SHALIST=$(cat "$SHA1FILE" 2> /dev/null)
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	152 if [ ! -z "$SHALIST" ]
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	153 then
1599 e8d846fb56eb Make sha1-for-source.txt only test .patch files out of sources/patches so stupid ".orig" files don't cause perpetual re-extraction. Rob Landley <rob@landley.net>* parents: 1582 diff changeset	154 for i in "$SHA1TAR" $(sha1file "$PATCHDIR/$PACKAGE"-*.patch 2>/dev/null)
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	155 do
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	156 # Is this sha1 in the file?
1582 6a8114c148d1 Lots of small improvements: check for toybox instead of busybox for host-tools $PATH adjustment, better manifest generation (with toybox in list), make package_cache function to find extracted source (so root-image.sh works when packages contains linux-git snapshot), fix more/record-commands.sh path adjustment. Rob Landley <rob@landley.net> parents: 1553 diff changeset	157 if [ -z "$(echo "$SHALIST" \| grep "$i")" ]
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	158 then
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	159 SHALIST=missing
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	160 break
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	161 fi
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	162 # Remove it
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	163 SHALIST="$(echo "$SHALIST" \| sed "s/$i//" )"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	164 done
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	165 # If we matched all the sha1sums, nothing more to do.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	166 [ -z "$SHALIST" ] && return 0
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	167 fi
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	168
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	169 # Re-extract the package, deleting the old one (if any)..
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	170
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	171 echo -n "Extracting '$PACKAGE'"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	172 (
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	173 UNIQUE=$(readlink /proc/self)
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	174 trap 'rm -rf "$BUILD/temp-'$UNIQUE'"' EXIT
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	175 rm -rf "$SRCTREE/$PACKAGE" 2>/dev/null
1410 7373c4c7ecee Don't create build/packages _and_ $SRCTREE, they should be the same and when they aren't the first is wrong. Rob Landley <rob@landley.net> parents: 1358 diff changeset	176 mkdir -p "$BUILD/temp-$UNIQUE" "$SRCTREE" \|\| dienow
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	177
1515 63d3f491b92f Make tarball extract failure detection not rely on "kill" command. Rob Landley <rob@landley.net> parents: 1486 diff changeset	178 { tar -xv${DECOMPRESS} -f "$FILENAME" -C "$BUILD/temp-$UNIQUE" &&
63d3f491b92f Make tarball extract failure detection not rely on "kill" command. Rob Landley <rob@landley.net> parents: 1486 diff changeset	179 # Wildcards only expand when they ponit to something that exists,
63d3f491b92f Make tarball extract failure detection not rely on "kill" command. Rob Landley <rob@landley.net> parents: 1486 diff changeset	180 # and even then they have to be an argument to a command.
63d3f491b92f Make tarball extract failure detection not rely on "kill" command. Rob Landley <rob@landley.net> parents: 1486 diff changeset	181 TARDIR="$(readlink -f "$BUILD/temp-$UNIQUE"/*)" &&
63d3f491b92f Make tarball extract failure detection not rely on "kill" command. Rob Landley <rob@landley.net> parents: 1486 diff changeset	182 touch "$TARDIR/$SHA1NAME"
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	183 } \| dotprogress
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	184
1515 63d3f491b92f Make tarball extract failure detection not rely on "kill" command. Rob Landley <rob@landley.net> parents: 1486 diff changeset	185 [ -e "$BUILD/temp-$UNIQUE"/*/"$SHA1NAME" ] &&
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	186 mv "$BUILD/temp-$UNIQUE/"* "$SRCTREE/$PACKAGE" &&
1515 63d3f491b92f Make tarball extract failure detection not rely on "kill" command. Rob Landley <rob@landley.net> parents: 1486 diff changeset	187 echo "$SHA1TAR" >> "$SHA1FILE"
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	188 )
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	189
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	190 [ $? -ne 0 ] && dienow
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	191
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	192 patch_package
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	193 }
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	194
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	195 # Confirm that a file has the appropriate checksum (or exists but SHA1 is blank)
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	196 # Delete invalid file.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	197
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	198 confirm_checksum()
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	199 {
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	200 SUM="$(sha1file "$SRCDIR/$FILENAME" 2>/dev/null)"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	201 if [ x"$SUM" == x"$SHA1" ] \|\| [ -z "$SHA1" ] && [ -f "$SRCDIR/$FILENAME" ]
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	202 then
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	203 if [ -z "$SHA1" ]
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	204 then
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	205 echo "No SHA1 for $FILENAME ($SUM)"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	206 else
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	207 echo "Confirmed $FILENAME"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	208 fi
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	209
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	210 # Preemptively extract source packages?
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	211
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	212 [ -z "$EXTRACT_ALL" ] && return 0
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	213 extract_package "$BASENAME"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	214 return $?
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	215 fi
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	216
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	217 # If there's a corrupted file, delete it. In theory it would be nice
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	218 # to resume downloads, but wget creates "*.1" files instead.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	219
1486 d398569effd2 Suppress some warnings. Rob Landley <rob@landley.net> parents: 1461 diff changeset	220 rm -f "$SRCDIR/$FILENAME"
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	221
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	222 return 1
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	223 }
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	224
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	225 # Attempt to obtain file from a specific location
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	226
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	227 download_from()
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	228 {
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	229 # Return success if we already have a valid copy of the file
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	230
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	231 confirm_checksum && return 0
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	232
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	233 # If we have another source, try to download file from there.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	234
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	235 [ -z "$1" ] && return 1
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	236 wget -t 2 -T 20 -O "$SRCDIR/$FILENAME" "$1" \|\|
1486 d398569effd2 Suppress some warnings. Rob Landley <rob@landley.net> parents: 1461 diff changeset	237 (rm -f "$SRCDIR/$FILENAME"; return 2)
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	238 touch -c "$SRCDIR/$FILENAME"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	239
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	240 confirm_checksum
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	241 }
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	242
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	243 # Confirm a file matches sha1sum, else try to download it from mirror list.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	244
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	245 download()
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	246 {
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	247 FILENAME=`echo "$URL" \| sed 's .*/ '`
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	248 [ -z "$RENAME" ] \|\| FILENAME="$(echo "$FILENAME" \| sed -r "$RENAME")"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	249
1335 ffb07a48a960 If sha1sum isn't installed, download falls back through all the mirrors repeatedly downloading stuff, without explaining why it failed. Rob Landley <rob@landley.net> parents: 1300 diff changeset	250 if [ -z "$(sha1sum < /dev/null)" ]
ffb07a48a960 If sha1sum isn't installed, download falls back through all the mirrors repeatedly downloading stuff, without explaining why it failed. Rob Landley <rob@landley.net> parents: 1300 diff changeset	251 then
ffb07a48a960 If sha1sum isn't installed, download falls back through all the mirrors repeatedly downloading stuff, without explaining why it failed. Rob Landley <rob@landley.net> parents: 1300 diff changeset	252 echo "Error: please install sha1sum" >&2
ffb07a48a960 If sha1sum isn't installed, download falls back through all the mirrors repeatedly downloading stuff, without explaining why it failed. Rob Landley <rob@landley.net> parents: 1300 diff changeset	253 exit 1
ffb07a48a960 If sha1sum isn't installed, download falls back through all the mirrors repeatedly downloading stuff, without explaining why it failed. Rob Landley <rob@landley.net> parents: 1300 diff changeset	254 fi
ffb07a48a960 If sha1sum isn't installed, download falls back through all the mirrors repeatedly downloading stuff, without explaining why it failed. Rob Landley <rob@landley.net> parents: 1300 diff changeset	255
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	256 echo -ne "checking $FILENAME\r"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	257
1552 c3b91b70cc42 Remove USE_ALT option, I have a better idea (upcoming patch). Rob Landley <rob@landley.net> parents: 1540 diff changeset	258 # Update timestamp on tarball (if it exists) so cleanup_oldfiles keeps it
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	259
1552 c3b91b70cc42 Remove USE_ALT option, I have a better idea (upcoming patch). Rob Landley <rob@landley.net> parents: 1540 diff changeset	260 touch -c "$SRCDIR"/"$FILENAME" 2>/dev/null
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	261
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	262 # Give package name, minus file's version number and archive extension.
1257 1461d20bb6a7 Genericize the noversion() regex a bit, and make sure to only apply patches in the sources/patches directory that end in ".patch". Rob Landley <rob@landley.net> parents: 1253 diff changeset	263 BASENAME="$(noversion "$FILENAME")"
1553 da21ff27331e Add ability to check out repositories in packages, IGNORE_REPOS to build from tarballs anyway, a FAQ entry on it, and more/repo.sh to set up/update some of the common ones. Rob Landley <rob@landley.net> parents: 1552 diff changeset	264 ! is_in_list "$BASENAME" "$IGNORE_REPOS" &&
da21ff27331e Add ability to check out repositories in packages, IGNORE_REPOS to build from tarballs anyway, a FAQ entry on it, and more/repo.sh to set up/update some of the common ones. Rob Landley <rob@landley.net> parents: 1552 diff changeset	265 [ -d "$SRCDIR/$BASENAME" ] && echo "Using $SRCDIR/$BASENAME" && return 0
1253 d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	266
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	267 # If environment variable specifies a preferred mirror, try that first.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	268
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	269 if [ ! -z "$PREFERRED_MIRROR" ]
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	270 then
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	271 download_from "$PREFERRED_MIRROR/$FILENAME" && return 0
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	272 fi
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	273
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	274 # Try original location, then mirrors.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	275 # Note: the URLs in mirror list cannot contain whitespace.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	276
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	277 download_from "$URL" && return 0
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	278 for i in $MIRROR_LIST
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	279 do
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	280 download_from "$i/$FILENAME" && return 0
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	281 done
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	282
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	283 # Return failure.
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	284
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	285 echo "Could not download $FILENAME"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	286 echo -en "\e[0m"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	287 return 1
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	288 }
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	289
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	290 # Clean obsolete files out of the source directory
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	291
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	292 START_TIME=`date +%s`
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	293
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	294 cleanup_oldfiles()
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	295 {
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	296 # wait for asynchronous downloads to complete
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	297
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	298 wait
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	299
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	300 for i in "${SRCDIR}"/*
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	301 do
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	302 if [ -f "$i" ] && [ "$(date +%s -r "$i")" -lt "${START_TIME}" ]
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	303 then
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	304 echo Removing old file "$i"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	305 rm -rf "$i"
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	306 fi
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	307 done
d4b251e78fc0 Break out download_functions.sh into their own file. Rob Landley <rob@landley.net> parents: diff changeset	308 }

Mercurial > hg > aboriginal

annotate sources/download_functions.sh @ 1622:ca0cf2dc37c2