aboutsummaryrefslogtreecommitdiff
path: root/Tools
Commit message (Collapse)AuthorAgeFilesLines
* Newline should go after the terminating punctuation (fix typo).Dima Dorfman2004-11-071-1/+1
| | | | | | | Approved by: will Notes: svn path=/head/; revision=121024
* Fix a couple of bugs in the extraction of this from dopackages, andKris Kennaway2004-10-251-33/+20
| | | | | | | | flatten the setupnode() function into the main body since it's only called from one place. Notes: svn path=/head/; revision=120107
* Redefine the -norestr option to mean 'don't build the list ofKris Kennaway2004-10-251-78/+73
| | | | | | | | | | | | | restricted ports' instead of 'don't build any restricted ports' since the former is useful when we're not intending to publish the results of a build, but the latter is not. Move the build preprocessing (directory setup, old build rotation, etc) out from under -nobuild, so that we can set up a new build using that option. Notes: svn path=/head/; revision=120106
* If the symlink under latest/ exists, then use that to find the most recentKris Kennaway2004-10-251-1/+6
| | | | | | | | error log for the port, instead of assuming it exists in the current build. Notes: svn path=/head/; revision=120105
* We now also do 6.x builds in the '6' branch.Kris Kennaway2004-10-251-3/+3
| | | | Notes: svn path=/head/; revision=120104
* Use the absolute path to nc(1) to deal with it not being in the PATH.Kris Kennaway2004-10-251-3/+1
| | | | | | | Remove useless sleep(1). Notes: svn path=/head/; revision=120103
* Clean up the 'latest error log' symlink when the build is successful.Kris Kennaway2004-10-251-0/+4
| | | | Notes: svn path=/head/; revision=120102
* When a port build fails, create a symlink to the error log underKris Kennaway2004-10-251-3/+4
| | | | | | | | | | ${arch}/${branch}/latest/${portdir}. We will use this in the processfail script, so that the "new package build errors" webpages do not have out-of-date links but instead link to the most recent copy of the build error. Notes: svn path=/head/; revision=120101
* Strip out date/time from +++/--- headers in generated patchfiles, so that theyMaxim Sobolev2004-10-141-11/+12
| | | | | | | don't generate extra hunk each time patchfile is updated. Notes: svn path=/head/; revision=119419
* Support ${client_user} and ${sudo_cmd} for running commands as root onKris Kennaway2004-09-261-1/+5
| | | | | | | | | | machines where we first connect to a non-root user and then increase privileges. Approved by: portmgr (self) Notes: svn path=/head/; revision=118451
* * Break out setupnode into the dosetupnode script so it may be calledKris Kennaway2004-09-261-37/+3
| | | | | | | | | | | manually * Support 6.x builds Approved by: portmgr (self) Notes: svn path=/head/; revision=118450
* Break out the setupnode function of dopackages into a separate script soKris Kennaway2004-09-261-0/+47
| | | | | | | | | | | | | | | | | | that it may be called by hand. Support new portbuild.conf variables client_user = user to connect to on the client (not necessarily root). This user must have write permission to the /var/portbuild tree if disconnected=1 (i.e. we're going to run rsync). rsync_gzip = set to "-z" to enable compression on low-bandwidth disconnected clients. Approved by: portmgr (self) Notes: svn path=/head/; revision=118449
* * Bail out if we can't claim a chroot directory in the client (e.g. ifKris Kennaway2004-09-261-13/+17
| | | | | | | | | | | | | | | | | | ssh times out) * Support new portbuild.conf settings: client_user = user to connect to on the client (not necessarily root) sudo_cmd = If ssh'ing to a non-root user, run this command to gain root privs (set to empty string for client_user=root, or sudo for !root). Cannot require interactivity, of course. Approved by: portmgr (self) Notes: svn path=/head/; revision=118448
* Refuse to "clean" / if something goes wrong upstream and we are passedKris Kennaway2004-09-261-0/+5
| | | | | | | | | the wrong path Approved by: portmgr (self) Notes: svn path=/head/; revision=118447
* Pull in portbuild.$(hostname) for per-client customizationKris Kennaway2004-09-261-1/+3
| | | | | | | Approved by: portmgr (self) Notes: svn path=/head/; revision=118446
* Support 6.x buildsKris Kennaway2004-09-262-1/+5
| | | | | | | Approved by: portmgr (self) Notes: svn path=/head/; revision=118445
* * Support 6.x buildsKris Kennaway2004-09-261-2/+12
| | | | | | | | | | * Default 4.x to XFree86 4.x, 5.x and above to x.org * Don't pick up local customizations from OPTIONS Approved by: portmgr (self) Notes: svn path=/head/; revision=118444
* Use a consistent PATH in both portbuild and dopackages.Joe Marcus Clarke2004-08-271-1/+1
| | | | Notes: svn path=/head/; revision=117441
* Avoid possible GNATS timeouts by using query-pr.real direclty.Kirill Ponomarev2004-07-281-1/+1
| | | | Notes: svn path=/head/; revision=114914
* Clean up the specified chroot (umount filesystems, and scrub theKris Kennaway2004-07-141-0/+36
| | | | | | | directories likely to contain changed files). Notes: svn path=/head/; revision=113628
* Atomically claim an unused chroot directory. The previous algorithmKris Kennaway2004-07-141-0/+52
| | | | | | | | was not atomic and could have ended up with two port builds trying to use the same chroot. Report the claimed directory back to the caller. Notes: svn path=/head/; revision=113627
* Bump up the cleanup timeout to directories that have been in use forKris Kennaway2004-07-141-3/+3
| | | | | | | >48 hours, to cope with monster ports like openoffice and atlas. Notes: svn path=/head/; revision=113625
* Back out the attempt to list the error type in the failure summary;Kris Kennaway2004-07-141-14/+1
| | | | | | | | | | because this file is a chronological history of port builds that have failed, the files listed may not be present in the current set of error logs, and we currently have no easy way to find the most recent failure log to use instead. Notes: svn path=/head/; revision=113624
* Generate permanent URLs in the error reports. The directories likeKris Kennaway2004-07-141-1/+10
| | | | | | | | | | | i386-5-latest that are linked to from the index.html are symlinks to dated directories (e.5.`date`), so the URLs in the error reports will expire with the start of the next build when the symlink is repointed. This change makes the URLs in the error reports use the realpath of the target file, so they do not expire. Notes: svn path=/head/; revision=113623
* * Add support for ${TRYBROKEN}.Kris Kennaway2004-07-141-6/+67
| | | | | | | | | | | | | | | | | | | | | | | * Clients no longer have ssh access to the master, so we need to push/pull everything on the client from here. This means we need to know where the build took place so we can go in and get the files after it finishes. Introduce the claim-chroot script which atomically claims a free chroot directory on the host and returns the name. This directory is later populated by the portbuild script if it does not already contain an extracted bindist. * Use the per-node portbuild.$(hostname) config file to decide where in the filesystem to claim the chroot on the build host. * If a port failed unexpectedly (i.e. is not marked BROKEN), or if something strange happened when trying to pull in build results from a client, then send me email (XXX should be configurable). * Clean up after the build finishes and we have everything we need, by dispatching the clean-chroot script on the client. Notes: svn path=/head/; revision=113622
* * Add a cleanup() function and use it so that we archive the ${WRKDIR}Kris Kennaway2004-07-141-68/+122
| | | | | | | | | | | | | | | | | | | | | | if requested (".keep" file in the port directory), no matter where we fail. * Add package dependencies before the corresponding build stage (e.g. FETCH_DEPENDS before 'make fetch'), and remove them again afterwards. This allows us to catch ports that list their dependencies too early/late. * No need to check for set[ug]id files here, the security-check target in bsd.port.mk does it for us. * Exclude some more directories and files from showing up in the mtree before/after comparison, to trim down the false-positive in the pkg-plist check. * Other minor changes Notes: svn path=/head/; revision=113621
* * Remove the quickports stuff, it didn't (ever?) work as designed andKris Kennaway2004-07-141-112/+156
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | it's done properly^Wbetter in makeparallel * Script accepts new arguments: -nodoccvs: skip cvs update of the doc tree -trybroken: try to build BROKEN ports (off by default because the i386 cluster is fast enough now that when doing incremental builds we were spending most of the time rebuilding things we know are probably going to fail anyway. Conversely, the other clusters are slow enough that we also usually don't want to waste time on BROKEN ports). -incremental: compare the interesting fields of the new INDEX with the previous one, remove packages and log files for the old ports that have changed, and rebuild the rest. This substantially cuts down on build times since we don't rebuild ports that we know have not changed. XXX checkpoint of work-in-progress, not yet working as committed. * When setting up the nodes, read in per-node config files ("portbuild.$(hostname)") before dispatching the setupnode script on each node. For disconnected nodes (which don't mount the master via NFS), we also rsync the interesting files required by the builds (ports/src/doc trees, bindist tarballs, scripts) into place on the client. They will be mounted locally via nullfs in the build chroots. * Break out the restricted.sh generation into a makerestr script so it can be called manually as needed. * Remove the -nocvsup argument which has been unused for a long time. * For now, don't prune the list of failed ports with prunefail, since when -trybroken is not specified, every BROKEN port ends up in the duds file (so the build is skipped), and as a result we would prune almost everything from the list of failed ports. XXX prunefailure should be run conditionally on -trybroken, or I should find a way to prune in both cases. * Don't run index in the background, it was thrashing against makeduds and not saving any time by doing it concurrently. * Build with 'make quickports all' to kick off the quickports builds earlier. * Delete restricted and/or cdrom distfiles *after* post-processing the distfiles, otherwise the script doesn't remove any of them since they're not in the expected place. * Miscellaneous other minor changes and cleanups Notes: svn path=/head/; revision=113620
* Parse the slightly-different format of the build logs.Kris Kennaway2004-07-141-1/+1
| | | | Notes: svn path=/head/; revision=113619
* Parse the slightly-different format of the build logs.Kris Kennaway2004-07-141-3/+3
| | | | Notes: svn path=/head/; revision=113617
* * Accept another argument, the value of the disconnected variable thatKris Kennaway2004-07-141-25/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | tells us whether the node has NFS access to the master. * Also copy the bindist-$(hostname).tar file to allow local customization of the build chroots (e.g. resolv.conf and make.conf files for disconnected systems) * For disconnected hosts, we don't copy the bindist files from the master, but just set up the local directories and let the server rsync them into place later. Also set up dangling symlinks to the bindist files in the build area, which will be filled in by the server too (in the NFS case it makes sense to cache the bindist files locally to avoid extra NFS traffic, but here we know the file is local so a symlink is fine) * Remove an apparently spurious 'killall fetch' that snuck in for what were probably transient reasons. * Forcibly clean up old chroot directories since we are preparing to start another build and don't want old (possibly orphaned) builds to skew the job scheduling or use up resources. Notes: svn path=/head/; revision=113616
* Allow local customization of the node using the portbuild.$(hostname) file.Kris Kennaway2004-07-141-0/+1
| | | | Notes: svn path=/head/; revision=113615
* Send me mail when a previously-broken port builds successfully.Kris Kennaway2004-07-141-2/+5
| | | | | | | XXX Should not be a hard-coded address Notes: svn path=/head/; revision=113614
* * Initial support for disconnected nodes (not on same LAN as buildKris Kennaway2004-07-141-83/+88
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | host), specified by disconnected=1 in portbuild.$(hostname) file. These do not mount via NFS, so we need to maintain a local copy of things needed by the build (like the ports/src/doc trees) on the build host, which are mounted into the chroot by read-only nullfs. These local files are maintained in the dopackages script via rsync. * Download packages via http instead of NFS. Allow fetching via a local http proxy (http_proxy variable in per-node portbuild.$(hostname) file). Caching package dependencies saves about 85% of package fetches and similar reduction in package fetch traffic by byte count. * Support a per-node tarball (bindist-$(hostname).tar) to customize the build chroots. This is used for things like local resolv.conf and make.conf files on disconnected nodes. * Make sure we don't use a chroot until it is finished extracting. * Don't set '.' in PATH; this is bad practise, and fortunately nothing seems to rely on it. * Only try to build broken packages if requested * Try harder to unmount leftover linprocfs mounts in the chroot, by copying in the 5.x mount binary and supporting libraries from the host system. The 5.x mount is able to unmount by FSID in situations where the 4.x umount becomes confused. * Don't clean up when we are signalled, that is done by the build master from outside. * Suppress some code relating to jail builds, which are not yet ready for use. * Don't push results of the build back to the master; the master now pulls them from the client when the build completes. Clients no longer need ssh access into the master; this is good for security as well as significantly reducing the load on the master since it is not thrashed by dozens of sshd processes. Notes: svn path=/head/; revision=113613
* Hack around the sparc64 64-bit time_t test in 'installworld' sinceKris Kennaway2004-07-141-0/+1
| | | | | | | | we're installing (into a temporary staging area) on a non-sparc64 machine that does not have 64-bit time_t. Notes: svn path=/head/; revision=113609
* * Break out the 'parallel' target from /usr/ports/Makefile. OneKris Kennaway2004-07-141-6/+40
| | | | | | | | | | | | | | | | | | | | | | | | | advantage is that here we know the value of PKGSUFFIX (.tgz/.tbz) for the build via buildenv. * Add a list of 'quickports', which are ports with long dependency chains that we should kick off straight away to try and avoid bottlenecks later on when most of the cluster idles waiting for one or two ports to build. Ideally we'd build dependencies of these ports exclusively first and only build other ports when we run out (i.e. a build slot becomes free), but I couldn't work out how to do this. As a compromise, we now do 'make -k -j<#> quickports all' which doesn't give quite as high a priority to the quickports (i.e. we also build other ports from the beginning while there are quickport dependencies still to build), but is better than nothing. * Pass in the FETCH/EXTRACT/PATCH/BUILD/RUN_DEPENDS separately via env variables when dispatching a job. This allows us to add and remove the dependencies at the corresponding build stage to catch ports with dependencies listed too early/late. Notes: svn path=/head/; revision=113608
* Set INDEX_JOBS to control index build concurrencyKris Kennaway2004-07-141-0/+3
| | | | Notes: svn path=/head/; revision=113607
* Drop the ignorelist concurrency back to 1 because I'm occasionally seeingKris Kennaway2004-07-141-1/+1
| | | | | | | | overlapping I/O. Something similar to how parallel index builds are now done will need to be done here too. Notes: svn path=/head/; revision=113606
* * If we time out connecting to a machine, give it a weight of 999 to makeKris Kennaway2004-07-141-2/+2
| | | | | | | | | | sure we don't try and schedule jobs on it even if all other machines are busy * Remove sleep in outer loop, this isn't needed or worthwhile now that there are so many machines being monitored Notes: svn path=/head/; revision=113605
* Chase bento to pointyhatKris Kennaway2004-07-141-1/+1
| | | | Notes: svn path=/head/; revision=113604
* * Be even more explicit about partial ports trees being unsupportedKris Kennaway2004-07-141-13/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | for INDEX builds [1] * Remove the parallel target from Makefile; this is heavily tied to the package build cluster and can be better done in the makeparallel script (commit to follow) [2] * Extend the format of INDEX to separately list the EXTRACT/PATCH/FETCH_DEPENDS instead of lumping them all in together with BUILD_DEPENDS. The three new fields are appended to the end of the record in that order. [2] * Change BROKEN to IGNORE in BROKEN_WITH_MYSQL failure code [3] * Support non-default PREFIX for perl 5.00503 [5] * Use pkg_info -I instead of ls when searching for conflicts [6] * Allow local customization of the port subdirectories by including ${.CURDIR}/Makefile.local in bsd.subdir.mk if it exists [7] * Fix 'make search' when ${PORTSDIR} is a symlink to a directory name containing extended regexp metacharacters [8] Submitted by: linimon [1] [3], kris [2], lth [4], sem [5], eik [5] [6], Roman Neuhauser <neuhauser@chello.cz> [7] PR: 68299 [1], 67705 [3], 67264 [4], 59696 [5], 66568 [6], 68072 [7] Notes: svn path=/head/; revision=113603
* Modernize these two scripts a little bit to make it easier to dealMurray Stokely2004-06-082-6/+16
| | | | | | | | | with 5.X or 4.X package sets (i.e., INDEX vs INDEX-5 and tgz vs tbz) OKed by: marcus Notes: svn path=/head/; revision=111108
* Hack in master/slave supportOliver Eikemeier2004-06-041-27/+69
| | | | Notes: svn path=/head/; revision=110856
* Add dopackages.wrapper, a wrapper script around dopackages that sets upJoe Marcus Clarke2004-06-031-0/+53
| | | | | | | | | | | | | | | build locking, log files, and cleans things up if a build fails. This script is the primary starting point for a package build. Symlinks should be created in the form of dopackages.${branch} -> dopackages.wrapper where ${branch} is currently one of 4, 4-exp, or 5. This script takes the place of the unofficial (i.e. uncommitted) dopackages.steveX scripts. Ok'd by: kris Tested by: 4.10-RELEASE package build Notes: svn path=/head/; revision=110725
* parse $FreeBSD$ with yyyy-mm-dd format datesOliver Eikemeier2004-05-301-1/+1
| | | | Notes: svn path=/head/; revision=110428
* protect $FreeBSD: ... $ from being expandedOliver Eikemeier2004-05-301-1/+1
| | | | | | | Noticed by: mat Notes: svn path=/head/; revision=110413
* don't terminate the log entry with the first hyphenOliver Eikemeier2004-05-301-1/+1
| | | | Notes: svn path=/head/; revision=110412
* - make somewhat more robust against a corrupted ports treeOliver Eikemeier2004-05-301-30/+80
| | | | | | | | | | | | | - CC committers and maintainer [1] - include affected ports in the subject line [2] - do a CVS log of the version checked out [3] Suggsted by: Ade Lovett <ade@FreeBSD.org> [1] Bjoern A. Zeeb <bzeeb-lists@lists.zabbadoz.net> [2] Pav Lucistnik <pav@FreeBSD.org> [3] Notes: svn path=/head/; revision=110402
* - Remove 'pkg-plist' from required filesKirill Ponomarev2004-05-271-2/+1
| | | | | | | Approved by: sobomax Notes: svn path=/head/; revision=110121
* - fix a bug handling the maintainer field when ALLPORTS is usedOliver Eikemeier2004-05-211-35/+56
| | | | | | | | | - add an option to watch for maintainer changes - change watch regex to match PKGORIGIN, not PKGNAME Notes: svn path=/head/; revision=109655
* Turn chkversion into an all-dancing, all-singing notification tool.Oliver Eikemeier2004-05-191-35/+138
| | | | | | | | | | | You can even get notified of version changes in your favourite perl modules by setting WATCH_REGEX='p5-.*' Plus, it has a nice configurable nagging option. Notes: svn path=/head/; revision=109506
generated by cgit v1.2.3 (git 2.25.1) at 2024-10-10 14:28:53 +0000