aboutsummaryrefslogtreecommitdiff
path: root/Tools
Commit message (Collapse)AuthorAgeFilesLines
* Document the check-latest-link and tindex scriptsKris Kennaway2005-01-241-1/+5
| | | | Notes: svn path=/head/; revision=127163
* Add the script I use to look for duplicate LATEST_LINK entries and toKris Kennaway2005-01-241-0/+72
| | | | | | | send nag-mails to the responsible maintainers. Notes: svn path=/head/; revision=127162
* This is the script I use to build indexes for all supported ports branches,Kris Kennaway2005-01-241-0/+164
| | | | | | | | | which are copied to www.freebsd.org for use by 'make fetchindex', as well as the source of build failure reports on ports@FreeBSD.org when it is broken. Notes: svn path=/head/; revision=127161
* Remove traling spaces.Kirill Ponomarev2005-01-0928-52/+52
| | | | Notes: svn path=/head/; revision=125941
* Always set up the bindist-${hostname}.tar file since it's usually small.Kris Kennaway2005-01-061-1/+1
| | | | | | | | | This should probably be changed to only update it if it changes. Approved by: portmgr (self) Notes: svn path=/head/; revision=125740
* Unset the __MAKE_CONF variable after setting up buildenv, because weKris Kennaway2005-01-061-0/+3
| | | | | | | | | want to use the default value in the build chroot. Approved by: portmgr (self) Notes: svn path=/head/; revision=125739
* Allow machines to use an alternate address for polling for job load (e.g.Kris Kennaway2005-01-061-1/+3
| | | | | | | | | | if the machine is behind a NAT and is not directly addressable, we can use a localhost port forward to contact it) Approved by: portmgr (self) Notes: svn path=/head/; revision=125738
* * Include portbuild.conf before portbuild.${hostname} to reset the defaultKris Kennaway2005-01-061-4/+6
| | | | | | | | | | | value of variables * Allow krion to drink from the email firehose Approved by: portmgr (self) Notes: svn path=/head/; revision=125737
* * The mlist file now only has one field per line.Kris Kennaway2005-01-061-4/+9
| | | | | | | | | | | * Correctly calculate job concurrency in pass two as well * In pass two run dosetupnode -rsync to avoid wasting time. Approved by: portmgr (self) Notes: svn path=/head/; revision=125736
* krion wants to drink from the pointyhat email firehose as well.Kris Kennaway2005-01-061-1/+1
| | | | | | | | | XXX This should be a portbuild.conf tunable Approved by: portmgr (self) Notes: svn path=/head/; revision=125735
* Now that I have so many remote build machines, don't force trying toKris Kennaway2005-01-061-7/+0
| | | | | | | | | | | fetch from ftp-master and pointyhat; they'll just get timeouts. Instead, each machine is expected to set up their own MASTER_SITE_* variables in etc/make.conf via a bindist-${hostname}.tar file. Approved by: portmgr (self) Notes: svn path=/head/; revision=125734
* * Support the -norsync argument to only remove the existing build chrootsKris Kennaway2004-12-281-2/+8
| | | | | | | | | | | | | | | | | | | | on a disconnected client, without running the time-consuming rsyncs. This is useful when a build is interrupted and needs to be restarted. * After we have cleaned up the machine, reset the queue counter by using pollmachine -queue. This has a race condition if other builds are being dispatched to the machine (e.g. builds on another branch): getmachine can claim a directory and increment the counter, then the machine is polled and finds e.g. 0 chroots in use, and resets the counter to 0, then claim-chroot is run and the build dispatched, with the counter now off-by-one. This could be fixed by running claim-chroot with the .lock held, but this turns out to be too time-consuming. A two-level lock approach might also fix this efficiently. Notes: svn path=/head/; revision=125320
* * It does in fact make sense to specify -incremental and -noindex at theKris Kennaway2004-12-281-7/+7
| | | | | | | | | | | | | same time, assuming that the admin has already built the INDEX and INDEX.old in advance. * Adapt to new method of calculating build concurrency, by summing the value of ${maxjobs} listed in every portbuild.${machine} * Support 5-exp builds Notes: svn path=/head/; revision=125319
* Fix the Xvfb code to run Xvfb before the build phase if detectedKris Kennaway2004-12-281-15/+25
| | | | | | | | | | | | | | (i.e. if the package lists a dependency on the relevant package in the PACKAGE_BUILDING case). This allows packages that require an available DISPLAY to again build (with some forthcoming fixes to existing ports). Improve the reporting of detected filesystem anomalies (extra files left behind after deinstallation, changes to and removal of pre-existing files) Notes: svn path=/head/; revision=125318
* Overhaul of the job scheduler. The new scheduler runs buildsKris Kennaway2004-12-284-29/+119
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | synchronously instead of probabilistically scheduling jobs, which means that the job load on a machine never exceeds a desired threshold, and we can preferentially use faster machines when they are available. This has a dramatic effect on package build throughput, although I don't yet have precise measurements of the performance improvements. Specifically, the changes are: * Introduce the new variable maxjobs in portbuild. This replaces the build scheduling weights previously listed in the mlist file, which now changes format to list the build machines only, ranked in order of preference for job dispatches (i.e. faster machines first). * The ${arch}/queue directory is used to list machines available for jobs (file content is the number of jobs currently running on the machine). Changes to files in this directory are serialized using lockf on the .lock file. * Claim a machine with the getmachine script, with the .lock held. This picks the machine with the fewestnumber of jobs running, which is listed highest in the mlist file in case of multiple machines with equal load. The job counter is incremented, and the file removed if the counter reaches ${maxjobs} for that machine. If all machines are busy, sleep for 15 seconds and retry. * After we have claimed a machine, we run claim-chroot on it to claim an empty chroot, as before. If the claim fails, release the job from the queue with the releasemachine script and retry after a 15 second wait. * When the build is finished, decrement the job counter with the releasemachine script, with .lock held. * The checkmachines script now exists only to poll the load averages for admin convenience (every 2 minutes), and to ping for unreachable machines. When a machine cannot be reached, remove the entry in the queue directory to stop further job dispatches to it. This needs more work to deal with reinitialization of machines after they become available again. Notes: svn path=/head/; revision=125317
* Overhaul of the job scheduler. The new scheduler runs buildsKris Kennaway2004-12-281-26/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | synchronously instead of probabilistically scheduling jobs, which means that the job load on a machine never exceeds a desired threshold, and we can preferentially use faster machines when they are available. This has a dramatic effect on package build throughput, although I don't yet have precise measurements of the performance improvements. Specifically, the changes are: * Introduce the new variable maxjobs in portbuild. This replaces the build scheduling weights previously listed in the mlist file, which now changes format to list the build machines only, ranked in order of preference for job dispatches (i.e. faster machines first). * The ${arch}/queue directory is used to list machines available for jobs (file content is the number of jobs currently running on the machine). Changes to files in this directory are serialized using lockf on the .lock file. * Claim a machine with the getmachine script, with the .lock held. This picks the machine with the fewestnumber of jobs running, which is listed highest in the mlist file in case of multiple machines with equal load. The job counter is incremented, and the file removed if the counter reaches ${maxjobs} for that machine. If all machines are busy, sleep for 15 seconds and retry. * After we have claimed a machine, we run claim-chroot on it to claim an empty chroot, as before. If the claim fails, release the job from the queue with the releasemachine script and retry after a 15 second wait. * When the build is finished, decrement the job counter with the releasemachine script, with .lock held. * The checkmachines script now exists only to poll the load averages for admin convenience (every 2 minutes), and to ping for unreachable machines. When a machine cannot be reached, remove the entry in the queue directory to stop further job dispatches to it. This needs more work to deal with reinitialization of machines after they become available again. Additional changes to this file: * Exit if passed a null package name, to avoid badness later on * Send a nag-mail if pkg-plist errors are detected in the build Notes: svn path=/head/; revision=125316
* * Catch up to change in formatting of 'extra files' section in logsKris Kennaway2004-12-281-4/+3
| | | | | | | * Other minor changes Notes: svn path=/head/; revision=125315
* /rescue/mount_linprocfs isn't functional when called byKris Kennaway2004-12-281-1/+2
| | | | | | | | | | | | /rescue/mount -t linprocfs, so assume that the i386 build hosts have statically-built copies of the necessary binaries in /sbin, until this is fixed. Create /usr/X11R6 inside the chroot so that mtree has something to do, since this directory is otherwise orphaned. Notes: svn path=/head/; revision=125314
* Support 5-exp buildsKris Kennaway2004-12-284-4/+14
| | | | Notes: svn path=/head/; revision=125313
* Use X_WINDOW_SYSTEM for 4.x builds too.Kris Kennaway2004-12-101-2/+2
| | | | Notes: svn path=/head/; revision=123622
* Belatedly catch up to XFREE86_VERSION -> X_WINDOW_SYSTEM change; applyKris Kennaway2004-12-101-3/+3
| | | | | | | the BSD.x11.dist if set to xfree86-3, otherwise use BSD.x11-4.dist. Notes: svn path=/head/; revision=123621
* The explanations of each build error no longer live on bento, they liveMark Linimon2004-12-092-7/+7
| | | | | | | on pointyhat. Notes: svn path=/head/; revision=123529
* Construct permanent links to the error logs here too.Kris Kennaway2004-12-051-4/+4
| | | | | | | | Submitted by: linimon Reminded by: marcus Notes: svn path=/head/; revision=123155
* Copy the statically-linked binaries from /rescue on the host, insteadKris Kennaway2004-12-051-4/+11
| | | | | | | | of needing to also include the dynamic libraries that were confusing some ports. We still need libc.so.[56] for killall to work, though :( Notes: svn path=/head/; revision=123154
* Be silent about attempts to unmount filesystems from within the chrootKris Kennaway2004-12-051-4/+4
| | | | Notes: svn path=/head/; revision=123153
* Clean up the symlink in ${pb}/${arch}/${branch}/latest/${dir} when we findKris Kennaway2004-11-251-0/+5
| | | | | | | a port that should not be listed here Notes: svn path=/head/; revision=122365
* If sudo_cmd is set, assume we need to chown the files we created onKris Kennaway2004-11-251-0/+3
| | | | | | | the client to the client_user so they can later write to them Notes: svn path=/head/; revision=122362
* Rework the -incremental support slightly; this should now be robust enoughKris Kennaway2004-11-251-3/+5
| | | | | | | to use. Notes: svn path=/head/; revision=122360
* Catch up to the changed format of the logs.Kris Kennaway2004-11-251-4/+4
| | | | | | | | | | List the extra/removed/changed files separately, and colour-code the serious errors (files left behind outside of /usr/local and /usr/X11R^; files removed that were installed by another port, and files with changed permissions or ownership) Notes: svn path=/head/; revision=122357
* * Recursively list all files detected as having been left behind afterKris Kennaway2004-11-251-7/+22
| | | | | | | | | | the port deinstall; mtree does not recurse into subdirectories it does not know about * Break out the 'files incorrectly removed' and 'files incorrectly changed' into their own sections Notes: svn path=/head/; revision=122354
* * Fix INDEX build if MAINTAINER is not defined. [1]Kirill Ponomarev2004-11-191-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Remove USE_QT2 since it's obsolete now. [2] * Clarify comments about ARCH. [3] * Speedup 'make readmes'. Add a perl script "Tools/make_readmes" and modify bsd.port.subdir.mk to avoid recursing into individual port directories to create README.html. [4] * Fix 'make search' to allow case insensitive search on 5-x/6-x. [5] * Add the possibility to search the ports by category. [6] * Remove tk42 and tcl76 from virtual categories since they're obsolete. [7] * Introduce new variable - DISTVERSION, vendor version of the distribution, that can be set instead of PORTVERSION and is automatically converted in a conforming PORTVERSION. [8] * Use --suffix instead of -b option for patch(1) to make it compatible with BSD patch(1) [9] * Fix {WANT,WITH}_MYSQL_VER behavior, to deal with conflicting versions. [10] PR: ports/68895 [1], ports/69486 [2], ports/68539 [3], ports/70018 [4], ports/68896 [5], ports/73299 [6], ports/73570 [7], ports/67171 [8], ports/72182 [9] Submitted by: linimon [1][3], arved [2][7], cperciva [4], Matthew Seaman <m.seaman@infracaninophile.co.uk> [5], Radek Kozlowski <radek@raadradd.com> [6], eik [8], Andreas Hauser <andy-freebsd@splashground.de> [9], clement [10] Notes: svn path=/head/; revision=121961
* Newline should go after the terminating punctuation (fix typo).Dima Dorfman2004-11-071-1/+1
| | | | | | | Approved by: will Notes: svn path=/head/; revision=121024
* Fix a couple of bugs in the extraction of this from dopackages, andKris Kennaway2004-10-251-33/+20
| | | | | | | | flatten the setupnode() function into the main body since it's only called from one place. Notes: svn path=/head/; revision=120107
* Redefine the -norestr option to mean 'don't build the list ofKris Kennaway2004-10-251-78/+73
| | | | | | | | | | | | | restricted ports' instead of 'don't build any restricted ports' since the former is useful when we're not intending to publish the results of a build, but the latter is not. Move the build preprocessing (directory setup, old build rotation, etc) out from under -nobuild, so that we can set up a new build using that option. Notes: svn path=/head/; revision=120106
* If the symlink under latest/ exists, then use that to find the most recentKris Kennaway2004-10-251-1/+6
| | | | | | | | error log for the port, instead of assuming it exists in the current build. Notes: svn path=/head/; revision=120105
* We now also do 6.x builds in the '6' branch.Kris Kennaway2004-10-251-3/+3
| | | | Notes: svn path=/head/; revision=120104
* Use the absolute path to nc(1) to deal with it not being in the PATH.Kris Kennaway2004-10-251-3/+1
| | | | | | | Remove useless sleep(1). Notes: svn path=/head/; revision=120103
* Clean up the 'latest error log' symlink when the build is successful.Kris Kennaway2004-10-251-0/+4
| | | | Notes: svn path=/head/; revision=120102
* When a port build fails, create a symlink to the error log underKris Kennaway2004-10-251-3/+4
| | | | | | | | | | ${arch}/${branch}/latest/${portdir}. We will use this in the processfail script, so that the "new package build errors" webpages do not have out-of-date links but instead link to the most recent copy of the build error. Notes: svn path=/head/; revision=120101
* Strip out date/time from +++/--- headers in generated patchfiles, so that theyMaxim Sobolev2004-10-141-11/+12
| | | | | | | don't generate extra hunk each time patchfile is updated. Notes: svn path=/head/; revision=119419
* Support ${client_user} and ${sudo_cmd} for running commands as root onKris Kennaway2004-09-261-1/+5
| | | | | | | | | | machines where we first connect to a non-root user and then increase privileges. Approved by: portmgr (self) Notes: svn path=/head/; revision=118451
* * Break out setupnode into the dosetupnode script so it may be calledKris Kennaway2004-09-261-37/+3
| | | | | | | | | | | manually * Support 6.x builds Approved by: portmgr (self) Notes: svn path=/head/; revision=118450
* Break out the setupnode function of dopackages into a separate script soKris Kennaway2004-09-261-0/+47
| | | | | | | | | | | | | | | | | | that it may be called by hand. Support new portbuild.conf variables client_user = user to connect to on the client (not necessarily root). This user must have write permission to the /var/portbuild tree if disconnected=1 (i.e. we're going to run rsync). rsync_gzip = set to "-z" to enable compression on low-bandwidth disconnected clients. Approved by: portmgr (self) Notes: svn path=/head/; revision=118449
* * Bail out if we can't claim a chroot directory in the client (e.g. ifKris Kennaway2004-09-261-13/+17
| | | | | | | | | | | | | | | | | | ssh times out) * Support new portbuild.conf settings: client_user = user to connect to on the client (not necessarily root) sudo_cmd = If ssh'ing to a non-root user, run this command to gain root privs (set to empty string for client_user=root, or sudo for !root). Cannot require interactivity, of course. Approved by: portmgr (self) Notes: svn path=/head/; revision=118448
* Refuse to "clean" / if something goes wrong upstream and we are passedKris Kennaway2004-09-261-0/+5
| | | | | | | | | the wrong path Approved by: portmgr (self) Notes: svn path=/head/; revision=118447
* Pull in portbuild.$(hostname) for per-client customizationKris Kennaway2004-09-261-1/+3
| | | | | | | Approved by: portmgr (self) Notes: svn path=/head/; revision=118446
* Support 6.x buildsKris Kennaway2004-09-262-1/+5
| | | | | | | Approved by: portmgr (self) Notes: svn path=/head/; revision=118445
* * Support 6.x buildsKris Kennaway2004-09-261-2/+12
| | | | | | | | | | * Default 4.x to XFree86 4.x, 5.x and above to x.org * Don't pick up local customizations from OPTIONS Approved by: portmgr (self) Notes: svn path=/head/; revision=118444
* Use a consistent PATH in both portbuild and dopackages.Joe Marcus Clarke2004-08-271-1/+1
| | | | Notes: svn path=/head/; revision=117441
* Avoid possible GNATS timeouts by using query-pr.real direclty.Kirill Ponomarev2004-07-281-1/+1
| | | | Notes: svn path=/head/; revision=114914
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-11 03:30:52 +0000