aboutsummaryrefslogtreecommitdiff
path: root/Tools
Commit message (Collapse)AuthorAgeFilesLines
* Add a mechanism for reporting client machine error conditions back to theKris Kennaway2006-02-151-1/+14
| | | | | | | | | | | | | | server. Error conditions are flagged by other processes by creating a named dotfile in ${scratchdir}. If these files are found, report the error status instead of the number of running jobs. Currently report "ERR" for all error conditions; I will probably change this to a per-condition message. Currently only "squid not running" and "disk space low" conditions are reported. Notes: svn path=/head/; revision=156098
* Trim some unused cruftKris Kennaway2006-02-151-69/+12
| | | | | | | | | | If the package copy fails, bail out immediately instead of later on when we try to pkg_add it. Also trap signals and bail out. Both conditions will cause a retry of the package build. Notes: svn path=/head/; revision=156096
* Save the output of portbuild run on the client to a temporary log file.Kris Kennaway2006-02-151-1/+11
| | | | | | | | | | | | | If portbuild bailed out unexpectedly, mail the log to ${mailto}. Add some XXX comments about improving robustness of this script. Sleep for 2 minutes before retrying builds, to avoid spamming ${mailto} with a high rate of failure logs. In future we might be smarter about attempting to automatically correct common failure modes. Notes: svn path=/head/; revision=156095
* Add some sanity checking of the build environment:Kris Kennaway2006-02-151-1/+30
| | | | | | | | | | | | | | * Test whether squid is running. If not, try to kick off the rc script in the background in case it can be restarted cleanly. * Test for at least 100MB of free space on the scratch partition. If either condition fails, set an exception flag and bail out. This will be reported back to the server via reportload. Notes: svn path=/head/; revision=156092
* * Don't try to pkg_delete packages that are not installed (the usualKris Kennaway2006-02-151-6/+9
| | | | | | | | | | | | | cause is because it was specified in the list twice) * Don't panic when the list of packages to delete becomes empty * When unexpected filesystem changes are detected, bail immediately instead of proceeding and hiding the error in the middle of the log Notes: svn path=/head/; revision=156091
* Add support for 6-exp builds (FreeBSD 6.x is our new reference platform)Kris Kennaway2006-02-154-4/+14
| | | | Notes: svn path=/head/; revision=156090
* * Switch back to using pkg_delete -f; we are not yet ready to dealKris Kennaway2006-02-131-25/+26
| | | | | | | | | | | | with all the errors from broken pkg_delete scripts * As threatened in previous commit, move the pristine mtree spec generation to phase 1, and avoid having to delete and re-add the FETCH_DEPENDS. We still have to keep them installed until after 'make extract' though Notes: svn path=/head/; revision=155961
* * In kill_procs(), pass in the directory and mountpoint as separateKris Kennaway2006-02-111-2/+8
| | | | | | | | | | | | | | arguments (cosmetic) * Detect if a chroot was used to run a jailed build, and first attempt to gracefully shut it down by killing everything within using pgrep(1) This has a much higher chance of succeeding that relying on fstat to identify processes that might interfere with our attempts to clean up mountpoints, which is fragile (libkvm-dependent), and inherently unreliable at best. Notes: svn path=/head/; revision=155730
* Support for building ports in a jail instead of a chroot. If use_jail=1Kris Kennaway2006-02-111-9/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | in portbuild.conf (or per-machine .conf), then construct a 127.0.0.0/8 IP address based on the build directory ID (i.e. unique for each build instance). This is bound to the lo0 interface for the duration of the 'phase 2' build. We cannot build 'phase 1' in a jail since 'make fetch' doesn't always work through a proxy (e.g. squid sometimes mangles files fetched through FTP, I think by performing CR/LF translation in FTP ASCII mode). Pass in the HTTP_PROXY variable to the jail, if set. This allows FTP/HTTP access from within the jail if the proxy is suitably configured (some ports legitimately need to fetch additional files during the build, e.g. if they have a BUILD_DEPENDS=...:configure target that needs to fetch additional distfiles). Not all ports can be built in jails (most notably the linux_base ports since they want to mount/umount linprocfs), so we will need to come up with a way to deal with this. Some ports require SYSV IPC, so security.jail.sysvipc_allowed=1 might be required. Some other ports attempt to perform DNS lookups, ping, or outbound TCP connections during the build. When it works, this provides better compartmentalization of package builds, e.g. easier termination of builds without the possibility of daemonized processes staying active; no possibility of accidental interference between jails, etc. It also allows for admin monitoring using jls(1). Notes: svn path=/head/; revision=155729
* * Use ${mailto} portbuild.conf variable instead of hard-coded list [1]Kris Kennaway2006-02-111-4/+7
| | | | | | | | | | | * Remove old logs and possible compressed logs before attempting the build Requested by: lofi [1] Submitted by: linimon [1] No more accidental portbuild spam: kris and krion [1] Notes: svn path=/head/; revision=155728
* Use ${mailto} portbuild.conf variable instead of hard-coded listKris Kennaway2006-02-111-1/+1
| | | | | | | | | Requested by: lofi Submitted by: linimon No more accidental portbuild spam: kris and krion Notes: svn path=/head/; revision=155727
* Checkpoint a major round of changes:Kris Kennaway2006-02-111-64/+201
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Only keep distfiles if the port passes 'make fetch', so we don't accidentally keep files with invalid checksums * Use cleanup() instead of directly exiting in some error conditions * When cleanup() is called indicating an unexpected error (possibly leaving the filesystem in an inconsistent state), mark the chroot as dirty so it will not be reused by another build * Remove packages in dependency order instead of with pkg_delete -f in possibly incorrect order. This paves the way for focusing on errors generated by pkg_delete (e.g. @dirrm that should be @dirrmtry) in the future. [1] * Detect when packages were left behind because they were still in use by other packages, indicating an incorrect or incomplete port dependency list * Partial support for ccache builds (not yet complete) * Support non-standard LOCALBASE/X11BASE settings * Delete FETCH_DEPENDS after the 'make fetch' stage. We have to add them again before 'make extract' since, due to a lack of a 'fetch cookie', 'make extract' actually *always* runs 'make fetch' again, even when distfiles have already been fetched. We need to delete them in order to: * Record an mtree spec of the 'pristine' filesystem state, for later comparison. # XXX Perhaps this can be done in stage 1 before the # 'make fetch', removing the need to delete-and-readd. * Also record an mtree spec of the filesystem state prior to the build phase. Compare this to the state of the filesystem immediately before running the install phase, to detect files that were inappropriately installed during the build phase. Doing so is a fatal error. * Prior to installing, try to run a 'regression-test' port makefile target, if it exists. This allows ports to hook their internal regression suites into the package build. This needs further infrastructure support, e.g. a default NOP target in bsd.port.mk. For now this is run with 'make -k', so regression failures will not yet actually cause package build failures. * Separate the 'make install' from 'make package' phases rather than let the latter implicitly do the install. * After the newly packaged port has been deleted, compare the state of the filesystem to the state before 'make install'. * After removing BUILD and RUN dependencies, compare the filesystem state to the pristine state before the start of the build. This also detects package dependencies that did not clean themselves up properly when deinstalling. It also detects dependencies that were 'missing' from the port INDEX: these were not pkg_added into place, so the package build had to compile them from scratch (a big waste of time and effort), so this is now also a fatal error. PR: ports/85746 (inspired by) [1] Submitted by: Boris B. Samorodov <bsam@ipt.ru> [1] Notes: svn path=/head/; revision=155726
* Flip from noclean to cleanKris Kennaway2006-02-031-3/+3
| | | | Notes: svn path=/head/; revision=155147
* Don't bother chowning the client /var/portbuild/, it just takes too longKris Kennaway2006-02-031-3/+0
| | | | | | | and this doesn't happen in practise unless you screwed up manually. Notes: svn path=/head/; revision=155146
* Use clean-chroot with clean=2 to remove the chroot instead of duplicatingKris Kennaway2006-02-031-5/+1
| | | | | | | some of the code Notes: svn path=/head/; revision=155145
* Flip "noclean" to "clean"Kris Kennaway2006-02-031-3/+3
| | | | Notes: svn path=/head/; revision=155144
* Instead of passing in a "noclean" boolean, pass in a "clean" level:Kris Kennaway2006-02-031-4/+11
| | | | | | | | | 0 = don't clean chroot 1 = sanitize commonly-modified directories 2 = remove entire chroot Notes: svn path=/head/; revision=155143
* Also look for chroots that are marked 'dirty' (previous build left chrootKris Kennaway2006-02-031-1/+5
| | | | | | | in unclean state) and run clean-chroot on them to remove them. Notes: svn path=/head/; revision=155142
* Don't try to remove the lock file now that we use lockf to manage itKris Kennaway2006-02-031-2/+0
| | | | | | | Noticed by: linimon Notes: svn path=/head/; revision=155141
* Fix bogus comments. No code change.Mark Linimon2006-02-021-5/+5
| | | | Notes: svn path=/head/; revision=155040
* Correctly set SRCBASE; the version inherited from buildenv is wrong forKris Kennaway2006-01-271-0/+1
| | | | | | | the client chroots. Notes: svn path=/head/; revision=154617
* Don't try to rsync docs, we no longer need to distribute themKris Kennaway2006-01-271-1/+0
| | | | Notes: svn path=/head/; revision=154616
* Update the list of quickportsKris Kennaway2006-01-271-1/+1
| | | | Notes: svn path=/head/; revision=154615
* Warn if there are duplicate MD5/SHA256/SIZE statements for a single fileEdwin Groothuis2006-01-241-3/+33
| | | | Notes: svn path=/head/; revision=154391
* Small tool to check the necessary lines in the distinfo files, basedEdwin Groothuis2006-01-241-0/+105
| | | | | | | | | | | | | | | | | | on the contents of the distinfo files. # # Small tool to find distinfo with missing MD5/SHA256/SIZE statements, # based on the assumption that if there is one of the MD5/SHA256/SIZE # statements, then there should be all of them (except for SIZE # when MD5/SHA256 is set to IGNORE). # # Usage: distinfochecker [-v] [-d directory] # -v - verbose (print) # -d - use directory instead of /usr/ports # Notes: svn path=/head/; revision=154320
* Support 7.x INDEX buildsKris Kennaway2006-01-221-16/+37
| | | | | | | | Add a few more local configuration variables Minor cleanups Notes: svn path=/head/; revision=154191
* Don't bother mounting doc tree, it's not needed by any portsKris Kennaway2006-01-221-12/+8
| | | | | | | | | Use LOCALBASE/X11BASE values from buildenv xorg uses the stock BSD.x11-4.dist now. Notes: svn path=/head/; revision=154130
* Remove -nodoccvs option, it turns out mounting the doc tree is notKris Kennaway2006-01-221-14/+1
| | | | | | | required by any ports Notes: svn path=/head/; revision=154129
* Also clean up linprocfs mountsKris Kennaway2006-01-221-0/+1
| | | | Notes: svn path=/head/; revision=154128
* Pull in buildenv for LOCALBASE/X11BASE locationsKris Kennaway2006-01-221-2/+4
| | | | | | | Don't bother cleaning up doc mounts, they are not needed. Notes: svn path=/head/; revision=154127
* Use LOCALBASE and X11BASE from buildenvKris Kennaway2006-01-221-1/+4
| | | | Notes: svn path=/head/; revision=154126
* Allow for the possibility of overriding LOCALBASE and X11BASEKris Kennaway2006-01-221-0/+2
| | | | Notes: svn path=/head/; revision=154125
* Exit with the correct error code when a make target failsKris Kennaway2006-01-151-3/+3
| | | | | | | | PR: ports/64930 Submitted by: Damir Kiramov <damirycha@damirycha.net.ru> Notes: svn path=/head/; revision=153536
* Back out last commit, it was the wrong fileKris Kennaway2006-01-151-17/+15
| | | | Notes: svn path=/head/; revision=153535
* Exit with the correct error code when a make target failsKris Kennaway2006-01-151-15/+17
| | | | | | | | PR: ports/64930 Submitted by: Damir Kiramov <damirycha@damirycha.net.ru> Notes: svn path=/head/; revision=153531
* Add portsvar.sh schell script into Tools/scripts.Daichi GOTO2006-01-111-0/+133
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | portsvar.sh - print out variable defined through bsd.port.mk example: % portsvar.sh PORTSDIR OSVERSION PORTSDIR = /usr/ports OSVERSION = 600034 % portsvar.sh -e "PORTS[^_]" PORTSDIR = /usr/ports PORTSSUPFILE = /usr/share/examples/cvsup/ports-supfile % portsvar.sh USE_RUBY=yes RUBY_VER RUBY_ARCH RUBY_VER = 1.8 RUBY_ARCH = i386-freebsd6 % portsvar.sh -w ruby RUBY_VER RUBY_ARCH RUBY_VER = 1.8 RUBY_ARCH = i386-freebsd6 % portsvar.sh -w kde -e KDE_ KDE_MAINTAINER = will@FreeBSD.org KDE_ORIGVER = ${KDE_VERSION} KDE_PREFIX = ${LOCALBASE} KDE_VERSION = 3.5.0 MASTER_SITE_KDE_kde = ${kmaster:S@%SUBDIR%/@@g} % portsvar.sh -a list all variables through bsd.port.mk % portsvar.sh -a -w kde list all variables through bsd.port.mk including kde portsvar.sh is easy tool to check out the variables used in /usr/ports/Mk/bsd.*.mk. Notes: svn path=/head/; revision=153236
* This script may be run on a client machine to force it to flush its squidKris Kennaway2005-12-111-0/+15
| | | | | | | | cache. This is useful if the client is holding a bad or stale cached copy of a package that is causing problems for package builds. Notes: svn path=/head/; revision=150855
* Bump the file size limit to 300MB now that openoffice distfiles exceedKris Kennaway2005-12-111-5/+5
| | | | | | | 256MB. Clarify a comment. Notes: svn path=/head/; revision=150854
* Shuffle the order of a cdKris Kennaway2005-12-111-2/+1
| | | | Notes: svn path=/head/; revision=150853
* Remove 0-sized entries to avoid misbehaviourKris Kennaway2005-12-111-0/+4
| | | | Notes: svn path=/head/; revision=150852
* Don't use a lockfile, the script will be called with lockf -t0 instead toKris Kennaway2005-12-112-16/+0
| | | | | | | avoid problems with stale lockfiles after a reboot. Notes: svn path=/head/; revision=150851
* Don't use a lockfile, the script will be called with lockf -t0 insteadKris Kennaway2005-12-111-6/+0
| | | | | | | to avoid problems with stale lockfiles after reboots. Notes: svn path=/head/; revision=150850
* Update the list of quickportsKris Kennaway2005-12-111-1/+1
| | | | Notes: svn path=/head/; revision=150849
* Don't set -nocopy when -norsync is set, otherwise the setupnode scriptKris Kennaway2005-12-111-1/+1
| | | | | | | doesn't completely prep the client. Notes: svn path=/head/; revision=150848
* Don't use a lock file, the script will be called with lockf -t 0 instead.Kris Kennaway2005-12-111-6/+0
| | | | | | | This avoids problems from stale lockfiles after reboots. Notes: svn path=/head/; revision=150847
* Mount the md image async since this gives much better performanceKris Kennaway2005-12-111-1/+1
| | | | Notes: svn path=/head/; revision=150846
* Utility: A tool to identify similar pkg-descr filesEdwin Groothuis2005-11-261-0/+24
| | | | | | | | | | | | | The ports tree contains many tools with almost-similar names and no helpful pkg-descr difference. The goal of pkg-descr would be to aid the user in his choice of port version. Ergo, similar pkg-descr files are bad. PR: ports/89432 Submitted by: Jonathan <afarsec@012.net.il> Notes: svn path=/head/; revision=149525
* Remove, don't comment out, unused variablesKris Kennaway2005-10-111-2/+0
| | | | Notes: svn path=/head/; revision=144900
* Remove unused variables. Don't bother protecting against negativeKris Kennaway2005-10-111-4/+2
| | | | | | | queue lengths since that doesn't arise. Notes: svn path=/head/; revision=144899
* Update lockf to lockf -k in example usageKris Kennaway2005-10-111-1/+1
| | | | Notes: svn path=/head/; revision=144898
generated by cgit v1.2.3 (git 2.25.1) at 2025-02-21 06:13:25 +0000