Tinbuild2 - automatic building of LibreOffice ============================================= tinbuild2 is an evolution of tinbuild - it periodically pulls from the LibreOffice git, builds the thing, and optionally mails those who broke the build, or uploads the build results to http://dev-builds.libreoffice.org/daily/ Short version ------------- is a name you chose for you profile... usually use 'master' for a tinderbox that run the master branch. but you can call it anything you want. You can create different profile that do different kind of build, following the same procedure. Read http://wiki.documentfoundation.org/Development/Tinderbox . # clone tinbuild git clone git://gerrit.libreoffice.org/buildbot /lo/buildbot # clone LibreOffice git clone git://gerrit.libreoffice.org/core /lo/core # make the LibreOffice build cd /lo/core ./autogen.sh [ which ever argument are needed ] make make dev-install # after the above succeeded, configure buildbot mkdir -p ~/.tinbuild/config mkdir -p ~/.tinbuild/metadata/ cat > ~/.tinbuild/config/.cfg < SMTPUSER= SMTPPW= TINDER_NAME= # [ see https://wiki.documentfoundation.org/Development/Tinderbox for naming conventions ] TINDER_ID= # [ see https://wiki.documentfoundation.org/Development/Tinderbox for how to get an ID ] OWNER= METADATA_DIR=~/.tinbuild/meta/ # this can be any existing directory you want # if METADATA_DIR is not specified the tinderbox metadata are stored in the core repo itself GERRIT_PLATFORM=(MAC|LINUX|WINDOWS) # platform this TB is building when doing gerrit path verification EOF # save you autogen configuration. mkdir -p ~/.tinbuild/autogen cp /lo/core/autogen.lastrun ~/.tinbuild/autogen/.autogen # check the configuration cd /lo/core /lo/buildbot/bin/tinbuild2 -p -c # prime the tinderbox cd /lo/core /lo/buildbot/bin/tinbuild2 -p -z # and if that succeeds, start the periodical building cd /lo/core /lo/buildbot/bin/tinbuild2 -m all -p # to also build gerrit patches /lo/buildbot/bin/tinbuild2 -m all -p --mode=fair Configuration ------------- The configuration of tinbuild2 is stored in ~/.tinbuild/* tinbuild2 is 'profile' based. each profile comport a config file in ~/.tinbuild/config/.cfg and an assocaited autogen configuration for the build located at ~/.tinbuild/autogen/.autogen optionnaly each profile can source an 'extension/override' script located at ~/.tinbuild/phases/.sh that file, if it exsist will be sourced in tinbuild2 script. tinbuild2 maintain some metadata information in order to keep track ot the state of affair for a given profile. by default these files are sotre in the directory of the core repo, but if you specify a METADATA_DIR in the configuration of the profile then that directory will be used to store these files. Note: the METADATA_DIR, if specified, must exist. Using a METADATA_DIR allow to keep the repo clean, allow to use r/o repo, all to use git clean on the repo without risk of destroying the state of the tinderbox. config file ----------- the config file for a profile is located at ~/.tindbuild/config/.cfg The following parameter can od must be specified. Mandatory parameters: SMTPHOST : hostname of the smtp service to send email. SMTPUSER : user to pass to the smtp server to send email. SMTPPW : password for the smp service SMTPUSER and SMTPPW may be ommited if your smtp service connection without login. That is usually not the case. TINDER_NAME : name of yur tinderbox, please refer to https://wiki.documentfoundation.org/Development/Tinderbox for tinderbox naming conventions TINDER_ID : numerical id of your tinderbox. Please refer to https://wiki.documentfoundation.org/Development/Tinderbox for how to assign an id to your tinderbox OWNER : email address of the owner of this tinderbox. Optional parameters METADATA_DIR : directory where tinbuild2 will store the metadata information about the state of the tinderbox if specified, the directory must exist (it is _not_ created by tinbuild2) if this variable is not specified, then the metadata fiels are store in the root of the core repository. ARTIFACTDIR : git repo where to deliver binary build. tinbuild2 can collect binary build to form a bibisect git repo this parameter is mandatody if you specifyc the '-x' command line argument when running tinbuild2 BIBISECT_GC : Y/N. In case of collection of binary artifact, make tinbuild2 to run a git gc command after the collection of each new binary version BIBSECT_PUSH : Y/N. In case of collection of binary artifact, make tinbuild2 push the bibisect repo to its remote 'origin' after the collection of each new binary version GERRIT_PLATFORM : MAC/LINUX/WINDOWS. when using tinbuild2 to do gerrit patch verification, indicate which class of build that tinderbox build. This parameter may be needed if the auto-termination that tinbuild2 do using 'uname' is uncorrect, or possibly in the future to support cross-compile tinderbox. PAUSE_SECONDS : . Duration, in seconds, to pause after each build, before trying to do another one. The default value is 900 seconds (15 minutes). For a dedicated tindebox machine, this should probably be 60 or less. command-line syntax: tinbuild2 -p [options] -0 Do incremental build; that is do not invoke the clean phase between build. Not recommended for any mode of operation that involve gerrit. In general that should be reserved to pure tinderbox mode and that tinderbox need to be monitored closely to detect repeated failure due to problem with incremental build. -a for a run that do not report to the tinderbox server, but is used to to a 'private' test of a particular branch -- like a feature branch for instance -- this specify the email of the 'author' to notify of the tinderbox activity. this allow you to run a tinderbox to test a feature branch and automatically send the result to a given person. This work in conjonction with '-m author'. See below. -h print help. -c Execute a 'sanity check'. Verify that the configuration is sane and send a test email to the owner, to verify that the smtp mechanism works. -g To a test build using the given gerrit ref to checkout a specific commit A gerrit ref is in the form of : refs/changes///

Message are sent in the Review section of the assocaited gerrit patch to indicate that a build as started and to indicate the result of the build. This option imply a one shot build. The tinbuild2 terminate after that one build. -i run with ionice -c3 -k To not overrite autogen.lastrun if present. This allow to use -z to do a quick test of the current config -m all|tb|owner|author|none Set the email level in regular tindeebox run This does not apply to gerrit run or priming run. all : send email to the tinderbox server, to the owner and to all the committer involved in a failed build. tb : send email to the tinderbox server and the owner owner : send only tinderbox to the owner. author : send email to the specified author (see -a above) none : do not send email at all. --mode tb|gerrit|tb-gerrit|gerrit-tb|fair Mode of operation tb : normal tinderbox mode. Wait on new commit to show on core and build as new commits are found... gerrit : gerrit patch verification mode. Query gerrit for patch to test-build build them and report back, and repeat. tb-gerrit: a mix of tb and gerrit, where if there is no new commit to be build in tb mode, we attempt to get a task from gerrit This mode favor the tb mode, in the sens that gerrit task are only attempted if there is nothing left to do in tb mode gerrit-tb : like tb-gerrit, except the the scheduling favor gerrit in priority. fair : dual mode tb and gerrit like tb-gerrit, except that the scheduling priority is given alternatively to tb and gerrit. The default mode is tb. --mode is not to be used conjunction with -z or -g which are meant to trigger one-shot build. -n Run with 'nice' -r Indicate that we want to push daily upload with a maximum upload bandwidth as indicated in KB/s This will attempt to upload a build at most once a day. This require that you have obtained ssh keys to be able to upload to dev-build.libreoffice.org -t Run the full suite of test after a sucessfull build -v Print more message while tinbuild2 is running -x Push sucessfull binary build, at most once a day to a bibisect repo This require that ARTIFACTDIR be specified in the configuration of the profile and point to a bibisect git repo. -z Run a 'Priming' build. In order for tinbuild2 to be able to notify the appropriate subset of committers in case of failure, we need a sucessfull 'reference' build. You are required to run a sucessfull primer build before you can run tinbuild2 in a loop. This is required even if you do not actually email committers with failures. phases customization -------------------- The design is loosely inspired from Gentoo's portage. the idea is to use the ability of bash to redefine function. so the function to build the product, do_build, is implemented with tb_call() { declare -F "$1" > /dev/null && $1 } phase() { local f=${1} for x in {pre_,do_,post_}${f} ; do tb_call ${x} done } do_build() { retval=0 for p in autogen clean make test push ; do echo "call $p" phase $p done } so essentially the following functions will be called if implemented pre_autogen, do_autogen, post_autogen, pre_clean, do_clean, post_clean, pre_make, do_make, post_make, pre_test, do_test, post_test, pre_push, do_push, post_push Note that the implementation of these function need to test ${retval} to find out if a step before them had failed. if so they may decide to do nothing (most cases) or may decide to still do something. reciprocally, every function should put a non-zero value in retval in case of failure. Now, the base function are implemented in tinbuild_phases.sh but these function can be overrided by a user-implementation, typically store in ~/.tinbphuid/phases/.sh which is sourced by tinbuild2 if it exists. In the same spirit, platform dependent hack can be done in tinbuild_internals_.sh The core default is meant to run on Linux, using bash and all the GNU tools that are expected to normally be there on a Linux box. Other platform will re-implementent incompatible stuff in their own .sh