summaryrefslogtreecommitdiff
path: root/tools/buildman
AgeCommit message (Collapse)Author
2018-01-28host-tools: use python2 explicitly for shebangMasahiro Yamada
All of these host tools are apparently written for Python2, not Python3. Use 'python2' in the shebang line according to PEP 394 (https://www.python.org/dev/peps/pep-0394/). Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2018-01-16tools: Update python "help" tests to cope with "more" odditiesTom Rini
In some cases when "more" is told to page a given file it will prepend the output with: :::::::::::::: /PATH/TO/THE/FILE :::::::::::::: And when this happens the output will not match the expected length. Further, if we use a different pager we will instead fail the coverage tests as we will not have 100% coverage. Update the help test to remove the string in question. Reviewed-by: Simon Glass <sjg@chromium.org> Signed-off-by: Tom Rini <trini@konsulko.com>
2017-11-22buildman: Fix up testsSimon Glass
The tests were broken by two separate commits which adjusted the output when boards are listed. Fix this by adding back a PowerPC board and putting the name of each board in the test. Fixes: b9f7d881 (powerpc, 5xx: remove some "5xx" remains) Fixes: 8d7523c5 (buildman: Allow showing the list of boards with -n) Signed-off-by: Simon Glass <sjg@chromium.org>
2017-11-22buildman: Allow skipping of tests which use the networkSimon Glass
Accessing the network slows down the test and limits the environment in which it can be run. Add an option to disable network tests. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-09-29kconfiglib: update with 'imply' supportUlf Magnusson
Corresponds to 375506d (File writing nit) from upstream (https://github.com/ulfalizer/Kconfiglib). Adds proper 'imply' support and fixes a few minor issues, one of which previously triggered the following weird warning: configs/taurus_defconfig: /tmp/tmpisI45S:6: warning: assignment to SPL_LDSCRIPT changes mode of containing choice from "arch/$(ARCH)/cpu/u-boot-spl.lds" to "y" The change in 8639f69 (genconfig.py: Print defconfig next to warnings) was reapplied. tools/moveconfig.py previously depended on a hack that merged 'select's with 'imply's. It was modified to look at the union of Symbol.get_selected_symbols() and Symbol.get_implied_symbols(), which should give the same behavior. tools/genboardscfg.py was verified to produce identical board.cfg's before and after the change. Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2017-09-11genconfig.py: Print defconfig next to warningsSimon Glass
At present we sometimes see warnings of the form: /tmp/tmpMA89kB:36: warning: overriding the value of CMD_SPL. Old value: "y", new value: "y". This is not very useful as it does not show whch defconfig file it relates to. Update the tool to show this. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-06avr32: Retire AVR32 for goodAndy Shevchenko
AVR32 is gone. It's already more than two years for no support in Buildroot, even longer there is no support in GCC (last version is heavily patched 4.2.4). Linux kernel v4.12 got rid of it (and v4.11 didn't build successfully). There is no good point to keep this support in U-Boot either. Reviewed-by: Simon Glass <sjg@chromium.org> Reviewed-by: Tom Rini <trini@konsulko.com> Reviewed-by: Heiko Schocher <hs@denx.de> Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
2017-06-16powerpc, 5xx: remove some "5xx" remainsHeiko Schocher
we removed 5xx support. So delete some forgotten remains. Signed-off-by: Heiko Schocher <hs@denx.de>
2017-06-09buildman: properly translate strings for log and err files to ASCIIDaniel Schwierzeck
The build output can still produce unicode encoded output. But in the buildman's log and err files we only want plain ASCII characters. To handle all situations with unicode and non-unicode output, encode the stdout and stderr strings to UTF-8 and afterwards to ASCII with replacing all special characters. Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
2017-06-09buildman: disable localized and unicode output of all build toolsDaniel Schwierzeck
Build tools like Make, gcc or binutils support localized output or unicode encoded output dependent on the default system locale. This is not useful for buildman, where we want reproducible warning or error messages or where the output of binutils is further processed. Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
2017-06-08buildman: Fix bloat option when 'new' only drops functionsTom Rini
In the case where a new build only decreases sizes and does not increase any size we still want to report what functions have been dropped when doing a bloat comparison. Cc: Simon Glass <sjg@chromium.org> Signed-off-by: Tom Rini <trini@konsulko.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2017-04-18buildman: Allow 'gnueabihf' toolchains for ARMTom Rini
Many toolchains for ARM use the 'gnueabihf' suffix rather than just 'gnueabi', so allow these to be used, but with a lower priority than 'gnueabi' ones. Cc: Simon Glass <sjg@chromium.org> Signed-off-by: Tom Rini <trini@konsulko.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2017-04-14buildman: Translate more strings to latin-1Tom Rini
When writing out some of our results we may now have UTF-8 characters in there as well. Translate these to latin-1 and ignore any errors (as this is for diagnostic and given the githash anything else can be reconstructed by the user. Reviewed-by: Simon Glass <sjg@chromium.org> Signed-off-by: Tom Rini <trini@konsulko.com>
2017-04-13buildman: Handle commit subjects containing unicodeSimon Glass
One of these has crept in in this commit: 40a808f1 ARCv2: SLC: Make sure busy bit is set properly on SLC flushing Adjust buildman to handle it. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-03-09kconfiglib.py: Kludge in 'imply' supportTom Rini
Currently upstream does not yet understand the imply keyword. For what we use kconfiglib.py for today, this is OK. We only need to be able to evaluate in order to make boards.cfg and none of those choices will depend on how imply evaluates out. Signed-off-by: Tom Rini <trini@konsulko.com>
2017-02-08buildman: Allow showing the list of boards with -nSimon Glass
As well as showing the number of boards, allow showing the actual list of boards that would be built, if -v is provided. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-01-28buildman: Update link to the most recent prebuilt ARC toolachinAlexey Brodkin
To troubleshoot unexpected bhavior during building and what's more important during execution it is strongly recommended to use recent ARC toolchain, and so we're now referring to arc-2016.09 which is the latest as of today. Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2016-12-02buildman: Rename do_build to config_onlySimon Glass
This variable name is needlessly confusion. Adjust it to use a 'positive' name instead. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-12-02buildman: Clean up odd characters on the terminalSimon Glass
At present buildman leaves behind a few characters during its progress updates, which looks odd. Fix it. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-12-02buildman: Squash useless output from -KSimon Glass
When using #define CONFIG_SOME_OPTION, the value it set to '1'. When using defconfig (i.e. CONFIG_SOME_OPTION=y) the value is set to 'y'. This results in differences showing up with -K. These differences are seldom useful. Adjust buildman to suppress these differences by default. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-12-02buildman: Add documentation for CONFIG checkingSimon Glass
The -K option is not mentioned in the README at present. Add some notes to describe how this is used. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-12-02buildman: Add an option to just create the configSimon Glass
Normally buildman does a full build of a board. This includes creating the u-boot.cfg file which contains all the configuration options. Buildman uses this file with the -K option, to show differences in effective configuration for each commit. Doing a full build of U-Boot just to create the u-boot.cfg file is wasteful. Add a -D option which causes buildman to only create the configuration. This is enough to support use of -K and can be done much more quickly (typically 5-10 times faster). Signed-off-by: Simon Glass <sjg@chromium.org>
2016-11-06buildman: Fix building based on 'options' fieldTom Rini
The README for buildman says that we can use any field in boards.cfg to decide what to build. However, we were not saving the options field correctly. Cc: Simon Glass <sjg@chromium.org> Signed-off-by: Tom Rini <trini@konsulko.com> Acked-by: Simon Glass <sjg@chromium.org>
2016-10-09tools: buildman: Add compiler wrapperYork Sun
Now we can use compiler wrapper such as ccache or distcc for buildman. Signed-off-by: York Sun <york.sun@nxp.com> CC: Simon Glass <sjg@chromium.org> Acked-by: Simon Glass <sjg@chromium.org>
2016-10-09tools: buildman: Remove duplicated codeYork Sun
Signed-off-by: York Sun <york.sun@nxp.com> CC: Simon Glass <sjg@chromium.org> Fixed commit subject: Signed-off-by: Simon Glass <sjg@chromium.org> Acked-by: Simon Glass <sjg@chromium.org>
2016-10-09buildman: Drop the 'alive' flag in BuilderThreadSimon Glass
This is not used, so drop it. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09buildman: Don't show a stacktrace on Ctrl-CSimon Glass
When Ctrl-C is pressed, just exited quietly. There is no sense in displaying a stack trace since buildman will always be in the same place: waiting for threads to complete building all the jobs on the queue. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09buildman: Drop the 'active' flag in the builderSimon Glass
This serves no real purpose, since when we are not active, we exit. Drop it. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09buildman: Allow builds to terminate cleanlySimon Glass
It is annoying that buildman does not respond cleanly to Ctrl-C or SIGINT, particularly on machines with lots of CPUS. Unfortunately queue.join() blocks the main thread and does not allow it to see the signal. Use a separate thread instead, Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09buildman: Put our local libraries first in the pathSimon Glass
If patman is installed on the machine (e.g. in the standard dist-packages directory), it will find libraries from there in preference to our local libraries. Adjust the order of the path to ensure that local libraries are found first. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09buildman: Print a message indicating the build is startingSimon Glass
Make it clear when buildman actually starts building. This happens when it has prepared the threads, working directory and output directories. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09buildman: Print a message when removing old directoriesSimon Glass
When buildman starts, it prepares its output directory by removing any old build directories which will not be used this time. This can happen if a previous build left directories around for commit hashes which are no-longer part of the branch. This can take quite a while, so print a message to indicate what is going on. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09buildman: Tidy up the 'cloning' messageSimon Glass
On a machine with a lot of CPUs this prints a lot of useless lines of the form: Cloning repo for thread <n> Adjust the output so that these all appear on one line, and disappear when the cloning is complete. Note: This cloning is actually unnecessary and very wasteful on disk space (about 3.5GB each time). It would be better to create symlinks. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-09-18patman: Adjust command.Output() to raise an error by defaultSimon Glass
It is more useful to have this method raise an error when something goes wrong. Make this the default and adjust the few callers that don't want to use it this way. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31buildman: Add a quick-start noteSimon Glass
For those who just want to build a board, it is useful to see a quick hint right at the start of the documentation. Add a few commands showing how to download toolchains and build a board. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31buildman: Avoid overwriting existing toolchain entriesSimon Glass
The current code for setting up the toolchain config always writes the new paths to an item called 'toolchain'. This means that it will overwrite any existing toolchain item with the same name. In practice, this means that: buildman --fetch-arch all will fetch all toolchains, but only the path of the final one will be added to the config. This normally works out OK, since most toolchains are the same version (e.g. gcc 4.9) and will be found on the same path. But it is not correct and toolchains for archs which don't use the same version will not function as expected. Adjust the code to use a complete glob of the toolchain path. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31buildman: Drop the toolchain error when downloading toolchainsSimon Glass
It doesn't make sense to complain about missing toolchains when the --fetch-arch option is being used. The user is presumably aware that there is a toolchain problem and is actively correcting it by running with this option. Refactor the code to avoid printing this confusing message. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31buildman: Fix a typo in TestSettingsHasPath()Simon Glass
The function comment should say 'buildman'. Fix it. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31buildman: Improve the toolchain progress/error outputSimon Glass
Use colour to make it easier to see what is going on. Also print a message before downloading a new toolchain. Mention --fetch-arch in the message that is shown when there are no available toolchains, since this is the quickest way to resolve the problem. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31buildman: Allow the toolchain error to be suppressedSimon Glass
When there are no toolchains a warning is printed. But in some cases this is confusing, such as when the user is fetching new toolchains. Adjust the function to supress the warning in this case. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31buildman: Fix the 'help' test to use the correct pathSimon Glass
When buildman is run via a symlink, this test fails. Fix it to work the same way as buildman itself. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31buildman: Automatically create a config file if neededSimon Glass
If there is no ~/.buildman file, buildman currently complains and exists. To make things a little more friendly, create an empty one automatically. This will not allow things to be built, but --fetch-arch can be used to handle that. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31buildman: Tidy up the README a littleSimon Glass
Tidy up some problems found by a recent review. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-05-17buildman: allow more incremental buildingStephen Warren
One use-case for buildman is to continually run it interactively after each small step in a large refactoring operation. This gives more immediate feedback than making a number of commits and then going back and testing them. For this to work well, buildman needs to be extremely fast. At present, a couple issues prevent it being as fast as it could be: 1) Each time buildman runs "make %_defconfig", it runs "make mrproper" first. This throws away all previous build results, requiring a from-scratch build. Optionally avoiding this would speed up the build, at the cost of potentially causing or missing some build issues. 2) A build tree is created per thread rather than per board. When a thread switches between building different boards, this often causes many files to be rebuilt due to changing config options. Using a separate build tree for each board would avoid this. This does put more strain on the system's disk cache, but it is worth it on my system at least. This commit adds two command-line options to implement the changes described above; -I ("--incremental") turns of "make mrproper" and -P ("--per-board-out-dir") creats a build directory per board rather than per thread. Tested: ./tools/buildman/buildman.py tegra ./tools/buildman/buildman.py -I -P tegra ./tools/buildman/buildman.py -b tegra_dev tegra ./tools/buildman/buildman.py -b tegra_dev -I -P tegra ... each once after deleting the buildman result/work directory, and once "incrementally" after a previous identical invocation. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Tom Rini <trini@konsulko.com> Acked-by: Simon Glass <sjg@chromium.org> # v1 Tested-by: Simon Glass <sjg@chromium.org> # v1 Acked-by: Simon Glass <sjg@chromium.org>
2016-03-17buildman: Clarify the use of -VSimon Glass
This option outputs to the log file, not to the terminal. Clarify that in the help, and add a mention of it in the README. Signed-off-by: Simon Glass <sjg@chromium.org> Reported-by: Stephen Warren <swarren@wwwdotorg.org> Reviewed-by: Stephen Warren <swarren@nvidia.com>
2016-03-17buildman: Add a way to specific a full toolchain prefixSimon Glass
At present buildman allows you to specify the directory containing the toolchain, but not the actual toolchain prefix. If there are multiple toolchains in a single directory, this can be inconvenient. Add a new 'toolchain-prefix' setting to the settings file, which allows the full prefix (or path to the C compiler) to be specified. Update the documentation to match. Suggested-by: Stephen Warren <swarren@wwwdotorg.org> Reviewed-by: Joe Hershberger <joe.hershberger@ni.com> Signed-off-by: Simon Glass <sjg@chromium.org> Tested-by: Stephen Warren <swarren@nvidia.com>
2016-03-17buildman: Allow branch names which conflict with directoriesSimon Glass
At present if you try to use buildman with the branch 'test' it will complain that it is unsure whether you mean the branch or the directory. This is a feature of the 'git log' command that buildman uses. Fix it by resolving the ambiguity. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-14buildman: Allow the toolchain architecture to be specifiedSimon Glass
At present the architecture is deduced from the toolchain filename. Allow it to be specified by the caller. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Joe Hershberger <joe.hershberger@ni.com
2016-03-14buildman: Allow the toolchain priority to be specifiedSimon Glass
At present the priority of a toolchain is calculated from its filename based on hard-coded rules. Allow it to be specified by the caller. We will use this in a later patch. Also display the priority and provide a message when it is overriden by another toolchain of higher priority. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-14buildman: Fix up a few code inconsistencies in toolchain.pySimon Glass
Normally we use a single quote for strings unless there is a reason not to (such as an embedded single quote). Fix a few counter-examples in this file. Also add a missing function-argument comment. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>