summaryrefslogtreecommitdiff
path: root/tools/moveconfig.py
AgeCommit message (Collapse)Author
2020-08-23moveconfig: Skip binary and ELF filesSimon Glass
Add a few more file extensions to the list of files that should not be processed. This avoids unicode errors, for example. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26patman: Tidy up sys.path changesSimon Glass
Now that we are using absolute paths we can remove some of the sys.path mangling that appears in the tools. We only need to add the path to 'tools/' so that everything can find modules relative to that directory. The special paths for finding pylibfdt remain. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26buildman: Move to absolute importsSimon Glass
At present buildman sets the python path on startup so that it can access the libraries it needs. If we convert to use absolute imports this is not necessary. Move buildman to use absolute imports. Also adjust moveconfig.py too since it uses some buildman modules and cannot work without this. Signed-off-by: Simon Glass <sjg@chromium.org>
2020-02-25moveconfig: convert ps.stderr to stringMarkus Klotzbuecher
Printing the error message in verbose mode fails, since python3 doesn't implicitely convert bytes to strings. Signed-off-by: Markus Klotzbuecher <mk@mkio.de> Cc: Simon Glass <sjg@chromium.org> Cc: Tom Rini <trini@konsulko.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2020-02-25moveconfig: replace unsafe eval with astevalMarkus Klotzbuecher
Commit b237d358b425 ("moveconfig: expand simple expressions") added support for expanding expressions in configs, but used the unsafe python built-in "eval". This patch fixes this by replacing eval with the asteval module. Signed-off-by: Markus Klotzbuecher <mk@mkio.de> Cc: Heinrich Schuchardt <xypron.glpk@gmx.de> Cc: Heiko Schocher <hs@denx.de> Cc: Tom Rini <trini@konsulko.com> Cc: Simon Glass <sjg@chromium.org> Cc: Joe Hershberger <joe.hershberger@ni.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2019-11-23moveconfig.py: Fix more Python3 UTF issuesTom Rini
With the move to using Python 3 for real, we encounter two different issues. First, the file include/video_font_data.h includes at least one UTF-16 character. Given that it does not include any CONFIG symbols it is easiest to just ignore this file. Next, we encounter similar problems with some dts/dtsi files that come from Linux. In this case it's easiest to simply ignore all dts/dtsi files as there will not be CONFIG symbols for us to migrate in them. Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Tom Rini <trini@konsulko.com>
2019-11-04move_config: Convert to Python 3Simon Glass
Convert this tool to Python 3 and make it use that, to meet the 2020 deadline. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-10-08kconfiglib: Update to the 12.14.0 releaseTom Rini
A large number of changes have happened upstream since our last sync which was to 375506d. The reason to do the upgrade at this point is for improved Python 3 support. As part of this upgrade we need to update moveconfig.py and genboardscfg.py the current API. This is: - Change "kconfiglib.Config" calls to "kconfiglib.Kconfig" - Change get_symbol() calls to syms.get(). - Change get_value() to str_value. Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Tom Rini <trini@konsulko.com>
2019-07-09moveconfig: expand simple expressionsMarkus Klotzbuecher
Add support for expanding simple expressions and sizes such as "(4 * 1024)", "(512 << 10)" or "(SZ_256K)". This can help to significantly reduce the number of "suspicious" moves, such as 'CONFIG_ENV_SIZE="(64 << 10)"' was removed by savedefconfig. If the expansion fails, it falls back to the original string. Signed-off-by: Markus Klotzbuecher <markus.klotzbuecher@kistler.com> Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Heiko Schocher <hs@denx.de>
2019-02-09moveconfig: add a second pass for empty #if/#endif blocksChris Packham
Moveconfig already attempts to remove empty #if/#endif blocks when there is a matching CONFIG_ being moved. Add a second pass which covers files without a match. Signed-off-by: Chris Packham <judge.packham@gmail.com>
2018-05-07SPDX: Convert all of our single license tags to Linux Kernel styleTom Rini
When U-Boot started using SPDX tags we were among the early adopters and there weren't a lot of other examples to borrow from. So we picked the area of the file that usually had a full license text and replaced it with an appropriate SPDX-License-Identifier: entry. Since then, the Linux Kernel has adopted SPDX tags and they place it as the very first line in a file (except where shebangs are used, then it's second line) and with slightly different comment styles than us. In part due to community overlap, in part due to better tag visibility and in part for other minor reasons, switch over to that style. This commit changes all instances where we have a single declared license in the tag as both the before and after are identical in tag contents. There's also a few places where I found we did not have a tag and have introduced one. Signed-off-by: Tom Rini <trini@konsulko.com>
2017-10-15moveconfig: fix error message in do_autoconf()Chris Packham
Move the % arch outside the double quote so that the missing toolchain message is displayed correctly. Signed-off-by: Chris Packham <judge.packham@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-09-29kconfiglib: update with 'imply' supportUlf Magnusson
Corresponds to 375506d (File writing nit) from upstream (https://github.com/ulfalizer/Kconfiglib). Adds proper 'imply' support and fixes a few minor issues, one of which previously triggered the following weird warning: configs/taurus_defconfig: /tmp/tmpisI45S:6: warning: assignment to SPL_LDSCRIPT changes mode of containing choice from "arch/$(ARCH)/cpu/u-boot-spl.lds" to "y" The change in 8639f69 (genconfig.py: Print defconfig next to warnings) was reapplied. tools/moveconfig.py previously depended on a hack that merged 'select's with 'imply's. It was modified to look at the union of Symbol.get_selected_symbols() and Symbol.get_implied_symbols(), which should give the same behavior. tools/genboardscfg.py was verified to produce identical board.cfg's before and after the change. Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2017-09-11moveconfig: Use fd.write() instead of print >>Simon Glass
Adjust this code so that it can work with Python 2 and 3. Fixes: d73fcb1 (moveconfig: Support building a simple config database) Reported-by: Chris Packham <judge.packham@gmail.com> Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-22RFC: moveconfig: Use toolchains from buildmanSimon Glass
It is annoying to have to set up and maintain two sets of toolchains, one for buildman and one for moveconfig. Adjust moveconfig to make use to buildman's toolchains. This should make things easier. One missing feature is the ability to specify the toolchain on the command line with a special environment variable, e.g. CROSS_COMPILE_ARM. I'm not sure if that is useful, but if it is it could be implemented in buildman. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com> Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
2017-07-22moveconfig: Tidy up imply flag parsingSimon Glass
Add an option to specify 'all' to enable all flags. Also print an error if an unrecognised flag is used. At present it just prints usage information which is not very helpful. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-11moveconfig: Allow automatic location and adding of 'imply'Simon Glass
By using a Kconfig parser we can find the location of each option in the Kconfig tree. Using the information from the database we can then automatically add an 'imply' option into the right place if requested by the user. Add a -a option to support adding 'imply' options. Display the location of any existing 'imply' option so that progress can be examined. Add a -A option to hide any existing 'imply' options so that already-completed additions need not be considered further. Also add documentation for this feature. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-11moveconfig: Allow control of which implying configs are shownSimon Glass
Sometimes it is useful to display CONFIG_TARGET or CONFIG_CMD configs. Add an option to control this. Also we generally ignore implying configs which affect fewer than 5 boards. But sometimes it is useful to show those those, so add an option that reduces the minimum to two. ERRATUM configs are never useful for implying things, so ignore those. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-11moveconfig: Allow piping in 'git show --stat' outputSimon Glass
It is useful to be able to process only a subset of boards to save time. Often that subset is defined by the defconfig files in a git commit. This change allows things like: # Build the database ./tools.moveconfig.py -b # Find some implying configs ./tools/moveconfig.py -i CONFIG_X # Add some 'imply' statements to Kconfig files ./tools/moveconfig.py -i CONFIG_X -a CONFIG_A,CONFIG_B # Reprocess the defconfig files to see if we can drop some changes git show --stat | ./tools/moveconfig.py -s -d - # Update the commit, with fewer defconfig changes gii commit -au Where the commit contains defconfig files, this will reprocess them to take account of the imply statements that you added. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-11moveconfig: Support looking for implied CONFIG optionsSimon Glass
Some CONFIG options can be implied by others and this can help to reduce the size of the defconfig files. For example, CONFIG_X86 implies CONFIG_CMD_IRQ, so we can put 'imply CMD_IRQ' under 'config X86' and all x86 boards will have that option, avoiding adding CONFIG_CMD_IRQ to each of the x86 defconfig files. Add a -i option which searches for such options. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Heiko Schocher <hs@denx.de>
2017-07-11moveconfig: Support building a simple config databaseSimon Glass
Add a -b option which scans all the defconfigs and builds a database of all the CONFIG options used by each. This is useful for querying later. At present this only works with the separate -b option, which does not move any configs. It would be possible to adjust the script to build the database automatically when moving configs, but this might not be useful as the database does not change that often. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-11moveconfig: Add a constant for auto.confSimon Glass
This filename is used a few times. Move it to a constant before adding further uses. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-11moveconfig: Tidy up the documentation and add hintsSimon Glass
The newest clean-up features are not mentioned in the docs. Fix this and add a few hints for particular workflows that are hopefully helpful. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-11moveconfig: Allow reading the defconfig list from stdinSimon Glass
Support passes in a defconfig filename of '-' to read the list from stdin instead of from a file. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-11moveconfig: Support providing a path to the defconfig filesSimon Glass
It is convenient to provide the full patch to the defconfig files in some situations, e.g. when the file was generated by a shell command (e.g. 'ls configs/zynq*'). Add support for this, and move the globbing code into a function with its own documentation. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-06avr32: Retire AVR32 for goodAndy Shevchenko
AVR32 is gone. It's already more than two years for no support in Buildroot, even longer there is no support in GCC (last version is heavily patched 4.2.4). Linux kernel v4.12 got rid of it (and v4.11 didn't build successfully). There is no good point to keep this support in U-Boot either. Reviewed-by: Simon Glass <sjg@chromium.org> Reviewed-by: Tom Rini <trini@konsulko.com> Reviewed-by: Heiko Schocher <hs@denx.de> Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
2017-05-08tools: moveconfig: cleanup README entiresChris Packham
The Kconfig description replaces the description in the README file so as options are migrated they can be removed from the README. Signed-off-by: Chris Packham <judge.packham@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2017-05-08tools: moveconfig: cleanup whitelist entriesChris Packham
After moving to KConfig and removing from all headers options should be removed from config_whitelist.txt so the build starts complaining if someone adds them back. Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com> Signed-off-by: Chris Packham <judge.packham@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2017-05-08tools: moveconfig: extract helper function for user confirmationChris Packham
Avoid repetitive code dealing with asking the user for confirmation. Signed-off-by: Chris Packham <judge.packham@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2017-04-18tools: moveconfig: remove GCC prefix of obsolete architectureMasahiro Yamada
Recently, U-Boot removed support for these architectures. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2016-10-24tools: moveconfig: support wildcards in --defconfigs fileMasahiro Yamada
Supporting shell-style wildcards for the --defconfigs option will be useful to run the moveconfig tool against a specific platform. For example, "uniphier*" in the file passed by --defconfigs option will be expanded to defconfig files that start with "uniphier". This is easier than listing out all defconfig files you are interested in. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-09-16moveconfig: Add an option to commit changesSimon Glass
The moveconfig tool is quite clever and generally produces results that are suitable for sending as a patch without further work. The main required step is to add the changes to a commit. Add an option to do this automatically. This allows moveconfig to be used from a script to convert multiple CONFIG options, once per commit. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-09-16moveconfig: Add an option to skip promptsSimon Glass
At present it is not easy to use moveconfig from a script since it asks for user input a few times. Add a -y option to skip this and assume that 'y' was entered. Signed-off-by: Simon Glass <sjg@chromium.org>
2016-09-06tools: moveconfig: add --spl option to move options for SPL buildMasahiro Yamada
Prior to this commit, the tool could not move options guarded by CONFIG_SPL_BUILD ifdef conditionals because they do not show up in include/autoconf.mk. This new option, if given, makes the tool parse spl/include/autoconf.mk instead of include/autoconf.mk, which is probably preferred behavior when moving options for SPL. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-09-06tools: moveconfig: warn loudly if moved option has no entry in KconfigMasahiro Yamada
Currently, the tool gives up moving an option quietly if its entry was not found in Kconfig. If the option is not defined in the config header in the first place, it is no problem (as the Kconfig entry may have been hidden by reasonable "depends on"). However, if the option is defined in the config header, the missing Kconfig entry is a sign of possible behavior change. It is highly recommended to manually check if the option has been moved as expected. In this case, let's add "suspicious" in the log and change the log color (if --color option is given) to make it stand out. This was suggested by Tom in [1]. [1] http://lists.denx.de/pipermail/u-boot/2016-July/261988.html Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Suggested-by: Tom Rini <trini@konsulko.com> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-09-06tools: moveconfig: use sets instead of lists for failed/suspicious boardsMasahiro Yamada
The sets feature is handier for adding unique elements. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Joe Hershberger <joe.hershberger@ni.com>
2016-09-06tools: moveconfig: remove document about deprecated error messageMasahiro Yamada
Since commit cc008299f852 ("tools: moveconfig: do not rely on type and default value given by users"), we do not have this error case. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Joe Hershberger <joe.hershberger@ni.com>
2016-08-26treewide: fix "followings" to "following"Masahiro Yamada
Most of them are my mistakes. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2016-08-26tools: moveconfig: add Xtensa GCC prefix to CROSS_COMPILE listMasahiro Yamada
This is needed to move CONFIG options for the recently-added xtfpga_defconfig. The tarball of the pre-built toolchain can be downloaded from: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.9.0/ Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2016-08-05tools: moveconfig: support CONFIG_SYS_EXTRA_OPTIONS cleaningMasahiro Yamada
We mostly move config options from board header files to Kconfig, but sometimes config defines come from CONFIG_SYS_EXTRA_OPTIONS. Historically, CONFIG_SYS_EXTRA_OPTIONS originates in boards.cfg, which was used as a central database of configuration prior to the Kconfig conversion. Now, we want to migrate to primary entries in Kconfig rather than option list in CONFIG_SYS_EXTRA_OPTIONS, so it should be helpful to have the tool to cleanup CONFIG_SYS_EXTRA_OPTIONS automatically. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-08-05tools: moveconfig: make getting all defconfigs into helper functionMasahiro Yamada
I want to reuse this routine in the next commit. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-08-05tools: moveconfig: fix cleanup of defines across multiple linesMasahiro Yamada
Correct the clean-up of such defines that continue across multiple lines, like follows: #define CONFIG_FOO "this continues to the next line " \ "this line should be removed too" \ "this line should be removed as well" Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-08-05tools: moveconfig: show diffs of cleaned headers in colorMasahiro Yamada
Show code diff in color if --color option is given. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-08-05tools: moveconfig: show result of header cleaning in unified diffMasahiro Yamada
The header cleanup feature of this tool now removes empty ifdef's, successive blank lines as well as moved option defines. So, we want to see a little more context to check which lines were deleted. It is true that we can see it by "git diff", but it would not work in the --dry-run mode. So, here, this commit. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2016-08-05tools: moveconfig: trim garbage lines after header cleanupsMasahiro Yamada
The tools/moveconfig.py has a feature to cleanup #define/#undef's of moved config options, but I want this tool to do a better job. For example, when we are moving CONFIG_FOO and its define is surrounded by #ifdef ... #endif, like follows: #ifdef CONFIG_BAR # define CONFIG_FOO #endif The header cleanup will leave empty #ifdef ... #endif: #ifdef CONFIG_BAR #endif Likewise, if a define line between two blank lines <blank line> #define CONFIG_FOO <blank lines. ... is deleted, the result of the clean-up will be successive empty lines, which is a coding-style violation. It is tedious to remove left-over garbage lines manually, so I want the tool to take care of this. The tool's job is still not perfect, so we should check the output of the tool, but I hope our life will be much easier with this patch. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2016-08-05tools: moveconfig: do not check clean tree and compilers for -H optionMasahiro Yamada
The clean tree (make mrproper) and compilers are required when moving config options, but not needed when we only cleanup headers. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-08-05tools: moveconfig: do not cleanup headers in include/generatedMasahiro Yamada
The files in include/generated are generated during build and removed by "make mrproper", so it has no point to touch them by this tool. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Tom Rini <trini@konsulko.com>
2016-06-22tools: moveconfig: show suspicious boards with possible misconversionMasahiro Yamada
There are some cases where config options are moved, but they are ripped off at the final savedefconfig stage: - The moved option is not user-configurable, for example, due to a missing prompt in the Kconfig entry - The config was not defined in the original config header despite the Kconfig specifies it as non-bool type - The config define in the header contains reference to another macro, for example: #define CONFIG_CONS_INDEX (CONFIG_SYS_LPC32XX_UART - 2) The current moveconfig does not support recursive macro expansion. In these cases, the conversion is very likely to be an unexpected result. That is why I decided to display the log in yellow color in commit 5da4f857beac ("tools: moveconfig: report when CONFIGs are removed by savedefconfig"). It would be nice to display the list of suspicious boards when the tool finishes processing. It is highly recommended to check the defconfigs once again when this message is displayed. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-06-22tools: moveconfig: simplify show_failed_boards() and show more infoMasahiro Yamada
Since commit 1d085568b3de ("tools: moveconfig: display log atomically in more readable format"), the function color_text() is clever enough to exclude LF from escape sequences. Exploit it for removing the "for" loops from Slots.show_failed_boards(). Also, display "(the list has been saved in moveconfig.failed)" if there are failed boards. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-06-22tools: moveconfig: simplify source tree switchingMasahiro Yamada
The subprocess.Popen() does not change the child process's working directory if cwd=None is given. Let's exploit this fact to refactor the source directory handling. We no longer have to pass "-C <reference_src_dir>" to the sub-process because self.current_src_dir tracks the source tree against which we want to run defconfig/autoconf. The flag self.use_git_ref is not necessary either because we can know the current state by checking whether the self.current_src_dir is a valid string or None. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>