summaryrefslogtreecommitdiff
path: root/tools/patman
AgeCommit message (Collapse)Author
2019-07-29binman: Add a bit of logging in entries when packingSimon Glass
Use the new logging feature to log information about progress with packing. This is useful to see how binman is figuring things out. Also update elf.py to use the same feature. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29patman: Update tout to avoid open-coding the debug levelsSimon Glass
Use the debug level constants instead of open-coding them in the file. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Use print() to print outputSimon Glass
At present tout writes directly to stdout. This is not necessary and it prevents tests from redirecting output. Change it to use print() for the non-progress output. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Allow for logging information to be displayedSimon Glass
Binman generally operates silently but in some cases it is useful to see what Binman is actually doing at each step. Enable some logging output with different logging levels selectable via the -v flag. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Allow easy importing of entry modulesSimon Glass
At present entry modules can only be accessed using Entry.Lookup() or Entry.Create(). Most of the time this is fine, but sometimes a module needs to provide constants or helper functions useful to other modules. It is easier in this case to use 'import'. Add an __init__ file to permit this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Convert to use ArgumentParserSimon Glass
This class is the new way to handle arguments in Python. Convert binman over to use it. At the same time, introduce commands so that we can separate out the different parts of binman functionality. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24patman: Add a function to write ifwitoolSimon Glass
This tool has quite a few arguments and options, so put the functionality in a function so that we call it from one place and hopefully get it right. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23patman: Add functions to compress and decompress dataSimon Glass
Add utility functions to compress and decompress using lz4 and lzma algorithms. In the latter case these use the legacy lzma support favoured by coreboot's CBFS. No tests are provided as these functions will be tested by the CBFS tests in a separate patch. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23patman: Add a way to set the search path for toolsSimon Glass
Sometimes tools can be located by looking in other locations. Add a way to direct the search. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23tools: Drop duplicate raise_on_error argumentSimon Glass
If kwargs contains raise_on_error then this function generates an error due to a duplicate argument. Fix this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Update cover-coverage tests for Python 3Simon Glass
We need slightly different commands to run code coverage with Python 3. Update the RunTestCoverage() function to handle this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Allow reading files in text modeSimon Glass
While reading files in binary mode is the norm, sometimes we want to use text mode. Add an optional parameter to handle this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10dtoc: Use byte type instead of str in fdtSimon Glass
In Python 3 bytes and str are separate types. Use bytes to ensure that the code functions correctly with Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10dtoc: Updates BytesToValue() for Python 3Simon Glass
The difference between the bytes and str types in Python 3 requires a number of minor changes to this function. Update it to handle the input data using the 'bytes' type. Create two useful helper functions which can be used by other modules too. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Don't require Python 2Simon Glass
Update the shebang to allow either Python 2 or Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Tidy up a few more unicode conversionsSimon Glass
Use the new functions in the tools module to handle conversion. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Adjust functional tests for Python 3Simon Glass
Change the code so that it works on both Python 2 and Python 3. This works by using unicode instead of latin1 for the test input, and ensuring that the output is converted to a string rather than a unicode object on Python 2. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Avoid unicode type in settings unit testsSimon Glass
The unicode type does not exist in Python 3 and when displaying strings they do not have the 'u' prefix. Adjusts the settings unit tests to deal with this difference, by converting the comparison value to a string, thus dropping the 'u'. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Sort series output for repeatabilySimon Glass
We use sets to produce the list of To and Cc lines for a series. This does not result in stable ordering of the recipients. Sort each list to ensure that the output is repeatable. This is necessary for tests. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Move unicode helpers to toolsSimon Glass
Create helper functions in the tools module to deal with the differences between unicode in Python 2 (where we use the 'unicode' type) and Python 3 (where we use the 'str' type). Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Support use of stringIO in Python 3Simon Glass
With Python 3 this class has moved. Update the code to handle both cases. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10binman: Handle repeated bytes for Python 3Simon Glass
The method of multiplying a character by a number works well for creating a repeated string in Python 2. But in Python 3 we need to use bytes() instead, to avoid unicode problems, since 'bytes' is no-longer just an alias of 'str'. Create a function to handle this detail and call it from the relevant places in binman. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Provide a way to get program output in binary modeSimon Glass
At present cros_subprocess and the tools library use a string to obtain stdout from a program. This works fine on Python 2. With Python 3 we end up with unicode errors in some cases. Fix this by providing a binary mode, which returns the data as bytes() instead of a string. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Use items() instead of iteritems()Simon Glass
Python 3 requires this, and Python 2 allows it. Convert the code over to ensure compatibility with Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Convert print statements to Python 3Simon Glass
Update all print statements to be functions, as required by Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10patman: Update cros_subprocess to use bytesSimon Glass
At present this function uses lists and strings. This does not work so well with Python 3, and testing against '' does not work for a bytearray. Update the code to fix these issues. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08binman: Run tests concurrentlySimon Glass
At present the tests run one after the other using a single CPU. This is not very efficient. Bring in the concurrencytest module and run the tests concurrently, using one process for each CPU by default. A -P option allows this to be overridden, which is necessary for code-coverage to function correctly. This requires fixing a few tests which are currently not fully independent. At some point we might consider doing this across all pytests in U-Boot. There is a pytest version that supports specifying the number of processes to use, but it did not work for me. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08patman: Don't clear progress in tout unless it was usedSimon Glass
At present calling Uninit() always called ClearProgress() which outputs a \r character as well as spaces to remove any progress information on the line. This can mess up the normal output of binman and other tools. Fix this by outputing this only when progress information has actually been previous written. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08tools: Set an initial value for indirSimon Glass
This variable is not documented or set up in the module. Fix this. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08patman: Handle unicode in _ProjectConfigParser testsSimon Glass
With Python 2.7.15rc1, ConfigParser.SafeConfigParser has unfortunately started returning unicode, for unknown reasons. Adjust the code to handle this by converting everything to unicode. We cannot convert things to ASCII since email addresses may be encoded with UTF-8. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29binman: Support adding filesSimon Glass
In some cases it is useful to add a group of files to the image and be able to access them at run-time. Of course it is possible to generate the binman config file with a set of blobs each with a filename. But for convenience, add an entry type which can do this. Add required support (for adding nodes and string properties) into the state module. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28patman: Detect missing tools and report themSimon Glass
When tools are needed but not present, at present we just get an error which can be confusing for the user. Try to be helpful by reporting the tool as missing and suggesting a possible remedy. Also update the Run() method to support this. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-08-08patman: Correct unit test failureSimon Glass
A recent rename of the function did not rename the test file. Fix this. Fixes: 12308b128fa (lib: fdtdec: Rename routine fdtdec_setup_memory_size()) Signed-off-by: Simon Glass <sjg@chromium.org>
2018-08-01patman: Show the current directory in GetInputFilename()Simon Glass
When this fails it is useful to see the current directory, since U-Boot's build system will typically change into the output directory during the build. Add this information to the error. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-08-01patman: Add a few more helpers to the tools librarySimon Glass
Add functions to read and write a file, looking through a list of search paths to find it. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-08-01patman: Allow test commands to fall back to real onesSimon Glass
Tests use the 'test_result' feature to return a predetermined command result for particular commands. The avoids needing to have the real command available just to run a test. It works by calling the function provided by the test, to get the value. However sometimes the test does need to run the real command. Allow it to fall back to do this when the function does not return a result. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-07-26patman: Split alias entries on arbitrary whitespaceAdam Sampson
Previously the first three words in a git-mailrc alias entry could only be separated by spaces. git-send-email and Mutt both allow arbitrary whitespace here. Signed-off-by: Adam Sampson <ats@offog.org> Reviewed-by: Simon Glass <sjg@chromium.org>
2018-07-19lib: fdtdec: Rename routine fdtdec_setup_memory_size()Siva Durga Prasad Paladugu
This patch renames the routine fdtdec_setup_memory_size() to fdtdec_setup_mem_size_base() as it now fills the mem base as well along with size. Signed-off-by: Siva Durga Prasad Paladugu <siva.durga.paladugu@xilinx.com> Signed-off-by: Michal Simek <michal.simek@xilinx.com> Reviewed-by: Tom Rini <trini@konsulko.com>
2018-07-09binman: Move capture_sys_output() to test_utilSimon Glass
This function is useful in various tests. Move it into the common test utility module. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-07-09binman: Move coverage logic into a new test_util fileSimon Glass
At present only binman has the logic for determining Python test coverage but this is useful for other tools also. Move it out into a separate file so it can be used by other tools. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-06-23patman: Support using a particular SMTP serverSimon Glass
Some environments require providing the '--smtp-server' argument to 'git send-email'. Add support for this. Signed-off-by: Simon Glass <sjg@chromium.org>
2018-06-23patman: add test for SPDX licenseChris Packham
Add a test to exercise the check for a valid SPDX license. Signed-off-by: Chris Packham <judge.packham@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2018-06-23patman: add option for limiting the Cc listChris Packham
Many mailing-lists consider a long Cc list a sign of spam and will either drop the message or mark it for moderation. Because patman automatically invokes get_maintainer.pl the Cc list can expand unexpectedly. Allow the user to specify a limit for the Cc list. This limit is applied after removing any known bouncing addresses. By default no limit is applied. Signed-off-by: Chris Packham <judge.packham@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2018-06-07patman: Fix unit tests for SPDXSimon Glass
The format of this line has changed. Update the patman test to suit. Signed-off-by: Simon Glass <sjg@chromium.org> Tested-by: Alex Kiernan <alex.kiernan@gmail.com>
2018-05-07SPDX: Convert all of our single license tags to Linux Kernel styleTom Rini
When U-Boot started using SPDX tags we were among the early adopters and there weren't a lot of other examples to borrow from. So we picked the area of the file that usually had a full license text and replaced it with an appropriate SPDX-License-Identifier: entry. Since then, the Linux Kernel has adopted SPDX tags and they place it as the very first line in a file (except where shebangs are used, then it's second line) and with slightly different comment styles than us. In part due to community overlap, in part due to better tag visibility and in part for other minor reasons, switch over to that style. This commit changes all instances where we have a single declared license in the tag as both the before and after are identical in tag contents. There's also a few places where I found we did not have a tag and have introduced one. Signed-off-by: Tom Rini <trini@konsulko.com>
2018-01-28host-tools: use python2 explicitly for shebangMasahiro Yamada
All of these host tools are apparently written for Python2, not Python3. Use 'python2' in the shebang line according to PEP 394 (https://www.python.org/dev/peps/pep-0394/). Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2018-01-21patman: Unquote output from get_maintainer.plStefan Brüns
get_maintainer.pl quotes names which it considers unsafe, i.e. anything containing [^a-zA-Z0-9_ \-]. This confuses patman, it will duplicate addresses which are also in Series-to/cc. Strip the quotes. Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de> Reviewed-by: Simon Glass <sjg@chromium.org>
2017-11-22patman: Fix up tests to pass with newest checkpatchSimon Glass
The checkpatch tool was updated but the patman tests were not. Fix this. Signed-off-by: Simon Glass <sjg@chromium.org>
2017-09-13patman: Fix error when the email blacklist is missingSimon Glass
This section of the settings file may be missing. Handle that gracefully rather than emitting an error. Also update patman to write this section when a new settings file is created. Fixes: e11aa602 (patman: add support for omitting bouncing addresses) Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Chris Packham <judge.pckham@gmail.com> Tested-by: Bin Meng <bmeng.cn@gmail.com>
2017-09-11patman: add support for omitting bouncing addressesChris Packham
Add support for reading a list of bouncing addresses from a in-tree file (doc/bounces) and from the ~/.patman config file. These addresses are stripped from the Cc list. Signed-off-by: Chris Packham <judge.packham@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org> Reviewed-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com <mailto:philipp.tomsich@theobroma-systems.com>>