summaryrefslogtreecommitdiff
path: root/tools
AgeCommit message (Collapse)Author
2019-07-24buildman: fix invocation examples typosBaruch Siach
Also, make command line example indentation consistent. Signed-off-by: Baruch Siach <baruch@tkos.co.il>
2019-07-24binman: Add a test for nested and aligned sectionsSimon Glass
Current test coverage is likely sufficient for the logic used to place sections in the image. However it seems useful to add a test specifically for nested sections, since these could have some unusual interactions. Add a new test for this and aligned sections. This test failed before the refactor to drop the bsection.py file (Section class), but passes now. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Add an 'extract' commandSimon Glass
It is useful to be able to extract all binaries from the image, or a subset of them. Add a new 'extract' command to handle this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Support reading from CBFS entriesSimon Glass
CBFS is a bit like a section but with a custom format. Provide the list of entries and the compression type to binman so that it can extract the data from the CBFS, just like any other part of the image. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Allow reading an entry from an imageSimon Glass
It is useful to be able to extract entry contents from an image to see what is inside. Add a simple function to read the contents of an entry, decompressing it by default. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Allow for logging information to be displayedSimon Glass
Binman generally operates silently but in some cases it is useful to see what Binman is actually doing at each step. Enable some logging output with different logging levels selectable via the -v flag. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Support listing an imageSimon Glass
Add support for listing the entries in an image. This relies on the image having an FDT map. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Convert Image to a subclass of EntrySimon Glass
When support for sections (and thus hierarchical images) was added to binman, the decision was made to create a new Section class which could be used by both Image and an Entry_section class. The decision between using inheritance and composition was tricky to make, but in the end it was decided that Image was different enough from Entry that it made sense to put the implementation of sections in an entirely separate class. It also has the advantage that core Image code does have to rely on an entry class in the etype directory. This work was mostly completed in commit: 8f1da50ccc "binman: Refactor much of the image code into 'section' As a result of this, the Section class has its own version of things like offset and size and these must be kept in sync with the parent Entry_section class in some cases. In the last year it has become apparent that the cost of keeping things in sync is larger than expected, since more and more code wants to access these properties. An alternative approach, previously considered and rejected, now seems better. Adjust Image to be a subclass of Entry_section. Move the code from Section (in bsection.py) to Entry_section and delete Section. Update all tests accordingly. This requires substantial changes to Image. Overall the changes reduce code size by about 240 lines. While much of that is just boilerplate from Section, there are quite a few functions in Entry_section which now do not need to be overiden from Entry. This suggests the change is beneficial even without further functionality being added. A side benefit is that the properties of sections are now consistent with other entries. This fixes a problem in testListCmd() where some properties are missing for sections. Unfortunately this is a very large commit since it is not feasible to do the migration piecemeal. Given the substantial tests available and the 100% code coverage of binman, we should be able to do this safely. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Support reading an image into an Image objectSimon Glass
It is possible to read an Image, locate its FDT map and then read it into the binman data structures. This allows full access to the entries that were written to the image. Add support for this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Support locating an image headerSimon Glass
Add support for locating an image header in an image. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Support locating an FDT mapSimon Glass
Add support for locating an image's Fdt map which is used to determine the contents and structure of the image. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Allow listing the entries in an imageSimon Glass
It is useful to be able to summarise all the entries in an image, e.g. to display this to this user. Add a new ListEntries() method to Entry, and set up a way to call it through the Image class. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Detect bad CBFS file typesSimon Glass
Detect when an unknown or unsupported file type is specified and report an error. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Support FDT update for CBFSSimon Glass
It is useful to add the CBFS file information (offset, size, etc.) into the FDT so that the layout is complete. Add support for this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Use the cbfs memlen field only for uncompressed lengthSimon Glass
The purpose of this badly named field is a bit ambiguous. Adjust the code to use it only to store the uncompressed length of a file, leaving it set to None if there is no compression used. This makes it easy to see if the value in this field is relevant / useful. Also set data_len for compressed fields, since it should be the length of the compressed data, not the uncompressed data. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Provide the actual data address for cbfs filesSimon Glass
At present a file with no explicit CBFS offset is placed in the next available location but there is no way to find out where it ended up. Update and rename the get_data() function to provide this information. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Allow device-tree entries to be compressedSimon Glass
At present the logic skips the blob class' handling of compression, so this is not supported with device tree entries. Fix this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Allow entries to expand after packingSimon Glass
Add support for detecting entries that change size after they have already been packed, and re-running packing when it happens. This removes the limitation that entry size cannot change after PackEntries() is called. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Add a control for post-pack entry expansionSimon Glass
We plan to support changing the size of entries after they have been packed. For now it will always be enabled. But to aid testing of both cases (in the event that we want to add a command-line flag, for example), add a setting to control it. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Add a return value to ProcessContentsUpdate()Simon Glass
At present if this function tries to update the contents such that the size changes, it raises an error. We plan to add the ability to change the size of entries after packing is completed, since in some cases it is not possible to determine the size in advance. An example of this is with a compressed device tree, where the values of the device tree change in SetCalculatedProperties() or ProcessEntryContents(). While the device tree itself does not change size, since placeholders for any new properties have already bee added by AddMissingProperties(), we cannot predict the size of the device tree after compression. If a value changes from 0 to 0x1234 (say), then the compressed device tree may expand. As a first step towards supporting this, make ProcessContentsUpdate() return a value indicating whether the content size is OK. For now this is always True (since otherwise binman raises an error), but later patches will adjust this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Call ProcessUpdateContents() consistentlySimon Glass
SetContents() should only be called to set the contents of an entry from within the ObtainContents() call, since it has no guard against increasing the size of the contents, thus triggering incorrect operation. Change all such calls to use ProcessUpdateContents() instead. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Fix up ProcessUpdateContents error and commentsSimon Glass
This function raises an exception with its arguments around the wrong way so the message is incorrect. Fix this as well as a few minor comment problems. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Allow easy importing of entry modulesSimon Glass
At present entry modules can only be accessed using Entry.Lookup() or Entry.Create(). Most of the time this is fine, but sometimes a module needs to provide constants or helper functions useful to other modules. It is easier in this case to use 'import'. Add an __init__ file to permit this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Drop an unused arg in Entry.Lookup()Simon Glass
The first argument is not used. Remove it. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Move compression into the Entry base classSimon Glass
Compression is currently available only with blobs. However we want to report the compression algorithm and uncompressed size for all entries, so that other entry types can support compression. This will help with the forthcoming 'list' feature which lists entries in the image. Move the compression properties into the base class. Also fix up the docs which had the wrong property name. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Convert to use ArgumentParserSimon Glass
This class is the new way to handle arguments in Python. Convert binman over to use it. At the same time, introduce commands so that we can separate out the different parts of binman functionality. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Add an image headerSimon Glass
It is useful to be able to quickly locate the FDT map in the image. An easy way to do this is with a pointer at the start or end of the image. Add an 'image header' entry, which places a magic number followed by a pointer to the FDT map. This can be located at the start or end of the image, or at a chosen location. As part of this, update GetSiblingImagePos() to detect missing siblings. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Add an FDT mapSimon Glass
An FDT map is an entry which holds a full description of the image entries, in FDT format. It can be discovered using the magic string at its start. Tools can locate and read this entry to find out what entries are in the image and where each entry is located. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Add a convenience functions for real-DTB testsSimon Glass
Quite a few tests will use a real device tree and need it updated with the binman metadata. Add a helper function for this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Update help for new featuresSimon Glass
A few new features have been added. This has rendered part of the README obsolete. Update it. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Update future featuresSimon Glass
A few features have been completed and a few items are added. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Simplify the entry testSimon Glass
The current test for the 'entry' module is a bit convoluted since it has to import the module multiple times. It also relies on ordering, in that test1EntryNoImportLib() must run before test2EntryImportLib() if they are running in the same Python process. This is unreliable since neither the ordering of tests nor the process that they run in is defined. Fix this by always reloading the entry in these two tests. Also add a check that the expected value of have_importlib is obtained. This corrects a code-coverage problem in the 'entry' module on some systems. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Add support for fixed-offset files in CBFSSimon Glass
A feature of CBFS is that it allows files to be positioned at particular offset (as with binman in general). This is useful to support execute-in-place (XIP) code, since this may not be relocatable. Add a new cbfs-offset property to control this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Pad empty areas of the CBFS with filesSimon Glass
When there is lots of open space in a CBFS it is normally padded with 'empty' files so that sequentially scanning the CBFS can skip from one to the next without a break. Add support for this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Add support for Intel IFWI entriesSimon Glass
An Integrated Firmware Image is used to hold various binaries used for booting with Apollolake and some later devices. Add support for this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Add support for CBFS entriesSimon Glass
Add support for putting CBFSs (Coreboot Filesystems) in an image. This allows binman to produce firmware images used by coreboot to boot. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Add a utility library for coreboot CBFSSimon Glass
Coreboot uses a simple flash-based filesystem called Coreboot Filesystem (CBFS) to organise files used during boot. This allows files to be named and their position in the flash to be set. It has special features for dealing with x86 devices which typically memory-map their SPI flash to the top of 32-bit address space and need a 'boot block' ending there. Create a library to help create and read CBFS files. This includes a writer class, a reader class and associated other helpers. Only a subset of features are currently supported. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24patman: Add a function to write ifwitoolSimon Glass
This tool has quite a few arguments and options, so put the functionality in a function so that we call it from one place and hopefully get it right. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-24binman: Pass the toolpath to testsSimon Glass
Tools like ifwitool may not be available in the PATH, but are available in the build. These tools may be needed by tests, so allow tests to use the --toolpath flag. Also use this flag with travis. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23binman: Allow preserving test directoriesSimon Glass
Sometimes when debugging tests it is useful to keep the input and output directories so they can be examined later. Add an option for this and update the binman tests to support it. This affects both the test class and the tearDown() function called after each test. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23binman: Allow verbosity control when running testsSimon Glass
At present the -v flag is ignored with tests, so that (for example) -v2 does not have any effect. Update binman to pass this flag through to tests so that they work just like running binman normally, except in a few special cases where we are actually testing behaviour with different levels of verbosity. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23binman: Fix up the _DoTestFile() function -u argumentSimon Glass
This should be -u, not -up, since we don't need to preserve the output directory in this case. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23binman: Support ELF files for TPLSimon Glass
We currenty support using the ELF file in U-Boot proper and SPL, but not TPL. Add this as it is useful both with sandbox and for CBFS to allow adding TPL as a 'stage'. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23binman: Correct comment in u_boot_spl_elfSimon Glass
This comment mentions the wrong default filename. Fix it. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23binman: Use tools compression function for blob handlingSimon Glass
Avoid duplicate code here by using the new compression function in the tools module. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23binman: Drop unnecessary debug handlingSimon Glass
The -D option enables debug mode, but we only need to add -D to the command line once. Drop the duplicate code. Also drop the comment about enabling debugging since this can be done with -D. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23binman: Use the tools.Decompress methodSimon Glass
Update the compression test to use the tools module to decompress the output data. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23patman: Add functions to compress and decompress dataSimon Glass
Add utility functions to compress and decompress using lz4 and lzma algorithms. In the latter case these use the legacy lzma support favoured by coreboot's CBFS. No tests are provided as these functions will be tested by the CBFS tests in a separate patch. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23binman: Allow text directly in the nodeSimon Glass
At present text entries use an indirect method to specify the text to use, with a label pointing to the text itself. Allow the text to be directly written into the node. This is more convenient in cases where the text is constant. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-23binman: Update entry.SetOffsetSize to be optionalSimon Glass
At present this function always sets both the offset and the size of entries. But in some cases we want to set only one or the other, for example with the forthcoming ifwi entry, where we only set the offset. Update the function to handle this. Signed-off-by: Simon Glass <sjg@chromium.org>