summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2019-07-29Merge branch '2019-07-29-master-imports'Tom Rini
- Assorted bug fixes
2019-07-29Makefile: fix implementation of BINMAN_DEBUGStephen Warren
binman only accepts the -D argument early on the command-line, yet the Makefile currently passes it near the end. This causes the build to fail if this feature is used. Re-order the command-line to fix this. Signed-off-by: Stephen Warren <swarren@nvidia.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Add command-line support for replacing entriesSimon Glass
Add a 'replace' command to binman to permit entries to be replaced, either individually or all at once (using a filter). Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Correct the error message for invalid pathSimon Glass
At present this message references -o for output file. But binman uses -f now. Fix it. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Split control.WriteEntryToImage() into separate functionsSimon Glass
This code has three distinct phases: 1. The image is loaded and the state module is set up 2. The entry is written to the image 3. The image is repacked and written back to the file Split the code out with three separate functions, one for each phase. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Update control.WriteEntry() to support writing the mapSimon Glass
Add the ability to write a new map file. Also tidy up a few comments and rename a misleading variable. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Move control.WriteEntry further down the fileSimon Glass
Move this function after the extraction logic so we can keep the writing logic in one place. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Clean up all output directories in testsSimon Glass
At present some tests leave behind output directories. This happens because some tests call binman, which sets up an output directory, then call it again, which sets up another output directory and leaves the original one behind. Fix this by using a separate temporary directory when binman is called twice, or by manually removing the output directory. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Add a test function to clean up the output dirSimon Glass
Put tearDown()'s logic into a new _CleanupOutputDir() function so that it can be called from elsewhere. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Update state when replacing device-tree entriesSimon Glass
Since the state module holds references to all the device trees used by binman, it must be updated when the device trees are updated. Add support for this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29patman: Reset the output directory when it is removedSimon Glass
At present outdir remains set ever after the output directory has been removed. Fix this to avoid trying to access it when it is not present. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Support replacing data in a cbfsSimon Glass
At present binman cannot replace data within a CBFS since it does not allow rewriting of the files in that CBFS. Implement this by using the new WriteData() method to handle the case. Add a header to compressed data so that the amount of compressed data can be determined without reference to the size of the containing entry. This allows the entry to be larger that the contents, without causing errors in decompression. This is necessary to cope with a compressed device tree being updated in such a way that it shrinks after the entry size is already set (an obscure case). It is not used with CBFS since it has its own metadata for this. Increase the number of passes allowed to resolve the position of entries, to handle this case. Add a test for this new logic. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Update Entry.WriteData() to handle special sectionsSimon Glass
At present this method assumes that the parent section does not need to recalculate its position or adjust any metadata it may contain. But when the entry changes size this may not be true. Also if the parent section is more than just a container (e.g. it is a CBFS) then the section may need to regenerate its output. Add a new WriteChildData() method to sections and call this from the WriteData() method, to handle this situation. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Update Entry.ReadEntry() to work through classesSimon Glass
At present we simply extract the data directly from entries using the image_pos information. This happens to work on current entry types, but cannot work if the entry type encodes the data in some way. Update the ReadData() method to provide the data by calling a new ReadChildData() method in the parent. This allows the entry_Section class, or possibly any other container class, to return the correct data in all cases. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Add a prefix before CBFS hex offsetsSimon Glass
Add a 0x prefix to these errors to avoid confusion. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Add a few more features to the wishlistSimon Glass
Add mention of a few other desirable features that may be implemented in the future. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Place Intel descriptor at image startSimon Glass
The Intel descriptor must always appear at the start of an (x86) image, so it is supposed to position itself there always. However there is no explicit test for this. Add one and fix a bug introduced by the recent change to adjust Entry to read the node in a separate call. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Adjust fmap to ignore CBFS filesSimon Glass
The FMAP is not intended to show the files inside a CBFS. The FMAP can be used to locate the CBFS itself, but then the CBFS must be read to find out what is in it. Update the FMAP to work this way and add some debugging while we are here. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29libfdt: Copy the struct region in fdt_resize()Simon Glass
At present this function appears to copy only the data before the struct region and the data in the string region. It does not seem to copy the struct region itself. >From the arguments of this function it seems that it should support fdt and buf being different. This patch attempts to fix this problem. Upstream commit: c72fa77 libfdt: Copy the struct region in fdt_resize() Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Support shrinking a entry after packingSimon Glass
Sometimes an entry may shrink after it has already been packed. In that case we must repack the items. Of course it is always possible to just leave the entry at its original size and waste space at the end. This is what binman does by default, since there is the possibility of the entry changing size every time binman calculates its contents, thus causing a loop. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Update the _testing entry to support shrinkageSimon Glass
Sometimes entries shrink after packing. As a start towards supporting this, update the _testing entry to handle the test case. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Allow updating entries that change sizeSimon Glass
So far we don't allow entries to change size when repacking. But this is not very useful since it is common for entries to change size after an updated binary is built, etc. Add support for this, respecting the original offset/size/alignment constraints of the image layout. For this to work the original image must have been created with the 'allow-repack' property. This does not support entry types with sub-entries such as files and CBFS, but it does support sections. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Add more tests for image header positionSimon Glass
The positioning does not currently work correctly if at the end of an image with no fixed size. Also if the header is in the middle of an image it can cause a gap in the image since the header position is normally at the image end, so entries after it are placed after the end of the image. Fix these problems and add more tests to cover these cases. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Move Image.BuildImage() into a single functionSimon Glass
Now that an Image is an Entry_section, there is no need for the separate BuildSection() function. Drop it and add a bit of logging. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Write the original input fdtmap to a fileSimon Glass
When reading an image in, write its fdtmap to a file in the output directory. This is useful for debugging. Update the 'ls' command to set up the output directory; otherwise it will fail. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Update documentation for image creationSimon Glass
There are a few more steps in the process now. Update the documentation to reflect this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Add info to allow safely repacking an image laterSimon Glass
At present it is not possible to discover the contraints to repacking an image (e.g. maximum section size) since this information is not preserved from the original image description. Add new 'orig-offset' and 'orig-size' properties to hold this. Add them to the main device tree in the image. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Support updating entries in an existing imageSimon Glass
While it is useful and efficient to build images in a single pass from a unified description, it is sometimes desirable to update the image later. Add support for replace an existing file with one of the same size. This avoids needing to repack the file. Support for more advanced updates will come in future patches. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Tidy up _SetupDtb() to use its own temporary fileSimon Glass
At present EnsureCompiled() uses an file from the 'output' directory (in the tools module) when compiling the device tree. This is fine in most cases, allowing useful inspection of the output files from binman. However in functional tests, _SetupDtb() creates an output directory and immediately removes it afterwards. This serves no benefit and just confuses things, since the 'official' output directory is supposed to be created and destroyed in control.Binman(). Add a new parameter for the optional temporary directory to use, and use a separate temporary directory in _SetupDtb(). Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Allow the fdtmap to remain unchangedSimon Glass
When updating an existing image where the size of all entries remains the same, we should not need to regenerate the fdtmap. Update the entry to return the same fdtmap as was read from the image. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Add a constant for common entry propertiesSimon Glass
We use this same combination of properties several times in tests. Add a constant for it to avoid typos, etc. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Add a function to obtain the image for an EntrySimon Glass
At present we have an 'image' property in the entry for this purpose, but this is not necessary and seems error-prone in the presence of inheritance. Add a function instead. The Entry_section class overrides this with a special version, since top-level sections are in fact images, since Image inherits Entry_section. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Adjust Entry to read the node in a separate callSimon Glass
At present the Entry constructor sets up the object and then immediately reads its device-tree node to obtain its properties. This breaks a convention that constructors should not do any processing. A consequence is that we must pass all arguments to the constructor and cannot have the node-reading proceed in a different way unless we pass flags to that constructor. We already have a 'test' flag in a few cases, and now need to control whether the 'orig_offset' and 'orig_size' properties are set or not. Adjust the code to require a separate call to ReadNode() after construction. The Image class remains as it was. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Add an image name into the fdtmapSimon Glass
Since binman supports multiple images it is useful to know which one created the image that has been read. Then it is possible to look up that name in the 'master' device tree (containing the description of all images). Add a property for this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Store the entry in output_fdt_filesSimon Glass
In some cases we want to access the Entry object for a particular device tree. This allows us to read its contents or update it. Add this information to output_fdt_files and provide a function to read it. Also rename output_fdt_files since its name is no-longer descriptive. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Allow state functions to fail to return dataSimon Glass
At present these state functions raise an exception if they cannot find what is requested. But in some cases the information is optional (e.g. an fdtmap in a coming patch) so it is better to return gracefully. Update these two functions to return None when the data cannot be found. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Support loading entry data from a fileSimon Glass
When modifying an image it is convenient to load the data from the file into each entry so that it can be reprocessed. Add a new LoadData() method to handle this. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Store image fdtmap when loading from a fileSimon Glass
This data provides all the information about the position and size of each entry. Store it for later use when loading an image. It can be reused as is if the image is modified without changing offsets/sizes. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29dtoc: Update Fdt.GetNode() to handle the root nodeSimon Glass
This function currently fails if the root node is requested. Requesting the root node is sometimes useful, so fix the bug. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29dtoc: Update Fdt.FromData() to allow a nameSimon Glass
It is confusing when something goes wrong with a device tree which was created from data rather than a file, since there is no identifying filename. Add an option to provide this. Use the filename as the name, where available Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Show a helpful error when a DT property is missingSimon Glass
At present a Python exception is raised which does not show the node information. Add a more helpful exception in this case. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Add a bit of logging in entries when packingSimon Glass
Use the new logging feature to log information about progress with packing. This is useful to see how binman is figuring things out. Also update elf.py to use the same feature. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29patman: Update tout to avoid open-coding the debug levelsSimon Glass
Use the debug level constants instead of open-coding them in the file. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Drop state.fdt_set as this is not neededSimon Glass
We can iterate through the output files so don't need this global anymore. Remove it. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Simplify state.fdt_subsetSimon Glass
At present this excludes the device tree passed in to binman since it is always returned first by GetAllFdts(). However, this is easy to ensure by adding a check in that function. Change this dict to includes all device trees, and rename it to fdt_set. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Adjust state.fdt_files to be keyed by entry typeSimon Glass
It makes more sense to use entry type as the key for this dictionary, since the filename can in principle be anything. Make this change and also rename fdt_files and add a comment to explain it better. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Adjust GetFdt() to be keyed by etypeSimon Glass
At present the FDTs are keyed by their default filename (not their actual filename). It seems easier to key by the entry type, since this is always the same for each FDT type. To do this, add a new Entry method called GetFdtEtype(). This is necessary since some entry types contain a device tree which are not the simple three entry types 'u-boot-dtb', 'u-boot-spl' or 'u-boot-tpl'. The code already returns a dict for GetFdt(). Update the value of that dict to include the filename so that existing code can work. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Rename state.GetFdt()Simon Glass
This function name conflicts with Fdt.Node.GetFdt() which has a different purpose. Rename it to avoid confusion. The new name suggests it is indexed by entry type rather than filename. This will be tidied up in a future commit. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Rename state.GetFdts()Simon Glass
This function name conflicts with Entry.GetFdts() which has a different purpose. Rename it to avoid confusion. Also update a stale comment relating to this function. Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-29binman: Convert GetFdtSet() to use a dictSimon Glass
At present this function returns a set of device-tree filenames. It has no way of returning the actual device-tree object. Change it to a dictionary so that we can add this feature in a future patch. Also drop fdt_set since it is no-longer used. Signed-off-by: Simon Glass <sjg@chromium.org>