* Root cause of the issue is improper naming of variables. Turns out,
there was no variable to even denote where the blob should have been
searched for, at "src".
* Previously there was one such variable, suggestively called "TARGET", that
was desperately trying to serve as both, depending on who +2d hacks harder.
* One such example is "c982836 extract_utils: Fix makefile generation issues".
That patch deliberately trimmed the "src:" from a spec (therefore
obviously breaking the search at src) but enabling the searching at
dst, via the good-for-all TARGET variable.
* This patch introduces the following variables:
- SRC_FILE: absolute path corresponding to SPEC_SRC_FILE in the
Android filesystem.
- DST_FILE: absolute path corresponding to SPEC_DST_FILE in the
Android filesystem. Somewhat analogous to the old TARGET variable,
but actually contains the leading / as well (/system/bin/adsprpcd
instead of system/bin/adsprpcd).
* Use existing common get_file() function (which previously was
impossible to use, because it was impossible to determine calling
arguments properly) to reduce complexity of handling adb and disk
image as blob sources.
* Via the new SRC_FILE and DST_FILE variables, search for a blob in all
possible locations (including paths stripped of "/system" which
transforms an absolute path in the Android filesystem into a proper
relative path to that file in a disk image).
Change-Id: Ic40fb4dc93541d8b3f33fde586b773199cf4ded2
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
* This denotes the path of the file that results from the extraction
process, relative to the "proprietary" directory.
* This is a cleanup patch.
Change-Id: I38e759bb6ed697f0a31ca35a7aa9b9b92f8b6793
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
* The write_product_copy_files() and write_product_packages() functions
rely on its undocumented behavior of keeping target_args in the
returned list, because they are users of target_args (such as
";PRESIGNED" etc).
* Make the behavior documented.
Change-Id: If71595dca32abd40039706d4fed2d7f12e005365
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
* Strip target_args from target_file at callee instead of at caller
* WARNING! Changes (improperly documented) behavior of prefix_match()
function, which is expected to not strip target_args(), and the root
cause why stripping target_args was currently done at caller. Will be
addressed in next patch.
Change-Id: I820d2350aa64ff41374809fcb22f812257132652
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
This reverts commit c982836ca6.
This breaks extracting from src in "src:dst", such as from a disk image.
Will be addressed in the cleanup commit that follows.
Change-Id: Iff84a926f0c3bf908320b43ba40235e0a89db644
* proprietary-files.txt entries such as
"-app/TimeService/TimeService.apk:priv-app/TimeService/TimeService.apk" should
generate a "LOCAL_SRC_FILES := proprietary/priv-app/TimeService/TimeService.apk"
in the Makefile definition.
* However, currently, the prefix_match function is being called on the whole
PRODUCT_PACKAGES_LIST entry (whole line, including ":"), and therefore,
TimeService.apk would be included in the APPS list instead of PRIV_APPS.
* Furthermore, because of the incorrect prefix_match, the generated
LOCAL_SRC_FILES is "proprietary/app/priv-app/TimeService/TimeService.apk",
which breaks the build because there is no file at that wrong path.
* The fix is to make the extract function match up with write_product_packages
by applying the target_file function on all BUILD_PREBUILT source files.
Change-Id: Ib4859b8854db0f2142bb3f28cce8dd25f7141732
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
Trailing slashes cause 'patchset' to be an empty string, which
cannot be converted to an int. If we have a trailing slash, ignore
it and pick the current patchset.
Change-Id: Ifad60b7f6ff3dcfd9dd80b2e93b1f909aa360666
vdexExtractor is a tool made by anestisb that is written in C++
with code taken directly from art. However, anestisb has also added
a quicken decompiler to oatdump that was merged upstream, so we only
need vdexExtractor for 8.1 and 8.0.
Change-Id: Ic2cf2dc627a1ad2fa4d500d02d9eac8b8a9577b5
Signed-off-by: Joe Maples <joe@frap129.org>
* Oreo expects VNDK compatible files to be listed as LOCAL_VENDOR_FILE,
not LOCAL_PROPRIETARY_FILE.
Change-Id: Ia2384c4f3ab3a99b79df52c796c53dc25a0f4a88
* Fix makefile generation for packages that have set
a different target destination
* Thanks to rashed and javelinanddart for their help
in debugging and solving this issue.
Change-Id: I1f5a1abd6929e4a7e0ccd6370d3a3dd986f94fed
* Use requests if installed. If not fall back to urllib. This is
done because users may or may not have requests installed and
requiring them to do so for simple http stuff isn't really
reasonable.
* If requests is installed and a .gerritrc file exists in the
user's HOME directory, try to get credentials out of it
for the given gerrit instance. If auth for the correct gerrit
instance exists, use it to auth to gerrit. If no .gerritrc
exists, just use requests with no auth.
Example ~/.gerritrc entry:
review.lineageos.org|invisiblek|httppasswordhere
Change-Id: I95be26d51bfd31b53f3613e8dbfb7bba46324571
Skip the extraction of pinned files if the ones currently available
have the expected sha1. If we don't, we will overwrite pinned files
with potentially incorrect files when the current vendor files are
not moved to a temporary directory (i.e. when not cleaning vendor).
Change-Id: I640d6bf2ed98eb366a4df17f0ebeaec81cb5274b
We want to cleanup our temporary files independently on the signal,
so just execute a trap on 0. This will ensure temporary files are
always removed and doesn't require any extra care when trapping
signals such as SIGINT that require an explicit exit call.
Change-Id: Ieff4f15c44a9ac9d5a543d14c140ebd72c0e7344
To be honest, this name is a little misleading, this is how it should
have been done in the first place. This allows devices to copy vendor
files to the proper location depending on TARGET_COPY_OUT_VENDOR rather
than hardcoding system/vendor. This allows devices with dedicated vendor
partitions to copy directly to vendor. The only reason it's optional
is that some nexi set TARGET_COPY_OUT_VENDOR to system which would cause
some weird breakage.
Change-Id: Ic46bc1086737835340abef9f61693d386bc6a5dc
* Make a tempdir using the mktemp command rather than just making a
dir in /tmp to accomodate for systems that don't set proper perms
or dont have /tmp
* Fix the deodex procedure to pull files from the right path
Change-Id: I181863599b6670e3a149069dbb7b13ebf73bae8e
Oreo introduces new optimized dex files (or vdex). While smali/baksmali
have supported vdex since 2.2.1, you actually have to have the .vdex in
the same location as the .oat or else the smali will fail due to not
finding the .vdex. Copy it (if it exists) and echo a warning since vdex
deodexing is still experimental.
Change-Id: Ic612751be45a8b6e54b8794b426a4d5adbbf13d2
Sometime blob directories are contained in dirs not named system
(ex. extracting blobs from another vendor dir) so check if the file
exists without system/ before appending "system/" to filenames.
Change-Id: I4ba946e178260b4f4804b1ac4422645e3085c665
Sometimes the need arises to update one specific subset of blobs
while holding others steady. Introduce a way to specify the "section"
of blobs to update (obviously requires the proprietary-files to be
properly split up first)
Requires additional changes in device tree extract_files.sh to support
passing the additional arguments
Change-Id: I98feab56f8bfade2818b1b5264e019440d4c57c5
In case of conflict, reset to initial state by aborting the cherry-pick,
leaving the working directory clean
Change-Id: I2353b92254b67ea49835bc88b6fc9bc3910b5107
* When using an ssh gerrit instance (-g) it is perfectly
reasonable to pick a draft commit. Don't require force
picking (-f) in order to do so.
Change-Id: I420f566ee1bb1f0b0d2a2ef29d95e0545db70b89
When a line from the XML file contains a character which grep classifies
as a binary one, it will omit this line at all. Moreover it will append
'Binary file matches' to the stdout. This behavior breaks some XML files.
To fix this issue, let's tell grep to process a binary file as if it were
text.
Change-Id: I6ec5689f73a0926870e502f3f8ee77346b47d05f
This change correctly handles devices with underscores in their
name even if there is a device with the same name after the last
underscore (spyder, umts_spyder) by not allowing any underscores
in the manufacturers name.
Without this change, spyder gets resolved to:
device/motorola_umts/spyder
device/motorola/spyder
Only the last one is valid.
With this change, both spyder and umts_spyder get resolved correctly.
Change-Id: I0c95ff126d827a2d6b58875dfcb9327e999f4913
* Specify the ota zip name as the only parameter to extract-files.sh
* Will extract to $CM_ROOT/system_dump
* Bail out on A/B OTA zips. We cannot support these.
* Handles block based OTA zips by using sdat2img.py
* Store the zip's MD5 and check if its already extracted. If so, don't
bother extracting again
Change-Id: I03038e38dac51e6cb60d493c7e6362754d1daf02
* Fixes errors like UnicodeEncodeError: 'ascii' codec can't encode
character u'\xb0' in position 51: ordinal not in range(128) with weird
characters in a commit message
Change-Id: I700198481630b4b29fd6d5d1f0c79bb94e59ad86
* On some machines, LC_ALL=C is set, causing the sort
order of makefiles to be different than those without
this locale change that is set
* Set LC_ALL=C on all machines for consistent ordering
* From 'man sort':
The locale specified by the environment affects sort order.
Set LC_ALL=C to get the traditional sort order that uses native byte values.
* Thanks to haggertk and rashed for verifying this for me
Change-Id: I5bbb94b0dfe599a67036de4312e8af913e9b8d30
Signed-off-by: Paul Keith <javelinanddart@aidenswann.com>
Currently, the priority of the src and dest is inverted between
pulling from adb and pulling from a system dump.
Assume that we have a camera wrapper and write the proprietary-files.txt
lib/hw/camera.msm8996.so:lib/hw/camera.vendor.msm8996.so
If we pull from a phone running Lineage that has both files, we get
camera.vendor.msm8996.so
as the pulled blob. If we take the exact same build and pull it
from the system dump (aka, your own $OUT directory that built the
installed software) you get
camera.msm8996.so
pull instead!
Make both paths follow the same logic so that you get the same
file independent of the source.
Change-Id: I479e0ae765339cc38fa05fcaad7943c528129463
* Avoids:
Cleaning output directory (./../../../vendor/.../.../proprietary)..
mv: cannot stat './../../../vendor/.../.../proprietary/*':
No such file or directory
Change-Id: I3258e64f3997f139d8db4fb4e07d14964a469c78
Let a device specify INITIAL_COPYRIGHT_YEAR=XXXX in setup-makefiles.sh
so that the correct copyright(s) is/are included in vendor makefiles.
The CyanogenMod copyright is retained when INITIAL_COPYRIGHT_YEAR<2017
and the LineageOS copyright is now included for >=2017.
Change-Id: I6895b5b69bc7ba399042ac3c29e17f3209d15f1b
-e no longer exists starting from v2.2b3, so update the
command. While at it, add the logic to deal with Android N
separated boot oat files.
Change-Id: Iab8e3946d6421824fea74e26dc43d2db38573022
* Some projects with multiple merges and
discountinuous histories could trigger
an error here when the HEAD~{?} does
not exist, hence avoid this by ignoring
the commit and continue the search
Change-Id: Ice28c87d1bf8897da52236b637bbb0c5d349f848
* In a situation where the Android tree is already
fetched and entirely set for a device tree,
all additions to cm.dependencies in a common
project were ignored due to being already fetched
* Add a new array of projects to verify but not
necessarily fetch, and fill it upon fetch request
or if the project starts with "android_device_",
to let the iterations follow the common trees
Change-Id: I9eea0ab9c46a4bed8366baa37a4e7e8696481504
Signed-off-by: Adrian DC <radian.dc@gmail.com>
Do not prepend app or priv-app to the src when a path is explicitly
given as an argument. This fixes the SRC path in such conditions.
Change-Id: I43370f17fe224df323fcfa9b0d5eae4ee2996524
In some cases we may not want to check against TARGET_DEVICE so allow
setting a custom variable to check against
Change-Id: Ia2fb338f453137a95a59c6940b0cc16b261505bf
In some circumstances we might want to set a custom vendor.mk
filename. If not set, then just default to the current device name
Change-Id: Icb59ebee67ce17b41ac613020685773efb01103d
* We might want to disable pinning entirely for an extract run.
* Set DISABLE_PINNING=1 to turn it off and override everything.
Change-Id: I1be90dda68f0de1c5f5a70c946052d70bfaab7ed
* No need to look for /sbin/recovery as adbd will report
the device as being in recovery mode already. Also, this
check appears to hang on N.
Change-Id: I3bcb427835ae3cd37c7ea56cbc57bba0d18fcc04
* In many cases, we would like to keep certain files which do not
exactly match what might be extracted from a factory ROM. This
becomes extremely annoying over time to manually reconstruct,
and it's easy to miss these special cases when updating to a
new vendor release. It's also useful to flag additions which
aren't found in the upstream release at all.
* To solve this, we can now "pin" files to a specific sha1 hash.
Simply append the sha1sum of the file to the appropriate line
in your bloblist, prepended by a | delimiter.
* This works by backing up the current files first, running the
extraction, then checking if any pinned files need to be
restored.
* Also add an exit trap to clean up all of our tempfiles
Change-Id: I2010b5175b5701e19a3efb112e8907062ca37d66
* Not gonna work without preparsing all the lists combined together.
This reverts commit b7b7f88443.
Change-Id: I63ceead414d7f8416e2c2c8e4b12098077b545e2
* A final "rootfs" tag in the last column will place
the chosen file into the ramdisk
* Currently only supports files in /sbin
* Cleaned up usage of class-specific flags
* Simplified code
Change-Id: Id823d70aab51f8767522f697eaf65f52fc64b94b
* Instead of having to enumerate packages and do a bunch of tedious
shit, we now have a way to do it automatically based only on
the blob lists.
* Devices still need to implement a short script, but this library
handles all the core functions.
* Yes, we should probably be doing this in Python- Bash is
absolutely horrible at dealing with arrays and lists.
Change-Id: I73b30ff4531c4d1b33ac53db33abf62555f500b4
Vladimir Oltean
Gabriele M
Joe Maples
Joey
Michael Bestas
Ethan Chen
Paul Keith
LuK1337
nailyk
Dan Pasanen
conbold
Adrian DC
Sam Mortimer
Rashed Abdel-Tawab
Christian Oder
M1cha
Luca Stefani
Harry Youd
Dobroslaw Kijowski
Michael Gernoth
Paul Keith
Christopher R. Palmer
Matt Mower
Luca Stefani
Ricardo Cerqueira
Simon Shields
Elektroschmock
HashBang
Michael Bestas
Steve Kondik
Louis Popi
Bruno Martins
Adnan Begovic