Database Info

Download of RC19 from Google Found!!!

https://android.clients.google.com/updates/signed-kila-ota-114235-prereq.TC4-RC19.zip
Not sure when this is from. I was tooling around trying to find a way to flash myself off of the RC29 test build that I stupidly put on. Stumbled across this. Tried to flash it over my RC29-Test, and it failed with the following error:
Code:
E:Failure at line 1:
assert file_contains(SYSTEM:build.prop", "test-keys"0 == "true" || file_contains
("SYSTEM:build.prop", "ro.build.fingerprint=tmobile/kila/dream/trout:1.0/
TC4-RC19/109652:user/ota-rel-keys,release-keys") == "true"
Installation aborted.
My build contains the following line:
Code:
ota-rel-keys,for-testing,release-keys
Let me know what y'all think.

I think its something good to add to my collection Thx

Darkrift said:
I think its something good to add to my collection Thx
Click to expand...
Click to collapse
I think it will be good for doing file diffs when we get to that point.

Thanks, It is good to have this...

Has anyone thought of trying to apply these updates to the RC1 from the emulator?

yeah, some guy named neoobs thought of it, not sure if he tried it yet though

Darkrift said:
yeah, some guy named neoobs thought of it, not sure if he tried it yet though
Click to expand...
Click to collapse
If I knew how to apply them I would have tried... but how do you hold the home button and the power button on the emulator? LOL

can you mis-match specific files?
I wonder if you can mis-match specific files in the updates.
I.e I wonder if you copied an RC19 file and built a custom version - but also copied the signatures into the appropriate files - since the file is still signed and it would still verify, I think it would still work to grab old versions and mix in with new versions.
The usefullness of making a frankenbuild is probably "not much" - but perhaps if there is a vulnerable file in one version it could be injected into a later version.

RyeBrye said:
I wonder if you can mis-match specific files in the updates.
I.e I wonder if you copied an RC19 file and built a custom version - but also copied the signatures into the appropriate files - since the file is still signed and it would still verify, I think it would still work to grab old versions and mix in with new versions.
The usefullness of making a frankenbuild is probably "not much" - but perhaps if there is a vulnerable file in one version it could be injected into a later version.
Click to expand...
Click to collapse
That won't work because there are 2 files with signatures... one of which is signed in the other. So if you change the signature file it will not validate. Pretty smart if you ask me.

neoobs said:
That won't work because there are 2 files with signatures... one of which is signed in the other. So if you change the signature file it will not validate. Pretty smart if you ask me.
Click to expand...
Click to collapse
I don't see where the one signs the other - the only META-INF file that is signed is the update-script that I can see in either file.
I do see in the code where it EXPLICITLY does NOT verify the .SF files however:
Code:
for (i = 0; i < mzZipEntryCount(pArchive); ++i) {
const ZipEntry *entry = mzGetZipEntryAt(pArchive, i);
UnterminatedString fn = mzGetZipEntryFileName(entry);
int len = mzGetZipEntryUncompLen(entry);
// Don't validate: directories, the manifest, *.RSA, and *.SF.
if (entry == mfEntry) {
LOGV("Skipping manifest %.*s\n", fn.len, fn.str);
} else if (fn.len > 0 && fn.str[fn.len-1] == '/' && len == 0) {
LOGV("Skipping directory %.*s\n", fn.len, fn.str);
} else if (!strncasecmp(fn.str, "META-INF/", 9) && (
!strncasecmp(fn.str + fn.len - 4, ".RSA", 4) ||
!strncasecmp(fn.str + fn.len - 3, ".SF", 3))) {
LOGV("Skipping signature %.*s\n", fn.len, fn.str);
} else {
unverified[i] = true;
totalBytes += len;
}
}
So I think it is entirely possible to swap in other files that are signed with the same key from an older version. If they change keys, I don't think it will work.

Another thought...
It doesn't make sense to me that the public key would be distributed along with the update - it looks like the phone is using the public key to verify the signed hashes - which is fine - they can see that the hashes were signed by the private-key pair of that RSA file...
But the .RSA is explicitly not verified, and I don't see in the verify.c file where it checks against any kind of keystore on the device...
I wonder if it is just checking that the files are signed, but not necessarily who they are signed by? Meaning - it's making sure the update hasn't been corrupted or tampered with - but not who made the update in the first place?

RyeBrye said:
I don't see where the one signs the other - the only META-INF file that is signed is the update-script that I can see in either file.
I do see in the code where it EXPLICITLY does NOT verify the .SF files however:
Code:
for (i = 0; i < mzZipEntryCount(pArchive); ++i) {
const ZipEntry *entry = mzGetZipEntryAt(pArchive, i);
UnterminatedString fn = mzGetZipEntryFileName(entry);
int len = mzGetZipEntryUncompLen(entry);
// Don't validate: directories, the manifest, *.RSA, and *.SF.
if (entry == mfEntry) {
LOGV("Skipping manifest %.*s\n", fn.len, fn.str);
} else if (fn.len > 0 && fn.str[fn.len-1] == '/' && len == 0) {
LOGV("Skipping directory %.*s\n", fn.len, fn.str);
} else if (!strncasecmp(fn.str, "META-INF/", 9) && (
!strncasecmp(fn.str + fn.len - 4, ".RSA", 4) ||
!strncasecmp(fn.str + fn.len - 3, ".SF", 3))) {
LOGV("Skipping signature %.*s\n", fn.len, fn.str);
} else {
unverified[i] = true;
totalBytes += len;
}
}
So I think it is entirely possible to swap in other files that are signed with the same key from an older version. If they change keys, I don't think it will work.
Click to expand...
Click to collapse
in one of the SF's (Signature Files HAHA never caught that till now) It has the SHA1-digest of the other. I believe it is Cert.sf, at the top it signs manifest.sf
This is why you can not alter the SF's at all because they both have all the files signed with one of them signing the other. Making it impossible to get around. And with SHA1 it just makes it hard to even think about it. Might as well do as others have said and find exploits.

RyeBrye said:
It doesn't make sense to me that the public key would be distributed along with the update - it looks like the phone is using the public key to verify the signed hashes - which is fine - they can see that the hashes were signed by the private-key pair of that RSA file...
But the .RSA is explicitly not verified, and I don't see in the verify.c file where it checks against any kind of keystore on the device...
I wonder if it is just checking that the files are signed, but not necessarily who they are signed by? Meaning - it's making sure the update hasn't been corrupted or tampered with - but not who made the update in the first place?
Click to expand...
Click to collapse
I like that idea... anyone know how to sign the files with your own rsa?

RyeBrye said:
It doesn't make sense to me that the public key would be distributed along with the update - it looks like the phone is using the public key to verify the signed hashes - which is fine - they can see that the hashes were signed by the private-key pair of that RSA file...
But the .RSA is explicitly not verified, and I don't see in the verify.c file where it checks against any kind of keystore on the device...
I wonder if it is just checking that the files are signed, but not necessarily who they are signed by? Meaning - it's making sure the update hasn't been corrupted or tampered with - but not who made the update in the first place?
Click to expand...
Click to collapse
Aperantly not many people know about private-public key cryptography.
Please, if you have the time, wiki private-public key cryptography or download the podcast from TWiT (This Week in Tech) Security now on that subject (episode ~15 or so, it was early on).
A simple explination:
Two keys are generated at random (or user input through an algorithm). A private key and a public key. The private key is used to encrypt files an ONLY the public key can decrypt it. IT IS NOT REVERSABLE.
The public key can be "publicly known", hence the name. It actually has to either be with the files or on the device that needs to unencrypt said files.
Now here's the awsome part a brute force attack on SHA1 would take millions of years to crack ON ONE KEY. (Much more detail to that)
Another added mesure of security is hash checking. Run a file through an algorythm and you get an unique ID for that file. You change as much as ONE BIT on that file and the output of the algorythm is COMPLETLY different.
Not imposible to crack, imposible to stay alive long enough to see it done.

Thats the Thing with PKey and Private Key and it Impossible to crack unless the Boot loader is not checking for Signing Authority.
Please Understand Signing a File and Encrypting the File is Different thing we have G1 Update.zip as Normal Zip file but The moment we change any byte in Zip Store the Checking gets failed. if G1 Boot loader is really not checking Singer and only rely on Signed Object whosmever may have signed then we got the Solution but i dont either HTC or Google is that foolish to do that.
We must have to look at G1 Source code for Details.
I think what cmonex and Olipro does is better. Get Source code of Boot loader Recompile it with disabled Signing Funda. Load the Boot Loader into Memory and that Jump to that Location. here Root access to Device is Necesory once we are Succefull to run Patched Bootloader from Memory Flashing Customised Image may not be a Problem

snoslicer8 said:
https://android.clients.google.com/updates/signed-kila-ota-114235-prereq.TC4-RC19.zip
Not sure when this is from. I was tooling around trying to find a way to flash myself off of the RC29 test build that I stupidly put on. Stumbled across this. Tried to flash it over my RC29-Test, and it failed with the following error:
Code:
E:Failure at line 1:
assert file_contains(SYSTEM:build.prop", "test-keys"0 == "true" || file_contains
("SYSTEM:build.prop", "ro.build.fingerprint=tmobile/kila/dream/trout:1.0/
TC4-RC19/109652:user/ota-rel-keys,release-keys") == "true"
Installation aborted.
My build contains the following line:
Code:
ota-rel-keys,for-testing,release-keys
Let me know what y'all think.
Click to expand...
Click to collapse
can anyone post this file somewhere i can get it? it's not there anymore. My PC no longer see's my device, i'm wondering if flashing back to RC19 will work.

canOpoop said:
can anyone post this file somewhere i can get it? it's not there anymore. My PC no longer see's my device, i'm wondering if flashing back to RC19 will work.
Click to expand...
Click to collapse
First, you can't flash the RC19 if you have RC29.
Second, what do you mean your PC doesn't see your device? If your talking about the SD card through the phone, at the top of your phone when you have connected, you will have a notification at the top of your phone.
Either drag the notification bar or press menu on your home screen and click on Notifications. Select the USB selectionan it will prompt you, select "Mount"

quedijo said:
First, you can't flash the RC19 if you have RC29.
Second, what do you mean your PC doesn't see your device? If your talking about the SD card through the phone, at the top of your phone when you have connected, you will have a notification at the top of your phone.
Either drag the notification bar or press menu on your home screen and click on Notifications. Select the USB selectionan it will prompt you, select "Mount"
Click to expand...
Click to collapse
First: check this out. http://www.blogsdna.com/1256/how-to-quickly-update-t1-mobile-g1-phone-firmware-manually-rc29.htm
i'm thinking if i get the file (RC19) i can flashback.
Second: It had worked before (before the OTA update). I have tried 3 different PC's 2 different USB cables, REFlash of RC29, and a hard reset (to Factory). i do not ever get a notification to mount in Notification bar. Windows throws out the error "One of the USB Devices attached to this computer has malfuctioned, and windows does not recognize it." And yes i have tried different USB ports.
i'm open to any more ideas...

canOpoop said:
First: check this out. http://www.blogsdna.com/1256/how-to-quickly-update-t1-mobile-g1-phone-firmware-manually-rc29.htm
i'm thinking if i get the file (RC19) i can flashback.
Second: It had worked before (before the OTA update). I have tried 3 different PC's 2 different USB cables, REFlash of RC29, and a hard reset (to Factory). i do not ever get a notification to mount in Notification bar. Windows throws out the error "One of the USB Devices attached to this computer has malfuctioned, and windows does not recognize it." And yes i have tried different USB ports.
i'm open to any more ideas...
Click to expand...
Click to collapse
Use anycut and create a shortcut to the activity "SD Card" and see what it says when you open it.

neoobs said:
Use anycut and create a shortcut to the activity "SD Card" and see what it says when you open it.
Click to expand...
Click to collapse
Tmobile is replacing device. I should get a new on Tuesday the 11th. they are out of stock.

[App] [2.3+] OTA Verifier

Whenever there's an over-the-air (OTA) update sent-out or made available (either leaked or official), there's always a large number of users that end-up reporting that their installations have failed with the infamous and dreaded "E:Error in /sdcard/xxx.zip (Status 7) installation aborted." error message. So, I decided to try to write an app that would help folks figure-out what the issues might be so that they can get their OTA update installed.
The OTA Verifier app can be used to evaluate an over-the-air (OTA) update.zip or other flashable .zip file before you attempt the install or afterwards to help you figure-out why the installation may have failed. OTA Verifier will point-out what files and conditions are being tested so that you can more easily attempt to correct these issues.
Play Market Link (free!): OTA Verifier (beta)
Do I have to be rooted to use this?
No, but rooted devices will have more conditions that the app will be able to test/evalute because it will have access to protected files that non-rooted devices don't.
Will the app actually do the installation or change anything on my device?
No. The app will only evaluate the expressions and conditions contained inside the updater-script file inside the .zip file that's trying to be installed. None of the functions or commands that try to modify your device (i.e., delete/patch/format/extract, etc.) will be evaluated or executed. The app basically operates in "read-only" mode with respect to the .zip file being evaluated.
Will this app work on older devices or flashable .zip files that use the amend update-script files?
No, this app evaluates and interprets the newer edify updater-script files (notice the "r" in "updater").
How do I use this app?
Install and launch the app
Click the "Select File" button and navigate to desired .zip file
Long-press (press and hold) the file entry for the desired .zip file
Click the "Verify OTA .zip" file pop-up
Wait for the app to process the file (should take under a minute, depending on your device)
View the displayed results (text will also be copied to the clipboard)
Miscellaneous info:
1. Savvy root users probably already know that after a failed .zip file installation, you should be able to view the /cache/recovery/log file to view the information about what might have failed. OTA Verifier will try to uncover all of the issues and not just the first one that causes the installation to fail.
2. While I wrote this app principally to evaluate OTA update.zip files, the edify updater-script files are used and written by ROM devs and others who created flashable .zip files. This app can be used to evaluate those file's updater-script files, too.
How does this all work?
Here's the basic outline/structure of what the app does:
1. the .zip file is selected by the user via the file selector
2. list of frozen system apps are identified for later reporting
3. the updater-script file is extracted from the META-INF/com/google/android folder in the .zip file
4. this updater-script file is parsed and converted into reverse polish notation (RPN) for execution
5. edify functions that might modify your device are NOT evaluated; these include apply_patch, delete, delete_recursive, format, mount, package_extract_dir/file, run_program, set_perm[_recursive], symlink, unmount, write_raw_image, etc.
6. note: the update-binary executable is not used by or involved in this app; the edify script interpreter that the app uses was written by me, in Java, specifically for this app
7. the remaining edify script functions that test conditions (such as apply_patch_check, apply_patch_space, assert, concat, file_exists, file_getprop, getprop, greater_than_int, if-then-else-endif, ifelse, is_substring, less_than_int, read_file, ui_print, and various operators (!, !=, &&, (, ), ;, ||, +, ==)) are evaluated using an operand/operator stack from the RPN expression parsed from the updater-script statements
8. expressions that fail (return a null-string) or are bypassed (usually because a resource (file/partition) is protected/secured against read-access) are reported for the user
9. after the entire script has been processed, the results are displayed in a pop-up window on the device and the text of those results are copied to the clipboard
Planned / future features:
- preferences / settings
- logging of the output to a file on the /sdcard
- test if .zip file is signed or not
- display more stats
- "explanation mode" to interpret, in English, what the edify commands are testing
Successfully tested on:
Samsung Galaxy Nexus (CDMA) 4.1.1
Samsung Galaxy Nexus (GSM) 4.1.2
Asus Nexus 7 tablet 4.1.2
Motorola Droid X 2.3.4
The following edify commands/functions/operators are supported:
Code:
[COLOR="blue"]apply_patch_check[/COLOR] // apply_patch_check ( <filepath>, <sha1-checksum> [ , <sha1-checksum> ... ] )
// apply_patch_check ( <compoundvalue> )
<compoundvalue> := [ EMMC : <filepath> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
[ MTD : <dev-block> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
[ vfat : <dev-block> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
<dev-block> := filename in /dev/block
[COLOR="Blue"]apply_patch_space[/COLOR] // apply_patch_space ( <bytes> )
[COLOR="blue"]assert[/COLOR] // assert ( <expression> )
[COLOR="blue"]concat[/COLOR] // conact ( <string> , <string> )
[COLOR="blue"]file_exists[/COLOR] // file_exists ( <filepath> )
[COLOR="blue"]file_getprop[/COLOR] // file_getprop ( <filepath>, <propertyname> )
[COLOR="blue"]getprop[/COLOR] // getprop ( <propertyname> )
[COLOR="blue"]greater_than_int[/COLOR] // greater_than_int ( <integer>, <integer> )
[COLOR="blue"]if then else endif[/COLOR] // if <condition> then <expression> [ else <expression> ] endif
[COLOR="blue"]ifelse[/COLOR] // ifelse ( <condition>, <then-expression> [ <else-expression> ] )
[COLOR="blue"]is_substring[/COLOR] // is_substring ( <string> , <string> )
[COLOR="blue"]less_than_int[/COLOR] // less_than_int ( <integer>, <integer> )
[COLOR="blue"]read_file[/COLOR] // read_file ( <filepath> )
[COLOR="blue"]ui_print[/COLOR] // ui_print ( <string> ) parsed & executed, but output not displayed
[COLOR="blue"]![/COLOR] // logical NOT operator
[COLOR="blue"]!=[/COLOR] // not equals operator
[COLOR="blue"]&&[/COLOR] // logical AND operator
[COLOR="blue"]([/COLOR] // open paren: precedence / grouping
[COLOR="blue"])[/COLOR] // close paren: precedence / grouping
[COLOR="blue"];[/COLOR] // sequence/imperative (terminates & separates statements)
[COLOR="blue"]||[/COLOR] // logical OR operator
[COLOR="blue"]+[/COLOR] // string concatenation operator
[COLOR="blue"]==[/COLOR] // equals operator
[COLOR="blue"]#[/COLOR] // comment line
The following edify functions are NOT supported (the majority of them because they are intended to modify your device):
Code:
[COLOR="blue"]apply_patch[/COLOR]
[COLOR="blue"]delete[/COLOR]
[COLOR="blue"]delete_recursive[/COLOR]
[COLOR="blue"]format[/COLOR]
[COLOR="blue"]mount[/COLOR]
[COLOR="blue"]package_extract_dir[/COLOR]
[COLOR="blue"]package_extract_file[/COLOR]
[COLOR="blue"]run_program[/COLOR]
[COLOR="blue"]set_perm[/COLOR]
[COLOR="blue"]set_perm_recursive[/COLOR]
[COLOR="blue"]symlink[/COLOR]
[COLOR="blue"]unmount[/COLOR]
[COLOR="blue"]write_raw_image[/COLOR]
[COLOR="blue"]is_mounted[/COLOR] // innocuous, but still not supported
[COLOR="blue"]stdout[/COLOR] // innocuous, but still not supported
[COLOR="blue"]show_progress[/COLOR] // innocuous, but still not supported
[COLOR="blue"]set_progress[/COLOR] // innocuous, but still not supported
Keywords:
OTA, over-the-air, updater-script, edify, amend
Screenshots:
updated 9-May-2014

updated to version 1.3
Updated app with root-enabled file browser for navigating to protected/secured directories like /cache to select OTA's update zip file.
Full change log to date:
[v1.3 - 04-Nov-2012]:
- for rooted devices, change file browser to allow navigating and selecting .zip files in protected directories (like /cache)
[v1.2 - 29-Oct-2012]:
- minor fix to handle two reports of null pointer exceptions
[v1.1 - 28-Oct-2012]:
- remove unused SD card write permission
[v1.0 - 28-Oct-2012]:
- initial Play Market release

This is pretty cool.
Any chance you could direct me to an explanation for sha1-size (for apply_patch_check) is though? I'm working on some edify scripting stuff and trying to incorporate some asserts for a specific bootloader (important to what I'm doing), and this is honestly the best reference I've found for it yet.

osm0sis said:
This is pretty cool.
Any chance you could direct me to an explanation for sha1-size (for apply_patch_check) is though? I'm working on some edify scripting stuff and trying to incorporate some asserts for a specific bootloader (important to what I'm doing), and this is honestly the best reference I've found for it yet.
Click to expand...
Click to collapse
You bet!
I'm heading home from work here soon and I'll dig-up the details for you .
(yeah, I scoured the internet for detailed information on Edify scripting as well as reviewed the actual Google/Android code--there's not a complete compendium, unfortunately).
I'll ping you back after a bit...
Cheers!

osm0sis said:
Any chance you could direct me to an explanation for sha1-size (for apply_patch_check) is though?
Click to expand...
Click to collapse
Okay, here's what I know about the apply_patch_check command:
There are basically two formats:
Code:
apply_patch_check // apply_patch_check ( <filepath>, <sha1-checksum> [ , <sha1-checksum> ... ] )
// apply_patch_check ( <compoundvalue> )
<compoundvalue> := [ EMMC : <filepath> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
[ MTD : <dev-block> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
[ vfat : <dev-block> : <sha1-size> : <sha1-checksum> [ <sha1-size> : <sha1-checksum> ... ]
The first of which is what I think you typically see in updater-script files where you have a filename, followed by one or more SHA1 digests. Here's an example:
assert(apply_patch_check("/system/app/ApplicationsProvider.apk", "41bb5aaaa2791e68b55622fcca13f0e4efa757b2", "fc845332ae7f706824de73f64ae47f93866ad245"));​
The second format is what I call a compound value format, where the file or partition to be checked and the SHA1 digests to be compared, are all concatenated together in a single, colon-separated value. For example:
assert(apply_patch_check("EMMC:/dev/block/platform/omap/omap_hsmmc.0/by-name/boot:4247552:3f4d4f9549d307d152f8503983ee4ff5f46b0a43:4470784:fbd13c778b34fdb7917c1ccee6389aa9b13a92bd")); ​
In the above, I've colored the sizes in red of file/partition on which to compute the SHA1 checksum (colored in black). I figured this out and verified this by using dd to copy the portions of the partition in question and calculating the SHA1 checksums on it.
My app only supports parsing and evaluating two sets of lengths/SHA1s for the compound format at this time (that's all I've encountered so far in the scripts that I've viewed).
Does that help and/or answer your questions?
Cheers!

Thanks! Yeah that helps a lot, but unfortunately it doesn't seem to solve the problem I've been seeing..
Using dd to grab the partition img I can sha1sum that and that SHA1 doesn't allow it to pass the assert, or even a direct sha1_check of the partition.
Code:
run_program("/sbin/busybox", "dd", "if=/dev/block/platform/sdhci-tegra.3/by-name/USP"), "of=/tmp/bootloader.img")
ui_print(sha1_check(read_file("/tmp/bootloader.img"))); = 8c206a1a87599f532ce68675536f0b1546900d7a (also, bootloader.img is 2097152 in size)
ui_print(sha1_check(read_file("/dev/block/platform/sdhci-tegra.3/by-name/USP"))); = da39a3ee5e6b4b0d3255bfef95601890afd80709
assert(apply_patch_check("EMMC:/dev/block/platform/sdhci-tegra.3/by-name/USP:2097152:8c206a1a87599f532ce68675536f0b1546900d7a")); = Fail
They just don't match and I'm completely baffled by it. I can work around it by using dd to pull the img first and using that for my asserts but it's not as clean as I'd like it.

osm0sis said:
Yeah that helps but I guess unfortunately it doesn't entirely solve the problem I've been seeing.
Using dd to grab the partition img I can sha1sum that and that SHA1 doesn't allow it to pass the assert, or even a direct sha1_check of the partition.
They just don't match and I'm completely baffled by it. I can work around it by using dd to pull the img and using that for my asserts but it's not as clean as I'd like it.
Click to expand...
Click to collapse
Is it for the Gnex or the N7 (I have those same devices)?
I'd be happy to test something for you...just PM me or send me an email ([email protected]).
I don't know if you're using my app to test with, but you can also manually run the update-binary directly on/from your phone (i.e., you don't have to run it in recovery, but you obviously need to be careful what your updater-script does ).
I'm still looking for the exact syntax in my notes, but I'll edit this post when I find it...brb.
~ ~ ~
edit: http://wiki.opticaldelusion.org/wiki/Update-binary shows this:
update-binary <api> <pipefd> <zip>
Click to expand...
Click to collapse
I know I did this during my early testing, but I can't remember the exact syntax...I'll try it and re-re-edit .

Man thanks so much, PM sent.
Sorry for cluttering up your thread with this semi-OT stuff.

osm0sis said:
Man thanks so much, PM sent.
Sorry for cluttering up your thread with this semi-OT stuff.
Click to expand...
Click to collapse
LOL, not a problem...it's all Edify-related and updater-script related...I'm betting that's what'll lead most folks here.
Happy to help .

==================
Detailed Analysis
==================
- you have no frozen system apps (good!)
- there were 2 failed expressions; see below for the details about these tests that will cause your OTA install to fail
- there were no bypassed expressions (good!)
-------------------------------
statistics:
frozen system apps: 0
success count: 526
fail count: 2
ignore count: 1587
partition count: 2
protected count: 2
bypassed count: 0
-------------------------------
updater-script analysis details:
FAILED: line #268:
assert(apply_patch_check("/system/app/XT9IME.apk", "8aba56a4406128e78f5729753252c3d93bc21cb4", "965b437bce65018eeb31ff9a381c3687542099e0"));
----------------------
FAILED: line #1038:
assert(apply_patch_check("EMMC:/dev/block/platform/sdhci-tegra.3/by-name/LNX:5013504:c48f8e86c73fb2c2ba1794f5ec98e27c9e206ed5:5060608:319331fae14fec8a88063751475fce26bae328e0"));
So a question, could this failure above be causing my 32 GB nexus 7 to have system update issues? (Not necessarily XT9IME.apk (which I shouldn't have deleted), more the other one) Is there a fix?
Sent from my Nexus 7 using Tapatalk HD
---------- Post added at 10:44 AM ---------- Previous post was at 10:39 AM ----------
==================
Detailed Analysis
==================
- you have no frozen system apps (good!)
- there were 2 failed expressions; see below for the details about these tests that will cause your OTA install to fail
- there were no bypassed expressions (good!)
-------------------------------
statistics:
frozen system apps: 0
success count: 526
fail count: 2
ignore count: 1587
partition count: 2
protected count: 2
bypassed count: 0
-------------------------------
updater-script analysis details:
FAILED: line #268:
assert(apply_patch_check("/system/app/XT9IME.apk", "8aba56a4406128e78f5729753252c3d93bc21cb4", "965b437bce65018eeb31ff9a381c3687542099e0"));
----------------------
FAILED: line #1038:
assert(apply_patch_check("EMMC:/dev/block/platform/sdhci-tegra.3/by-name/LNX:5013504:c48f8e86c73fb2c2ba1794f5ec98e27c9e206ed5:5060608:319331fae14fec8a88063751475fce26bae328e0"));
Sent from my Nexus 7 using Tapatalk HD
Click to expand...
Click to collapse
So a question, could this failure above be causing my 32 GB nexus 7 to have system update issues? (Not necessarily XT9IME.apk (which I shouldn't have deleted), more the other one) Is there a fix?

modwilly said:
So a question, could this failure above be causing my 32 GB nexus 7 to have system update issues? (Not necessarily XT9IME.apk (which I shouldn't have deleted), more the other one) Is there a fix?
Click to expand...
Click to collapse
Looks like you've updated the boot (LNX) partition in addition to deleting (or renaming/moving) the XT9IMG.apk?
So yes, the OTA will not install until you've put both items back to their expected state.
The fix, of course, depends on exactly what you did to change them in the first place.

version 2.1 uploaded to Play Store
Major re-write of app done for version 2.0 and above.
Recent change log:
[version 2.1 - 6-May-2014]:
- fix issue w/identification of non-existent files
[version 2.0 - 5-May-2014]:
- major app update: edify parsing, RPN processing and execution engine completely re-written
- original core app behavior and functionality remain the same, but code cleanup and re-write should mean better and more robust handling of future OTA updater-script expressions
Click to expand...
Click to collapse
Thanks and let me know if you have any questions.

@scary alien could you please tell me how to get apply_patch_space bytes? i am making ota script and i got almost all, still need to generate:
apply_patch_space(bytes) || abort("Not enough free space on /cache to apply patches.");
Click to expand...
Click to collapse
i just dunno how

ZduneX25 said:
@scary alien could you please tell me how to get apply_patch_space bytes? i am making ota script and i got almost all, still need to generate:
i just dunno how
Click to expand...
Click to collapse
It's just the available space (in bytes) in the /cache partition that you want to make sure is available (i.e., free space) for any operations your updater-script file will do concerning /cache.
For example:
apply_patch_space(1000000) || abort("Not enough free space on /cache to apply patches.");
Click to expand...
Click to collapse
Would verify that there is at least 1MB (1,000,000) free bytes available in /cache.
Does that answer what you're asking?
Lemme know--happy to help if I can.
Cheers!

@scary alien not really, i mean i know how it works, just dunno how to generate proper size in updater, for example:
i create regular ota: multiple .p files and some images, zipped, signed OTA.zip size is 20mb, /patch size is 7 mb, images 10 mb and /system (new files) 3mb, how do i know how many bytes i should set to make this ota install in recovery?
should i summarize .p files size or target (patched apk, jar) files size (would be around 90mb) or what else?

I think the size would depend on when your patch files are cleaned-up...i.e., after each patching operation or at the end. If its at the end, you'll of course need to account to all of the space you might use in /cache.
I don't know of a good way to tell you what the high water mark would be other than testing and recording the output of a "df /cache" command at various points in your updater-script file.

I could do that however I don't see it universal or handy, each ota has different size so it would need more/less free space every time.
There is how Google gets this value : https://github.com/MiCode/patchrom_tools/blob/kitkat/releasetools/edify_generator.py#L131 maybe you will understand better.

ZduneX25 said:
I could do that however I don't see it universal or handy, each ota has different size so it would need more/less free space every time.
There is how Google gets this value : https://github.com/MiCode/patchrom_tools/blob/kitkat/releasetools/edify_generator.py#L131 maybe you will understand better.
Click to expand...
Click to collapse
I think I understood what you meant...there's no easy way to make this universal for each OTA except to evaluate each file that might need to use space resources from the /cache partition.
Even the edify-generator git that you referenced does this dynamically:
releasetools/ota_from_target_files:
Code:
for diff in diffs:
tf, sf, d = diff.GetPatch()
if d is None or len(d) > tf.size * OPTIONS.patch_threshold:
# patch is almost as big as the file; don't bother patching
tf.AddToZip(output_zip)
verbatim_targets.append((tf.name, tf.size))
else:
common.ZipWriteStr(output_zip, "patch/" + tf.name + ".p", d)
patch_list.append((tf.name, tf, sf, tf.size, common.sha1(d).hexdigest()))
[COLOR="Red"]largest_source_size[/COLOR] = max([COLOR="red"]largest_source_size[/COLOR], sf.size)
:
:
:
if patch_list or updating_recovery or updating_boot:
script.[COLOR="Blue"][B]CacheFreeSpaceCheck[/B][/COLOR]([COLOR="red"]largest_source_size[/COLOR])

I have seen it, but here comes largest_source_size:
Code:
largest_source_size = 0
https://github.com/MiCode/patchrom_tools/blob/kitkat/releasetools/ota_from_target_files#L532

ZduneX25 said:
I have seen it, but here comes largest_source_size:
Code:
largest_source_size = 0
https://github.com/MiCode/patchrom_tools/blob/kitkat/releasetools/ota_from_target_files#L532
Click to expand...
Click to collapse
Yeah, that's just the initialization of that variable...it gets updated in the for-loop that I included above.
(you know that's not my tool, software, right?)

Adding new languages - testers welcome!

Hello! In 2015 i'm bulded patch for increased languages support for Galaxy S6 Edge Sprint. Now i want to make new patch for Note 5 Sprint. I haven't Note 5 and walking in the darkness - I need your help for testing!
MAKE BACKUP!
Requirements:
- Root
- Custom recovery
- Android 5.1.1 OI or higher firmware.
Additional: DeODEXed firmware(Sometimes working bad on odexed firemwares)
Installation:
- Flash Zip via TWRP
- Go Settings - Language and Input and choose your language
Download: Click Me!
Builded from OL2 version source. Post here about issues.

Installing ' /sdcard/N920P.zip '...
Checking for MD5 file...
Skipping MD5 check: no MD5 file found
Error flashing zip' /sdcard/N920P.zip
Updating partion details...
...done
failed

MA7MOD_GSM said:
Installing ' /sdcard/N920P.zip '...
Checking for MD5 file...
Skipping MD5 check: no MD5 file found
Error flashing zip' /sdcard/N920P.zip
Updating partion details...
...done
failed
Click to expand...
Click to collapse
Let's try this one link

Skarxhead said:
Let's try this one link
Click to expand...
Click to collapse
not work same prob

MA7MOD_GSM said:
not work same prob
Click to expand...
Click to collapse
Strange thing. What version of fimware on your phone?

Add languages
well work i fix your file with my little brain they have Bugs And issues

NEED to talk
hello brother can possible to can talk private ???

Any update on this or on how to fix it?

[DEV][LINUX/OSX] IMG Patch Tools | sdat2img for OTA zips

{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
most part of Introduction from taken sdat2img thread
if you want extract non OTA zip i suggest to use sdat2img tool
FOR AB OTA ZIPS USE THIS SCRIPT AND PAYLOAD EXTRACTOR​
in this tools i made block_image_update() function from android recovery to a binary to run on PC and patch system.img with DAT files on OTA package to update system.img by OTA
also we have apply_patch() function for patching boot.img and other files like firmwares
Introduction
You probably know already that starting from Android 5.x (Lollipop) compiled roms (aosp,cm,stock) are not compressed anymore the way they used to be on previous android versions. On previous versions all content inside /system folder that has to be extracted within our device was either uncompressed (simple /system folder inside our flashable zip) or compressed in a system.img file, which it is a ext4 compressed file; both of these, anyway, were readable and we could see all system files (app,framework, etc).
The problem comes in >=5.0 versions, this method is not used anymore. Why? Because roms started to be always larger, so it is necessary to compress them even more.
What does new Android zips (full roms, but also OTAs) contain?
New Android flashable zips are made this way:
boot.img (kernel)
file_contexts (selinux related)
META-INF (folder containing scripts)
system.new.dat (compressed /system partition)
system.patch.dat (for OTAs)
system.transfer.list (see explanation below)
and other patch files (.p only on OTA zip)​
What does updater-script contains then?
The updater-script uses a brand new function: block_image_update(), this method basically decompresses necessary files inside the device. Let's study it.
From google git source code, if we go inside the new file /bootable/recovery/updater/blockimg.c, we find at the end of it the registration of the function block_image_update() as the method BlockImageUpdateFn() which starts at line 254. Here finally we find all information we need to know about the decompression of the .dat file(s). First file we analyze is system.transfer.list which Google tells us:
The transfer list is a text file containing commands to transfer data from one place to another on the target partition.
Click to expand...
Click to collapse
But what each line means?:
First line is the version number of the transfer list; 1 for android 5.0.x, 2 for android 5.1.x, 3 for android 6.0.x, 4 for android 7.x
Second line is the total number of blocks being written
Third line is how many stash entries are needed simultaneously (only on versions >= 2)
Fourth line is the maximum number of blocks that will be stashed simultaneously (only on versions >= 2)
Fifth line and subsequent lines are all individual transfer commands.
Click to expand...
Click to collapse
all transfer commands is :
bsdiff
*erase
free
imgdiff
move
*new
stash
*zero
BlockImageUpdate is reading system.transfer.list and executing all commands
But BlockImageVerify doesn’t execute * commands, which not to make changes on system.img just verifying update can happen
original sdat2img tool only support "new" command
Ok, but how to Patch the system.img with OTA files?
All instructions are below. binaries are involved. Please read carefully step by step.
You can use/modify these files and/or include them in your work as long as proper credits and a link to this thread are given.
If you have questions or problems
write here
Thanks
- @xpirt , for original sdat2img tool and useful thread
XDA:DevDB Information
IMG Patch Tools, sdat2img for OTA zips, Tool/Utility for all devices (see above for details)
Contributors
erfanoabdi
Source Code: https://github.com/erfanoabdi/imgpatchtools
Version Information
Status: Testing
Created 2017-07-21
Last Updated 2020-01-23

Usage
Code:
./BlockImageUpdate <system.img> <system.transfer.list> <system.new.dat> <system.patch.dat>
args:
<system.img> = block device (or file) to modify in-place
<system.transfer.list> = transfer list (blob) from OTA/rom zip
<system.new.dat> = new data stream from OTA/rom zip
<system.patch.dat> = patch stream from OTA/rom zip
Code:
./ApplyPatch <file> <target> <tgt_sha1> <size> <init_sha1(1)> <patch(1)> [init_sha1(2)] [patch(2)]...
args:
<file> = source file from rom zip
<target> = target file (use "-" to patch source file)
<tgt_sha1> = target SHA1 Sum after patching
<size> = file size
<init_sha1> = file SHA1 sum
<patch> = patch file (.p) from OTA zip
Code:
usage: ./scriptpatcher.sh <updater-script>
args:
<updater-script> = updater-script from OTA zip to patch recovery commands
Example
for example from updater-script of OTA we have:
Code:
block_image_update("/dev/block/bootdevice/by-name/system", package_extract_file("system.transfer.list"), "system.new.dat", "system.patch.dat")
apply_patch("EMMC:/dev/block/bootdevice/by-name/boot:33554432:f32a854298814c18b12d56412f6e3a31afc95e42:33554432:0041a4df844d4b14c0085921d84572f48cc79ff4",
"-", 0041a4df844d4b14c0085921d84572f48cc79ff4, 33554432,
f32a854298814c18b12d56412f6e3a31afc95e42,
package_extract_file("patch/boot.img.p"))
after getting system.img and boot.img from firmware This is equals of previous functions on PC with this tools:
Code:
~$ ./BlockImageUpdate system.img system.transfer.list system.new.dat system.patch.dat
~$ ./ApplyPatch boot.img - 0041a4df844d4b14c0085921d84572f48cc79ff4 33554432 f32a854298814c18b12d56412f6e3a31afc95e42
scriptpatcher.sh will generate all commands automatically from updater script so run it like:
Code:
~$ ./scriptpatcher.sh META-INF/com/google/android/updater-script > fullpatch.sh
check fullpatch.sh your self, you need to provide all images and files in correct name and patch as mentioned in mount and other commands of fullpatch.sh
Building Requirements
For Building this tool you need :
zlib
libbz2
openssl
It currently supports Linux x86/x64 & MacOS, Not tested on Windows.
Compile and Build command:
Code:
make
Youtube

Download links:
GitHub Release
Changelog:
GitHub Commits
Known Bugs/Issues:
Plz test and report

Nice work

Very nice
Sent from my LG-LS997 using Tapatalk

Wow, I tried to apply OTAs manually myself, but never got it to work. Getting the propper tools from you is awesome!
One tiny question though: Does applying/verifying a system update to system.img require a lot of temporary space or memory?
I am running Ubuntu x64 in a VM (VirtualBox) on Windows 10. Verifying a system update doesn't really do much, except keeping the CPU busy.
This is the output of "./BlockImageVerify system.img system.transfer.list system.new.dat system.patch.dat":
Code:
performing verification
creating cache dir cache
cache dir: cache
blockimg version is 3
maximum stash entries 236
creating stash cache/537ed49fb5f6c32dc3d205d78b6084fe54f70cd3/
The cache folder remains empty, there is no harddisk activity and only the CPU is working at 100 %. I interrupted BlockImageVerify after 30 minutes.

daniel_m said:
Wow, I tried to apply OTAs manually myself, but never got it to work. Getting the propper tools from you is awesome!
One tiny question though: Does applying/verifying a system update to system.img require a lot of temporary space or memory?
I am running Ubuntu x64 in a VM (VirtualBox) on Windows 10. Verifying a system update doesn't really do much, except keeping the CPU busy.
This is the output of "./BlockImageVerify system.img system.transfer.list system.new.dat system.patch.dat":
The cache folder remains empty, there is no harddisk activity and only the CPU is working at 100 %. I interrupted BlockImageVerify after 30 minutes.
Click to expand...
Click to collapse
Ah actually yes you need free space on RAM as much as both patch.dat and new.dat sizes
It's on my todo list to fix it, https://github.com/erfanoabdi/imgpatchtools/blob/master/blockimg/blockimg.cpp#L383

How to extract the file system.new.dat

evangelico793 said:
How to extract the file system.new.dat
Click to expand...
Click to collapse
You can find it in OTA zip
Motorola? See this : motorola.erfanabdi.ir

erfanoabdi said:
You can find it in OTA zip
Motorola? See this : motorola.erfanabdi.ir
Click to expand...
Click to collapse
No longer look at the picture

erfanoabdi said:
You can find it in OTA zip
Motorola? See this : motorola.erfanabdi.ir
Click to expand...
Click to collapse
Something Looks Wrong ReCheck inputs Or No Update Available
for moto g4 play 6.0.1

evangelico793 said:
No longer look at the picture
Click to expand...
Click to collapse
Please send picture again
404 not found
evangelico793 said:
Something Looks Wrong ReCheck inputs Or No Update Available
for moto g4 play 6.0.1
Click to expand...
Click to collapse
You are doing something wrong see thread for help and examples

@erfanoabdi
Thank you a ton for these tools!
They work like a charm :good:
I knew there must be a way to do this, but it's definitely above my paygrade.

@erfanoabdi
I've been trying scriptpatcher.sh on various updater-scripts and have noticed some bugs.
If you're interested in looking at the files, let me know and I'll post them.

Q9Nap said:
@erfanoabdi
I've been trying scriptpatcher.sh on various updater-scripts and have noticed some bugs.
If you're interested in looking at the files, let me know and I'll post them.
Click to expand...
Click to collapse
Yeah, there's lots of bugs in that script
I appreciate any kind of help
So far i know :
Only supporting /dev/block/boot.../by-name partitions
Can't mount ext4 in macOS
I disabled verify, but we can verify images
And i haven't tested it so much
If it gets more complicated Maybe we have to rewrite it on python.

erfanoabdi said:
Yeah, there's lots of bugs in that script
I appreciate any kind of help
So far i know :
Only supporting /dev/block/boot.../by-name partitions
Can't mount ext4 in macOS
I disabled verify, but we can verify images
And i haven't tested it so much
If it gets more complicated Maybe we have to rewrite it on python.
Click to expand...
Click to collapse
*removed*

@erfanoabdi
I noticed that there was an update to the source today to enable the bonus file argument.
I tried to create a recovery.img with a boot.img, recovery-from-boot.p, and recovery-resource.dat.
I tried patching to create a new recovery image and also tried patching the boot image in place.
It always returns "Unknown patch file format".
I am able to use these files to create recovery with the on-device applypatch binary, so not sure why it's throwing the "unknown patch file format" error.

Q9Nap said:
@erfanoabdi
I noticed that there was an update to the source today to enable the bonus file argument.
I tried to create a recovery.img with a boot.img, recovery-from-boot.p, and recovery-resource.dat.
I tried patching to create a new recovery image:
#Output:
creating cache dir cache
cache dir: cache
patch boot.img:
failed to stat "recovery.img": No such file or directory
Unknown patch file format
Done with error code : 1
and also tried patching the boot image in place:
#Output:
creating cache dir cache
cache dir: cache
patch boot.img:
Unknown patch file format
Done with error code : 1
I am able to use these files to create recovery with the on-device applypatch binary, so not sure why it's throwing the "unknown patch file format" error.
Any ideas? Is my syntax incorrect?
Click to expand...
Click to collapse
That was chainfire commit changes how can i say "no" to his PR
I didn't had time to check it
I'll check it out when I could

Q9Nap said:
@erfanoabdi
I noticed that there was an update to the source today to enable the bonus file argument.
I tried to create a recovery.img with a boot.img, recovery-from-boot.p, and recovery-resource.dat.
I tried patching to create a new recovery image:
#Output:
creating cache dir cache
cache dir: cache
patch boot.img:
failed to stat "recovery.img": No such file or directory
Unknown patch file format
Done with error code : 1
and also tried patching the boot image in place:
#Output:
creating cache dir cache
cache dir: cache
patch boot.img:
Unknown patch file format
Done with error code : 1
I am able to use these files to create recovery with the on-device applypatch binary, so not sure why it's throwing the "unknown patch file format" error.
Any ideas? Is my syntax incorrect?
Click to expand...
Click to collapse
I've tested recovery-from-boot by chainfire and i can confirm its working
Also i fixed no such file error
Try to compile new source

*edit*
Fixed!

Installing a package by TWRP fails: ERROR 1

Hello! I am trying to install Kali Nethunter on my Moto G3 2015 (from here), but no matter what I do, I get the same error: Updater process ended with ERROR: 1. I tried both the 1.1 and 1.2 versions on their respective supported systems (1.1 on cyanogenmod 14.1 and 1.2 on LineageOS 15.1). TWRP log says this before aborting the update: /tmp/updater[129]: .: env.sh: No such file or directory. My phone has Magisk installed.

The reason for this ERROR 1 issue is due to a corrupt package: file env.sh is missing therein.

An env.sh file is present inside the package. If you don't believe me, download it for yourself.