Database Info

[Q] What's in the firmware blob?

I'm currently looking at the latest firmware update zip file (WW_epad_user_10_4_4_18_UpdateLauncher.zip), hoping it contains driver binaries etc. that could be reused in a custom rom (a script that downloads the latest official firmware and extracts needed bits is legal -- just distributing the drivers and all might not be).
The content is another zip file, that in turn contains a META-INF directory with information and an updater, and one huge file called "blob" that probably contains all we care about.
Has anyone investigated that "blob" file yet?
Unfortunately it doesn't seem to be an ext4 image like what you'd flash with "fastboot flash system"...
If nobody else has done it, I'll look into this more myself, but obviously if it's already been done, no need to start over.

The blob can be unpacked using BlobTools (https://github.com/AndroidRoot/BlobTools). It contains partition images:
PT ... Tegra Partition Table
EBT ... bootloader
SOS ... recovery kernel image
LNX ... Android kernel image
APP ... /system partition
The kernel images can again be split into the kernel itself and the initramfs image, and APP is an ext4 image.

Image 2 Image [ magisk module ] system, vendor image converter

Android system, vendor image converter tool
############## IMAGE 2 IMAGE ##############
Module Image Converter tools & setup v1.1
## MMT-EX .v 20.4 -Zackptg5
## Python .v 2.7.9 -Python.org
## Python .v 3.4.2 -Python.org
## Bash .v 4.3.4.2 -Bash GNU
## BusyBox .v 1.31.1 -topjohnwu
## Brotli .v 1.0.7-2 -brotli.org
Info:
For unpacking, repacking an creating system.new.dat.br, system.new.dat, system.img an vendor.new.dat.br vendor.new.dat vendor.img on android devices.
# STEPS:
Use a good android terminal emulator app , like Nethunter terminal app. (highly Recommended)
Download :
https://store.nethunter.com/repo/com.offsec.nhterm_2020030100.apk
-Grant your terminal app root: in Nethunter you would choose, android su
** You can also type " sudo " then press enter for root access **
Type without quotes " bash " then press enter
Type without quotes " ziona " then press enter
First time setup !
# INTERNAL STORAGE USAGE:
.You will be asked about which storage your using were the files to convert are located
if using internal storage then leave terminal an locate the folder named internal in that storage
place system.new.dat or dat.br an system.transfer.list here !!
use this location for your dat, dat.br an system|vendor image files
# EXTERNAL STORAGE USAGE:
.You will be asked about which storage your using were the files to convert are located
if using external storage then leave terminal an locate the folder named external in that storage
place system.new.dat or dat.br an system.transfer.list here !!
use this location for your dat, dat.br an system|vendor image files
Once your files are in one of the storage locations ,
Now go back to terminal app menu an press Enter .
# FIRST MENU:
Choose which storage you placed the files to convert
# SECOND MENU AND THIRD MENU:
Choose which convert action to do unpacking or repacking of the image file
Options:
1. unpack system.new.dat.br -------(to) system.new.dat
2. repack system.new.dat.br ------ (from) system.new.dat
3. unpack vendor.new.dat.br -------(to) vendor.new.dat
4. repack vendor.new.dat.br -------(from) vendor.new.dat
5. unpack system.new.dat ----------(to) {raw} system.img
6. repack system.new.dat ----------(from) {sparse} system.img
7. unpack vendor.new.dat ---------(to) {raw} vendor.img
8. repack vendor.new.dat ---------(from) {sparse} vendor.img
9. unpack sparse system.img -----(to) {raw} system.img
10. repack sparse system.img -----(from) {raw} system.img
11. unpack sparse vendor.img -----(to) {raw} system.img
12. repack sparse vendor.img -----(from) {raw} vendor.img
13. mount system.img
14. mount vendor.img
15. umount system.img
16. umount vendor.img
17. makefs raw system.img --------(from) {raw} system.img mounted
18. makefs raw vendor.img --------(from) {raw} vendor.img mounted
Mounting of raw system.img images made after unpacking are done automatically
Mounting of raw vendor.img images made after unpacking are done automatically
.You can also Manually mount system.img an umount system.img in internal|external
.You can also Manually mount vendor.img an umount vendor.img in internal|external
.You can type in terminal m1 to mount a system|vendor image in internal|external
.You can type in terminal u1 to unmount a system|vendor image in internal|external
Usage for storage internal|external folder convertions:
-Copy an Place your images files here for everything -- Unpacking & Repacking Also Mounting Images
system.transfer.list & system.new.dat -- needed for converting
system.transfer.list & system.new.dat.br -- needed for converting
vendor.transfer.list & vendor.new.dat -- optional for converting
file_contexts.bin & system.img -- optional to create a modified new image
.When image is converted from .dat to system.img (raw) it will be mounted
within the same external/internal storage folder look inside /system1 or 2 & /vendor1 or 2
.You can do whatever modifications there before making new image or just repacking after umount.
# NOTE CONVERTIONS:
Examples for image processing:
From; {raw} system.img > {sparse} system.img > system.new.dat & optional finally > system.new.dat.br
From; system.new.dat.br > system.new.dat > {raw} system.img
From; {sparse} system.img > {raw} system.img
~ You can repack raw system.img to sparse system.img
used this to flash images in twrp recovery to system or vendor partitions.
Remember on sparse images:
~ To unpack a sparse image to raw system.img or vendor.img you can sparse unpack,
you can also covert it to system|vendor.new.dat then to raw system|vendor.img.
(You can call help in menu by typing 4 an then enter . At main screen first menu only)
All info will display if successful or errors occur .
General logs are found in folders internal|external/logs .
Android 6 to 11 images:
.Both system an vendor images tested. Works on custom roms Lineage, CM, RR, Mokee rom images an more .
Terminal support for:
busybox, dash, bash, python2 & python3 - Shell Commands that are standalone in terminal.
For best results:
•Have 4GB or more of free internal or external storage available on device
•Operating convertions in internal storage is 3xxx faster than using external storage (sdcards)
•ALWAYS have a backup of your original images before converting an using delete old image option when preserving device storage space.
•Remember normally yellow text in the terminal when converting anything is successful and red is a error happened.
•If a image won't automatically or manually mount its more than likely because it's a bad system|vendor image.
TroubleShooting:
--Error logs are made within system*.logs in external or internal logs folder to report.
--Scroll up to view full menu options they are over 18 to choose from in terminal menu1-2 options, internal|external storage.
--If using Nethunter Terminal app if display freezes or weird behavior occurs restart app or start a new window.
--Always have just system.transfer.list with system image files in folder for convertions.
--Always have just vendor.transfer.list with vendor image files in folder for convertions.
--Always remove system.transfer.list an system image files from folder before convertions if using vendor files.
--Always have file_context.bin when creating new images from a mounted system|vendor image
--Have a image that wont convert even with the necessary files present it can mean a bad image or just incompatible with this module.
--If Nethunter Terminal app enter key stops working press the three dots in upper right corner,
and tap Preferences, now tap Function keys, then untick the (tab) option and go back to first terminal window,
now on top of keyboard theirs special functions a new (enter) tab should be on the end right corner.
--Full support for use on arm devices only
# Credits: For Tools & ideas
Kali NetHunter
@andii_nr
@Zackptg5
@topjohnwu
Python.org
Brotli.org
Bash GNU
Creator:
@Ziona
Created:
13/10/2020
Disclaimer :
* I am not responsible for bricked devices, dead SD cards, thermonuclear war,
* or you getting fired because the alarm app failed.
* Please do some research if you have any concerns about this feature included
* in the product you find here before flashing it!
* YOU are choosing to make these modifications.
This should be safe to use when properly install.
Click to expand...
Click to collapse
For arm devices only !
Download backup link:
image2image_v1.1 -
https://www.androidfilehost.com/?fid=10763459528675564171
.

Reserved

@Ziona
Nice work!
If you're looking for some more up to date binaries like bash and brotli, you can grab them from here: https://github.com/Zackptg5/Cross-Compiled-Binaries-Android

Zackptg5 said:
@Ziona
Nice work!
If you're looking for some more up to date binaries like bash and brotli, you can grab them from here: https://github.com/Zackptg5/Cross-Compiled-Binaries-Androidy
Click to expand...
Click to collapse
Thanks
Cool no prob, will be sure to add what i can for next release. With credits to you again also.

file seems to be corrupt, tried downloading many times but it always stops at 4.5mb

dead0 said:
file seems to be corrupt, tried downloading many times but it always stops at 4.5mb
Click to expand...
Click to collapse
Thanks for pointing that out, xda server seems to be experiencing difficulties.
Theirs a new working link above.

Ziona said:
Thanks for pointing that out, xda server seems to be experiencing difficulties.
Theirs a new working link above.
Click to expand...
Click to collapse
thanks
been trying to unpack the vendor image from miui 12 for poco f2 global. unfortunately the furthest i get is the creation of vendor.img which i am unable to mount - seems to be an error on header magi which im not sure what it means

dead0 said:
thanks
been trying to unpack the vendor image from miui 12 for poco f2 global. unfortunately the furthest i get is the creation of vendor.img which i am unable to mount - seems to be an error on header magi which im not sure what it means
Click to expand...
Click to collapse
What is the default vendor image type before converting? .bin .img .dat or .dat.br . (sparse or raw format)

Ziona said:
What is the default vendor image type before converting? .bin .img .dat or .dat.br . (sparse or raw format)
Click to expand...
Click to collapse
its dat.br in raw format from here: http://airtel.bigota.d.miui.com/V12...IGlobal_V12.0.4.0.QJKMIXM_72b1da9230_10.0.zip

dead0 said:
its dat.br in raw format from here: http://airtel.bigota.d.miui.com/V12...IGlobal_V12.0.4.0.QJKMIXM_72b1da9230_10.0.zip
Click to expand...
Click to collapse
Generally you know it would go like this: vendor.img (raw) > vendor.img (sparse) > vendor.new.dat (data sparse format) > finally compressed again using brotli > vendor.new.dat.br (data sparse format) .
Things to consider
You get header errors also when unpacking images that are bad or when using the wrong decompression tool on it.
Is the .dat.br image for 32 or 64 bit device.
Using google drive, mega or android host create a link here to only the vendor.new.dat.br for me to try

Ziona said:
Generally you know it would go like this: vendor.img (raw) > vendor.img (sparse) > vendor.new.dat (data sparse format) > finally compressed again using brotli > vendor.new.dat.br (data sparse format) .
Things to consider
You get header errors also when unpacking images that are bad or when using the wrong decompression tool on it.
Is the .dat.br image for 32 or 64 bit device.
Using google drive, mega or android host create a link here to only the vendor.new.dat.br for me to try
Click to expand...
Click to collapse
i'll upload it after work. the dat.br is for 64bit. I can extract on the pc using CRB from xda chef central but cant compress it back again

dead0 said:
i'll upload it after work. the dat.br is for 64bit. I can extract on the pc using CRB from xda chef central but cant compress it back again
Click to expand...
Click to collapse
Ok send it then, btw did you try the options in the module on your device an followed the steps to unpack an repack dat.br.
Am not clear on if you did or not

Ziona said:
Ok send it then, btw did you try the options in the module on your device an followed the steps to unpack an repack dat.br.
Am not clear on if you did or not
Click to expand...
Click to collapse
i have tried the steps but only gone as far as creating the vendor.img, after that i get a mounting error

dead0 said:
i have tried the steps but only gone as far as creating the vendor.img, after that i get a mounting error
Click to expand...
Click to collapse
I'll have to check that vendor file, you have busybox installed in your system

Ziona said:
I'll have to check that vendor file, you have busybox installed in your system
Click to expand...
Click to collapse
yes busybox installed. heres the link for the vendor image: https://mega.nz/file/UdEHSCjB#XVgUjFMd2KEfoiaH53z_AyDIB_TGhYWRwyQYRap-2uI

dead0 said:
yes busybox installed. heres the link for the vendor image: https://mega.nz/file/UdEHSCjB#XVgUjFMd2KEfoiaH53z_AyDIB_TGhYWRwyQYRap-2uI
Click to expand...
Click to collapse
Ok
Is this the vendor.new.dat.br image unconverted? If so its really large for a compressed vendor image file

Ziona said:
Ok
Is this the vendor.new.dat.br image unconverted? If so its really large for a compressed vendor image file
Click to expand...
Click to collapse
yes. this is from stock

dead0 said:
yes. this is from stock
Click to expand...
Click to collapse
The vendor image is good .
Android 10
Redmi K30
Xiaomi
It unpacks fine.
It mounts fine.
It repacks fine.
So your sure it was mounted , prop files from the image !

Ziona said:
The vendor image is good .
Android 10
Redmi K30
Xiaomi
It unpacks fine.
It mounts fine.
It repacks fine.
So your sure it was mounted , prop files from the image !
Click to expand...
Click to collapse
Strange. I'll try it again 2moro but using stock ROM as base os. I was testing it with a custom ROM. Any chance u can upload the repacked image?

dead0 said:
Strange. I'll try it again 2moro but using stock ROM as base os. I was testing it with a custom ROM. Any chance u can upload the repacked image?
Click to expand...
Click to collapse
I'll upload it.
You may have to tell me step process your doing to unpack an repack so can point out anything wrong

Repack AP_---.tar.md5 file with all whatever original signatures

The device is a Sam Gal A11W.
I want to change one text file in the /system partition.
In order to do that, I want to start from the stock firmware I already downloaded, make the change then flash the whole modified firmware.
There are 5 .tar.md5 files (4 if we consider only one of CSC_ and HOME_CSC_) in this firmware, which can be flashed just fine in their original form.
What I want to do is to:
1 - unpack the AP_***.tar file (contains meta-data/fota.zip, boot.img.lz4, carrier.img.lz4, dtbo.img.lz4, metadata.img.lz4, recovery.img.lz4, super.img.lz4, userdata.img.lz4, vbmeta.img.lz4)
2 - unlz4 the super.img.lz4,
3 - unsparse the super.img to super.ext4.img (using simg2img)
4 - lpunpack the super.ext4.img into individual dynamic partitions (there are only 4: odm.img, product.img, system.img, vendor.img - this is NOT an A/B device !!)
5 - modify the text file on system.img
6 - lpmake the super.ext4.img back from odm.img, product.img, system.img, vendor.img
7 - re-sparse the super.ext4.img back into super.img (using img2simg)
8 - lz4 the super.img back into super.img.lz4
9 - Re-TAR the super.img.lz4 file plus all the rest of the original .img.lz4 files back into the AP_***.tar file
10 - Flash the AP_***.tar file to this mother****ing piece of Samsung crap without errors, and that is WITHOUT UNLOCKED BOOTLOADER, because the scumbags eliminated the OEM Unlock option from the Dev Options.
What I already know is:
- How to do the whole process without preserving any signatures.
This is already explained in this thread:
Editing system.img inside super.img and flashing our modifications
I'm trying to modify my system.img (/system/build.prop) to include support for multi users. After struggling a lot, I've succeeded following your guide (that's an awesome work btw) to unpack, mount, modify, umount and repack super.img. Then...
forum.xda-developers.com
but the author assumes the bootloader can be unlocked and therefore doesn't deal with any signatures / security-crypto-baloney checks.
What I (still) need to know is:
1 - How many crypto-security-baloney-whatever-signatures are there on all these files ?
2 - How can I restore them during repackaging so the flashing process does not get screwed up by the locked phone ?
If anyone here knows the answer to these 2 things ... well I guess they must be the Android Godfather Almighty !!

have you figured this out?

FugkGoocle said:
The device is a Sam Gal A11W.
I want to change one text file in the /system partition.
In order to do that, I want to start from the stock firmware I already downloaded, make the change then flash the whole modified firmware.
There are 5 .tar.md5 files (4 if we consider only one of CSC_ and HOME_CSC_) in this firmware, which can be flashed just fine in their original form.
What I want to do is to:
1 - unpack the AP_***.tar file (contains meta-data/fota.zip, boot.img.lz4, carrier.img.lz4, dtbo.img.lz4, metadata.img.lz4, recovery.img.lz4, super.img.lz4, userdata.img.lz4, vbmeta.img.lz4)
2 - unlz4 the super.img.lz4,
3 - unsparse the super.img to super.ext4.img (using simg2img)
4 - lpunpack the super.ext4.img into individual dynamic partitions (there are only 4: odm.img, product.img, system.img, vendor.img - this is NOT an A/B device !!)
5 - modify the text file on system.img
6 - lpmake the super.ext4.img back from odm.img, product.img, system.img, vendor.img
7 - re-sparse the super.ext4.img back into super.img (using img2simg)
8 - lz4 the super.img back into super.img.lz4
9 - Re-TAR the super.img.lz4 file plus all the rest of the original .img.lz4 files back into the AP_***.tar file
10 - Flash the AP_***.tar file to this mother****ing piece of Samsung crap without errors, and that is WITHOUT UNLOCKED BOOTLOADER, because the scumbags eliminated the OEM Unlock option from the Dev Options.
What I already know is:
- How to do the whole process without preserving any signatures.
This is already explained in this thread:
Editing system.img inside super.img and flashing our modifications
I'm trying to modify my system.img (/system/build.prop) to include support for multi users. After struggling a lot, I've succeeded following your guide (that's an awesome work btw) to unpack, mount, modify, umount and repack super.img. Then...
forum.xda-developers.com
but the author assumes the bootloader can be unlocked and therefore doesn't deal with any signatures / security-crypto-baloney checks.
What I (still) need to know is:
1 - How many crypto-security-baloney-whatever-signatures are there on all these files ?
2 - How can I restore them during repackaging so the flashing process does not get screwed up by the locked phone ?
If anyone here knows the answer to these 2 things ... well I guess they must be the Android Godfather Almighty !!
Click to expand...
Click to collapse
Try using modified/patched Odin. As for trying to bypass unlocking the bootloader to flash the modified firmware or trying to "mimic" or "fake" the original signature, that isn't going to work. Samsung's proprietary signature is an enigma that can't be cracked. Though you might be able to match the MD5 by adding a dummy file to the file you are modifying and filing it bit/byte by bit/byte, one step at a time, the goal is to add characters to the file until it is large enough to make your modified file match the original file's MD5. That is if your modified file is smaller than the original file, if your modified file is larger than the original file, you can delete unimportant files from the modified MD5 file until it is smaller than the original MD5 and then create the dummy file filled with dummy characters until it exactly matches the original MD5 bit for bit. Then try flashing your MD5 file once you gets it's MD5 code matching bit for bit. Try the patched version of Odin to flash your modified file. No guarantees that it will work but part of the security checks during flashing checks the MD5 for a match/mismatch.
Rab_DaJew said:
have you figured this out?
Click to expand...
Click to collapse

[REQUEST] Build system.img

Hello,
I have an ASUS K01A tablet with a blank system partition and unusable recovery. I can flash img files to their respective partitions just fine (boot.img, droidboot.img) in fastboot, but 'system' in the stock firmware zip is a directory instead of a flashable image. Of course, this (and other reasons) means I can't just flash the stock firmware in fastboot, nor flash over the entire 'system' directory.
I need a system.img to recover my tablet from softbrick, but I can't for the life of me compile one. I've exhausted every method and tool I could find in both Windows and Ubuntu.
I beseech the community; if there is anyone reading this who could build such a file for me, I would be eternally grateful. I believe I only need the system.img to recover the tablet.
The stock firmware is at the ASUS website. Here is the download link: UL-K01A-WW-12.8.1.41-user.zip.
Thanks in advance!

Why not extract the system folder of the ZIP linked to, then pack it into a ZIP what you then convert to IMG using an online-convertor that does the job?

jwoegerbauer said:
Why not extract the system folder of the ZIP linked to, then pack it into a ZIP what you then convert to IMG using an online-convertor that does the job?
Click to expand...
Click to collapse
"Why not" indeed; that sounds like it would more than suffice. Can you please recommend one?

Anyone? I have been unable to find this "online-converter" despite days of searching; still at square one.

Only as example ( found via Google search !!! )
Archive Converter - Convert your a7z, rar, zip, gzip, bzip2, tar archives to another format online
Free Archive Converter. Convert your archive files to a7z, rar, zip, gzip, bzip2, tar format online for free using OnlineConvertFree.
onlineconvertfree.com

jwoegerbauer said:
Only as example ( found via Google search !!! )
Archive Converter - Convert your a7z, rar, zip, gzip, bzip2, tar archives to another format online
Free Archive Converter. Convert your archive files to a7z, rar, zip, gzip, bzip2, tar format online for free using OnlineConvertFree.
onlineconvertfree.com
Click to expand...
Click to collapse
Thank you, but I've tried this one and all the others on Google, and none will handle the filesize (~650 MB). It must be purpose-made for Android .img files it seems, which I've been unable to find.

Kehkou said:
Hello,
I have an ASUS K01A tablet with a blank system partition and unusable recovery. I can flash img files to their respective partitions just fine (boot.img, droidboot.img) in fastboot, but 'system' in the stock firmware zip is a directory instead of a flashable image. Of course, this (and other reasons) means I can't just flash the stock firmware in fastboot, nor flash over the entire 'system' directory.
I need a system.img to recover my tablet from softbrick, but I can't for the life of me compile one. I've exhausted every method and tool I could find in both Windows and Ubuntu.
I beseech the community; if there is anyone reading this who could build such a file for me, I would be eternally grateful. I believe I only need the system.img to recover the tablet.
The stock firmware is at the ASUS website. Here is the download link: UL-K01A-WW-12.8.1.41-user.zip.
Thanks in advance!
Click to expand...
Click to collapse
maybe that video helps you...
good luck
solong
speedson

I built it finally using IMGBurn to create the raw img and Sparse Converter to convert it to sparse img fragments.
But now the tab has formatting error, and with -u option, flashing errors (all partitions and imgs now). Ah, well.
Thank you to everyone! Hopefully someone else reading this will have better luck than me.

Kehkou said:
I built it finally using IMGBurn to create the raw img and Sparse Converter to convert it to sparse img fragments.
But now the tab has formatting error, and with -u option, flashing errors (all partitions and imgs now). Ah, well.
Thank you to everyone! Hopefully someone else reading this will have better luck than me.
Click to expand...
Click to collapse
Download the attached file from the link at the bottom of this post and unzip it somewhere on your PC. It is a temporary recovery launcher. We'll come back to it in a moment. Based on your original post I'm assuming you already have your stock firmware downloaded as well as ADB and Fastboot setup. From your stock firmware unpack the following files: fastboot.img, splashscreen.img, and boot.img and place them in your ADB/Fastboot folder. Copy the stock firmware to your sd card (in its original, unzipped form...the way it was when you first downloaded it). Put the tablet in fastboot mode. Run the following fastboot commands from a command prompt.
- fastboot flash fastboot fastboot.img
- fastboot flash boot boot.img
- fastboot flash splashscreen splashscreen.img
Reboot back to fastboot mode. Go to wherever you unzipped the temp recovery that you downloaded earlier. Double click on launcher.bat. Type accept in the window, then type T2 and your device should boot into CWM recovery. Now choose install zip>install from se card>and choose the stock firmware.zip that you previously copied to your sd card then let it do its thing. Once its finished go back to the first page of the CWM recovery that you're in and select reboot system now>no>no. You should be golden now. Post back if you have any issues.
Download Disposable Recovery

Modifying system.img

Hello guys. I have a simple question. I am trying to modify ROM for xiaomi a2. I have official .iso images. For test i trying to remove "grep" binary file. I mount system.img, change it, unmount, pack and flash using fastboot. But the file is still is visble using adb.
When i try TWRP file manager, the file is absent. How is it possible?
Then i try to change one file directly in .iso using hex editor, but the changes is not visible. But when i install magisk, all changes became visible. Can android store copy of system.img somewhere else?

Not sure what you have done or what you mean with "official .iso images" but consider three possibilities:
Magisk is systemless-root method, it provides overlay for modifying /system content without modifying system partition
A/B devices have two slots, system_a and system_b partition
dm-verity provides FEC feature to correct modifications on file system