Database Info

[Q] Boot Process

I've looked in quite a few threads, including the custom mix ROMs. What I want to know is who has knowledge of how this thing boots? I've noticed on the 2 droids I've messed with you upgrade them with an update.zip and basically an empty file on the SD card.
I'm guessing the update.zip contains a squashfs or something similar along with a kernel, but I don't know for sure. Who can explain exactly what happens on boot, what devices it looks for filesystems/loop filesystems on etc?

muqali said:
I've looked in quite a few threads, including the custom mix ROMs. What I want to know is who has knowledge of how this thing boots? I've noticed on the 2 droids I've messed with you upgrade them with an update.zip and basically an empty file on the SD card.
I'm guessing the update.zip contains a squashfs or something similar along with a kernel, but I don't know for sure. Who can explain exactly what happens on boot, what devices it looks for filesystems/loop filesystems on etc?
Click to expand...
Click to collapse
The best way to learn about this stuff is by tearing down existing .zip flashers and reading about how boot.img files are built and trying to build your own.
The boot process is controlled by the bootloader (the thing that displays the Viewsonic birds logo and decides whether you are trying to boot into recovery, APX mode, or a standard boot).
The ZIP files that you flash actually run a script that writes an .IMG file contents to one or more partitions.
Most ROM flashers have two images they flash - a boot.img and a system.img. The system.img is a filesystem image of the system partition. The boot.img consists of two parts - a kernel and a ramdisk image (well, and a boot header). The kernel is the Linux kernel that's been compiled to provide all the core system functionality for Android. The ramdisk image (which is a GZIPed CPIO file) contains the init.rc file that does all the initialization, mounting of the main filesystem images and so on.
Once everything is mounted and configured, the Android ROM itself boots up.
I recommend reading this page for more info:
http://android-dls.com/wiki/index.php?title=HOWTO:_Unpack,_Edit,_and_Re-Pack_Boot_Images
Hope that helps!

That's more or less what I wanted, but for our device do you know the exact boot order? Mtd partitions then SD sd2 and maybe a USB disk? I'd really like to put a custom Roman on the sd2 and leave the rom factory.

muqali said:
That's more or less what I wanted, but for our device do you know the exact boot order? Mtd partitions then SD sd2 and maybe a USB disk? I'd really like to put a custom Roman on the sd2 and leave the rom factory.
Click to expand...
Click to collapse
I *think* (could be wrong) that both the externalSD (/sdcard2) and the USBDISK (/usbdisk) are mounted dynamically, via vold.
At least that's the way I understood it...
There's a thread here in this forum, about (dual) booting Ubuntu, that does something along the lines of what you're asking about...
Jim

rcgabriel said:
The best way to learn about this stuff is by tearing down existing .zip flashers and reading about how boot.img files are built and trying to build your own.
The boot process is controlled by the bootloader (the thing that displays the Viewsonic birds logo and decides whether you are trying to boot into recovery, APX mode, or a standard boot).
The ZIP files that you flash actually run a script that writes an .IMG file contents to one or more partitions.
Most ROM flashers have two images they flash - a boot.img and a system.img. The system.img is a filesystem image of the system partition. The boot.img consists of two parts - a kernel and a ramdisk image (well, and a boot header). The kernel is the Linux kernel that's been compiled to provide all the core system functionality for Android. The ramdisk image (which is a GZIPed CPIO file) contains the init.rc file that does all the initialization, mounting of the main filesystem images and so on.
Once everything is mounted and configured, the Android ROM itself boots up.
I recommend reading this page for more info:
http://android-dls.com/wiki/index.php?title=HOWTO:_Unpack,_Edit,_and_Re-Pack_Boot_Images
Hope that helps!
Click to expand...
Click to collapse
rcgabriel,
When you do an nvflash (ala bekit's), does the contents of bootloader.bin end up getting stored permanently on the Gtab's internal SDcard? And, whenever the Gtab is booted, the code in that bootloader is the first code that runs, deciding, as you said, whether or not to do a recovery or execute the boot.img?
Thanks,
Jim

Isn't the bootloader on one of the MTD partitions and not the sdcard? What I'm having the hardest time wrapping my head around is how this system boots in comparison to an x86 Debian 5.0 system.
The basic picture I have is that there is the rom/flash portion of the boot process and then the kernel and real filesystem are loaded and mounted. But using these custom roms, is it changing the flash and bootloader or just the kernel and OS? Meaning where exactly is the first thing the CPU hits to boot? Flash MTD partitions for the bootloader. Then is the kernel on the mtd partitions or on some separate partition on sdcard.
Since you can basically stick 2 files on an external uSD and send it into upgrade mode, can't you customize the filesystem contained in a zip and then stick it in and have a custom system with whatever you wanted added or removed?

[Q][RK2818|Archos 7HTv2] "No Space left on Device" during flash

A little background, I am trying to flash a Honeycomb rom to my Archos 7HTv2. This is a custom rom that I put together. I know that Honeycomb requires much more power than my Archos can give me. I know that it would probably be unusable, but I was curious, and it brought up a problem.
The system.img for the HC rom is ~112mbs, the total update.img is ~123mbs
The system.img for the stock rom is ~51mbs the total update.img is ~54mbs
I first tried to flash it and got a "not enough space" error when flashing the system. I assumed that the stock rom barely fit in the system partition, so I went looking for how to make it bigger.
Flashing on the Archos is by putting an update.img in the SDcard, or by using the "BatchUpgrade" utility from the computer, which copies it to some temp on the device. Then the recovery takes over and installs the rom.
Code:
Writing KERNEL:...
Checking KERNEL:...
Writing BOOT:...
Checking BOOT:...
Writing SYSTEM:...
E:Write failed(No space left on device)
E:Failure at line 16:
write_image PACKAGE:system SYSTEM:
Update failed, please reboot and update again!
Line 16 refers to the update-script file which has the list of Writing & Checking.
When I flash the rom from the computer using "BatchUpgrade" it fails before it can attempt unpacking. It just tells me "FAILURE" and no specifics as to why. Because the img never begins unpacking makes me believe it might be the temp area is not big enough for the .img and the extracted components.
I found this mtd information in the HWDEF file (part of update.img)
Code:
#Format: part_name offset:size
parameter 0x00000000:0x00002000:fh
misc 0x00002000:0x00002000:f
kernel 0x00004000:0x00004000:f
boot 0x00008000:0x00002000:f
recovery 0x0000A000:0x00004000:f
system 0x0000E000:0x00030000:f
backup 0x0003E000:0x0003A000:
cache 0x00078000:0x0003A000:
userdata 0x000B2000:0x00100000:
user 0x001B2000:0xFFFFFFFF:
END
I changed the system and backup partitions to be bigger (the above is stock) but nothing happened. I then found the PARAMETER file which had a similar string of mtd definitions, I updated them as above. Still no luck.
Anyone have any ideas?
Thanks

Do you had any luck?

[RECOVERY] [TWRP] Android system recovery <3e> universal checksum disabler

This is a Flashable Zip for restoring backups created from different recoveries. No more problems with checksums, checksums will be ignored.
- Android system recovery <3e>
- CWM
- TWRP
- ADB Backup
- MIUI Backup
This is a alpha release, in this state you can test it for Android system recovery <3e> only!
I need testers for proof of concept. If you have a Mediatek Phone with encrypted userdata_20160823_100259.backup, please try to restore with bckp2raw.zip from TWRP. Please give feedback if failed or vote :good: if worked.
bckp2raw.zip (v1.0.3) 32-bit
bckp2raw.zip (v1.0.5) universal
(other backup formats following soon, feel free to request)

TUTORIAL - under construction

aIecxs said:
TUTORIAL - under construction
Click to expand...
Click to collapse
Is this solves problem of Android 7 stock recovery backup?
I got second phone stock recovery problem for 7 version
stock recovery do not want to recover his backup. complains - "it is corrupted, please reset your phone"
I install twrp recovery and try to install this .zip
bit it give error 134 (or something like that)

sorry this project is abandoned. in generally, backup can not restored to other device because of the checksum mismatch. Thats where this zip comes in game. If you upload recovery.log right after installing this zip i will try to fix the errors..

aIecxs said:
sorry this project is abandoned. in generally, backup can not restored to other device because of the checksum mismatch. Thats where this zip comes in game. If you upload right after installing this zip i will try to fix the errors..
Click to expand...
Click to collapse
I meant - I got another (second) phone looped reboot, same as in 2018. And try to stock recovery backup (55Gbyte and ) restore.
BTW - first phone is still waiting to restore 11Gbyte stock recovery backup!
last file: userdata_20191106_110350.backup27
Android 7 again same as in topic: https://forum.xda-developers.com/showthread.php?t=2528969&page=7
I copied lines from recovery.log:
I: operation_start: 'Flashing'
Installing zip file '/external_sd/bckp2raw.zip'
Checking for Digest file...
Skipping Digest check: no Digest file found
I:Update binary zip
I:Zip does not contain SELinux file_contexts file in its root.
I:Legacy property environment initialized.
Archive: /external_sd/bckp2raw.zip
inflating: bckp2raw.sh
Archive: /external_sd/bckp2raw.zip
inflating: busybox-arm
Archive: /external_sd/bckp2raw.zip
inflating: tar-arm
CANNOT LINK EXECUTABLE "/system/bin/sh": "/system/lib/libc++.so" is 32-bit instead of 64-bit
Aborted
Updater process ended with ERROR: 134
I:Legacy property environment disabled.
I:Install took 0 second(s).
Error installing zip file '/external_sd/bckp2raw.zip'
Dont want to boring and get you, if you would give help - you can save many people with same problem with android-7

ah i see, script does not even start. i don't have a x64 device for testing, i assume twrp is not able to run mksh from /system. i have added paths please try this one instead...
update-binary
Code:
LD_LIBRARY_PATH=/system/lib64:/system/vendor/lib64:/system/lib:/system/vendor/lib
bckp2raw.zip (v1.0.5) universal

for those who want to quickly convert TWRP ADB backup.ab into data.ext4.win
usage for unpacking files (don't do this on windows disk)
Code:
bash twab.sh backup.ab
cd 20*
mkdir data
sudo tar --ignore-command-error --numeric-owner --selinux --xattrs -xpf data.ext4.win -ivC data 2> /dev/null

data partition is encrypted
Hi , I've been trying a lot of methods to get my data back from my .backup files. All with no results. Your bckp2raw script seems to be one of the most successfull , but it ends by a " data partition is encrypted" ... Do you think I can still recover my files ... The original phone is actually working but it has been factory wiped ...

that message just means that your current /data partition is encrypted (message for encrypted backup is different)
- do a backup of metadata partition (just in case)
- do a backup of /sdcard to PC (adb/mtp)
- format userdata partition
- reboot TWRP
- check userdata partition is unencrypted
- restore userdata*.backup from zip
- restore metadata partition from backup
- restore /sdcard backup
the restoring process will deny if userdata is not empty. please note if backup is encrypted it is most likely useless without corresponding backup of metadata partition

aIecxs said:
that message just means that your current /data partition is encrypted (message for encrypted backup is different)
- do a backup of metadata partition (just in case)
- do a backup of /sdcard to PC (adb/mtp)
- format userdata partition
- reboot TWRP
- check userdata partition is unencrypted
- restore userdata*.backup from zip
- restore metadata partition from backup
- restore /sdcard backup
the restoring process will deny if userdata is not empty. please note if backup is encrypted it is most likely useless without corresponding backup of metadata partition
Click to expand...
Click to collapse
Thank you very much for your quick reply. Ok so I tried it , the restoring part worked , but when I rebooted it asked me for my code and said "password is correct but data is corrupted" ....
Do you have any idea for me to get my data back .... at least my pictures ...?

sounds like the backup is encrypted, can you confirm? more important is this the source phone of this backup? or do you try to restore on other device?

aIecxs said:
sounds like the backup is encrypted, can you confirm? more important is this the source phone of this backup? or do you try to restore on other device?
Click to expand...
Click to collapse
I'm pretty sure it is encrypted. Yes the phone is the source phone of the backups

encryption should work in theory, but you need a copy of metadata partition from the day you created backup. you most likely don't have it, therefore your data is lost forever

aIecxs said:
encryption should work in theory, but you need a copy of metadata partition from the day you created backup. you most likely don't have it, therefore your data is lost forever
Click to expand...
Click to collapse
Yes I don't have the metadatas from that day, is there a chance that they have been backup with the android backup ?
Thanks for your help anyways, I'm sad

no, nothing you can do. even a old metadata from the first day when you buyed this device would probably do it. metadata partition is formatted on factory reset.
for some devices the crypto footer is not located at separate partition, but at the end of userdata partition instead. But this is not the case for your device, otherwise this zip would have restored. you can double check, search with hex editor in last -16384 bytes of last file for string aes-cbc-essiv:sha256 (or aes-xts)

aIecxs said:
no, nothing you can do. even a old metadata from the first day when you buyed this device would probably do it. metadata partition is formatted on factory reset.
for some devices the crypto footer is not located at separate partition, but at the end of userdata partition instead. But this is not the case for your device, otherwise this zip would have restored. you can double check, search with hex editor in last -16384 bytes of last file for string aes-cbc-essiv:sha256
Click to expand...
Click to collapse
Cannot find anything like this but i'm not used to HEX reading .... Is there a chance that the metadata is on the SD card i was using for (external) storage on the phone (not the ones for the backups) ?

Hello there and thanks for creating such a productive thread, just what I needed for my current situation. so am trying to retrieve .backup files on cxAir android 7.0 after root through twrp but apparently, when running the script from the zip file you've shared I get the Error: data partition not mounted.
I'm still new in the dev world but am totally excited about it and I may not understand a lot of things but am willing to work my ass off to figure out the most of what I can so please I'd really appreciate it if I could get some help on this one, thanks in advance

mount data partition and try again?

Repack AP_---.tar.md5 file with all whatever original signatures

The device is a Sam Gal A11W.
I want to change one text file in the /system partition.
In order to do that, I want to start from the stock firmware I already downloaded, make the change then flash the whole modified firmware.
There are 5 .tar.md5 files (4 if we consider only one of CSC_ and HOME_CSC_) in this firmware, which can be flashed just fine in their original form.
What I want to do is to:
1 - unpack the AP_***.tar file (contains meta-data/fota.zip, boot.img.lz4, carrier.img.lz4, dtbo.img.lz4, metadata.img.lz4, recovery.img.lz4, super.img.lz4, userdata.img.lz4, vbmeta.img.lz4)
2 - unlz4 the super.img.lz4,
3 - unsparse the super.img to super.ext4.img (using simg2img)
4 - lpunpack the super.ext4.img into individual dynamic partitions (there are only 4: odm.img, product.img, system.img, vendor.img - this is NOT an A/B device !!)
5 - modify the text file on system.img
6 - lpmake the super.ext4.img back from odm.img, product.img, system.img, vendor.img
7 - re-sparse the super.ext4.img back into super.img (using img2simg)
8 - lz4 the super.img back into super.img.lz4
9 - Re-TAR the super.img.lz4 file plus all the rest of the original .img.lz4 files back into the AP_***.tar file
10 - Flash the AP_***.tar file to this mother****ing piece of Samsung crap without errors, and that is WITHOUT UNLOCKED BOOTLOADER, because the scumbags eliminated the OEM Unlock option from the Dev Options.
What I already know is:
- How to do the whole process without preserving any signatures.
This is already explained in this thread:
Editing system.img inside super.img and flashing our modifications
I'm trying to modify my system.img (/system/build.prop) to include support for multi users. After struggling a lot, I've succeeded following your guide (that's an awesome work btw) to unpack, mount, modify, umount and repack super.img. Then...
forum.xda-developers.com
but the author assumes the bootloader can be unlocked and therefore doesn't deal with any signatures / security-crypto-baloney checks.
What I (still) need to know is:
1 - How many crypto-security-baloney-whatever-signatures are there on all these files ?
2 - How can I restore them during repackaging so the flashing process does not get screwed up by the locked phone ?
If anyone here knows the answer to these 2 things ... well I guess they must be the Android Godfather Almighty !!

have you figured this out?

FugkGoocle said:
The device is a Sam Gal A11W.
I want to change one text file in the /system partition.
In order to do that, I want to start from the stock firmware I already downloaded, make the change then flash the whole modified firmware.
There are 5 .tar.md5 files (4 if we consider only one of CSC_ and HOME_CSC_) in this firmware, which can be flashed just fine in their original form.
What I want to do is to:
1 - unpack the AP_***.tar file (contains meta-data/fota.zip, boot.img.lz4, carrier.img.lz4, dtbo.img.lz4, metadata.img.lz4, recovery.img.lz4, super.img.lz4, userdata.img.lz4, vbmeta.img.lz4)
2 - unlz4 the super.img.lz4,
3 - unsparse the super.img to super.ext4.img (using simg2img)
4 - lpunpack the super.ext4.img into individual dynamic partitions (there are only 4: odm.img, product.img, system.img, vendor.img - this is NOT an A/B device !!)
5 - modify the text file on system.img
6 - lpmake the super.ext4.img back from odm.img, product.img, system.img, vendor.img
7 - re-sparse the super.ext4.img back into super.img (using img2simg)
8 - lz4 the super.img back into super.img.lz4
9 - Re-TAR the super.img.lz4 file plus all the rest of the original .img.lz4 files back into the AP_***.tar file
10 - Flash the AP_***.tar file to this mother****ing piece of Samsung crap without errors, and that is WITHOUT UNLOCKED BOOTLOADER, because the scumbags eliminated the OEM Unlock option from the Dev Options.
What I already know is:
- How to do the whole process without preserving any signatures.
This is already explained in this thread:
Editing system.img inside super.img and flashing our modifications
I'm trying to modify my system.img (/system/build.prop) to include support for multi users. After struggling a lot, I've succeeded following your guide (that's an awesome work btw) to unpack, mount, modify, umount and repack super.img. Then...
forum.xda-developers.com
but the author assumes the bootloader can be unlocked and therefore doesn't deal with any signatures / security-crypto-baloney checks.
What I (still) need to know is:
1 - How many crypto-security-baloney-whatever-signatures are there on all these files ?
2 - How can I restore them during repackaging so the flashing process does not get screwed up by the locked phone ?
If anyone here knows the answer to these 2 things ... well I guess they must be the Android Godfather Almighty !!
Click to expand...
Click to collapse
Try using modified/patched Odin. As for trying to bypass unlocking the bootloader to flash the modified firmware or trying to "mimic" or "fake" the original signature, that isn't going to work. Samsung's proprietary signature is an enigma that can't be cracked. Though you might be able to match the MD5 by adding a dummy file to the file you are modifying and filing it bit/byte by bit/byte, one step at a time, the goal is to add characters to the file until it is large enough to make your modified file match the original file's MD5. That is if your modified file is smaller than the original file, if your modified file is larger than the original file, you can delete unimportant files from the modified MD5 file until it is smaller than the original MD5 and then create the dummy file filled with dummy characters until it exactly matches the original MD5 bit for bit. Then try flashing your MD5 file once you gets it's MD5 code matching bit for bit. Try the patched version of Odin to flash your modified file. No guarantees that it will work but part of the security checks during flashing checks the MD5 for a match/mismatch.
Rab_DaJew said:
have you figured this out?
Click to expand...
Click to collapse

Modifying system.img

Hello guys. I have a simple question. I am trying to modify ROM for xiaomi a2. I have official .iso images. For test i trying to remove "grep" binary file. I mount system.img, change it, unmount, pack and flash using fastboot. But the file is still is visble using adb.
When i try TWRP file manager, the file is absent. How is it possible?
Then i try to change one file directly in .iso using hex editor, but the changes is not visible. But when i install magisk, all changes became visible. Can android store copy of system.img somewhere else?

Not sure what you have done or what you mean with "official .iso images" but consider three possibilities:
Magisk is systemless-root method, it provides overlay for modifying /system content without modifying system partition
A/B devices have two slots, system_a and system_b partition
dm-verity provides FEC feature to correct modifications on file system