Related
Disclaimer:These tools and instructions can EASILY permanently brick your phone if you do not know what you're doing. This is for developers ONLY.
Background: Some of the images below have been successfully flashed to our EVO, some have not.
A PC36IMG.zip is the same as a rom.zip extracted from an RUU.exe.
Goal: Understand and confirm successful flash of the images from the files released/leaked by HTC to the partitions on our EVO.
Tools: As far as I know there are basically two tools we have access to flash individual partitions with: flash_image and fastboot.
Downloads:
flash_image.zip (extract from .zip please!)
eng-PC36IMG.zip (eng build rom.zip root part 2)
ud-PC36IMG.zip (userdebug rom.zip root part 1)
rom.zip from RUU .1
rom.zip from RUU .6
EVO Partition Structure:
the phone reports its partition information as:
cat /proc/mtd
dev: size erasesize name
mtd0: 00c00000 00020000 "wimax"
mtd1: 000a0000 00020000 "misc"
mtd2: 00500000 00020000 "recovery"
mtd3: 00280000 00020000 "boot"
mtd4: 15e00000 00020000 "system"
mtd5: 0a000000 00020000 "cache"
mtd6: 1aba0000 00020000 "userdata"
EVO Partition Size:
cat /proc/partitions
major minor #blocks name
7 0 2111 loop0
7 1 14585 loop1
31 0 12288 mtdblock0
31 1 640 mtdblock1
31 2 4608 mtdblock2
31 3 3072 mtdblock3
31 4 358400 mtdblock4
31 5 163840 mtdblock5
31 6 437888 mtdblock6
254 0 2110 dm-0
254 1 14584 dm-1
rom.zip File Structure:
All the files located within a PC36IMG.zip or RUU rom.zip - Same file structure, different names (PC36IMG.zip or rom.zip) depending on the tool being used to flash it.
Example - Eng Build PC36IMG.zip
ls -l
Code:
size image name confirmed partition
75 android-info.txt
2168832 boot.img (mtd3 - kernel and ramdisk)
524288 hboot_0.76.2000.img (SPL)
6892 microp_v04.img
655360 NV.img (PRI)
22282240radio_1.36.00.04.02.img (radio)
131072 rcdata.img
3561472 recovery.img (mtd2 - recovery)
786432 splash1.nb0 (initial boot graphic)
215548608system.img (mtd4 - /system)
73081 tp_atmel224_16ab.img [url=http://forum.xda-developers.com/showpost.php?p=6848555&postcount=2]Atmel maXTouch 224-channel touchscreen[/url] - [url=http://www.shipped-roms.com/shipped/Incredible/]HTC Incredible[/url]
71761 tp_atmelc03_16ac.img
24178176userdata.img (mtd6 - /data)
8781824 wimax_24722.img (mtd0 - wimax)
Confirmed Working with flash_image:
flash_image boot boot.img (custom/stock kernel boot.img flashed to mtd3)
flash_image recovery recovery.img (custom/stock recovery recovery.img flashed to mtd2)
Confirmed Working with fastboot:
fastboot flash recovery recovery-RA-evo-v1.7.0.1.img
fastboot flash zimage zImage
fastboot boot boot.img
fastboot flash splash1 splash1.nb0
UPDATED FASTBOOT INFO - from toastcfh
By going into supercid mode on fastboot, we are able to flash our own PC36IMG.zip files w/o having HTC signed keys through the bootloader.
Entering supercid mode is done through two commands:
fastboot oem h
fastboot oem changeCid 00000000
Code:
INFO[hboot]:(RAW) block start=497, size=6 (768 KB)
INFO[misc3]:(RAW) block start=503, size=7 (896 KB)
INFO[mfg]:(RAW) block start=510, size=6 (768 KB)
INFO[sp1]:(RAW) block start=516, size=9 (1152 KB)
INFO[wifi]:(RAW) block start=525, size=5 (640 KB)
INFO[recovery]:(RAW) block start=530, size=40 (5120 KB)
INFO[boot]:(RAW) block start=570, size=20 (2560 KB)
INFO[system]:(YAFFS) block start=590, size=2800 (369600 KB)
INFO[cache]:(YAFFS) block start=3390, size=1280 (168960 KB)
INFO[userdata]:(YAFFS) block start=4670, size=3421 (451572 KB)
INFO[wimax]:(RAW) block start=8091, size=96 (12288 KB)
INFO[misc]:(RAW) block start=8187, size=5 (640 KB)
INFO[rcdata]:(OTHER) block start=0, size=0 (0 KB)
INFO[cpld]:(OTHER) block start=0, size=0 (0 KB)
INFO[microp]:(OTHER) block start=0, size=0 (0 KB)
INFO[a1026]:(OTHER) block start=0, size=0 (0 KB)
INFO[tp]:(OTHER) block start=0, size=0 (0 KB)
INFO[nv]:(OTHER) block start=0, size=0 (0 KB)
OKAY [ 0.089s]
Directions:
Please post your successful and unsuccessful experiences here! I will keep this thread updated until we've been able to flash everything possible to every possible partition!
Toast recommends having the phone in recovery mode as the safest state to flash from.
Use either flash_image or fastboot to flash any of the image files located within any of the PC36IMG/rom.zip files.
Please post back
Name of PC36IMG.zip/rom.zip package:
Name of .img file flashed:
Method used to flash: fastboot or flash_image
Command used to flash:
Results: Successful/Unsuccessful (errors)
so tp_atmel224_16ab.img is for the Atmel maXTouch 224-channel touchscreen sensor (you can see it on the fixit teardown)
but what is the tp_atmelc03_16ac.img for? Closest thing I could find was AT89C51CC03 8-bit microcontroller with CAN controller and 64-Kbyte Flash. 2-Kbyte RAM, 2-Kbyte EEPROM, SPI. Power Fail Detect (no need for external brown- out). Pinout compatible with T89C51CC01.
Vinny75 said:
so tp_atmel224_16ab.img is for the Atmel maXTouch 224-channel touchscreen sensor (you can see it on the fixit teardown)
but what is the tp_atmelc03_16ac.img for? Closest thing I could find was AT89C51CC03 8-bit microcontroller with CAN controller and 64-Kbyte Flash. 2-Kbyte RAM, 2-Kbyte EEPROM, SPI. Power Fail Detect (no need for external brown- out). Pinout compatible with T89C51CC01.
Click to expand...
Click to collapse
perfect! updated the main post with the info on atmel224_16ab ! thanks!
joeykrim said:
perfect! updated the main post with the info on atmel224_16ab ! thanks!
Click to expand...
Click to collapse
I wonder why these are separate imgs.... is that common on other android devices? just thought the driver would be in the boot or system img
Vinny75 said:
I wonder why these are separate imgs.... is that common on other android devices? just thought the driver would be in the boot or system img
Click to expand...
Click to collapse
driver probably is in the kernel like most linux systems, but my guess is maybe this is the firmware?
joeykrim said:
driver probably is in the kernel like most linux systems, but my guess is maybe this is the firmware?
Click to expand...
Click to collapse
Hurm...makes sense... does the Incredible suffer from the same 30fps cap? It uses the MTX224 as well... would be interesting to see the file is in an Incredible rom...
Check out the incredible ruu on shipped roms or I'm sure its posted in the incredible forum
Wait when we flashed both of the pci.zips didn't it skip those both times I'm pretty sure it did if I'm not mistaken
bcnice20 said:
Check out the incredible ruu on shipped roms or I'm sure its posted in the incredible forum
Wait when we flashed both of the pci.zips didn't it skip those both times I'm pretty sure it did if I'm not mistaken
Click to expand...
Click to collapse
Yeah at work now... so can't check yet. Could have skipped because it was the same version already?
bcnice20 said:
Check out the incredible ruu on shipped roms or I'm sure its posted in the incredible forum
Wait when we flashed both of the pci.zips didn't it skip those both times I'm pretty sure it did if I'm not mistaken
Click to expand...
Click to collapse
i marked my notes at home but i remember it skipped one and flashed the other, but dont remember which one is which. ill update my post when i get access to my notes or anybody who is goin thru the root process can observe.
I already got it extracted out of the ruu on my computer ill check it out soon as I get home
tp_atmel224_16ab.img IS in the incredible rom but
tp_atmelc03_16ac.img ISN'T
npace said:
tp_atmel224_16ab.img IS in the incredible rom but
tp_atmelc03_16ac.img ISN'T
Click to expand...
Click to collapse
i'll update the main post. do you have a link for reference?
joeykrim said:
i'll update the main post. do you have a link for reference?
Click to expand...
Click to collapse
I got the RUU from here http://www.shipped-roms.com/shipped/Incredible/
then got the rom.zip and looked in the archive.
It's not for the EVO so I dont think you should be putting it on the main post. Don't want anyone flashing the Incredible .img's
npace said:
I got the RUU from here http://www.shipped-roms.com/shipped/Incredible/
then got the rom.zip and looked in the archive.
It's not for the EVO so I dont think you should be putting it on the main post. Don't want anyone flashing the Incredible .img's
Click to expand...
Click to collapse
thanks for the link and research!
link is labeled HTC Incredible. like to cite sources for future reference.
if anybody unknowingly flashes HTC Incredible ROMs to their EVO they prob have more serious issues than a bricked EVO ...
npace said:
tp_atmel224_16ab.img IS in the incredible rom but
tp_atmelc03_16ac.img ISN'T
Click to expand...
Click to collapse
Did you do a bin compare on the two to see if they are the same?
Vinny75 said:
Did you do a bin compare on the two to see if they are the same?
Click to expand...
Click to collapse
As a matter of fact, they are!
EDIT: just to be clear:
EVO's tp_atmel224_16ab.img is identical to Incredible's tp_atmel224_16ab.img
I have successfully flashed splash1 via fastboot.
Is it possible to do it via flash tool from with in the shell in recovery?
npace said:
As a matter of fact, they are!
EDIT: just to be clear:
EVO's tp_atmel224_16ab.img is identical to Incredible's tp_atmel224_16ab.img
Click to expand...
Click to collapse
tp_atmel224_ab.img is only in "RUU_IncredibleC_Verizon_WWE_1.13.605.1_Radio_1.00.00.02.09_release_149249_hboot_076.0000_NoDriver.exe" and not "RUU_IncredibleC_Verizon_WWE_1.22.605.2_Radio_1.00.03.04.06_hboot_0.79_release_161494_NoDriver_0514.exe"
Nice work. I'm working on kernel source atm but will try to flash the wimax image here in bit. Will report back
Xyber3364 said:
I have successfully flashed splash1 via fastboot.
Is it possible to do it via flash tool from with in the shell in recovery?
Click to expand...
Click to collapse
can you paste the full command you used? if you have a link to a thread about it too, could you paste the link and i'll link back to it?
Most of the ROMs with appropriate kernels come with ext2 for /system
and then convert the filesystems with the .convert file to data-ext4 and cache-ext2
so while i was messing around i tried this:
Required:
Kernel supporting filesystems,
factoryfs.rfs is already ext2 and has cwm recovery in it
stl7 and stl8 converted to desired filesystems
use a freshly flashed rom without any private data on /cache and /data for "dd" if you packaging it for people to download (just in case, it wont matter because it will get formatted on first boot itself)
create cache.rfs (12,45,184 bytes) and datafs.rfs (14,82,752 bytes) using terminal editor/adb
For datafs.rfs
Code:
# dd if=/dev/block/stl7 of=/sdcard/datafs.rfs bs=4096 count=362
it will create a datafs.rfs file in your sdcard which is (4096 x 362 = 14,82,752 bytes)
For cache.rfs
Code:
# dd if=/dev/block/stl8 of=/sdcard/cache.rfs bs=4096 count=304
it will create a cache.rfs file in your sdcard which is (4096 x 304 = 12,45,184 bytes)
now i packed it in a tar with a factoryfs.rfs (ext2) with recovery in it and flashed the rom
It showed CWM recovery on first boot,
Here i did a factory reset which formatted data cache and sd-ext into clean filesystems, then i fixed permissions superstitiously
This way ROM gets flashed and at the first boot itself the fs are converted to desired fs.
I don't know the quality of what i did or whether it has been done before or is it even correct..
but it worked
If you like it, thank me
if you don't, Ignore
Attached: my data (ext4) & cache (ext2) 2011-07-24
revant said:
Most of the ROMs with appropriate kernels come with ext2 for /system
and then convert the filesystems with the .convert file to data-ext4 and cache-ext2
so while i was messing around i tried this:
Required:
Kernel supporting filesystems,
factoryfs.rfs is already ext2 and has cwm recovery in it
stl7 and stl8 converted to desired filesystems
create cache.rfs (12,45,184 bytes) and datafs.rfs (14,82,752 bytes) using terminal editor/adb
For datafs.rfs
Code:
# dd if=/dev/block/stl7 of=/sdcard/datafs.rfs bs=4096 count=362
it will create a datafs.rfs file in your sdcard which is (4096 x 362 = 14,82,752 bytes)
For cache.rfs
Code:
# dd if=/dev/block/stl8 of=/sdcard/cache.rfs bs=4096 count=304
it will create a cache.rfs file in your sdcard which is (4096 x 304 = 12,45,184 bytes)
now i packed it in a tar with a factoryfs.rfs (ext2) with recovery in it and flashed the rom
It showed CWM recovery on first boot,
Here i did a factory reset which formatted data cache and sd-ext into clean filesystems, then i fixed permissions superstitiously
This way ROM gets flashed and at the first boot itself the fs are converted to desired fs.
I don't know the quality of what i did or whether it has been done before or is it even correct..
but it worked
If you like it, thank me
if you don't, Ignore
Click to expand...
Click to collapse
Thanks a lot, i was searching the same thing for appearing cwm on first boot after flashing rom. when i flash new rom i will check it and i m sure that it will work
I dont know why the CWM is seen on first boot.
May be because of error in filesystem
the output Cache.rfs and datafs.rfs are just initial bytes of the partition, if you get whole partition cache will be 6.0+ mB and datafs around 270 mB
But with this it is somehow managing to say to cwm that the fs is bad and CWM stops for us.. CWM seems to think it is certain type of fs and formats it to same type.
If you dont format (factory reset) the phone wont boot and the logcat will show error that it cannot create anything on /data
I tried "mkdir 123" in /data with adb and it gives IO error
but after format everything works fine.
and if you accidentally reboot and miss the CWM first time and the phone doesnt boot.. Just manually go to recovery and do factory reset
I used the following command to create cache.rfs as my cache.rfs is 13 MB
Code:
dd if=/dev/block/stl8 of=/sdcard/cache.rfs bs=4096 count=3344
Re-packaging OTA
It's a good topic and helps very peoples like me.
Thanks for sharing
I spent some time in analyzing of flash layout. The comprehensive description below attempts to map each byte of the flash and describes way to extract it.
I would be glad if somebody could provide more detailed info about bootloader, signatures, DRM etc.
Patches are welcome.
Code:
mmcblk0 layout
All dumps were done on Asus Eee Pad Transformer Infinity TF700T, 64GB version, firmware 9.4.5.26, locked
mmcblk0 off-partition section
Offset: 0 (0x0)
Size: 38273024 (0x2480000)
Read command: busybox dd if=/dev/block/mmcblk0 of=/mnt/sdcard/mmcblk0pre1.img bs=524288 count=73
Offset: 0 (0x0)
Size: 3670016 (0x380000)
Contains: Zeroes
Purpose: Unknown
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s1.img bs=3670016 count=1
Process command: tr -d '\0' <mmcblk0pre1s1.img >mmcblk0pre1s1nz.img # mmcblk0pre1s1nz.img must be empty file
Offset: 3670016 (0x380000)
Contains: Recovery kernel image followed by zeroes
Size: 8388608 (0x800000)
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s2.img bs=524288 skip=7 count=16
Process commands:
perl split_bootimg.pl mmcblk0pre1s2.img
mkdir mmcblk0pre1s2.img-ramdisk
cd mmcblk0pre1s2.img-ramdisk
zcat ../mmcblk0pre1s2.img-ramdisk.gz | cpio -i
cd ..
# end Process commands
Offset: 12058624 (0xb80000)
Contains: Regular boot kernel image followed by zeroes
Size: 8388608 (0x800000)
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s3.img bs=524288 skip=23 count=16
Process commands:
perl split_bootimg.pl mmcblk0pre1s3.img
mkdir mmcblk0pre1s3.img-ramdisk
cd mmcblk0pre1s3.img-ramdisk
zcat ../mmcblk0pre1s3.img-ramdisk.gz | cpio -i
cd ..
# end Process commands
Offset: 20447232 (0x1380000)
Contains: Block of 16 bytes followed by 0x2de0 hexadecimal numbers followed by FF
Size: 12288 (0x3000)
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s4.img bs=524288 skip=39
Vital data:
Extract command: dd if=mmcblk0pre1s4.img of=mmcblk0pre1s4ss2.img bs=4096 skip=3
Binary part of vital data:
Extract command: dd if=mmcblk0pre1s4ss1.img of=mmcblk0pre1s4ss1ch1.img bs=16 count=1
Hexadecimal part of vital data:
Extract command: dd if=mmcblk0pre1s4ss1.img of=mmcblk0pre1s4ss1ch2.img bs=16 count=734 skip=1
Process command: unhex <mmcblk0pre1s4ss1ch2.img >mmcblk0pre1s4ss1ch2bin.img
FF part of vital data:
Extract command: dd if=mmcblk0pre1s4ss1.img of=mmcblk0pre1s4ss1ch3.img bs=16 skip=735
Process command: tr -d '\377' <mmcblk0pre1s4ss1ch3.img >mmcblk0pre1s4ss1ch3nff.img # mmcblk0pre1s4ss1ch3nff.img must be empty file
Zeroes:
Extract command: dd if=mmcblk0pre1s4.img of=mmcblk0pre1s4ss1.img bs=4096 count=3
Process command: tr -d '\0' <mmcblk0pre1s4ss2.img >mmcblk0pre1s4ss2nz.img # mmcblk0pre1s4ss2nz.img must be empty file
Purpose: Probably encrypted bootloader
mmcblk0p1
Offset: 38273024 (0x2480000)
Size: 805306368 (0x30000000)
File system size: 196608 * 4096 = 805306368 (fully occupies partition)
Format: Linux ext4 filesystem
Mounted at: /system
Mount options: read only, extended attributes, ACL
Permissions: only root can manipulate
Contains: Base system and embedded applications
Purpose: Base system
mmcblk0p2
Offset: 843579392 (0x32480000)
Size: 448790528 (0x1ac00000)
File system size: 109568 * 4096 = 448790528 (fully occupies partition)
Format: Linux ext4 filesystem
Mounted at: /cache
Mount options: read/write, no SUID, no device nodes, no atime
Permissions: only root can manipulate, UID system and GID cache can read and write
Contains: Cache
Purpose: Application cache
Note: The volume has the same UUID as mmcblk0p1
mmcblk0p3
Offset: 1292369920 (0x4d080000)
Size: 2097152 (0x200000)
File system size: 512 * 4096 = 2097152 (fully occupies partition)
Linux rev 1.0 ext3 filesystem
Not mounted
Permissions: GID system can manipulate
Contains: Empty file system
Purpose: Recovery /misc
Referenced by: /system/lib/libandroid_runtime.so recovery ramdisk: /etc/recovery.fstab
Note: File system is referenced in recovery as emmc, not ext3!
mmcblk0p4
Offset: 1294467072 (0x4d280000)
Size: 855638016 (0x33000000)
File system size: 208896 * 4096 = 855638016
Linux rev 1.0 ext3 filesystem
Not mounted
Permissions: GID system can manipulate
Contains: Empty file system
Purpose: Recovery /staging
Referenced by: recovery ramdisk: init.rc /etc/recovery.fstab
mmcblk0p5
Offset: 2150105088 (0x80280000)
Size: 5242880 (0x500000)
File system size: 5092 * 1024 = 5147488
Format: FAT32 file system, no partition table, MS-DOS "Non-system disk" boot block
Not mounted
Permissions: only root can manipulate
Contains: File system with files:
Serial numbers (ISN, PPID, SSN, UUID)
Calibration data (AL3010 light sensor, AMI304 magnetic sensor, KXTF9 motion sensor)
Purpose: Device specific unique system data, mounted as /btmac during Android boot
Referenced by: /system/bin/wifimacwriter /system/bin/brcm_patchram_plus /system/bin/sensors-config /system/bin/sixpair ramdisk: /init recovery ramdisk: /etc/recovery.fstab /init
mmcblk0p5 off file-system area
Offset in section: 5147488 (0x4e8b60)
Size: 28672 (0x7000)
Read command: busybox dd if=/dev/block/mmcblk0p5 of=/mnt/sdcard/mmcblk0p5s2.img bs=1024 skip=5092
Process command: tr -d '\0' <mmcblk0p5s2.img >mmcblk0p5s2nz.img # mmcblk0p5s2nz.img must be empty file
mmcblk0p6
Offset: 2155347968 (0x80780000)
Size: 524288 (0x80000)
Format: binary data
Permissions: UID drm can manipulate
Contains: 208 bytes of binary data, the rest are zeroes
Purpose: DRM, probably contains encrypted DRM key
Referenced by: /system/bin/wvdrmserver /system/vendor/lib/drm/libdrmwvmplugin.so
mmcblk0p7
Offset: 2155872256 (0x80800000)
Size: 5242880 (0x500000)
Format: empty
Contains: Zeroes
Purpose: Unknown
mmcblk0p8
Offset: 2161115136 (0x80d00000)
Size: 61415620608 (0xe4ca80000)
File system size: 14994040 * 4096 = 61415587840
Format: Linux ext4 filesystem
Mounted at: /data
Mount options: read/write, no SUID, no device nodes, no atime
Permissions: only root can manipulate, read and write are directory specific
Contains: User applications, user data, and virtual internal SD card
Note: /data/media is mounted via UID/GID stripping FUSE as /mnt/sdcard
mmcblk0p8 off file-system area
Offset in section: 61415587840 (0xe4ca78000)
Size: 32768 (0x8000)
Read command: busybox dd if=/dev/block/mmcblk0p8 of=/mnt/sdcard/mmcblk0p8s2.img bs=4096 skip=14994040
mmcblk0 off-partition section
Offset: 63576735744 (0xecd780000)
Size: 524288 (0x80000)
Read command: busybox dd if=/dev/block/mmcblk0 of=/mnt/sdcard/mmcblk0post8.img bs=524288 skip=121263
Process command: tr -d '\0' <mmcblk0p8s2.img >mmcblk0p8s2nz.img # mmcblk0p8s2nz.img must be empty file
Offset: 63576735744 (0xecd780000)
Offset in section: 0 (0x0)
Size: 507392 (0x7be00)
Contains: Zeroes
Purpose: Unknown
Extract command: dd if=mmcblk0post8.img of=mmcblk0post8s1.img bs=507392 count=1
Process command: tr -d '\0' <mmcblk0post8s1.img >mmcblk0post8s1nz.img # mmcblk0post8s1nz.img must be empty file
Offset: 63577243136 (0xecd7fbe00)
Offset in section: 507392 (0x7be00)
Size: 16896 (0x4200)
Contains: EFI Partition table (partition names: APP, CAC, MSC, USP, PER, YTU, CRA, UDA)
Extract command: dd if=mmcblk0post8.img of=mmcblk0post8s2.img bs=512 skip=991
Purpose: Partition table
Total size of mmcblk0: 63577260032 (0xecd800000)
Notes:
can manipulate = can read, write partition vital data, only root can mount
can read, write = can read, write partition file system contents
Read commands are ran on the Transformer
Extract and process commands are run anywhere, with pre-read image file in the current directory.
You need dd with large files support. Vanilla dd on TF700T does not support large files. Busybox dd does.
Dropbox link to Asus_Transformer_Infinity_TF700T/flash_layout.txt
Wow, thanks for this detailed analysis - much more detailed than mine.
So what can I add to your research?
Tegra-based systems have another partition table, which has a proprietary layout and an unknown purpose (maybe just important for NVFlash and for flashing blobs?). Looking at the flash.cfg in the NVFlash package from AndroidRoot.mobi, we can get the Tegra partition layout and partition names:
Partition number 1 is missing in the list, maybe it contains the extremely well-hidden APX mode recovery code or even the answer to life, the universe and everything.
The following 3 partitions are located at the beginning of mmcblk0 and their contents are apparently encrypted with a device-specific key. For some reason, with ICS-based ROMs it reads as all zeros; in JB-based ROMs additional mmcblk0boot0 and mmcblk0boot1 partitions appear which together cover this area. The "bricksafe.img" in the nvflash guide covers these 3 partitions.
2 BCT: Tegra Boot Configuration Table - 3145728 bytes
3 PT: Tegra Partition Table - 524288 bytes
4 EBT: Bootloader - 8388608 bytes
You already know the following 2:
5 SOS: Recovery kernel - 8388608 bytes
6 LNX: Linux kernel - 8388608 bytes
Then some more funny ones:
7 CER: I think this stands for "Certificate" and contains the bootloader unlock token. - 8388608 bytes. If I calculated correctly, this is at 0x1380000 into mmcblk0. Saved as "unlock-token.img" in the nvflash guide.
8 IMG: no idea what this is for - 8388608 bytes
9 GP1: space for a GPT partition table, maybe unused - 1048576 bytes
Now the regular partitions follow (p1 to p8):
10 APP: p1 = /system (Android OS)
11 CAC: p2 = /cache (for communication between Android and recovery)
12 MSC: p3 ="misc", whatever that is. On the TF101 it was used for bootloader commands.
13 USP: p4 = The update staging partition. Update blobs are copied here and flashed to the correct partition by the bootloader.
14 PER: p5 = device-specific config in a FAT filesystem
15 YTU: p6 = Apparently the DRM key. Confirmed to be overwritten with 0 by the unlocking process.
16 CRA: p7 = unknown (reserved for crash dumps?)
17 UDA: p8 = /data (Android user data)
And finally:
18 GPT: the EFI partition table that is actually used by the kernel
Well, it seems, that something (ICS stock kernel, hardware) hides contents of the first (at most) 0x380000 bytes of flash.
I am locked, and I have some token at 0x1380000 as well.
I am still thinking about a way to unlock, keep access to nvflash, and upgrade to JB keeping DRM working, even at cost of using stock system. That is why I wanted to backup and analyze everything and find all keys and signatures.
It would be also nice to know, whether there are areas of flash with hardware or kernel write lock.
utx said:
Well, it seems, that something (ICS stock kernel, hardware) hides contents of the first (at most) 0x380000 bytes of flash.
I am locked, and I have some token at 0x1380000 as well.
Click to expand...
Click to collapse
Yes, before unlocking I had something very similar to you there - a 16 byte header followed by some hexdump. I don't know what it was. It was overwritten by the unlock process with a 4 byte data block prefixed with a "-SIGNED-BY-SIGNBLOB-" header and followed by 256 bytes of what looks like a digital signature, very similar to the signed update blobs.
utx said:
I am still thinking about a way to unlock, keep access to nvflash, and upgrade to JB keeping DRM working, even at cost of using stock system. That is why I wanted to backup and analyze everything and find all keys and signatures.
Click to expand...
Click to collapse
Definitely back up the YTU partition before unlocking (p6) and then make the nvflash backups - but maybe the key must match something that is broken by the unlocking process, or it is renewed periodically, etc., so it might not help. Maybe try using DRM before unlocking and watch if the content of the partition changes over time.
utx said:
It would be also nice to know, whether there are areas of flash with hardware or kernel write lock.
Click to expand...
Click to collapse
Never tried to write directly to the block device - too scared to break something.
---------- Post added at 09:32 PM ---------- Previous post was at 09:28 PM ----------
Another small addition:
Note: /data/media is mounted via UID/GID stripping FUSE as /mnt/sdcard
Click to expand...
Click to collapse
This FUSE trick also makes /mnt/sdcard case-insensitive.
I just thought of something. What if you launched a data recovery process and recovered the DRM keys for the device?
ostar2 said:
I just thought of something. What if you launched a data recovery process and recovered the DRM keys for the device?
Click to expand...
Click to collapse
How do you define "data recovery process"? You cannot recover data that has been overwritten.
_that said:
How do you define "data recovery process"? You cannot recover data that has been overwritten.
Click to expand...
Click to collapse
Well, if the DRM partition is write enabled, it may be possible to restore its contents, if you backed it up before unlock (it is probably per-device unique). But it could be insufficient. Locked bootloader can be different than unlocked bootloader, and may drop cipher needed for DRM decihering. It is just a theory. Somebody could proof it or falsify, if:
1) Backed all accessible data before unlock.
2) Unlocked (and to be safe, also made brickproof image).
3) Recovered the data creates in step 1.
Will DRM work then? Or did we need the contents of (currently inaccessible) locked stock data of the first megabytes?
But I see no way, how to back-up first megabytes of locked device (on ICS; JB is not as interesting for us, once you upgrade to JB, you cannot create brickproof image for nvflash).
I even don't know, which part of the subsystem causes these megabytes being reported as zeroes. Is it stock Asus ICS kernel? Is it bootloader? Is it a hardware lock on the flash device?
Good idea, but what I meant by "Data Recovey". Is restoring the deleted data from that filesystem/partition.
ostar2 said:
Good idea, but what I meant by "Data Recovey". Is restoring the deleted data from that filesystem/partition.
Click to expand...
Click to collapse
I see, so I assume you assume you had a backup before.
Somebody (maybe you?) could try roughly the following sequence:
- get new TF700
- update to 9.4.5.26. if it's already newer, forget nvflash, but the rest could still work.
- root it using debugfs
- make a backup of /dev/block/mmcblk0p6
- do some DRM-dependent stuff and check that it works
- after some days, make another backup of /dev/block/mmcblk0p6 and compare if anything has changed. If the key is static, maybe restoring after unlocking could work. If not, chances are high that it doesn't work.
- unlock (this erases mmcblk06 and voids warranty)
- optional, but very useful: install AndroidRoot hacked bootloader to make blobs for nvflash, then use nvflash to backup all partitions
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works
_that said:
I see, so I assume you assume you had a backup before.
Somebody (maybe you?) could try roughly the following sequence:
- get new TF700
- update to 9.4.5.26. if it's already newer, forget nvflash, but the rest could still work.
- root it using debugfs
- make a backup of /dev/block/mmcblk0p6
- optional, but very useful: install AndroidRoot hacked bootloader to make blobs for nvflash, then use nvflash to backup all partitions
- do some DRM-dependent stuff and check that it works
- after some days, make another backup of /dev/block/mmcblk0p6 and compare if anything has changed. If the key is static, maybe restoring after unlocking could work. If not, chances are high that it doesn't work.
- unlock (this erases mmcblk06 and voids warranty)
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works
Click to expand...
Click to collapse
To install AndroidRoot bootloader and by that getting nvflash blobs, you have to unlock first... The order of your steps is therefore wrong.
firetech said:
To install AndroidRoot bootloader and by that getting nvflash blobs, you have to unlock first... The order of your steps is therefore wrong.
Click to expand...
Click to collapse
Oops, thanks for noticing. I edited my post.
what if we were to read from the NAND externally (RAW)....xbox 360 style...wouldn't that be the same as nvflash....
except that the three partitions in question are encrypted with a key that is probably unique per Tegra...
2 BCT: Tegra Boot Configuration Table - 3145728 bytes
3 PT: Tegra Partition Table - 524288 bytes
4 EBT: Bootloader - 8388608 bytes
but I would suppose it wouldn't be a problem since a raw flash would restore everything back to normal...even if we can't read it..the CPU can..and that's all that matters.
---------- Post added at 11:21 AM ---------- Previous post was at 11:13 AM ----------
never mind...its a BGA
_that said:
I see, so I assume you assume you had a backup before.
Somebody (maybe you?) could try roughly the following sequence:
- get new TF700
- update to 9.4.5.26. if it's already newer, forget nvflash, but the rest could still work.
- root it using debugfs
- make a backup of /dev/block/mmcblk0p6
- do some DRM-dependent stuff and check that it works
- after some days, make another backup of /dev/block/mmcblk0p6 and compare if anything has changed. If the key is static, maybe restoring after unlocking could work. If not, chances are high that it doesn't work.
- unlock (this erases mmcblk06 and voids warranty)
- optional, but very useful: install AndroidRoot hacked bootloader to make blobs for nvflash, then use nvflash to backup all partitions
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works
Click to expand...
Click to collapse
Correct order maybe.
- get new TF700
- update to 9.4.5.26.
- root it using debugfs
- make a backup of /dev/block/*.*
- unlock (this erases mmcblk06 and voids warranty)
- install AndroidRoot hacked bootloader to make blobs for nvflash
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works
Q1:If i backed up 9.4.5.26 all block image.After i updated 9.4.5.30 can i get the nvflash blob from backed up images?No way to dig out the blob key from the backup?
W3ber said:
Q1:If i backed up 9.4.5.26 all block image.After i updated 9.4.5.30 can i get the nvflash blob from backed up images?No way to dig out the blob key from the backup?
Click to expand...
Click to collapse
No way - the BCT, bootloader, etc. is not visible to the kernel at all (so it's not included in your images), and I don't know which kind of magic the blob creation tool uses, but I assume it's more than reading stuff from the nand.
I modified the new stock kernel with the ramdisk features from CleanROM:
- insecure adbd
- init.d support
Flashable ZIP:
View attachment kernel-10.4.4.25-insecure.zip
Update 2013-01-21: Here is a version that does not set lcd_density, so that custom dpi set in /system/build.prop work. Verified to work with CROMI 3.2.
View attachment kernel-10.4.4.25-insecure-nodpi.zip
Working fine here with a mix of CleanROM 2.3, custom mods and some cherry-picked changes from CROMI. This kernel should be compatible with any stock ROM since 10.4.4.18 or CleanROM/Inheritance since 2.3.
My MicroSD card (Sandisk Mobile Ultra UHS-1) seems to work more stable now - will try All2SD tomorrow.
For the tutorial how to unpack and repack a kernel, see post #4.
Update 2013-01-21: Updated with info how to unlock DPI.
_that said:
I modified the new stock kernel with the ramdisk features from CleanROM:
- insecure adbd
- init.d support
Flashable ZIP:
View attachment 1657033
Working fine here with a mix of CleanROM 2.3, custom mods and some cherry-picked changes from CROMI. This kernel should be compatible with any stock ROM since 10.4.4.18 or CleanROM/Inheritance since 2.3.
My MicroSD card (Sandisk Mobile Ultra UHS-1) seems to work more stable now - will try All2SD tomorrow.
Click to expand...
Click to collapse
Nice work!!:good:
_that said:
I modified the new stock kernel with the ramdisk features from CleanROM:
- insecure adbd
- init.d support
Flashable ZIP:
View attachment 1657033
Working fine here with a mix of CleanROM 2.3, custom mods and some cherry-picked changes from CROMI. This kernel should be compatible with any stock ROM since 10.4.4.18 or CleanROM/Inheritance since 2.3.
My MicroSD card (Sandisk Mobile Ultra UHS-1) seems to work more stable now - will try All2SD tomorrow.
Click to expand...
Click to collapse
Many thanks - I've been struggling with linux to unpack the blob.LNX file. Blob tools isn't working for me for some reason as ./blobunpack blob.LNX says unrecognised blob format.
What is the secret to get it working?
sbdags said:
Many thanks - I've been struggling with linux to unpack the blob.LNX file. Blob tools isn't working for me for some reason as ./blobunpack blob.LNX says unrecognised blob format.
What is the secret to get it working?
Click to expand...
Click to collapse
I firmly believe that this should not be a secret, so I'll explain in detail how to do it and not only give you steps but also a lot of background info. Many steps in the following tutorial are optional and useful for learning, for checking things and for troubleshooting - I don't want you to blindly follow a recipe without understanding what you are doing.
Unpacking the blob
We start with the full firmware, unpacked once - in this case, WW_epad-user-10.4.4.25.zip. First extract the blob from the zip:
Code:
[email protected] ~/android/tf700/kernel $ unzip WW_epad-user-10.4.4.25.zip blob
Archive: WW_epad-user-10.4.4.25.zip
signed by SignApk
inflating: blob
This is a signed blob. It begins with "-SIGNED-BY-SIGNBLOB-". You can check this with hexdump (this step is completely optional, just for illustration):
Code:
[email protected] ~/android/tf700/kernel $ hexdump -C -n 48 blob
00000000 2d 53 49 47 4e 45 44 2d 42 59 2d 53 49 47 4e 42 |-SIGNED-BY-SIGNB|
00000010 4c 4f 42 2d e9 1a d7 30 00 01 00 00 4d 53 4d 2d |LOB-...0....MSM-|
00000020 52 41 44 49 4f 2d 55 50 44 41 54 45 00 00 01 00 |RADIO-UPDATE....|
00000030
Also note the MSM-RADIO-UPDATE after the signature header - it is the start of an unsigned blob.
But what exactly is a blob? In database terminology, it stands for "binary large object" - a bunch of data that is just stored with no further interpretation by the database.The blobs for flashing NVIDIA Tegra devices have a specific structure - they contain one or more raw images to be flashed to one or more partitions on the device.
The full firmware blob for the TF700 contains images for 5 partitions, which we will now extract from the blob. If you don't have blobunpack, get the sources from https://github.com/AndroidRoot/BlobTools - how to do that and how to compile them may be part of another tutorial. I also assume you have blobunpack somewhere in your $PATH.
Code:
[email protected] ~/android/tf700/kernel $ blobunpack blob
Header size: 60
5 partitions starting at offset 0x3C
Blob version: 65536
Partition 0
Name: PT
Offset: 140 (0x8C)
Size: 524288 (0x80000)
Writing file blob.PT (524288 bytes)
Partition 1
Name: EBT
Offset: 524428 (0x8008C)
Size: 1566813 (0x17E85D)
Writing file blob.EBT (1566813 bytes)
Partition 2
Name: SOS
Offset: 2091241 (0x1FE8E9)
Size: 6410496 (0x61D100)
Writing file blob.SOS (6410496 bytes)
Partition 3
Name: LNX
Offset: 8501737 (0x81B9E9)
Size: 5595392 (0x556100)
Writing file blob.LNX (5595392 bytes)
Partition 4
Name: APP
Offset: 14097129 (0xD71AE9)
Size: 805306368 (0x30000000)
Writing file blob.APP (805306368 bytes)
OK, now we have separate files for each partition. The next step is again optional, just to illustrate what happened:
Code:
[email protected] ~/android/tf700/kernel $ ls -l
total 2059512
-rw-rw-r-- 1 that users 468028852 28. Dez 12:33 WW_epad-user-10.4.4.25.zip
-rw-r--r-- 1 that users 819403781 22. Mär 2011 blob
-rw-r--r-- 1 that users 805306368 19. Jän 19:39 blob.APP
-rw-r--r-- 1 that users 1566813 19. Jän 19:39 blob.EBT
-rw-r--r-- 1 that users 5595392 19. Jän 19:39 blob.LNX
-rw-r--r-- 1 that users 524288 19. Jän 19:39 blob.PT
-rw-r--r-- 1 that users 6410496 19. Jän 19:39 blob.SOS
What's inside a firmware blob
blob.APP is a raw image of a Linux ext4 partition. If you are interested in the contents, you can loop mount this file and access all files inside directly on your Linux computer. This is the filesystem that is mounted as /system - the ROM as you all know it.
blob.EBT is the bootloader.
blob.LNX is the boot image - the Linux kernel and the ramdisk, which is later seen as "/". This is the one we are after.
blob.PT is the Tegra partition table. This is a proprietary format, not a PC-compatible partition table.
blob.SOS is the recovery boot image - another Linux kernel and another ramdisk, but this ramdisk contains the stock recovery (incl. the dreaded "dead android").
The boot image
Now, what's actually a boot image, and how do we recognize one if we see one? As usual, the best answer is: look inside! (again - optional, just to learn something)
Code:
[email protected] ~/android/tf700/kernel $ hexdump -C -n 16 blob.LNX
00000000 41 4e 44 52 4f 49 44 21 58 0c 4e 00 00 80 00 10 |ANDROID!X.N.....|
00000010
An Android boot image begins with the text "ANDROID!". Easy.
(maybe sbdags now starts to smack his head and realizes why blobunpack does not work here - and if anyone is still in doubt - a boot image is not a blob)
Splitting a boot image
Good, but how do we crack this thing open? Fortunately there are two small perl scripts I extracted from dsixda's Android kitchen (they are in tools/extract_boot_files):
Code:
[email protected] ~/android/tf700/kernel $ extract-kernel.pl blob.LNX
[email protected] ~/android/tf700/kernel $ extract-ramdisk.pl blob.LNX
1932 blocks
OK, what happened?
Code:
[email protected] ~/android/tf700/kernel $ ls -l
total 2064524
-rw-rw-r-- 1 that users 468028852 28. Dez 12:33 WW_epad-user-10.4.4.25.zip
-rw-r--r-- 1 that users 819403781 22. Mär 2011 blob
-rw-r--r-- 1 that users 805306368 19. Jän 19:39 blob.APP
-rw-r--r-- 1 that users 1566813 19. Jän 19:39 blob.EBT
-rw-r--r-- 1 that users 5595392 19. Jän 19:39 blob.LNX
drwxr-xr-x 8 that users 4096 19. Jän 19:59 blob.LNX-ramdisk
-rw-r--r-- 1 that users 524288 19. Jän 19:39 blob.PT
-rw-r--r-- 1 that users 6410496 19. Jän 19:39 blob.SOS
-rw-r--r-- 1 that users 5114968 19. Jän 19:58 zImage
We got a new file (zImage) and a directory (blob.LNX-ramdisk)!
zImage is the kernel image. It is the result of compiling a kernel from sources. Today, we will not compile a new kernel, just repack this image with our modified ramdisk.
What blob.LNX-ramdisk contains should be obvious.
Modifying the ramdisk
Now we can start modifying the ramdisk. A CleanROM ramdisk differs from a stock ramdisk in only 3 files:
- sbin/adbd
- default.prop
- init.rc
sbin/adbd is the adb daemon - the program that handles the tablet's side of the adb protocol. The stock version runs under the "shell" user account, which has limited permissions and cannot access all files on the tablet. The CleanROM version of adbd runs as "root", so "adb push" and "adb pull" can write or read any file on the device, and "adb shell" goes directly to a root prompt.
I don't know where scrosler got this binary from, I just copied it from his CleanROM kernel's ramdisk. You can do the same - now you already know how to extract a ramdisk, so just do the same with the CleanROM kernel in another directory.
default.prop contains only a few lines. Here is the already modified CleanROM version:
Code:
#
# ADDITIONAL_DEFAULT_PROPERTIES
#
ro.secure=0
ro.allow.mock.location=0
ro.debuggable=1
persist.sys.usb.config=mtp,adb
In the original file, ro.secure=1 (which would limit adb to the "shell" account again), ro.debuggable=0 (I'd have to look up what exactly this does), and persist.sys.usb.config=mtp without adb (I think this modification is what automatically enables "USB debugging" at boot time).
Finally, init.rc has a few additional lines to enable init.d support. This means the boot sequence will run any scripts in /etc/init.d - this a simplified version of a usual convention for desktop Linux systems. CleanROM uses it to set up various tweaks (look into /system/etc/init.d on your device).
To illustrate the changes in init.rc, I use the "diff" command (and you can too, if you want - just use the correct paths for your setup instead of mine):
Code:
[email protected] ~/android/tf700/kernel $ diff -u blob.LNX-ramdisk/init.rc ~/android/tf700/cleanrom/blob.LNX-ramdisk/init.rc
--- blob.LNX-ramdisk/init.rc 2013-01-19 19:59:03.000000000 +0100
+++ /home/that/android/tf700/cleanrom/blob.LNX-ramdisk/init.rc 2013-01-19 01:24:43.000000000 +0100
@@ -309,6 +309,8 @@
# Set this property so surfaceflinger is not started by system_init
setprop system_init.startsurfaceflinger 0
+ start sysinit
+
class_start core
class_start main
@@ -413,6 +415,10 @@
user drm
group drm system inet drmrpc sdcard_r
+service sysinit /system/bin/logwrapper /system/xbin/busybox run-parts /system/etc/init.d
+ disabled
+ oneshot
+
service media /system/bin/mediaserver
class main
user media
If you have never read a diff before (also called a patch), here a quick guide:
- Lines beginning with "---" and "+++" identify the two files that were compared.
- Lines beginning with "@@" specify line numbers in the two files. -309,6 +309,8 means: The next block contains 6 lines starting with line 309 from file 1 (---), and 8 lines starting with line 309 in file 2 (+++). In other words, this means that two lines were inserted there.
- The following lines start with a blank - they are context lines. These lines are common in both files and only used to see some context around the change.
- Lines beginning with "+" exist only in file 2 (+++). You can easily remember all this if you know that a diff describes how to edit file 1 to be identical to file 2 - you delete lines marked with "-" and insert lines marked with "+". With only these two operations, you can transform any text file to any other - in the worst case you will delete all lines of the original file and insert all lines of the new file.
The second block works like the first one - here we insert 4 lines near line 413 (which has become line 415 after our first insertion).
You have the following options to apply this change:
1. Simply copy over the changed version from the previous CleanROM kernel. Do a diff before to verify that Asus didn't change anything, otherwise you will undo Asus' changes.
2. Use the "patch" tool to apply the diff file from above. This will automatically perform the edit steps in the diff file and it also has some tolerance if something changed in other sections of the file.
3. Use a text editor and perform the changes manually. If you copy lines from the diff (and not the original init.rc), make sure you don't accidentally copy the leading "+" - this is only for the diff and should not part of the file.
Update 2013-01-21 Start
Unlocking DPI
As usual I don't just tell you how to do it, but some background info first:
There is a read-only system property in Android named "lcd_density" that specifies the display resolution. This is done to ensure that the approximate size of display items is kept consistent across the wide range of devices Android runs on.
For normal users this is great, because they don't have to buy a magnifying glass and a pointed stylus just to use their super-hires-devices, but users here on xda-developers of course see a setting that can be modified and therefore it *must* be modified. Smaller values produce smaller icons and text (and thus more visible content at once); bigger values produce bigger icons and text.
The ASUS stock firmware is not only used on the TF700, but also for several other Transformers. The file /system/build.prop sets ro.sf.lcd_density=160, which means 160 dpi. This is good for the majority of tablets that have 1280*800 displays, but the TF700's display has a resolution of 224 pixels per inch, and with a setting of 160 dpi everything would be very tiny.
The bootloader passes an important parameter to the kernel: "androidboot.productid=0x04", This value depends on the Transformer model, and it is 4 for the TF700. If you look into the ramdisk, there are several files named init.00.rc to init.0d.rc. These are device-specific, and you guessed right if you think that init.04.rc is for the TF700. This file sets lcd_density to 240 dpi, and since it is a read-only property, it keeps its value even if the system's build.prop (which is read later) tries to set lcd_density to 160.
So how do we unlock dpi? Delete the line that says "setprop ro.sf.lcd_density 240" in init.04.rc! Of course this requires you to set the desired dpi value in /system/build.prop. If you install CleanROM Inheritance, you can choose this value in the installer. If you run this kernel with the stock ROM, everything will become very tiny, and now you know why.
Update 2013-01-21 End
Re-packing the boot image
OK, phew, we finally have a modified ramdisk - now we need to repack everything, basically the reverse of the first few steps. For this I use a self-written shell script. First let's just run it:
Code:
[email protected] ~/android/tf700/kernel $ repack-kernel.sh
Found 1 partitions as commandline arguments
Partname: LNX Filename: blob.LNX
Size: 60
1 partitions starting at offset 0x3C
Offset: 76
8+0 records in
8+0 records out
8 bytes (8 B) copied, 0,000136275 s, 58,7 kB/
OK, what happened here? Since I wrote this script myself, I can explain it line by line:
Code:
[email protected] ~/android/tf700/kernel $ cat ~/bin/repack-kernel.sh
#!/bin/bash
~/android/Android-Kitchen/tools/mkboot/mkbootfs blob.LNX-ramdisk | gzip > ramdisk.gz
~/android/Android-Kitchen/tools/mkboot/mkbootimg --kernel zImage --ramdisk ramdisk.gz -o blob.LNX
blobpack blobtmp LNX blob.LNX
echo -n "-SIGNED-BY-SIGNBLOB-" > blob
dd if=/dev/zero bs=1 count=8 >> blob
cat blobtmp >> blob
The ramdisk is repacked and zipped again. This creates ramdisk.gz.
The kernel is merged with the ramdisk to a new boot image, to create blob.LNX. If you paid attention before, yes, this will overwrite the old blob.LNX. If you want, you can check again with hexdump that it is really a boot image (it begins with "ANDROID!").
Re-packing a blob
Now we pack the boot image into a blob. However, the "original" version of blobtools does not create a signed blob. See:
Code:
[email protected] ~/android/tf700/kernel $ hexdump -C -n 16 blobtmp
00000000 4d 53 4d 2d 52 41 44 49 4f 2d 55 50 44 41 54 45 |MSM-RADIO-UPDATE|
00000010
That was the blob format for the TF101. For later Transformers, we need to prepend a signature header. As we don't have Asus' cryptographic keys, we cannot create a valid signature - but fortunately, since we have an unlocked bootloader, we don't actually need to add a proper signature - the header is enough. There are two ways to create this header - there is a modified version of blobtools in circulation that has a command line option for it, but I do it "manually" with echo and dd.
The final output is again simply called "blob" - this is the new blob that now contains only the kernel. (It overwrites the original blob, which is not really intended, but I just didn't care).
For comparison, now we have this:
Code:
[email protected] ~/android/tf700/kernel $ ls -l
total 1274980
-rw-rw-r-- 1 that users 468028852 28. Dez 12:33 WW_epad-user-10.4.4.25.zip
-rw-r--r-- 1 that users 5599336 19. Jän 20:35 blob
-rw-r--r-- 1 that users 805306368 19. Jän 19:39 blob.APP
-rw-r--r-- 1 that users 1566813 19. Jän 19:39 blob.EBT
-rw-r--r-- 1 that users 5599232 19. Jän 20:35 blob.LNX
drwxr-xr-x 8 that users 4096 19. Jän 19:59 blob.LNX-ramdisk
-rw-r--r-- 1 that users 524288 19. Jän 19:39 blob.PT
-rw-r--r-- 1 that users 6410496 19. Jän 19:39 blob.SOS
-rw-r--r-- 1 that users 5599308 19. Jän 20:35 blobtmp
-rw-r--r-- 1 that users 479752 19. Jän 20:35 ramdisk.gz
-rw-r--r-- 1 that users 5114968 19. Jän 19:58 zImage
Congratulations, you have a flashable blob with your new kernel. But how do you get it onto your device?
Installation
Again, there are two ways:
Installing the blob directly with dd
The first option is to copy the blob to your tablet's staging partition manually and reboot:
Code:
[email protected] ~/android/tf700/kernel $ adb push blob /sdcard/kernel-10.4.4.25.blob
1324 KB/s (5599336 bytes in 4.128s)
[email protected] ~/android/tf700/kernel $ adb shell
sh-3.2# dd if=/sdcard/kernel-10.4.4.25.blob of=/dev/block/mmcblk0p4
10936+1 records in
10936+1 records out
5599336 bytes transferred in 0.129 secs (43405705 bytes/sec)
Be very careful with the dd command and double-check that you didn't swap "if" and "of", and that you specified the correct target partition *before* you hit Enter.
Now reboot your tablet, and the bootloader will flash your new kernel from the blob to the actual kernel partition (notice the progress bar on the boot screen with the slightly broken graphics, as usual). The tablet will reboot again automatically, and if you did everything right, your shiny new kernel will be running!
Creating a zip for flashing in the recovery
OK, now to option 2 - pack the blob into a flashable zip that you can flash in the recovery. The easiest way to do this is to reuse the original firmware zip. Just replace the blob inside the big firmware zip, and that's it - you have a recovery-flashable zip!
Code:
[email protected] ~/android/tf700/kernel $ mv WW_epad-user-10.4.4.25.zip kernel-10.4.4.25.zip
[email protected] ~/android/tf700/kernel $ zip -d kernel-10.4.4.25.zip blob
deleting: blob
[email protected] ~/android/tf700/kernel $ zip kernel-10.4.4.25.zip blob
adding: blob (deflated 1%)
This would not be a good tutorial if I didn't explain how and why this actually works. Let's look inside!
Code:
[email protected] ~/android/tf700/kernel $ unzip kernel-10.4.4.25.zip META-INF/*
Archive: kernel-10.4.4.25.zip
signed by SignApk
inflating: META-INF/com/google/android/resource
inflating: META-INF/com/google/android/rule
inflating: META-INF/com/google/android/update-binary
inflating: META-INF/com/google/android/updater-script
inflating: META-INF/com/android/otacert
inflating: META-INF/MANIFEST.MF
inflating: META-INF/CERT.SF
inflating: META-INF/CERT.RSA
The core of a flashable zip is the updater-script. For a full firmware, it is very short:
Code:
[email protected] ~/android/tf700/kernel $ cat META-INF/com/google/android/updater-script
assert(package_extract_file("blob", "/tmp/blob"), write_raw_image("/tmp/blob", "staging"),delete("/tmp/blob"));
This line instructs the recovery to extract the blob from the zip file to /tmp/blob and then copy it to the staging partition - essentially it is exactly the same as we did manually in option #1 above!
OK, now the secret is fully disclosed - I hope you learned something. If you have further questions, or if you need help compiling some of the mentioned binaries, don't hesitate to ask (if it's not something like "how do I use the Linux command line" ).
uh huh, uh huh, What?
I think I need to go to blob school....:silly:
_that said:
I firmly believe that this should not be a secret, so I'll explain in detail how to do it and not only give you steps but also a lot of background info. Many steps in the following tutorial are optional and useful for learning, for checking things and for troubleshooting - I don't want you to blindly follow a recipe without understanding what you are doing.
We start with the full firmware, unpacked once - in this case, WW_epad-user-10.4.4.25.zip. First extract the blob from the zip:
Code:
[email protected] ~/android/tf700/kernel $ unzip WW_epad-user-10.4.4.25.zip blob
Archive: WW_epad-user-10.4.4.25.zip
signed by SignApk
inflating: blob
This is a signed blob. It begins with "-SIGNED-BY-SIGNBLOB-". You can check this with hexdump (this step is completely optional, just for illustration):
Code:
[email protected] ~/android/tf700/kernel $ hexdump -C -n 48 blob
00000000 2d 53 49 47 4e 45 44 2d 42 59 2d 53 49 47 4e 42 |-SIGNED-BY-SIGNB|
00000010 4c 4f 42 2d e9 1a d7 30 00 01 00 00 4d 53 4d 2d |LOB-...0....MSM-|
00000020 52 41 44 49 4f 2d 55 50 44 41 54 45 00 00 01 00 |RADIO-UPDATE....|
00000030
Also note the MSM-RADIO-UPDATE after the signature header - it is the start of an unsigned blob.
But what exactly is a blob? In database terminology, it stands for "binary large object" - a bunch of data that is just stored with no further interpretation by the database.The blobs for flashing NVIDIA Tegra devices have a specific structure - they contain one or more raw images to be flashed to one or more partitions on the device.
The full firmware blob for the TF700 contains images for 5 partitions, which we will now extract from the blob. If you don't have blobunpack, get the sources from https://github.com/AndroidRoot/BlobTools - how to do that and how to compile them may be part of another tutorial. I also assume you have blobunpack somewhere in your $PATH.
Code:
[email protected] ~/android/tf700/kernel $ blobunpack blob
Header size: 60
5 partitions starting at offset 0x3C
Blob version: 65536
Partition 0
Name: PT
Offset: 140 (0x8C)
Size: 524288 (0x80000)
Writing file blob.PT (524288 bytes)
Partition 1
Name: EBT
Offset: 524428 (0x8008C)
Size: 1566813 (0x17E85D)
Writing file blob.EBT (1566813 bytes)
Partition 2
Name: SOS
Offset: 2091241 (0x1FE8E9)
Size: 6410496 (0x61D100)
Writing file blob.SOS (6410496 bytes)
Partition 3
Name: LNX
Offset: 8501737 (0x81B9E9)
Size: 5595392 (0x556100)
Writing file blob.LNX (5595392 bytes)
Partition 4
Name: APP
Offset: 14097129 (0xD71AE9)
Size: 805306368 (0x30000000)
Writing file blob.APP (805306368 bytes)
OK, now we have separate files for each partition. The next step is again optional, just to illustrate what happened:
Code:
[email protected] ~/android/tf700/kernel $ ls -l
total 2059512
-rw-rw-r-- 1 that users 468028852 28. Dez 12:33 WW_epad-user-10.4.4.25.zip
-rw-r--r-- 1 that users 819403781 22. Mär 2011 blob
-rw-r--r-- 1 that users 805306368 19. Jän 19:39 blob.APP
-rw-r--r-- 1 that users 1566813 19. Jän 19:39 blob.EBT
-rw-r--r-- 1 that users 5595392 19. Jän 19:39 blob.LNX
-rw-r--r-- 1 that users 524288 19. Jän 19:39 blob.PT
-rw-r--r-- 1 that users 6410496 19. Jän 19:39 blob.SOS
blob.APP is a raw image of a Linux ext4 partition. If you are interested in the contents, you can loop mount this file and access all files inside directly on your Linux computer. This is the filesystem that is mounted as /system - the ROM as you all know it.
blob.EBT is the bootloader.
blob.LNX is the boot image - the Linux kernel and the ramdisk, which is later seen as "/". This is the one we are after.
blob.PT is the Tegra partition table. This is a proprietary format, not a PC-compatible partition table.
blob.SOS is the recovery boot image - another Linux kernel and another ramdisk, but this ramdisk contains the stock recovery (incl. the dreaded "dead android").
Now, what's actually a boot image, and how do we recognize one if we see one? As usual, the best answer is: look inside! (again - optional, just to learn something)
Code:
[email protected] ~/android/tf700/kernel $ hexdump -C -n 16 blob.LNX
00000000 41 4e 44 52 4f 49 44 21 58 0c 4e 00 00 80 00 10 |ANDROID!X.N.....|
00000010
An Android boot image begins with the text "ANDROID!". Easy.
(maybe sbdags now starts to smack his head and realizes why blobunpack does not work here - and if anyone is still in doubt - a boot image is not a blob)
Good, but how do we crack this thing open? Fortunately there are two small perl scripts I extracted from dsixda's Android kitchen (they are in tools/extract_boot_files):
Code:
[email protected] ~/android/tf700/kernel $ extract-kernel.pl blob.LNX
[email protected] ~/android/tf700/kernel $ extract-ramdisk.pl blob.LNX
1932 blocks
OK, what happened?
Code:
[email protected] ~/android/tf700/kernel $ ls -l
total 2064524
-rw-rw-r-- 1 that users 468028852 28. Dez 12:33 WW_epad-user-10.4.4.25.zip
-rw-r--r-- 1 that users 819403781 22. Mär 2011 blob
-rw-r--r-- 1 that users 805306368 19. Jän 19:39 blob.APP
-rw-r--r-- 1 that users 1566813 19. Jän 19:39 blob.EBT
-rw-r--r-- 1 that users 5595392 19. Jän 19:39 blob.LNX
drwxr-xr-x 8 that users 4096 19. Jän 19:59 blob.LNX-ramdisk
-rw-r--r-- 1 that users 524288 19. Jän 19:39 blob.PT
-rw-r--r-- 1 that users 6410496 19. Jän 19:39 blob.SOS
-rw-r--r-- 1 that users 5114968 19. Jän 19:58 zImage
We got a new file (zImage) and a directory (blob.LNX-ramdisk)!
zImage is the kernel image. It is the result of compiling a kernel from sources. Today, we will not compile a new kernel, just repack this image with our modified ramdisk.
What blob.LNX-ramdisk contains should be obvious.
Now we can start modifying the ramdisk. A CleanROM ramdisk differs from a stock ramdisk in only 3 files:
- sbin/adbd
- default.prop
- init.rc
sbin/adbd is the adb daemon - the program that handles the tablet's side of the adb protocol. The stock version runs under the "shell" user account, which has limited permissions and cannot access all files on the tablet. The CleanROM version of adbd runs as "root", so "adb push" and "adb pull" can write or read any file on the device, and "adb shell" goes directly to a root prompt.
I don't know where scrosler got this binary from, I just copied it from his CleanROM kernel's ramdisk. You can do the same - now you already know how to extract a ramdisk, so just do the same with the CleanROM kernel in another directory.
default.prop contains only a few lines. Here is the already modified CleanROM version:
Code:
#
# ADDITIONAL_DEFAULT_PROPERTIES
#
ro.secure=0
ro.allow.mock.location=0
ro.debuggable=1
persist.sys.usb.config=mtp,adb
In the original file, ro.secure=1 (which would limit adb to the "shell" account again), ro.debuggable=0 (I'd have to look up what exactly this does), and persist.sys.usb.config=mtp without adb (I think this modification is what automatically enables "USB debugging" at boot time).
Finally, init.rc has a few additional lines to enable init.d support. This means the boot sequence will run any scripts in /etc/init.d - this a simplified version of a usual convention for desktop Linux systems. CleanROM uses it to set up various tweaks (look into /system/etc/init.d on your device).
To illustrate the changes in init.rc, I use the "diff" command (and you can too, if you want - just use the correct paths for your setup instead of mine):
Code:
[email protected] ~/android/tf700/kernel $ diff -u blob.LNX-ramdisk/init.rc ~/android/tf700/cleanrom/blob.LNX-ramdisk/init.rc
--- blob.LNX-ramdisk/init.rc 2013-01-19 19:59:03.000000000 +0100
+++ /home/that/android/tf700/cleanrom/blob.LNX-ramdisk/init.rc 2013-01-19 01:24:43.000000000 +0100
@@ -309,6 +309,8 @@
# Set this property so surfaceflinger is not started by system_init
setprop system_init.startsurfaceflinger 0
+ start sysinit
+
class_start core
class_start main
@@ -413,6 +415,10 @@
user drm
group drm system inet drmrpc sdcard_r
+service sysinit /system/bin/logwrapper /system/xbin/busybox run-parts /system/etc/init.d
+ disabled
+ oneshot
+
service media /system/bin/mediaserver
class main
user media
If you have never read a diff before (also called a patch), here a quick guide:
- Lines beginning with "---" and "+++" identify the two files that were compared.
- Lines beginning with "@@" specify line numbers in the two files. -309,6 +309,8 means: The next block contains 6 lines starting with line 309 from file 1 (---), and 8 lines starting with line 309 in file 2 (+++). In other words, this means that two lines were inserted there.
- The following lines start with a blank - they are context lines. These lines are common in both files and only used to see some context around the change.
- Lines beginning with "+" exist only in file 2 (+++). You can easily remember all this if you know that a diff describes how to edit file 1 to be identical to file 2 - you delete lines marked with "-" and insert lines marked with "+". With only these two operations, you can transform any text file to any other - in the worst case you will delete all lines of the original file and insert all lines of the new file.
The second block works like the first one - here we insert 4 lines near line 413 (which has become line 415 after our first insertion).
You have the following options to apply this change:
1. Simply copy over the changed version from the previous CleanROM kernel. Do a diff before to verify that Asus didn't change anything, otherwise you will undo Asus' changes.
2. Use the "patch" tool to apply the diff file from above. This will automatically perform the edit steps in the diff file and it also has some tolerance if something changed in other sections of the file.
3. Use a text editor and perform the changes manually. If you copy lines from the diff (and not the original init.rc), make sure you don't accidentally copy the leading "+" - this is only for the diff and should not part of the file.
OK, phew, we finally have a modified ramdisk - now we need to repack everything, basically the reverse of the first few steps. For this I use a self-written shell script. First let's just run it:
Code:
[email protected] ~/android/tf700/kernel $ repack-kernel.sh
Found 1 partitions as commandline arguments
Partname: LNX Filename: blob.LNX
Size: 60
1 partitions starting at offset 0x3C
Offset: 76
8+0 records in
8+0 records out
8 bytes (8 B) copied, 0,000136275 s, 58,7 kB/
OK, what happened here? Since I wrote this script myself, I can explain it line by line:
Code:
[email protected] ~/android/tf700/kernel $ cat ~/bin/repack-kernel.sh
#!/bin/bash
~/android/Android-Kitchen/tools/mkboot/mkbootfs blob.LNX-ramdisk | gzip > ramdisk.gz
~/android/Android-Kitchen/tools/mkboot/mkbootimg --kernel zImage --ramdisk ramdisk.gz -o blob.LNX
blobpack blobtmp LNX blob.LNX
echo -n "-SIGNED-BY-SIGNBLOB-" > blob
dd if=/dev/zero bs=1 count=8 >> blob
cat blobtmp >> blob
The ramdisk is repacked and zipped again. This creates ramdisk.gz.
The kernel is merged with the ramdisk to a new boot image, to create blob.LNX. If you paid attention before, yes, this will overwrite the old blob.LNX. If you want, you can check again with hexdump that it is really a boot image (it begins with "ANDROID!").
Now we pack the boot image into a blob. However, the "original" version of blobtools does not create a signed blob. See:
Code:
[email protected] ~/android/tf700/kernel $ hexdump -C -n 16 blobtmp
00000000 4d 53 4d 2d 52 41 44 49 4f 2d 55 50 44 41 54 45 |MSM-RADIO-UPDATE|
00000010
That was the blob format for the TF101. For later Transformers, we need to prepend a signature header. As we don't have Asus' cryptographic keys, we cannot create a valid signature - but fortunately, since we have an unlocked bootloader, we don't actually need to add a proper signature - the header is enough. There are two ways to create this header - there is a modified version of blobtools in circulation that has a command line option for it, but I do it "manually" with echo and dd.
The final output is again simply called "blob" - this is the new blob that now contains only the kernel. (It overwrites the original blob, which is not really intended, but I just didn't care).
For comparison, now we have this:
Code:
[email protected] ~/android/tf700/kernel $ ls -l
total 1274980
-rw-rw-r-- 1 that users 468028852 28. Dez 12:33 WW_epad-user-10.4.4.25.zip
-rw-r--r-- 1 that users 5599336 19. Jän 20:35 blob
-rw-r--r-- 1 that users 805306368 19. Jän 19:39 blob.APP
-rw-r--r-- 1 that users 1566813 19. Jän 19:39 blob.EBT
-rw-r--r-- 1 that users 5599232 19. Jän 20:35 blob.LNX
drwxr-xr-x 8 that users 4096 19. Jän 19:59 blob.LNX-ramdisk
-rw-r--r-- 1 that users 524288 19. Jän 19:39 blob.PT
-rw-r--r-- 1 that users 6410496 19. Jän 19:39 blob.SOS
-rw-r--r-- 1 that users 5599308 19. Jän 20:35 blobtmp
-rw-r--r-- 1 that users 479752 19. Jän 20:35 ramdisk.gz
-rw-r--r-- 1 that users 5114968 19. Jän 19:58 zImage
Congratulations, you have a flashable blob with your new kernel. But how do you get it onto your device?
Again, there are two ways:
The first option is to copy the blob to your tablet's staging partition manually and reboot:
Code:
[email protected] ~/android/tf700/kernel $ adb push blob /sdcard/kernel-10.4.4.25.blob
1324 KB/s (5599336 bytes in 4.128s)
[email protected] ~/android/tf700/kernel $ adb shell
sh-3.2# dd if=/sdcard/kernel-10.4.4.25.blob of=/dev/block/mmcblk0p4
10936+1 records in
10936+1 records out
5599336 bytes transferred in 0.129 secs (43405705 bytes/sec)
Be very careful with the dd command and double-check that you didn't swap "if" and "of", and that you specified the correct target partition *before* you hit Enter.
Now reboot your tablet, and the bootloader will flash your new kernel from the blob to the actual kernel partition (notice the progress bar on the boot screen with the slightly broken graphics, as usual). The tablet will reboot again automatically, and if you did everything right, your shiny new kernel will be running!
OK, now to option 2 - pack the blob into a flashable zip that you can flash in the recovery. The easiest way to do this is to reuse the original firmware zip. Just replace the blob inside the big firmware zip, and that's it - you have a recovery-flashable zip!
Code:
[email protected] ~/android/tf700/kernel $ mv WW_epad-user-10.4.4.25.zip kernel-10.4.4.25.zip
[email protected] ~/android/tf700/kernel $ zip -d kernel-10.4.4.25.zip blob
deleting: blob
[email protected] ~/android/tf700/kernel $ zip kernel-10.4.4.25.zip blob
adding: blob (deflated 1%)
This would not be a good tutorial if I didn't explain how and why this actually works. Let's look inside!
Code:
[email protected] ~/android/tf700/kernel $ unzip kernel-10.4.4.25.zip META-INF/*
Archive: kernel-10.4.4.25.zip
signed by SignApk
inflating: META-INF/com/google/android/resource
inflating: META-INF/com/google/android/rule
inflating: META-INF/com/google/android/update-binary
inflating: META-INF/com/google/android/updater-script
inflating: META-INF/com/android/otacert
inflating: META-INF/MANIFEST.MF
inflating: META-INF/CERT.SF
inflating: META-INF/CERT.RSA
The core of a flashable zip is the updater-script. For a full firmware, it is very short:
Code:
[email protected] ~/android/tf700/kernel $ cat META-INF/com/google/android/updater-script
assert(package_extract_file("blob", "/tmp/blob"), write_raw_image("/tmp/blob", "staging"),delete("/tmp/blob"));
This line instructs the recovery to extract the blob from the zip file to /tmp/blob and then copy it to the staging partition - essentially it is exactly the same as we did manually in option #1 above!
OK, now the secret is fully disclosed - I hope you learned something. If you have further questions, or if you need help compiling some of the mentioned binaries, don't hesitate to ask (if it's not something like "how do I use the Linux command line" ).
Click to expand...
Click to collapse
Wow! Many thanks! :victory:
Will digest this tomorrow but yes I recognised my smack head moment! :laugh:
Hmmm, my extreme pro 16gb seems to work fine with this kernel. Using/testing it as an external storage, formatted as fat32. Before it wasen't even recognized when inserted. In recovery (twrp) the card is still not recognized but in android (cleanrom 3.1) it works fine.
New kernel from _that so bad,this kernel dont have OC and voltage control,DPI and most important for me balance power mode 1600...performance power mode do nothing.
But i still want asus changes from new kernel.I hope someone create normal kernel like was Clemsyn's kernel or may be _that do this?
But still thanks for your work and sorry if it's all sounds not very good...
Blue cat said:
New kernel from _that so bad,this kernel dont have OC and voltage control,DPI and most important for me balance power mode 1600...performance power mode do nothing.
But i still want asus changes from new kernel.I hope someone create normal kernel like was Clemsyn's kernel or may be _that do this?
Click to expand...
Click to collapse
That's a bit harsh don't you think.
sbdags said:
That's a bit harsh don't you think.
Click to expand...
Click to collapse
Yes,sorry if it's sounds harsh,forgot add in my post gratitude for his work,cant edit next 5 min.I just very want OC and new kernel.
Blue cat said:
Yes,sorry if it's sounds harsh,forgot add in my post gratitude for his work,cant edit next 5 min.I just very want OC and new kernel.
Click to expand...
Click to collapse
_that has only worked on the Asus Stock kernel, for which I am extremely grateful. It gives those users who don't want to OC anything the ability to have a custom rom on the new kernel. From my testing it works better than the old kernel and is very stable.
I will be looking at the Clemsyn kernels in the mean time to see if I can bring them onto the new kernel base and the benefits they bring.
Please be patient, these are not 5 minute jobs. It can take many hours to unpack, decompile, edit, recompile, repack a kernel into a working flashable package. Then we have to test it and live with bootloops and soft bricks if we get it wrong. I can not tell you how many times I have had to restore my Infinity due to one line of code wrong or a badly compiled apk. It's all so you don't have to.
Anyway I'm sure you understand :good: Why don't you read the guides and have a go yourself? Always good to have more people able to do this type of work
sbdags said:
Why don't you read the guides and have a go yourself?
Click to expand...
Click to collapse
I tried to understand,but it's all very hard for me and ok i will be wait your new kernel with OC
Blue cat said:
New kernel from _that so bad,this kernel dont have OC and voltage control,DPI and most important for me balance power mode 1600...performance power mode do nothing.
But i still want asus changes from new kernel.I hope someone create normal kernel like was Clemsyn's kernel or may be _that do this?
Click to expand...
Click to collapse
This is a 100% stock kernel with just a custom ramdisk. Stock kernels never have any of the features you describe here.
Asus have not released their sources for 10.4.4.25 yet, and Clemsyn's kernel sources are a big mess. As soon as sources are released, it is possible to build a new kernel with OC features.
sbdags said:
I will be looking at the Clemsyn kernels in the mean time to see if I can bring them onto the new kernel base and the benefits they bring.
Click to expand...
Click to collapse
Do you have a clue how to build the kernels in your kernel installer from Clemsyn's sources?
sbdags said:
Please be patient, these are not 5 minute jobs.
Click to expand...
Click to collapse
Right, this kernel was more like 10 minutes work. Writing the tutorial took more than 2 hours.
_that said:
Do you have a clue how to build the kernels in your kernel installer from Clemsyn's sources?
/QUOTE]
Nope. Just unpacked yours and clemsyn's. Right, that's not going to happen anytime fast!
:laugh:
Click to expand...
Click to collapse
Great job, _that, you have my sincere thanks. I still cannot find the Thanks button on my screen because of all the blood and brain tissue that got on there when my head exploded. I'll take a walk over to the ED department to get that fixed with a printed plastic cranium, glue it shut, cover it up with a few bandages, get back to my own department and then clean up for a bit. When I finally regain something of a normal level of consciousness I'll try and step into the shadown you throw upon me, and press the damn button.
EDIT: button pressed. Speech still slurring a bit, minor tremor in the left hand and incontinent for both faeces and urine. Apart from that, I'm fine, thanks for asking.
Just kidding, obviously. I've read along, and I get most parts theoretically. Thanks for making me learn -- the most important mental facility we humans have.
Hey very good tutorial
I just unpacked your kernel and take a look inside the ramdisk and repacked the blob.
And it boots up normaly without problems.
Now I am waiting for Asus to release their latest kernel sources that I can modify the kernel itself, too and not only the ramdisk.
Thanks for helping me with that blob stuff!
blackmambazzz said:
Hey very good tutorial
I just unpacked your kernel and take a look inside the ramdisk and repacked the blob.
And it boots up normaly without problems.
Click to expand...
Click to collapse
:good: Great!
blackmambazzz said:
Now I am waiting for Asus to release their latest kernel sources that I can modify the kernel itself, too and not only the ramdisk.
Click to expand...
Click to collapse
Do you have any specific mods in mind?
_that said:
Do you have any specific mods in mind?
Click to expand...
Click to collapse
I think about this features but nothing for sure
more governors
more io-scheduler
compiled with linaro and -o3 optimizations
add fsync control
add oc (by this one i need help I think)
increase readahead
NODPI Update!
Excellent work _that! I will implement the nodpi kernel into the kernel flasher tomorrow and when the odex rom is finished I'll put it in instead of the one there now. :good::good::good:
Thank you, _that, for your hard work. Your kernel seems to be faster than Clemysn's, at least on first sight (scr** the benchmark hippies -- it feels faster, OK? ).
Flashing this was so fast, I thought I had accidentally pressed "reboot system now" instead of actually flashing your kernel. Off to testing a bit more! ; )
Hello everybody!
This will be my first XDA-wide release of my new utility for all LG phones.
What is this?
This is a utility to extract the new format KDZ files that LG distributes, specifically the 'compressed' ones.
LG frequently distributes firmware for phones as KDZ files, which are essentially a firmware image of the eMMC and a DLL file that is used by the downloader utility to communicate with the phone.
In the past, there were utilities to extract KDZ files to a DLL file and a DZ file, but no further (at least to my knowledge).
This utility lets you break the KDZ file into it's respective partitions (aboot, rpm, tz, and so on)
What good does this do me?
If you're an phone modder, haxxor, or just an enthusiast that has access to their phone's KDZ file and would like to have a copy of the actual partitions stored within, this will let you.
As an example, firmware for the new LG G2 on many device models is distributed as a KDZ file only.
Other phones use a TOT file, which is essentially a disk image of the eMMC with no compression.
If someone with a KDZ firmware-only phone wiped a partition (for example, modem) and wanted to get it back without flashing the whole phone all over again, they would be stuck.
TOT files are easily extractable, as there is software available currently for that but until today there was none (to my knowledge) for these new KDZ files.
How do I use this?
Glad you asked.
Inside the ZIP file you'll see two Python scripts, KDZFileTools.py and DZFileTools.py.
There's also a README.txt file for more in-depth information if you're curious.
Both scripts respond to --help or -h, so if you're even more curious, try that too!
KDZ files contain DZ files and DLL files, so the first step will be to split those into their respective parts:
LAS_V08d_pre3_00.kdz is the name of the KDZ file that I've copied to the working directory for this example.
Code:
# python KDZFileTools.py -l -f LAS_V08d_pre3_00.kdz
[+] KDZ Partition List
=========================================
0 : LAS_V08d_pre3_00.dz (1428092632 bytes)
1 : LGUP_8974.dll (1477632 bytes)
This shows me that there are two files inside the KDZ file: LAS_V08d_pre3_00.dz and LGUP_8974.dll
You can now extract them by ID by using the -s option, or by using -x to extract all of the files.
Code:
# python KDZFileTools.py -f LAS_V08d_pre3_00.kdz -x
[+] Extracting all partitions!
[+] Extracting LAS_V08d_pre3_00.dz to kdzextracted\LAS_V08d_pre3_00.dz
[+] Extracting LGUP_8974.dll to kdzextracted\LGUP_8974.dll
Now you'll see a folder called "kdzextracted" in your current working directory, which will contain the extracted files.
The next step would be to extract the DZ file to get the partitions it contains:
Code:
# python DZFileTools.py -f kdzextracted/LAS_V08d_pre3_00.dz -l
[+] DZ Partition List
=========================================
0 : PrimaryGPT_0.bin (4299 bytes)
1 : modem_32768.bin (25719664 bytes)
2 : sbl1_163840.bin (179443 bytes)
3 : dbi_165888.bin (10505 bytes)
4 : aboot_229376.bin (288082 bytes)
5 : rpm_231424.bin (93084 bytes)
6 : boot_262144.bin (8959565 bytes)
7 : tz_294912.bin (149388 bytes)
8 : persist_393216.bin (23621 bytes)
9 : recovery_458752.bin (10454494 bytes)
10 : laf_622592.bin (14244284 bytes)
11 : system_7176192.bin (66791740 bytes)
12 : system_7438336.bin (2651 bytes)
13 : system_7440008.bin (2313 bytes)
14 : system_7444120.bin (103727934 bytes)
15 : system_7704592.bin (114239263 bytes)
16 : system_7964296.bin (2313 bytes)
17 : system_7968408.bin (103349001 bytes)
18 : system_8228880.bin (121921125 bytes)
19 : system_8488584.bin (2313 bytes)
20 : system_8492696.bin (101078725 bytes)
21 : system_8753168.bin (125454806 bytes)
22 : system_9012872.bin (2313 bytes)
23 : system_9016984.bin (105806605 bytes)
24 : system_9277456.bin (115830981 bytes)
25 : system_9537160.bin (2313 bytes)
26 : system_9541272.bin (108458465 bytes)
27 : system_9801744.bin (83280847 bytes)
28 : system_10063888.bin (67940827 bytes)
29 : system_10326032.bin (91997923 bytes)
30 : system_10588176.bin (58015487 bytes)
31 : system_10846208.bin (2314 bytes)
32 : system_11108352.bin (2314 bytes)
33 : system_11370496.bin (2314 bytes)
34 : system_11632640.bin (2314 bytes)
35 : system_11894784.bin (2314 bytes)
36 : system_12156928.bin (2314 bytes)
37 : system_12419072.bin (2314 bytes)
38 : system_12681216.bin (2314 bytes)
39 : system_12943360.bin (2314 bytes)
40 : system_13205504.bin (2314 bytes)
41 : system_13467648.bin (2314 bytes)
42 : system_13729792.bin (2652 bytes)
43 : system_13731464.bin (2314 bytes)
44 : BackupGPT_61070336.bin (4286 bytes)
Excellent! All the files are there!
Large images are split up by LG, and can be combined with "cat" or something like that.
The filename actually is in the form "partname_offset.bin" where "offset" is the actual location that the file should be written to on the phone's eMMC (handy!)
You can substitute -l in the options for -x again to extract all the partitions to the folder "dzextracted" in the current working directory as well.
The option --out or -o will change the output directory, so it doesn't have to output to {kdz|dz}extracted
Where can I download this?
Code:
[URL="http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip"]http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip[/URL]
Please don't mirror this file, as having it in one location makes it easy for me to keep it up to date, or pull it if there are problems with this script
It doesn't work? Can you help me?!
Sure, join #lg-g2 on Freenode and ask for IOMonster and I'll try to help out as best I can.
I will not answer questions in PM or via email, please don't PM me about this script!
As usual, if this script eats your dog/cat/uncle please don't blame me either. I didn't mean to do it, I swear!
Whoo! Thanks! It works! How can I say thanks?
"Thanking" this thread is great!
If you feel that I absolutely saved you many hours of time, feel free to click on the "Donate to Me" button next to my username in this post.
This project was funded by a boring day at work and lots of caffiene
Alright guys, let me know if you've got any questions and I'll see what I can do!
Ha! Just what I was looking for but now to get it to work in Windows...
Thanks!
Edit
So I installed Python for Windows and this was the output...
C:\Python33>KDZFileTools.py -l -f F320K11D_00.kdz
File "C:\Python33\KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
C:\Python33>python.exe DZFileTools.py -l -f D80210C_00.kdz
File "DZFileTools.py", line 96
print "[!] Bad DZ sub header!"
^
SyntaxError: invalid syntax
C:\Python33>python.exe KDZFileTools.py -l -f D80210C_00.kdz
File "KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
AndroidUser00110001 said:
Ha! Just what I was looking for but now to get it to work in Windows...
Thanks!
Edit
So I installed Python for Windows and this was the output...
C:\Python33>KDZFileTools.py -l -f F320K11D_00.kdz
File "C:\Python33\KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
C:\Python33>python.exe DZFileTools.py -l -f D80210C_00.kdz
File "DZFileTools.py", line 96
print "[!] Bad DZ sub header!"
^
SyntaxError: invalid syntax
C:\Python33>python.exe KDZFileTools.py -l -f D80210C_00.kdz
File "KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
Click to expand...
Click to collapse
Not sure what's happening here, but I wrote this in Windows, so it works for me...
Try using Python 2.7 ? I haven't tested it with Py3k...
If it still gives you invalid headers after you get rid of the SyntaxError messages, let me know and I'll take a look. I've only verified that this works on a handful of KDZ files.
Thanks for such a awesome tool. After extracting D80210E_00.kdz, I found that system_xxxxxxx.bin files is not contiguous. Merge into one file, you can not mount it.
would you like to develop a tool more for system partitions merge?
honentan said:
Thanks for such a awesome tool. After extracting D80210E_00.kdz, I found that system_xxxxxxx.bin files is not contiguous. Merge into one file, you can not mount it.
would you like to develop a tool more for system partitions merge?
Click to expand...
Click to collapse
Excellent timing I'm actually working on that right now.
It would appear that the KDZ file contains only the 'sparse' data from large partitions.
Basically, it's the same as if you fed the utility "hexdump" a large file with multiple sections will have the empty sections 'collapsed' and shown as a "*".
My utility needs to take the beginning position of the first system_xxxx.bin file into account, and the end system_xxxx.bin 's position and size.
From there, it knows how big the image file needs to be, then it can simply seek to the position in the new file as defined by each .bin file and write it's contents there.
Ultimately, I'd like to have in-place extraction from the KDZ files, without the need to extract the DZ file itself (since it really is just copying the bytes from the right position to a new file on the hdd), and a utility to merge split partitions, like system.
Until then, if you need to create this image yourself, you can do so with a hex editor or dd
1. Take the first system_xxxxx.bin file and write down the value of xxxxx somewhere. We're going to use this as an offset.
2. Take the last system_xxxxx.bin file and write down it's xxxxx as well. Subtract the two. You should now get the size of the system.img file
3. Use "dd" to create a file of proper length: dd if=/dev/zero of=system.img bs=1 count=SIZE_FROM_EARLIER
4. Now, let's put our files into the right places. The first file can just be merged into it with dd if=system_xxxxx.bin of=system.img conv=notrunc
5. For every file after the first one, you'll have to take the xxxxx value from the file, and subtract the offset you found from step 1. use dd if=system_xxxxx.bin of=system.img seek=VALUE_YOU_JUST_FOUND
6. The end file is no different. Subtract it's xxxxx value again and run the above DD command.
If I'm right, this should leave you with a working partition image.
I haven't verified yet, but it seems logical and should work
thanks for your reply. I am going to try the method as you mentioned just now to merge the system.img using winhex, but I have not verified yet too. I will try it tomorrow.
as a advice, new tool should be able to extract and repack new kdz file.
honentan said:
Thanks for such a awesome tool. After extracting D80210E_00.kdz, I found that system_xxxxxxx.bin files is not contiguous. Merge into one file, you can not mount it.
would you like to develop a tool more for system partitions merge?
Click to expand...
Click to collapse
thecubed said:
Excellent timing I'm actually working on that right now.
It would appear that the KDZ file contains only the 'sparse' data from large partitions.
Basically, it's the same as if you fed the utility "hexdump" a large file with multiple sections will have the empty sections 'collapsed' and shown as a "*".
My utility needs to take the beginning position of the first system_xxxx.bin file into account, and the end system_xxxx.bin 's position and size.
From there, it knows how big the image file needs to be, then it can simply seek to the position in the new file as defined by each .bin file and write it's contents there.
Ultimately, I'd like to have in-place extraction from the KDZ files, without the need to extract the DZ file itself (since it really is just copying the bytes from the right position to a new file on the hdd), and a utility to merge split partitions, like system.
Until then, if you need to create this image yourself, you can do so with a hex editor or dd
1. Take the first system_xxxxx.bin file and write down the value of xxxxx somewhere. We're going to use this as an offset.
2. Take the last system_xxxxx.bin file and write down it's xxxxx as well. Subtract the two. You should now get the size of the system.img file
3. Use "dd" to create a file of proper length: dd if=/dev/zero of=system.img bs=1 count=SIZE_FROM_EARLIER
4. Now, let's put our files into the right places. The first file can just be merged into it with dd if=system_xxxxx.bin of=system.img conv=notrunc
5. For every file after the first one, you'll have to take the xxxxx value from the file, and subtract the offset you found from step 1. use dd if=system_xxxxx.bin of=system.img seek=VALUE_YOU_JUST_FOUND
6. The end file is no different. Subtract it's xxxxx value again and run the above DD command.
If I'm right, this should leave you with a working partition image.
I haven't verified yet, but it seems logical and should work
Click to expand...
Click to collapse
I merged system.img using script as you mentioned, but it couldn't be mounted also.
Progress was obtained, I got all files in system.img using ext2explore(version2.1).
Code:
dd if=/dev/zero of=system.img bs=1 count=5505024
dd if=system_819200.bin of=system.img conv=notrunc seek=0
dd if=system_1081344.bin of=system.img conv=notrunc seek=262144
dd if=system_1082760.bin of=system.img conv=notrunc seek=263560
dd if=system_1086808.bin of=system.img conv=notrunc seek=267608
dd if=system_1347536.bin of=system.img conv=notrunc seek=528336
dd if=system_1607048.bin of=system.img conv=notrunc seek=787848
dd if=system_1611096.bin of=system.img conv=notrunc seek=791896
dd if=system_1871824.bin of=system.img conv=notrunc seek=1052624
dd if=system_2131336.bin of=system.img conv=notrunc seek=1312136
dd if=system_2135384.bin of=system.img conv=notrunc seek=1316184
dd if=system_2396112.bin of=system.img conv=notrunc seek=1576912
dd if=system_2655624.bin of=system.img conv=notrunc seek=1836424
dd if=system_2659672.bin of=system.img conv=notrunc seek=1840472
dd if=system_2920400.bin of=system.img conv=notrunc seek=2101200
dd if=system_3179912.bin of=system.img conv=notrunc seek=2360712
dd if=system_3183960.bin of=system.img conv=notrunc seek=2364760
dd if=system_3444688.bin of=system.img conv=notrunc seek=2625488
dd if=system_3706832.bin of=system.img conv=notrunc seek=2887632
dd if=system_3968976.bin of=system.img conv=notrunc seek=3149776
dd if=system_4231120.bin of=system.img conv=notrunc seek=3411920
dd if=system_4493264.bin of=system.img conv=notrunc seek=3674064
dd if=system_4755408.bin of=system.img conv=notrunc seek=3936208
dd if=system_5017552.bin of=system.img conv=notrunc seek=4198352
dd if=system_5275648.bin of=system.img conv=notrunc seek=4456448
dd if=system_5537792.bin of=system.img conv=notrunc seek=4718592
dd if=system_5799936.bin of=system.img conv=notrunc seek=4980736
dd if=system_6062080.bin of=system.img conv=notrunc seek=5242880
dd if=system_6324224.bin of=system.img conv=notrunc seek=5505024
ext2explore log:
Partition Table Error on D:/Android-ROM/980VS/system.img
Invalid End of sector markerBlock size 4096, inp 8064, inodesize 256
Linux Partition found on disk 1 partition 0
Inode 727 with file size 0
thecubed said:
Not sure what's happening here, but I wrote this in Windows, so it works for me...
Try using Python 2.7 ? I haven't tested it with Py3k...
If it still gives you invalid headers after you get rid of the SyntaxError messages, let me know and I'll take a look. I've only verified that this works on a handful of KDZ files.
Click to expand...
Click to collapse
Thanks, I got it working using 2.7 I have everything extracted and now I need to learn how to merge all the system files.
Thanks!
AndroidUser00110001 said:
Thanks, I got it working using 2.7 I have everything extracted and now I need to learn how to merge all the system files.
Thanks!
Click to expand...
Click to collapse
I'm working on a modified version of the DZFileUtils.py script, since the DZ file actually contains the proper information to regenerate the full system.img easier than using DD or a hex editor.
Work has been a little crazy for the past few days, so I may not get a chance to work on the script until next week, but it won't be that hard to have this script put the system.img files back together.
I could modify the whole toolsets to do in-place extraction from a KDZ file (skipping the intermediate DZ file) but for now this is the easiest and accomplishes my goal the fastest (which is to allow extraction of bootloader stacks from KDZ files found on the interwebs)
I have no plans on creating a utility to generate KDZ files, since myself and @Shelnutt2 are in the process of writing a utility to flash images without needing any LG software on your PC.
I used dd for windows and was able to merge all the files with the script that someone posted. Sorry on XDA App and can't see full thread but thanks for the script.
If you guys need help with testing let me know.
Sent from my LG-D800 using XDA Premium 4 mobile app
thecubed said:
Hello everybody!
This will be my first XDA-wide release of my new utility for all LG phones.
What is this?
This is a utility to extract the new format KDZ files that LG distributes, specifically the 'compressed' ones.
LG frequently distributes firmware for phones as KDZ files, which are essentially a firmware image of the eMMC and a DLL file that is used by the downloader utility to communicate with the phone.
In the past, there were utilities to extract KDZ files to a DLL file and a DZ file, but no further (at least to my knowledge).
This utility lets you break the KDZ file into it's respective partitions (aboot, rpm, tz, and so on)
What good does this do me?
If you're an phone modder, haxxor, or just an enthusiast that has access to their phone's KDZ file and would like to have a copy of the actual partitions stored within, this will let you.
As an example, firmware for the new LG G2 on many device models is distributed as a KDZ file only.
Other phones use a TOT file, which is essentially a disk image of the eMMC with no compression.
If someone with a KDZ firmware-only phone wiped a partition (for example, modem) and wanted to get it back without flashing the whole phone all over again, they would be stuck.
TOT files are easily extractable, as there is software available currently for that but until today there was none (to my knowledge) for these new KDZ files.
How do I use this?
Glad you asked.
Inside the ZIP file you'll see two Python scripts, KDZFileTools.py and DZFileTools.py.
There's also a README.txt file for more in-depth information if you're curious.
Both scripts respond to --help or -h, so if you're even more curious, try that too!
KDZ files contain DZ files and DLL files, so the first step will be to split those into their respective parts:
LAS_V08d_pre3_00.kdz is the name of the KDZ file that I've copied to the working directory for this example.
Code:
# python KDZFileTools.py -l -f LAS_V08d_pre3_00.kdz
[+] KDZ Partition List
=========================================
0 : LAS_V08d_pre3_00.dz (1428092632 bytes)
1 : LGUP_8974.dll (1477632 bytes)
This shows me that there are two files inside the KDZ file: LAS_V08d_pre3_00.dz and LGUP_8974.dll
You can now extract them by ID by using the -s option, or by using -x to extract all of the files.
Code:
# python KDZFileTools.py -f LAS_V08d_pre3_00.kdz -x
[+] Extracting all partitions!
[+] Extracting LAS_V08d_pre3_00.dz to kdzextracted\LAS_V08d_pre3_00.dz
[+] Extracting LGUP_8974.dll to kdzextracted\LGUP_8974.dll
Now you'll see a folder called "kdzextracted" in your current working directory, which will contain the extracted files.
The next step would be to extract the DZ file to get the partitions it contains:
Code:
# python DZFileTools.py -f kdzextracted/LAS_V08d_pre3_00.dz -l
[+] DZ Partition List
=========================================
0 : PrimaryGPT_0.bin (4299 bytes)
1 : modem_32768.bin (25719664 bytes)
2 : sbl1_163840.bin (179443 bytes)
3 : dbi_165888.bin (10505 bytes)
4 : aboot_229376.bin (288082 bytes)
5 : rpm_231424.bin (93084 bytes)
6 : boot_262144.bin (8959565 bytes)
7 : tz_294912.bin (149388 bytes)
8 : persist_393216.bin (23621 bytes)
9 : recovery_458752.bin (10454494 bytes)
10 : laf_622592.bin (14244284 bytes)
11 : system_7176192.bin (66791740 bytes)
12 : system_7438336.bin (2651 bytes)
13 : system_7440008.bin (2313 bytes)
14 : system_7444120.bin (103727934 bytes)
15 : system_7704592.bin (114239263 bytes)
16 : system_7964296.bin (2313 bytes)
17 : system_7968408.bin (103349001 bytes)
18 : system_8228880.bin (121921125 bytes)
19 : system_8488584.bin (2313 bytes)
20 : system_8492696.bin (101078725 bytes)
21 : system_8753168.bin (125454806 bytes)
22 : system_9012872.bin (2313 bytes)
23 : system_9016984.bin (105806605 bytes)
24 : system_9277456.bin (115830981 bytes)
25 : system_9537160.bin (2313 bytes)
26 : system_9541272.bin (108458465 bytes)
27 : system_9801744.bin (83280847 bytes)
28 : system_10063888.bin (67940827 bytes)
29 : system_10326032.bin (91997923 bytes)
30 : system_10588176.bin (58015487 bytes)
31 : system_10846208.bin (2314 bytes)
32 : system_11108352.bin (2314 bytes)
33 : system_11370496.bin (2314 bytes)
34 : system_11632640.bin (2314 bytes)
35 : system_11894784.bin (2314 bytes)
36 : system_12156928.bin (2314 bytes)
37 : system_12419072.bin (2314 bytes)
38 : system_12681216.bin (2314 bytes)
39 : system_12943360.bin (2314 bytes)
40 : system_13205504.bin (2314 bytes)
41 : system_13467648.bin (2314 bytes)
42 : system_13729792.bin (2652 bytes)
43 : system_13731464.bin (2314 bytes)
44 : BackupGPT_61070336.bin (4286 bytes)
Excellent! All the files are there!
Large images are split up by LG, and can be combined with "cat" or something like that.
The filename actually is in the form "partname_offset.bin" where "offset" is the actual location that the file should be written to on the phone's eMMC (handy!)
You can substitute -l in the options for -x again to extract all the partitions to the folder "dzextracted" in the current working directory as well.
The option --out or -o will change the output directory, so it doesn't have to output to {kdz|dz}extracted
Where can I download this?
Code:
[URL="http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip"]http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip[/URL]
Please don't mirror this file, as having it in one location makes it easy for me to keep it up to date, or pull it if there are problems with this script
It doesn't work? Can you help me?!
Sure, join #lg-g2 on Freenode and ask for IOMonster and I'll try to help out as best I can.
I will not answer questions in PM or via email, please don't PM me about this script!
As usual, if this script eats your dog/cat/uncle please don't blame me either. I didn't mean to do it, I swear!
Whoo! Thanks! It works! How can I say thanks?
"Thanking" this thread is great!
If you feel that I absolutely saved you many hours of time, feel free to click on the "Donate to Me" button next to my username in this post.
This project was funded by a boring day at work and lots of caffiene
Alright guys, let me know if you've got any questions and I'll see what I can do!
Click to expand...
Click to collapse
Hi, i make my phone soft bricked. no other method works. i need bin file to flash my phone.i have the orignal kdz file.how can i make a single bin file to flash it using LGNPST? If you want to read my whole phone bricking story, please follow this
http://forum.xda-developers.com/showthread.php?t=2492105
AAAAwesome!!!
Thank you very much!
Able to customize LG's ROM easier than ever!
thecubed said:
I'm working on a modified version of the DZFileUtils.py script, since the DZ file actually contains the proper information to regenerate the full system.img easier than using DD or a hex editor.
Work has been a little crazy for the past few days, so I may not get a chance to work on the script until next week, but it won't be that hard to have this script put the system.img files back together.
I could modify the whole toolsets to do in-place extraction from a KDZ file (skipping the intermediate DZ file) but for now this is the easiest and accomplishes my goal the fastest (which is to allow extraction of bootloader stacks from KDZ files found on the interwebs)
I have no plans on creating a utility to generate KDZ files, since myself and @Shelnutt2 are in the process of writing a utility to flash images without needing any LG software on your PC.
Click to expand...
Click to collapse
Any word on the DZfileUtils.py script? I am trying to convert the dz to the system img and all I get is errors when trying to use dd. If there is a better way than dd for windows could someone let me know.
Well I think I have found where my error was at with what I was trying to do.
Thank you for all your hard work on this
honentan said:
I merged system.img using script as you mentioned, but it couldn't be mounted also.
Progress was obtained, I got all files in system.img using ext2explore(version2.1).
ext2explore log:
Partition Table Error on D:/Android-ROM/980VS/system.img
Invalid End of sector markerBlock size 4096, inp 8064, inodesize 256
Linux Partition found on disk 1 partition 0
Inode 727 with file size 0
Click to expand...
Click to collapse
I can confirm I could not mount or use normal methods I have used in the past for getting the system image extracted but I also used ext2explore on windows and it worked like a charm. Thank you
Thanks for tools:good:
but I got this error when I tried with KDZ file.
C:\Users\VM\Desktop\Compressed_KDZUtils>KDZFileTools.py -f L01E20A_00.kdz -x
[!] Error: Unsupported KDZ file format.
[ ] Expected: 0x28 0x5 0x0 0x0 0x34 0x31 0x25 0x80 ,
but received 0x31 0x23 0x53 0x7f 0x89 0x1e 0xe6 0x9b .
Click to expand...
Click to collapse
and when I got DZ file from other KDZ extracter and tried DZ file I got this
C:\Users\VM\Desktop\Compressed_KDZUtils>DZFileTools.py -f L01E20A_00.dz -l
[!] Error: Unsupported DZ file format.
[ ] Expected: 0x32 0x96 0x18 0x74 ,
but received 0x22 0x12 0x84 0x19 .
Traceback (most recent call last):
File "C:\Users\VM\Desktop\Compressed_KDZUtils\DZFileTools.py", line 212, in
<module>
dztools.main()
File "C:\Users\VM\Desktop\Compressed_KDZUtils\DZFileTools.py", line 196, in
main
self.openFile(args.dzfile)
File "C:\Users\VM\Desktop\Compressed_KDZUtils\DZFileTools.py", line 173, in
openFile
sys.exit(0)
NameError: global name 'sys' is not defined
Click to expand...
Click to collapse
I never run python script but I think I did it right.
I'm installed python 2.7. and KDZ file is for Optimus G
rquiett said:
I can confirm I could not mount or use normal methods I have used in the past for getting the system image extracted but I also used ext2explore on windows and it worked like a charm. Thank you
Click to expand...
Click to collapse
honentan said:
I merged system.img using script as you mentioned, but it couldn't be mounted also.
Progress was obtained, I got all files in system.img using ext2explore(version2.1).
Click to expand...
Click to collapse
in linux
Code:
mkdir -p /mnt/dsk
mount -o loop system.img /mnt/dsk
this will return an error
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
Click to expand...
Click to collapse
run dmesg
Code:
dmesg | tail
there will be a line like this:
EXT4-fs (loop0): bad geometry: block count 851968 exceeds size of device (819537 blocks)
Click to expand...
Click to collapse
now truncate the file with the value of the first number
Code:
truncate -o -s 851968
remount, and hey presto.
thank you @Flinny for this
edit: not needed anymore, just ignore it. keeping it here for reference though
merging the bins on linux
ok, so there have been a few issues with getting this done on any platform. I was asked to try to get into a kdz for someone, so I did it
included below is a python script that takes advantage of dd. I've set it up for the system partition (as that's the only one that is split)
I have left all my comments and tweaks in so you can see how it was formed
to run:
Code:
python SystemMerger.py
It should run with python3 (untested). can someone please report any errors and I'll see if I can fix them up.
after running the script, just mount the image, and you're away
Happy Hacking
ps, don't forget to change the extension to .py
ok final version here, should support windows now
https://github.com/cybojenix/random-scripts/blob/master/SystemMerger.py
it should also support python3 still, but someone please check, same with windows (though I'm sceptical about the way I write zeros. might not work on winblows...)
anyway, happy hacking again
cybojenix said:
ok final version here, should support windows now
https://github.com/cybojenix/random-scripts/blob/master/SystemMerger.py
it should also support python3 still, but someone please check, same with windows (though I'm sceptical about the way I write zeros. might not work on winblows...)
anyway, happy hacking again
Click to expand...
Click to collapse
good job! awesome! thanks.
AndroidUser00110001 said:
Ha! Just what I was looking for but now to get it to work in Windows...
Thanks!
Edit
So I installed Python for Windows and this was the output...
C:\Python33>KDZFileTools.py -l -f F320K11D_00.kdz
File "C:\Python33\KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
C:\Python33>python.exe DZFileTools.py -l -f D80210C_00.kdz
File "DZFileTools.py", line 96
print "[!] Bad DZ sub header!"
^
SyntaxError: invalid syntax
C:\Python33>python.exe KDZFileTools.py -l -f D80210C_00.kdz
File "KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
Click to expand...
Click to collapse
just put bracket on each print params, then it should work
like this print ("[!] Error: Unsupported KDZ file format.")
also the KDZFileTools not work on LG G2, the header doesnt match with required
here
i debug to find what is the actual value on verify_header
D80210b.kdz header found
>>> verify_header
b'(\x05\x00\x0041%\x80'
kdz_header = '0x28 0x5 0x0 0x0 0x34 0x31 0x25 0x80'
KDZFileTools error log
[!] Error: Unsupported KDZ file format.
Traceback (most recent call last):
File "C:\Downloads\KDZFileTools.py", line 197, in <module>
kdztools.main()
File "C:\Downloads\KDZFileTools.py", line 182, in main
self.openFile(args.kdzfile)
File "C:\Downloads\KDZFileTools.py", line 160, in openFile
print ("[ ] Expected: %s ,\n\tbut received %s ." % (" ".join(hex(ord) for n in self.kdz_header), " ".join(hex(ord) for n in verify_header)))
File "C:\Downloads\KDZFileTools.py", line 160, in <genexpr>
print ("[ ] Expected: %s ,\n\tbut received %s ." % (" ".join(hex(ord) for n in self.kdz_header), " ".join(hex(ord) for n in verify_header)))
TypeError: ord() expected string of length 1, but int found
Click to expand...
Click to collapse