Database Info

Automated NVFlash backup for Linux

Inspired by http://forum.xda-developers.com/showthread.php?t=1010659
I set out to make a similar script for Linux. You still need nvflash installed and working. Nothing is changed on the tablet, but your mileage may vary, past performance is not an indicator of future results, and void where prohibited by law
Code:
#!/bin/bash
### Version 1.0 2 April 2011, wd5gnr
### Usage: dobackup
### Puts backup files $ROOT/backup/YYYYMMDDHHMM/
### by default
### Edit the variables below to set things up for your setup
###
### If you already have the bootloader running
### You can use dobackup -r
### To reconnect
# note: for some reason partition 6 and 11 on my device
# fail to read properly. 6 is the MSC partition on mine
# if I reformat it and reload it from stock, its ok. But
# something during running is blowing it again
# Partition 11 apparently is supposed to fail
# based on comments in the script
# from: http://forum.xda-developers.com/showthread.php?t=1010659
# because of that you may want to try the full PARTLIST
# and then edit to suit your situation
#PARTLIST=2 3 4 5 6 7 8 9 10 11
PARTLIST="2 3 4 5 7 8 9 10 11"
# Set the path for backups etc.
ROOT=~/android/nvflash
# Set the path for NVFLASH
NVFLASH=$ROOT/nvflash
# and bootloader
NVBOOT=$ROOT/bootloader.bin
# This gets the date in YYYYMMDDHHMM format
BSTAMP=`date +%Y%m%d%H%M`
# This picks the name of the backup (using the date)
BDIR=$ROOT/backup/$BSTAMP
# make directories
mkdir -p $BDIR
# I did not check to see that this succeeded
# we assume you have permission etc.
# if -r option we never have to load the bootloader
RESUME="--bl $NVBOOT"
if [ "$1" == '-r' ]
then RESUME=-r
fi
function donvf {
if $NVFLASH [email protected]
then true
else
echo NVFlash error 1>&2
rmdir $BDIR >/dev/null 2>&1 # delete an empty backup directory
exit 1
fi
}
# Get BCT first and prime the boot loader
donvf $RESUME --getbct --bct $BDIR/backup.bct --go
# get partition table
donvf -r --getpartitiontable $BDIR/parttable.txt
# for some reason #1 is not required or absent and #12 is the cache
# but you could add it if necessary (look at parttable.txt)
# Part 11 may give an error, but is ok
# If 11 gives error APX will be down and you'll have to reboot to
# do any more things
for I in $PARTLIST
do
if [ $I == 11 ]
then
echo You may safely ignore errors on partition 11 apparently 1>&2
fi
donvf -r --read $I $BDIR/part$I.img
done
exit 0 # because of partition 11, you won't get here unless you aren't doing 11
###
### Note BCT is hardware info (4080 bytes)
### Partition guesses
# 2 = BCT = ?
# 3 = PT = ?
# 4 = EBT = ?
# 5 = MBT = ?
# 6 = MSC = ? Some kind of recovery thing; 7 in stock
# 7 = BLO = Boot logo (6 in stock)
# 8 = OGO
# 9 = SOS = Recovery image (clockwork or stock)
# 10 = LNX = Linux?
# 11 = APP = Applications?
# 12 = CAC = Cache
Funny things are partition 6 (my MSC partition). I can reformat it and flash it with stock and it works. But later it will hang nvflash trying to read it. I can't find any info on what is in there anyway.
Also partition 11 gives an error, but apparently its not just me. See comments in the script.

Can you try running dmesg (e.g. via adb shell or using a terminal on the gtab), right after booting, then looking at the outout for bad block msgs?
I've been guessing that the reason is because of bad blocks...
Jim

MSC partition is actually the /misc folder, you can verify this by comparing content with the file inanop.log
inandop.sh create it, it is designed to make sure the partition is splited in 2 section 2/16 GB and run in init.rc
could be the reason people trying to partition 4GB/12 where having problem...
@ Jim, Seem like you are right, bad sector are really the source of read error in my unit...
I am not sure if we will encounter issue when writing from the OS since this is handled byt the actual file system but writing directly from nvflash seem to be bypassing any kinf of error/wear levelling provided by yaffs file system or even fat32
I would think that writing with the -v (Verify on) would be better to prevent write error but that's a feeling not based on fact (you need to disable this flag to restore a partition with bad block)
I also believe that bad block at the end of the partition 11 are not a concern, they are probably normal for this kind of inand memory... mine appear at 209.584Mb out of 209.715MB so the partition is still 209MB

Nothing in /misc looks exotic -- just a log file and a lost+found. Very strange that it hangs the nvflash read. I noticed it the other day and reformatted it from nvflash and rolled back the stock image and then I was able to read it back. But today it hangs my backup script again.
Oh yeah. And my dmesg log shows Falled Ddk Rd. Bad block error code=0xb0010 at chip=0 block=65. That repeats a few times. Not unique to me however: http://forum.xda-developers.com/showthread.php?t=933128

wd5gnr said:
Nothing in /misc looks exotic -- just a log file and a lost+found. Very strange that it hangs the nvflash read. I noticed it the other day and reformatted it from nvflash and rolled back the stock image and then I was able to read it back. But today it hangs my backup script again.
Oh yeah. And my dmesg log shows Falled Ddk Rd. Bad block error code=0xb0010 at chip=0 block=65. That repeats a few times. Not unique to me however: http://forum.xda-developers.com/showthread.php?t=933128
Click to expand...
Click to collapse
Hi,
Thanks for that info. If you look thru the dmesg, there'll be a listing of the partitions that the kernel is "creating" (I think, more like "assigning"), and I think that by looking at that, relative to the bad block msgs that will be interspersed, you can tell which partition (e.g., on my gtab, one of them is in /system) the bad block is occurring in.
Jim

P00r said:
MSC partition is actually the /misc folder, you can verify this by comparing content with the file inanop.log
inandop.sh create it, it is designed to make sure the partition is splited in 2 section 2/16 GB and run in init.rc
could be the reason people trying to partition 4GB/12 where having problem...
@ Jim, Seem like you are right, bad sector are really the source of read error in my unit...
I am not sure if we will encounter issue when writing from the OS since this is handled byt the actual file system but writing directly from nvflash seem to be bypassing any kinf of error/wear levelling provided by yaffs file system or even fat32
I would think that writing with the -v (Verify on) would be better to prevent write error but that's a feeling not based on fact (you need to disable this flag to restore a partition with bad block)
I also believe that bad block at the end of the partition 11 are not a concern, they are probably normal for this kind of inand memory... mine appear at 209.584Mb out of 209.715MB so the partition is still 209MB
Click to expand...
Click to collapse
Hi,
A couple of comments:
- The problem that I think may occur if you use the -v is that it may prevent the download of the partition to be completed. In other words, if you do --download with -v, and if nvflash hits a verify error in the middle of the partition, it may (I don't know) just stop at that point, then you end up with a half-populated partition, which could be pretty useless.
- Thanks for the info re. part 11, which I believe is "CAC", or "cache". I wonderinf if\ that partition really needs to be restored for a complete system restore?
BTW, what got me started with all of this was that way-back-when, when I first started working with my gtab, I wanted to find a way to do what I call a "complete bare-metal" backup/image of my gtab. That got me started working with nvflash, but then I ran into the --read problems.
As mentioned, one of the bad blocks on my gtab is in cache partition, which may or not needed for a full restore, but the worse thing is that the 2nd bad block that I have is apparently near the end of the system (/system) partition, so that prevents me from doing a successful read of the system partition to back it up.
I've tried a number of things, including doing repartitioning system using CWM, and also some messing with the system partition via adb while in CWM, i.e., I deleted the system partition completely, and the restored it. From what I've read, yaffs partitions don't need to be pre-formatted (don't need to be formatted before being used), so I was hoping (had seem some posts in some other forums) that that might eliminate the bad blocks if they were "soft", but it didn't.
So, I'm still stuck with 2 bad blocks on my gtab, and can't complete a full nvflash image of all the partitions on it ...
BTW, I just got a new gtab via woot, and I ran dmesg on it, and that too has 1 bad block, so it seems like probably all devices have some... Just some info...
Jim

wd5gnr,
So, I guess, at this point, is who is brave enough to try to do a full restore from the partition files that they pulled from their gtab (albeit with read errors)?
Have you tried that yet?
Just to be clear, I have NOT, mainly because of the read errors that I encountered when trying to backup, i.e., if I can't read the partitions, then I have doubts about the integrity of the files that I ended up with.
Jim

I have not. Partition 11 is the system partition, not the cache (at least on mine). On mine 12 is the cache.
If you look at the "inspired by" link above they mention that the image is still usable which implies there is slack space at the end maybe? But I don't know why/how they know that. And no, I have not reflashed my whole system from these backups although I have done a few on partition basis.

wd5gnr said:
I have not. Partition 11 is the system partition, not the cache (at least on mine). On mine 12 is the cache.
If you look at the "inspired by" link above they mention that the image is still usable which implies there is slack space at the end maybe? But I don't know why/how they know that. And no, I have not reflashed my whole system from these backups although I have done a few on partition basis.
Click to expand...
Click to collapse
I was in that thread you linked, and re-read it, and maybe I missed something, but I don't see where it says the image is still usable?
Jim

You have to open up the batch file and read the comments in the code.
Code:
REM If you get error in partition #11 keep it anyway it is still usable
REM this seem to be a normal "feature" of the nand chips used in the tablet...

Ok, I see where you found that comment, in the zip file.
Jim

Hi,
I've been doing some more detailed analysis of my dmesg.
First of all, here's snippets with info from dmesg, showing:
1) kernel boot parameters, with partition sizes
2) status of MTD partitions
mtdparts=tegra_nand:
[email protected](misc),
[email protected](recovery),
[email protected](boot),
[email protected](system),
[email protected](cache),
[email protected](bootbmp),
[email protected](logodata)
<5>[ 7.086731] 7 cmdlinepart partitions found on MTD device tegra_nand
<5>[ 7.092987] Creating 7 MTD partitions on "tegra_nand":
<5>[ 7.098130] 0x000000740000-0x000001740000 : "misc"
<5>[ 7.103663] 0x000003ce0000-0x000004ce0000 : "recovery"
<5>[ 7.109346] 0x000004d60000-0x000005d60000 : "boot"
<5>[ 7.114670] 0x000005de0000-0x0000125e0000 : "system"
<6>[ 7.119773] tegra_nand tegra_nand: Block 0x43b is bad [chip=0,offset=0x8760000]
<5>[ 7.127928] 0x000012680000-0x00001ff80000 : "cache"
<5>[ 7.133741] 0x0000017c0000-0x000001bc0000 : "bootbmp"
<5>[ 7.139340] 0x000001c40000-0x000003c40000 : "logodata"
<6>[ 7.144567] tegra_nand tegra_nand: Block 0x186 is bad [chip=0,offset=0x30c0000]
When I do nvflash --read on all the partitions, I get read errors on partition 8 and partition 11, so I tried to figure out which partition those are, from the dmesg msgs and the sizes shown:
Partition 8
Name=OGO
DeviceId=19
StartSector=12928
NumSectors=16384
BytesPerSector=2048
16834 x 2048 = 34476032 BYTES = 33,668 KBYTES ==>
ALSO:
0x000001c40000-0x000003c40000 = 2000000 = 33554432 BYTES = 32,768 KBYTES ==> THIS IS THE 'logodata' partition
[See: http://forum.xda-developers.com/archive/index.php/t-859884-p-2.html,
"Additionally, the boot animation is left over from another mod. To remove it and see the Vegan boot animation, go into /logodata and delete everything in it."]
So, it appears this bad block is occurring in the 'logodata' partition, which apparently has something to do with the boot animation.
Partition 11
Name=APP
DeviceId=19
StartSector=45696
NumSectors=102400
BytesPerSector=2048
102400 x 2048 = 209715200 BYTES = 204,800 KBYTES ==> THIS IS 'system' partition
ALSO:
0x000005de0000-0x0000125e0000 = C800000 = 209715200 ??
So, it appears this bad block is occurring in the 'system' partition, which I believe is /system.
I'm particularly concerned about the one in the 'system' partition, i.e., probably if I pull that partition using nvflash, and then later try to restore it, /system would be bad/corrupted?
Jim

TF700T complete flash layout

I spent some time in analyzing of flash layout. The comprehensive description below attempts to map each byte of the flash and describes way to extract it.
I would be glad if somebody could provide more detailed info about bootloader, signatures, DRM etc.
Patches are welcome.
Code:
mmcblk0 layout
All dumps were done on Asus Eee Pad Transformer Infinity TF700T, 64GB version, firmware 9.4.5.26, locked
mmcblk0 off-partition section
Offset: 0 (0x0)
Size: 38273024 (0x2480000)
Read command: busybox dd if=/dev/block/mmcblk0 of=/mnt/sdcard/mmcblk0pre1.img bs=524288 count=73
Offset: 0 (0x0)
Size: 3670016 (0x380000)
Contains: Zeroes
Purpose: Unknown
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s1.img bs=3670016 count=1
Process command: tr -d '\0' <mmcblk0pre1s1.img >mmcblk0pre1s1nz.img # mmcblk0pre1s1nz.img must be empty file
Offset: 3670016 (0x380000)
Contains: Recovery kernel image followed by zeroes
Size: 8388608 (0x800000)
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s2.img bs=524288 skip=7 count=16
Process commands:
perl split_bootimg.pl mmcblk0pre1s2.img
mkdir mmcblk0pre1s2.img-ramdisk
cd mmcblk0pre1s2.img-ramdisk
zcat ../mmcblk0pre1s2.img-ramdisk.gz | cpio -i
cd ..
# end Process commands
Offset: 12058624 (0xb80000)
Contains: Regular boot kernel image followed by zeroes
Size: 8388608 (0x800000)
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s3.img bs=524288 skip=23 count=16
Process commands:
perl split_bootimg.pl mmcblk0pre1s3.img
mkdir mmcblk0pre1s3.img-ramdisk
cd mmcblk0pre1s3.img-ramdisk
zcat ../mmcblk0pre1s3.img-ramdisk.gz | cpio -i
cd ..
# end Process commands
Offset: 20447232 (0x1380000)
Contains: Block of 16 bytes followed by 0x2de0 hexadecimal numbers followed by FF
Size: 12288 (0x3000)
Extract command: dd if=mmcblk0pre1.img of=mmcblk0pre1s4.img bs=524288 skip=39
Vital data:
Extract command: dd if=mmcblk0pre1s4.img of=mmcblk0pre1s4ss2.img bs=4096 skip=3
Binary part of vital data:
Extract command: dd if=mmcblk0pre1s4ss1.img of=mmcblk0pre1s4ss1ch1.img bs=16 count=1
Hexadecimal part of vital data:
Extract command: dd if=mmcblk0pre1s4ss1.img of=mmcblk0pre1s4ss1ch2.img bs=16 count=734 skip=1
Process command: unhex <mmcblk0pre1s4ss1ch2.img >mmcblk0pre1s4ss1ch2bin.img
FF part of vital data:
Extract command: dd if=mmcblk0pre1s4ss1.img of=mmcblk0pre1s4ss1ch3.img bs=16 skip=735
Process command: tr -d '\377' <mmcblk0pre1s4ss1ch3.img >mmcblk0pre1s4ss1ch3nff.img # mmcblk0pre1s4ss1ch3nff.img must be empty file
Zeroes:
Extract command: dd if=mmcblk0pre1s4.img of=mmcblk0pre1s4ss1.img bs=4096 count=3
Process command: tr -d '\0' <mmcblk0pre1s4ss2.img >mmcblk0pre1s4ss2nz.img # mmcblk0pre1s4ss2nz.img must be empty file
Purpose: Probably encrypted bootloader
mmcblk0p1
Offset: 38273024 (0x2480000)
Size: 805306368 (0x30000000)
File system size: 196608 * 4096 = 805306368 (fully occupies partition)
Format: Linux ext4 filesystem
Mounted at: /system
Mount options: read only, extended attributes, ACL
Permissions: only root can manipulate
Contains: Base system and embedded applications
Purpose: Base system
mmcblk0p2
Offset: 843579392 (0x32480000)
Size: 448790528 (0x1ac00000)
File system size: 109568 * 4096 = 448790528 (fully occupies partition)
Format: Linux ext4 filesystem
Mounted at: /cache
Mount options: read/write, no SUID, no device nodes, no atime
Permissions: only root can manipulate, UID system and GID cache can read and write
Contains: Cache
Purpose: Application cache
Note: The volume has the same UUID as mmcblk0p1
mmcblk0p3
Offset: 1292369920 (0x4d080000)
Size: 2097152 (0x200000)
File system size: 512 * 4096 = 2097152 (fully occupies partition)
Linux rev 1.0 ext3 filesystem
Not mounted
Permissions: GID system can manipulate
Contains: Empty file system
Purpose: Recovery /misc
Referenced by: /system/lib/libandroid_runtime.so recovery ramdisk: /etc/recovery.fstab
Note: File system is referenced in recovery as emmc, not ext3!
mmcblk0p4
Offset: 1294467072 (0x4d280000)
Size: 855638016 (0x33000000)
File system size: 208896 * 4096 = 855638016
Linux rev 1.0 ext3 filesystem
Not mounted
Permissions: GID system can manipulate
Contains: Empty file system
Purpose: Recovery /staging
Referenced by: recovery ramdisk: init.rc /etc/recovery.fstab
mmcblk0p5
Offset: 2150105088 (0x80280000)
Size: 5242880 (0x500000)
File system size: 5092 * 1024 = 5147488
Format: FAT32 file system, no partition table, MS-DOS "Non-system disk" boot block
Not mounted
Permissions: only root can manipulate
Contains: File system with files:
Serial numbers (ISN, PPID, SSN, UUID)
Calibration data (AL3010 light sensor, AMI304 magnetic sensor, KXTF9 motion sensor)
Purpose: Device specific unique system data, mounted as /btmac during Android boot
Referenced by: /system/bin/wifimacwriter /system/bin/brcm_patchram_plus /system/bin/sensors-config /system/bin/sixpair ramdisk: /init recovery ramdisk: /etc/recovery.fstab /init
mmcblk0p5 off file-system area
Offset in section: 5147488 (0x4e8b60)
Size: 28672 (0x7000)
Read command: busybox dd if=/dev/block/mmcblk0p5 of=/mnt/sdcard/mmcblk0p5s2.img bs=1024 skip=5092
Process command: tr -d '\0' <mmcblk0p5s2.img >mmcblk0p5s2nz.img # mmcblk0p5s2nz.img must be empty file
mmcblk0p6
Offset: 2155347968 (0x80780000)
Size: 524288 (0x80000)
Format: binary data
Permissions: UID drm can manipulate
Contains: 208 bytes of binary data, the rest are zeroes
Purpose: DRM, probably contains encrypted DRM key
Referenced by: /system/bin/wvdrmserver /system/vendor/lib/drm/libdrmwvmplugin.so
mmcblk0p7
Offset: 2155872256 (0x80800000)
Size: 5242880 (0x500000)
Format: empty
Contains: Zeroes
Purpose: Unknown
mmcblk0p8
Offset: 2161115136 (0x80d00000)
Size: 61415620608 (0xe4ca80000)
File system size: 14994040 * 4096 = 61415587840
Format: Linux ext4 filesystem
Mounted at: /data
Mount options: read/write, no SUID, no device nodes, no atime
Permissions: only root can manipulate, read and write are directory specific
Contains: User applications, user data, and virtual internal SD card
Note: /data/media is mounted via UID/GID stripping FUSE as /mnt/sdcard
mmcblk0p8 off file-system area
Offset in section: 61415587840 (0xe4ca78000)
Size: 32768 (0x8000)
Read command: busybox dd if=/dev/block/mmcblk0p8 of=/mnt/sdcard/mmcblk0p8s2.img bs=4096 skip=14994040
mmcblk0 off-partition section
Offset: 63576735744 (0xecd780000)
Size: 524288 (0x80000)
Read command: busybox dd if=/dev/block/mmcblk0 of=/mnt/sdcard/mmcblk0post8.img bs=524288 skip=121263
Process command: tr -d '\0' <mmcblk0p8s2.img >mmcblk0p8s2nz.img # mmcblk0p8s2nz.img must be empty file
Offset: 63576735744 (0xecd780000)
Offset in section: 0 (0x0)
Size: 507392 (0x7be00)
Contains: Zeroes
Purpose: Unknown
Extract command: dd if=mmcblk0post8.img of=mmcblk0post8s1.img bs=507392 count=1
Process command: tr -d '\0' <mmcblk0post8s1.img >mmcblk0post8s1nz.img # mmcblk0post8s1nz.img must be empty file
Offset: 63577243136 (0xecd7fbe00)
Offset in section: 507392 (0x7be00)
Size: 16896 (0x4200)
Contains: EFI Partition table (partition names: APP, CAC, MSC, USP, PER, YTU, CRA, UDA)
Extract command: dd if=mmcblk0post8.img of=mmcblk0post8s2.img bs=512 skip=991
Purpose: Partition table
Total size of mmcblk0: 63577260032 (0xecd800000)
Notes:
can manipulate = can read, write partition vital data, only root can mount
can read, write = can read, write partition file system contents
Read commands are ran on the Transformer
Extract and process commands are run anywhere, with pre-read image file in the current directory.
You need dd with large files support. Vanilla dd on TF700T does not support large files. Busybox dd does.
Dropbox link to Asus_Transformer_Infinity_TF700T/flash_layout.txt

Wow, thanks for this detailed analysis - much more detailed than mine.
So what can I add to your research?
Tegra-based systems have another partition table, which has a proprietary layout and an unknown purpose (maybe just important for NVFlash and for flashing blobs?). Looking at the flash.cfg in the NVFlash package from AndroidRoot.mobi, we can get the Tegra partition layout and partition names:
Partition number 1 is missing in the list, maybe it contains the extremely well-hidden APX mode recovery code or even the answer to life, the universe and everything.
The following 3 partitions are located at the beginning of mmcblk0 and their contents are apparently encrypted with a device-specific key. For some reason, with ICS-based ROMs it reads as all zeros; in JB-based ROMs additional mmcblk0boot0 and mmcblk0boot1 partitions appear which together cover this area. The "bricksafe.img" in the nvflash guide covers these 3 partitions.
2 BCT: Tegra Boot Configuration Table - 3145728 bytes
3 PT: Tegra Partition Table - 524288 bytes
4 EBT: Bootloader - 8388608 bytes
You already know the following 2:
5 SOS: Recovery kernel - 8388608 bytes
6 LNX: Linux kernel - 8388608 bytes
Then some more funny ones:
7 CER: I think this stands for "Certificate" and contains the bootloader unlock token. - 8388608 bytes. If I calculated correctly, this is at 0x1380000 into mmcblk0. Saved as "unlock-token.img" in the nvflash guide.
8 IMG: no idea what this is for - 8388608 bytes
9 GP1: space for a GPT partition table, maybe unused - 1048576 bytes
Now the regular partitions follow (p1 to p8):
10 APP: p1 = /system (Android OS)
11 CAC: p2 = /cache (for communication between Android and recovery)
12 MSC: p3 ="misc", whatever that is. On the TF101 it was used for bootloader commands.
13 USP: p4 = The update staging partition. Update blobs are copied here and flashed to the correct partition by the bootloader.
14 PER: p5 = device-specific config in a FAT filesystem
15 YTU: p6 = Apparently the DRM key. Confirmed to be overwritten with 0 by the unlocking process.
16 CRA: p7 = unknown (reserved for crash dumps?)
17 UDA: p8 = /data (Android user data)
And finally:
18 GPT: the EFI partition table that is actually used by the kernel

Well, it seems, that something (ICS stock kernel, hardware) hides contents of the first (at most) 0x380000 bytes of flash.
I am locked, and I have some token at 0x1380000 as well.
I am still thinking about a way to unlock, keep access to nvflash, and upgrade to JB keeping DRM working, even at cost of using stock system. That is why I wanted to backup and analyze everything and find all keys and signatures.
It would be also nice to know, whether there are areas of flash with hardware or kernel write lock.

utx said:
Well, it seems, that something (ICS stock kernel, hardware) hides contents of the first (at most) 0x380000 bytes of flash.
I am locked, and I have some token at 0x1380000 as well.
Click to expand...
Click to collapse
Yes, before unlocking I had something very similar to you there - a 16 byte header followed by some hexdump. I don't know what it was. It was overwritten by the unlock process with a 4 byte data block prefixed with a "-SIGNED-BY-SIGNBLOB-" header and followed by 256 bytes of what looks like a digital signature, very similar to the signed update blobs.
utx said:
I am still thinking about a way to unlock, keep access to nvflash, and upgrade to JB keeping DRM working, even at cost of using stock system. That is why I wanted to backup and analyze everything and find all keys and signatures.
Click to expand...
Click to collapse
Definitely back up the YTU partition before unlocking (p6) and then make the nvflash backups - but maybe the key must match something that is broken by the unlocking process, or it is renewed periodically, etc., so it might not help. Maybe try using DRM before unlocking and watch if the content of the partition changes over time.
utx said:
It would be also nice to know, whether there are areas of flash with hardware or kernel write lock.
Click to expand...
Click to collapse
Never tried to write directly to the block device - too scared to break something.
---------- Post added at 09:32 PM ---------- Previous post was at 09:28 PM ----------
Another small addition:
Note: /data/media is mounted via UID/GID stripping FUSE as /mnt/sdcard
Click to expand...
Click to collapse
This FUSE trick also makes /mnt/sdcard case-insensitive.

I just thought of something. What if you launched a data recovery process and recovered the DRM keys for the device?

ostar2 said:
I just thought of something. What if you launched a data recovery process and recovered the DRM keys for the device?
Click to expand...
Click to collapse
How do you define "data recovery process"? You cannot recover data that has been overwritten.

_that said:
How do you define "data recovery process"? You cannot recover data that has been overwritten.
Click to expand...
Click to collapse
Well, if the DRM partition is write enabled, it may be possible to restore its contents, if you backed it up before unlock (it is probably per-device unique). But it could be insufficient. Locked bootloader can be different than unlocked bootloader, and may drop cipher needed for DRM decihering. It is just a theory. Somebody could proof it or falsify, if:
1) Backed all accessible data before unlock.
2) Unlocked (and to be safe, also made brickproof image).
3) Recovered the data creates in step 1.
Will DRM work then? Or did we need the contents of (currently inaccessible) locked stock data of the first megabytes?
But I see no way, how to back-up first megabytes of locked device (on ICS; JB is not as interesting for us, once you upgrade to JB, you cannot create brickproof image for nvflash).
I even don't know, which part of the subsystem causes these megabytes being reported as zeroes. Is it stock Asus ICS kernel? Is it bootloader? Is it a hardware lock on the flash device?

Good idea, but what I meant by "Data Recovey". Is restoring the deleted data from that filesystem/partition.

ostar2 said:
Good idea, but what I meant by "Data Recovey". Is restoring the deleted data from that filesystem/partition.
Click to expand...
Click to collapse
I see, so I assume you assume you had a backup before.
Somebody (maybe you?) could try roughly the following sequence:
- get new TF700
- update to 9.4.5.26. if it's already newer, forget nvflash, but the rest could still work.
- root it using debugfs
- make a backup of /dev/block/mmcblk0p6
- do some DRM-dependent stuff and check that it works
- after some days, make another backup of /dev/block/mmcblk0p6 and compare if anything has changed. If the key is static, maybe restoring after unlocking could work. If not, chances are high that it doesn't work.
- unlock (this erases mmcblk06 and voids warranty)
- optional, but very useful: install AndroidRoot hacked bootloader to make blobs for nvflash, then use nvflash to backup all partitions
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works

_that said:
I see, so I assume you assume you had a backup before.
Somebody (maybe you?) could try roughly the following sequence:
- get new TF700
- update to 9.4.5.26. if it's already newer, forget nvflash, but the rest could still work.
- root it using debugfs
- make a backup of /dev/block/mmcblk0p6
- optional, but very useful: install AndroidRoot hacked bootloader to make blobs for nvflash, then use nvflash to backup all partitions
- do some DRM-dependent stuff and check that it works
- after some days, make another backup of /dev/block/mmcblk0p6 and compare if anything has changed. If the key is static, maybe restoring after unlocking could work. If not, chances are high that it doesn't work.
- unlock (this erases mmcblk06 and voids warranty)
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works
Click to expand...
Click to collapse
To install AndroidRoot bootloader and by that getting nvflash blobs, you have to unlock first... The order of your steps is therefore wrong.

firetech said:
To install AndroidRoot bootloader and by that getting nvflash blobs, you have to unlock first... The order of your steps is therefore wrong.
Click to expand...
Click to collapse
Oops, thanks for noticing. I edited my post.

what if we were to read from the NAND externally (RAW)....xbox 360 style...wouldn't that be the same as nvflash....
except that the three partitions in question are encrypted with a key that is probably unique per Tegra...
2 BCT: Tegra Boot Configuration Table - 3145728 bytes
3 PT: Tegra Partition Table - 524288 bytes
4 EBT: Bootloader - 8388608 bytes
but I would suppose it wouldn't be a problem since a raw flash would restore everything back to normal...even if we can't read it..the CPU can..and that's all that matters.
---------- Post added at 11:21 AM ---------- Previous post was at 11:13 AM ----------
never mind...its a BGA

_that said:
I see, so I assume you assume you had a backup before.
Somebody (maybe you?) could try roughly the following sequence:
- get new TF700
- update to 9.4.5.26. if it's already newer, forget nvflash, but the rest could still work.
- root it using debugfs
- make a backup of /dev/block/mmcblk0p6
- do some DRM-dependent stuff and check that it works
- after some days, make another backup of /dev/block/mmcblk0p6 and compare if anything has changed. If the key is static, maybe restoring after unlocking could work. If not, chances are high that it doesn't work.
- unlock (this erases mmcblk06 and voids warranty)
- optional, but very useful: install AndroidRoot hacked bootloader to make blobs for nvflash, then use nvflash to backup all partitions
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works
Click to expand...
Click to collapse
Correct order maybe.
- get new TF700
- update to 9.4.5.26.
- root it using debugfs
- make a backup of /dev/block/*.*
- unlock (this erases mmcblk06 and voids warranty)
- install AndroidRoot hacked bootloader to make blobs for nvflash
- restore backup of /dev/block/mmcblk0p6
- try if DRM still works
Q1:If i backed up 9.4.5.26 all block image.After i updated 9.4.5.30 can i get the nvflash blob from backed up images?No way to dig out the blob key from the backup?

W3ber said:
Q1:If i backed up 9.4.5.26 all block image.After i updated 9.4.5.30 can i get the nvflash blob from backed up images?No way to dig out the blob key from the backup?
Click to expand...
Click to collapse
No way - the BCT, bootloader, etc. is not visible to the kernel at all (so it's not included in your images), and I don't know which kind of magic the blob creation tool uses, but I assume it's more than reading stuff from the nand.

[Tool][python] LG Compressed KDZ Extractor

Hello everybody!
This will be my first XDA-wide release of my new utility for all LG phones.
What is this?
This is a utility to extract the new format KDZ files that LG distributes, specifically the 'compressed' ones.
LG frequently distributes firmware for phones as KDZ files, which are essentially a firmware image of the eMMC and a DLL file that is used by the downloader utility to communicate with the phone.
In the past, there were utilities to extract KDZ files to a DLL file and a DZ file, but no further (at least to my knowledge).
This utility lets you break the KDZ file into it's respective partitions (aboot, rpm, tz, and so on)​
What good does this do me?
If you're an phone modder, haxxor, or just an enthusiast that has access to their phone's KDZ file and would like to have a copy of the actual partitions stored within, this will let you.
As an example, firmware for the new LG G2 on many device models is distributed as a KDZ file only.
Other phones use a TOT file, which is essentially a disk image of the eMMC with no compression.
If someone with a KDZ firmware-only phone wiped a partition (for example, modem) and wanted to get it back without flashing the whole phone all over again, they would be stuck.
TOT files are easily extractable, as there is software available currently for that but until today there was none (to my knowledge) for these new KDZ files.​
How do I use this?
Glad you asked.
Inside the ZIP file you'll see two Python scripts, KDZFileTools.py and DZFileTools.py.
There's also a README.txt file for more in-depth information if you're curious.
Both scripts respond to --help or -h, so if you're even more curious, try that too!
KDZ files contain DZ files and DLL files, so the first step will be to split those into their respective parts:
LAS_V08d_pre3_00.kdz is the name of the KDZ file that I've copied to the working directory for this example.
Code:
# python KDZFileTools.py -l -f LAS_V08d_pre3_00.kdz
[+] KDZ Partition List
=========================================
0 : LAS_V08d_pre3_00.dz (1428092632 bytes)
1 : LGUP_8974.dll (1477632 bytes)
This shows me that there are two files inside the KDZ file: LAS_V08d_pre3_00.dz and LGUP_8974.dll
You can now extract them by ID by using the -s option, or by using -x to extract all of the files.
Code:
# python KDZFileTools.py -f LAS_V08d_pre3_00.kdz -x
[+] Extracting all partitions!
[+] Extracting LAS_V08d_pre3_00.dz to kdzextracted\LAS_V08d_pre3_00.dz
[+] Extracting LGUP_8974.dll to kdzextracted\LGUP_8974.dll
Now you'll see a folder called "kdzextracted" in your current working directory, which will contain the extracted files.
The next step would be to extract the DZ file to get the partitions it contains:
Code:
# python DZFileTools.py -f kdzextracted/LAS_V08d_pre3_00.dz -l
[+] DZ Partition List
=========================================
0 : PrimaryGPT_0.bin (4299 bytes)
1 : modem_32768.bin (25719664 bytes)
2 : sbl1_163840.bin (179443 bytes)
3 : dbi_165888.bin (10505 bytes)
4 : aboot_229376.bin (288082 bytes)
5 : rpm_231424.bin (93084 bytes)
6 : boot_262144.bin (8959565 bytes)
7 : tz_294912.bin (149388 bytes)
8 : persist_393216.bin (23621 bytes)
9 : recovery_458752.bin (10454494 bytes)
10 : laf_622592.bin (14244284 bytes)
11 : system_7176192.bin (66791740 bytes)
12 : system_7438336.bin (2651 bytes)
13 : system_7440008.bin (2313 bytes)
14 : system_7444120.bin (103727934 bytes)
15 : system_7704592.bin (114239263 bytes)
16 : system_7964296.bin (2313 bytes)
17 : system_7968408.bin (103349001 bytes)
18 : system_8228880.bin (121921125 bytes)
19 : system_8488584.bin (2313 bytes)
20 : system_8492696.bin (101078725 bytes)
21 : system_8753168.bin (125454806 bytes)
22 : system_9012872.bin (2313 bytes)
23 : system_9016984.bin (105806605 bytes)
24 : system_9277456.bin (115830981 bytes)
25 : system_9537160.bin (2313 bytes)
26 : system_9541272.bin (108458465 bytes)
27 : system_9801744.bin (83280847 bytes)
28 : system_10063888.bin (67940827 bytes)
29 : system_10326032.bin (91997923 bytes)
30 : system_10588176.bin (58015487 bytes)
31 : system_10846208.bin (2314 bytes)
32 : system_11108352.bin (2314 bytes)
33 : system_11370496.bin (2314 bytes)
34 : system_11632640.bin (2314 bytes)
35 : system_11894784.bin (2314 bytes)
36 : system_12156928.bin (2314 bytes)
37 : system_12419072.bin (2314 bytes)
38 : system_12681216.bin (2314 bytes)
39 : system_12943360.bin (2314 bytes)
40 : system_13205504.bin (2314 bytes)
41 : system_13467648.bin (2314 bytes)
42 : system_13729792.bin (2652 bytes)
43 : system_13731464.bin (2314 bytes)
44 : BackupGPT_61070336.bin (4286 bytes)
Excellent! All the files are there!
Large images are split up by LG, and can be combined with "cat" or something like that.
The filename actually is in the form "partname_offset.bin" where "offset" is the actual location that the file should be written to on the phone's eMMC (handy!)
You can substitute -l in the options for -x again to extract all the partitions to the folder "dzextracted" in the current working directory as well.
The option --out or -o will change the output directory, so it doesn't have to output to {kdz|dz}extracted​
Where can I download this?
Code:
[URL="http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip"]http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip[/URL]
Please don't mirror this file, as having it in one location makes it easy for me to keep it up to date, or pull it if there are problems with this script​
It doesn't work? Can you help me?!
Sure, join #lg-g2 on Freenode and ask for IOMonster and I'll try to help out as best I can.
I will not answer questions in PM or via email, please don't PM me about this script!
As usual, if this script eats your dog/cat/uncle please don't blame me either. I didn't mean to do it, I swear!​
Whoo! Thanks! It works! How can I say thanks?
"Thanking" this thread is great!
If you feel that I absolutely saved you many hours of time, feel free to click on the "Donate to Me" button next to my username in this post.
This project was funded by a boring day at work and lots of caffiene ​
Alright guys, let me know if you've got any questions and I'll see what I can do!

Ha! Just what I was looking for but now to get it to work in Windows...
Thanks!
Edit
So I installed Python for Windows and this was the output...
C:\Python33>KDZFileTools.py -l -f F320K11D_00.kdz
File "C:\Python33\KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
C:\Python33>python.exe DZFileTools.py -l -f D80210C_00.kdz
File "DZFileTools.py", line 96
print "[!] Bad DZ sub header!"
^
SyntaxError: invalid syntax
C:\Python33>python.exe KDZFileTools.py -l -f D80210C_00.kdz
File "KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax

AndroidUser00110001 said:
Ha! Just what I was looking for but now to get it to work in Windows...
Thanks!
Edit
So I installed Python for Windows and this was the output...
C:\Python33>KDZFileTools.py -l -f F320K11D_00.kdz
File "C:\Python33\KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
C:\Python33>python.exe DZFileTools.py -l -f D80210C_00.kdz
File "DZFileTools.py", line 96
print "[!] Bad DZ sub header!"
^
SyntaxError: invalid syntax
C:\Python33>python.exe KDZFileTools.py -l -f D80210C_00.kdz
File "KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
Click to expand...
Click to collapse
Not sure what's happening here, but I wrote this in Windows, so it works for me...
Try using Python 2.7 ? I haven't tested it with Py3k...
If it still gives you invalid headers after you get rid of the SyntaxError messages, let me know and I'll take a look. I've only verified that this works on a handful of KDZ files.

Thanks for such a awesome tool. After extracting D80210E_00.kdz, I found that system_xxxxxxx.bin files is not contiguous. Merge into one file, you can not mount it.
would you like to develop a tool more for system partitions merge?

honentan said:
Thanks for such a awesome tool. After extracting D80210E_00.kdz, I found that system_xxxxxxx.bin files is not contiguous. Merge into one file, you can not mount it.
would you like to develop a tool more for system partitions merge?
Click to expand...
Click to collapse
Excellent timing I'm actually working on that right now.
It would appear that the KDZ file contains only the 'sparse' data from large partitions.
Basically, it's the same as if you fed the utility "hexdump" a large file with multiple sections will have the empty sections 'collapsed' and shown as a "*".
My utility needs to take the beginning position of the first system_xxxx.bin file into account, and the end system_xxxx.bin 's position and size.
From there, it knows how big the image file needs to be, then it can simply seek to the position in the new file as defined by each .bin file and write it's contents there.
Ultimately, I'd like to have in-place extraction from the KDZ files, without the need to extract the DZ file itself (since it really is just copying the bytes from the right position to a new file on the hdd), and a utility to merge split partitions, like system.
Until then, if you need to create this image yourself, you can do so with a hex editor or dd
1. Take the first system_xxxxx.bin file and write down the value of xxxxx somewhere. We're going to use this as an offset.
2. Take the last system_xxxxx.bin file and write down it's xxxxx as well. Subtract the two. You should now get the size of the system.img file
3. Use "dd" to create a file of proper length: dd if=/dev/zero of=system.img bs=1 count=SIZE_FROM_EARLIER
4. Now, let's put our files into the right places. The first file can just be merged into it with dd if=system_xxxxx.bin of=system.img conv=notrunc
5. For every file after the first one, you'll have to take the xxxxx value from the file, and subtract the offset you found from step 1. use dd if=system_xxxxx.bin of=system.img seek=VALUE_YOU_JUST_FOUND
6. The end file is no different. Subtract it's xxxxx value again and run the above DD command.
If I'm right, this should leave you with a working partition image.
I haven't verified yet, but it seems logical and should work

thanks for your reply. I am going to try the method as you mentioned just now to merge the system.img using winhex, but I have not verified yet too. I will try it tomorrow.
as a advice, new tool should be able to extract and repack new kdz file.

honentan said:
Thanks for such a awesome tool. After extracting D80210E_00.kdz, I found that system_xxxxxxx.bin files is not contiguous. Merge into one file, you can not mount it.
would you like to develop a tool more for system partitions merge?
Click to expand...
Click to collapse
thecubed said:
Excellent timing I'm actually working on that right now.
It would appear that the KDZ file contains only the 'sparse' data from large partitions.
Basically, it's the same as if you fed the utility "hexdump" a large file with multiple sections will have the empty sections 'collapsed' and shown as a "*".
My utility needs to take the beginning position of the first system_xxxx.bin file into account, and the end system_xxxx.bin 's position and size.
From there, it knows how big the image file needs to be, then it can simply seek to the position in the new file as defined by each .bin file and write it's contents there.
Ultimately, I'd like to have in-place extraction from the KDZ files, without the need to extract the DZ file itself (since it really is just copying the bytes from the right position to a new file on the hdd), and a utility to merge split partitions, like system.
Until then, if you need to create this image yourself, you can do so with a hex editor or dd
1. Take the first system_xxxxx.bin file and write down the value of xxxxx somewhere. We're going to use this as an offset.
2. Take the last system_xxxxx.bin file and write down it's xxxxx as well. Subtract the two. You should now get the size of the system.img file
3. Use "dd" to create a file of proper length: dd if=/dev/zero of=system.img bs=1 count=SIZE_FROM_EARLIER
4. Now, let's put our files into the right places. The first file can just be merged into it with dd if=system_xxxxx.bin of=system.img conv=notrunc
5. For every file after the first one, you'll have to take the xxxxx value from the file, and subtract the offset you found from step 1. use dd if=system_xxxxx.bin of=system.img seek=VALUE_YOU_JUST_FOUND
6. The end file is no different. Subtract it's xxxxx value again and run the above DD command.
If I'm right, this should leave you with a working partition image.
I haven't verified yet, but it seems logical and should work
Click to expand...
Click to collapse
I merged system.img using script as you mentioned, but it couldn't be mounted also.
Progress was obtained, I got all files in system.img using ext2explore(version2.1).
Code:
dd if=/dev/zero of=system.img bs=1 count=5505024
dd if=system_819200.bin of=system.img conv=notrunc seek=0
dd if=system_1081344.bin of=system.img conv=notrunc seek=262144
dd if=system_1082760.bin of=system.img conv=notrunc seek=263560
dd if=system_1086808.bin of=system.img conv=notrunc seek=267608
dd if=system_1347536.bin of=system.img conv=notrunc seek=528336
dd if=system_1607048.bin of=system.img conv=notrunc seek=787848
dd if=system_1611096.bin of=system.img conv=notrunc seek=791896
dd if=system_1871824.bin of=system.img conv=notrunc seek=1052624
dd if=system_2131336.bin of=system.img conv=notrunc seek=1312136
dd if=system_2135384.bin of=system.img conv=notrunc seek=1316184
dd if=system_2396112.bin of=system.img conv=notrunc seek=1576912
dd if=system_2655624.bin of=system.img conv=notrunc seek=1836424
dd if=system_2659672.bin of=system.img conv=notrunc seek=1840472
dd if=system_2920400.bin of=system.img conv=notrunc seek=2101200
dd if=system_3179912.bin of=system.img conv=notrunc seek=2360712
dd if=system_3183960.bin of=system.img conv=notrunc seek=2364760
dd if=system_3444688.bin of=system.img conv=notrunc seek=2625488
dd if=system_3706832.bin of=system.img conv=notrunc seek=2887632
dd if=system_3968976.bin of=system.img conv=notrunc seek=3149776
dd if=system_4231120.bin of=system.img conv=notrunc seek=3411920
dd if=system_4493264.bin of=system.img conv=notrunc seek=3674064
dd if=system_4755408.bin of=system.img conv=notrunc seek=3936208
dd if=system_5017552.bin of=system.img conv=notrunc seek=4198352
dd if=system_5275648.bin of=system.img conv=notrunc seek=4456448
dd if=system_5537792.bin of=system.img conv=notrunc seek=4718592
dd if=system_5799936.bin of=system.img conv=notrunc seek=4980736
dd if=system_6062080.bin of=system.img conv=notrunc seek=5242880
dd if=system_6324224.bin of=system.img conv=notrunc seek=5505024
ext2explore log:
Partition Table Error on D:/Android-ROM/980VS/system.img
Invalid End of sector markerBlock size 4096, inp 8064, inodesize 256
Linux Partition found on disk 1 partition 0
Inode 727 with file size 0

thecubed said:
Not sure what's happening here, but I wrote this in Windows, so it works for me...
Try using Python 2.7 ? I haven't tested it with Py3k...
If it still gives you invalid headers after you get rid of the SyntaxError messages, let me know and I'll take a look. I've only verified that this works on a handful of KDZ files.
Click to expand...
Click to collapse
Thanks, I got it working using 2.7 I have everything extracted and now I need to learn how to merge all the system files.
Thanks!

AndroidUser00110001 said:
Thanks, I got it working using 2.7 I have everything extracted and now I need to learn how to merge all the system files.
Thanks!
Click to expand...
Click to collapse
I'm working on a modified version of the DZFileUtils.py script, since the DZ file actually contains the proper information to regenerate the full system.img easier than using DD or a hex editor.
Work has been a little crazy for the past few days, so I may not get a chance to work on the script until next week, but it won't be that hard to have this script put the system.img files back together.
I could modify the whole toolsets to do in-place extraction from a KDZ file (skipping the intermediate DZ file) but for now this is the easiest and accomplishes my goal the fastest (which is to allow extraction of bootloader stacks from KDZ files found on the interwebs)
I have no plans on creating a utility to generate KDZ files, since myself and @Shelnutt2 are in the process of writing a utility to flash images without needing any LG software on your PC.

I used dd for windows and was able to merge all the files with the script that someone posted. Sorry on XDA App and can't see full thread but thanks for the script.
If you guys need help with testing let me know.
Sent from my LG-D800 using XDA Premium 4 mobile app

thecubed said:
Hello everybody!
This will be my first XDA-wide release of my new utility for all LG phones.
What is this?
This is a utility to extract the new format KDZ files that LG distributes, specifically the 'compressed' ones.
LG frequently distributes firmware for phones as KDZ files, which are essentially a firmware image of the eMMC and a DLL file that is used by the downloader utility to communicate with the phone.
In the past, there were utilities to extract KDZ files to a DLL file and a DZ file, but no further (at least to my knowledge).
This utility lets you break the KDZ file into it's respective partitions (aboot, rpm, tz, and so on)​
What good does this do me?
If you're an phone modder, haxxor, or just an enthusiast that has access to their phone's KDZ file and would like to have a copy of the actual partitions stored within, this will let you.
As an example, firmware for the new LG G2 on many device models is distributed as a KDZ file only.
Other phones use a TOT file, which is essentially a disk image of the eMMC with no compression.
If someone with a KDZ firmware-only phone wiped a partition (for example, modem) and wanted to get it back without flashing the whole phone all over again, they would be stuck.
TOT files are easily extractable, as there is software available currently for that but until today there was none (to my knowledge) for these new KDZ files.​
How do I use this?
Glad you asked.
Inside the ZIP file you'll see two Python scripts, KDZFileTools.py and DZFileTools.py.
There's also a README.txt file for more in-depth information if you're curious.
Both scripts respond to --help or -h, so if you're even more curious, try that too!
KDZ files contain DZ files and DLL files, so the first step will be to split those into their respective parts:
LAS_V08d_pre3_00.kdz is the name of the KDZ file that I've copied to the working directory for this example.
Code:
# python KDZFileTools.py -l -f LAS_V08d_pre3_00.kdz
[+] KDZ Partition List
=========================================
0 : LAS_V08d_pre3_00.dz (1428092632 bytes)
1 : LGUP_8974.dll (1477632 bytes)
This shows me that there are two files inside the KDZ file: LAS_V08d_pre3_00.dz and LGUP_8974.dll
You can now extract them by ID by using the -s option, or by using -x to extract all of the files.
Code:
# python KDZFileTools.py -f LAS_V08d_pre3_00.kdz -x
[+] Extracting all partitions!
[+] Extracting LAS_V08d_pre3_00.dz to kdzextracted\LAS_V08d_pre3_00.dz
[+] Extracting LGUP_8974.dll to kdzextracted\LGUP_8974.dll
Now you'll see a folder called "kdzextracted" in your current working directory, which will contain the extracted files.
The next step would be to extract the DZ file to get the partitions it contains:
Code:
# python DZFileTools.py -f kdzextracted/LAS_V08d_pre3_00.dz -l
[+] DZ Partition List
=========================================
0 : PrimaryGPT_0.bin (4299 bytes)
1 : modem_32768.bin (25719664 bytes)
2 : sbl1_163840.bin (179443 bytes)
3 : dbi_165888.bin (10505 bytes)
4 : aboot_229376.bin (288082 bytes)
5 : rpm_231424.bin (93084 bytes)
6 : boot_262144.bin (8959565 bytes)
7 : tz_294912.bin (149388 bytes)
8 : persist_393216.bin (23621 bytes)
9 : recovery_458752.bin (10454494 bytes)
10 : laf_622592.bin (14244284 bytes)
11 : system_7176192.bin (66791740 bytes)
12 : system_7438336.bin (2651 bytes)
13 : system_7440008.bin (2313 bytes)
14 : system_7444120.bin (103727934 bytes)
15 : system_7704592.bin (114239263 bytes)
16 : system_7964296.bin (2313 bytes)
17 : system_7968408.bin (103349001 bytes)
18 : system_8228880.bin (121921125 bytes)
19 : system_8488584.bin (2313 bytes)
20 : system_8492696.bin (101078725 bytes)
21 : system_8753168.bin (125454806 bytes)
22 : system_9012872.bin (2313 bytes)
23 : system_9016984.bin (105806605 bytes)
24 : system_9277456.bin (115830981 bytes)
25 : system_9537160.bin (2313 bytes)
26 : system_9541272.bin (108458465 bytes)
27 : system_9801744.bin (83280847 bytes)
28 : system_10063888.bin (67940827 bytes)
29 : system_10326032.bin (91997923 bytes)
30 : system_10588176.bin (58015487 bytes)
31 : system_10846208.bin (2314 bytes)
32 : system_11108352.bin (2314 bytes)
33 : system_11370496.bin (2314 bytes)
34 : system_11632640.bin (2314 bytes)
35 : system_11894784.bin (2314 bytes)
36 : system_12156928.bin (2314 bytes)
37 : system_12419072.bin (2314 bytes)
38 : system_12681216.bin (2314 bytes)
39 : system_12943360.bin (2314 bytes)
40 : system_13205504.bin (2314 bytes)
41 : system_13467648.bin (2314 bytes)
42 : system_13729792.bin (2652 bytes)
43 : system_13731464.bin (2314 bytes)
44 : BackupGPT_61070336.bin (4286 bytes)
Excellent! All the files are there!
Large images are split up by LG, and can be combined with "cat" or something like that.
The filename actually is in the form "partname_offset.bin" where "offset" is the actual location that the file should be written to on the phone's eMMC (handy!)
You can substitute -l in the options for -x again to extract all the partitions to the folder "dzextracted" in the current working directory as well.
The option --out or -o will change the output directory, so it doesn't have to output to {kdz|dz}extracted​
Where can I download this?
Code:
[URL="http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip"]http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip[/URL]
Please don't mirror this file, as having it in one location makes it easy for me to keep it up to date, or pull it if there are problems with this script​
It doesn't work? Can you help me?!
Sure, join #lg-g2 on Freenode and ask for IOMonster and I'll try to help out as best I can.
I will not answer questions in PM or via email, please don't PM me about this script!
As usual, if this script eats your dog/cat/uncle please don't blame me either. I didn't mean to do it, I swear!​
Whoo! Thanks! It works! How can I say thanks?
"Thanking" this thread is great!
If you feel that I absolutely saved you many hours of time, feel free to click on the "Donate to Me" button next to my username in this post.
This project was funded by a boring day at work and lots of caffiene ​
Alright guys, let me know if you've got any questions and I'll see what I can do!
Click to expand...
Click to collapse
Hi, i make my phone soft bricked. no other method works. i need bin file to flash my phone.i have the orignal kdz file.how can i make a single bin file to flash it using LGNPST? If you want to read my whole phone bricking story, please follow this
http://forum.xda-developers.com/showthread.php?t=2492105

AAAAwesome!!!
Thank you very much!
Able to customize LG's ROM easier than ever!

thecubed said:
I'm working on a modified version of the DZFileUtils.py script, since the DZ file actually contains the proper information to regenerate the full system.img easier than using DD or a hex editor.
Work has been a little crazy for the past few days, so I may not get a chance to work on the script until next week, but it won't be that hard to have this script put the system.img files back together.
I could modify the whole toolsets to do in-place extraction from a KDZ file (skipping the intermediate DZ file) but for now this is the easiest and accomplishes my goal the fastest (which is to allow extraction of bootloader stacks from KDZ files found on the interwebs)
I have no plans on creating a utility to generate KDZ files, since myself and @Shelnutt2 are in the process of writing a utility to flash images without needing any LG software on your PC.
Click to expand...
Click to collapse
Any word on the DZfileUtils.py script? I am trying to convert the dz to the system img and all I get is errors when trying to use dd. If there is a better way than dd for windows could someone let me know.
Well I think I have found where my error was at with what I was trying to do.
Thank you for all your hard work on this

honentan said:
I merged system.img using script as you mentioned, but it couldn't be mounted also.
Progress was obtained, I got all files in system.img using ext2explore(version2.1).
ext2explore log:
Partition Table Error on D:/Android-ROM/980VS/system.img
Invalid End of sector markerBlock size 4096, inp 8064, inodesize 256
Linux Partition found on disk 1 partition 0
Inode 727 with file size 0
Click to expand...
Click to collapse
I can confirm I could not mount or use normal methods I have used in the past for getting the system image extracted but I also used ext2explore on windows and it worked like a charm. Thank you

Thanks for tools:good:
but I got this error when I tried with KDZ file.
C:\Users\VM\Desktop\Compressed_KDZUtils>KDZFileTools.py -f L01E20A_00.kdz -x
[!] Error: Unsupported KDZ file format.
[ ] Expected: 0x28 0x5 0x0 0x0 0x34 0x31 0x25 0x80 ,
but received 0x31 0x23 0x53 0x7f 0x89 0x1e 0xe6 0x9b .
Click to expand...
Click to collapse
and when I got DZ file from other KDZ extracter and tried DZ file I got this
C:\Users\VM\Desktop\Compressed_KDZUtils>DZFileTools.py -f L01E20A_00.dz -l
[!] Error: Unsupported DZ file format.
[ ] Expected: 0x32 0x96 0x18 0x74 ,
but received 0x22 0x12 0x84 0x19 .
Traceback (most recent call last):
File "C:\Users\VM\Desktop\Compressed_KDZUtils\DZFileTools.py", line 212, in
<module>
dztools.main()
File "C:\Users\VM\Desktop\Compressed_KDZUtils\DZFileTools.py", line 196, in
main
self.openFile(args.dzfile)
File "C:\Users\VM\Desktop\Compressed_KDZUtils\DZFileTools.py", line 173, in
openFile
sys.exit(0)
NameError: global name 'sys' is not defined
Click to expand...
Click to collapse
I never run python script but I think I did it right.
I'm installed python 2.7. and KDZ file is for Optimus G

rquiett said:
I can confirm I could not mount or use normal methods I have used in the past for getting the system image extracted but I also used ext2explore on windows and it worked like a charm. Thank you
Click to expand...
Click to collapse
honentan said:
I merged system.img using script as you mentioned, but it couldn't be mounted also.
Progress was obtained, I got all files in system.img using ext2explore(version2.1).
Click to expand...
Click to collapse
in linux
Code:
mkdir -p /mnt/dsk
mount -o loop system.img /mnt/dsk
this will return an error
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
Click to expand...
Click to collapse
run dmesg
Code:
dmesg | tail
there will be a line like this:
EXT4-fs (loop0): bad geometry: block count 851968 exceeds size of device (819537 blocks)
Click to expand...
Click to collapse
now truncate the file with the value of the first number
Code:
truncate -o -s 851968
remount, and hey presto.
thank you @Flinny for this
edit: not needed anymore, just ignore it. keeping it here for reference though

merging the bins on linux
ok, so there have been a few issues with getting this done on any platform. I was asked to try to get into a kdz for someone, so I did it
included below is a python script that takes advantage of dd. I've set it up for the system partition (as that's the only one that is split)
I have left all my comments and tweaks in so you can see how it was formed
to run:
Code:
python SystemMerger.py
It should run with python3 (untested). can someone please report any errors and I'll see if I can fix them up.
after running the script, just mount the image, and you're away
Happy Hacking
ps, don't forget to change the extension to .py

ok final version here, should support windows now
https://github.com/cybojenix/random-scripts/blob/master/SystemMerger.py
it should also support python3 still, but someone please check, same with windows (though I'm sceptical about the way I write zeros. might not work on winblows...)
anyway, happy hacking again

cybojenix said:
ok final version here, should support windows now
https://github.com/cybojenix/random-scripts/blob/master/SystemMerger.py
it should also support python3 still, but someone please check, same with windows (though I'm sceptical about the way I write zeros. might not work on winblows...)
anyway, happy hacking again
Click to expand...
Click to collapse
good job! awesome! thanks.

AndroidUser00110001 said:
Ha! Just what I was looking for but now to get it to work in Windows...
Thanks!
Edit
So I installed Python for Windows and this was the output...
C:\Python33>KDZFileTools.py -l -f F320K11D_00.kdz
File "C:\Python33\KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
C:\Python33>python.exe DZFileTools.py -l -f D80210C_00.kdz
File "DZFileTools.py", line 96
print "[!] Bad DZ sub header!"
^
SyntaxError: invalid syntax
C:\Python33>python.exe KDZFileTools.py -l -f D80210C_00.kdz
File "KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
Click to expand...
Click to collapse
just put bracket on each print params, then it should work
like this print ("[!] Error: Unsupported KDZ file format.")
also the KDZFileTools not work on LG G2, the header doesnt match with required
here
i debug to find what is the actual value on verify_header
D80210b.kdz header found
>>> verify_header
b'(\x05\x00\x0041%\x80'
kdz_header = '0x28 0x5 0x0 0x0 0x34 0x31 0x25 0x80'
KDZFileTools error log
[!] Error: Unsupported KDZ file format.
Traceback (most recent call last):
File "C:\Downloads\KDZFileTools.py", line 197, in <module>
kdztools.main()
File "C:\Downloads\KDZFileTools.py", line 182, in main
self.openFile(args.kdzfile)
File "C:\Downloads\KDZFileTools.py", line 160, in openFile
print ("[ ] Expected: %s ,\n\tbut received %s ." % (" ".join(hex(ord) for n in self.kdz_header), " ".join(hex(ord) for n in verify_header)))
File "C:\Downloads\KDZFileTools.py", line 160, in <genexpr>
print ("[ ] Expected: %s ,\n\tbut received %s ." % (" ".join(hex(ord) for n in self.kdz_header), " ".join(hex(ord) for n in verify_header)))
TypeError: ord() expected string of length 1, but int found
Click to expand...
Click to collapse

[GUIDE][UTIL][MT65xx] Create Scatter File and Dump Full ROM

[GUIDE][UTIL][MT65xx] Create Scatter File / Dump Full ROM
For any MT65xx device, no matter how obscure.
I will discuss a method for:
* making your own SPFT scatter file
* dumping your entire ROM (without root)
* dicing up your entire ROM into partition blocks
This is somewhat of a manual process. rua1's MTK Droid Root & Tools circumvents the need for doing most of this. I applaude his work, it's a big undertaking and he supports it well.
Here are a few reasons to use the method I discuss here:
* you want a ROM dump without rooting
* you want a ROM dump without your OS booted (clone cold system is safe)
* you want a guaranteed way to restore exactly current state (safety and return to store)
* you are just plain worried something will go wrong
-------------------------------------
I have two methods for creating a scatter file.
Method #1
Find your scatter information in SP Flash Tool's Logs
Well first, you have to make SP Flash Tool happy enough to give you some information about your device.
* Obtain a SPFT ROM that is known good for any phone/tablet, preferrably with the same chip as yours. Make sure that scatter file loads into SPFT without error, SPFT checks the PRELOADER and DSP_BL and if they aren't in the scatter directory, it will fail and maybe crash.
* Close SPFT, now modify the MT*scatter.txt file and introduce a few errors in the partition names, but don't change PRELOADER or DSP_BL. An example, instead of "ANDROID" replace it with "DEADBEEF". You ask why? Well you want to make SURE that the "Download" feature fails. You DON'T want the "Download" to actually work and write your random SPFT ROM to your new device. After loading the freshly broken scatter file, click "Download" and hook up your MTK device in preloader / META Mode. It will say your PMT block does not match the scatter file. ERROR:
* Go to SPFT's menu "Help / Open logs folder" and find the latest date. Open the BROM_DLL*.log in your text editor. Search for text that looks like this, I'd first search for "CMD_ReadPartitionInfo():
Code:
11/14/13 09:01:27.382 BROM_DLL[3836][2208]: DA_cmd::CMD_ReadPartitionInfo(): command is allowed. (FlashToolLib/sv5/common/generic/src/da_c
md.cpp:5242)
11/14/13 09:01:27.382 BROM_DLL[3836][2208]: DA_cmd::CMD_ReadPartitionInfo(): getting 20 partitions .. (FlashToolLib/sv5/common/generic/src
/da_cmd.cpp:5269)
11/14/13 09:01:27.397 BROM_DLL[3836][2208]: DA_cmd::CMD_ReadPartitionInfo(): dump 20 partitions (FlashToolLib/sv5/common/generic/src/da_cm
d.cpp:5279)
11/14/13 09:01:27.397 BROM_DLL[3836][2208]: PART[0 ](PRELOADER ) - offset (0x0000000000000000) - size (0x0000000000040000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.397 BROM_DLL[3836][2208]: PART[1 ](DSP_BL ) - offset (0x0000000000040000) - size (0x00000000005C0000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.397 BROM_DLL[3836][2208]: PART[2 ](MBR ) - offset (0x0000000000600000) - size (0x0000000000004000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[3 ](EBR1 ) - offset (0x0000000000604000) - size (0x000000000005C000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[4 ](__NODL_PMT ) - offset (0x0000000000660000) - size (0x0000000000400000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[5 ](__NODL_NVRAM ) - offset (0x0000000000A60000) - size (0x0000000000300000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[6 ](__NODL_SECCFG ) - offset (0x0000000000D60000) - size (0x0000000000020000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[7 ](UBOOT ) - offset (0x0000000000D80000) - size (0x0000000000060000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[8 ](BOOTIMG ) - offset (0x0000000000DE0000) - size (0x0000000000600000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[9 ](RECOVERY ) - offset (0x00000000013E0000) - size (0x0000000000600000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[10](SEC_RO ) - offset (0x00000000019E0000) - size (0x0000000000600000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[11](__NODL_MISC ) - offset (0x0000000001FE0000) - size (0x0000000000060000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[12](LOGO ) - offset (0x0000000002040000) - size (0x0000000000300000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[13](__NODL_EXPDB ) - offset (0x0000000002340000) - size (0x00000000000A0000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[14](EBR2 ) - offset (0x00000000023E0000) - size (0x0000000000004000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[15](FAC ) - offset (0x00000000023E4000) - size (0x0000000010000000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[16](ANDROID ) - offset (0x00000000123E4000) - size (0x0000000020100000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[17](CACHE ) - offset (0x00000000324E4000) - size (0x0000000020100000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[18](USRDATA ) - offset (0x00000000525E4000) - size (0x0000000020100000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
11/14/13 09:01:27.413 BROM_DLL[3836][2208]: PART[19](FAT ) - offset (0x00000000726E4000) - size (0x0000000000000000) - mask (0x0
000000000000000) (FlashToolLib/sv5/common/generic/src/da_cmd.cpp:5283)
Make sure you aren't looking at a dump of your scatter, but looking at the partition dump from your device. (If it contains DEADBEEF flag from above, you are looking at the wrong part of the log.) You can manually create a scatter file from that, or you can paste the lines with partition info into a text file and run this shell script:
Code:
cat pastedlog.txt | sed -n 's/.*](\([^ ]*\)[^(]*(\([^)]*\).*)/\1 \2\n{\n}/p' | sed 's/x00000000/x/g' > mynewscatter.txt
This will get you really close, but the last entry you see in a typical scatter file is BMTPOOL. I don't know much about BMTPOOL or __NODL_BMPPOOL other than it's not a real partition. It might not even be required? (You might be able to fake that entry if you look at similar scatter files. If not, read on)
Method #2
An alternate and supplemental way of getting scatter info is to use ADB on a running device. I'm sure this is similar to MTK Droid Root & Tools method.
Code:
adb pull /proc/dumchar_info
Now that you have that file local on your PC, you can run this python script:
Code:
import sys
import string
import re
ins = open( "dumchar.txt", "rb" )
outs = open( "scatter.txt", "wb" )
for line in ins:
linesp = re.split('\W+', line)
name = linesp[0].upper()
start = int(linesp[2],16)
block = linesp[5]
if block != 'misc':
start = start + 0x600000
outs.write(name + " " + string.replace(hex(start), "L", "") + "\n{\n}\n")
ins.close()
outs.close()
This method gives you the BMTPOOL entry that the other method does not, but it lacks the __NODL_ designator for all partitions. If you aren't familiar with that prefix, SPFT uses that to know if it should typically allow you to DownLoad over the top of the paritition.
With both scatter files, throw them side-by-side in a text editor (or diff tool) and with a brain, it's easy enough to merge one with the __NODL_ prefix and the one with the BMTPOOL entry.
Scatter file complete!
PS - Possibly another way, is to use SPFT to do a "Read back" (read memory) at 0x660000 of size 0x400000. Save this as the PMT block and analyse it with a hex editor. I believe the PMT block is always at address 0x66000 judging from a dozen different scatter files.
Dump Full ROM
Perfect Total Backup of your Firmware
The safe way to do this with or without a proper scatter file is the "Read back" feature of SP Flash Tool. There are MANY reasons to dislike MediaTek, but this feature is so nice that I can forgive them for most of their wrong doing.
Most of this section I will generalize from my Lenovo A2107A Guide.
Here is a cookbook for doing a total backup of your MTK device with MediaTek's SP Flash Tool. No rooting, you might even do this before you ever boot! I have basically done this with both of my devices before I fiddled too much. I recommend doing it before you do anything really.
1. Install VCOM Drivers.
2. Install SP Flash Tool.
3. Grab an SPFT ROM, really anything should work, you just have to make SP Flash Tool happy. SPFT validates the scatter file against some of the image files, so you have to calm SPFT down by giving it something it can make sense of. We won't use the scatter file or image files while we do the "Read back" operation.
4. Run SP Flash Tool, Open Scatter File
5. Don't play with anything, go into the "Read back" tab (This will read your flash to a file on your PC)
6. Click on any items in the list, then click the "Remove" button
7. Now click the "Add" button
8. Double click on the "N/A" under Read Flag
9. Type a file name to write to, like "WHOLE_ROM"
10. Now it will popup a window "Readback block start address"
11. Leave "Hex" selected, Start Address" 0x0000, Length: 0x40000000, Click OK (NOTE: this will get the first GIG of flash)
12. Click the "Read back" button
13. SPFT now waits for you to connect your device and put it in Meta Mode
14. Without plugging your phone/tablet in, tap the Reset Button or make sure it's fully off
15. Hold VolUp, plug in USB, Release VolUp (putting it in Meta Mode) <--- Important
16. You will see the progress bar moving. Total backup takes forever, because in this mode SPFT seems to not do USB HIGHSPEED
That's IT! It'll take a few hours, so go to bed.
If you ever restore, just go into Recovery and Wipe Data and Cache. (as these are large and we probably didn't back them up above)
Note: "Length" in Step 11 is probably long enough for most devices. You can reference the scatter file you made above to make sure get everything, but you don't need USRDATA or CACHE, as those get wiped anyway.
Dicing up Full ROM image into partition images
I've made a little bash shell script to dice up a whole ROM according to a scatter file. This creates files exactly the size of the partitions. Some post processing needs done below the script:
dice.sh
Code:
#!/bin/bash
scatterfile=$1
rom=$2
pdir=raw.partitions
rm -rf $pdir
mkdir $pdir
cat $scatterfile | grep "x" | while read line; do
name=$(echo $line | sed 's/ .*//g')
name=$(echo $name | sed 's/^__NODL_//g')
start=$(echo $line | sed 's/.* //g')
# echo $name
if [[ -n $last_name ]]; then
echo "Processing: $last_name"
echo " start: $last_start"
size=$(( $start - $last_start ))
if [[ $size -lt 0 ]]; then
size=0xFFFFF000
else
size=0x$(echo "obase=16; $size" | bc)
fi
echo " size: $size"
short_start=$(echo $last_start | sed 's/000$//g')
short_size=$(echo $size | sed 's/000$//g')
echo dd if=$rom of=$pdir/$last_name bs=$(( 0x1000 )) \
skip=$(( $short_start )) count=$(( $short_size ))
dd if=$rom of=$pdir/$last_name bs=$(( 0x1000 )) \
skip=$(( $short_start )) count=$(( $short_size ))
fi
last_name=$name
last_start=$start
done
Now there is some post processing done. Truncate MBR, EBR1, EBR2 to 512 bytes. And remove trailing bytes of 0000 or FFFF on the end of PRELOADER and DSP_BL.
Here is a one off script for example use:
clean.sh
Code:
#!/bin/bash
pdir=raw.partitions
odir=out
rm -rf $odir
mkdir $odir
dd if=$pdir/PRELOADER of=$odir/preloader.bin bs=$(( 0x800 )) skip=1
./trim.sh $odir/preloader.bin
./trimFF.sh $odir/preloader.bin
dd if=$pdir/DSP_BL of=$odir/DSP_BL bs=$(( 0x8000 )) count=1
./trimFF.sh $odir/DSP_BL
dd if=$pdir/MBR of=$odir/MBR bs=512 count=1
dd if=$pdir/EBR1 of=$odir/EBR1 bs=512 count=1
dd if=$pdir/EBR2 of=$odir/EBR2 bs=512 count=1
cp $pdir/UBOOT $odir/uboot.bin
cp $pdir/LOGO $odir/logo.bin
./trim.sh $odir/logo.bin
cp $pdir/SEC_RO $odir/secro.img
cp $pdir/RECOVERY $odir/recovery.img
cp $pdir/BOOTIMG $odir/boot.img
cp $pdir/FAC $odir/fac.img
cp $pdir/ANDROID $odir/system.img
cp MT*.txt $odir/
And quickly, Here is my hack to remove 0000 and FFFF from the end of files:
trim.sh
Code:
#!/bin/bash
truncate -s $(( 4 + $(cat $1 | hexdump -v -e '/4 "%_ad: " ' -e '4/1 "%02X" "\n"' \
| grep -v ": 00000000" | tail -n 1 | sed 's/:.*//') )) $1
trimFF.sh
Code:
#!/bin/bash
truncate -s $(( 4 + $(cat $1 | hexdump -v -e '/4 "%_ad: " ' -e '4/1 "%02X" "\n"' \
| grep -v ": FFFFFFFF" | tail -n 1 | sed 's/:.*//') )) $1
You should be able to read the clean.sh script and figure out only in just those few cases, is special post processing needed. If you don't post process, SPFT will give errors.
I hope this helps. If you have any questions, ask... I'll try to clarify this first post.

syserr said:
[GUIDE][UTIL][MT65xx] Create Scatter File / Dump Full ROM
For any MT65xx device, no matter how obscure.
Dicing up Full ROM image into partition images
I've made a little bash shell script to dice up a whole ROM according to a scatter file. This creates files exactly the size of the partitions. Some post processing needs done below the script:
dice.sh
Code:
#!/bin/bash
scatterfile=$1
rom=$2
pdir=raw.partitions
rm -rf $pdir
mkdir $pdir
cat $scatterfile | grep "x" | while read line; do
name=$(echo $line | sed 's/ .*//g')
name=$(echo $name | sed 's/^__NODL_//g')
start=$(echo $line | sed 's/.* //g')
# echo $name
if [[ -n $last_name ]]; then
echo "Processing: $last_name"
echo " start: $last_start"
size=$(( $start - $last_start ))
if [[ $size -lt 0 ]]; then
size=0xFFFFF000
else
size=0x$(echo "obase=16; $size" | bc)
fi
echo " size: $size"
short_start=$(echo $last_start | sed 's/000$//g')
short_size=$(echo $size | sed 's/000$//g')
echo dd if=$rom of=$pdir/$last_name bs=$(( 0x1000 )) \
skip=$(( $short_start )) count=$(( $short_size ))
dd if=$rom of=$pdir/$last_name bs=$(( 0x1000 )) \
skip=$(( $short_start )) count=$(( $short_size ))
fi
last_name=$name
last_start=$start
done
Click to expand...
Click to collapse
hi syserr,
i m tryin to run the script dice.sh and this is the trace i get
Code:
[email protected] ~/TESTbcup $ sh -x ./dice.sh
+
: not found2: ./dice.sh:
+ scatterfile=
+ rom=
+ pdir=raw.partitions
+
: not found6: ./dice.sh:
+ rm -rf raw.partitions
+ mkdir raw.partitions
+
: not found9: ./dice.sh:
./dice.sh: 37: ./dice.sh: Syntax error: end of file unexpected (expecting "then")
[email protected] ~/TESTbcup $
any advice would be welcome. I dumb a complete rom and try to dice it.
thank you syserr

fragargon said:
hi syserr,
i m tryin to run the script dice.sh and this is the trace i get
Code:
[email protected] ~/TESTbcup $ sh -x ./dice.sh
+
: not found2: ./dice.sh:
+ scatterfile=
+ rom=
+ pdir=raw.partitions
+
: not found6: ./dice.sh:
+ rm -rf raw.partitions
+ mkdir raw.partitions
+
: not found9: ./dice.sh:
./dice.sh: 37: ./dice.sh: Syntax error: end of file unexpected (expecting "then")
[email protected] ~/TESTbcup $
any advice would be welcome. I dumb a complete rom and try to dice it.
thank you syserr
Click to expand...
Click to collapse
Sorry Sorry Sorry. It takes 2 arguments, I should do some error checking and make sure the user supplies them.
Code:
./dice.sh scatterfile.txt FULLROM.img
It might have some syntax that is specific to bash, so "sh" might throw errors too.

Dumchar.txt
I did dumchar_info, I've renamed it dumchar.txt.
Run script , and it gives me error(He created and an scatter.txt), but empty - see image

Dumchar.txt
What happens and what is the solution?

Alex1948 said:
I did dumchar_info, I've renamed it dumchar.txt.
Run script , and it gives me error(He created and an scatter.txt), but empty - see image
Click to expand...
Click to collapse
I'm sorry, I didn't see this message before, I'm following about 10 threads and this got lost.
I also copied this over from the rua1 thread. I don't think he appreciates our conversation over there, mainly because it's somewhat off topic.
Alex1948 said:
For @syserr
I do not know what all you write here, Python, etc.start adress + lenght , I think we know how to add two numbers in base 16 , no other ....MTKdroid is a software that solves everything,In Python a script of yours, gives me error.
I posted a picture of treadh and you did not answer why(Or you can not quite explain , and we do not understand) - see image
Click to expand...
Click to collapse
Answers:
I've never run it without putting it in a file. Put that python text in dumchar2scatter.py, or something like that. I'm 99% sure that will get rid of the error and give you stuff in your scatter file. You are using Python2, that is good!
You are free to PM me why you feel I didn't see a post. Again, sorry.
Because I think you are genuinely interested in learning, I will describe what my little script is doing. (btw, this is about my third program/script in Python ever)
Code:
import sys
import string
import re
ins = open( "dumchar.txt", "rb" ) [COLOR=Blue]# creates a file handle (ins) to read in dumchar.txt[/COLOR]
outs = open( "scatter.txt", "wb" ) [COLOR=Blue]# creates a file handle (outs) to write to scatter.txt[/COLOR]
for line in ins: [COLOR=Blue]# loops through each line of the input file and puts each line in variable "line"[/COLOR]
linesp = re.split('\W+', line) [COLOR=Blue]# this splits the line varable based on whitespace/spaces, results go into linesp array[/COLOR]
name = linesp[0].upper() [COLOR=Blue]# grabs the first thing in the line, uppercases it, puts it in a varable called name[/COLOR]
start = int(linesp[2],16) [COLOR=Blue]# start variable gets the value of the 3rd thing in the line, but also converts it to an int from text[/COLOR]
block = linesp[5] [COLOR=Blue]# block variable gets 6th thing in line - usually misc or blk[/COLOR]
if block != 'misc': [COLOR=Blue]# if the block variable is NOT 'misc' (only preloader and dsp_bl are these) then ...[/COLOR]
start = start + 0x600000 [COLOR=Blue]# add the offset to the start variable[/COLOR]
outs.write(name + " " + string.replace(hex(start), "L", "") + "\n{\n}\n") [COLOR=Blue]# write out name with start address followed by { } on newlines[/COLOR]
ins.close() [COLOR="Blue"]# just close the file handles here, to clean up, outs might need last writes to be flushed to file.[/COLOR]
outs.close()
I've previewed the code block above on this forum. Why is it so narrow, sorry you will need to use the scrollbar a lot.
UPDATE: Also it's important to have SPACES in a python file, not tabs. And the spaces are critical for python to know code blocks, it acts like {} in Java/C. So, when you make your python file, use spaces to make sure lines line up just right.

syserr said:
I'm sorry, I didn't see this message before, I'm following about 10 threads and this got lost.
I also copied this over from the rua1 thread. I don't think he appreciates our conversation over there, mainly because it's somewhat off topic.
Answers:
I've never run it without putting it in a file. Put that python text in dumchar2scatter.py, or something like that. I'm 99% sure that will get rid of the error and give you stuff in your scatter file. You are using Python2, that is good!
You are free to PM me why you feel I didn't see a post. Again, sorry.
Because I think you are genuinely interested in learning, I will describe what my little script is doing. (btw, this is about my third program/script in Python ever)
Code:
import sys
import string
import re
ins = open( "dumchar.txt", "rb" ) [COLOR=Blue]# creates a file handle (ins) to read in dumchar.txt[/COLOR]
outs = open( "scatter.txt", "wb" ) [COLOR=Blue]# creates a file handle (outs) to write to scatter.txt[/COLOR]
for line in ins: [COLOR=Blue]# loops through each line of the input file and puts each line in variable "line"[/COLOR]
linesp = re.split('\W+', line) [COLOR=Blue]# this splits the line varable based on whitespace/spaces, results go into linesp array[/COLOR]
name = linesp[0].upper() [COLOR=Blue]# grabs the first thing in the line, uppercases it, puts it in a varable called name[/COLOR]
start = int(linesp[2],16) [COLOR=Blue]# start variable gets the value of the 3rd thing in the line, but also converts it to an int from text[/COLOR]
block = linesp[5] [COLOR=Blue]# block variable gets 6th thing in line - usually misc or blk[/COLOR]
if block != 'misc': [COLOR=Blue]# if the block variable is NOT 'misc' (only preloader and dsp_bl are these) then ...[/COLOR]
start = start + 0x600000 [COLOR=Blue]# add the offset to the start variable[/COLOR]
outs.write(name + " " + string.replace(hex(start), "L", "") + "\n{\n}\n") [COLOR=Blue]# write out name with start address followed by { } on newlines[/COLOR]
ins.close() [COLOR="Blue"]# just close the file handles here, to clean up, outs might need last writes to be flushed to file.[/COLOR]
outs.close()
I've previewed the code block above on this forum. Why is it so narrow, sorry you will need to use the scrollbar a lot.
UPDATE: Also it's important to have SPACES in a python file, not tabs. And the spaces are critical for python to know code blocks, it acts like {} in Java/C. So, when you make your python file, use spaces to make sure lines line up just right.
Click to expand...
Click to collapse
OK, I get it("" I don't think he appreciates our conversation over there, mainly because it's somewhat off topic.""
, and why I posted here.
I do not know python, but try to understand more about these phones and their software.
A carefully read what I've written for other questions, as I have, and across from method 1, and there I have some questions.
Especially that last for hours at a READ BACK ,Dump Full ROM , with start adress and lenght by you.
I understand your explanation, written in the program right lines.I did not need them, I know what each instruction written there.
I do not know python but I want to understand .Look - I send you dumchar_info and firmware.info , with partitions start adress and lenght and see what you get.
And finally you get scatter file , my SCATTER FILE , block info - start adress , size , etc.
It's OK. ?As we do not just theory.
I liked your idea, what you wrote but I want to see it completed

Crashes
I did as you said. Look :
Send and dumchar.txt and dumchar2scatter.py
It's ok, I'm sorry you do not answer.I also sent a file and I wanted to enlighten and method 1.
I feel that all that script and dump rom are stupid
I think you should post here on this thred, not on MTKDroid Tool.
Sorry, if you prove me wrong
I'm sorry special excuse but if the script does not work , I posted the file dumchar.txt ,the idea that a fix you
It again sorry, did not mean to upset

Scatter.txt
Successful scatter.txt.As shown in the picture has the same start adress with MTKDroid scatter file , without __NODL_ to 10 partitions.
SP Flash Tool , scatter - loading , load all - see image 2.
For Read Back why use it or one obtained with MTKDroid(see image 3)
Tks.

Alex1948 said:
It's ok, I'm sorry you do not answer.
Click to expand...
Click to collapse
Hi Alex, I figured out why I wasn't catching your posts here. I'm normally looking at my Subscribed threads, and I hadn't subscribed to this thread. I should subscribe and look at my notifications too.
I will look at your messages today. My script is "dumb" as it's looking for a certain path for the block device in the output of dumchar. If that output changes, my script will probably fail.

Alex1948 said:
Successful scatter.txt.As shown in the picture has the same start adress with MTKDroid scatter file , without __NODL_ to 10 partitions.
SP Flash Tool , scatter - loading , load all - see image 2.
For Read Back why use it or one obtained with MTKDroid(see image 3)
Tks.
Click to expand...
Click to collapse
I see it's working for you now. Great. We could make great tools, but MediaTek is changing things that cause problems for even rua1's MTKDRT. Personally, I think it's good to understand things. I'm glad you want to understand too.

Nice share :fingers-crossed:

WHOLE_ROM
4. Run SP Flash Tool, Open Scatter File :
- What scatter file upload ? images 1(that obtained with MTKDroid) or scatter.txt (renamed) and otained with dumchar , script pyton....see images 2. ???
- Start adress is 0x00000000 , but Lenght can pass 0xE7F20000 (start adress FAT=0xC81C0000+FAT lenght=0x1FD60000) END adress FAT ???
With this file, probably higher, than 1GB ,WHOLE_ROM - without extension , What do I do next ?
- to get all the MTKDroid A5_Duo_130806_ForFlashtoolFromReadBack_131007-212823 - see image 3(for this we used start adress 0x00000000 and Lenght 0x3c9c0000 - my CACHE start adress) and My ROM_0 has - 969 MB
And another CWM recovery, more personalized for this phone with can be done?Android-Kitchen-0.224 ? Need 2 files(system.img and boot.img)
I installed and cygwin but does not work, crashes

Alex1948 said:
4. Run SP Flash Tool, Open Scatter File :
- What scatter file upload ? images 1(that obtained with MTKDroid) or scatter.txt (renamed) and otained with dumchar , script pyton....see images 2. ???
- Start adress is 0x00000000 , but Lenght can pass 0xE7F20000 (start adress FAT=0xC81C0000+FAT lenght=0x1FD60000) END adress FAT ???
With this file, probably higher, than 1GB ,WHOLE_ROM - without extension , What do I do next ?
- to get all the MTKDroid A5_Duo_130806_ForFlashtoolFromReadBack_131007-212823 - see image 3(for this we used start adress 0x00000000 and Lenght 0x3c9c0000 - my CACHE start adress) and My ROM_0 has - 969 MB
And another CWM recovery, more personalized for this phone with can be done?Android-Kitchen-0.224 ? Need 2 files(system.img and boot.img)
I installed and cygwin but does not work, crashes
Click to expand...
Click to collapse
#3 and #4 go hand in hand. You just need to make SPFT happy. In all my testing, the scatter and img files can be WRONG... *IF* all you are doing is using "Read back" and "Write memory". It makes sense that to do raw reads and writes, you wouldn't need to know the structure. But this tool forces you to have a scatter and images so it will perform the read/write tasks.
On your second issue, the "FAT" block is not very important and at least on ICS 4.0.4 on my device, if it is all zeros, then on first boot Android will ask to format it. On 4.0.3, I think I had to create a FAT image, which is pretty easy on Linux.

I used this "tool".
But with clean.sh I get a error about DSP_BL. In my scatterfile (MT6589) it is not there. I attached scatterfile.
Is there another "replacement" which i need to clean ?

Thanks for the guide!

Lost calibration data - Unresponsive screen
Hey guys, I have Star N9800 phone. During the last week I've been trying hard to fix my touchscreen driver since I flashed a stock ROM. The problems I was facing were checking and comparing lk.bin (lcm driver) with other people who also have same phone, downloading different ROMs and boot images, tried almost everything. I even opened my phone and disconnected the digitizer to check if it's the problem. When it's connected it says: touchscreen: ili2113a when I check version in factory mode (VOL- + HOME + POWER). When it's disconnected it says: touchscreen: (null), so obviously digitizer is working fine.
Today I think i've found the problem. While using SP Flash Tool I've used manual format from 0x00000000 to whatever it said. It deleted calibration data. And apparently calibration data is stored in nvram.bin which can be backed up using MTK Droid Tools while running a working ROM.
I have flashed original stock ROM now, I'm rooted and I can control my phone from the PC but can't use the screen. It's fully unresponsive. I've checked some answers here on XDA and some guy said I should make a nvram backup of stock ROM and then flash userdata_nvram_only.tar as USERDATA in SP Flash Tool. When I tried unzipping that .tar it clearly has /data/nvram/ folder which also contains some calibration and other files. But when I flashed that .tar as USERDATA my phone isn't booting anymore. I've tried flashing different boot.img but the problem is still here.
Does anybody know how to fix my touchscreen?
I contacted few people on www.NeedRom.com to upload userdata_nvram_only.tar for me, but I don't know whether they are going to do it or not.
I appreciate all help I can get, and I'd seriously hate if I had to send the phone back to china. I wouldn't really do it.

Posted my reply to his PM:
JoeSip said:
Wow you cleared some things for me now. I have Star N9800 smartphone. I've flashed 15 ROMs and tried disconnecting and connecting back the digitizer but screen is still unresponsive. It doesn't react at all. It could be because I used Format option and manual format option in SP Flash tool. Do you have any clue how to get my screen working again? Apparently when doing manual format from adress 0x00000000 calibration data gets removed. Please help me, i'm really desperate.
I can ask people from www.NeedRom.com to help me. They already tried because some of us have the same problems...
You could be the saviour of us all.
Click to expand...
Click to collapse
I think you are totally on track. The NVRAM block/partition is special and it really bugs me that I didn't know this up front when I killed my first MTK device, and people I trusted (ahead of me) didn't talk about it. (FYI, I bought an identical device to repair my device's NVRAM block)
The trick here is... most MTK SP Flash ROMs available don't have the NVRAM block. This is because, they are unique to the device (IMEI and MAC) and if you don't Format with SP Flash, then you don't need them.
What you need... someone with an identical device that is willing to run SP Flash and do a "Read Memory" on the address range of NVRAM block. You would get the address range from the scatter file.

JoeSip said:
Hey guys, I have Star N9800 phone. During the last week I've been trying hard to fix my touchscreen driver since I flashed a stock ROM. The problems I was facing were checking and comparing lk.bin (lcm driver) with other people who also have same phone, downloading different ROMs and boot images, tried almost everything. I even opened my phone and disconnected the digitizer to check if it's the problem. When it's connected it says: touchscreen: ili2113a when I check version in factory mode (VOL- + HOME + POWER). When it's disconnected it says: touchscreen: (null), so obviously digitizer is working fine.
Today I think i've found the problem. While using SP Flash Tool I've used manual format from 0x00000000 to whatever it said. It deleted calibration data. And apparently calibration data is stored in nvram.bin which can be backed up using MTK Droid Tools while running a working ROM.
I have flashed original stock ROM now, I'm rooted and I can control my phone from the PC but can't use the screen. It's fully unresponsive. I've checked some answers here on XDA and some guy said I should make a nvram backup of stock ROM and then flash userdata_nvram_only.tar as USERDATA in SP Flash Tool. When I tried unzipping that .tar it clearly has /data/nvram/ folder which also contains some calibration and other files. But when I flashed that .tar as USERDATA my phone isn't booting anymore. I've tried flashing different boot.img but the problem is still here.
Does anybody know how to fix my touchscreen?
I contacted few people on www.NeedRom.com to upload userdata_nvram_only.tar for me, but I don't know whether they are going to do it or not.
I appreciate all help I can get, and I'd seriously hate if I had to send the phone back to china. I wouldn't really do it.
Click to expand...
Click to collapse
I'll say a couple more things:
Yes, right now your NVRAM block is probably all zeros. And yes, you can use MTK Droid Tools to back up a good NVRAM... I like using SP Flash Tool to read the memory range of the NVRAM block, then I know it's in a shutdown state etc. All MKDRT is doing is running an adb command and doing something like "dd if=/dev/mc***** of=nvram.bin bs=1M count=1"... You could do that too with adb.
You don't need the USERDATA block at all. I would correct your request, to just ask for NVRAM. As USERDATA/USRDATA/DATA (all aliases of the same thing) is a block for your installed apps and data. It is what gets wiped when you do a Factory Reset with typical ROMs or with CWM etc.

Thank you very much man!
So now I'm supposed to just get backup of NVRAM that someone has done in MTK Droid Tools and not the whole ROM?
What if I flash full backup of someone's whole ROM? Will that save me too?

Editing system.img inside super.img and flashing our modifications

Hello and welcome to my first post.
today I will talk about editing super.img and modifying system.img inside of it.
In android 10 and bigger, sometimes there is no system.img in ROM it because google starting use Dynamic Partitions for more flexible images size - more details here
instead of system.img we will see super.img that include few partition inside.
So In this case, in order to do our modifications in the rom we should unpack the super.img and after that to unpack the system.img and then build it again.
requirements:
1) I will use ubuntu in vbox so you need a linux machine.
2) some super.img for editing.
steps:
1) unpacking super.img
2) resizing system.img in order to insert our MODS to the rom
3) mounting system.img
4) do our modifications
5) umounting system
6) shrink edited system.img to the minimal size
7) generating new super.img
8) flashing it to our device
Let's Start!
1) unpacking super image:
First of all the super.img file might be in sparse format so we need to make it raw image
open termianl in super.img directory and type:
Code:
simg2img super.img super.ext4.img
now we got new file named super.ext4.img: we are working with this file.
There are multiple ways to unpack super.img:
for example: using imjtool or using lpunpack
If you use imjtool follow this
Open terminal in imjtool path and super.img path and type
Code:
./imjtool.ELF64 super.img extract
if you got and error run it as superuser by
Code:
sudo ./imjtool.ELF64 super.img extract
I will use lpunpck tool (Official tool from google)
locate lpunpack and super.ext4.img and open terminal in this folder.
Then type:
Code:
./lpunpack super.ext4.img
wait for it it may take couple of minutes.
now our folder looks like this:
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
2)resizing system.img in order to make enough space for modifications
turn back into your terminal and type:
Code:
fallocate -l 2G system.img
This command allocates more space for system.img (Changing to 2GB)
After that type:
Code:
resize2fs system.img 2G
This command increases the file system size of the partition to 2G
3) mounting system.img
Create new folder and mount:
Code:
mkdir system
Code:
sudo mount -t ext4 -o loop system.img system
Now system dir contains all system.img files.
4)edit it as you want
***IMPORTANT***
In order to make changes you should use superuser:
when you typing a command while you modifying use sudo prefix.
you can use the file explorer with superuser permissions by typing:
Code:
sudo nautilus
5) umount system
*)make sure the terminal is not in system directory or some sub dir of it
if Yes just type
Code:
cd ..
to go up in the directories tree (do it many times until you will be in the same directory like the image in the top)
type:
Code:
sudo umount system
Code:
e2fsck -yf system.img
(fixes file system errors)
6) shrink the edit system.img into the minimum possible size
by:
Code:
resize2fs -M system.img
fix the file system again:
Code:
e2fsck -yf system.img
7) generating new super.img
as you can see at the image above, in my case the super.ext4.img contaings 3 partitions:
1)system.img(that we worked with)
2)vendor.img
3)product.img
in you case it may be different so follow the logic of the following command and not copy paste it.
another tool from google that called lpmake using for packing super.img.
How this tool works
it creates empty super.img with all headers stuff and pushing partitions into it
so let's start with generating:
this is the documentation of the tool: link
at first we should get each partitions file size
we can do it by:
Code:
stat -c '%n %s' system.img
do it for all partitions files
And this is tricky and critical step:
***DO NOT COPY!!***
Code:
./lpmake --metadata-size 65536 --super-name super --metadata-slots 1 --device super:4294967296 --group main:3139354624 --partition system:readonly:1434726400:main --image system=./system.img --partition vendor:readonly:330866688:main --image vendor=./vendor.img --partition product:readonly:1373761536:main --image product=./product.img --sparse --output ./super.new.img
explanation:
1) --metadata-size: The maximum size that partition metadata may consume. A partition entry uses 64 bytes and an extent entry uses 16 bytes. I think 65536 should work in most cases.
2) --metadata-slots - The number of slots available for storing metadata. This should match the number of update slots on the device, 2 for A/B devices and 1 for non-A/B.
3)--device super: The size of the “super” partition on the device. It must match exactly, and it must be evenly divisible by the sector size (512 bytes).
4) --group main:4293513600: sum of all partitions files sizes
5) --partition system:readonly:1577095168:main --image system=./system.img : Every parition file size with permissions(readonly) and input img file
good work we got new custom rom!
credit to(for repacking step):
XDA Post: https://forum.xda-developers.com/showpost.php?p=82241115&postcount=70
8) flashing the new rom to a physical device
First of all we should unlock the boot loader without unlocking we may brick the phone
It can be done by allowing oem unlock in developers options and rebooting into bootloader and type:
Code:
fastboot flashing unlock
But it may not work - it depends on your device so check in google.
After you unlocked the bootloader flash the new rom by:
Code:
fastboot flash super super.new.img
If your devices alert for unlocked bootloader(orange state)
you can remove this annoying alert
just search on google.
if you have mediatek check this link:
https://forum.hovatek.com/thread-31664.html

I'm trying to modify my system.img (/system/build.prop) to include support for multi users. After struggling a lot, I've succeeded following your guide (that's an awesome work btw) to unpack, mount, modify, umount and repack super.img. Then, after flashing the new super.img, nothing changes on the phone. I've tried adding and removing files to the system.img before reflashing, and after flashing, i've verified using adb shell. It seem that the image flashed do not contains my modifications. In other words, I succeed to repack a modified super.img, but flashing the new super.img do not modify the phone itself. Any clues? Can you help me?
Phone is Umidigi A7 Pro 64GB

saga1900 said:
I'm trying to modify my system.img (/system/build.prop) to include support for multi users. After struggling a lot, I've succeeded following your guide (that's an awesome work btw) to unpack, mount, modify, umount and repack super.img. Then, after flashing the new super.img, nothing changes on the phone. I've tried adding and removing files to the system.img before reflashing, and after flashing, i've verified using adb shell. It seem that the image flashed do not contains my modifications. In other words, I succeed to repack a modified super.img, but flashing the new super.img do not modify the phone itself. Any clues? Can you help me?
Phone is Umidigi A7 Pro 64GB
Click to expand...
Click to collapse
Hey dear thank you for the compliment, please tell me more about how did you flash the new image.
hey dear did you flash it by using sp flash tool? did you unlocked the bootloader? did you see orange state warning when the device booted after flashing?

Brepro1 said:
Hey dear thank you for the compliment, please tell me more about how did you flash the new image.
hey dear did you flash it by using sp flash tool? did you unlocked the bootloader? did you see orange state warning when the device booted after flashing?
Click to expand...
Click to collapse
Hi, thx for your reply!
I've unlocked the bootloader as you oriented, and after that, I'm seeing the orange warning!I tried flashing using two methods. Using SP Flash Tool, and using flashboot.
Using SP Flash Tool was straight forward. Just selected the Scatter File, then picked the new super.img. Successfully flashed, but no changes on the phone.
Using flashboot: ./flashboot flash super super.modified.img
What a did, in short, was:
1.) simg2img super.img super.raw
2.) imjtool super.raw extract (also tried using lpunpack as you oriented)
3.) mount -t ext4 -o loop system.img /mount/point
4.) edit everything needed
5.) umount /mount/point
6.) repacked super.img using lpmake
7.) flashed the new super.img using SPFT or flashboot. Boths succeeds, but none really modifies the phone
To double check that the new super.img is correct, I've extracted it again, and double checked my changes. and they are there.

Vary strange, I don't have an idea, sorry

Brepro1 said:
Vary strange, I don't have an idea, sorry
Click to expand...
Click to collapse
NP, thanks anyway. I'm reading about a few things like disabling vbmeta verity and other things. Also, I did not mentioned, but I've downloaded and modified a stock rom, and my phone is not rooted. If i found anything that worths mention here, I'll update this thread

i cant mounr system.img . Cant you help me
#sudo mount -t ext4 -o loop system.img system
mount: /home/nam/system: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error

NguyenNhutNam said:
i cant mounr system.img . Cant you help me
#sudo mount -t ext4 -o loop system.img system
mount: /home/nam/system: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error
Click to expand...
Click to collapse
what does the command
file path://system.img
show up?

Anyone can help how to repack an unpacked super.img?
Try to delete an app in the product and system.img and post the instructions to repack everything into a super.img again.
My goal is to remove most of the preinstalled Google apps.
Here is the firmware link
UMIDIGI_A9_PRO_128GB_V1-0_20210127
MediaFire is a simple to use free service that lets you put all your photos, documents, music, and video in a single place so you can access them anywhere and share them everywhere.
www.mediafire.com
How do you get this size, from the scatter file super partition size?
3)--device super: The size of the “super” partition on the device. It must match exactly, and it must be evenly divisible by the sector size (512 bytes).
After repacking I also get this error, is it normal?
'Invalid sparse file format at header magic'

Brepro1 said:
Hello and welcome to my first post.
today I will talk about editing super.img and modifying system.img inside of it.
In android 10 and bigger, sometimes there is no system.img in ROM it because google starting use Dynamic Partitions for more flexible images size - more details here
instead of system.img we will see super.img that include few partition inside.
So In this case, in order to do our modifications in the rom we should unpack the super.img and after that to unpack the system.img and then build it again.
requirements:
1) I will use ubuntu in vbox so you need a linux machine.
2) some super.img for editing.
steps:
1) unpacking super.img
2) resizing system.img in order to insert our MODS to the rom
3) mounting system.img
4) do our modifications
5) umounting system
6) shrink edited system.img to the minimal size
7) generating new super.img
8) flashing it to our device
Let's Start!
1) unpacking super image:
First of all the super.img file might be in sparse format so we need to make it raw image
open termianl in super.img directory and type:
Code:
simg2img super.img super.ext4.img
now we got new file named super.ext4.img: we are working with this file.
There are multiple ways to unpack super.img:
for example: using imjtool or using lpunpack
If you use imjtool follow this
Open terminal in imjtool path and super.img path and type
Code:
./imjtool.ELF64 super.img extract
if you got and error run it as superuser by
Code:
sudo ./imjtool.ELF64 super.img extract
I will use lpunpck tool (Official tool from google)
locate lpunpack and super.ext4.img and open terminal in this folder.
Then type:
Code:
./lpunpack super.ext4.img
wait for it it may take couple of minutes.
now our folder looks like this:
2)resizing system.img in order to make enough space for modifications
turn back into your terminal and type:
Code:
fallocate -l 2G system.img
This command allocates more space for system.img (Changing to 2GB)
After that type:
Code:
resize2fs system.img 2G
This command increases the file system size of the partition to 2G
3) mounting system.img
Create new folder and mount:
Code:
mkdir system
Code:
sudo mount -t ext4 -o loop system.img system
Now system dir contains all system.img files.
4)edit it as you want
***IMPORTANT***
In order to make changes you should use superuser:
when you typing a command while you modifying use sudo prefix.
you can use the file explorer with superuser permissions by typing:
Code:
sudo nautilus
5) umount system
*)make sure the terminal is not in system directory or some sub dir of it
if Yes just type
Code:
cd ..
to go up in the directories tree (do it many times until you will be in the same directory like the image in the top)
type:
Code:
sudo umount system
Code:
e2fsck -yf system.img
(fixes file system errors)
6) shrink the edit system.img into the minimum possible size
by:
Code:
resize2fs -M system.img
fix the file system again:
Code:
e2fsck -yf system.img
7) generating new super.img
as you can see at the image above, in my case the super.ext4.img contaings 3 partitions:
1)system.img(that we worked with)
2)vendor.img
3)product.img
in you case it may be different so follow the logic of the following command and not copy paste it.
another tool from google that called lpmake using for packing super.img.
How this tool works
it creates empty super.img with all headers stuff and pushing partitions into it
so let's start with generating:
this is the documentation of the tool: link
at first we should get each partitions file size
we can do it by:
Code:
stat -c '%n %s' system.img
do it for all partitions files
And this is tricky and critical step:
***DO NOT COPY!!***
Code:
./lpmake --metadata-size 65536 --super-name super --metadata-slots 1 --device super:4294967296 --group main:3139354624 --partition system:readonly:1434726400:main --image system=./system.img --partition vendor:readonly:330866688:main --image vendor=./vendor.img --partition product:readonly:1373761536:main --image product=./product.img --sparse --output ./super.new.img
explanation:
1) --metadata-size: The maximum size that partition metadata may consume. A partition entry uses 64 bytes and an extent entry uses 16 bytes. I think 65536 should work in most cases.
2) --metadata-slots - The number of slots available for storing metadata. This should match the number of update slots on the device, 2 for A/B devices and 1 for non-A/B.
3)--device super: The size of the “super” partition on the device. It must match exactly, and it must be evenly divisible by the sector size (512 bytes).
4) --group main:4293513600: sum of all partitions files sizes
5) --partition system:readonly:1577095168:main --image system=./system.img : Every parition file size with permissions(readonly) and input img file
good work we got new custom rom!
credit to(for repacking step):
XDA Post: https://forum.xda-developers.com/showpost.php?p=82241115&postcount=70
8) flashing the new rom to a physical device
First of all we should unlock the boot loader without unlocking we may brick the phone
It can be done by allowing oem unlock in developers options and rebooting into bootloader and type:
Code:
fastboot flashing unlock
But it may not work - it depends on your device so check in google.
After you unlocked the bootloader flash the new rom by:
Code:
fastboot flash super super.new.img
If your devices alert for unlocked bootloader(orange state)
you can remove this annoying alert
just search on google.
if you have mediatek check this link:
https://forum.hovatek.com/thread-31664.html
Click to expand...
Click to collapse
when I put the command line
./lpmake --metadata-size 65536 --super-name super --metadata-slots 2 --device super: 6836715520 --group main: 6642450432 --partition system: readonly: 5244977152: main --image system =. / system.img --partition odm: readonly: 4349952: main --image odm =. / odm.img --partition product: readonly: 752545792: main --image product =. / product.img --partition vendor: readonly: 640577536: main --image vendor =. / Vendor.img --sparse --output ./super_new.img
it says this: Invalid sparse file format at header magic
why ????

I edited the vendor.img but something strange happens:
$stat -c '%n %s' *
super.img 3758096384
product.img 1596944384
system.img 1128718336
vendor.img 544976896
$../otatools/bin/lpmake --metadata-size 65536 --super-name super --metadata-slots 1 --device super:3758096384 --group main:3270639616 --partition system:readonly:1128718336:main --image system=./system.img --partition vendor:readonly:544976896:main --image vendor=./vendor.img --partition product:readonly:1596944384:main --image product=./product.img --sparse --output ./super.new.img
lpmake I 02-17 12:18:27 2646704 2646704 builder.cpp:1012] [liblp]Partition system will resize from 0 bytes to 1128718336 bytes
lpmake I 02-17 12:18:27 2646704 2646704 builder.cpp:1012] [liblp]Partition vendor will resize from 0 bytes to 544976896 bytes
lpmake I 02-17 12:18:27 2646704 2646704 builder.cpp:1012] [liblp]Partition product will resize from 0 bytes to 1596944384 bytes
Invalid sparse file format at header magic
Invalid sparse file format at header magic
Invalid sparse file format at header magic
BUT....
$stat -c '%n %s' super.new.img
super.new.img 3248851200
which is not divisible by 512!
Shouldn't it be 3758096384 ?

This is for android 10 custom os. Or it will work on all android 10 mobile. Can we modify FRP partition using this.

Me too, sorry

@Brepro1 thank you so much for writing this detailed guide.
Thanks to your detailed guide I was able to create an automated bash script that performs all of these steps automatically and makes all read only partitions inside super.img (system, vendor , product, etc...) into read write-able partitions again and flash to device as a brand new super.img.
It would be an honor for me if you could please try it and let me know if it works on your device. Thanks.
Here is the link:
https://forum.xda-developers.com/t/script-mount-system-as-read-write-android-10.4240703/

I have same issue as already was mentioned by NguyenNhutNam on Jan-20, i.e.:
In response to
sudo mount -t ext4 -o loop system.img edit
I'm getting this:
wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
And Linux tells me this:
file system.img
system.img: Linux rev 1.0 ext2 filesystem data, UUID=4729639d-b5f2-5cc1-a120-9ac5f788683c (extents) (large files) (huge files)
Of course, I tried:
sudo mount -t ext2 -o loop system.img edit
only to get this:
wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
Any ideas?

Figured out the reason.
Topicstarter confuses people with incorrect instructions.

The code below works great in Ubuntu
Code:
sudo su
mkdir /mnt/dir
mount -t ext4 -o loop,rw ./system.img /mnt/dir
But as soon as I try it directly inside Android I get this error:
Code:
mount: '/dev/block/loop10'->'/mnt/dir': Block device required
I guess this must be some kind of Android limitation...

use busybox mount applet instead of toybox

Brepro1 said:
Hello and welcome to my first post.
today I will talk about editing super.img and modifying system.img inside of it.
In android 10 and bigger, sometimes there is no system.img in ROM it because google starting use Dynamic Partitions for more flexible images size - more details here
Click to expand...
Click to collapse
None of this is working on the Moto One 5g Ace. All I want to do is get into /product/media/audio/ringtones to delete the crappy ones like I do with EVERY VERSION OF ANDROID and replace them with my own. I can't even open the images up and mod them because of the Read only xx. Please help and give detailed instructions like I'm a 5 year old. Have tried on linux and windows to ZERO success and LeBigMac's script didn't do xxxxx either.
Mod Edit: Post edited.

why don't use magisk native systemless method? just overlay magisk module with desired mods
https://topjohnwu.github.io/Magisk/guides.html#easy-replace

For those who can't mount system image see this twitter conversation by
https://twitter.com/i/web/status/1170404631865778177