Related
Hi,
Because I don't run Windows nor NetBSD, I rewrote rkflash from scratch with the use of libusb-1.0, so you can now read and write your rk2818-based tablet's flash memory under Linux (also w/o the need to root your tablet). Credit for reverse-engineering the protocol goes to the original author of rkflash (see source).
Small guide
- unzip the file
- compile
Linux (Debian, Ubuntu, ...)
Code:
sudo apt-get install libusb-1.0-0-dev
gcc -o rkflashtool rkflashtool.c -lusb-1.0 -O2 -W -Wall -s
Mac OS X (thanks to surfer63, binary here)
Code:
sudo port install libusb
gcc -I/opt/local/include -I/opt/local/include/libusb-1.0 \
-L/opt/local/lib -o rkflashtool rkflashtool.c -lusb-1.0 -O2 -W -Wall
Preparation
- powerdown your tablet
- disconnect all cables
To get into flash mode differs for many tablets. Google around or use trial and error
- insert the USB cable in computer
- hold vol+ (or put on/off/locked-switch in the locked position)
- insert the other end of your cable in the tablet
- wait a few seconds
- release vol+
Now if you run lsusb, the following line should appear:
Bus 001 Device 044: ID 2207:281a (290a for rk2918 based tablets)
Bus and device number may be different. The screen of your tablet stays black.
The USB device must be readable and writable for the user running rkflashtool. If that's not the case, you'll see an error like this:
Code:
$ ./rkflashtool b
libusb couldn't open USB device /dev/bus/usb/001/048: Permission denied.
libusb requires write access to USB device nodes.
rkflashtool: fatal: cannot open device
This can be fixed in several ways (chmod, run as root, udev rules) but that's beyond the scope of this posting. For now, chmod 666 the device mentioned in the error message.
Usage of rkflashtool
Code:
$ ./rkflashtool
rkflashtool: fatal: usage:
rkflashtool b reboot device
rkflashtool r offset size >file read flash
rkflashtool w offset size <file write flash
offset and size are in units of 512 bytes
On my tablet, the boot partition starts at offset 0x8000 (in blocks of 512 bytes)
Its size is 0x2000 blocks
To backup the partition, issue:
Code:
$ ./rkflashtool r 0x8000 0x2000 >boot.img.backup
rkflashtool: info: interface claimed
rkflashtool: info: reading flash memory at offset 0x00008000
rkflashtool: info: reading flash memory at offset 0x00008020
.......
rkflashtool: info: reading flash memory at offset 0x00009fe0
To write a new boot.img or an old backup back to the device:
Code:
$ ./rkflashtool w 0x8000 0x2000 <boot.img.backup
rkflashtool: info: interface claimed
rkflashtool: info: writing flash memory at offset 0x00008000
rkflashtool: info: writing flash memory at offset 0x00008020
.......
rkflashtool: info: writing flash memory at offset 0x00009fe0
You can find a list of all partitions of your tablet in the HWDEF file, which is inside the update.img for your tablet. If no such file is available, you can also look at /proc/cmdline on a running device (either through adb or a terminal app running on the device itself). Depending on the tablet, you might need root access to view /proc/cmdline. Another option is dumping the first 0x2000 blocks of nand flash by issuing rkflashtool r 0x0000 0x2000 >parm. View the file with hexedit, xxd, or a similar program. The kernel parameters contain a description of several mtd partitions (sizes and offsets).
After reading and writing at will, you can reboot your tablet by issuing
./rkflashtool b
Note that if your tablet has an on/off/locked-switch and it is still in the locked position, rebooting won't work.
If the file you are writing is smaller than the specified size, the rest is padded with zeroes. If it's bigger, it will be truncated. This is different from rkflash, which will overwrite blocks beyond the partition size.
rkflashtool does not support flashing a new bootloader directly.
If you have a different tablet, please try rkflashtool b and r first before flashing (w) something new.
Standard DISCLAIMER with regard to bricking your tablet applies.
Enjoy!
EDIT: better build instructions, clean up text
EDIT2: works on rk2918 tablets too (tested on Arnova 7 G2) if you change the USB product id from 0x281a to 0x290a before compilation
EDIT3: released version 2 of rkflashtool. now supports rk2918 tablets out of the box. if it doesn't find one, it falls back to rk2808/rk2818. also, updated the wording a bit.
EDIT4: new mac osx binary
EDIT5: more ways to find offsets and sizes of partitions on your tablet
EDIT6: small emphasis changes above and...
version 1 is here ONLY for archival purposes or if version 2 does not work on your rk28xx tablet. In all other cases, you need to download rkflashtool-v2.zip
Thanks a lot for this flash tool. I'm on MacOSX and Ubuntu and don't have Windows either. I tried the original rkflash as well but couldn't get it to work. On my Ubuntu boxes your rkflashtool compiles and works fine.
My Archos 7 HT V2 presents itself also as:
Code:
Bus 002 Device 004: ID 2207:281a
Reading partitions works fine and so does writing.
I did a quick modification of a system.img (left some files out) of my custom froyo rom and wrote it to my tablet.
That works fine. As /data is a separate partition I even have all my downloaded apps, data, settings, etc. This makes modifying a new rom much faster then building a complete update.img, flashing it, restore some data and then start testing.
Nice work.
great! finally I can remove one line from my todo list
thank you!
EDIT:
random notes (I don't see your code yet so it may be fixed, then sorry)
* I always specify b(reboot) for rk2818 tablets with my rkflash because it hanged easily if I try to write multiple times without b
* parameter file need to be converted with rkcrc -p. official RKAndroid tools flashed it 5 times with offsets. (read & check 1st 0x0-0x2000 block)
* I logged how to update bootloader, but it's complicated and I could not understand probably bootloader can be updated via misc partition. see update-script in update.img. (but not recommended/no reason to do it)
EDIT2:
there is libusb for Windows and OS X. rkflashtool may work on them.
on Windows, there is RKAndroidTool.exe (not batchupgrade). but "read" function in rkflash/rkflashtool may be useful on some case on Windows
Good to hear it works for others, too! I have not had a hanging tablet after several writes in one session, but this might depend on the tablet.
Thanks for mentioning that it should also work on other platforms supported by libusb. I'd forgotten to do that.
About using update.img to flash a new bootloader, this can be done, but if you brick the tablet by flashing a wrong/faulty bootloader, you can only unbrick it with the Windows tools
Which leads me to the question: could you send me the snooped log of updating the bootloader? Two people see more than one and perhaps we can eventually manage to do this through libusb too.
ivop said:
About using update.img to flash a new bootloader, this can be done, but if you brick the tablet by flashing a wrong/faulty bootloader, you can only unbrick it with the Windows tools
Click to expand...
Click to collapse
probably you also need a needle to short pins of NAND chip
so I don't recommend to flash bootloader
ivop said:
Which leads me to the question: could you send me the snooped log of updating the bootloader? Two people see more than one and perhaps we can eventually manage to do this through libusb too.
Click to expand...
Click to collapse
I made that log several months ago with another windows machine which is not used lately. I'm not sure log is still exist... if I find it, I'll send it to you (but please don't expect)
probably you may also get log on Windows on VM on Linux. it seems VMware has log function (refer http://vusb-analyzer.sourceforge.net/tutorial.html) or there is "usbmon" function in Linux.
actually I didn't try this way myself so it may be wrong, sorry.
I've tryed a couple of firmwares, cooking my own.
Every time after flashing, tablet shows boot animation and after few seconds display becomes dark.
My investigation led me to following:
Log shows:
Code:
ERROR/Lights(865): write_int failed to open /sys/class/backlight/rk28_button_light/brightness
in /sys/class/backlight I found symlink (rk28_bl):
rk28_bl -> ../../devices/platform/rk28_backlight/backlight/rk28_bl
Shouldn't be there another symlink named r28_button_light ?
I'm using MANTA MID001 from Poland.
fun_ said:
EDIT2:
there is libusb for Windows and OS X. rkflashtool may work on them.
Click to expand...
Click to collapse
ivop said:
Good to hear it works for others, too! I have not had a hanging tablet after several writes in one session, but this might depend on the tablet.
Click to expand...
Click to collapse
I did a couple of successive writes as well from ubuntu.
ivop said:
Thanks for mentioning that it should also work on other platforms supported by libusb. I'd forgotten to do that.
Click to expand...
Click to collapse
My main platform is OSX and I immediately added libusb. So far I have not been able to compile rkflashtool despite declaring all kind of CFLAGS, CXXFLAGS and/or LDFLAGS.
Trying a little bit more.
Could you post the compiler warnings/errors here? I might be able to help out.
ivop said:
Could you post the compiler warnings/errors here? I might be able to help out.
Click to expand...
Click to collapse
I managed to compile it. It took a lot of hurdles. I used the build environment I also use for Hugin for which I'm the OSX maintainer.
I now built a single combined 32/64bit (i386/x86_64) rkflashtool that will run on 10.4.x/10.5.x/10.6.x/10.7.x (building multi-architecture, multi-version binaries/libraries in one binary/library is possible on OSX. I'm not going to explain that here but it's a feature of OSX).
The compiled version is attached. You can also attach it to your first post if you like.
It works fine. I did some reading/writing of images without issues.
If you are on OSX and have macports installed, you can do the following to build rkflashtool.
Install libusb from Macports:
Code:
sudo port install libusb
cd into the folder where your rkflashtool.c is is and run the following command:
Code:
gcc -I/opt/local/include -I/opt/local/include/libusb-1.0 \
-L/opt/local/lib -o rkflashtool rkflashtool.c -lusb-1.0 -W -Wall
This will build rkflashtool for your native environment (OSX version, hardware and config).
--- removed the rest of the post as well as the attachments. He/She who is interested in building a complete universal distributable rkflashtool can ask via this thread ---
UPDATE: Works on rk2918 tablet too
Yesterday I have tested the tool on an Arnova 7 G2 tablet, which has an rk2918 CPU. If you change the ProductID before compilation, like this:
... libusb_open_device_with_vid_pid(c, 0x2207, 0x281a) ...
to
... libusb_open_device_with_vid_pid(c, 0x2207, 0x290a) ...
it'll work, except for rebooting the device if the tablet is still locked. To boot the tablet in bootloader mode, turn off the tablet completely, put the on/off-switch in the locked position and connect it to your computer. It should be visible now with lsusb. For further instructions, see first post. I advise dumping the first 0x2000 blocks at offset 0x0000 first as this contains the parameter block in which you can see where each partition starts and how big it is.
ivop said:
UPDATE: Works on rk2918 tablet too
Yesterday I have tested the tool on an Arnova 7 G2 tablet, which has an rk2918 CPU. If you change the ProductID before compilation, like this:
... libusb_open_device_with_vid_pid(c, 0x2207, 0x281a) ...
to
... libusb_open_device_with_vid_pid(c, 0x2207, 0x290a) ...
Click to expand...
Click to collapse
Feature request :
I's nice but could you also make it a startup option, like the b,r,w options, with an if-else option in the source code? Something like (RK)2818 and (RK)2918 and maybe even for the older ones: (RK)2808.
In that case you only need one binary. Users who are going to use the tool will definitely know what CPU they have.
surfer63 said:
Feature request :
I's nice but could you also make it a startup option, like the b,r,w options, with an if-else option in the source code? Something like (RK)2818 and (RK)2918 and maybe even for the older ones: (RK)2808.
In that case you only need one binary. Users who are going to use the tool will definitely know what CPU they have.
Click to expand...
Click to collapse
I released a new version and updated the first post. It now tries to connect to an rk2918 tablet and if it doesn't find one, it falls back to rk2818.
The V2 version works fine too on MacOSX. The compilation is still the same for a "my machine only" version.
I compiled a universal Intel 32bit/64bit 10.4/10.5/10.6/10.7 V2 version as well.
See attached.
Note: I don't have a RK2918 so I can only test for a RK2818 tablet.
Hi,
Thanks for your thread it's very intersting.
I succeed flashing my boot partition with your tool but I don't success in remount,rw my system partition. It's cramFS and in init.rk28board.rc you can see those line :
Code:
# Mount /system rw first to give the filesystem a chance to save a checkpoint
mount cramfs [email protected] /system
mount cramfs [email protected] /system ro remount
I tried everything like replacing ro by rw, deleting the second line but my system stills in ReadOnly, don't understand why. I also tried deleting those lines to test if my flash process works properly and it's worked... So I'm lost. Any idea ?
----
Other thing, if I want to do same as flashing boot partition but with system partition is it possible with the same process ? Unfortunately I don't know the beginning offset of the partition. I don't know where to find HWDEF file. The size of partition is 00038000 (hex) bytes => 229376 (dec) bytes
Here is my /proc/mtd :
Code:
dev: size erasesize name
mtd0: 00002000 00000010 "misc"
mtd1: 00004000 00000010 "kernel"
mtd2: 00002000 00000010 "boot"
mtd3: 00004000 00000010 "recovery"
mtd4: 00038000 00000010 "system"
mtd5: 0003a000 00000010 "backup"
mtd6: 0003a000 00000010 "cache"
mtd7: 00080000 00000010 "userdata"
mtd8: 00534000 00000010 "user"
mtd9: 00020000 00000010 "pagecache"
mtd10: 00020000 00000010 "swap"
Thank you for your great job
My problem is solved. I was searching for a while but ivop gave the answer in a previous post
I advise dumping the first 0x2000 blocks at offset 0x0000 first as this contains the parameter block in which you can see where each partition starts and how big it is.
Click to expand...
Click to collapse
So I did it, after I opened an Hex Editor like GHex on Ubuntu and I can saw this :
Code:
[email protected](misc),
[email protected](kernel),
[email protected](boot),
[email protected](recovery),
[email protected](system),
[email protected](backup),
[email protected](cache),
[email protected](userdata),
[email protected](user)
So system partition starts at E000 and has a length of 38000 (hex) bytes.
Thanks for your help this thread is now in my bookmarks
And really nice job with this flashtool
I pushed latest my rkutils to https://github.com/naobsd/rkutils
rkunpack can unpack RKFW image used in RK2918 ROM, RKAF image (update.img), KRNL/PARM image used in some single partition image. unpack will be done recursively.
rkcrc can make KRNL/PARM images with -k/-p.
rkafpack can make RKAF image. (I need to write docs/howtos...)
little off-topic,
latest RK2918 ROMs which is based on "SDK2.0", new format for boot.img/recovery.img is introduced. it's almost same as common boot.img format for android. unpackbootimg/mkbootimg can be used to unpack/repack it with one exception...
there is SHA1 hash value in header of boot.img (offset 0x240 bytes). Rockchip changes it by some unknown way. normal mkbootimg can't generate same hash value as Rockchip, so we can't make custom boot.img with new format
fortunately, we can split new boot.img, and we can make separate kernel.img and boot.img(ramdisk) like as pre-SDK2.0 RK2918 ROMs, which is loadable with new bootloader in SDK2.0 ROMs.
--
btw I just found interesting one: https://github.com/jhonxie/rk2918_tools
relsyou said:
My problem is solved. I was searching for a while but ivop gave the answer in a previous post
So I did it, after I opened an Hex Editor like GHex on Ubuntu and I can saw this :
Code:
[email protected](misc),
[email protected](kernel),
[email protected](boot),
[email protected](recovery),
[email protected](system),
[email protected](backup),
[email protected](cache),
[email protected](userdata),
[email protected](user)
So system partition starts at E000 and has a length of 38000 (hex) bytes.
Thanks for your help this thread is now in my bookmarks
And really nice job with this flashtool
Click to expand...
Click to collapse
I'll add that to my first post. Also, you can view /proc/cmdline to see a list of partitions. It's part of the kernel command line.
Note that the lengths are not in bytes but in blocks of 512 bytes. This happens to be the same as the requirements of the rkflashtool btw (length in blocks).
As for having a writable system partition, currently the system partition is cramfs which cannot be written to. Ever. If you want a writable system partition, you need to change it to ext3 for example. That means unpacking fun_'s system.img and recreating it as an ext3 partition.
In short:
Unpack cramfs img with cramfsck -x (as root, so you preserve permissions and uid/gid)
Create an empty file the size of your system partition (dd if=/dev/null of=fubar.img bs=512 count=...... et cetera, do the math)
mkfs.ext3 fubar.img
mount -o loop fubar.img /someplacemountable
copy contents of old image to /someplacemountable (use cp -a to preserve ownership etc)
umount
flash fubar.img to system partition
change init.rk28board.rc to reflect the changes
reflash boot.img
reboot device
This is untested, but should work in theory.
Another option is to keep the system partition read-only and use unionfs to overlay a writable partition. I'm not sure if this can be a file on your userdata partition mounted with -o loop, but I suppose it can. This depends on your kernel having unionfs and loopback support though.
fun_ said:
I pushed latest my rkutils to https://github.com/naobsd/rkutils
Click to expand...
Click to collapse
Nice! I was thinking about creating an rkpack(tool ) myself, but I see it's not necessary anymore.
here is an example for rkafpack
Code:
$ rkunpack N3NET-2.3-20110722.img
[COLOR="Red"][B]FIRMWARE_VER:1.0.0[/B][/COLOR]
[COLOR="Red"][B]MACHINE_MODEL:rk2818sdk[/B][/COLOR]
MACHINE_ID:
[COLOR="Red"][B]MANUFACTURER:rock-chips[/B][/COLOR]
unpacking 12 files
-------------------------------------------------------------------------------
00000800-00000fff [COLOR="Red"][B]HWDEF:HWDEF[/B][/COLOR] 797 bytes
00001000-000017ff [COLOR="Red"][B]package-file:package-file[/B][/COLOR] 532 bytes
00001800-00021fff [COLOR="Red"][B]bootloader:RK28xxLoader(L).bin[/B][/COLOR] 131700 bytes
00022000-000227ff [COLOR="Red"][B]parameter:parameter:[email protected][/B][/COLOR] 506 bytes
00022800-0002e7ff [COLOR="Red"][B]misc:Image/misc.img:[email protected][/B][/COLOR] 49152 bytes
0002e800-0066bfff [COLOR="Red"][B]kernel:Image/kernel.img:[email protected][/B][/COLOR] 6541946 bytes
0066c000-006947ff [COLOR="Red"][B]boot:Image/boot.img:[email protected][/B][/COLOR] 163844 bytes
00694800-008e8fff [COLOR="Red"][B]recovery:Image/recovery.img:[email protected][/B][/COLOR] 2441220 bytes
008e9000-085fc7ff [COLOR="Red"][B]system:Image/system.img:[email protected][/B][/COLOR] 131149828 bytes
----------------- [COLOR="Red"][B]backup:SELF:[email protected][/B][/COLOR] (N3NET-2.3-20110722.img) 140498948 bytes
085fc800-085fcfff [COLOR="Red"][COLOR="Red"][B]update-script:update-script[/B][/COLOR][/COLOR] 1294 bytes
085fd000-085fd7ff [COLOR="Red"][B]recover-script:recover-script[/B][/COLOR] 266 bytes
-------------------------------------------------------------------------------
unpacked
$ rkafpack \
[COLOR="Red"][B]FIRMWARE_VER:1.0.0[/B][/COLOR] \
[COLOR="Red"][B]MACHINE_MODEL:rk2818sdk[/B][/COLOR] \
[COLOR="Red"][B]MANUFACTURER:rock-chips[/B][/COLOR] \
[COLOR="Red"][B]HWDEF:HWDEF[/B][/COLOR] \
[COLOR="Red"][B]package-file:package-file[/B][/COLOR] \
'[COLOR="Red"][B]bootloader:RK28xxLoader(L).bin[/B][/COLOR]' \
[COLOR="Red"][B]parameter:parameter:[email protected][/B][/COLOR] \
[COLOR="Red"][B]misc:Image/misc.img:[email protected][/B][/COLOR] \
[COLOR="Red"][B][B]kernel:Image/kernel.img:[email protected][/B][/B][/COLOR] \
[COLOR="Red"][B]boot:Image/boot.img:[email protected][/B][/COLOR] \
[COLOR="Red"][B]recovery:Image/recovery.img:[email protected][/B][/COLOR] \
[COLOR="Red"][B]system:Image/system.img:[email protected][/B][/COLOR] \
[COLOR="Red"][B]backup:SELF:[email protected][/B][/COLOR] \
[COLOR="Red"][B]update-script:update-script[/B][/COLOR] \
[COLOR="Red"][B]recover-script:recover-script[/B][/COLOR] \
> new.img
$ sha1sum N3NET-2.3-20110722.img new.img
e758a6c47dca7f09f0b8a82ad89b0cd7c7c8e826 N3NET-2.3-20110722.img
e758a6c47dca7f09f0b8a82ad89b0cd7c7c8e826 new.img
some values are empty in RK2818 ROM.
--
here is how to make RKFW image
Code:
$ rkunpack N50-2.3-20111103-ZZ-SDK2.0.img
VERSION:2.0.3
unpacking
00000000-00000065 N50-2.3-20111103-ZZ-SDK2.0.img-HEAD 102 bytes
00000066-00022623 N50-2.3-20111103-ZZ-SDK2.0.img-BOOT 140734 bytes
00022624-0c342627 update.img 204603396 bytes
unpacking update.img
================================================================================
FIRMWARE_VER:0.2.3
MACHINE_MODEL:rk29sdk
MACHINE_ID:007
MANUFACTURER:RK29SDK
unpacking 10 files
-------------------------------------------------------------------------------
00000800-00000fff package-file:package-file 540 bytes
00001000-000237ff bootloader:RK29xxLoader(L)_V2.08.bin 140734 bytes
00023800-00023fff parameter:parameter:[email protected] 610 bytes
00024000-0002ffff misc:Image/misc.img:[email protected] 49152 bytes
00030000-006a3fff boot:Image/boot.img:[email protected] 6766592 bytes
006a4000-01167fff recovery:Image/recovery.img:[email protected] 11288576 bytes
01168000-0c31efff system:Image/system.img:[email protected] 186346496 bytes
----------------- backup:SELF:[email protected] (update.img) 204603396 bytes
0c31f000-0c31f7ff update-script:update-script 933 bytes
0c31f800-0c31ffff recover-script:recover-script 266 bytes
-------------------------------------------------------------------------------
================================================================================
00022624-0c342627 N50-2.3-20111103-ZZ-SDK2.0.img-MD5 32 bytes
unpacked
$ cat N50-2.3-20111103-ZZ-SDK2.0.img-HEAD N50-2.3-20111103-ZZ-SDK2.0.img-BOOT update.img > new.img
$ md5sum new.img
[COLOR="Red"][B]5191abc65649eacf8d2476e37d84a046[/B][/COLOR] new.img
$ cat N50-2.3-20111103-ZZ-SDK2.0.img-MD5
5191abc65649eacf8d2476e37d84a046
$ echo -n [COLOR="Red"][B]5191abc65649eacf8d2476e37d84a046[/B][/COLOR] >> new.img
$ sha1sum N50-2.3-20111103-ZZ-SDK2.0.img new.img
3120b13df8886e0ddfae0e35379443c27c925572 N50-2.3-20111103-ZZ-SDK2.0.img
3120b13df8886e0ddfae0e35379443c27c925572 new.img
Hello everybody!
This will be my first XDA-wide release of my new utility for all LG phones.
What is this?
This is a utility to extract the new format KDZ files that LG distributes, specifically the 'compressed' ones.
LG frequently distributes firmware for phones as KDZ files, which are essentially a firmware image of the eMMC and a DLL file that is used by the downloader utility to communicate with the phone.
In the past, there were utilities to extract KDZ files to a DLL file and a DZ file, but no further (at least to my knowledge).
This utility lets you break the KDZ file into it's respective partitions (aboot, rpm, tz, and so on)
What good does this do me?
If you're an phone modder, haxxor, or just an enthusiast that has access to their phone's KDZ file and would like to have a copy of the actual partitions stored within, this will let you.
As an example, firmware for the new LG G2 on many device models is distributed as a KDZ file only.
Other phones use a TOT file, which is essentially a disk image of the eMMC with no compression.
If someone with a KDZ firmware-only phone wiped a partition (for example, modem) and wanted to get it back without flashing the whole phone all over again, they would be stuck.
TOT files are easily extractable, as there is software available currently for that but until today there was none (to my knowledge) for these new KDZ files.
How do I use this?
Glad you asked.
Inside the ZIP file you'll see two Python scripts, KDZFileTools.py and DZFileTools.py.
There's also a README.txt file for more in-depth information if you're curious.
Both scripts respond to --help or -h, so if you're even more curious, try that too!
KDZ files contain DZ files and DLL files, so the first step will be to split those into their respective parts:
LAS_V08d_pre3_00.kdz is the name of the KDZ file that I've copied to the working directory for this example.
Code:
# python KDZFileTools.py -l -f LAS_V08d_pre3_00.kdz
[+] KDZ Partition List
=========================================
0 : LAS_V08d_pre3_00.dz (1428092632 bytes)
1 : LGUP_8974.dll (1477632 bytes)
This shows me that there are two files inside the KDZ file: LAS_V08d_pre3_00.dz and LGUP_8974.dll
You can now extract them by ID by using the -s option, or by using -x to extract all of the files.
Code:
# python KDZFileTools.py -f LAS_V08d_pre3_00.kdz -x
[+] Extracting all partitions!
[+] Extracting LAS_V08d_pre3_00.dz to kdzextracted\LAS_V08d_pre3_00.dz
[+] Extracting LGUP_8974.dll to kdzextracted\LGUP_8974.dll
Now you'll see a folder called "kdzextracted" in your current working directory, which will contain the extracted files.
The next step would be to extract the DZ file to get the partitions it contains:
Code:
# python DZFileTools.py -f kdzextracted/LAS_V08d_pre3_00.dz -l
[+] DZ Partition List
=========================================
0 : PrimaryGPT_0.bin (4299 bytes)
1 : modem_32768.bin (25719664 bytes)
2 : sbl1_163840.bin (179443 bytes)
3 : dbi_165888.bin (10505 bytes)
4 : aboot_229376.bin (288082 bytes)
5 : rpm_231424.bin (93084 bytes)
6 : boot_262144.bin (8959565 bytes)
7 : tz_294912.bin (149388 bytes)
8 : persist_393216.bin (23621 bytes)
9 : recovery_458752.bin (10454494 bytes)
10 : laf_622592.bin (14244284 bytes)
11 : system_7176192.bin (66791740 bytes)
12 : system_7438336.bin (2651 bytes)
13 : system_7440008.bin (2313 bytes)
14 : system_7444120.bin (103727934 bytes)
15 : system_7704592.bin (114239263 bytes)
16 : system_7964296.bin (2313 bytes)
17 : system_7968408.bin (103349001 bytes)
18 : system_8228880.bin (121921125 bytes)
19 : system_8488584.bin (2313 bytes)
20 : system_8492696.bin (101078725 bytes)
21 : system_8753168.bin (125454806 bytes)
22 : system_9012872.bin (2313 bytes)
23 : system_9016984.bin (105806605 bytes)
24 : system_9277456.bin (115830981 bytes)
25 : system_9537160.bin (2313 bytes)
26 : system_9541272.bin (108458465 bytes)
27 : system_9801744.bin (83280847 bytes)
28 : system_10063888.bin (67940827 bytes)
29 : system_10326032.bin (91997923 bytes)
30 : system_10588176.bin (58015487 bytes)
31 : system_10846208.bin (2314 bytes)
32 : system_11108352.bin (2314 bytes)
33 : system_11370496.bin (2314 bytes)
34 : system_11632640.bin (2314 bytes)
35 : system_11894784.bin (2314 bytes)
36 : system_12156928.bin (2314 bytes)
37 : system_12419072.bin (2314 bytes)
38 : system_12681216.bin (2314 bytes)
39 : system_12943360.bin (2314 bytes)
40 : system_13205504.bin (2314 bytes)
41 : system_13467648.bin (2314 bytes)
42 : system_13729792.bin (2652 bytes)
43 : system_13731464.bin (2314 bytes)
44 : BackupGPT_61070336.bin (4286 bytes)
Excellent! All the files are there!
Large images are split up by LG, and can be combined with "cat" or something like that.
The filename actually is in the form "partname_offset.bin" where "offset" is the actual location that the file should be written to on the phone's eMMC (handy!)
You can substitute -l in the options for -x again to extract all the partitions to the folder "dzextracted" in the current working directory as well.
The option --out or -o will change the output directory, so it doesn't have to output to {kdz|dz}extracted
Where can I download this?
Code:
[URL="http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip"]http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip[/URL]
Please don't mirror this file, as having it in one location makes it easy for me to keep it up to date, or pull it if there are problems with this script
It doesn't work? Can you help me?!
Sure, join #lg-g2 on Freenode and ask for IOMonster and I'll try to help out as best I can.
I will not answer questions in PM or via email, please don't PM me about this script!
As usual, if this script eats your dog/cat/uncle please don't blame me either. I didn't mean to do it, I swear!
Whoo! Thanks! It works! How can I say thanks?
"Thanking" this thread is great!
If you feel that I absolutely saved you many hours of time, feel free to click on the "Donate to Me" button next to my username in this post.
This project was funded by a boring day at work and lots of caffiene
Alright guys, let me know if you've got any questions and I'll see what I can do!
Ha! Just what I was looking for but now to get it to work in Windows...
Thanks!
Edit
So I installed Python for Windows and this was the output...
C:\Python33>KDZFileTools.py -l -f F320K11D_00.kdz
File "C:\Python33\KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
C:\Python33>python.exe DZFileTools.py -l -f D80210C_00.kdz
File "DZFileTools.py", line 96
print "[!] Bad DZ sub header!"
^
SyntaxError: invalid syntax
C:\Python33>python.exe KDZFileTools.py -l -f D80210C_00.kdz
File "KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
AndroidUser00110001 said:
Ha! Just what I was looking for but now to get it to work in Windows...
Thanks!
Edit
So I installed Python for Windows and this was the output...
C:\Python33>KDZFileTools.py -l -f F320K11D_00.kdz
File "C:\Python33\KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
C:\Python33>python.exe DZFileTools.py -l -f D80210C_00.kdz
File "DZFileTools.py", line 96
print "[!] Bad DZ sub header!"
^
SyntaxError: invalid syntax
C:\Python33>python.exe KDZFileTools.py -l -f D80210C_00.kdz
File "KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
Click to expand...
Click to collapse
Not sure what's happening here, but I wrote this in Windows, so it works for me...
Try using Python 2.7 ? I haven't tested it with Py3k...
If it still gives you invalid headers after you get rid of the SyntaxError messages, let me know and I'll take a look. I've only verified that this works on a handful of KDZ files.
Thanks for such a awesome tool. After extracting D80210E_00.kdz, I found that system_xxxxxxx.bin files is not contiguous. Merge into one file, you can not mount it.
would you like to develop a tool more for system partitions merge?
honentan said:
Thanks for such a awesome tool. After extracting D80210E_00.kdz, I found that system_xxxxxxx.bin files is not contiguous. Merge into one file, you can not mount it.
would you like to develop a tool more for system partitions merge?
Click to expand...
Click to collapse
Excellent timing I'm actually working on that right now.
It would appear that the KDZ file contains only the 'sparse' data from large partitions.
Basically, it's the same as if you fed the utility "hexdump" a large file with multiple sections will have the empty sections 'collapsed' and shown as a "*".
My utility needs to take the beginning position of the first system_xxxx.bin file into account, and the end system_xxxx.bin 's position and size.
From there, it knows how big the image file needs to be, then it can simply seek to the position in the new file as defined by each .bin file and write it's contents there.
Ultimately, I'd like to have in-place extraction from the KDZ files, without the need to extract the DZ file itself (since it really is just copying the bytes from the right position to a new file on the hdd), and a utility to merge split partitions, like system.
Until then, if you need to create this image yourself, you can do so with a hex editor or dd
1. Take the first system_xxxxx.bin file and write down the value of xxxxx somewhere. We're going to use this as an offset.
2. Take the last system_xxxxx.bin file and write down it's xxxxx as well. Subtract the two. You should now get the size of the system.img file
3. Use "dd" to create a file of proper length: dd if=/dev/zero of=system.img bs=1 count=SIZE_FROM_EARLIER
4. Now, let's put our files into the right places. The first file can just be merged into it with dd if=system_xxxxx.bin of=system.img conv=notrunc
5. For every file after the first one, you'll have to take the xxxxx value from the file, and subtract the offset you found from step 1. use dd if=system_xxxxx.bin of=system.img seek=VALUE_YOU_JUST_FOUND
6. The end file is no different. Subtract it's xxxxx value again and run the above DD command.
If I'm right, this should leave you with a working partition image.
I haven't verified yet, but it seems logical and should work
thanks for your reply. I am going to try the method as you mentioned just now to merge the system.img using winhex, but I have not verified yet too. I will try it tomorrow.
as a advice, new tool should be able to extract and repack new kdz file.
honentan said:
Thanks for such a awesome tool. After extracting D80210E_00.kdz, I found that system_xxxxxxx.bin files is not contiguous. Merge into one file, you can not mount it.
would you like to develop a tool more for system partitions merge?
Click to expand...
Click to collapse
thecubed said:
Excellent timing I'm actually working on that right now.
It would appear that the KDZ file contains only the 'sparse' data from large partitions.
Basically, it's the same as if you fed the utility "hexdump" a large file with multiple sections will have the empty sections 'collapsed' and shown as a "*".
My utility needs to take the beginning position of the first system_xxxx.bin file into account, and the end system_xxxx.bin 's position and size.
From there, it knows how big the image file needs to be, then it can simply seek to the position in the new file as defined by each .bin file and write it's contents there.
Ultimately, I'd like to have in-place extraction from the KDZ files, without the need to extract the DZ file itself (since it really is just copying the bytes from the right position to a new file on the hdd), and a utility to merge split partitions, like system.
Until then, if you need to create this image yourself, you can do so with a hex editor or dd
1. Take the first system_xxxxx.bin file and write down the value of xxxxx somewhere. We're going to use this as an offset.
2. Take the last system_xxxxx.bin file and write down it's xxxxx as well. Subtract the two. You should now get the size of the system.img file
3. Use "dd" to create a file of proper length: dd if=/dev/zero of=system.img bs=1 count=SIZE_FROM_EARLIER
4. Now, let's put our files into the right places. The first file can just be merged into it with dd if=system_xxxxx.bin of=system.img conv=notrunc
5. For every file after the first one, you'll have to take the xxxxx value from the file, and subtract the offset you found from step 1. use dd if=system_xxxxx.bin of=system.img seek=VALUE_YOU_JUST_FOUND
6. The end file is no different. Subtract it's xxxxx value again and run the above DD command.
If I'm right, this should leave you with a working partition image.
I haven't verified yet, but it seems logical and should work
Click to expand...
Click to collapse
I merged system.img using script as you mentioned, but it couldn't be mounted also.
Progress was obtained, I got all files in system.img using ext2explore(version2.1).
Code:
dd if=/dev/zero of=system.img bs=1 count=5505024
dd if=system_819200.bin of=system.img conv=notrunc seek=0
dd if=system_1081344.bin of=system.img conv=notrunc seek=262144
dd if=system_1082760.bin of=system.img conv=notrunc seek=263560
dd if=system_1086808.bin of=system.img conv=notrunc seek=267608
dd if=system_1347536.bin of=system.img conv=notrunc seek=528336
dd if=system_1607048.bin of=system.img conv=notrunc seek=787848
dd if=system_1611096.bin of=system.img conv=notrunc seek=791896
dd if=system_1871824.bin of=system.img conv=notrunc seek=1052624
dd if=system_2131336.bin of=system.img conv=notrunc seek=1312136
dd if=system_2135384.bin of=system.img conv=notrunc seek=1316184
dd if=system_2396112.bin of=system.img conv=notrunc seek=1576912
dd if=system_2655624.bin of=system.img conv=notrunc seek=1836424
dd if=system_2659672.bin of=system.img conv=notrunc seek=1840472
dd if=system_2920400.bin of=system.img conv=notrunc seek=2101200
dd if=system_3179912.bin of=system.img conv=notrunc seek=2360712
dd if=system_3183960.bin of=system.img conv=notrunc seek=2364760
dd if=system_3444688.bin of=system.img conv=notrunc seek=2625488
dd if=system_3706832.bin of=system.img conv=notrunc seek=2887632
dd if=system_3968976.bin of=system.img conv=notrunc seek=3149776
dd if=system_4231120.bin of=system.img conv=notrunc seek=3411920
dd if=system_4493264.bin of=system.img conv=notrunc seek=3674064
dd if=system_4755408.bin of=system.img conv=notrunc seek=3936208
dd if=system_5017552.bin of=system.img conv=notrunc seek=4198352
dd if=system_5275648.bin of=system.img conv=notrunc seek=4456448
dd if=system_5537792.bin of=system.img conv=notrunc seek=4718592
dd if=system_5799936.bin of=system.img conv=notrunc seek=4980736
dd if=system_6062080.bin of=system.img conv=notrunc seek=5242880
dd if=system_6324224.bin of=system.img conv=notrunc seek=5505024
ext2explore log:
Partition Table Error on D:/Android-ROM/980VS/system.img
Invalid End of sector markerBlock size 4096, inp 8064, inodesize 256
Linux Partition found on disk 1 partition 0
Inode 727 with file size 0
thecubed said:
Not sure what's happening here, but I wrote this in Windows, so it works for me...
Try using Python 2.7 ? I haven't tested it with Py3k...
If it still gives you invalid headers after you get rid of the SyntaxError messages, let me know and I'll take a look. I've only verified that this works on a handful of KDZ files.
Click to expand...
Click to collapse
Thanks, I got it working using 2.7 I have everything extracted and now I need to learn how to merge all the system files.
Thanks!
AndroidUser00110001 said:
Thanks, I got it working using 2.7 I have everything extracted and now I need to learn how to merge all the system files.
Thanks!
Click to expand...
Click to collapse
I'm working on a modified version of the DZFileUtils.py script, since the DZ file actually contains the proper information to regenerate the full system.img easier than using DD or a hex editor.
Work has been a little crazy for the past few days, so I may not get a chance to work on the script until next week, but it won't be that hard to have this script put the system.img files back together.
I could modify the whole toolsets to do in-place extraction from a KDZ file (skipping the intermediate DZ file) but for now this is the easiest and accomplishes my goal the fastest (which is to allow extraction of bootloader stacks from KDZ files found on the interwebs)
I have no plans on creating a utility to generate KDZ files, since myself and @Shelnutt2 are in the process of writing a utility to flash images without needing any LG software on your PC.
I used dd for windows and was able to merge all the files with the script that someone posted. Sorry on XDA App and can't see full thread but thanks for the script.
If you guys need help with testing let me know.
Sent from my LG-D800 using XDA Premium 4 mobile app
thecubed said:
Hello everybody!
This will be my first XDA-wide release of my new utility for all LG phones.
What is this?
This is a utility to extract the new format KDZ files that LG distributes, specifically the 'compressed' ones.
LG frequently distributes firmware for phones as KDZ files, which are essentially a firmware image of the eMMC and a DLL file that is used by the downloader utility to communicate with the phone.
In the past, there were utilities to extract KDZ files to a DLL file and a DZ file, but no further (at least to my knowledge).
This utility lets you break the KDZ file into it's respective partitions (aboot, rpm, tz, and so on)
What good does this do me?
If you're an phone modder, haxxor, or just an enthusiast that has access to their phone's KDZ file and would like to have a copy of the actual partitions stored within, this will let you.
As an example, firmware for the new LG G2 on many device models is distributed as a KDZ file only.
Other phones use a TOT file, which is essentially a disk image of the eMMC with no compression.
If someone with a KDZ firmware-only phone wiped a partition (for example, modem) and wanted to get it back without flashing the whole phone all over again, they would be stuck.
TOT files are easily extractable, as there is software available currently for that but until today there was none (to my knowledge) for these new KDZ files.
How do I use this?
Glad you asked.
Inside the ZIP file you'll see two Python scripts, KDZFileTools.py and DZFileTools.py.
There's also a README.txt file for more in-depth information if you're curious.
Both scripts respond to --help or -h, so if you're even more curious, try that too!
KDZ files contain DZ files and DLL files, so the first step will be to split those into their respective parts:
LAS_V08d_pre3_00.kdz is the name of the KDZ file that I've copied to the working directory for this example.
Code:
# python KDZFileTools.py -l -f LAS_V08d_pre3_00.kdz
[+] KDZ Partition List
=========================================
0 : LAS_V08d_pre3_00.dz (1428092632 bytes)
1 : LGUP_8974.dll (1477632 bytes)
This shows me that there are two files inside the KDZ file: LAS_V08d_pre3_00.dz and LGUP_8974.dll
You can now extract them by ID by using the -s option, or by using -x to extract all of the files.
Code:
# python KDZFileTools.py -f LAS_V08d_pre3_00.kdz -x
[+] Extracting all partitions!
[+] Extracting LAS_V08d_pre3_00.dz to kdzextracted\LAS_V08d_pre3_00.dz
[+] Extracting LGUP_8974.dll to kdzextracted\LGUP_8974.dll
Now you'll see a folder called "kdzextracted" in your current working directory, which will contain the extracted files.
The next step would be to extract the DZ file to get the partitions it contains:
Code:
# python DZFileTools.py -f kdzextracted/LAS_V08d_pre3_00.dz -l
[+] DZ Partition List
=========================================
0 : PrimaryGPT_0.bin (4299 bytes)
1 : modem_32768.bin (25719664 bytes)
2 : sbl1_163840.bin (179443 bytes)
3 : dbi_165888.bin (10505 bytes)
4 : aboot_229376.bin (288082 bytes)
5 : rpm_231424.bin (93084 bytes)
6 : boot_262144.bin (8959565 bytes)
7 : tz_294912.bin (149388 bytes)
8 : persist_393216.bin (23621 bytes)
9 : recovery_458752.bin (10454494 bytes)
10 : laf_622592.bin (14244284 bytes)
11 : system_7176192.bin (66791740 bytes)
12 : system_7438336.bin (2651 bytes)
13 : system_7440008.bin (2313 bytes)
14 : system_7444120.bin (103727934 bytes)
15 : system_7704592.bin (114239263 bytes)
16 : system_7964296.bin (2313 bytes)
17 : system_7968408.bin (103349001 bytes)
18 : system_8228880.bin (121921125 bytes)
19 : system_8488584.bin (2313 bytes)
20 : system_8492696.bin (101078725 bytes)
21 : system_8753168.bin (125454806 bytes)
22 : system_9012872.bin (2313 bytes)
23 : system_9016984.bin (105806605 bytes)
24 : system_9277456.bin (115830981 bytes)
25 : system_9537160.bin (2313 bytes)
26 : system_9541272.bin (108458465 bytes)
27 : system_9801744.bin (83280847 bytes)
28 : system_10063888.bin (67940827 bytes)
29 : system_10326032.bin (91997923 bytes)
30 : system_10588176.bin (58015487 bytes)
31 : system_10846208.bin (2314 bytes)
32 : system_11108352.bin (2314 bytes)
33 : system_11370496.bin (2314 bytes)
34 : system_11632640.bin (2314 bytes)
35 : system_11894784.bin (2314 bytes)
36 : system_12156928.bin (2314 bytes)
37 : system_12419072.bin (2314 bytes)
38 : system_12681216.bin (2314 bytes)
39 : system_12943360.bin (2314 bytes)
40 : system_13205504.bin (2314 bytes)
41 : system_13467648.bin (2314 bytes)
42 : system_13729792.bin (2652 bytes)
43 : system_13731464.bin (2314 bytes)
44 : BackupGPT_61070336.bin (4286 bytes)
Excellent! All the files are there!
Large images are split up by LG, and can be combined with "cat" or something like that.
The filename actually is in the form "partname_offset.bin" where "offset" is the actual location that the file should be written to on the phone's eMMC (handy!)
You can substitute -l in the options for -x again to extract all the partitions to the folder "dzextracted" in the current working directory as well.
The option --out or -o will change the output directory, so it doesn't have to output to {kdz|dz}extracted
Where can I download this?
Code:
[URL="http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip"]http://downloads.codefi.re/thecubed/androidtools/Compressed_KDZUtils.zip[/URL]
Please don't mirror this file, as having it in one location makes it easy for me to keep it up to date, or pull it if there are problems with this script
It doesn't work? Can you help me?!
Sure, join #lg-g2 on Freenode and ask for IOMonster and I'll try to help out as best I can.
I will not answer questions in PM or via email, please don't PM me about this script!
As usual, if this script eats your dog/cat/uncle please don't blame me either. I didn't mean to do it, I swear!
Whoo! Thanks! It works! How can I say thanks?
"Thanking" this thread is great!
If you feel that I absolutely saved you many hours of time, feel free to click on the "Donate to Me" button next to my username in this post.
This project was funded by a boring day at work and lots of caffiene
Alright guys, let me know if you've got any questions and I'll see what I can do!
Click to expand...
Click to collapse
Hi, i make my phone soft bricked. no other method works. i need bin file to flash my phone.i have the orignal kdz file.how can i make a single bin file to flash it using LGNPST? If you want to read my whole phone bricking story, please follow this
http://forum.xda-developers.com/showthread.php?t=2492105
AAAAwesome!!!
Thank you very much!
Able to customize LG's ROM easier than ever!
thecubed said:
I'm working on a modified version of the DZFileUtils.py script, since the DZ file actually contains the proper information to regenerate the full system.img easier than using DD or a hex editor.
Work has been a little crazy for the past few days, so I may not get a chance to work on the script until next week, but it won't be that hard to have this script put the system.img files back together.
I could modify the whole toolsets to do in-place extraction from a KDZ file (skipping the intermediate DZ file) but for now this is the easiest and accomplishes my goal the fastest (which is to allow extraction of bootloader stacks from KDZ files found on the interwebs)
I have no plans on creating a utility to generate KDZ files, since myself and @Shelnutt2 are in the process of writing a utility to flash images without needing any LG software on your PC.
Click to expand...
Click to collapse
Any word on the DZfileUtils.py script? I am trying to convert the dz to the system img and all I get is errors when trying to use dd. If there is a better way than dd for windows could someone let me know.
Well I think I have found where my error was at with what I was trying to do.
Thank you for all your hard work on this
honentan said:
I merged system.img using script as you mentioned, but it couldn't be mounted also.
Progress was obtained, I got all files in system.img using ext2explore(version2.1).
ext2explore log:
Partition Table Error on D:/Android-ROM/980VS/system.img
Invalid End of sector markerBlock size 4096, inp 8064, inodesize 256
Linux Partition found on disk 1 partition 0
Inode 727 with file size 0
Click to expand...
Click to collapse
I can confirm I could not mount or use normal methods I have used in the past for getting the system image extracted but I also used ext2explore on windows and it worked like a charm. Thank you
Thanks for tools:good:
but I got this error when I tried with KDZ file.
C:\Users\VM\Desktop\Compressed_KDZUtils>KDZFileTools.py -f L01E20A_00.kdz -x
[!] Error: Unsupported KDZ file format.
[ ] Expected: 0x28 0x5 0x0 0x0 0x34 0x31 0x25 0x80 ,
but received 0x31 0x23 0x53 0x7f 0x89 0x1e 0xe6 0x9b .
Click to expand...
Click to collapse
and when I got DZ file from other KDZ extracter and tried DZ file I got this
C:\Users\VM\Desktop\Compressed_KDZUtils>DZFileTools.py -f L01E20A_00.dz -l
[!] Error: Unsupported DZ file format.
[ ] Expected: 0x32 0x96 0x18 0x74 ,
but received 0x22 0x12 0x84 0x19 .
Traceback (most recent call last):
File "C:\Users\VM\Desktop\Compressed_KDZUtils\DZFileTools.py", line 212, in
<module>
dztools.main()
File "C:\Users\VM\Desktop\Compressed_KDZUtils\DZFileTools.py", line 196, in
main
self.openFile(args.dzfile)
File "C:\Users\VM\Desktop\Compressed_KDZUtils\DZFileTools.py", line 173, in
openFile
sys.exit(0)
NameError: global name 'sys' is not defined
Click to expand...
Click to collapse
I never run python script but I think I did it right.
I'm installed python 2.7. and KDZ file is for Optimus G
rquiett said:
I can confirm I could not mount or use normal methods I have used in the past for getting the system image extracted but I also used ext2explore on windows and it worked like a charm. Thank you
Click to expand...
Click to collapse
honentan said:
I merged system.img using script as you mentioned, but it couldn't be mounted also.
Progress was obtained, I got all files in system.img using ext2explore(version2.1).
Click to expand...
Click to collapse
in linux
Code:
mkdir -p /mnt/dsk
mount -o loop system.img /mnt/dsk
this will return an error
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
Click to expand...
Click to collapse
run dmesg
Code:
dmesg | tail
there will be a line like this:
EXT4-fs (loop0): bad geometry: block count 851968 exceeds size of device (819537 blocks)
Click to expand...
Click to collapse
now truncate the file with the value of the first number
Code:
truncate -o -s 851968
remount, and hey presto.
thank you @Flinny for this
edit: not needed anymore, just ignore it. keeping it here for reference though
merging the bins on linux
ok, so there have been a few issues with getting this done on any platform. I was asked to try to get into a kdz for someone, so I did it
included below is a python script that takes advantage of dd. I've set it up for the system partition (as that's the only one that is split)
I have left all my comments and tweaks in so you can see how it was formed
to run:
Code:
python SystemMerger.py
It should run with python3 (untested). can someone please report any errors and I'll see if I can fix them up.
after running the script, just mount the image, and you're away
Happy Hacking
ps, don't forget to change the extension to .py
ok final version here, should support windows now
https://github.com/cybojenix/random-scripts/blob/master/SystemMerger.py
it should also support python3 still, but someone please check, same with windows (though I'm sceptical about the way I write zeros. might not work on winblows...)
anyway, happy hacking again
cybojenix said:
ok final version here, should support windows now
https://github.com/cybojenix/random-scripts/blob/master/SystemMerger.py
it should also support python3 still, but someone please check, same with windows (though I'm sceptical about the way I write zeros. might not work on winblows...)
anyway, happy hacking again
Click to expand...
Click to collapse
good job! awesome! thanks.
AndroidUser00110001 said:
Ha! Just what I was looking for but now to get it to work in Windows...
Thanks!
Edit
So I installed Python for Windows and this was the output...
C:\Python33>KDZFileTools.py -l -f F320K11D_00.kdz
File "C:\Python33\KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
C:\Python33>python.exe DZFileTools.py -l -f D80210C_00.kdz
File "DZFileTools.py", line 96
print "[!] Bad DZ sub header!"
^
SyntaxError: invalid syntax
C:\Python33>python.exe KDZFileTools.py -l -f D80210C_00.kdz
File "KDZFileTools.py", line 159
print "[!] Error: Unsupported KDZ file format."
^
SyntaxError: invalid syntax
Click to expand...
Click to collapse
just put bracket on each print params, then it should work
like this print ("[!] Error: Unsupported KDZ file format.")
also the KDZFileTools not work on LG G2, the header doesnt match with required
here
i debug to find what is the actual value on verify_header
D80210b.kdz header found
>>> verify_header
b'(\x05\x00\x0041%\x80'
kdz_header = '0x28 0x5 0x0 0x0 0x34 0x31 0x25 0x80'
KDZFileTools error log
[!] Error: Unsupported KDZ file format.
Traceback (most recent call last):
File "C:\Downloads\KDZFileTools.py", line 197, in <module>
kdztools.main()
File "C:\Downloads\KDZFileTools.py", line 182, in main
self.openFile(args.kdzfile)
File "C:\Downloads\KDZFileTools.py", line 160, in openFile
print ("[ ] Expected: %s ,\n\tbut received %s ." % (" ".join(hex(ord) for n in self.kdz_header), " ".join(hex(ord) for n in verify_header)))
File "C:\Downloads\KDZFileTools.py", line 160, in <genexpr>
print ("[ ] Expected: %s ,\n\tbut received %s ." % (" ".join(hex(ord) for n in self.kdz_header), " ".join(hex(ord) for n in verify_header)))
TypeError: ord() expected string of length 1, but int found
Click to expand...
Click to collapse
This is in relation to this and my post on xda.
The main reason I want to make a custom kernel is to gain root and once I successfully have then add other CPU governors, and since this is considered a low activity device on xda I will have to do this myself. Also I you are just gonna say use twrp to flash SuperSU, well I can't as it seems to not work with the device when its running Lollipop 5.1.1
Device Specifications:
Current Android Version: Android Lollipop 5.1.1
Chipset: Marvell Armada PXA1908 (Note: Due to this being a rarely used chip, the CF-Auto root won't work)
Custom Recovery Status: TWRP 3.0.2-0 (More on this later on)
Root Status (This is why I am here): Android KitKat 4.4.4 (Root) , Android Lollipop 5.1.1 (NO ROOT Yet)
ARMv8 64-bit
Now let's get into my steps up to the point and then my problem.
Note: In the kernel readme it states to use the toolchain 4.8 but when I use it, it complains of not being able to find gcc. Also in the read me it states "get Toolchain download and install arm-eabi-4.8 toolchain for ARM EABI.(64bit)" and when reading up on it, https://developer.android.com/ndk/guides/standalone_toolchain.html#syt , it says to use aarch64 for ARM 64 Bit devices.
Device Source Code can be found at Here
Code:
cd ~/android
export CROSS_COMPILE=~/android/ndk/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-
cd ~/android/kernel
make ARCH=arm64 pxa1908_xcover3lte_eur_defconfig
make ARCH=arm64
This outputs: Image, Image.gz, .dts and .dtb files.
Where's the kernel readme (I believe this hasn't been update since kitkat) says the output will be,
- Kernel : Kernel/arch/arm/boot/zImage
- module : Kernel/drivers/*/*.ko
Note: when trying to compile with the 32- bit ARM toolchain it fails, as the config is found in arm64, wheres other configs are found in arm.
So know I have a kernel (Image or Image.gz), and some .dts and .dtb files. Now to unpack boot.img, this is where problems occur. When trying to use tools like abootimg or the various different versions of unmkbootimg, they complain about non-standard boot.img.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
or
While also try saving it as a zImage when its meant to be a Image.gz, or they extract it without throwing any errors, but when looking at the extracted files with a hex editor, it is all 00 throughout the files, therefore a useless file.
So their for I tried manually unpacking with a hex editor and managed to get the kernel. Left is my Compiled and Right is the hex version.
Notice the difference in size, is this because the kernel in boot.img is stripped of its debugging items while mine isn't? If so I should look up on how to fix that.
But am having troubles trying to extract the ramdisk via hex.
So is anyone able to either:
a) Help me extract the boot.img properly (with tools like unmkbootimg or with a hex editor)
or
b) do this for me and explain how you achieved it so I myself is am able to do it when needed.
I have attached necessary files if you want to have a look at them yourself.
Any help is appreciated.
FIles(XDA attachments not working): https://drive.google.com/file/d/0B_5mtquWAP3MZjJQay1ERFprbnM/view?usp=sharing
After Numerous trial and error, I finally managed to output ramdisk.cpio.gz.
The start of a Gzip file in hex is, 1F 8B 08, therefor when using the search function in you had editor application you can narrow down your results to 1 or 2 files (2 Files for me as my kernel and ramdisk are both gzipped). You then follow it all the way down till you find a big bunch of zeros(seems like they are passing between files). When you reach the bunch of zeros include the first "00" at the end of the other hexidecial. E.G. End of one of my gzip files is "CE 24 00 00 00....00 (ZERO PADDING BETWEEN FILES), Threaded the end of my file is "CE 24 00".
Knowing this I was able to successfully extract and verify both my kernel and ramdisk files are correct.
Perl script for unpacking
Code:
#!/usr/bin/perl
######################################################################
#
# File : split_bootimg.pl
# Author(s) : William Enck <[email protected]>
# Description : Split appart an Android boot image created
# with mkbootimg. The format can be found in
# android-src/system/core/mkbootimg/bootimg.h
#
# Thanks to alansj on xda-developers.com for
# identifying the format in bootimg.h and
# describing initial instructions for splitting
# the boot.img file.
#
# Last Modified : Tue Dec 2 23:36:25 EST 2008
# By : William Enck <[email protected]>
#
# Copyright (c) 2008 William Enck
#
######################################################################
use strict;
use warnings;
# Turn on print flushing
$|++;
######################################################################
## Global Variables and Constants
my $SCRIPT = __FILE__;
my $IMAGE_FN = undef;
# Constants (from bootimg.h)
use constant BOOT_MAGIC => 'ANDROID!';
use constant BOOT_MAGIC_SIZE => 8;
use constant BOOT_NAME_SIZE => 16;
use constant BOOT_ARGS_SIZE => 512;
# Unsigned integers are 4 bytes
use constant UNSIGNED_SIZE => 4;
# Parsed Values
my $PAGE_SIZE = undef;
my $KERNEL_SIZE = undef;
my $RAMDISK_SIZE = undef;
my $SECOND_SIZE = undef;
my $DT_SIZE = undef;
######################################################################
## Main Code
&parse_cmdline();
&parse_header($IMAGE_FN);
=format (from bootimg.h)
** +-----------------+
** | boot header | 1 page
** +-----------------+
** | kernel | n pages
** +-----------------+
** | ramdisk | m pages
** +-----------------+
** | second stage | o pages
** +-----------------+
**
** n = (kernel_size + page_size - 1) / page_size
** m = (ramdisk_size + page_size - 1) / page_size
** o = (second_size + page_size - 1) / page_size
** p = (dt_size + page_size - 1) / page_size
**
** 0. all entities are page_size aligned in flash
** 1. kernel and ramdisk are required (size != 0)
** 2. second is optional (second_size == 0 -> no second)
** 3. load each element (kernel, ramdisk, second) at
** the specified physical address (kernel_addr, etc)
** 4. prepare tags at tag_addr. kernel_args[] is
** appended to the kernel commandline in the tags.
** 5. r0 = 0, r1 = MACHINE_TYPE, r2 = tags_addr
** 6. if second_size != 0: jump to second_addr
** else: jump to kernel_addr*/
=cut
my $n = int(($KERNEL_SIZE + $PAGE_SIZE - 1) / $PAGE_SIZE);
my $m = int(($RAMDISK_SIZE + $PAGE_SIZE - 1) / $PAGE_SIZE);
my $o = int(($SECOND_SIZE + $PAGE_SIZE - 1) / $PAGE_SIZE);
my $p = int(($DT_SIZE + $PAGE_SIZE - 1) / $PAGE_SIZE);
my $k_offset = $PAGE_SIZE;
my $r_offset = $k_offset + ($n * $PAGE_SIZE);
my $s_offset = $r_offset + ($m * $PAGE_SIZE);
my $t_offset = $s_offset + ($o * $PAGE_SIZE);
(my $base = $IMAGE_FN) =~ s/.*\/(.*)$/$1/;
my $k_file = "kernel.gz";
my $r_file = "ramdisk.gz";
my $s_file = "second.gz";
my $t_file = "dt.img";
# The kernel is always there
print "Writing $k_file ...";
&dump_file($IMAGE_FN, $k_file, $k_offset, $KERNEL_SIZE);
print " complete.\n";
# The ramdisk is always there
print "Writing $r_file ...";
&dump_file($IMAGE_FN, $r_file, $r_offset, $RAMDISK_SIZE);
print " complete.\n";
# The Second stage bootloader is optional
unless ($SECOND_SIZE == 0) {
print "Writing $s_file ...";
&dump_file($IMAGE_FN, $s_file, $s_offset, $SECOND_SIZE);
print " complete.\n";
}
# The DT.img stage is optional
unless ($DT_SIZE == 0) {
print "Writing $t_file ...";
&dump_file($IMAGE_FN, $t_file, $t_offset, $DT_SIZE);
print " complete.\n";
}
######################################################################
## Supporting Subroutines
=header_format (from bootimg.h)
struct boot_img_hdr
{
unsigned char magic[BOOT_MAGIC_SIZE];
unsigned kernel_size; /* size in bytes */
unsigned kernel_addr; /* physical load addr */
unsigned ramdisk_size; /* size in bytes */
unsigned ramdisk_addr; /* physical load addr */
unsigned second_size; /* size in bytes */
unsigned second_addr; /* physical load addr */
uint32_t dt_size; /* device tree size in bytes */
uint32_t dt_addr; /* device tree address in bytes */
unsigned tags_addr; /* physical addr for kernel tags */
unsigned page_size; /* flash page size we assume */
unsigned char name[BOOT_NAME_SIZE]; /* asciiz product name */
unsigned char cmdline[BOOT_ARGS_SIZE];
unsigned id[8]; /* timestamp / checksum / sha1 / etc */
};
=cut
sub parse_header {
my ($fn) = @_;
my $buf = undef;
open INF, $fn or die "Could not open $fn: $!\n";
binmode INF;
# Read the Magic
read(INF, $buf, BOOT_MAGIC_SIZE);
unless ($buf eq BOOT_MAGIC) {
die "Android Magic not found in $fn. Giving up.\n";
}
# Read kernel size and address (assume little-endian)
read(INF, $buf, UNSIGNED_SIZE * 2);
my ($k_size, $k_addr) = unpack("VV", $buf);
# Read ramdisk size and address (assume little-endian)
read(INF, $buf, UNSIGNED_SIZE * 2);
my ($r_size, $r_addr) = unpack("VV", $buf);
# Read second size and address (assume little-endian)
read(INF, $buf, UNSIGNED_SIZE * 2);
my ($s_size, $s_addr) = unpack("VV", $buf);
# Read dt size and address (assume little-endian)
read(INF, $buf, UNSIGNED_SIZE * 2);
my ($t_size, $t_addr) = unpack("VV", $buf);
# Ignore tags_addr
read(INF, $buf, UNSIGNED_SIZE);
# get the page size (assume little-endian)
read(INF, $buf, UNSIGNED_SIZE);
my ($p_size) = unpack("V", $buf);
# Read the name (board name)
read(INF, $buf, BOOT_NAME_SIZE);
my $name = $buf;
# Read the command line
read(INF, $buf, BOOT_ARGS_SIZE);
my $cmdline = $buf;
# Ignore the id
read(INF, $buf, UNSIGNED_SIZE * 8);
# Close the file
close INF;
# Print important values
printf "Page size: %d (0x%08x)\n", $p_size, $p_size;
printf "Kernel size: %d (0x%08x)\n", $k_size, $k_size;
printf "Ramdisk size: %d (0x%08x)\n", $r_size, $r_size;
printf "Second size: %d (0x%08x)\n", $s_size, $s_size;
printf "dt size: %d (0x%08x)\n", $t_size, $t_size;
printf "Board name: $name\n";
printf "Command line: $cmdline\n";
# Save the values
$PAGE_SIZE = $p_size;
$KERNEL_SIZE = $k_size;
$RAMDISK_SIZE = $r_size;
$SECOND_SIZE = $s_size;
$DT_SIZE = $t_size;
}
sub dump_file {
my ($infn, $outfn, $offset, $size) = @_;
my $buf = undef;
open INF, $infn or die "Could not open $infn: $!\n";
open OUTF, ">$outfn" or die "Could not open $outfn: $!\n";
binmode INF;
binmode OUTF;
seek(INF, $offset, 0) or die "Could not seek in $infn: $!\n";
read(INF, $buf, $size) or die "Could not read $infn: $!\n";
print OUTF $buf or die "Could not write $outfn: $!\n";
close INF;
close OUTF;
}
######################################################################
## Configuration Subroutines
sub parse_cmdline {
unless ($#ARGV == 0) {
die "Usage: $SCRIPT boot.img\n";
}
$IMAGE_FN = $ARGV[0];
}
The reason why none of tools support our image is because it has different header format. For example mkbootimg:
Code:
unsigned tags_addr; /* physical addr for kernel tags */
unsigned page_size; /* flash page size we assume */
unsigned unused[2]; /* future expansion: should be 0 */
but we need for our kernel such a code
Code:
unsigned dt_size; /* device tree size in bytes */
unsigned dt_addr; /* device tree address in bytes */
unsigned tags_addr; /* physical addr for kernel tags */
unsigned page_size; /* flash page size we assume */
akuhak said:
Perl script for unpacking
The reason why none of tools support our image is because it has different header format. For example mkbootimg:but we need for our kernel such a code
Click to expand...
Click to collapse
Yes @akuhak I had realised that about a week ago (header sizes), also another thing is most of these tools deal with a zImage, where's I boot.img has a hOmage (which is a 64 byte uImage header followed by the kernel. I have succesfully unpacked this boot.img (the kernel, ramdisk and dt.img).
Now if this script works it will be good for other users, but for me my need has already been done.
I am just having trouble trying to get this booting up on my phone when I repack it (I haven't been paying to much attention to this project becuse of exam revision)
Edit: I have tested it and verified thatvit works by comparing the files produced by hand and the files produced by this script via the sha1sum command. I have uploaded it onto my xCover3 post linking back here and giving the credit to you for posting it here. Thanks for your help
It easier to found already existing tool. Our device (I am owning Xcover 3 too) has pxa1088 board. It is known that similar boards has similar structure. So i found these thread github(dot)com/kumajaya/degas-mkbootimg (he has also topic here in xda with his researchs)
It has the same board so these tools works fine for us The only difference I noticed - header unknown value is 0x03000000 for our device and 0x02000000 for Galaxy Tab 4. Maybe something wrong with dtb image - I didn't check these yet.
BTW our phone has same characteristics as Samsung SM-G531F Galaxy Grand Prime (same board, same cpu, same gpu, only screen a bit different) and grandprimevelte has 5.1.1 android onboard and working TWRP recovery (I was trying to flash it but was unsuccessful).
As for same board - For Example Xcover 3 Value Edition has completely different board (exynos3475 with mali-t760mp8 gpu). The same characteristics as for Samsung SM-J200 Galaxy J2. So we can use root methods from j2lte. This means - flash TWRP then install SuperSu zip archive - thats all.
But we cannot use TWRP from VE cause of very different hardwares.
Now Im working on improving degas utilities...
akuhak said:
It easier to found already existing tool. Our device (I am owning Xcover 3 too) has pxa1088 board. It is known that similar boards has similar structure. So i found these thread github(dot)com/kumajaya/degas-mkbootimg (he has also topic here in xda with his researchs)
It has the same board so these tools works fine for us The only difference I noticed - header unknown value is 0x03000000 for our device and 0x02000000 for Galaxy Tab 4. Maybe something wrong with dtb image - I didn't check these yet.
BTW our phone has same characteristics as Samsung SM-G531F Galaxy Grand Prime (same board, same cpu, same gpu, only screen a bit different) and grandprimevelte has 5.1.1 android onboard and working TWRP recovery (I was trying to flash it but was unsuccessful).
As for same board - For Example Xcover 3 Value Edition has completely different board (exynos3475 with mali-t760mp8 gpu). The same characteristics as for Samsung SM-J200 Galaxy J2. So we can use root methods from j2lte. This means - flash TWRP then install SuperSu zip archive - thats all.
But we cannot use TWRP from VE cause of very different hardwares.
Now Im working on improving degas utilities...
Click to expand...
Click to collapse
TWRP for xcover3 value editio came out about 7 says ago, and apparently one user has tried it and is working on their value edition version. (When I had a quick look at the source code yesterdau I happenend to notice it was a exynos board). I had previously tried the dagas scripts and they didn't work for our image at the time, yet your providerd version does. Great idea for searching up the board name (Marvell) as that is one of the only things I hadn't thought to Google.
Feel free to compile TWRP under android 5.1.1 branch. I can/will chime in with relevant info, as well as you can use info from TWRPs source code on the other twrp versions of our device. The only reason I haven't tried it as of yet is becuse i using my my mobile data 99% of the time and just don't have the data to so are downloading source code.
As for the dt.img, I veriefed the scripts output to my on manually extraction and it's the same for both files. (Sha1). But there is a little bit of trailing data (anything past SEANDROIDENFORCING) which is ignored, I am not sure id it's importance as of yet(other then SuperSu omits it from the boot.img when using SuperSu.zip.
Will pop back into here later today.
Ok I completed my tools collection for pxa1088
Source Code: github(dot)com/AKuHAK/pxa1088-mkbootimg
All necessary utilities can be achieved via my modified android_img_repack_tools repo: github(dot)com/AKuHAK/android_img_repack_tool
By just typing ./configure and make
If you are scared by traffic I released tools in one archive: github(dot)com/AKuHAK/android_img_repack_tools/releases/tag/1st.
How to use? I created README in github: github(dot)com/AKuHAK/pxa1088-mkbootimg/blob/master/README
Tools can extract and than pack back boot.img and recovery.img from XCover3. If you didn't extract uImage and dtb.img you can repack boot.img (recovery.img) without hash sum changes. If you extract zImage from uImage and than pack again in uImage your resulting uImage will be different cause uIimage header contain timestamp which will be taken from your PC settings. But resulting uImage still HAVE to be valid.
akuhak said:
Ok I completed my tools collection for pxa1088
Source Code: github(dot)com/AKuHAK/pxa1088-mkbootimg
All necessary utilities can be achieved via my modified android_img_repack_tools repo: github(dot)com/AKuHAK/android_img_repack_tool
By just typing ./configure and make
If you are scared by traffic I released tools in one archive: github(dot)com/AKuHAK/android_img_repack_tools/releases/tag/1st.
How to use? I created README in github: github(dot)com/AKuHAK/pxa1088-mkbootimg/blob/master/README
Tools can extract and than pack back boot.img and recovery.img from XCover3. If you didn't extract uImage and dtb.img you can repack boot.img (recovery.img) without hash sum changes. If you extract zImage from uImage and than pack again in uImage your resulting uImage will be different cause uIimage header contain timestamp which will be taken from your PC settings. But resulting uImage still HAVE to be valid.
Click to expand...
Click to collapse
Cool, will have a look tomorrow, it's hitting 1am for me now. Just one quick correction in the above post of yours. You mention zImage is extracted from the uImage. That is incorrect, is is in fact just a gzipped kernel image (Image.gz) not a zImage.
Yes I know both types of formats are compressed, but our device runs a armv8 cpu which is 64 bits (albeit our kernel is using a 32 but instruction set not a 64 bit instruction set and this means or OS is also 32 bits then). Apon reading the documentation found in the source code, you will realise that it can only handle a) the uncompressed kernel image (Image) or b) a compressed version of the image in gzip format (Image.gz). This is further proven when looking at the boot.img with a hex editor as you can clearly see 2 Files that start with the gzip header format (1F 8B 08) which are the gzipped kernel (also if you go back by 64 bytes then you will be at the start of the uImage header which holds or the appropriate info about the kernel.uImage file) and our ramdisk.
So hopefully you may have just made a mistake in the above post, but if not then you are outputting the incorrect file, which may confuse some users of they don't know much but just wanted to compile the kernel from source.
Sorry about the above rant, will now have a quick look at your github. Thanks for your work you have put into this phone.
Edit: when you mention the android img repack tools, did you use this, http://forum.xda-developers.com/showthread.php?t=2600364 , and if so what modification have you done since I already have the downloaded (used the last of my mobile data last month to get it)
EDIT2: Gonna have a mess around with them tomorrow. One question, sorry for sounding like a noon, but how does the compression differs between normal gzip and minigzip, how will these differences affect a repacked ramdisk and what got you onto that piece of info.
And I had a quick look at your github, seen all your ps2 back related source code.you using FMCB, a hardmod (a chip) or a internal hardrive via an Ethernet adapter. I see sp193, doctorxyz, have commuted to your repo, so sounds like your a ps2 dev, which is cool as ****
1) About zImage. Maybe I need to be more clear: Xcover 3 kernel has uImage onboard with gzip compression. You can extract kernel from that gzipped uImage - I just thought that zImage is name for extracted kernel Sorry if I made mistake in this case.
uImage can be created with use of u-boot tool (denx(dot)de/wiki/U-Boot/WebHome)
For example
Code:
mkimage -I boot.uImage
will provide such an information for our device
Code:
Image Name: pxa1928dkb linux
Created: Wed May 18 15:13:06 2016
Image Type: AArch64 Linux Kernel Image (gzip compressed)
Data Size: 6616640 Bytes = 6461.56 kB = 6.31 MB
Load Address: 01000000
Entry Point: 01000000
This information is extracted from uImage kernel (64 bytes). After header we will see our gzipped kernel - you are right. I just simply extracted it with 7zip from boot.img-uImage.
The difference is that your way didn't use uImage header at all - so you will be unable to pack it back without mistakes - or your gzipped kernel have to be completely the same in size which is almost impossible to achieve. With use of mkimage you can alter kernels (of course only if samsung doesnt check something else) and get correct uImage in output.
2) About difference between minigzip and gzip. In fact I don't know why its happened but I didn't found a reason why ramdisk have to be packed exactly with minigzip. I tried almost all flag combinations with gzip but I get the same result that was in output file only with minigzip
As about kernel it is packed with maximum compression without name with normal gzip.
So assuming minigzip for ramdisk, gzip -n -9 for kernel. But Im completely sure that we can use any combination of gzipers and image still will be valid (but of course will be different in hash). We need to use exactly this combination only if we need to get the same file.
3) Yes Im using exactly this kitchen - I just removed all branches except 5.1.1 added minigzip, my tools and u-boot mkimage tool generation.
4) Yes Im one of still alive ps2 developers I just realized that my phone isn't rooted and started to dig what I can do with it.
akuhak said:
1) About zImage. Maybe I need to be more clear: Xcover 3 kernel has uImage onboard with gzip compression. You can extract kernel from that gzipped uImage - I just thought that zImage is name for extracted kernel Sorry if I made mistake in this case.
uImage can be created with use of u-boot tool (denx(dot)de/wiki/U-Boot/WebHome)
For example will provide such an information for our device
This information is extracted from uImage kernel (64 bytes). After header we will see our gzipped kernel - you are right. I just simply extracted it with 7zip from boot.img-uImage.
The difference is that your way didn't use uImage header at all - so you will be unable to pack it back without mistakes - or your gzipped kernel have to be completely the same in size which is almost impossible to achieve. With use of mkimage you can alter kernels (of course only if samsung doesnt check something else) and get correct uImage in output.
2) About difference between minigzip and gzip. In fact I don't know why its happened but I didn't found a reason why ramdisk have to be packed exactly with minigzip. I tried almost all flag combinations with gzip but I get the same result that was in output file only with minigzip
As about kernel it is packed with maximum compression without name with normal gzip.
So assuming minigzip for ramdisk, gzip -n -9 for kernel. But Im completely sure that we can use any combination of gzipers and image still will be valid (but of course will be different in hash). We need to use exactly this combination only if we need to get the same file.
3) Yes Im using exactly this kitchen - I just removed all branches except 5.1.1 added minigzip, my tools and u-boot mkimage tool generation.
4) Yes Im one of still alive ps2 developers I just realized that my phone isn't rooted and started to dig what I can do with it.
Click to expand...
Click to collapse
1) yes a zImage is one type of you can get when compiling for ARM Devices, you Device is ARM64 so thierare differences. As for extracting the kernel I have always include the 64 byte header (since the middle of last week when I realised that it was there after running binwalk against the file). So yea your assumption was correct up untill a week ago.
2) Interesting. I will be sure to make changes accordingly.
3) cool I have 2 out of four so will download accordingly
4) cool, that the development scene is still going since the ps2 reached it 16 birthday this year.
5) as for root on this device chainfire has needed to go Systemless after/on 6.0 device or 5.1.1. Samsung devices, so to achive root on those devices it requires the modification of boot.img. what his SuperSu.zip does is patch the sepolicy file to allow 3 rules to run in permissive, modify other bits of the ramdisk accordingly to allow for it to run services and mount su.img at boot and it creates and place the necessary files inside su.img. I have completed all that by hand after reading his update-script numerous times. My only roadblock has been try to get the boot.img to boot.I was currently in the process, of try (I have tried alot of different ways including mkbootimg etc.) Manuallying replacing the ramdisk contents with the modified version, and then modify the bootmimg header to continue any modifed value (ramdisk size and dt offset) but am only partially done as I haven't had the chance the last few days to do it.
Awesome tool, unpack, repacked without a single modification and then ran "sha1sum" to both boot.img's and they are exactly the same. You are amazing . I have referenced your tools in this thread (http://forum.xda-developers.com/android/development/4-4-4-5-1-1-6-0-1-samsung-xcover3-t3465132 ) straight to your github. Now to try my modified ramdisk and see If my phone can boot it. Will post results soon.
======
Yes I can boot custom boot.img, without the SEANDORIDENFORCEING showing up. I can see my ramdisk changes, work, I can type adb root and I don't get the product in build message, but trying to do anything that requires root e.g. trying to push su.img to /data or /cache, gets me the error permission denied, but hey I am half way there to getting root.
WARNING
This should go without saying, but you MUST have your bootloader unlocked (check OEM UNLOCK in developer options AND fastboot oem unlock). If you don't, you will probably brick your phone.
If you deviate from this procedure, and think: "I can just skip a step, or I can do this on my own Linux install". Don't complain if you brick your phone.
PREREQUISITES:
You need to grab FWUL (version 2.7 or later) and burn it to a USB stick: link
Even if you have Linux, and you think you can install the dependencies, don't. I know this works from FWUL.
PROCEDURE PART 1: Installing TWRP
Boot from your FWUL USB stick. If your PC has secureboot enabled, you will have to disable it in BIOS
Put your phone into download mode. With the phone powered off, hold vol up and plug in the USB cable. You do not need to touch the power button -- the phone will power on and enter download mode.
Once booted, login. The password is: linux
Double click the LG folder that is on the desktop
Double click on LG LAF (runningnak3d) icon and you will be at a terminal prompt.
The following are the commands that you enter into that terminal. You can copy / paste them if you like.
Code:
git pull
git checkout v10-miscwrte
./step1.sh
When you are told to, pull the USB cable, and the phone will power off. You now have TWRP installed. At this point you can flash a ROM, or Magisk or whatever you like.
OPTIONAL:
If you don't know what to do with TWRP, and you just want to run rooted stock, this is for you....
First boot into TWRP - with the phone off, hold vol down and power at the same time. The second the LG logo appears, release power for a split second, then then press and hold power again (you never let go of vol down).
When you get a screen asking you to factory reset, you can let go of both buttons. hit vol down to select yes -- two times -- this will take you to TWRP.
PROCEDURE PART 2: Rooting and cleanup
Now that you are in TWRP:
./step2.sh
If you ran step2.sh you have TWRP on recovery, and you are rooted. If you only ran step1.sh, then you have TWRP on recovery. Either way, enjoy!
CREDITS:
Lekensteyn -- His base work on the G2 / G3 gave me a GREAT headstart!
@steadfasterX - He added some real nice features, great guy to bounce ideas off, and just testing crazy ideas because he wasn't afraid to brick his phone Also, for FWUL
tuxuser - Helping with my lacking in Python
@smitel - His original reverse engineering of LG UP. Great inspiration!
-- Brian
Entering recovery [ READ THIS ]
To enter recovery power off the phone then hold both the down volume and power at the same time. When you see the black LG screen briefly release the power button and then press it again while not letting the volume down up.
You will see a screen asking if you want to delete all user settings. Say YES
You will see a screen asking if you want to delete all user data. Say YES
You will briefly see the black LG bootup screen.
TWRP or factory recovery will load.
------------------------------------------------------------------
Thanks for remembering the V10 users!
For those wondering, to get into download mode power off your phone then hold down volume up at the same time as you are plugging in the usb cable. It should go into download mode. [I recently installed twrp to the laf partition so I have it in two places...if somehow my main twrp gets wiped out I can still get to it via download mode.]
Might be worth mentioning once booted up into TWRP magisk is the preferred root method since it provides modules to add xposed and can help pass safetynet so android pay/pokemon go continue to work even while rooted.
https://forum.xda-developers.com/apps/magisk/official-magisk-v7-universal-systemless-t3473445
Magisk also has alot of REALLY nice modules to add all sorts of features to our vanilla rom including methods to debloat all the t-mobile and LG apps.
runningnak3d said:
WARNING
This should go without saying, but
Click to expand...
Click to collapse
Thanks for the effort and time! Much appreciated. Too bad I'm not in FL to see about this beers!
Will be back later after my virgin V10 (100% stock nougat) sacrifice is complete...
Sent from my LG-H901 using XDA Labs
Just a heads up -- you can always flash again if it fails, but if you want to check before you reboot, you can run:
./partitions.py --dump test.img recovery
sha256sum test.img and compare it to the hash of TWRP.
Flashing via this method has no retries, so if there is noise on the cable or the bus, you will have a bad flash.
-- Brian
runningnak3d said:
Download this vdi: fwul.zip
Click to expand...
Click to collapse
Getting an error on the vdi..
Code:
This download file is not currently available (it was deleted or disabled).
Is the referenced file the same as this one?
Code:
FWUL_v2.3_x86_64_15GB.zip
runningnak3d said:
Just a heads up -- you can always flash again if it fails, but if you want to check before you reboot, you can run:
./partitions.py --dump test.img recovery
sha256sum test.img and compare it to the hash of TWRP.
Flashing via this method has no retries, so if there is noise on the cable or the bus, you will have a bad flash.
-- Brian
Click to expand...
Click to collapse
Why not add an auto hash check post flash with a prompt to reflash if they don't match?
That is if you plan to customize it or use the generic lglaf.
---------- Post added at 08:09 PM ---------- Previous post was at 08:05 PM ----------
NYLimited said:
Getting an error on the vdi..
Code:
This download file is not currently available (it was deleted or disabled).
Click to expand...
Click to collapse
Some alternatives here: https://forum.xda-developers.com/an.../live-iso-adb-fastboot-driver-issues-t3526755
famewolf said:
Thanks for remembering the V10 users!
Might be worth mentioning once booted up into TWRP magisk is the preferred root method since it provides modules to add xposed and can help pass safetynet so android pay/pokemon go continue to work even while rooted.
https://forum.xda-developers.com/apps/magisk/official-magisk-v7-universal-systemless-t3473445
Magisk also has alot of REALLY nice modules to add all sorts of features to our vanilla rom including methods to debloat all the t-mobile and LG apps.
Click to expand...
Click to collapse
Agreed and let's not forget that SuperSU development seems to have stalled since Chanfire moved on...
Sorry about that, I had to upload a version that did hash checks. I will update the link now.
-- Brian
@famewolf There are a LOT of things that I am going to add. This will eventually be a full blown replacement for LG UP with ARB checking, etc.
LG is getting ready to relate Oreo for the V20, so I wanted to get it out there ASAP.
-- Brian
runningnak3d said:
@famewolf There are a LOT of things that I am going to add. This will eventually be a full blown replacement for LG UP with ARB checking, etc.
LG is getting ready to relate Oreo for the V20, so I wanted to get it out there ASAP.
-- Brian
Click to expand...
Click to collapse
Are you planning to make an oreo available for the V10 or at least willing to work with a few of us on it? I'm not sure what would have to be changed to allow a v20 rom to run for us or if there is a closer match now that nougat can be rooted.
---------- Post added at 09:15 PM ---------- Previous post was at 09:12 PM ----------
famewolf said:
Are you planning to make an oreo available for the V10 or at least willing to work with a few of us on it? I'm not sure what would have to be changed to allow a v20 rom to run for us or if there is a closer match now that nougat can be rooted.
Click to expand...
Click to collapse
Oh you may want to implement automatic backup of recovery or laf prior to flashing a new one...maybe with a timestamp in filename so multiple revisions can be saved....that way for example someone could go back to normal download mode. I wrote some shell scripts that run under twrp or rooted system and allow you to backup all the partitions to images on the microsd and then pull them via adb to the pc. Something similar might be worthwhile for a backup option. I'm excellent with suggestions of hard work for others. ;P
NYLimited said:
Getting an error on the vdi..
This download file is not currently available (it was deleted or disabled).
Click to expand...
Click to collapse
famewolf said:
Some alternatives here: https://forum.xda-developers.com/an.../live-iso-adb-fastboot-driver-issues-t3526755
Click to expand...
Click to collapse
I grabbed the 15 and 32 GB persistent files but they need to be converted from .img to .vdi which takes a while... I suppose I could post the converted file for d/l if anyone wants them.
Also, VirtualBox did NOT give me Arch Linux 64 bit option (only 32 bit)..
By all means keep the suggestions coming. I don't suppose you are any good with Python? I would love the help. For the things you are talking about you don't need to know the protocol, although I would be glad to teach you / give you all the documentation.
Yes, if it is possible, I will port Oreo to the V10, or at least help. Unless they do something so radical that it just isn't feasible. Nougat on the V20 isn't much different than Nougat on the V10.
Heck, they are so cheap now, I am ordering me another V10 just so I can help out.
-- Brian
runningnak3d said:
By all means keep the suggestions coming. I don't suppose you are any good with Python? I would love the help. For the things you are talking about you don't need to know the protocol, although I would be glad to teach you / give you all the documentation.
Yes, if it is possible, I will port Oreo to the V10, or at least help. Unless they do something so radical that it just isn't feasible. Nougat on the V20 isn't much different than Nougat on the V10.
Heck, they are so cheap now, I am ordering me another V10 just so I can help out.
-- Brian
Click to expand...
Click to collapse
I don't know python although I can usually manage simple mods to it....I do ok in quick and dirty bash shell scripts....virtualbox seems to be causing more complications than it helps...it may be worthwhile to consider a bootable iso to burn to a cd with a shellscript on the desktop that says "run me" which automated the downloading of twrp and then the running of the commands....that's something I could probably manage but it wouldn't be "pretty".
(I'm working with NYLimited in email).
---------- Post added at 10:22 PM ---------- Previous post was at 10:19 PM ----------
runningnak3d said:
By all means keep the suggestions coming. I don't suppose you are any good with Python? I would love the help. For the things you are talking about you don't need to know the protocol, although I would be glad to teach you / give you all the documentation.
Yes, if it is possible, I will port Oreo to the V10, or at least help. Unless they do something so radical that it just isn't feasible. Nougat on the V20 isn't much different than Nougat on the V10.
Heck, they are so cheap now, I am ordering me another V10 just so I can help out.
-- Brian
Click to expand...
Click to collapse
The V10 isn't even my primary device anymore but it had so much potential and LG just crippled it so badly. With your recent root, the updated twrp (the latest version we had was 3.0 previously) and bringing people up to 30c at least it has a decent starting point.
So glad to see this!
A quick question, on the final step I get this error message:
./partitions.py --restoremisc ~/Downloads/TWRP_3.2.1_H901.img recovery
Traceback (most recent call last):
File "./partitions.py", line 460, in <module>
main()
File "./partitions.py", line 410, in main
lglaf.try_hello(comm)
File "/home/android/lglaf/lglaf.py", line 401, in try_hello
data = comm.read(0x20, timeout=HELLO_READ_TIMEOUT)
File "/home/android/lglaf/lglaf.py", line 240, in read
buff = self._read(need, timeout=timeout)
File "/home/android/lglaf/lglaf.py", line 359, in _read
array = self.usbdev.read(self.ep_in, 2**14, timeout=timeout)
File "/usr/lib/python3.6/site-packages/usb/core.py", line 988, in read
self.__get_timeout(timeout))
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 833, in bulk_read
timeout)
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 936, in __read
_check(retval)
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 595, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 110] Operation timed out
Any suggestions? I haven't had any trouble with the USB cable and there were no installation issues.
chin'ah.girl said:
So glad to see this!
A quick question, on the final step I get this error message:
./partitions.py --restoremisc ~/Downloads/TWRP_3.2.1_H901.img recovery
Traceback (most recent call last):
File "./partitions.py", line 460, in <module>
main()
File "./partitions.py", line 410, in main
lglaf.try_hello(comm)
File "/home/android/lglaf/lglaf.py", line 401, in try_hello
data = comm.read(0x20, timeout=HELLO_READ_TIMEOUT)
File "/home/android/lglaf/lglaf.py", line 240, in read
buff = self._read(need, timeout=timeout)
File "/home/android/lglaf/lglaf.py", line 359, in _read
array = self.usbdev.read(self.ep_in, 2**14, timeout=timeout)
File "/usr/lib/python3.6/site-packages/usb/core.py", line 988, in read
self.__get_timeout(timeout))
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 833, in bulk_read
timeout)
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 936, in __read
_check(retval)
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 595, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 110] Operation timed out
Any suggestions? I haven't had any trouble with the USB cable and there were no installation issues.
Click to expand...
Click to collapse
JUst as a suggestion you may want to copy/paste all the text in your terminal session to help identify possible issues. The phone is in download mode (power it off then hold vol up while plugging in the usb cable) and says so on the screen. If you try "./partitions.py --list" what do you get?
Check out this post to ensure dependencies are installed: https://forum.xda-developers.com/showpost.php?p=76134256&postcount=97
Also if you are in ~/lglaf you may want to use ../Downloads/TWRP_3.2.1_H901.img or cp the img file directly into same dir and use ./TWRP_3.2.1_H901.img
Hi @runningnak3d, thank you so much for not abandonning us lg v10 and for your hard work, really appreciated. Please this link https://forum.xda-developers.com/devdb/project/dl/?id=29075 doesn't support "resume", i already tried 5 times and always failed because the download can't be resumed. Please could you add another androifilehost link to dowload this fwul.zip? Thank you. I wanna try to root mine with your awesome method then disable 2big cores that give bootloop to my phone.
@famewolf
I made sure the phone is in download mode. Unfortunately that command pretty much generates the same results...
[[email protected] ~]$ cd lglaf
[[email protected] lglaf]$ git pull
Already up to date.
[[email protected] lglaf]$ git checkout v10-miscwrte
Already on 'v10-miscwrte'
Your branch is up to date with 'origin/v10-miscwrte'.
[[email protected] lglaf]$ ./partitions.py --list
Traceback (most recent call last):
File "./partitions.py", line 460, in <module>
main()
File "./partitions.py", line 410, in main
lglaf.try_hello(comm)
File "/home/android/lglaf/lglaf.py", line 401, in try_hello
data = comm.read(0x20, timeout=HELLO_READ_TIMEOUT)
File "/home/android/lglaf/lglaf.py", line 240, in read
buff = self._read(need, timeout=timeout)
File "/home/android/lglaf/lglaf.py", line 359, in _read
array = self.usbdev.read(self.ep_in, 2**14, timeout=timeout)
File "/usr/lib/python3.6/site-packages/usb/core.py", line 988, in read
self.__get_timeout(timeout))
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 833, in bulk_read
timeout)
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 936, in __read
_check(retval)
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 595, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 110] Operation timed out
The phone appears as its supposed to in the VM Devices > USB menu as well.
chin'ah.girl said:
@famewolf
I made sure the phone is in download mode. Unfortunately that command pretty much generates the same results...
[[email protected] ~]$ cd lglaf
[[email protected] lglaf]$ git pull
Already up to date.
[[email protected] lglaf]$ git checkout v10-miscwrte
Already on 'v10-miscwrte'
Your branch is up to date with 'origin/v10-miscwrte'.
[[email protected] lglaf]$ ./partitions.py --list
Traceback (most recent call last):
File "./partitions.py", line 460, in <module>
main()
File "./partitions.py", line 410, in main
lglaf.try_hello(comm)
File "/home/android/lglaf/lglaf.py", line 401, in try_hello
data = comm.read(0x20, timeout=HELLO_READ_TIMEOUT)
File "/home/android/lglaf/lglaf.py", line 240, in read
buff = self._read(need, timeout=timeout)
File "/home/android/lglaf/lglaf.py", line 359, in _read
array = self.usbdev.read(self.ep_in, 2**14, timeout=timeout)
File "/usr/lib/python3.6/site-packages/usb/core.py", line 988, in read
self.__get_timeout(timeout))
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 833, in bulk_read
timeout)
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 936, in __read
_check(retval)
File "/usr/lib/python3.6/site-packages/usb/backend/libusb1.py", line 595, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 110] Operation timed out
The phone appears as its supposed to in the VM Devices > USB menu as well.
Click to expand...
Click to collapse
Did you run the lines to verify dependencies? Specifically the pip lines to install PyUSB and the 2 others.... If all else fails simplify things..I think using virtualbox is causing more problems then helping.... download the ISO of his linux version or an ubuntu one...write it to a cd or to a usb drive...boot off that directly....do the pip commands to ensure dependencies....run his instructions with phone connected to pc or laptop. I use linux directly and didn't do virtualbox.....NYLimited is also having issues that may be attributed to Virtualbox.
@runningnak3d @famewolf
Thanks for the help and patience! I did a lot of little stuff tonight and found a few inconsistencies, some perhaps due to the fact that I had to improvise and grab another fwul version and such. My lack of linux background didn't help, of course.
Long story short, I got to the point of running the py script. It wrote 32796672 bytes but recovery did not load for me.
Pulling the image back from recovery via --dump yielded a consistent 41943040 bytes. Each time I flashed the img file the 32796672 bytes were consistent. So were the 41943040 bytes coming back. The computed hash sums differed from each other but the twrp image was always the same hash and the test dump from recovery was consistent with itself but different from twrp.
It almost seemed like I was writing to a place totally different than the place I was pulling data back from. Neither side ever changed from the previous version of itself but the two never matched each other. Recovery did not load, regardless.
Time to take a break (early day tomorrow) and will regroup again sometime tomorrow eve with hopefully fresh ideas.
This is the last flash and hash of the files. The numbers are consistent over multiple flashes:
Code:
[[email protected] lglaf]$ ./partitions.py --restoremisc ../Downloads/TWRP321.img recovery
2018-04-06 05:39:16,946 partitions: INFO: Done after writing 32796672 bytes from ../Downloads/TWRP321.img
[[email protected] lglaf]$ ./partitions.py --dump test.img recovery
[ 100 % ] 2018-04-06 05:40:49,401 partitions: INFO: Wrote 41943040 bytes to test.img
[[email protected] lglaf]$ sha256sum test.img
d78190b422733a84b2526558f36c5d8ab6915748096fd7569927ad84f509e6c1 test.img
[[email protected] lglaf]$ sha256sum ../Downloads/TWRP321.img
1a5667e8ac35784780d8cd7b5c3ad72a353889c39220d8002ac2498a92ff6f8e ../Downloads/TWRP321.img
[[email protected] lglaf]$ ./partitions.py --restoremisc ../Downloads/TWRP321.img recovery
2018-04-06 05:49:38,020 partitions: INFO: Done after writing 32796672 bytes from ../Downloads/TWRP321.img
[[email protected] lglaf]$ ./partitions.py --dump test.img recovery
[ 100 % ] 2018-04-06 05:51:12,507 partitions: INFO: Wrote 41943040 bytes to test.img
[[email protected] lglaf]$ sha256sum test.img
d78190b422733a84b2526558f36c5d8ab6915748096fd7569927ad84f509e6c1 test.img
[[email protected] lglaf]$
NYLimited said:
@runningnak3d @famewolf
Thanks for the help and patience! I did a lot of little stuff tonight and found a few inconsistencies, some perhaps due to the fact that I had to improvise and grab another fwul version and such. My lack of linux background didn't help, of course.
Long story short, I got to the point of running the py script. It wrote 32796672 bytes but recovery did not load for me.
Pulling the image back from recovery via --dump yielded a consistent 41943040 bytes. Each time I flashed the img file the 32796672 bytes were consistent. So were the 41943040 bytes coming back. The computed hash sums differed from each other but the twrp image was always the same hash and the test dump from recovery was consistent with itself but different from twrp.
It almost seemed like I was writing to a place totally different than the place I was pulling data back from. Neither side ever changed from the previous version of itself but the two never matched each other. Recovery did not load, regardless.
Time to take a break (early day tomorrow) and will regroup again sometime tomorrow eve with hopefully fresh ideas.
This is the last flash and hash of the files. The numbers are consistent over multiple flashes:
Code:
[[email protected] lglaf]$ ./partitions.py --restoremisc ../Downloads/TWRP321.img recovery
2018-04-06 05:39:16,946 partitions: INFO: Done after writing 32796672 bytes from ../Downloads/TWRP321.img
[[email protected] lglaf]$ ./partitions.py --dump test.img recovery
[ 100 % ] 2018-04-06 05:40:49,401 partitions: INFO: Wrote 41943040 bytes to test.img
[[email protected] lglaf]$ sha256sum test.img
d78190b422733a84b2526558f36c5d8ab6915748096fd7569927ad84f509e6c1 test.img
[[email protected] lglaf]$ sha256sum ../Downloads/TWRP321.img
1a5667e8ac35784780d8cd7b5c3ad72a353889c39220d8002ac2498a92ff6f8e ../Downloads/TWRP321.img
[[email protected] lglaf]$ ./partitions.py --restoremisc ../Downloads/TWRP321.img recovery
2018-04-06 05:49:38,020 partitions: INFO: Done after writing 32796672 bytes from ../Downloads/TWRP321.img
[[email protected] lglaf]$ ./partitions.py --dump test.img recovery
[ 100 % ] 2018-04-06 05:51:12,507 partitions: INFO: Wrote 41943040 bytes to test.img
[[email protected] lglaf]$ sha256sum test.img
d78190b422733a84b2526558f36c5d8ab6915748096fd7569927ad84f509e6c1 test.img
[[email protected] lglaf]$
Click to expand...
Click to collapse
If your twrp is showing the following:
[email protected] /workarea/android/v10 $ md5sum TWRP_3.2.1_H901.img
b89d341cd61da31a5348d8f6b3c75c97 TWRP_3.2.1_H901.img
then it's fine...as for the dump...I think empty space at the end would have to be stripped off for them to match. Will work with it more tomorrow..just drop me a line.
Hello and welcome to my first post.
today I will talk about editing super.img and modifying system.img inside of it.
In android 10 and bigger, sometimes there is no system.img in ROM it because google starting use Dynamic Partitions for more flexible images size - more details here
instead of system.img we will see super.img that include few partition inside.
So In this case, in order to do our modifications in the rom we should unpack the super.img and after that to unpack the system.img and then build it again.
requirements:
1) I will use ubuntu in vbox so you need a linux machine.
2) some super.img for editing.
steps:
1) unpacking super.img
2) resizing system.img in order to insert our MODS to the rom
3) mounting system.img
4) do our modifications
5) umounting system
6) shrink edited system.img to the minimal size
7) generating new super.img
8) flashing it to our device
Let's Start!
1) unpacking super image:
First of all the super.img file might be in sparse format so we need to make it raw image
open termianl in super.img directory and type:
Code:
simg2img super.img super.ext4.img
now we got new file named super.ext4.img: we are working with this file.
There are multiple ways to unpack super.img:
for example: using imjtool or using lpunpack
If you use imjtool follow this
Open terminal in imjtool path and super.img path and type
Code:
./imjtool.ELF64 super.img extract
if you got and error run it as superuser by
Code:
sudo ./imjtool.ELF64 super.img extract
I will use lpunpck tool (Official tool from google)
locate lpunpack and super.ext4.img and open terminal in this folder.
Then type:
Code:
./lpunpack super.ext4.img
wait for it it may take couple of minutes.
now our folder looks like this:
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
2)resizing system.img in order to make enough space for modifications
turn back into your terminal and type:
Code:
fallocate -l 2G system.img
This command allocates more space for system.img (Changing to 2GB)
After that type:
Code:
resize2fs system.img 2G
This command increases the file system size of the partition to 2G
3) mounting system.img
Create new folder and mount:
Code:
mkdir system
Code:
sudo mount -t ext4 -o loop system.img system
Now system dir contains all system.img files.
4)edit it as you want
***IMPORTANT***
In order to make changes you should use superuser:
when you typing a command while you modifying use sudo prefix.
you can use the file explorer with superuser permissions by typing:
Code:
sudo nautilus
5) umount system
*)make sure the terminal is not in system directory or some sub dir of it
if Yes just type
Code:
cd ..
to go up in the directories tree (do it many times until you will be in the same directory like the image in the top)
type:
Code:
sudo umount system
Code:
e2fsck -yf system.img
(fixes file system errors)
6) shrink the edit system.img into the minimum possible size
by:
Code:
resize2fs -M system.img
fix the file system again:
Code:
e2fsck -yf system.img
7) generating new super.img
as you can see at the image above, in my case the super.ext4.img contaings 3 partitions:
1)system.img(that we worked with)
2)vendor.img
3)product.img
in you case it may be different so follow the logic of the following command and not copy paste it.
another tool from google that called lpmake using for packing super.img.
How this tool works
it creates empty super.img with all headers stuff and pushing partitions into it
so let's start with generating:
this is the documentation of the tool: link
at first we should get each partitions file size
we can do it by:
Code:
stat -c '%n %s' system.img
do it for all partitions files
And this is tricky and critical step:
***DO NOT COPY!!***
Code:
./lpmake --metadata-size 65536 --super-name super --metadata-slots 1 --device super:4294967296 --group main:3139354624 --partition system:readonly:1434726400:main --image system=./system.img --partition vendor:readonly:330866688:main --image vendor=./vendor.img --partition product:readonly:1373761536:main --image product=./product.img --sparse --output ./super.new.img
explanation:
1) --metadata-size: The maximum size that partition metadata may consume. A partition entry uses 64 bytes and an extent entry uses 16 bytes. I think 65536 should work in most cases.
2) --metadata-slots - The number of slots available for storing metadata. This should match the number of update slots on the device, 2 for A/B devices and 1 for non-A/B.
3)--device super: The size of the “super” partition on the device. It must match exactly, and it must be evenly divisible by the sector size (512 bytes).
4) --group main:4293513600: sum of all partitions files sizes
5) --partition system:readonly:1577095168:main --image system=./system.img : Every parition file size with permissions(readonly) and input img file
good work we got new custom rom!
credit to(for repacking step):
XDA Post: https://forum.xda-developers.com/showpost.php?p=82241115&postcount=70
8) flashing the new rom to a physical device
First of all we should unlock the boot loader without unlocking we may brick the phone
It can be done by allowing oem unlock in developers options and rebooting into bootloader and type:
Code:
fastboot flashing unlock
But it may not work - it depends on your device so check in google.
After you unlocked the bootloader flash the new rom by:
Code:
fastboot flash super super.new.img
If your devices alert for unlocked bootloader(orange state)
you can remove this annoying alert
just search on google.
if you have mediatek check this link:
https://forum.hovatek.com/thread-31664.html
I'm trying to modify my system.img (/system/build.prop) to include support for multi users. After struggling a lot, I've succeeded following your guide (that's an awesome work btw) to unpack, mount, modify, umount and repack super.img. Then, after flashing the new super.img, nothing changes on the phone. I've tried adding and removing files to the system.img before reflashing, and after flashing, i've verified using adb shell. It seem that the image flashed do not contains my modifications. In other words, I succeed to repack a modified super.img, but flashing the new super.img do not modify the phone itself. Any clues? Can you help me?
Phone is Umidigi A7 Pro 64GB
saga1900 said:
I'm trying to modify my system.img (/system/build.prop) to include support for multi users. After struggling a lot, I've succeeded following your guide (that's an awesome work btw) to unpack, mount, modify, umount and repack super.img. Then, after flashing the new super.img, nothing changes on the phone. I've tried adding and removing files to the system.img before reflashing, and after flashing, i've verified using adb shell. It seem that the image flashed do not contains my modifications. In other words, I succeed to repack a modified super.img, but flashing the new super.img do not modify the phone itself. Any clues? Can you help me?
Phone is Umidigi A7 Pro 64GB
Click to expand...
Click to collapse
Hey dear thank you for the compliment, please tell me more about how did you flash the new image.
hey dear did you flash it by using sp flash tool? did you unlocked the bootloader? did you see orange state warning when the device booted after flashing?
Brepro1 said:
Hey dear thank you for the compliment, please tell me more about how did you flash the new image.
hey dear did you flash it by using sp flash tool? did you unlocked the bootloader? did you see orange state warning when the device booted after flashing?
Click to expand...
Click to collapse
Hi, thx for your reply!
I've unlocked the bootloader as you oriented, and after that, I'm seeing the orange warning!I tried flashing using two methods. Using SP Flash Tool, and using flashboot.
Using SP Flash Tool was straight forward. Just selected the Scatter File, then picked the new super.img. Successfully flashed, but no changes on the phone.
Using flashboot: ./flashboot flash super super.modified.img
What a did, in short, was:
1.) simg2img super.img super.raw
2.) imjtool super.raw extract (also tried using lpunpack as you oriented)
3.) mount -t ext4 -o loop system.img /mount/point
4.) edit everything needed
5.) umount /mount/point
6.) repacked super.img using lpmake
7.) flashed the new super.img using SPFT or flashboot. Boths succeeds, but none really modifies the phone
To double check that the new super.img is correct, I've extracted it again, and double checked my changes. and they are there.
Vary strange, I don't have an idea, sorry
Brepro1 said:
Vary strange, I don't have an idea, sorry
Click to expand...
Click to collapse
NP, thanks anyway. I'm reading about a few things like disabling vbmeta verity and other things. Also, I did not mentioned, but I've downloaded and modified a stock rom, and my phone is not rooted. If i found anything that worths mention here, I'll update this thread
i cant mounr system.img . Cant you help me
#sudo mount -t ext4 -o loop system.img system
mount: /home/nam/system: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error
NguyenNhutNam said:
i cant mounr system.img . Cant you help me
#sudo mount -t ext4 -o loop system.img system
mount: /home/nam/system: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error
Click to expand...
Click to collapse
what does the command
file path://system.img
show up?
Anyone can help how to repack an unpacked super.img?
Try to delete an app in the product and system.img and post the instructions to repack everything into a super.img again.
My goal is to remove most of the preinstalled Google apps.
Here is the firmware link
UMIDIGI_A9_PRO_128GB_V1-0_20210127
MediaFire is a simple to use free service that lets you put all your photos, documents, music, and video in a single place so you can access them anywhere and share them everywhere.
www.mediafire.com
How do you get this size, from the scatter file super partition size?
3)--device super: The size of the “super” partition on the device. It must match exactly, and it must be evenly divisible by the sector size (512 bytes).
After repacking I also get this error, is it normal?
'Invalid sparse file format at header magic'
Brepro1 said:
Hello and welcome to my first post.
today I will talk about editing super.img and modifying system.img inside of it.
In android 10 and bigger, sometimes there is no system.img in ROM it because google starting use Dynamic Partitions for more flexible images size - more details here
instead of system.img we will see super.img that include few partition inside.
So In this case, in order to do our modifications in the rom we should unpack the super.img and after that to unpack the system.img and then build it again.
requirements:
1) I will use ubuntu in vbox so you need a linux machine.
2) some super.img for editing.
steps:
1) unpacking super.img
2) resizing system.img in order to insert our MODS to the rom
3) mounting system.img
4) do our modifications
5) umounting system
6) shrink edited system.img to the minimal size
7) generating new super.img
8) flashing it to our device
Let's Start!
1) unpacking super image:
First of all the super.img file might be in sparse format so we need to make it raw image
open termianl in super.img directory and type:
Code:
simg2img super.img super.ext4.img
now we got new file named super.ext4.img: we are working with this file.
There are multiple ways to unpack super.img:
for example: using imjtool or using lpunpack
If you use imjtool follow this
Open terminal in imjtool path and super.img path and type
Code:
./imjtool.ELF64 super.img extract
if you got and error run it as superuser by
Code:
sudo ./imjtool.ELF64 super.img extract
I will use lpunpck tool (Official tool from google)
locate lpunpack and super.ext4.img and open terminal in this folder.
Then type:
Code:
./lpunpack super.ext4.img
wait for it it may take couple of minutes.
now our folder looks like this:
2)resizing system.img in order to make enough space for modifications
turn back into your terminal and type:
Code:
fallocate -l 2G system.img
This command allocates more space for system.img (Changing to 2GB)
After that type:
Code:
resize2fs system.img 2G
This command increases the file system size of the partition to 2G
3) mounting system.img
Create new folder and mount:
Code:
mkdir system
Code:
sudo mount -t ext4 -o loop system.img system
Now system dir contains all system.img files.
4)edit it as you want
***IMPORTANT***
In order to make changes you should use superuser:
when you typing a command while you modifying use sudo prefix.
you can use the file explorer with superuser permissions by typing:
Code:
sudo nautilus
5) umount system
*)make sure the terminal is not in system directory or some sub dir of it
if Yes just type
Code:
cd ..
to go up in the directories tree (do it many times until you will be in the same directory like the image in the top)
type:
Code:
sudo umount system
Code:
e2fsck -yf system.img
(fixes file system errors)
6) shrink the edit system.img into the minimum possible size
by:
Code:
resize2fs -M system.img
fix the file system again:
Code:
e2fsck -yf system.img
7) generating new super.img
as you can see at the image above, in my case the super.ext4.img contaings 3 partitions:
1)system.img(that we worked with)
2)vendor.img
3)product.img
in you case it may be different so follow the logic of the following command and not copy paste it.
another tool from google that called lpmake using for packing super.img.
How this tool works
it creates empty super.img with all headers stuff and pushing partitions into it
so let's start with generating:
this is the documentation of the tool: link
at first we should get each partitions file size
we can do it by:
Code:
stat -c '%n %s' system.img
do it for all partitions files
And this is tricky and critical step:
***DO NOT COPY!!***
Code:
./lpmake --metadata-size 65536 --super-name super --metadata-slots 1 --device super:4294967296 --group main:3139354624 --partition system:readonly:1434726400:main --image system=./system.img --partition vendor:readonly:330866688:main --image vendor=./vendor.img --partition product:readonly:1373761536:main --image product=./product.img --sparse --output ./super.new.img
explanation:
1) --metadata-size: The maximum size that partition metadata may consume. A partition entry uses 64 bytes and an extent entry uses 16 bytes. I think 65536 should work in most cases.
2) --metadata-slots - The number of slots available for storing metadata. This should match the number of update slots on the device, 2 for A/B devices and 1 for non-A/B.
3)--device super: The size of the “super” partition on the device. It must match exactly, and it must be evenly divisible by the sector size (512 bytes).
4) --group main:4293513600: sum of all partitions files sizes
5) --partition system:readonly:1577095168:main --image system=./system.img : Every parition file size with permissions(readonly) and input img file
good work we got new custom rom!
credit to(for repacking step):
XDA Post: https://forum.xda-developers.com/showpost.php?p=82241115&postcount=70
8) flashing the new rom to a physical device
First of all we should unlock the boot loader without unlocking we may brick the phone
It can be done by allowing oem unlock in developers options and rebooting into bootloader and type:
Code:
fastboot flashing unlock
But it may not work - it depends on your device so check in google.
After you unlocked the bootloader flash the new rom by:
Code:
fastboot flash super super.new.img
If your devices alert for unlocked bootloader(orange state)
you can remove this annoying alert
just search on google.
if you have mediatek check this link:
https://forum.hovatek.com/thread-31664.html
Click to expand...
Click to collapse
when I put the command line
./lpmake --metadata-size 65536 --super-name super --metadata-slots 2 --device super: 6836715520 --group main: 6642450432 --partition system: readonly: 5244977152: main --image system =. / system.img --partition odm: readonly: 4349952: main --image odm =. / odm.img --partition product: readonly: 752545792: main --image product =. / product.img --partition vendor: readonly: 640577536: main --image vendor =. / Vendor.img --sparse --output ./super_new.img
it says this: Invalid sparse file format at header magic
why ????
I edited the vendor.img but something strange happens:
$stat -c '%n %s' *
super.img 3758096384
product.img 1596944384
system.img 1128718336
vendor.img 544976896
$../otatools/bin/lpmake --metadata-size 65536 --super-name super --metadata-slots 1 --device super:3758096384 --group main:3270639616 --partition system:readonly:1128718336:main --image system=./system.img --partition vendor:readonly:544976896:main --image vendor=./vendor.img --partition product:readonly:1596944384:main --image product=./product.img --sparse --output ./super.new.img
lpmake I 02-17 12:18:27 2646704 2646704 builder.cpp:1012] [liblp]Partition system will resize from 0 bytes to 1128718336 bytes
lpmake I 02-17 12:18:27 2646704 2646704 builder.cpp:1012] [liblp]Partition vendor will resize from 0 bytes to 544976896 bytes
lpmake I 02-17 12:18:27 2646704 2646704 builder.cpp:1012] [liblp]Partition product will resize from 0 bytes to 1596944384 bytes
Invalid sparse file format at header magic
Invalid sparse file format at header magic
Invalid sparse file format at header magic
BUT....
$stat -c '%n %s' super.new.img
super.new.img 3248851200
which is not divisible by 512!
Shouldn't it be 3758096384 ?
This is for android 10 custom os. Or it will work on all android 10 mobile. Can we modify FRP partition using this.
Me too, sorry
@Brepro1 thank you so much for writing this detailed guide.
Thanks to your detailed guide I was able to create an automated bash script that performs all of these steps automatically and makes all read only partitions inside super.img (system, vendor , product, etc...) into read write-able partitions again and flash to device as a brand new super.img.
It would be an honor for me if you could please try it and let me know if it works on your device. Thanks.
Here is the link:
https://forum.xda-developers.com/t/script-mount-system-as-read-write-android-10.4240703/
I have same issue as already was mentioned by NguyenNhutNam on Jan-20, i.e.:
In response to
sudo mount -t ext4 -o loop system.img edit
I'm getting this:
wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
And Linux tells me this:
file system.img
system.img: Linux rev 1.0 ext2 filesystem data, UUID=4729639d-b5f2-5cc1-a120-9ac5f788683c (extents) (large files) (huge files)
Of course, I tried:
sudo mount -t ext2 -o loop system.img edit
only to get this:
wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
Any ideas?
Figured out the reason.
Topicstarter confuses people with incorrect instructions.
The code below works great in Ubuntu
Code:
sudo su
mkdir /mnt/dir
mount -t ext4 -o loop,rw ./system.img /mnt/dir
But as soon as I try it directly inside Android I get this error:
Code:
mount: '/dev/block/loop10'->'/mnt/dir': Block device required
I guess this must be some kind of Android limitation...
use busybox mount applet instead of toybox
Brepro1 said:
Hello and welcome to my first post.
today I will talk about editing super.img and modifying system.img inside of it.
In android 10 and bigger, sometimes there is no system.img in ROM it because google starting use Dynamic Partitions for more flexible images size - more details here
Click to expand...
Click to collapse
None of this is working on the Moto One 5g Ace. All I want to do is get into /product/media/audio/ringtones to delete the crappy ones like I do with EVERY VERSION OF ANDROID and replace them with my own. I can't even open the images up and mod them because of the Read only xx. Please help and give detailed instructions like I'm a 5 year old. Have tried on linux and windows to ZERO success and LeBigMac's script didn't do xxxxx either.
Mod Edit: Post edited.
why don't use magisk native systemless method? just overlay magisk module with desired mods
https://topjohnwu.github.io/Magisk/guides.html#easy-replace
For those who can't mount system image see this twitter conversation by
https://twitter.com/i/web/status/1170404631865778177