Related
[TUT] Simple way to add Flash Disk (a.k.a "Internal Storage") to your own ROM
It's so long time, no one answer this question "How to add Flash Disk to own-cook ROM?", so I tried to find it out myself and I had found 2 different ways to do that.
As promised, or just a way to say thanks to all this community, I will share the way which more simple than the other.
It worked with Universal devices and may work with others, too!
* What you need?
1. Your ROM file (OS.nb, a.k.a nk.nba, nk.fat), different kitchens use different file names.
2. Any HEX editor, I use XVI32 here.
3. My Flash Disk image template.
* How it work?
The maximum size for a Universal OS ROM is 63 MB (66.060.288 Bytes), normally, we use less than this size for our ROM, the remain size is useless at all, so we make it useful by partitioning it.
* Which things to modify?
- Nothing in your SYS and/or OEM.
- boot.rgu in your XIP (only if it is not XIP-ready )
- First 512 Bytes of your ROM file (called Master Boot Record) (only if it is not MBR-ready )
- First 512 Bytes of my Flash Disk template file (called Boot Sector or Boot Record), it is a must edit thing to do (determine the size of Flash Disk)
* Here we go!
1. Download my file below, extract it and your nk.nbf into the same folder.
2. Run HTC64, decode your nk.nbf into two files (nk.prj and nk.fat).
3. Run HEX editor (XVI32), open nk.fat and write down two values (or just remember if you can) RED value (from $1E6 to $1E8 [or from $1F6 to $1F8 if they are not all 00]) and BLUE value (from $1EA to $1EB [or from $1FA to $1FB if they are not all 00]).
- Note: I use luca16thebig 1.4.9 BETA1 ROM for example, values may different from your ROM or this ROM after rebuilt.
00000000 ...
...
000001E0 41 88 04 FF 41 F7 80 88 01 00 80 6F 00 00 00 00
000001F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA
Click to expand...
Click to collapse
4. Close the file nk.fat and don't save if there is anything changed. Open VNIntFlashDisk.template (also in HEX editor).
Match two RED and BLUE values which you have writen down into two place RED (from $1C to $1E) and BLUE (from $13 to $14). Double check if the values are corrected and fixed into its places and save the template.
- Note: I have already done it with luca16thebig 1.4.9 BETA1 ROM's values, so just goto next step
00000000 ...
00000010 02 00 02 80 6F F8 0C 00 3F 00 FF 00 80 88 01 00
Click to expand...
Click to collapse
5. Run combine.bat from my package, your original ROM file will be saved as nk.fat.original. The file nk.fat now is Flash Disk ready!
6. Run HTC64, re-encode two files (nk.prj and nk.fat) into nk.nbf and we are done!
* What is RED and BLUE values mean?
All two values are in HEX and reversed order, in this example we have:
- RED value is 80 88 01, so its true value is 1.88.80h, change it into decimal number, we have 100.480. It is the number of used sectors (1 sector = 512 Bytes). And... luca16thebig 1.4.9 BETA1 ROM's original size is 51.445.760 Bytes = 100.480 sectors x 512 Bytes.
- BLUE value is 80 6F, its true value is 6F.80h, in decimal is 28.544. It is the number of free sectors, so our Flash Disk size is 28.544 sectors x 512 Bytes = 14.614.528 Bytes ~ 14 MB.
- As you known, OS ROM maximum size is 66.060.288 Bytes = 51.445.760 Bytes + 14.614.528 Bytes.
If your ROM is MBR-ready, each time you rebuilt the ROM, all two values will automatically adjusted in MBR and you just match these values into my Flash Disk template. That all!
- Note: luca16thebig 1.4.9 BETA1 ROM is XIP-ready and MBR-ready. So, if your ROM doesn't work with this instruction, post your 512 Bytes MBR and your boot.rgu file here. I will analyse and make instructions how to make them ready.
When I release this instruction, I tried to make all things clearly from basic to advance but it seem so hard to... eat! So, I did it in reversed order, let you can do it first, then explain and/or clarify. More details will be add later (only when someone needed)!
Link: VNIntFlashDisk.rar
Questions and answers
* What is XIP-ready?
- boot.rgu in your XIP already have Flash Disk related key and values, they are:
[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\TRUEFFS_DOC\FATFS]
"FormatTfat"=dword:1
"EnableWriteBack"=dword:1
"MountAsROM"=dword:0
"MountHidden"=dword:0
"Folder"="Flash Disk"
Click to expand...
Click to collapse
- Note: no space in key [HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\TRUEFFS_DOC\FATFS], I don't know why there is a space before TRUEFFS_DOC, please manually delete that space when copy!
* What is MBR-ready?
MBR of your ROM already have Flash Disk partition, there are 4 partition slots in MBR.
00000000 ...
...
000001B0 __ __ __ __ __ __ __ __ __ __ __ __ __ __ 11 11
000001C0 11 11 11 11 11 11 11 11 11 11 11 11 11 11 22 22
000001D0 22 22 22 22 22 22 22 22 22 22 22 22 22 22 33 33
000001E0 33 33 33 33 33 33 33 33 33 33 33 33 33 33 44 44
000001F0 44 44 44 44 44 44 44 44 44 44 44 44 44 44 55 AA
Click to expand...
Click to collapse
- The last 2 bytes (55 AA) is a signature, it means END OF MBR.
- 5th byte of each slot is value for partition type, some mostly used values are:
+ 20 - BOOT: just a name, it isn't actually a boot partition and your MBR can be without it.
+ 23 - RAWFS: your XIP partition.
+ 25 - IMGFS: your SYS + OEM partition.
+ 00 - none: free slot, all value in this slot should be 00, too.
+ 01, 04, 06 - FATFS: our Flash Disk here! In some other devices, it's called "Internal Storage".
There are some other values but that's another matter!
So, if your MBR have a FATFS partition type in last used slot, you have a MBR-ready ROM.
- Two non-MBR-ready cases: (with and without BOOT partition slot)
00000000 ...
...
000001B0 __ __ __ __ __ __ __ __ __ __ __ __ __ __ 11 11
000001C0 11 11 20 11 11 11 11 11 11 11 11 11 11 11 22 22
000001D0 22 22 23 22 22 22 22 22 22 22 22 22 22 22 33 33
000001E0 33 33 25 33 33 33 33 33 33 33 33 33 33 33 00 00
000001F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA
Click to expand...
Click to collapse
00000000 ...
...
000001B0 __ __ __ __ __ __ __ __ __ __ __ __ __ __ 11 11
000001C0 11 11 23 11 11 11 11 11 11 11 11 11 11 11 22 22
000001D0 22 22 25 22 22 22 22 22 22 22 22 22 22 22 00 00
000001E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000001F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA
Click to expand...
Click to collapse
BLUE free slot must be entered manually (by HEX caculating) to make them MBR-ready! But... how to HEX caculate values? Maybe later with a MBR sample!
VNInt said:
It's so long time, no one answer this question "How to add Flash Disk to own-cook ROM?", so I tried to find it out myself and I had found 2 different ways to do that.
As promised, or just a way to say thanks to all this community, I will share the way which more simple than the other.
It worked with Universal devices and may work with others, too!
* What you need?
1. Your ROM file (OS.nb, a.k.a nk.nba, nk.fat), different kitchens use different file names.
2. Any HEX editor, I use XVI32 here.
3. My Flash Disk image template.
* How it work?
The maximum size for a Universal OS ROM is 63 MB (66.060.288 Bytes), normally, we use less than this size for our ROM, the remain size is useless at all, so we make it useful by partitioning it.
* Which things to modify?
- Nothing in your SYS and/or OEM.
- boot.rgu in your XIP (only if it is not XIP-ready )
- First 512 Bytes of your ROM file (called Master Boot Record) (only if it is not MBR-ready )
- First 512 Bytes of my Flash Disk template file (called Boot Sector or Boot Record), it is a must edit thing to do (determine the size of Flash Disk)
* Here we go!
1. Download my file below, extract it and your nk.nbf into the same folder.
2. Run HTC64, decode your nk.nbf into two files (nk.prj and nk.fat).
3. Run HEX editor (XVI32), open nk.fat and write down two values (or just remember if you can) RED value (from $1E6 to $1E8) and BLUE value (from $1EA to $1EB).
- Note: I use luca16thebig 1.4.9 BETA1 ROM for example, values may different from your ROM or this ROM after rebuilt.
4. Close the file nk.fat and don't save if there is anything changed. Open VNIntFlashDisk.template (also in HEX editor).
Match two RED and BLUE values which you have writen down into two place RED (from $1C to $1E) and BLUE (from $13 to $14). Double check if the values are corrected and fixed into its places and save the template.
- Note: I have already done it with luca16thebig 1.4.9 BETA1 ROM's values, so just goto next step
5. Run combine.bat from my package, your original ROM file will be saved as nk.fat.original. The file nk.fat now is Flash Disk ready!
6. Run HTC64, re-encode two files (nk.prj and nk.fat) into nk.nbf and we are done!
- Note: luca16thebig 1.4.9 BETA1 ROM is XIP-ready and MBR-ready. So, if your ROM doesn't work with this instruction, post your 512 Bytes MBR and your boot.rgu file here. I will analyse and make instructions how to make them ready.
More details will be add later (only when someone needed)
Link: VNIntFlashDisk.rar
Click to expand...
Click to collapse
Very Useful!!THX You!!Post More details,PLZ
very very good news thank you
WOW
guys it's just to hard to handle for me
I'll give it a try....thx for the info....
ps: VNInt, could you just add Luca's ROM with flash disk? that would be great....
Got me 13.90MB Flash disk!!!!...
Thanks VNInt.....
I can't download the file from www.4shared.com
who can apply another download link
thanks
VNInt said:
* Which things to modify?
- Nothing in your SYS and/or OEM.
- boot.rgu in your XIP (only if it is not XIP-ready )
- First 512 Bytes of your ROM file (called Master Boot Record) (only if it is not MBR-ready )
- First 512 Bytes of my Flash Disk template file (called Boot Sector or Boot Record), it is a must edit thing to do (determine the size of Flash Disk)
Click to expand...
Click to collapse
1st let me thank you for your post. it will be quite useful in the future.
2 questions just to clarify:
- how to find if rom is XIP ready or not?
- how to find if rom is MBR ready?
Maybe these are dumb questions, sory for that.
Cheers.
ultravox said:
- how to find if rom is XIP ready or not?
- how to find if rom is MBR ready?
Click to expand...
Click to collapse
Yeah! You asked must-answer questions!
When I release this instruction, I tried to make all things clearly from basic to advance but it seem so hard to... eat! So, I did it in reversed order, let you can do it first, then explain and/or clarify.
* XIP-ready: boot.rgu in your XIP already have Flash Disk related key and values, they are:
[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\TRUEFFS_DOC\FATFS]
"FormatTfat"=dword:1
"EnableWriteBack"=dword:1
"MountAsROM"=dword:0
"MountHidden"=dword:0
"Folder"="Flash Disk"
Click to expand...
Click to collapse
* MBR-ready: MBR of your ROM already have Flash Disk partition, there are 4 partition slots in MBR.
00000000 ...
...
000001B0 __ __ __ __ __ __ __ __ __ __ __ __ __ __ 11 11
000001C0 11 11 11 11 11 11 11 11 11 11 11 11 11 11 22 22
000001D0 22 22 22 22 22 22 22 22 22 22 22 22 22 22 33 33
000001E0 33 33 33 33 33 33 33 33 33 33 33 33 33 33 44 44
000001F0 44 44 44 44 44 44 44 44 44 44 44 44 44 44 55 AA
Click to expand...
Click to collapse
- The last 2 bytes (55 AA) is a signature, it means END OF MBR.
- 5th byte of each slot is value for partition type, some mostly used values are:
+ 20 - BOOT: just a name, it isn't actually a boot partition and your MBR can be without it.
+ 23 - RAWFS: your XIP partition.
+ 25 - IMGFS: your SYS + OEM partition.
+ 00 - none: free slot, all value in this slot should be 00, too.
+ 01, 04, 06 - FATFS: our Flash Disk here! In some other devices, it's called "Internal Storage".
There are some other values but that's another matter!
So, if your MBR have a FATFS partition type in last used slot, you have a MBR-ready ROM.
- Two non-MBR-ready cases: (with and without BOOT partition slot)
00000000 ...
...
000001B0 __ __ __ __ __ __ __ __ __ __ __ __ __ __ 11 11
000001C0 11 11 20 11 11 11 11 11 11 11 11 11 11 11 22 22
000001D0 22 22 23 22 22 22 22 22 22 22 22 22 22 22 33 33
000001E0 33 33 25 33 33 33 33 33 33 33 33 33 33 33 00 00
000001F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA
Click to expand...
Click to collapse
00000000 ...
...
000001B0 __ __ __ __ __ __ __ __ __ __ __ __ __ __ 11 11
000001C0 11 11 23 11 11 11 11 11 11 11 11 11 11 11 22 22
000001D0 22 22 25 22 22 22 22 22 22 22 22 22 22 22 00 00
000001E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000001F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA
Click to expand...
Click to collapse
BLUE free slot must be entered manually (by HEX caculating) to make them MBR-ready!
* Updated to second #2 post!
VNInt said:
* What is XIP-ready?
- boot.rgu in your XIP already have Flash Disk related key and values, they are:
- Note: no space in key [HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\TRUEFFS_DOC\FATFS], I don't know why there is a space before TRUEFFS_DOC, please manually delete that space when copy!
* What is MBR-ready?
MBR of your ROM already have Flash Disk partition, there are 4 partition slots in MBR.
- The last 2 bytes (55 AA) is a signature, it means END OF MBR.
- 5th byte of each slot is value for partition type, some mostly used values are:
+ 20 - BOOT: just a name, it isn't actually a boot partition and your MBR can be without it.
+ 23 - RAWFS: your XIP partition.
+ 25 - IMGFS: your SYS + OEM partition.
+ 00 - none: free slot, all value in this slot should be 00, too.
+ 01, 04, 06 - FATFS: our Flash Disk here! In some other devices, it's called "Internal Storage".
There are some other values but that's another matter!
So, if your MBR have a FATFS partition type in last used slot, you have a MBR-ready ROM.
- Two non-MBR-ready cases: (with and without BOOT partition slot)
BLUE free slot must be entered manually (by HEX caculating) to make them MBR-ready! But... how to HEX caculate values? Maybe later with a MBR sample!
Click to expand...
Click to collapse
how to HEX caculate values?
yanqichun9527 said:
how to HEX caculate values?
Click to expand...
Click to collapse
Do not only ask, just follow my instruction, if it do not work for you or your MBR are not ready, post your 512 bytes MBR and your boot.rgu here!
I will use them as samples to continue.
Excellent tutorial VNint..Thanks!
VNInt said:
Do not only ask, just follow my instruction, if it do not work for you or your MBR are not ready, post your 512 bytes MBR and your boot.rgu here!
I will use them as samples to continue.
Click to expand...
Click to collapse
It's Ready I think.Just Want To Know Answer
yanqichun9527 said:
It's Ready I think.Just Want To Know Answer
Click to expand...
Click to collapse
OK! As we know, values are saved in reversed order and maximum size is 1.F8.00h sectors, in decimal is 129.024 sectors = 66.060.288 Bytes (1 sector = 512 Bytes).
From now, we work with HEX only.
* Some definitions:
1. Value from $1x6 (4 Bytes) (x is C, D, E or F) is address offset of partition's first sector a.k.a number of used sectors by all previous partitions.
2. Value from $1xA (4 Bytes) (x is C, D, E or F) is number of sectors in partition a.k.a size of partition.
* Some formularies:
number of used sectors = size of last partition + address offset of last partition
size of your partition = maximum size - number of used sectors
In your sample:
number of used sectors = 1.BA.00h + 1A.80h = 1.D4.80h
size of your partition = 1.F8.00h - 1.D4.80h = 80.23h
lol how about to remove them from roms?
mr4r4n said:
lol how about to remove them from roms?
Click to expand...
Click to collapse
So easy! Just zero-fill FATFS slot in your MBR!
Nice tutorial VNInt.
Sorry
I have stupid Question....
Can we have Internal Device Storage Memory more than usually?
For example, ROM from VNInt has Internal Memory ~41MB, Storage ~9.9MB & Flashdisk ~21MB. Can we combine all of the memory (internal,storage & flashdisk) to One Place memory, so we can have Internal memory 41 + 9.9 + 21 = ~73MB? ..
Sorry 4 my bad English & my stupid Question
aladin said:
Sorry
I have stupid Question....
Can we have Internal Device Storage Memory more than usually?
For example, ROM from VNInt has Internal Memory ~41MB, Storage ~9.9MB & Flashdisk ~21MB. Can we combine all of the memory (internal,storage & flashdisk) to One Place memory, so we can have Internal memory 41 + 9.9 + 21 = ~73MB? ..
Sorry 4 my bad English & my stupid Question
Click to expand...
Click to collapse
The answer is NO, but this is an exciting idea!
I will try some hack to get it true if it can be!
aladin said:
Sorry
I have stupid Question....
Can we have Internal Device Storage Memory more than usually?
For example, ROM from VNInt has Internal Memory ~41MB, Storage ~9.9MB & Flashdisk ~21MB. Can we combine all of the memory (internal,storage & flashdisk) to One Place memory, so we can have Internal memory 41 + 9.9 + 21 = ~73MB? ..
Sorry 4 my bad English & my stupid Question
Click to expand...
Click to collapse
My Stupid Question has answered... look at this
Code:
http://forum.xda-developers.com/showthread.php?t=468776
My Question Again. How to make it?
aladin said:
My Stupid Question has answered... look at this
Code:
http://forum.xda-developers.com/showthread.php?t=468776
My Question Again. How to make it?
Click to expand...
Click to collapse
* Yeah! He got it hacked!
* How it work? This trick is made by repartitioning DiskOnChip.
- This is original DOC layout: (mtty/task 28)
Binary0 Size: 0x100000
FAT0 Size: 0x4000000 (67108864 bytes = 64 MB) (63 MB maximum ROM and Flash Disk size + 1 MB splash image)
FAT1 Size: 0xA00000 (10485760 bytes = 10 MB Extended ROM size)
FAT2 Size: 0x2C70000 (46596096 bytes = 44 MB Storage size)
All Size: 0x7770000
FAT0_ADDR=0x100000,FAT1_ADDR=0x4100000,FAT2_ADDR=0x4B00000
- And this is modified DOC layout after Cotulla's ROM:
Binary0 Size: 0x100000
FAT0 Size: 0x2A00000 (44040192 bytes = 42 MB maximum ROM size)
FAT1 Size: 0x40000 (262144 bytes = 256 KB Extended ROM resized but hidden away)
FAT2 Size: 0x4C20000 (79822848 bytes = 76 MB Storage gained size)
All Size: 0x7760000
FAT0_ADDR=0x100000,FAT1_ADDR=0x2B00000,FAT2_ADDR=0x2B40000
* How to make it?
There is a DOC repartitioning tool released by mamaich before with his source code. I'm trying to got it work unattendedly, so hard now.
ok so i followed ALL the steps in this
-thread http://forum.cyanogenmod.com/topic/4999-howto-sim-unlock-your-vibrant-galaxy-s/
I created a bml3.bak, and copied it into my c:/, downloaded sgux.exe also put it in my c:/. however when i open cmd and type, "c:\sgux.exe bml3.bak" It says
Searching code block...
found...
searching codes..
then it returns to normal state, where the code?!
i even tried it with a backup of my nv_data.bin, but i get the same results
no help on this one? , I could pm you a link to my nv_data.bin, and bml3.bak and would also donate if you help me find the code.! =(
I have the same issue!
HEX editing of nv_data.bin helped!! No sim lock anymore
mount the internal SD Card on your computer
make a backup copy of the nv_data.bin file on your computer
using your favorite HEX editor open the nv_data.bin on the sdcard
jump to address 0x181468
you should see a string like this
ff 01 00 00 00 00 46 46
there are 5 different types of locks in 5 different bytes
the FF byte should be left alone
the first byte after the FF is the network lock
the next byte is the network subset lock
the next byte is the sp lock
the next byte is the cp lock
the last byte appears to be a data lock.
the 46 46 should be left alone
Change any 0x01 to 0x00 (or 0x00 to 0x01 to lock for warranty)
save and close file
unmount SD Card
Click to expand...
Click to collapse
Hi
My sd card on my Desire has become corrupted. I have a2sd+ on there; I can start from scratch (I accept the risk of installing development ROMs!) but I could really do with getting to the photos I took this morning before the card crashed.
gpart won't find the partitions at all but there's definitely still data on the card - if I strings the card I can see the DOS directory structure.
I used ROM Manager to create a 512MB ext partition with 0MB swap; I tried creating a w95 FAT32 (lba) partition of all the disk except the last 512MB using fdisk, but the partitions won't mount when I do that.
Anyone got an idea what values I should pass to fdisk to get it back?
I have
Disk /dev/sdb: 8017 MB, 8017412096 bytes
247 heads, 62 sectors/track, 1022 cylinders
Units = cylinders of 15314 * 512 = 7840768 bytes
I figured the FAT32 part is probably first, so I calculated the size of a 512MB partition as 65 blocks and assigned the first 956 blocks to the FAT partition, with the rest as the ext partition, but it made no difference.
I tried getting RM to create a partition on a second card I have and when I look at it in fdisk it looks really weird - there's messages about "Partition 1 has different physical/logical beginnings (non-Linux?)" and "Partition 1 does not end on a cylinder boundary"
Is there a way to recreate the partitions that ROM Manager would create without wiping the existing data?
Thanks!
So.. You've plugged your SD to your linux box and want to created aligned partitions with fdisk?
Follow exactly what I'm doing and your partitions will be MB aligned.
At the end you'll get a FAT32 partition for your data and an ext4 partition for data2xxxx scripts.
If you need more partitions, just adapt this tutorial to you needs.
I know some scripts or recoveries do this automatically so no need to remind me
I initially posted this in SuperNova ROM thread but it might worth a separate thread.
I've done this with an old 1GB usb key just to show you the commands, you'll have to adapt to your SD depending on it's size.
When I plugged the flash drive, it got recognized by my system as /dev/sde. It can be different on your system and once again you'll have to figure-out yourself the correct device.
Here as an example I'm splitting the flash drive in 2 partitions. 1st (512Mb) will be for the fat32 partition and will have to be formatted later. The rest is the future ext4 and is about 512MB too.
I'm gonna use 2 magic parameters with fdisk, in order to get 1024Kb clusters (2048x512 bytes) and align easily the partitions. 64 heads and 32 sectors should give 64x32=2048 so that is going to be my params here
Code:
fdisk -H64 -S32 /dev/sde
Command (m for help): p
Disk /dev/sde: 1054 MB, 1054605312 bytes
64 heads, 32 sectors/track, 1005 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sde1 * 1 1005 1028043+ 6 FAT16
Partition 1 has different physical/logical beginnings (non-Linux?):
phys=(0, 3, 45) logical=(0, 7, 10)
Partition 1 has different physical/logical endings:
phys=(509, 63, 63) logical=(1004, 3, 32)
Partition 1 does not end on cylinder boundary.
So here I have an old partition on my key (sde1). I delete it first and check that there is no more partition on the drive.
Repeat the process if you have more than 1 partition on your SD.
Make sure that it's clean before proceeding to the next step.
Code:
Command (m for help): d
Selected partition 1
Command (m for help): p
Disk /dev/sde: 1054 MB, 1054605312 bytes
64 heads, 32 sectors/track, 1005 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
Empty! Great! The key is clean. Let's partition it now
Code:
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1005, default 1): 2
Last cylinder or +size or +sizeM or +sizeK (2-1005, default 1005): 512
Command (m for help): p
Disk /dev/sde: 1054 MB, 1054605312 bytes
64 heads, 32 sectors/track, 1005 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sde1 2 512 523264 83 Linux
Ok now I have a partition starting on cylinder 2 and 512Mb in size. If your SD is 16GB, you can use for instance a size of 15GB to leave 1GB for the ext4 partition.
That would give approximately 15x1024=15360 instead of 512. Got it?
Pay attention to the value below "End". Here it is 512 so I will use the next cylinder (513) for the 2nd partition:
Code:
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (1-1005, default 1): 513
Last cylinder or +size or +sizeM or +sizeK (513-1005, default 1005):
Using default value 1005
Command (m for help): p
Disk /dev/sde: 1054 MB, 1054605312 bytes
64 heads, 32 sectors/track, 1005 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sde1 2 512 523264 83 Linux
/dev/sde2 513 1005 504832 83 Linux
Now I have 2 partitions aligned to 1MB but declared as "linux". Let's change that to FAT32 for partition 1.
Code:
Command (m for help): t
Partition number (1-4): 1
Hex code (type L to list codes): c
Changed system type of partition 1 to c (W95 FAT32 (LBA))
Command (m for help): p
Disk /dev/sde: 1054 MB, 1054605312 bytes
64 heads, 32 sectors/track, 1005 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sde1 2 512 523264 c W95 FAT32 (LBA)
/dev/sde2 513 1005 504832 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks
That's all.
Now format 1st partition to FAT32 with this guide:
credits to kd8cgo
Code:
Making an aligned FAT32 format
Anyone can figure out alignment of the partition from many sources already posted on the web.
One thing I could not find was a guide on how to align that pesky FAT32 format so that each
cluster (allocation unit) is aligned inside the boundaries of erase blocks. Well, the Linux mkfs.vfat
utility has all the options we need to make this aligned format possible!
Please view this primer to FAT32 to get an idea of how the system is laid out on your disk.
You normally have the first sector in the partition which is your Volume ID, which sits in the
reserved space which is normally the first 32 sectors in the partition. This is followed
by (2) copies of the File Allocation Tables, which vary in length when created depending on
a variety of factors, including the chosen allocation unit size. The FAT size remains constant
after creation. Please keep in mind for this discussion the physical size of a sector, which is 512 bytes.
The Golden Nugget
We will be changing the reserved sectors from the default value of 32, to a number that
we will calculate from the reported size of the FAT tables after formatting. The goal will be to
make the FAT tables end right at a 128K boundary, so each cluster of the file system will fall
neatly within erase blocks on our disk!
First we will format our disk using FAT32 paying no mind to reserved sectors. This will
report to us our FAT size as so:
[email protected]:~$ sudo mkfs.vfat -F 32 -n UserData -s 32 -v /dev/sde1
mkfs.vfat 3.0.7 (24 Dec 2009)
/dev/sdd1 has 224 heads and 56 sectors per track,
logical sector size is 512,
using 0xf8 media descriptor, with 31309312 sectors;
file system has 2 32-bit FATs and 32 sectors per cluster.
FAT size is 7641 sectors, and provides 977937 clusters.
Volume ID is 40c250bd, volume label UserData .
The juicy bits are that 2nd to last line, it tells you the size on disk of 1 FAT table.
7641 sector FAT x 2 FATs x 512 bytes/sector = 7,824,384 bytes
The above formula shows you the exact amount of space the FAT tables are using at
the beginning of your disk. This number is not usually going to be evenly divisible by
128K (131,072 bytes) as you can see 7,824,384 / 131072 = 59.695 erase block sized
chunks. What we need to do is force the end of those FAT tables to end right at
60 blocks to do so we:
131,072 x 60 = 7,864,320 bytes in 60 erase blocks
7,864,320 - 7,824,384 = 39,936 bytes remainder
39,936 / 512 = 78 sectors remain
New reserved sector count for alignment = 78
Those are all the fundamentals required to align a FAT32 partition, so that
clusters on the disk fall in line with the erase blocks of the physical cell medium.
An example of the format command required:
sudo mkfs.vfat -F 32 -R 78 -n UserData -s 32 -v /dev/sde1
Breakdown:
sudo - super user privledges
mkfs.vfat - create a FAT file system
-F 32 - 32 bit FAT (FAT32)
-R 78 - Use 78 Reserved Sectors (instead of 32)
-n UserData - drive label, up to 11 characters
-s 32 - 32 sectors per cluster 32 x 512 - 16K allocation unit size
-v /dev/sdex - Device to format
Note: If you're getting a R=0, use 256 (256x512bytes=128Kb) as the minimum R value for FAT32 is 2.
Format 2nd partition with the following command, replacing xxx with your device
Code:
mke2fs -T ext4 -b 4096 -E stride=64,stripe-width=64,hash_alg=tea -O extent,^huge_file -m 0 -L userdata /dev/xxx2
You should be good to go now!
Reserved for later
thanks for this you absolute babe <3
Did you read post #7 by CitizenJohn a few posts below the original HOWTO: Increase write speed by 'aligning' FAT32
Tends to contradict the method posted by kd8cgo for Fat32
Presumably both methods can't be correct!
Of course I did. I read many posts about SSDs to come to this tutorial.
First of all I don't use the same geometry for creating partitions.
And so far this way of formatting has always been able to give me the best of all my flash memory based devices, SD and USB keys.
Have you tried both methods? Which of the two gave you the best results?
I'm gonna try it again, maybe I missed something the first time.
Of course if there is a proven better way to get even more from our flash drives and SDs, I'll gladly learn it and change the tutorial accordingly.
EDIT:
Posted atto results for both methods on a pretty old flash drive. Left is formatted with the post you mentioned and right is with the "standard" method.
Don't see much of a difference.
I have but quite a while apart, i didn't use any software to measure transfer speed but using a sd-card adapter according to Linux i get ~10Mbs sd-card to PC and ~8Mbs PC to sd-card with the method i mentioned can't imagine a class2 sd-card could get much faster
Damn just realized to PC was Fat partition to PC : To sd-card was to btrfs linux partition
The problem is that we don't know the actual erase block size of each flash drive.
And on top of that, fat32 has block sizes of 64K max. Max erase block size for SD is 256K but is usually less.
I also just tried both methods on a faster usb key (with write speeds up to 15Mb/s) and results are the same.
But each his own and if anybody can get a better data transfer rate with the 2nd method, please report all the details here.
No worries mate thanks for the TUT, the question came out of ignorance rather than any understanding of the methodology
ziggy1001 said:
No worries mate thanks for the TUT, the question came out of ignorance rather than any understanding of the methodology
Click to expand...
Click to collapse
Actually it was an interesting question.
We assume that for any device the erase block size is 128K
But the guy in your post is formatting also a usb flash thumb drive or an SD card.
If the erase block size of his flash drive is 16k, then his file was aligned
details:
8011776 / 131072 = 61.125 File "not aligned" according to him.
8011776 / 16384 = 489. Round number, file aligned.
To get even more speed, one has to use bigger fat32 blocks (up to 64k). But then each file will use blocks of 64Ko on the drive, even if it's real size is 1Ko !
Good compromises are 16 and 32Ko blocs.
... Or you can use GParted.
problem solved.
Terepin said:
... Or you can use GParted.
problem solved.
Click to expand...
Click to collapse
Could be a good idea but:
1/ Some people have reported issues with Gparted when aligning their partitions and even some 16GB SDs were not detected by the Desire after using Gparted. On the opposites, the phone could detect them by doing it manually.
2/ Unless I'm mistaken, Gparted is NOT optimizing the FAT32 formatting process nor the EXT4 one.
is there really a difference between Gparted live and your method ?
when you wipe your SDext with your recovery, does sdext returns to default value ?
t-minik said:
is there really a difference between Gparted live and your method ?
when you wipe your SDext with your recovery, sdext returns to default value ?
Click to expand...
Click to collapse
Yes there is: I don't think that Gparted is using 1024Ko cylinders.
But most of the time, choosing MB alignment is working to align partitions. However, it is not aligning the filesystem. And like I already said, some people reported issues, that's why I published this tutorial.
If Gparted is working for you, keep using it
yep, GP works ..... but I'm really curious
and what happens when wiping SDEXT ?
If you change -s to 128 does that give 64k block size?
sudo mkfs.vfat -F 32 -R 78 -n UserData-s 32 -v /dev/sde1
Well whatever it was transfer Fat32 to PC went from ~10Mbs to ~16Mbs
If you change -s to 128 does that give 64k block size?
sudo mkfs.vfat -F 32 -R 78 -n UserData-s 32 -v /dev/sde1
Well whatever it was transfer Fat32 to PC went from ~10Mbs to ~16Mbs
Click to expand...
Click to collapse
yep, s128 gives 64k blocks.
Did you calculate R78 with the given method?
Happy that you're getting higher speed, but usually this method goal is to increase write speed to the SD
Sent from my A70S using the XDA mobile application powered by Tapatalk
Does 4ext recovery does this aligned partitioning/formatting?
Great thread, Sibere.. Will modify my installation steps to point to this TUT than Sd formatter
Does 4ext recovery does this aligned partitioning/formatting?
Click to expand...
Click to collapse
According to the dev, it should. Didn't try myself though. but like Gparted, it won't optimze the FAT32 formatting part.
Sent from my A70S using the XDA mobile application powered by Tapatalk
LiooO said:
Does 4ext recovery does this aligned partitioning/formatting?
Click to expand...
Click to collapse
sibere said:
According to the dev, it should. Didn't try myself though. but like Gparted, it won't optimze the FAT32 formatting part.
Sent from my A70S using the XDA mobile application powered by Tapatalk
Click to expand...
Click to collapse
So according to your answer i can partiton SD card in 4Ext recovery or GParted and then just do Fat32 formatting according to your guide? Can someone confirm this pls?
- for Linux only -
Stock Recovery to TWRP Backup Converter for Android system recovery <3e>
This progam is basically written for unpacking stock recovery android backup userdata_20160823_100259.backup + convert it into custom recovery nandroid backup data.ext4.win000 (but you can create your own TWRP Backups from "any" data source, too)
content and usage of bckp2win.sh is similar to bckp2cwm.sh with some slight modifications. based on previous version, it skips the checksum and unpack /data partition from userdata_00000000_000000.backup then re-pack it as TWRP Backup. optionally the screenlock pattern can be unlocked.
Requirements:
- pc with linux
- ext4 formatted hard disk
may work on ntfs, give a try (in case backup is a partition image)
Requirements (source phone):
- Android system recovery <3e> with
- "backup user data" functionality
- data must not encrypted
- external sdcard
Requirements (target device):
- root
- TWRP custom recovery
- working identical ROM pre-installed (like source phone)
before you start:
download this flashable UPDATE-sdcard.Fix.Permissions-signed.zip from osm0sis @ xda-developers to your phones memory or external sdcard - you might need it later
http://forum.xda-developers.com/showthread.php?t=2239421
TWRP and Internal Storage:
even if TWRP recovery process claims not touching /data/media, it restores files anyway. this is a great advantage side effect as we can easily restore Pictures and Files by simply including it in the backup. However, this will overwrite existing data - please don't use this option unless you know what you're doing!
if apps crashing after restoring from TWRP, this might have to do with Internal Storage - the above flashable zip will fix permissions, ownership and selinux labels for /data/media in case you manually added some files (regarding /data - of course - there is no tool in the world, which can do the same for /data partition - be warned never copy files, just always move files from one linux file system to another, and never use a windows file system)
bckp2win.sh is a linux bash script using GNU tar for creating TWRP archive files from userdata_yyyymmdd_hhmmss.backup files.
in TWRP Backup each data.ext4.win000 file represent a standalone tarball archive - this means each single archive can be unpacked for its own - without concatenating them, or having splitted files spreaded over multiple archives. unfortunately i don't know how they do it (i think TWRP use its own tar implementation), so i decided to write another bash script wich is basically doing the same thing (creating multipart standalone tarball archives):
edit: this is the main converting script (and the only file you need)
multi_tar.sh is not limited to Android system recovery <3e> userdata backup and can be used for any scope of application.
This means you can simply create TWRP Backups from "any" data source. It is summarizing files in a index file until archive size is reached and then archiving from index with GNU tar. This is a very slow procedure but it works. optionally it uses GZIP compression. (i really dont know how to check compressed file size from bash without compressing it, therefore it is compressed twice in a 2 pass way, 1-st pass is for checking size only)
edit: do not download this script, try bckp2win.sh without multi_tar.sh first (press No when asked). it is for splitting large backups only and not required in most cases
twrp_sign.sh is another bash script for creating sha2 checksums especially for TWRP Backups. But checksums can be disabled in TWRP - therefore its optional.
needs ~ 120% of free disk space and takes time about ~ 30 min, enjoy your coffee
[TUTORIAL] How to convert stock backup into TWRP backup
First of all you need to know, that userdata_yyyymmdd_hhmmss.backup files contain user data only. it is NOT a full nandroid backup like TWRP / CWM.
So we can just restore data partition from TWRP:
userdata_20160823_100259.backup -> data.ext4.win000 -> /data
The data partition contains the Internal Virtual SD Card. It is usually not included in TWRP backup. It is your decision to manually copy back to phone (recommended). But you can also restore Internal Virtual SD Card from TWRP:
/data/media/0 -> /storage/emulated/0
IMEI and WiFi MAC-Address:
/data/nvram (NOT recommended)
If there is a copy of NVRAM partition in folder /data/nvram, the script will delete it by default. However, there is a option to use IMEI and WiFi MAC-Address from backup (clone), instead from Phone.
Windows Users please click here
download UNetbootin -> https://unetbootin.github.io <- scroll this page down for tutorial
format USB flash drive FAT32install any Linux Distro to USB flash drive
Reboot your PC from USB flash drive
to access the boot menu while booting your computer:
- press the appropriate key F11 or F12 during the initial startup screen
- select the USB boot option in the BIOS boot menu
- boot Default entry
congratulations, you have
sucessfully entered your
own working Linux system!
it's easy, isn't it?
Now, come back to forum.xda-developers.com
- find Start Menu on upper left corner
- open the "Web Browser" from Favorites
- search for "bckp2win" in google
you can add your Keyboard Layout from: Settings - Keyboard
Note: performance of Firefox badly depends on USB flash drive speed
if it is too slow, try another one, or disable persistence:
- find syslinux.cfg file on USB flash drive
- open syslinux.cfg file with editor
- delete --- persistent from Try Xubuntu without installing entry
- save the changes, reboot from USB flash drive Try Xubuntu without installing entry
However, without persistence it is a read-only Live system and will lose all settings on reboot
Let's begin with preparations
- copy UPDATE-sdcard.Fix.Permissions-signed.zip to microSD card / or
- download UPDATE-sdcard.Fix.Permissions-signed.zip to target phone
- connect the source phone USB cable / or
- insert the microSD card into the PC's SD Card Reader
- copy all userdata_20160823_100259.backup files to any folder on local disk
- download all the zip files from this thread
- unpack zip files to same folder on local disk
run the shell script in Terminal
- do right-click somewhere in the backup folder, select "Open Terminal Here"
- type "sudo bash bckp2win.sh" in Terminal
- check disk space
example: when backup is 2 GB,
Avail must > 2,4 GB (120%)
when backup is 55 GB,
Avail must > 66 GB (120%)
- check file system type
Type must not vfat, fat32
all others allowed
(ntfs, ext4, fuseblk, ...)
this is very important! otherwise it will fail at the end and waste your time, and you won't know the reason. so please check carefully
- select file number to extract
- wait a long time, depends on disk speed and backup size (1~2 min / 1 GB)
Halftime break
- when message appears Press 'y' to unlock: [y/n] - Congratulations! The backup is sucessfully extracted
(if you need to edit/modify/delete files within backup this is the point for break)
- to continue with re-packing:
answer all questions with No
(or just press Enter for default):
Android sparse image (simg2img) support -> No
Flash-Friendly File System (F2FS) support -> No
unlock screenlock pattern -> No
clone old IMEI and Wifi Mac Address -> No
Restore /storage/emulated/0 -> No
use gzip compression -> No
Extract Internal Virtual SD Card to local disk -> Yes
(this will extract /data/media to TWRP folder instead, but exclude it from backup)
- wait for the script is finished
- wait for background processes
(it happens sometimes script finishes too early, if there is a data.ext4.win000 + data.ext4.win0000 file, just wait for the target size 1 GB each file)
- if checksums missing or failed, please manually run twrp_sign.sh again
(successful checksums look like this)
Finally we can restore the new backup
- copy back TWRP folder to phone
- boot into recovery mode
- create a failsafe backup (just in case...)
- move folder to the right location via MTP, or
- Advanced -> File Manager -> TWRP/BACKUPS/<serialno>/<backup folder>
- Options (blue icon on the right bottom) -> Move -> TWRP/BACKUPS/<phone name> -> Swipe to confirm
(in case converted backup is not visible in Restore list...)
- restore converted backup from TWRP
- install the flashable UPDATE-sdcard.Fix.Permissions-signed.zip
(just in case you have added files to data/media...)
- boot the phone
- move/copy all your pictures videos etc from TWRP folder back to phone
Troubleshooting
If you fail at some point please try again with more disk space or gzip compression
phone is not showing up on PC
enable Settings -> Developer options -> Select USB Configuration -> MTP:
- goto Settings -> About phone
- tab Build number seven times until you see a message
- goto Settings -> Developer options
- enable Developer options, confirm Allow development settings
- scroll down to -> Select USB Configuration
- select MTP (Media Transfer Protocol)
My source backup file is not in "userdata_YYYYMMDD_hhmmss.backup" format
This script may work with other archives, but only accept "userdata_YYYYMMDD_hhmmss.backup" pattern as input file name. But you can specify any input file or folder as parameter. the script will scan the folder for known archive types and link files into script folder:
sudo bash bckp2win.sh ~/Android/Backup/TWRP/2016-08-23--10-02-59/data.ext4.win*
mkdir: cannot create directory ‘b2wtmp’: File exists
There was a previous session left, delete the folder ‘b2wtmp’ and try again. You can keep previous session for testing purposes with parameter -f force unpack:
sudo bash bckp2win.sh -f
ERROR: something goes wrong. check disk space
The GNU tar unpacking or archiving process may fail for various reasons. However, you can suppress error messages and skip this exit point for testing purposes with parameter -f force unpack. If you run out of disk space, you can exclude folders from backup with parameter -e --exclude PATTERN, for example:
sudo bash bckp2win.sh -e */com.google.android.googlequicksearchbox*
Bugs & Known Issues
extractTarFork() process ended with ERROR: 255
probably bug in script. at the moment, only solution is manually restoring backup files. i know its annoying but i don't know the reason, yet.
- download GNU tar for android
- unpack the zip and copy the tar binary to phone
- in TWRP, copy tar binary to /cache, then wipe data
- in TWRP, go back to -> Advanced -> Terminal
- in Terminal, change directory to backup folder, then run for each file
Code:
chmod 0755 /cache/tar
cd /external_sd/TWRP/BACKUPS/<phone>/2016-08-23--10-02-59*
/cache/tar --selinux --xattrs -P -vxpf data.ext4.win000
(or for compressed files)
busybox gzip -cd data.ext4.win000 | /cache/tar --selinux --xattrs -P -vxp
Please post here for support i will answer your questions
found a multipart image, merge files
==================
... merged
try to unpack ...failed
try to mount as *...failed
skip first 512 bytes and try to mount again as ext2, as rfs, as fsfs
... failed
no files in folder "data"
exiting script
What can I do ?
My phone model is Bluboo S1 with Android 7.0, before I wiped all out I was made backup with stock backup and now I have TWRP 3.2.1. Now I have phone flashed with BLUBOO_S1_Helio_P25_L_V04_20170908 and is working but I want put my old userdata because I think there is all my 230 apps already installed.
Please check if your userdata_20180926_141645.backup is ext4 image. Each file start with a 512 byte checksum header, followed by partition image. The ext4 Super Block will start at offset 1024 bytes (from partition). The ext4 magic number 0xEF53 you can find at offset 0x38 (from the Super Block start)
We can skip (512 bytes) checksum + (1024 bytes) unused padding, because the Super Block start is at (1536 bytes) = 0x600 in this case
Please note the ext4 magic 0xEF53 at offset 0x638
encrypted files not supported
If the files look like this (no zeros within, after skip 512 bytes checksum) it is probably a raw backup of encrypted data partition
you can check with hexdump
Code:
hexdump -C -n1600 userdata_20180926_141645.backup
Unfortunately, these userdata backups are pretty useless since android 7.0 (encrypted by default), because they will not backup efs/metadata. If you wipe data from stock recovery, the metadata is wiped, too. It is impossible to decrypt data without encryption key (which is stored in metadata).
If you are really lucky maybe its not to late, do a read back of metadata partition with SP Flash Tool. Furthermore, check if userdata backup is encrypted
. . .
If the files look like this (no zeros within, after skip 512 bytes checksum) it is probably a raw backup of encrypted data partition
you can check with hexdump
Code:
hexdump -C -n1600 userdata_20180926_141645.backup
[/QUOTE]
***
there was also too little disk space and I installed the new Ubuntu on the other computer so I'll try it later
if you prefer to re-pack data by yourself, stop the script when it ask for unlock screenlock pattern (CTRL + C)
enter parent directory of data folder
run multi_tar.sh - while first arg is destination and all other args are source folders
Note you can change the output to any file or folder:
replace "data.ext4.win" with "/media/xubuntu/my-drive/Android/my-output-folder/my-file-name"
Code:
sudo bash multi_tar.sh -z -L 1048576 data.ext4.win data --transform 's,^data,/data,'
(with parameters -z for compression and -L for split size, and some string replacement within the archive - replaceing "data" with "/data" in this case)
there are still some bugs i am struggle with, restoring in TWRP fails when a single file within backup is failed.
Please check out my other solution for encrypted backup. You can restore TWRP backup from this zip instead of restoring from TWRP menu
https://forum.xda-developers.com/showthread.php?t=3899918
aIecxs said:
We can skip (512 bytes) checksum + (1024 bytes) unused padding, because the Super Block start is at (1536 bytes) = 0x600 in this case
Click to expand...
Click to collapse
Hi,
i have exact these offset you wrote here but i do not have success to mount my data. Have you an idea what could be wrong on my side?
Code:
000001d0 ab 0a 44 a3 c5 ee 69 fa 44 78 c2 ca ec 13 bb f5 |..D...i.Dx......|
000001e0 38 f4 e7 ca f2 7c 49 e3 a2 a0 d8 1e e3 f9 94 c5 |8....|I.........|
000001f0 f5 f5 3e c2 6a bc 8a 58 1c ef 0e 8a 91 29 c4 99 |..>.j..X.....)..|
00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000600 00 40 02 00 80 fc 08 00 00 10 00 00 77 e3 00 00 |[email protected]|
00000610 da 10 02 00 00 00 00 00 02 00 00 00 02 00 00 00 |................|
00000620 00 80 00 00 00 80 00 00 00 20 00 00 0f 8e a4 54 |......... .....T|
00000630 0f 8e a4 54 6c 02 ff ff 53 ef 01 00 02 00 00 00 |...Tl...S.......|
Code:
try to mount as ext4
...failed
skip first 512 bytes and try to mount again as ext4
...failed
try to mount as ext3
...failed
skip first 512 bytes and try to mount again as ext3
...failed
try to mount as ext2
...failed
skip first 512 bytes and try to mount again as ext2
...failed
try to mount as rfs
...failed
skip first 512 bytes and try to mount again as rfs
...failed
try to mount as f2fs
...failed
skip first 512 bytes and try to mount again as f2fs
...failed
./bckp2win.sh: Zeile 796: cd: .//b2wtmp: Datei oder Verzeichnis nicht gefunden
WARNING: no files in folder "data"
exiting script
Thanks
script looks buggy, maybe wrong mount options, or it makes a difference when called with "sudo bash bckp2win.sh"
Hello all.
Is there a way to do the inverse (system.ext4.win -> system.img) ? Does anybody know a link to instruction ?
With that system.img, I can then do: Bootloader> fastboot -S 130M flash system system.img
(if bootloader is unlocked).
Thank you everyone.
yes it is possible, check my edited reply later in 10 hours
edit: see reply in other thread
https://forum.xda-developers.com/showthread.php?t=4015725
@e5e197740b what is the error message when using this tool?
aIecxs said:
@e5e197740b what is the error message when using this tool?
Click to expand...
Click to collapse
Do I need a rooted Tablet for this? I'm not sure I understood all the instructions..
Edit:
So I ran bckpwin.sh in an Ubuntu VM, here is the output:
Code:
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
#
#
# This script converts "Android system recovery <3e>" stock recovery file
# "userdata_YYYYMMDD_hhmmss.backup" into custom recovery nandroid backup file.
#
# executing file system must be ext2/3/4 otherwise app permissions will be lost.
#
#
run this script as root. type "sudo -i" or "sudo bash bckp2win.sh".
are you root?
Press 'y' to continue: [y/n] y
2,0G userdata_20200609_220537.backup
2,0G userdata_20200609_220537.backup1
2,0G userdata_20200609_220537.backup2
2,0G userdata_20200609_220537.backup3
2,0G userdata_20200609_220537.backup4
2,0G userdata_20200609_220537.backup5
110M userdata_20200609_220537.backup6
13G total
Dateisystem Typ Größe Benutzt Verf. Verw% Eingehängt auf
/dev/sda5 ext4 98G 45G 48G 49% /
WARNING: Make sure enough free disk space - NOT checked during process!!
1) userdata_20200609_220537.backup
select file number to extract (q to quit): 1
1) "userdata_20200609_220537.backup"
try to unpack as tar (multipart image support)
...failed
WARNING: No ext4 magic number detected. Try it anyway? [y/n] y
OPTION: install support for (EXT4) Android sparse image (simg2img)? [y/n] y
Paketlisten werden gelesen... Fertig
Abhängigkeitsbaum wird aufgebaut.
Statusinformationen werden eingelesen.... Fertig
Paket android-tools-fsutils ist nicht verfügbar, wird aber von einem anderen Paket
referenziert. Das kann heißen, dass das Paket fehlt, dass es abgelöst
wurde oder nur aus einer anderen Quelle verfügbar ist.
Doch die folgenden Pakete ersetzen es:
android-sdk-libsparse-utils android-sdk-ext4-utils
E: Für Paket »android-tools-fsutils« existiert kein Installationskandidat.
OPTION: install support for (F2FS) Flash-Friendly File System? [y/n] y
Paketlisten werden gelesen... Fertig
Abhängigkeitsbaum wird aufgebaut.
Statusinformationen werden eingelesen.... Fertig
Die folgenden zusätzlichen Pakete werden installiert:
libf2fs-format4 libf2fs5
Die folgenden NEUEN Pakete werden installiert:
f2fs-tools libf2fs-format4 libf2fs5
0 aktualisiert, 3 neu installiert, 0 zu entfernen und 3 nicht aktualisiert.
Es müssen 185 kB an Archiven heruntergeladen werden.
Nach dieser Operation werden 568 kB Plattenplatz zusätzlich benutzt.
Holen:1 http://de.archive.ubuntu.com/ubuntu focal/universe amd64 libf2fs5 amd64 1.11.0-1.1ubuntu1 [14,1 kB]
Holen:2 http://de.archive.ubuntu.com/ubuntu focal/universe amd64 libf2fs-format4 amd64 1.11.0-1.1ubuntu1 [16,6 kB]
Holen:3 http://de.archive.ubuntu.com/ubuntu focal/universe amd64 f2fs-tools amd64 1.11.0-1.1ubuntu1 [154 kB]
Es wurden 185 kB in 0 s geholt (608 kB/s).
Vormals nicht ausgewähltes Paket libf2fs5:amd64 wird gewählt.
(Lese Datenbank ... 189666 Dateien und Verzeichnisse sind derzeit installiert.)
Vorbereitung zum Entpacken von .../libf2fs5_1.11.0-1.1ubuntu1_amd64.deb ...
Entpacken von libf2fs5:amd64 (1.11.0-1.1ubuntu1) ...
Vormals nicht ausgewähltes Paket libf2fs-format4:amd64 wird gewählt.
Vorbereitung zum Entpacken von .../libf2fs-format4_1.11.0-1.1ubuntu1_amd64.deb ...
Entpacken von libf2fs-format4:amd64 (1.11.0-1.1ubuntu1) ...
Vormals nicht ausgewähltes Paket f2fs-tools wird gewählt.
Vorbereitung zum Entpacken von .../f2fs-tools_1.11.0-1.1ubuntu1_amd64.deb ...
Entpacken von f2fs-tools (1.11.0-1.1ubuntu1) ...
libf2fs5:amd64 (1.11.0-1.1ubuntu1) wird eingerichtet ...
libf2fs-format4:amd64 (1.11.0-1.1ubuntu1) wird eingerichtet ...
f2fs-tools (1.11.0-1.1ubuntu1) wird eingerichtet ...
Trigger für man-db (2.9.1-1) werden verarbeitet ...
Trigger für initramfs-tools (0.136ubuntu6) werden verarbeitet ...
update-initramfs: Generating /boot/initrd.img-5.4.0-37-generic
Trigger für libc-bin (2.31-0ubuntu9) werden verarbeitet ...
found a multipart image, merge files:
../userdata_20200609_220537.backup
../userdata_20200609_220537.backup1
../userdata_20200609_220537.backup2
../userdata_20200609_220537.backup3
../userdata_20200609_220537.backup4
../userdata_20200609_220537.backup5
../userdata_20200609_220537.backup6
(please wait - up to 15 min - don't worry computer is still alive)
...merged
try to unpack as sparse ext4 image (skipping ... )
...failed
try to mount as ext4
...failed
try to mount as ext3
...failed
try to mount as ext2
...failed
try to mount as rfs
...failed
try to mount as f2fs
...failed
no files in folder "data"
caching file to disk again for second run
(please wait - up to 15 min)
try to mount as ext4
...failed
skip first 512 bytes and try to mount again as ext4
...failed
try to mount as ext3
...failed
skip first 512 bytes and try to mount again as ext3
...failed
try to mount as ext2
...failed
skip first 512 bytes and try to mount again as ext2
...failed
try to mount as rfs
...failed
skip first 512 bytes and try to mount again as rfs
...failed
try to mount as f2fs
...failed
skip first 512 bytes and try to mount again as f2fs
...failed
./bckp2win.sh: Zeile 796: cd: .//b2wtmp: Datei oder Verzeichnis nicht gefunden
WARNING: no files in folder "data"
exiting script
So it had to load some stuff of the internet (namely simg2img and F2FS and its dependencies), but it seemed to run through properly, apart from not retrieving any data.
simg2img or f2fs is not required, it's from the days i wasn't aware of backup format
try to unpack as tar ...failed
mean it doesn't extract with tar(gz) with or without 512 bytes header
WARNING: No ext4 magic number detected
mean it did not find hex 53 ef with or without 512 bytes header (f2fs/ext4 is checked both)
but it tries to mount anyway
this could mean
a) script is not working (@matrix4you claimed this, too)
b) header is not 512 bytes
c) backup is encrypted
a) and b) can be double checked with unencrypted backup. do a factory reset then right after create another backup without booting android. this should give you 12 GB backup which can be zipped into less a few MiB because it is empty ext4 image
edit: did you run the script via 'sudo ./bckp2win.sh' that would explain the bug in line 796? usage is 'sudo bash bckp2win.sh' maybe behavior is different
aIecxs said:
simg2img or f2fs is not required, it's from the days i wasn't aware of backup format
try to unpack as tar ...failed
mean it doesn't extract with tar(gz) with or without 512 bytes header
WARNING: No ext4 magic number detected
mean it did not find hex 53 ef with or without 512 bytes header (f2fs/ext4 is checked both)
but it tries to mount anyway
this could mean
a) script is not working (@matrix4you claimed this, too)
b) header is not 512 bytes
c) backup is encrypted
a) and b) can be double checked with unencrypted backup. do a factory reset then right after create another backup without booting android. this should give you 12 GB backup which can be zipped into less a few MiB because it is empty ext4 image
edit: did you run the script via 'sudo ./bckp2win.sh' that would explain the bug in line 796? usage is 'sudo bash bckp2win.sh' maybe behavior is different
Click to expand...
Click to collapse
I did go sudo bash.
I can try the reset-backup thing.
aIecxs said:
simg2img or f2fs is not required, it's from the days i wasn't aware of backup format
try to unpack as tar ...failed
mean it doesn't extract with tar(gz) with or without 512 bytes header
WARNING: No ext4 magic number detected
mean it did not find hex 53 ef with or without 512 bytes header (f2fs/ext4 is checked both)
but it tries to mount anyway
this could mean
a) script is not working (@matrix4you claimed this, too)
b) header is not 512 bytes
c) backup is encrypted
a) and b) can be double checked with unencrypted backup. do a factory reset then right after create another backup without booting android. this should give you 12 GB backup which can be zipped into less a few MiB because it is empty ext4 image
edit: did you run the script via 'sudo ./bckp2win.sh' that would explain the bug in line 796? usage is 'sudo bash bckp2win.sh' maybe behavior is different
Click to expand...
Click to collapse
So what I did now:
Reset the device, without rebooting make a backup:
I still creates 7 files 12 gigs in total.
The are, as expected, seemingly completely empty, apart from the first 512 Bytes, allthough even those are empty in files 4-7.
The first 8 bytes are always the same, both in the empty and the real backup, if the first 512 exist at all.
Here are the first 512 bytes from the first file of the empty backup:
Code:
AB EF CA 9C B0 C5 4A 0A 38 3D BE C5 5E 4F B5 ED DB 14 73 B2 5C 52 64 1B FA 57 83 A1 6A 63 BF 7A 81 4F AF 09 8D 90 07 18 D6 00 68 0B 94 80 62 62 70 4E 35 8E 28 35 2A 65 63 8D 60 6C E6 1B BF 84 22 DC 8D 22 FF BE FA D8 40 A5 37 01 14 DA 80 C0 92 4A 41 24 EA 80 F6 B3 4B 2C 45 A1 17 99 E0 2D DF F5 7E 9D 10 3B 39 DD FE 1E 48 D4 A8 24 81 B4 E1 91 55 BC 2B 80 7D 9E 1B 1C 82 C5 BD 98 59 66 FF 07 00 1E 4F 41 0B 87 4F 17 CC C4 12 2A 50 1E 70 2F 96 61 09 EF 37 B6 A3 7C 01 9C 09 49 D5 0E 98 91 EB 68 09 FE 51 6D D1 7C 4A 15 02 FF 12 5B B2 CA 31 FD 9F 07 8E 26 FA 56 72 F7 BD 55 85 BB F6 F7 59 8C 11 54 B4 E8 04 3E 10 41 B9 32 54 19 A8 BF 0C 5B 40 5D 37 72 68 BA 46 2A E5 C6 98 28 C1 79 DD F6 8E 8F AD 75 43 BA DA 1B 48 DD B9 F8 AB BC 02 6E E3 62 E5 C8 E2 20 60 43 D9 C5 47 E4 81 04 13 A9 04 8C 1E B3 0B 9D 59 B7 D4 CF A9 2F D9 96 93 BA 02 F9 A8 4E D5 FD 7A B2 74 E4 DB 9C 8B 4C 94 CF CD 61 C6 98 E7 88 B7 51 79 E9 0B FB B2 DE 63 66 A5 08 44 24 CE 76 E2 F2 7D 16 80 DA BF 3E F4 58 B3 8C 76 83 62 ED 40 CC 73 6C 28 B8 C7 78 65 8E 11 BB 8D B7 64 4B 66 B7 32 F8 88 14 BB 54 17 D0 52 3D 5A 47 B8 06 D8 00 94 7C 0B FF 7B 07 6E 4A B7 23 A9 6E 91 BB FE 6F 61 C8 0B C5 B3 E9 96 FB 9F 21 33 F8 BF 0A 9A C7 3E ED 31 B5 F5 66 BD 12 1D E9 1F D7 B2 DA 2E FC 3C 22 F4 F4 B6 10 10 BA 7E 9A 41 76 B0 43 91 DE 2E 31 64 59 DD 1E A5 5D 20 D9 5E 17 78 59 12 35 66 35 3A E6 7B 67 D5 64 D7 93 9C F4 01 47 6C 0F 92 4D 45 F9 B8 34 81 0A EA 47 1A 6C 4D 68 C1 6E A0 6C DE 7D 3E 33 06 82 D2 BF 05 82 5D 0E CF 5B E3 15 98 BB 51 98 79 A2 05 8F 7B BC 70 D1 76 35 67 A8 A0 EF
Is there any chance left?
no there is nothing you can do. what does the script say to empty backup? does it mount ext4 image?
So I had to take a few days off, but I ran the script on the empty backup.
It does mount it as a ext4 image. But I can't find the folder it mounted it to.