Dear All,
I have been building some custom Android ROMS for the past few years, and my latest toy is a Flytouch 3 I wanted to share some of my findings and methods here, perhaps other newbies like me will find it useful.
1. FT3 ROM contents:
FT3 ROMs will be in RAR or ZIP format and have 2 or 3 files in them as explained below. Just unzip or unrar the package to use them:
firmware2 - flash tool which burns the firmware ( I guess...lol)
bootloader-discovery - 7zip file which contains the bootloader
IMPORTANT! Do not flash a wrong bootloader to your device or it will be bricked! Actually this is the only way you can brick the thing, it's pretty sturdy in that sense. If you are not sure, do not copy bootloader-recovery to your sdcard when flashing, it will boot with your existing bootloader anyway.
firmware-discovery - 7zip file which contains the firmware and it's the main file you want to edit.
2. firmware-discovery contents:
This is the easy part! Modifying this file will let you modify system and data partitions. You can also use different ramdisks and kernels compatible with your device. It's a 7zip file and you can unpack and repack easily with the 7-zip file manager (http://www.7-zip.org/).
I use 7-zip on Windows although I use Ubuntu Linux as my main OS. There is something wrong with my 7-zip installation most probably, but I was too lazy to check it If you are on Windows anyway, you are good, just download and use 7-zip from the URL above. I advise that you do all your firmware-discovery editing within 7-zip file manager. That way, you won't have to worry about re-packing with the correct 7-zip settings. At least, that's how I do it and had no problems.
firmware-discovery will have these files in the 7zip package:
uImage - kernel
Very important Your kernel will make the difference between having and not having certain functionality, even if your hardware supports it.
uImage_recovery - recovery file - I beleive this is what runs when you push menu+power.
ramdisk.img - ramdisk
This is one of the interesting files you might want to play with. Ramdisk is a simple file system with some crucial files, mainly init.rc that lets you control the boot process. You can also stick a initlogo.rle image here to display a splash image in initial boot. More details below at title 4.
data.tar - /data partition - tar archive
All user data is stored here, including apps and user definable settings. This is what gets wiped when you do a recovery by pushing menu+power.
system.tar - /system partition - tar archive
System data, drivers and apps, including Android OS itself (except kernel). Certainly the most interesting part for the average modder
3. Editing system.tar and data.tar:
This might be the bulk of your modding, and most probably what will be most visible to the end user.
I very strongly suggest that you use a Linux system to edit these files. If you use Windows, file permissions will not be saved! If Windows is your main OS, simply install a VM host such as VirtualBox and install your fav Linux distro inside it. Most of them will have an Archive Manager with a GUI and will certainly support TAR. Even without a GUI, you can use "tar" and "untar" commands from a terminal to unpack and repack these files. When you are on Linux, TAR files will preserve file ownership and file permissions. This can be very crucial in your modding.
For modders who are new to Linux, I can recommend Ubuntu. It's very user friendly and easy to use in my opinion.
When you are done editing system.tar and data.tar copy them to your Windows box and drop and them to firmware-recovery file within 7-zip file manager.
4. Unpack, Edit and Repack ramdisk.img:
Now we are on to more serious stuff ramdisk.img consists of a 64 byte U-Boot header, followed by a gzip compressed ramdisk cpio image. So we need to strip the first 64 bytes, and use gunzip to extract it to a folder. After making the changes in the folder, we need apply cpio+gzip compression to the folder and add the 64 byte u-boot header again. You will need to use Linux and install the mkimage tool for these operations.
Here's the command to install mkimage tool in Ubuntu:
Code:
sudo apt-get install uboot-mkimage
It sounds complicated, but no worries, I have some shell scripts that can do all that for you! Thanks to leeh33 from Techknow for the scripts below:
unpack_ramdisk.sh
Code:
mkdir ramdisk
cd ramdisk
dd if=../ramdisk.img bs=64 skip=1 of=ramdisk.gz
gzip -dc ramdisk.gz | cpio -idmv
rm ramdisk.gz
repack_ramdisk.sh
Code:
cd ramdisk
find . | cpio -o -H newc -C 512 | gzip -n > ../ramdisk.cpio.gz
cd ..
mkimage -A ARM -O Linux -T ramdisk -C none -a 0x41000000 -e 0x41000000 -n hRamdisk -d ramdisk.cpio.gz ramdisk.img
rm ramdisk.cpio.gz
Create the scripts above using a text editor or download the attachment to this post and put the scripts together with your ramdisk.img file in a new folder. Executing unpack_ramdisk.sh will create a folder named ramdisk and extract the ramdisk contents here. You can edit e.g. init.rc in this folder, or add a static bootlogo. To add a custom bootlogo create a initlogo.rle image file by following this guide: http://www.androidtablets.net/forum/android-tablet-hacking/438-how-setup-static-bootlogo-before-animated-boot-animation.html Then place the resulting initlogo.rle file directly in the ramdisk directory.
After you are done editing the ramdisk folder, simply run repack_ramdisk.sh and your new ramdisk.img will be created, replacing the old one. Copy the ramdisk.img file to your Windows box again and drop it to firmware-recovery file within 7-zip file manager.
That's it, you have a custom ROM now! Please feel free to ask questions or correct me if you think I have any errors in the info above.
Happy Modding!!!
Tolga
Big problem about my device...
Thank you for this useful post!
I have seen a good knowledge about these devices(flytouch 3) and decided to post my question here:
I have a big problem! I used IUW Burning tool to flash my device. my device is:
SuperPad III (As box says)
Model: ANDR P1001
I decided to try tim's custom rom. It doesn't work. After restarting the device, screen become black (There was no connection
with screen, it was not turned on). The device had vibration when I pressed power button. It continued vibrating periodically(like it was restarting).
After that I flashed my device using IUW with most likely firmware, but during the process I FORGOT TO CHOOSE userdata.img FILE!
Now the device can't even communicate with pc.
OTG method is not working. The device no more vibrates at start up! At motherboard led is lighting when pressing power button.So IS MY DEVICE DEAD? Or is there any solution in this situation(Alternative communication, or whatever...)
Ok, so I know many users are looking forward to create a custom rom. Where are you gonna get the system files from? Sure you can get it from a CWM backup. You can also extract the system files and the respective kernel from PDA.tar. Sure everyone knows it's in there but very few exactly know how to extract them. The system files are in the file "system.img".
So the 1st step is to open the PDA.tar using 7-zip and extract the system.img to a particular folder.
Download sgs2toext4.jar from attachment. It's a .jar file created by dphrozen to convert .img files to .ext4.img. Place that file anywhere in C:\ and then open the Command Prompt. In the command prompt, set the directory to the folder where sgs2toext4.jar is placed. Then type this command :-
Code:
java -jar sgs2toext4.jar
A window will open which is gonna ask you to drop the .img file in it. Simply drag the system.img to the window and the process of converting it to system.ext4.img will start. After the process is complete, you'll find system.ext4.img file in the same folder where your system.img file was placed.
Now, download Linux Reader. Install and then open it. At top left, go to Drives -> Mount Image. A window will open up. Browse to system.ext4.img and select it.
That's it! Now you can view all the system files. Export all of them to a particular folder. After exporting, you'll see an extension .efs.ntfs at the end of each file. Just delete that extension and you're good to go!
Hi/Hello/good evening!
I have a broken MTK phone waiting for spareparts and one backup file from its SDCard made with CWM
name userdata_20130615_140224.backup where I need to extract contact data from.
How to do it?
I have been searching a whole while now but could not find one single post about opening this archive.
Pls help!
Thank you,
Erwin
If u have any spare phone with Titanium Backup installed then u can transfer ur backup to tat mobile's sd card and use TB to extract data from Nanoid Backup..
Sent from my WT19i using xda premium
expanding file
hi,
unyaffs doesnt work on that .backup file and I dont have a spare phone lying around. I got the file on windows
and please after reading 20.000 tuts on "backups" can there be one that explains how to extract the file
with windows?
thank you,
Erwin
NOW!!!
ewoewo said:
hi,
unyaffs doesnt work on that .backup file and i dont have a spare phone lying around. I got the file on windows
and please after reading 20.000 tuts on "backups" can there be one that explains how to extract the file
with windows?
Thank you,
erwin
Click to expand...
Click to collapse
tell me the tec solution internet!
ewoewo said:
Hi/Hello/good evening!
I have a userdata#######.backup and I cant enter into recovery mode. once i try toenter recovery mode in boots and shuts down without entering into recovery mode. I want to restore this backup file. Help me out
Thank you!!
Click to expand...
Click to collapse
It's a 512 byte header in front of a "tar.gz"
hexdump -C userdata_xxxxxxxx_yyyyyy.backup | lessshowed the magical zip header "1f8b0800 00000000" at offset 0x200
So, skipping 512 bytes, the rest can be gunzip | tar
dd if=userdata_xxxxxxxx_yyyyyy.backup bs=512 skip=1 | gunzip -c | tar xv
Code:
dd if=/home/user/_backups/E380-android-phone/userdata_20160126_223433.backup bs=512 skip=1 > /home/user/_backups/E380-android-phone/userdata_20160126_223433.backup.dd-out.tar.gz
Worked for me. Then just used KDE's archive opener (Ark) to view the files, looks to have everything in there except keys and whatnot.
if you prefer a linux solution use this script it converts android backup to nandroid backup
http://forum.xda-developers.com/showthread.php?p=65374561
Hello! Super-Developers, mega-programmers and smart hackers!
Nobody does not know how to open *. backup file?
No one can't help in this problem suffering millions of android/windows users?
You suggest them to install Unix/Linux system, - everybody to become hackers and programmers?!
maybe in the World exists GUI- programme to open *. backup file?
Someone can solve this problem of the century?
Philoandr said:
Hello! Super-Developers, mega-programmers and smart hackers!
Nobody does not know how to open *. backup file?
No one can't help in this problem suffering millions of android/windows users?
You suggest them to install Unix/Linux system, - everybody to become hackers and programmers?!
maybe in the World exists GUI- programme to open *. backup file?
Someone can solve this problem of the century?
Click to expand...
Click to collapse
What do you expect. You are trying to edit a Linux based file. Not to mention a closed sourced compressed file.
There are free programs like OSFMount or DiskInternals Linux Reader for mounting (android) linux file system images on Windows. Also 7-Zip for Windows can extract files from linux file system images.
However, there is metadata like user identifier and group ownership, file permissions, SELinux context (user role type range) stored for each file, which gets lost once you have saved linux files to windows disk. Without this metadata you can not restore files to phone.
Regardless you can extract files in windows if you just care about pictures, music files etc. Maybe one day there is a developer who may write such a program no matter its useless for restoring complete user data to phone.
Regarding the userdata*.backup files (no cwm) this is a chinese feature not provided by google android open source project. Without source code its nearly impossible to analyze the checksum algorithm. You can not re-create a userdata*.backup file.
What you can do is unpacking it. make a guess. cut off the first 512 byte header of each file which is probably the checksum. then merge the files together. Now you have a file for further analysis. Its probably either a ext4 disk image or a (gz compressed) tarball archive. a disk image can be mounted as a new drive, metadata keeps preserved in this case. a tarball archive must extracted with gnu tar to linux file system. when extracting with 7-Zip you will lose metadata.
There is a risk of tarball archive is unheadered Tar, in this case 7-zip will not extract it. If your phone is encrypted, the disk image is probably encrypted disk image (thats where the hacker stuff starts).
If you want to remove the checksum in windows, there is a small problem. There is no such command. You can use a third party command line utility Trunc to try it. (truncate deletes only from tail)
Another way for Windows, you can use some linux utils with cygwin. I don't know exactly how to do it, but @Doc_cheilvenerdi.org released a Windows script Odinatrix which converts unheadered Tar to Ustar Tar. From this you can use some linux utils (i don't know if it works for converting backups, too)
1. unpack the Odinatrix-WINDOWS.zip file. copy the folder named "files" containing cygwin linux utils. rename the folder to bash (or any meaningful name)
2. create a new folder on Windows Desktop
3. move the backup files to <some folder>
4. open the Windows Command Prompt
goto Windows Start - Run...
type: %SystemRoot%\system32\cmd.exe - press OK
now type the following commands in Windows terminal console:
5. add bash folder to windows environment variable %PATH%
(example: PATH=%PATH%;C:\Program Files\bash)
Code:
PATH=%PATH%;<path to bash folder>
6. change directory and list backup files
Code:
cd %UserProfile%\Desktop\<some folder>
dir /a userdata*.backup*
7. cut off 512 byte header from each file using (cygwin) gnu "dd" linux util like @ektoric suggest in post #6.
do this for each file, skipping 512 bytes only (without unpacking). replace the wildcard * with <date>_<time>
Code:
dd if=userdata*.backup of=userdata*.part0 bs=512 skip=1
dd if=userdata*.backup1 of=userdata*.part1 bs=512 skip=1
dd if=userdata*.backup2 of=userdata*.part2 bs=512 skip=1
dd if=userdata*.backup3 of=userdata*.part3 bs=512 skip=1
8. merge the files with windows
Code:
copy /v /y userdata*.part0 /b + userdata*.part1 /b + userdata*.part2 /b + userdata*.part3 /b userdata*.img /b
9. try to mount the userdata*.img with OSFMount. if not working, try extracting with 7-zip.
Besides this linux is no hacker-os and just as easily operated as windows. You can boot a live distribution from usb-stick without installing.
for further questions, please visit the main thread
[Q] How to Extract data from .Backup file ? - Created with android system recovery
zelendel said:
What do you expect. You are trying to edit a Linux based file. Not to mention a closed sourced compressed file.
Click to expand...
Click to collapse
Thank you for the answer!
Please comment next:
1) Video how to extract files from
"Linux based, Not to mention, a closed sourced compressed" backup file with metadata:
Распаковка и извлечение файлов из резервной копии - YouTube
m.youtube.com/watch?v=HNXed7RGR2g
2) quote from Android Backup Extractor's README-TXT-instruction:
"Usage:-- Windows: very easy"?
Android Backup Extractor - -- sourceforge.net/projects/adbextractor/files
-sourceforge.net/projects/adbextractor/reviews?source=navbar
I tried to use Android Backup Extractor In WinXp SP3
but windows version of this app shows cygwin&some other files missing.
Do you know why it doesn't work?
I don't need to recover backup on its own place.
My target is to extract and open few files: photos, videos, documents...
Strangely 7zip also doesn't work. Thank you in advance!
adbextractor is for adb backups *.ab only (btw open source).
cygwin is a implementation of linux utils running in windows.
the above solution uses cygwin, did you try?
aIecxs said:
adbextractor is for adb backups *.ab only (btw open source).
cygwin is a implementation of linux utils running in windows.
the above solution uses cygwin, did you try?
Click to expand...
Click to collapse
Thank you for first very detailed answer!
It seems that you have a remarkable grasp of this subject,
that's why I ask you to comment my previous post about video and Android Backup Extractor.
Your second post is not clear for me: adbextractor is only for *.ab files??
I have few files cygwin, is it a separate program??
I attempted to run file start.exe - windows version of Android Backup Extractor with those few cygwin files
but unsuccessful.
I read and test a lot, but The problem is still unsettled.
i came up two years ago with same question, how to unpack, and ended up with learning linux. i have uploaded a bash script, but i don't know if it works. i am not a developer and still learning, and nobody give feedback.
what is exactly file name of your backup? suffix *.ab or *.backup? these are completely different, don't mix up tools.
if you mean userdata_20180313_161000.backup files, please try mini tut above beginning 1.
aIecxs said:
i came up two years ago with same question, how to unpack, and ended up with learning linux. ... i am not a developer and still learning
Click to expand...
Click to collapse
It is classic aphorismic sentence!! I like it.
What's the bash script? for what goal? it is for Windows? If yes, I'll try it. give me the link.
my files are userdata_20100102_045926.backup, -=-=.backup1, 2, 3......
I'm not Englishman, "mini tut" is mini tutorial?
You have 100 posts and 25 thanks - super rate for notdeveloper.
the tool in video (adbextractor) can not unpack your backup. your backup is a splitted ext4 partition image. OSFMount can open ext4 partition images from Windows
bckp2cwm.sh script is basically doing the same steps like mini tutorial above. unfortunately it is for linux. main goal is convert userdata*.backup to cwm backup, but its paused in the middle (after unpacking, before repacking)
as windows user, all you need is a usb-stick. there is a tool unetbootin running in windows. it is a one-click-solution for downloading and copying any linux distribution to usb-stick. you can boot into linux from this usb-stick, for example ubuntu
aIecxs said:
OSFMount can open ext4 partition images from Windows
Click to expand...
Click to collapse
OSFMount can extract few files from my backup?
Thank you very much for efficient advices!!!:highfive:
I'll try them in sequence.
although Linux is not my :angel:dream.
Solution for Windows
1. Download and install free hhd hex editor neo.
2. Open each of your .backup files, in select menu select range at 0 offset size 512 decimal, delete and save.
3. Use Windows copy command to join files as described in post#11 (Thanks to the author for free education).
4. Use any program that reads Linux disk image files. I used diskinternals (mount image from the drive menu first). Openext worked too.
This won't work on tar archives if I understand correctly. This took me days to figure out and only minutes to execute. No thanks to confusing android prompts while upgrading to a new OS.
yoyohelp said:
1. Download and install free hhd hex editor neo.
2. Open each of your .backup files, in select menu select range at 0 offset size 512 decimal, delete and save.
3. Use Windows copy command to join files as described in post#11 (Thanks to the author for free education).
4. Use any program that reads Linux disk image files. I used diskinternals (mount image from the drive menu first). Openext worked too.
This won't work on tar archives if I understand correctly. This took me days to figure out and only minutes to execute. No thanks to confusing android prompts while upgrading to a new OS.
Click to expand...
Click to collapse
Hi i have a problem i have 5 files from the backup each with 2gb when i do the command from post 11 it works nice but when goes to backup part 3 it gives an error, what could it be? Im i doing something wrong?
Hello, I'm needing to reflash the factory firmware to my SHIELD (from NVIDIA's dev site), but I am unable to extract the files from the .tgz archive. I don't have a Linux system, I don't want a Linux system, and I don't want to have to learn a new OS to unpack a few files. I have tried using WinBash and a couple of other utilities (I don't even want to get started on Git) with no success. If someone could provide a walkthrough on extracting this on a Windows 7 system, or post links to the extracted files, that would be awesome. I'm good on the ADB/Fastboot front, I just need access to the .tgz contents. The latest 7-Zip and WinRAR don't work with the file, either. Thanks in advance...
...actually...
Never mind, I answered my own question. For those who need to know who don't know...
1. Download CygWin and install
2. Extract the .tgz file from the system image zip from NVIDIA
3. Rename the .tgz to 'shield.tgz'
4. Go to CygWin parent directory and run 'CygWin.bat', leave open
5. Place shield.tgz into the CygWin\Home\[whatever your CygWin account name is]
6. Go back to Cygwin and type 'bash shield.tgz'
Enjoy!
Just a heads up for people looking/finding this thread via Google & what not, you can also use OSX terminal to extract the contents, so for anyone that has a Mac can do this as it just needs *inx commands & not linux per say as the instructions say on nVidia's site.
I "think" 7-zip on windows can do this.
It can't
I recently purchased a Nvidia Shield tablet and was happy to discover that some of the development aspects are pretty similar to Samsung devices. I wrote tutorials for several Samsung devices that detail this same process and thought it would be good to post one for the Nvidia Shield Tablet.
Now to start off, this does not automatically make you a Super Duper Developer, this does not show you how to make your own ROMs. What it does do is show you how to pull files from those stock leaked/official ROMs so that you can proceed to make yourself into a Super Duper Developer!
In essence if the Developer knows what they are doing they could use this method to extract all the files from the build and could then add them to a ZIP and have a stock odexed ROM within minutes and do it all through Windows!
So let's begin!
We've all seen the stock restore images Nvidia provides. How often have you wanted to look inside them and see what is inside before flashing? There are several ways to do it and the most common answer requires Linux and other utilities. This tutorial gives you the ability to do that through Windows and without having to extract the full ROM. These instructions are specific to the Nvidia Shield Tablet but with some tweaking can be used on several devices.
Each stock restore release contains multiple files. Each of these files corresponds to a specific partition on your device. These files are formatted to be mounted by your device but not by a computer. We will be converting them from a .img partition type to an ext4.img type partition so we can mount them and read them through Windows. For this tutorial we will be opening up the system.img (the system files partition).
Before you dive in there are a few things you'll need.
- A PC running Windows.
- A stock Nvidia restore.
- DiskInternals Linux Reader (must be installed on the Windows PC). You can download DiskInternals Linux Reader here.
- SGS2toEXT4 Utility available here. *Be sure to extract the JAR file from the ZIP archive.
- 7-Zip available here.
For the rest of the tutorial please visit my site: www.rwilco12.com
Enjoy!
The original tutorial is still available here, anyhow starting with 7-zip 15.08 you don't really need other programs to access ext4 images.
The only other step you need is to convert to that the sparse images nvidia provides.
Python script to convert sparse ext4 Android images (like cache.img[.ext4], hidden.img[.ext4] to openable ext4 images (like with 7zip)
Python script to convert sparse ext4 Android images (like cache.img[.ext4], hidden.img[.ext4] to openable ext4 images (like with 7zip) - sparse_img_to_ext4.py
gist.github.com
[DEV][Tools] simg2img for Windows
Hello, although I'm working in Linux (VM too) I rewrote the SIMG2IMG so far for Windows (PE32, x86) Usage: simg2img.exe -i -o Optional: -d for debugging messages, listing all chunks of the image...
forum.xda-developers.com