Ok, I'll admit it, for a lot of things, especially system administration, I prefer the command line.
Is there a reason why I would not use tar to do "full backup" of data (as opposed to TiBackup or the like)?
One advantage is that I can extract a tar on another box and, for example, check it in to a VC system.
Admittedly, busybox tar isn't the most full-featured implementation (no one-file-system option, for example), but I believe it does have an exclude-from-file option that works.
I may be wrong, but I don't think tar on our phones can save ownership and permissions (too lazy to try it out right now). I suppose it wouldn't be too hard to write a script to save any info that tar would not preserve into a properties file and then have a restore script that would read the properties file and chown and chmod every file after extracting.
xviiivx said:
I may be wrong, but I don't think tar on our phones can save ownership and permissions
Click to expand...
Click to collapse
busybox tar appears to work like "any" tar, at least for non-extended attributes (I don't know if the kernel supports xattrs )
Code:
$ tar tvf test.tar | head -n 10
drwxr-xr-x 0/0 0 2011-02-07 20:58 system/etc/
-rw-r--r-- 0/0 15346 2011-02-07 20:58 system/etc/apns-conf.xml
-rw-r--r-- 0/0 25 2011-02-07 20:58 system/etc/hosts
-rw-r--r-- 0/0 99026 2011-02-07 20:58 system/etc/NOTICE.html.gz
-r-xr-x--- 0/2000 1200 2011-02-07 20:58 system/etc/init.goldfish.sh
-rw-r--r-- 0/0 1707 2011-02-07 20:58 system/etc/vold.fstab
-rw-r--r-- 0/0 195 2011-02-07 20:58 system/etc/gps.conf
drwxr-xr-x 0/0 0 2011-02-07 20:58 system/etc/security/
-rw-r--r-- 0/0 91755 2011-02-07 20:58 system/etc/security/cacerts.bks
drwxr-xr-x 0/0 0 2011-02-07 20:58 system/etc/security/drm/
and does appear to preserve links as well
Code:
$ tar tvf test2.tar | head -n 10
lrwxrwxrwx 0/0 0 2011-03-15 12:09 etc -> /system/etc
These were created using busybox tar from /system/etc and of /etc, adb pull-ed, and extracted on another system (Cygwin, in this case).
If you look in the TitaniumBackup directory on the sdcard, each app has 3 files:
appname.apk.gz- I didn't check, but obviously is probably just a backup of the apk
appname.tar.gz (looks like a tar of the /data/data/app dir)
appname.properties- looks like it includes a huge chuck of data for the icon and some misc information, a lot of which appears to be... market information?
I don't know how important the third file is, but it looks like Titanium is using tar. You could download a junk app and try it with that app and see if it works.
Just a guess, but I bet untarring may not be enough, you'll probably have to run the command "pm install" on each apk.
Thanks Fubuya!
That leads me to two useful "conclusions"
tar probably works just fine
TiBackup files are likely extractable on a POSIX box
Yeah, on linux, one is just text and I extracted the other two just fine (tar -zxvf for one and gunzip for the other). It's probably just a matter of finding out if there are any extra tricks you need to know about. Looks like Tibackup has a forum here, if you run into problems maybe they'd be nice enough to translate what's happening on the backend of their backup process.
I didn't write this hack, but it appears to work with the new stock GB update.
I did edit the script and zip archive so it wouldn't need any external exe's or libs to apply. Just copy the newest framework.jar file to the script directory and run the HackHotspot.bat file. Then simply run these ADB commands from a dos window.
------
adb shell
mount /system
exit
adb push framework.jar /system/framework.jar
-------
clear Dalvik and fix permissions if desired
Reboot Phone
Done!
Much appreciated but way over my head (I've never done anything with ADB). Any chance of getting this, and the USB debugging notification fix, in a flashable form?
The debug fix is the one that doesn't display the statusbar icon when connected?
I'll see what I can do for ya
townsenk said:
Just copy the newest framework.jar file to the script directory and run the HackHotspot.bat file.
Click to expand...
Click to collapse
Awesome script. Thanks.
will this work for the revolutionary?
CAn i do this on a MAC?....
Can n e one verify that this works?
im getting errors in the script:
Code:
Extracting framework.jar
7-Zip 4.65 Copyright (c) 1999-2009 Igor Pavlov 2009-02-03
Processing archive: framework.jar
Extracting META-INF
Extracting META-INF\MANIFEST.MF
Extracting preloaded-classes
Everything is Ok
Folders: 1
Files: 2
Size: 65651
Compressed: 12413
Unpacking classes.dex
Can't find the file classes.dex
Putting HotspotHack into place
The system cannot find the path specified.
The system cannot find the path specified.
0 file(s) copied.
Repacking classes.dex
UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.RuntimeException: Cannot find file or directory "out/"
at org.jf.smali.main.main(main.java:173)
Could Not Find C:\AddHotspotHack\classes.dex
The system cannot find the file specified.
Updating framework.jar
1 file(s) copied.
classes.dex : no such file or directory
Cleaning up...
Could Not Find C:\AddHotspotHack\classes.dex
The system cannot find the file specified.
Done if no errors listed above!
C:\AddHotspotHack>
and shouldnt it be
adb push framework.jar /system/framework/framework.jar
Hello all,
My Chinese notwellknown phone model does not have TWRP, but I have made live TWRP/CWM (yes both) backup files:
system.ext4.win (TWRP) and its md5
system.ext4.tar and (CWM) and its md5.
If one day, I have a bootloop and want to restore those file via Bootloader: (Power button & Vol-)
then: fastboot -S 130M flash system system.img
So how do I create a "system.img" from one of the files above (system.ext4.win or system.ext4.tar) ?
A) Before someone sidetrack the topic, I have done MANY TIMES SUCCESSFULLY the command: fastboot -S 130M flash system "stock-system-file.img" with this phone ( .img extracted from stock zip/tar). (* 1)
B) I didn't ask: "Is there other way to restore ?" I asked: How to convert *.ext4.win (twrp) or *.ext4.tar (cwm) files to .img files just like the ones uncompressed from a stock zip/tar.
(*1) Of course there are other ways to restore, like temporarily fastboot boot twrp.img then restore from there or from twrp>adb. But that's not what the question is asking.
........... Thank you everyone ...........
sintoo said:
Hello all,
My Chinese notwellknown phone model does not have TWRP, but I have made live TWRP/CWM (yes both) backup files:
system.ext4.win (TWRP) and its md5
system.ext4.tar and (CWM) and its md5.
If one day, I have a bootloop and want to restore those file via Bootloader: (Power button & Vol-)
then: fastboot -S 130M flash system system.img
So how do I create a "system.img" from one of the files above (system.ext4.win or system.ext4.tar) ?
A) Before someone sidetrack the topic, I have done MANY TIMES SUCCESSFULLY the command: fastboot -S 130M flash system "stock-system-file.img" with this phone ( .img extracted from stock zip/tar). (* 1)
B) I didn't ask: "Is there other way to restore ?" I asked: How to convert *.ext4.win (twrp) or *.ext4.tar (cwm) files to .img files just like the ones uncompressed from a stock zip/tar.
(*1) Of course there are other ways to restore, like temporarily fastboot boot twrp.img then restore from there or from twrp>adb. But that's not what the question is asking.
........... Thank you everyone ...........
Click to expand...
Click to collapse
Here is a thread for converting TWRP/CWM backups into a flashable zip. I know you aren't looking for how to create a flashable zip but the guide instructs how to extract the system.ext4.win and system.ext4.tar to get the system.img from the backup, this is the part that you need.
https://forum.xda-developers.com/showthread.php?t=2746044
Sent from my SM-S767VL using Tapatalk
first, system.img usually can be downloaded somewhere, no need to restore twrp backup for system. don't you think you will find a download? second, if you restore system you may required to restore vendor and boot too. boot.emmc.win can be flashed from fastboot directly, but for vendor you need to convert. the same way. third, system.img can have different formats. you need to know which file system type, partition size, and if it is sparsed image or not. file system type is EXT4 in your case because twrp backup is named system.ext4.win
for "converting" the system.ext4.win* file(s) into system.img you do it in two steps. first you need to create a empty ext4 image file. after the empty disk image is mounted somewhere, you simply unpack the backup files into the mounted disk. so it's not really converting, more like copying.
You need a linux machine for this
so let's begin with partition size
on a working device you can simply check from system or recovery. find the symlink, resolve the symlink for system, get #blocks (=size) for respective block partition, for example mmcblk0p99
for some reason xda blocks code for ls -l (lower case -L there is no space between)
Code:
find /dev/block -name by-name
ls - l /dev/block/platform/bootdevice/by-name
cat /proc/partitions
Note: the above commands need to be run on target android device only. use adb shell or terminal emulator. all other commands below need to be run on host linux pc
if gathering partition size on the device itself is not an option for you, if you have a mediatek chipset you can simply look into scatter file. OTA zip files most likely contain scatter file. if you don't have a scatter file for your ROM version, you can create your own scatter file with WwR MTK
use hex2dec calculator for partition_size: 0x9C800000 = 2625634304 bytes = 2564096 KiB (#blocks)
if you don't have mediatek device i am running out of ideas. you can boot into twrp from fastboot and check, or find a ROM and check the file size hopefully it is not a sparsed image.
Once you know the partition size, now proceed to create a empty file (avoid to do this on fat32 host if size is bigger > 4 GiB)
and format the file to ext4 (or f2fs if needed)
Code:
dd if=/dev/zero of=system.img bs=1 count=0 seek=2625634304
mke2fs -t ext4 system.img
if all went well you can create a mount point on host and mount the empty disk image. the following commands need to be run with root permissions. you can do it from sudo or as root
Code:
mkdir system
sudo -i
cd /home/$SUDO_USER
mount -t ext4 -o loop,rw,noexec,noatime system.img system
you can now unpack the tar files into system. make sure the folder structure remains intact, if the archive contains the folder /system you should unpack to ~ (this will create all files in ~/system) otherwise you need to unpack to ~/system (or cd into ~/system)
for cwm all files need to be concatenated. for twrp do not concatenate each file is a standalone tar archive. if you have android > kitkat 4.4.2+ make sure you use the proper flags for selinux context
Code:
tar --selinux --xattrs --numeric-owner -vxpf system.ext4.win000
tar --selinux --xattrs --numeric-owner -vxpf system.ext4.win001
or
Code:
cat system.ext4.tar000 system.ext4.tar001 | tar --selinux --xattrs --numeric-owner -vxp
the unpacking may produce errors malformed header or something, make sure all files extracted anyway. i have read somewhere better use star instead of tar which can handle twrp files in the right way, unfortunately haven't tested yet
to avoid any problems with permissions you should check/set for system again
Code:
chown -h 1000:1000 system
chmod 0755 system
chcon -h --reference system/bin system
after unpacking just unmount the disk image
Code:
umount system
rmdir system
exit
If you want to create a sparse image you can use the img2simg tool
Code:
sudo apt-get install android-tools-fsutils
img2simg system.img system.smg
This method is just written on the fly especially for your request, everything untested. I really don't know if this file is flashable from fastboot, do it at your own risk!
aIecxs said:
first, system.img usually can be downloaded somewhere, no need to restore twrp backup for system. don't you think you will find a download? second, if you restore system you may required to restore vendor and boot too. boot.emmc.win can be flashed from fastboot directly, but for vendor you need to convert. the same way. third, system.img can have different formats. you need to know which file system type, partition size, and if it is sparsed image or not. file system type is EXT4 in your case because twrp backup is named system.ext4.win
for "converting" the system.ext4.win* file(s) into system.img you do it in two steps. first you need to create a empty ext4 image file. after the empty disk image is mounted somewhere, you simply unpack the backup files into the mounted disk. so it's not really converting, more like copying.
You need a linux machine for this
so let's begin with partition size
on a working device you can simply check from system or recovery. find the symlink, resolve the symlink for system, get #blocks (=size) for respective block partition, for example mmcblk0p99
for some reason xda blocks code for ls -l (lower case -L there is no space between)
Code:
find /dev/block -name by-name
ls - l /dev/block/platform/bootdevice/by-name
cat /proc/partitions
Note: the above commands need to be run on target android device only. use adb shell or terminal emulator. all other commands below need to be run on host linux pc
if gathering partition size on the device itself is not an option for you, if you have a mediatek chipset you can simply look into scatter file. OTA zip files most likely contain scatter file. if you don't have a scatter file for your ROM version, you can create your own scatter file with WwR MTK
use hex2dec calculator for partition_size: 0x9C800000 = 2625634304 bytes = 2564096 KiB (#blocks)
if you don't have mediatek device i am running out of ideas. you can boot into twrp from fastboot and check, or find a ROM and check the file size hopefully it is not a sparsed image.
Once you know the partition size, now proceed to create a empty file (avoid to do this on fat32 host if size is bigger > 4 GiB)
and format the file to ext4 (or f2fs if needed)
Code:
dd if=/dev/zero of=system.img bs=1 count=0 seek=2625634304
mke2fs -t ext4 system.img
if all went well you can create a mount point on host and mount the empty disk image. the following commands need to be run with root permissions. you can do it from sudo or as root
Code:
mkdir system
sudo -i
cd /home/$SUDO_USER
mount -t ext4 -o loop,rw,noexec,noatime system.img system
you can now unpack the tar files into system. make sure the folder structure remains intact, if the archive contains the folder /system you should unpack to ~ (this will create all files in ~/system) otherwise you need to unpack to ~/system (or cd into ~/system)
for cwm all files need to be concatenated. for twrp do not concatenate each file is a standalone tar archive. if you have android > kitkat 4.4.2+ make sure you use the proper flags for selinux context
Code:
tar --selinux --xattrs --numeric-owner -vxpf system.ext4.win000
tar --selinux --xattrs --numeric-owner -vxpf system.ext4.win001
or
Code:
cat system.ext4.tar000 system.ext4.tar001 | tar --selinux --xattrs --numeric-owner -vxp
the unpacking may produce errors malformed header or something, make sure all files extracted anyway. i have read somewhere better use star instead of tar which can handle twrp files in the right way, unfortunately haven't tested yet
to avoid any problems with permissions you should check/set for system again
Code:
chown -h 1000:1000 system
chmod 0755 system
chcon -h --reference system/bin system
after unpacking just unmount the disk image
Code:
umount system
rmdir system
exit
If you want to create a sparse image you can use the img2simg tool
Code:
sudo apt-get install android-tools-fsutils
img2simg system.img system.smg
This method is just written on the fly especially for your request, everything untested. I really don't know if this file is flashable from fastboot, do it at your own risk!
Click to expand...
Click to collapse
Cygwin works as well, it doesn't necessarily "have" to be Linux.
There is a post for using Cygwin as well in the thread that I linked.
More than one way to "skin this cat", so to speak. I'm sure they'll figure it out with the information they've been provided.
Sent from my SM-S767VL using Tapatalk
i don't think so, twrp archive doesn't contain a system.img it only contains folder /system. therefore you can't extract system.img but only the files inside, which have to be extracted in the right way including metadata. the cygwin untar.exe won't handle secontext flag. besides this it is generally bad idea to extract linux files to windows file system. ntfs is case insensitive and not all ascii chars allowed in file names, you will lose all file permissions owner/group and selinux context (except you untar it directly to ext4 image like i said). This might not be important for flashable zip because everything is lost inside a zip anyway (that's why META-INF is needed) but for creating a working ext4 image you must set everything to drw(x)r-(x)r-(x) system system ubject_r:system_file:s0 (i can be wrong always double check for your ROM) things become more complicated when we talking about data.ext4.win or vendor.ext4.win
even if you compile star or gnu tar and all other binaries for windows like mke2fs chcon or so, you won't be able to mount the ext4 disk image r/w in the right way on windows..
anyway thanks for suggesting cygwin, might be a simple alternative for windows freaks. i personally prefer booting live distro from usb stick
aIecxs said:
i don't think so, twrp archive doesn't contain a system.img it only contains folder /system. therefore you can't extract system.img but only the files inside, which have to be extracted in the right way including metadata. the cygwin untar.exe won't handle secontext flag. besides this it is generally bad idea to extract linux files to windows file system. ntfs is case insensitive and not all ascii chars allowed in file names, you will lose all file permissions owner/group and selinux context (except you untar it directly to ext4 image like i said). This might not be important for flashable zip because everything is lost inside a zip anyway (that's why META-INF is needed) but for creating a working ext4 image you must set everything to drw(x)r-(x)r-(x) system system ubject_r:system_file:s0 (i can be wrong always double check for your ROM) things become more complicated when we talking about data.ext4.win or vendor.ext4.win
even if you compile star or gnu tar and all other binaries for windows like mke2fs chcon or so, you won't be able to mount the ext4 disk image r/w in the right way on windows..
anyway thanks for suggesting cygwin, might be a simple alternative for windows freaks. i personally prefer booting live distro from usb stick
Click to expand...
Click to collapse
Yeah, that is true, it should only be the system folder.
Since they must use TWRP/CWM to create the backups, maybe it would have been better to suggest that they should instead, boot into TWRP then connect to adb to use adb shell commands or to use the terminal emulator that is built into TWRP to run a system dump or dd commands to dd a copy of the system.img over to PC.
Or maybe there is a way to mount/run the extracted system folder in terminal then using terminal to dump/dd a .img from that. I don't even know if that is possible(never heard of it, but it's a thought) but it seems to me that if the system.img is what is written to the system partition when flashed and this is what creates the system folder, there "should" be a way to pack the contents of the system folder back into what was originally contained in the system.img. It might miss a few things though, like the kernel for one, the kernel is sometimes part of the system.img but I don't know if that necessarily means that the kernel will be in the system partition/folder when the system.img is flashed. Thus, making it impossible to reverse engineer back into a proper system.img using only the contents of the system folder obtained from a nandroid backup.
A system dump, dd the .img or extract the system.img from the stock firmware file are the way to go, preferably, extracting from the stock firmware file, because that is easier and less risky than using shell, terminal and Linux for the uninitiated.
Sent from my SM-S767VL using Tapatalk
---------- Post added at 03:52 PM ---------- Previous post was at 03:47 PM ----------
aIecxs said:
i don't think so, twrp archive doesn't contain a system.img it only contains folder /system. therefore you can't extract system.img but only the files inside, which have to be extracted in the right way including metadata. the cygwin untar.exe won't handle secontext flag. besides this it is generally bad idea to extract linux files to windows file system. ntfs is case insensitive and not all ascii chars allowed in file names, you will lose all file permissions owner/group and selinux context (except you untar it directly to ext4 image like i said). This might not be important for flashable zip because everything is lost inside a zip anyway (that's why META-INF is needed) but for creating a working ext4 image you must set everything to drw(x)r-(x)r-(x) system system ubject_r:system_file:s0 (i can be wrong always double check for your ROM) things become more complicated when we talking about data.ext4.win or vendor.ext4.win
even if you compile star or gnu tar and all other binaries for windows like mke2fs chcon or so, you won't be able to mount the ext4 disk image r/w in the right way on windows..
anyway thanks for suggesting cygwin, might be a simple alternative for windows freaks. i personally prefer booting live distro from usb stick
Click to expand...
Click to collapse
I use VM, Live USB and dual boot, depending on which system I'm on(I have more than one rig) and depending on what I'm doing. In some cases, using Windows running Linux in VM is handy because some things are easier in Windows and some are easier in Linux, VM allows switching back and forth between Windows and Linux "on the fly" instead of having to move back and forth between two different systems. Another advantage to VM, your actual system is effectively immune to viruses when browsing the web inside the VM, only the OS installed in the VM is vulnerable, if infected, just wipe that OS and reinstall in the VM and you're clean again, your actual system that the VM is running on, never gets effected.
Sent from my SM-S767VL using Tapatalk
yeah i wonder how did create twrp backup without actually having twrp. busybox tar isn't able to do it, at least full tar binary is required. would be better to create backup in desired format from the very beginning. easiest way is adb pull /dev/block/bootdevice/by-name/system system.img (with proper path of course)
aIecxs said:
yeah i wonder how did create twrp backup without actually having twrp. busybox tar isn't able to do it, at least full tar binary is required. would be better to create backup in desired format from the very beginning. easiest way is adb pull /dev/block/bootdevice/by-name/system system.img (with proper path of course)
Click to expand...
Click to collapse
They booted a live temporary session of TWRP without actually flashing it to the recovery partition. I've done the same on an Intel tablet and a couple of RCA tablets. Booting it directly instead of flashing it onto the device doesn't trigger the locked bootloader. The locked bootloader won't allow booting unverified software that has been installed in the device's hardware itself, but it does not block booting unverified software from an external source.
Sent from my SM-S767VL using Tapatalk
After reading other source, someone said the ext4.win are just tar. One only needs to rename them to ext4.win.tar and you uncompress to img.
I guess the truth is more complicated than that because img (from stock, ready to be ODINed/Bootloader Fastboot) are raw images, including zeros. That's the difference.
In the end, if I had to do it again, I would have to dd the whole /mmcblkxxx(system) to a microSD. Yes 16-20Gb takes a much longer time than 2-3Gb (system.ext4.win) but that what <fastboot flash system system.img> requires (raw data and zeros).
sintoo said:
After reading other source, someone said the ext4.win are just tar. One only needs to rename them to ext4.win.tar and you uncompress to img.
Click to expand...
Click to collapse
Thats exactly how it works see post #3
if you add entry with file system type "emmc" in /etc/recovery.fstab TWRP produces flashable disk image system_image.emmc.win* instead of tar archive (which you can probably concatenate into system.img)
Code:
/system_image emmc /dev/block/platform/mtk-msdc.0/by-name/system flags=display="System Image";backup=1;flashimg=1
Hello,
i'm trying to unpark boot.img
But then i got this error here. how can that be fixed? (Android-Kitchen-0.224)
Copy boot.ing into -> bootimg_040621_161723
Code:
Creating folder /home/fredy/Desktop/Android-Kitchen-0.224/bootimg_040621_161723 ...
---> Place boot.img/recovery.img into the folder mentioned above <--
Press Enter to continue
Kernel found at offset 2048 in boot.img
Extracting kernel ...
Extracting ramdisk ...
Error: No ramdisk folder found!
Contents of bootimg_040621_161723:
total 9476
-rw-rw-r-- 1 fredy fredy 9701434 Apr 6 16:17 zImage
Press Enter to continue
Thanks for your time
007fred62 said:
Hello,
i'm trying to unpark boot.img
But then i got this error here. how can that be fixed? (Android-Kitchen-0.224)
Copy boot.ing into -> bootimg_040621_161723
Code:
Creating folder /home/fredy/Desktop/Android-Kitchen-0.224/bootimg_040621_161723 ...
---> Place boot.img/recovery.img into the folder mentioned above <--
Press Enter to continue
Kernel found at offset 2048 in boot.img
Extracting kernel ...
Extracting ramdisk ...
Error: No ramdisk folder found!
Contents of bootimg_040621_161723:
total 9476
-rw-rw-r-- 1 fredy fredy 9701434 Apr 6 16:17 zImage
Press Enter to continue
Thanks for your time
Click to expand...
Click to collapse
The ramdisk is likely in the recovery.img
sd_shadow said:
The ramdisk is likely in the recovery.img
Click to expand...
Click to collapse
ok
sd_shadow said:
The ramdisk is likely in the recovery.img
Click to expand...
Click to collapse
Hello,
can you help me with this? recreate boot.img
Code:
Boot image tools
----------------
Enter a choice:
a = Extract kernel+ramdisk from boot.img/recovery.img in any folder
x = Exit
NOTE: Other options may not be shown in this menu *until* a specific
action above is selected or a specific working folder is created.
(A working folder is created with Option 1 of the kitchen main menu!)
e.g. You won't see the option to re-pack a boot.img unless you unpack
the boot.img first! For re-packing you also need a working folder
with the original boot.img so that the kitchen can use its header
information to construct the new boot.img.
But there can i copy files too.. only a = extract no working folder