Related
INFORMATION
This guide is intended to make a full backup of your android phone (the entire memory block with all partitions) or a single partition (including sdcards, etc) directly to your computer, in either
Block level (with dd): for single partitions or whole memory block (all partitions in one piece). The backup always has the same size which is the size of the partition.
File level (with tar): only for individual partitions. This only includes files and folders, so occupies much less space, depending on how much filled is the partition.
It can be done with the phone powered on or from ClockWorkMod Recovery (from both ADB works, while in Fastboot doesn't so won't apply). Unless specified the commands meant to be used from Windows. For Linux and Unix is similar.
REQUIREMENTS
Rooted Android Phone
Busybox installed on your phone
If you are using Linux / OS X you have native tools, for Windows download Cygwin, and install with it netcat, pv and util-linux. Get them from Cygwin's setup.exe
ADB installed.
Make sure adb.exe is in your windows' path. See here and here, or use Path Manager.
Android phone with USB Debugging enabled, and the proper drivers installed on Windows so the phone is recognized. Typing 'adb devices' on a terminal should show your device.
PARTITION IDENTIFICATION
You now have to identify the partition or block device that you want to backup. For a single partition you can use either tar or dd, while for the entire memory block you can only use dd.
For example, on Galaxy Nexus you have the list of partitions here and for Galaxy S2 here.
Usually on android, the entire block containing all partitions is located at /dev/block/mmcblk0 and the data partitions is a subpartition of it. You can push parted with GPT support to your device and see all information on a partition or block.
Whole phone memory -> /dev/block/mmcblk0 (may vary, in some phones this is the sdcard)
Subpartitions -> depends on each device. Usually at /dev/block/platform/dw_mmc/by-name/ there are listed by name linking to the real device.
Back up of the whole memory block (via adb)
Connect the phone in ADB mode and unlock the screen.
Open one Cygwin Terminal and enter (replace mmcblk0 if needed):
Code:
adb forward tcp:5555 tcp:5555
adb shell
su
/system/xbin/busybox nc -l -p 5555 -e /system/xbin/busybox dd if=/dev/block/mmcblk0
You will see the cursor blinking at the left. Now the phone is waiting to send the block over the network.
Open another Cygwin terminal and type:
Code:
adb forward tcp:5555 tcp:5555
cd /path/to/store/the/backup
nc 127.0.0.1 5555 | pv -i 0.5 > mmcblk0.raw
You will see how the image size is growing until it finishes. Now you have the whole phone backed up in raw format. You can see the contents of the GPT partition with gptfdisk tool, available for windows, linux and such. See official website and sourceforge to get it. You can do it the same from ClockWorkMod Recovery but you have to mount first the /system partition since the busybox included with clockworkmod does not come with netcat and you have to use the one from the system partition.
With further linux tools you could edit or extract single partitions from the whole block.
You can use adb via wifi as well with applications like WiFi ADB.
Back up of the whole memory block (via wifi)
Original post: [Q] Nandroid directly to computer w/o sdcard
We need to install a FTP server on the computer or the other device, configure a user with a password if we want to, and set some port. It uses by default 21 but this example uses 40. We must set a home dir for the user with write permissions.
Usually is a good idea to put myfifo in /cache not in /data because we may overwrite sensitive data in case we want to use that raw image for data recovery.
Open one Cygwin terminal
Code:
adb shell
su
mkfifo /cache/myfifo
ftpput -v -u user -p pass -P 40 COMPUTER_IP block.raw /cache/myfifo
Open another Cygwin terminal
Code:
adb shell
su
dd if=/dev/block/mmcblk0p12 of=/cache/myfifo
Tips:
- Fifos only can be made on linux native filesystems, for example on a FAT partition is not possible.
- Reading from a partition does not modify it.
Now check on Filezilla Server the speed
Back up of the whole memory block (USB tethering, Wifi tethering)
To use tethering you have to disconnect the computer from all networks and connect it only to the phone with the type of connection you want.
Once you connect it, you can view the IP of the computer and the IP of the phone from connection properties. The ip is the computer ip and the gateway is the phone's ip.
Wifi Tethering: Computer <---Wifi---> Phone <---3G---> Internet
USB Tethering:
Computer <---USB---> Phone <---Wifi---> Internet
Conputer <---USB---> Phone <---3G---> Internet
This is exactly the same as via wifi, except that the transfer speed is much higher because the computer and the phone are directly connected, instead of using a router as a gateway. In this case, the gateway is the phone. USB tethering has the highest transfer rate.
Back up of a single partition (raw = every bit of the partition)
It is exactly the same as the the previous but replacing mmcblk0 by the corresponding partition. You can use in this particular case several software to read the partition from windows, depending on partition filesystem: DiskInternals Linux Reader, Ext2Read, Ext2 File System Driver for Windows, Ext4Explore, plugin for Total Commander and ImDisk Virtual Disk Driver. You can also use recovery software on individual partitions like Recuva in combination with VHD Tool or command line tools included with operating systems.
Back up of a single partition (tar = only files and folders)
In this case, you need the partition mounted. To see the list of mounted partitions type on Cygwin Terminal
Code:
adb shell mount
Now you need to know where is mounted the partition you want to backup, for example the firmware is mounted on /system, which is the ROM.
In this case you will have to open three terminals, because of android limitations:
Open one Cygwin terminal and create a fifo, in /cache, for example, and redirect the tar there
Code:
adb forward tcp:5555 tcp:5555
adb shell
su
/system/xbin/busybox mkfifo /cache/myfifo
/system/xbin/busybox tar -cvf /cache/myfifo /system
We have to do it this way because redirecting the tar to stdout (with - ) is broken on android and will corrupt the tar file.
Open a second Cygwin terminal and type:
Code:
adb forward tcp:5555 tcp:5555
adb shell
su
/system/xbin/busybox nc -l -p 5555 -e /system/xbin/busybox cat /cache/myfifo
Open a third Cygwin terminal and type:
Code:
adb forward tcp:5555 tcp:5555
cd /path/to/store/the/backup
nc 127.0.0.1 5555 | pv -i 0.5 > system.tar
You can browse the tar file with Winrar, Total Commander, PeaZip and almost any compression tool. Note that you shouldn't extract files or edit it since the tar format saves the permission and owner data for each file, that is lost when extracted to FAT / NTFS partitions and you will mess things when restoring.
LINKS
[GUIDE] Internal Memory Data Recovery - Yes We Can!
How to Create and Attach a Virtual Hard Disk in Windows 7
[Guide] Types of Android backups
mohsyn said:
On newer android versions (Im on 7.2) data folder has a folder media which is link to sdcard and one ends up backing up entire sd card. I had a 64gb backup which wasn't necessary
In order to avoid skipping the media folder i had to do some trial and error because busybox tar command is not completely the same as GNU tar.
Would appreciate if you can mention it in the mail guide to use the following command to backup /data folder without copying sdcard files
In first terminal
tar cv --exclude data/media/0 -f /cache/myfifo /data
in 3rd terminal
nc 127.0.0.1 5555 | pv -i 0.5 > data.tar
no change in second terminal
Cheers
Click to expand...
Click to collapse
Umm...how to restore back from computer?
Sent from MARVEL
I am a little new to this, I have installed Android sdk and i am able to see my device by using "adb devices" , i have also installed Cygwin, now i want to backup whole phone memory block so i tried executing the first line on cygin "adb forward tcp:5555 tcp:5555" i get an error saying -bash: adb :command not found.
I am sorry if i am missing any thing, please guide me, and also what do you mean by "download Cygwin, and install with it netcat, pv and util-linux"
Thanx a ton !!
aunriz said:
I am a little new to this, I have installed Android sdk and i am able to see my device by using "adb devices" , i have also installed Cygwin, now i want to backup whole phone memory block so i tried executing the first line on cygin "adb forward tcp:5555 tcp:5555" i get an error saying -bash: adb :command not found.
I am sorry if i am missing any thing, please guide me, and also what do you mean by "download Cygwin, and install with it netcat, pv and util-linux"
Thanx a ton !!
Click to expand...
Click to collapse
You've done almost everything! But you skipped the section "make sure adb is in your path"
Probably you have adb.exe in the path
Code:
C:\Program Files (x86)\android-sdk\platform-tools\adb.exe
So you have to just add it to the Cygwin's path (would be better if you had added it earlier to the windows' path and cygwin will import it automatically but it is ok)
Code:
export PATH="/cygdrive/c/Program Files (x86)/android-sdk/platform-tools":$PATH
Remember to backup the path previously if you want.
Code:
echo $PATH > mypathbackup.txt
scandiun said:
You've done almost everything! But you skipped the section "make sure adb is in your path"
Probably you have adb.exe in the path
Code:
C:\Program Files (x86)\android-sdk\platform-tools\adb.exe
So you have to just add it to the Cygwin's path (would be better if you had added it earlier to the windows' path and cygwin will import it automatically but it is ok)
Code:
export PATH="/cygdrive/c/Program Files (x86)/android-sdk/platform-tools":$PATH
Remember to backup the path previously if you want.
Code:
echo $PATH > mypathbackup.txt
Click to expand...
Click to collapse
Thanks for replying but i cant seem to run the 3rd line , see this
[email protected] ~
$ adb forward tcp:5555 tcp:5555
[email protected] ~
$ adb shell
$ /system/xbin/busybox nc -l -p 5555 -e /system/xbin/busybox dd if=/dev/block/mmcblk0
reloc_library[1311]: 10182 cannot locate 'android_reboot'...
CANNOT LINK EXECUTABLE
i hav sucessfully installed busybox v1.14.3, i am not sure what is causing the problem
EDIT:
i found that my directory ws system/bin instead of xbin
so i changed it and first part worked correctly, now i cant seem to get the second part
[email protected] ~
$ adb forward tcp:5555 tcp:5555
[email protected] ~
$ cd c:/
[email protected] /cygdrive/c
$ nc 127.0.0.1 5555 | pv -i 0.5 > mmcblk0.raw
-bash: nc: command not found
-bash: pv: command not found
aunriz said:
[email protected] /cygdrive/c
$ nc 127.0.0.1 5555 | pv -i 0.5 > mmcblk0.raw
-bash: nc: command not found
-bash: pv: command not found
Click to expand...
Click to collapse
You don't have installed pv and netcat on cygwin. Run the setup.exe and install them.
If you run
Code:
whereis pv
whereis nc
should give you some path (in cygwin) but applies similar inside android.
scandiun said:
You don't have installed pv and netcat on cygwin. Run the setup.exe and install them.
If you run
Code:
whereis pv
whereis nc
should give you some path (in cygwin) but applies similar inside android.
Click to expand...
Click to collapse
Now i hav installed cv and nc and commands run sucessfully, but i get the raw file as just 1kb
First terminal:
[email protected] ~
$ adb forward tcp:5555 tcp:5555
adb shell
[email protected] ~
$ adb shell
$ /system/bin/busybox nc -l -p 5555 -e /system/bin/busybox dd if=/dev/block/mmcblk0
/system/bin/busybox nc -l -p 5555 -e /system/bin/busybox dd if=/dev/block/mmcblk0
$
second terminal:
[email protected] ~
$ adb forward tcp:5555 tcp:5555
[email protected] ~
$ cd c:/
[email protected] /cygdrive/c
$ nc 127.0.0.1 5555 | pv -i 0.5 > mmcblk0.raw
55 B 0:00:00 [10.7kiB/s] [<=> ]
[email protected]
No idea what you may be doing wrong
Listen, I appreciate the guide, and it being basically the only one which popped up in google results, I can't gripe too much, but... you really, really need to make it more clear.
The point of a guide is to help a large group of people with varying degrees of knowledge (and if it's a guide targeted at a specific group of people, i.e tech savvy, then it needs to be indicated as such).
With that working definition in place, it follows that you should be as specific as possible; think of *everything*. By doing so, you not only avoid headaches for the people reading the guide, but for yourself as well since you don't have to reply to comments which might've otherwise been avoided.
I would post a question, but I'll probably have figured this out (with a good 1+ hours of searching no doubt) before anyone responds.
Here are some examples of what could be more specific:
"ADB installed." - what is ADB? Where's the link? It's not reasonable to assume people use these tools on a regular basis or remember them.
"You can push parted with GPT support to your device and see all information on a partition or block." - okay, so we know what it does but not how to install it or use it.
"ADB mode" - is this important? What is it? Not sure because it was glossed over.
These are just some examples. It's not the most horrendous trespass ever committed, but definitely annoying. Just spell it out from one step to the next, it works far better than topics with subheadings and unintuitive concepts being assumed as general knowledge on the behalf of noobs like me.
Edit: I'm just going to take everything off my SD card, use nandroid, and then copy the nandroid backup to computer as well. Please improve the guide, thanks.
Greatly appreciate this!
For me, a long-time UNIX and Linux administrator, this little guide was a breath of fresh air. Scandiun, *Thank You* for putting it together. It makes perfect sense to me -- just treat the phone as the linux machine it is. I'm becoming convinced that most of the more recent "developers" hanging around the android community have never used a linux machine before -- they don't seem to know what's going on, they go way out of their way to write overkill tools to do things clumsily that can already be done cleanly and quickly from the command line, and then they wrap those tools in so much mystery, black magic, and script-kiddie terminology that I can't figure out what they do either, and I certainly don't trust them doing things to my phone.
For example, I've got a new Galaxy S3, and just wasted a whole day digging around on xda, reviewing all of the "kewl rooting mods" until I got sick of it. Why the *heck* are people flashing entire partitions just to install a setuid /system/xbin/su on these devices? The rooting method I wound up using was dirt simple -- just find an rc exploit and use it to install an 'su' binary, by typing a few commands via adb. I used a variation of the exploit mentioned in http://seclists.org/fulldisclosure/2012/Aug/171, and elaborated in http://forum.xda-developers.com/showthread.php?t=1790104, http://forum.xda-developers.com/showthread.php?t=1792342, http://galaxys3root.com/galaxy-s3-root/how-to-root-u-s-canadian-dual-core-galaxy-s3-on-mac-osx/, and http://forum.xda-developers.com/showthread.php?t=1827518. If you are a UNIX person, you'll recognize what's going on with that exploit and be able to come up with something that suits your own needs. If you aren't a UNIX person, then you'll be completely lost. Sorta like this guide.
For anyone who doesn't yet know what adb is, or who's never used standard UNIX/Linux tools like dd, netcat, gparted, or busybox, I agree that this guide is not only not going to help you, but may actually aid you in shooting yourself in the foot with extreme efficiency. But please don't criticize or nag the OP in return for helpful advice freely given. You won't learn much about UNIX tools on an Android-related web site in any case. I recommend starting with a Linux systems administration book -- the Nemeth series is always good. But if you do take that route, you need to expect to take time to learn the basics.
Absolutely beautiful!
Thank you for this work. I was able to recover deleted files from my Galaxy Nexus' internal memory using this technique. I made a [GUIDE] using most of what you accomplished here: http://forum.xda-developers.com/showthread.php?p=34185439
Thank you, thank you, thank you! It can't be said enough. I had family photos that I would not have been able to reproduce. Much love to you and yours!
:good: :highfive: :victory:
so did anyone dare to restore the drive (all of storage, everything !)? without bricking the thing ?
mai77 said:
so did anyone dare to restore the drive? w.o bricking the thing ?
Click to expand...
Click to collapse
What drive? A partition?
scandiun said:
If you are using Linux / OS X you have native tools, for Windows download Cygwin, and install with it netcat, pv and util-linux. Get them from Cygwin's setup.exe
Click to expand...
Click to collapse
netcat is obsolete (mark to even find it) install net / nc instead
---------- Post added at 01:03 PM ---------- Previous post was at 01:03 PM ----------
scandiun said:
What drive? A partition?
Click to expand...
Click to collapse
all of storage I mean. the full monty ...
----------------------
parted seems to not work fully with Samsung galaxy Y = SGY proprietary rfs filesystem
on SGY mmcblk0 gives you the sd card, not internal storage with android.
backing up my sd card was a thing I could even do before I read this (lol)
mai77 said:
netcat is obsolete (mark to even find it) install net / nc instead
Click to expand...
Click to collapse
nc is an abbreviation for netcat. On Cygwin, you choose to install either, but for the original, written by the *Hobbit*, that allows direct execution of commands with -e and -t, you have to uncheck "Hide obsolete packages" on Cygwin's setup.exe.
The two of them are here:
Netcat 1.10 (netcat.traditional): http://www.netgull.com/cygwin/release-legacy/netcat/
Netcat 1.107 (netcat.openbsd): http://www.netgull.com/cygwin/release/nc/
mai77 said:
parted seems to not work fully with Samsung galaxy Y = SGY proprietary rfs filesystem
on SGY mmcblk0 gives you the sd card, not internal storage with android.
backing up my sd card was a thing I could even do before I read this (lol)
Click to expand...
Click to collapse
Can you post where is your whole memory block then or even the PIT file for that phone? You have an example here:
[Info] List of Samsung Galaxy S2 GT-I9100 devices and partitions
scandiun said:
Can you post where is your whole memory block then or even the PIT file for that phone? You have an example here:
[Info] List of Samsung Galaxy S2 GT-I9100 devices and partitions
Click to expand...
Click to collapse
I don't know where the "whole memory block" is or what it is. is it internal storage = NAND ?
here is the pit file:
mai77 said:
I don't know where the "whole memory block" is or what it is. is it internal storage = NAND ?
Click to expand...
Click to collapse
Yes it is the NAND. See your pit analysis here:
[INFO] Samsung Galaxy Y GT-S5360 PIT File Analysis
OK
scandiun said:
Yes it is the NAND. See your pit analysis here:
[INFO] Samsung Galaxy Y GT-S5360 PIT File Analysis
Click to expand...
Click to collapse
very interesting tool.
but how do I backup NAND in one piece on SGY ?
mmcblk0 = sd card
???blk0 = NAND
it must be somewhere ...
mai77 said:
but how do I backup NAND in one piece on SGY ?
mmcblk0 = sd card
???blk0 = NAND
it must be somewhere ...
Click to expand...
Click to collapse
Is almost sure that is under /dev/block. Please post the output of
Code:
ls -lR /dev/block
I bought a Moto G(XT1031) last week. I rooted it, and before removing any system apps, created a backup in TWRP of the system and boot partitions.
Now, I have the 8GB model(couldn't find a 16GB one cheap enough) and so I would like to create a .img of the system and boot partitions. I still have the TWRP backup, however I want to remove it from my phone as it's eating up almost a GB of precious space.
I've seen some assorted stuff on how to pull .img files from the phone, but I' m not sure which ones will work with mine. Any help would be appreciated!
You could use: adb pull /sdcard/TWRP c:/MotoG_Backup/TWRP
Now you can delete that folder from phone.
Then if you want to restore TWRP Backup: adb push c:/MotoG_Backup/TWRP /sdcard/TWRP/
Alternatively:
ADB Dumping Methods: *ROOT REQUIRED*
You need to grant the SuperSU permission popup that will appear on your phone.
System:
adb shell su -c "dd if=/dev/block/platform/msm_sdcc.1/by-name/system of=/sdcard/system.img"
adb pull /sdcard/system.img
lost101 said:
You could use: adb pull /sdcard/TWRP c:/MotoG_Backup/TWRP
Now you can delete that folder from phone.
Then if you want to restore TWRP Backup: adb push c:/MotoG_Backup/TWRP /sdcard/TWRP/
Alternatively:
ADB Dumping Methods: *ROOT REQUIRED*
You need to grant the SuperSU permission popup that will appear on your phone.
System:
adb shell su -c "dd if=/dev/block/platform/msm_sdcc.1/by-name/system of=/sdcard/system.img"
adb pull /sdcard/system.img
Click to expand...
Click to collapse
Why, thank you sir! I was unaware that you can push files to your device while in the recovery! That is why I wanted a system image file.
Thanks again!
Disclaimer
Code:
Whatever you do, it's your decision and you do it on your own risk!
I am not responsible for bricks, any kind of damages, data loss or any other unwanted result!
Everything you do is done on YOUR OWN RESPONSIBILITY!
And I am also not responsible if you lose the chat with your crush.
You might want to transfer your internal storage such as /sdcard to your computer directly and compressed as fast as possible.
So here is how you do it (on linux using bash):
Reboot your phone into TWRP, connect your device to your computer, and run the following command:
Code:
adb shell mkfifo /bk.pipe && adb exec-out 'tar -cvap data/media/0 2>/bk.pipe' | dd of=sdcard.tar & adb shell cat /bk.pipe &
In this example your data (in this case /data/media/0) will be backed up to your computer into the file sdcard.tar.
Compression is done automatically and you will get a nice output on your terminal on which file is being backed up currently.
When everything is finished you should get a message similar to this on your terminal:
Code:
79692237+2 records in
79692238+0 records out
40802425856 bytes (41 GB, 38 GiB) copied, 3550,38 s, 11,5 MB/s
(On the result of this you can see that the speed is around 12 MB/s, means that 1 GB backups in less than 1 minute)
When you get that, it means that your backup has finished and you can safely remove the usb cable.
Note that you can do this with any directory on your device, so if you desire to back up /data, just write data instead of data/media/0.
Example of the resulting sdcard.tar file (opened on my computer):
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Decompressing your data directly to your device might get a little trickier:
Code:
adb shell mkfifo /rst.pipe && adb push sdcard.tar /rst.pipe & adb shell 'cd /; cat /rst.pipe | tar -xv'
If that does not work, you can just extract the content of the tar file normally onto your computer and then copy the files over using adb push or copy the tar file to your phone and extract it using the following command:
Code:
adb push sdcard.tar /sdcard/sdcard.tar
adb shell 'cd /; tar -xvf /sdcard/sdcard.tar'
And that should do the job as well.
---
See also:
tar to pipe but keep -v verbose output separate from STDERR
Transferring binary data over ADB shell (ie fast file transfer using tar)
Compose and pipe a gzipped tar archive in one go
Bash tar and output to log file
GNU Tar
Encrypting and Compressing
How encrypt data/text stream instead of a file?
Post on my blog:
How to backup compressed data from your android device to your computer
Is it possible to do this with windows cmd?
Hmm, that looks really complicated. How about rsync instead?
(Note: This requires root and it requires rsync... It's present on CyanogenMod and probably other custom roms, but usually not on stock ROMs...)
Code:
1. Create rsync.conf:
[root]
path = /
read only = no
uid = root
gid = root
2. Upload the file to the phone
$ adb push rsync.conf /sdcard/
3. Restart adbd with root permissions
$ adb root
4. Set up TCP forwarding of the rsync daemon through adb
$ adb forward tcp:9873 tcp:873
5. Run the daemon on the phone
$ adb shell /system/xbin/rsync --daemon --config /sdcard/rsync.conf --no-detach
6. Copy files in both directions
rsync -v rsync://localhost:9873/root/
rsync --help
Edit: You can also run rsync via SSH, e.g. with the help of SSHdroid:
Code:
rsync -e "ssh -l root -p 2222" -v <phone-ip>:/
adbsync utility by @Renate NST is pretty perfect for this too. I've got a batch automation for Windows packaged up in my Odds and Ends thread but I'm sure it wouldn't be hard to get something similar setup for Linux.
Nice!
Is there a way to do this but instead of coping files, copy the raw data from the internal memory? So that I can recover deleted files from the device from the generated raw file.
Tarcis said:
Nice!
Is there a way to do this but instead of coping files, copy the raw data from the internal memory? So that I can recover deleted files from the device from the generated raw file.
Click to expand...
Click to collapse
Something like (run in the host console)
Code:
adb exec-out "cat <sdcard partition> 2>/dev/null" > sdcard.raw
Or
Code:
adb exec-out "dd if=<sdcard partition> bs=64k 2>/dev/null" > sdcard.raw
This works on Windows too.
Something like on the first post that works on Windows but without the progress report (which is what the FIFO is for):
Code:
adb exec-out "tar czf - /sdcard" > sdcard.tar.gz
adb exec-out is an undocumented adb command that works like adb shell but without creating a pseudoterminal thus it works with binary content.
Has it been confirmed that all files are backed up this way? Main issue I have when backing up via MTP is that not all files seem to be transferred properly e.g. Carbon/Helium backups missing the json file.
tech3475 said:
Has it been confirmed that all files are backed up this way? Main issue I have when backing up via MTP is that not all files seem to be transferred properly e.g. Carbon/Helium backups missing the json file.
Click to expand...
Click to collapse
As far as I know, tar backups all files recursively in the directory you specified. Hence that should work perfectly fine. I recommend to use TWRP 3+ for this, but you can also try it on android itself.
Tarcis said:
Nice!
Is there a way to do this but instead of coping files, copy the raw data from the internal memory? So that I can recover deleted files from the device from the generated raw file.
Click to expand...
Click to collapse
It depends on your device, but in general, you can transfer any binary data over exec-out in highspeed.
Code:
adb root
adb exec-out "dd if=/dev/block/mmcblk0" > mmcblk0.img
In this case it would copy the whole internal storage including all partitions and everything from 0 to end of your internal emmc to your computer.
On newer devices/systems you can also use the following to copy the single partitions (in this example /data) as raw image to your pc:
Code:
adb root
adb exec-out "dd if=/dev/block/bootdevice/by-name/userdata">userdata.img
Examples for partition names: system, data, userdata, cache, boot, LOGO, recovery, ...
If you want to compress all partitions separately into a tar archive, following command should do:
Code:
adb root && adb exec-out 'cd /dev/block/bootdevice/by-name && tar -ca ./' > allpartitions.tar
This would also work on windows.
char101 said:
Something like (run in the host console)
Code:
adb exec-out "cat <sdcard partition> 2>/dev/null" > sdcard.raw
Or
Code:
adb exec-out "dd if=<sdcard partition> bs=64k 2>/dev/null" > sdcard.raw
This works on Windows too.
Something like on the first post that works on Windows but without the progress report (which is what the FIFO is for):
Code:
adb exec-out "tar czf - /sdcard" > sdcard.tar.gz
adb exec-out is an undocumented adb command that works like adb shell but without creating a pseudoterminal thus it works with binary content.
Click to expand...
Click to collapse
xdvs23 said:
It depends on your device, but in general, you can transfer any binary data over exec-out in highspeed.
Code:
adb root
adb exec-out "dd if=/dev/block/mmcblk0" > mmcblk0.img
In this case it would copy the whole internal storage including all partitions and everything from 0 to end of your internal emmc to your computer.
On newer devices/systems you can also use the following to copy the single partitions (in this example /data) as raw image to your pc:
Code:
adb root
adb exec-out "dd if=/dev/block/bootdevice/by-name/userdata">userdata.img
Examples for partition names: system, data, userdata, cache, boot, LOGO, recovery, ...
If you want to compress all partitions separately into a tar archive, following command should do:
Code:
adb root && adb exec-out 'cd /dev/block/bootdevice/by-name && tar -ca ./' > allpartitions.tar
This would also work on windows.
Click to expand...
Click to collapse
Thank you so much for the commands, but I ran them on Windows, and I get no error, but the output file is 1kb.
I tried running on dos prompt, I´ll try again tomorrow on cygwin
thank but too hard for me
can i have another software to do this?
There are a few ways which you can make a Nandroid backup. The recommended way is to use a custom recovery to create one, and it’s the only way to restore from one. You should be able to use any custom recovery that offers Nandroid backup capabilities — if you don’t want to search around, the best choices are CWM and TWRP. Once you’ve flashed a custom recovery onto your device, you can boot into it and choose to create (or later on, restore from) a Nandroid backup. It’ll go through the process and create a backup file on your microSD card or other equivalent storage location.
Emma Tayler said:
There are a few ways which you can make a Nandroid backup. The recommended way is to use a custom recovery to create one, and it’s the only way to restore from one. You should be able to use any custom recovery that offers Nandroid backup capabilities — if you don’t want to search around, the best choices are CWM and TWRP. Once you’ve flashed a custom recovery onto your device, you can boot into it and choose to create (or later on, restore from) a Nandroid backup. It’ll go through the process and create a backup file on your microSD card or other equivalent storage location.
Click to expand...
Click to collapse
Then tell me how to create a nandroid backup from your internal storage (/sdcard or /data/media/0)
Tarcis said:
Thank you so much for the commands, but I ran them on Windows, and I get no error, but the output file is 1kb.
I tried running on dos prompt, I´ll try again tomorrow on cygwin
Click to expand...
Click to collapse
Then open the file using e.g. notepad and show me the contents. It could be an error message or something...
cuddas said:
thank but too hard for me
can i have another software to do this?
Click to expand...
Click to collapse
I'm actually making backup tools which you can then use to create backups easily, maybe I'll also make a program with a user interface for windows where you can do that much easier.
Of course I cant promise anything atm cuz I have some other things to do as well.
xdvs23 said:
Disclaimer
Code:
adb shell mkfifo /bk.pipe && adb exec-out 'tar -cvap data/media/0 2>/bk.pipe' | dd of=sdcard.tar & adb shell cat /bk.pipe &
Click to expand...
Click to collapse
replace with this one
Code:
adb shell mkfifo /bk.pipe && adb exec-out 'tar -cvap data/media/0 2>/bk.pipe' | dd of=sdcard.tar & adb shell cat /bk.pipe &
xdvs23 said:
Then open the file using e.g. notepad and show me the contents. It could be an error message or something...
Click to expand...
Click to collapse
good idea!
The files contained:
allpartitions.tar: /sbin/sh: syntax error: unterminated quoted string on one tr
mmcblk0.img: 0kb
sdcard.raw : /sbin/sh: syntax error: unexpected redirection
userdata:img: dd: can't open '/dev/block/bootdevice/by-name/userdata': No such file or directory
Does this help?
http://syedtahir16.blogspot.in/2014/11/the-ultimate-android-backup-solution.html
syedtahir16 said:
Does this help?
http://syedtahir16.blogspot.in/2014/11/the-ultimate-android-backup-solution.html
Click to expand...
Click to collapse
No, but thanks. The ideia is to be able to backup without being able to load the OS.
how to restore?
Actually a few days back I was unlocking my bootloader of redmi note 6 pro phone and I didn't know of that fact that it does a factory reset during this and accidentally I lost all my data but the most important data which I lost were some videos and photos which were clicked recently like 2-3 days ago and some videos I really need help coz I need to recover those data so anyone please help me in doing that thing.
Ask authorized service center whether they have the forensic tool(s) to recover the data of interest, and if so, let them try to recover them.
jwoegerbauer said:
Ask authorized service center whether they have the forensic tool(s) to recover the data of interest, and if so, let them try to recover them.
Click to expand...
Click to collapse
actually i can't reach any authorized or non-authorized centers the only thing I can do is by my own
If phone is rooted, and if it's accessible by ADB, only then all you can do at your own is to create a 1:1 ( means: bitwise ) copy of the partitions you are interested in and store this image on your computer - what later on by forensic tools may get made readable.
jwoegerbauer said:
If phone is rooted, and if it's accessible by ADB, only then all you can do at your own is to create a 1:1 ( means: bitwise ) copy of the partitions you are interested in and store this image on your computer - what later on by forensic tools may get made readable.
Click to expand...
Click to collapse
Like Can You Share Any Video Or Something Which Explains This Set By Step
I don't have a video at hand and I won't search for such one.
These are the ADB commands you have to run - one by one
Code:
adb devices
adb su -c 'mount -o rw,remount /<NAME-OF-PARTITION-TO-GET-COPIED>'
adb reboot
adb devices
adb exec-out "dd if=/<PARTITIONS-BLOCK-DEVICE-NAME-TO-GET-COPIED> bs=4096" > C:\COPY_OF_PARTITION.img
adb su -c 'mount -o ro,remount /<NAME-OF-PARTITION-TO-GET-COPIED>'
adb reboot
where <NAME-OF-PARTITION-TO-GET-COPIED> & <PARTITIONS-BLOCK-DEVICE-NAME> are placeholders what you have to substitute by their real names.
jwoegerbauer said:
I don't have a video at hand and I won't search for such one.
These are the ADB commands you have to run - one by one
Code:
adb devices
adb su -c 'mount -o rw,remount /<NAME-OF-PARTITION-TO-GET-COPIED>'
adb reboot
adb devices
adb exec-out "dd if=/<PARTITIONS-BLOCK-DEVICE-NAME-TO-GET-COPIED> bs=4096" > C:\COPY_OF_PARTITION.img
adb su -c 'mount -o ro,remount /<NAME-OF-PARTITION-TO-GET-COPIED>'
adb reboot
where <NAME-OF-PARTITION-TO-GET-COPIED> & <PARTITIONS-BLOCK-DEVICE-NAME> are placeholders what you have to substitute by their real names.
Click to expand...
Click to collapse
Which partition should I choose for recovery
Look inside here:
Extract Android Userdata Partition for Recovery and Forensics - Root101 | Fundamentals of Linux Administration
Open Source and Linux. Guides and Ideas for Designing Highly Available and Scalable Production Environments
www.root101.net
.
Using a Xiaomi Mi11 (rooted). Recently upgraded to MIUI 13 which is based on Android 12. Was going to do my usual TWRP backup before bringing the phone in to fix some minor problems with sticky volume keys, when I realized (!) my TWRP cannot even mount the /data partition, let alone decrypting and doing any backup.
So I started reading up on TWRP developments, and realized TWRP for now has lost its ability to see anything under /data if your phone is on Android 12.
Never a fan of things like Titanium backup where the backup is done on an app-by-app basis, so a lot of of settings like magisk modules / phone behavior, etc etc cannot be retained (at least that was my impression of it when I briefly tried those solutions). So when I decided to bring my phone in for repair anyway, I went ahead and wiped the phone clean, and had to live with losing 10 day's worth of my data - 10 days because fortunately I did a backup just before I upgraded from MIUI 12 to MIUI 13 10 days ago... (yeah could have done a lot of manual work to salvage some of the data before I wiped it clean, but I didn't bother with the tedious processes).
So I now have a fixed phone, no more sticky buttons, and restored my nandroid backup with the older MIUI 12 system (android 11 based), and not even considering moving back to MIUI 13 until there is a feasible way to do a TOTAL backup of the /data partition, in others words a nandroid backup on Android 12....
Question - is there any feasible method of doing a Nandroid Backup on an Android 12 system, with or without TWRP?
Thank you !!!
A NANDroid-backup is the bitwise 1:1 copy of existing Android system.
If phone's Anndroid OS is rooted then you always can launch a NANDroid-backup.
This can get achieved by pure ADB commands what of course requires ADB is enabled on phone.
xXx yYy said:
A NANDroid-backup is the bitwise 1:1 copy of existing Android system.
If phone's Anndroid OS is rooted then you always can launch a NANDroid-backup.
This can get achieved by pure ADB commands what of course requires ADB is enabled on phone.
Click to expand...
Click to collapse
Could you elaborate?
I can picture this issue -
if you do "adb shell" to enter terminal (or plain adb pull?) while your phone is switched on, a lot of files are being locked and/or being modified while the phone OS is running so how can someone just take a snapshot of everything under /data even with proper adb commands?
And if you go to recovery mode first, well at the present time no TWRP can access the data partition it seems. So again even with the appropriate adb commands, no copying is possible....?
Any clarification appreciated !
You would run
Code:
adb wait-for-device
adb root & adb shell "stop"
adb shell "mount -o rw,remount /data"
: run the backup command here
adb shell "start" & adb unroot
xXx yYy said:
You would run
Code:
adb wait-for-device
adb root & adb shell "stop"
adb shell "mount -o rw,remount /data"
: run the backup command here
adb shell "start" & adb unroot
Click to expand...
Click to collapse
Dear Android export @xXx yYy - wow ! This looks really promising !!
I just did a quick test by going straight to adb shell, "su", then "stop". My phone screen totally went blank, and I was amazed ! This is awesome !!! "start" and after a while the phone boots up again.
I then tried "top" while the phone is stopped. It seems to still have a few android related processes running, so I am not 100% sure if the whole system has been frozen. But you obviously know what you are talking about, and I have faith in you.
(by the way, I cannot "adb root", seems like after doing a quick search I will need to make my phone think it is a development build by patching the adbd daemon first on my phone.. suggestions on what to do appreciated)
You have just made me decide to spend the coming hours to test the following. Let me know if I should skip any of the steps below because you know it works so I don't need to waste time to validate:
1. Do a proper backup with TWRP first in case I screw up anything
2. start a terminal session with adb shell
3. "su", "stop"
4. "cd /data"
5. "tar -cvpzf /data/backup.tar.gz -C /data"
(If no error, this should be my nandroid backup...?)
6. flash phone and wipe everything clean, so it is back to brand new status, non-rooted
7. reboot phone, see if it is starting new as if I have just bought the phone
8. root the phone, then try and "stop", "delete everything under /data except /data/media", "delete everything under /data/media", "copy backup.tar.gz back to /data", "tar -xvzf /data/backup.tar.gz -C /"
9. If phone works and is back to the state immediately before backup, then restore successful
Take note that
Code:
adb root
is giving root access to adb ( adbd - read: adb daemon )
what has nothing to do with giving root access to current Android user with following shell command
Code:
adb shell "su"
Also take note that Android services aren't located in /data partition, the partition you want to back up.
With @xXx yYy 's help I think I am getting somewhere.
So essentially a "stop" command in android will stop Zygote (i.e. the mother of all app processes if I am not mistaken). Once you have stopped Zygote, I believe you are then free to make a duplicate of the entire /data environment.
So far that's exactly what I have done. Created a tar.gz file with a size of around 40GB. I believe I am halfway there in my quest to do Nandroid without TWRP, but what I still need to try is to restore the tar file after factory resetting the phone. Will be a time consuming process (as obviously I will also need to have a tried-and-true real backup created first in case I screw something up... I am doing everything on my main phone that I actually use everyday), so I will continue my experiment in the coming days.
One question I have already encountered however - I still cannot do "adb root", which would have allowed me to directly create the backup tar file AND pipe it to my PC all in one go. So far I have had to tar all within the phone, which means space will be a constraint, and it is more time consuming creating the backup file THEN think of a way to transfer that file out of the phone. Already posted a question here asking for help, and if anyone knows of a good way to get adbd to grant adb root request, please let me know.
Above all else, once I have a working method, and I have polished the process, I will be happy to share. I suspect many others are also yearning for a good backup / restore procedure on Android 12.
one can't backup /data partition this way, because tar is just a toybox applet not cabable of preserving secontext. get a gnu tar binary (for example from opengapps installer), set mount namespaces to global, set selinux permissive (if kernel allows it, important) and run from su
/storage/1234-5678 is exFAT and has enough free disk space
Code:
tar --selinux --xattrs --numeric-owner -vcpPf /storage/1234-5678/data.tar /data
/storage/1234-5678 is vfat or less free disk space
Code:
tar --selinux --xattrs --numeric-owner -vcpP /data | gzip | split -a 1 -b 1024m - /storage/1234-5678/data.tar.gz.
another approach would be loop mount some file and busybox cp -avc everything where the -c flag is responsible for secontext (proper busybox required)
--numeric-owner flag is recommended if you are planning to extract it on linux PC later
you could also exec-out straight to PC if no MicroSD Card available, but requires gzip or other compressed stream, otherwise windows will mess up linefeed with carriage return and render your file unreadable
Code:
adb exec-out "su -c 'tar --selinux --xattrs --numeric-owner -vcpP /data | gzip'" > data.tar.gz
restoring .tar.gz from TWRP is absolutely possible, it's just that TWRP can't handle encrypted userdata partition (yet)
Code:
cat /external_sd/data.tar.gz.* | gzip -d | tar --selinux --xattrs -vxpPC /
(where tar must called with full path to binary like /cache/tar or /tmp/tar, or unlink /sbin/tar applet and place binary /sbin, or just rename it gtar)
Note: bitwise 1:1 copy of apps is not possible/sufficient if you factory reset your device, because apps might save data in TEE TrustZone (which is flushed on factory reset)
Hi @seemebreakthis, very interesting discussion on Android 12 backup!
Did you reach a workable solution with this?
Since we can restore most apps from a Google backup, it seems the real issue is to recover the user settings etc. after the Google restore.
Interested in this. Any success thus far?
aIecxs said:
one can't backup /data partition this way, because tar is just a toybox applet not cabable of preserving secontext. get a gnu tar binary (for example from opengapps installer), set mount namespaces to global, set selinux permissive (if kernel allows it, important) and run from su
/storage/1234-5678 is exFAT and has enough free disk space
Code:
tar --selinux --xattrs --numeric-owner -vcpPf /storage/1234-5678/data.tar /data
/storage/1234-5678 is vfat or less free disk space
Code:
tar --selinux --xattrs --numeric-owner -vcpP /data | gzip | split -a 1 -b 1024m - /storage/1234-5678/data.tar.gz.
another approach would be loop mount some file and busybox cp -avc everything where the -c flag is responsible for secontext (proper busybox required)
--numeric-owner flag is recommended if you are planning to extract it on linux PC later
you could also exec-out straight to PC if no MicroSD Card available, but requires gzip or other compressed stream, otherwise windows will mess up linefeed with carriage return and render your file unreadable
Code:
adb exec-out "su -c 'tar --selinux --xattrs --numeric-owner -vcpP /data | gzip'" > data.tar.gz
restoring .tar.gz from TWRP is absolutely possible, it's just that TWRP can't handle encrypted userdata partition (yet)
Code:
cat /external_sd/data.tar.gz.* | gzip -d | tar --selinux --xattrs -vxpPC /
(where tar must called with full path to binary like /cache/tar or /tmp/tar, or unlink /sbin/tar applet and place binary /sbin, or just rename it gtar)
Note: bitwise 1:1 copy of apps is not possible/sufficient if you factory reset your device, because apps might save data in TEE TrustZone (which is flushed on factory reset)
Click to expand...
Click to collapse
You are a genius! That's excactly what i was searching for! Thank you!
This is a bit beyond me.. though I'm looking for a full ROM backup on Android 12. Does this work?
TWRP 3.7.0 is for Android 12 including Encryption Support (except for Samsung)