Hi folks,
I've always been annoyed by the lack of a true simple backup option for Android. A recent device failure reminded me of this situation. The purpose of this post is to share my experience, and a solution that seems to work for me.
Google has a built-in backup capability in Android that backs up your contacts (assuming that you keep your contacts with Google), and it'll keep track of what apps you have installed, and it may even back up app settings if the app developer has chosen to opt into that feature. That's a few "if's" already here. Things that will not get backed up for sure are your photos, movies, and anything on your SD card. Also, various customizations, such as home screen layout and even other Android settings don't seem to be restored. Various carriers seem to have their own cloud backup options, but this seems vaguely redundant with some aspects of the Android-level backup, it's not immediately clear if it backs up *everything*, and it may cost extra money every month to have stuff backed up online. No thanks.
So how do you back up your own Android phone? The requirements before proceeding:
Root Access
A custom recovery ROM, such as CWM
Does anybody else find this somewhat ridiculous?
My first venture into backing up into the phone was to go the CIFS route so that stuff could be easily moved off of the phone. (The LG G2 doesn't have an external card). One of the problems with this was that if I didn't unmount the share before leaving the wifi network, some things could behave differently. Back to the drawing board...
First, in order to get as close to a "bare metal" backup, you'll need to make a nandroid backup. And in order to do this, you'll need a recovery rom. There are supposedly apps that do an online nandroid backup, but I've heard that this technique can be problematic at times. The two popular recovery roms are CWM and TWRP. I attempted to load these roms manually, but I'd get an error: "Loki aboot version does not match device". I didn't feel like digging into this much. Some other options, such as ROM Manager said that my phone wasn't supported. In the end I used FreeGee, which can install CWM or TWRP with a single touch.
One other speed bump that I ran into is that at least with the CWM and TWRP versions installed by FreeGee (6.0.4.7, and 2.6.3.3, respectively at this point), If you encrypt your phone, you cannot make a nandroid backup. Really? You have to choose between extra security or having a backup? Android seems to still have a lot of growing up to do. But I digress...
One notable difference that I ran into between TWRP and CWM is that TWRP puts your nandroid backups out in the open on the SD card, while CWM puts them in a protected area. Either technique has a downside. This means that with TWRP backups, a rogue application might be able to access sensitive bits of your phone. With CWM backups, this means that a non-root-enabled application won't be able to access the backups, even if you want it to. We'll run into this later. The other thing I ran into with my LG phone is that backups are dated 1970. Apparently LG doesn't expose time capability to the environment that CWM/TWRP run in. My workaround here is to use ROM Manager to rename the backup accordingly.
Now we've got a nandroid backup, and the next step is to get the data off of the phone. Rsync backup for Android works quite well. The default sdcard backup field used by this app ended up backing up a single symlink to my NAS, so I ended up changing it to back up /storage/emulated/legacy instead, and that works fine. Now with a single click I can sync my photos and anything else that lives on the SD card. And rsync over ssh is quite efficient as well.
Now, how to get the nandroid backups from CWM to rsync. While they're physically on the SD card storage of the phone, they're logically protected and not visible on /sdcard for security reasons. How to we back them up? We need rsync to have root access. I've tried a couple of options to make this happen, either by playing with the command-line options in the rsync android app, as well as by trying to create a wrapper for the rsync binary. No luck with either route. While not as simple, my fallback technique was to run a script to relax the permissions on the directory where the CWM nandroid backups live. And then once I'm done rsyncing them, I run a different script to restore the permissions.
opencwm
#!/system/bin/sh
mount -o rw,remount /
chmod 755 /mnt/shell
mount -o ro,remount /
echo done!
closecwm
#!/system/bin/sh
mount -o rw,remount /
chmod 700 /mnt/shell
mount -o ro,remount /
echo done!
You can use whichever android script runner app you're comfortable with. The (separate) rsync job that I created for CWM backups works on: /mnt/shell/emulated/clockworkmod.
That's about it. What's a sane backup strategy to use for your phone? Once you've got your phone set up mostly how you like, reboot to CWM and create a backup of the OS. Rsync this as described above. As you normally use your phone, rsync the sdcard periodically. If you want to have backups of your apps and system data as well, use something like TitaniumBackup to back them up to the SD card. Encrypt them if you want them to be protected. Now, every time you rsync the SD card, you'll have backups of that app data as well.
So in the end, you'll have a nandroid backup as your "bare metal" backup. I'd only expect you to create these when you do major OS changes. The usual backup will be the normal sdcard rsync job, which functions as an incremental backup. Run this job as frequently as you like. Now if you ever need to restore your phone, go into CWM and restore the backup. Then go restore the SD card contents, and restore the Titanium Backup. You'll obviously need CWM on the new phone (if it was replaced), and if you've cleared the CWM backup off of the phone itself (e.g. due to space constraints) you'll need to copy it back to the phone (or otherwise make it available, e.g. via USB OTG) before proceeding.
I have the same opinion. My set up is Helium.
https://play.google.com/store/apps/details?id=com.koushikdutta.backup
You don't have to be rooted. You do need a PC to give Helium "temp root access" though. If you are rooted this is not required. Essentially it makes a back up of your apps then you can upload these to most of the cloud storage services.
This is not the full solution for settings and customization of the phone etc nor is it the ideal solution. It gives me all my apps, their settings and game saves that I'd be worried about after a factory reset or losing my phone. Google backs up my photos but Helium offers this service too.
Give it a shot, for a simple back up solution it works for me.
forrestgump2000 said:
One notable difference that I ran into between TWRP and CWM is that TWRP puts your nandroid backups out in the open on the SD card, while CWM puts them in a protected area.
Click to expand...
Click to collapse
I've recently switched to TWRP, and it seems that there is an option to encrypt the nandroid backup. So this makes backup easier, as the script to change permissions on the CWM backup location is no longer necessary.
forrestgump2000 said:
I've recently switched to TWRP, and it seems that there is an option to encrypt the nandroid backup. So this makes backup easier, as the script to change permissions on the CWM backup location is no longer necessary.
Click to expand...
Click to collapse
However, the backup is ridiculously slow when encryption is enabled.
Related
Well since this was my first root I guess I made a mistake or maybe not?
So this is the case :
I have an HTC One S (Rooted using SuperSu Method)
What I did for backup is just copy all the content from the phone (in hard drive mode) to a folder on my pc (which ended up being around 7gigs)
My question is how do I restore all that data\pics\apps\profiles back to my phone?
(Tried to copy past it back into the phones SD-Card and rebooting, The phone doesn't recognize any of the data)
Any way to actually make this work? or is it a lost cause?
Thanks for the help! :laugh:
As far as apps and their data goes, only some info is saved on the sdcard, the rest is saved on the devices internal memory.
Only restoring the stuff saved on the SD card aint gonna bring back any apps. They need to be reinstalled properly.
Goatshocker said:
As far as apps and their data goes, only some info is saved on the sdcard, the rest is saved on the devices internal memory.
Only restoring the stuff saved on the SD card aint gonna bring back any apps. They need to be reinstalled properly.
Click to expand...
Click to collapse
How about all the Contacts \ Messages and ect?
Because I have all the files on my PC (Exactly the same files that have been on the phone before the ROOT\Unblock)
Can I recover those back to the phone somehow?
Messages and contacts are stored on internal memory, unless you export them.
You dont have access to anything except /sdcard/ through the PC, unless you use ADB.
edit: What I mean is, if you have only backed up /sdcard/ (which seems to be the case), you have lost everything on the internal memory (such as messages and contacts etc).
Goatshocker said:
Messages and contacts are stored on internal memory, unless you export them.
You dont have access to anything except /sdcard/ through the PC, unless you use ADB.
Click to expand...
Click to collapse
So basically even tho I copied all the content from the phone itself to the pc, All I copied is squat.
All the raw data I get is pictures.
So no encrypted data can be found on the storage partition and I have to just start over from scratch?
And the next time I root or flash, just use Recovery backup?
You didnt copy all the content. You only copied what was stored on the /sdcard/ (which is more or less the only thing you have access to without root).
Theres a very slim amount of data android uses on the sdcard, unless the user specifically tells it to.
Next time you flash, you should back up all your apps you want to save with for example titanium backup (requires root, which you now have), save all the contacts to the sim card, and export all messages.
Then after you have wiped and flashed a new rom, just restore the backups with titanium backup again, and import contacts from sim.
The backup/restore in recovery backups/restores the WHOLE system, so when you restore it it will be exactly like the way it was when you made the backup. (edit: to clarify, if you flash a new rom, then restore a backup made on another rom, you will return to that rom the backup was made on)
Goatshocker said:
You didnt copy all the content. You only copied what was stored on the /sdcard/ (which is more or less the only thing you have access to without root).
Theres a very slim amount of data android uses on the sdcard, unless the user specifically tells it to.
Next time you flash, you should back up all your apps you want to save with for example titanium backup (requires root, which you now have), save all the contacts to the sim card, and export all messages.
Then after you have wiped and flashed a new rom, just restore the backups with titanium backup again, and import contacts from sim.
The backup/restore in recovery backups/restores the WHOLE system, so when you restore it it will be exactly like the way it was when you made the backup. (edit: to clarify, if you flash a new rom, then restore a backup made on another rom, you will return to that rom the backup was made on)
Click to expand...
Click to collapse
Got it
I actually found some encrypted data such as : Whats up message history, Some Contacts, Specific apps.
And I recovered all my contacts VIA my google account.
Only thing lost is the applications, which I will reinstall in like 30 minutes.
Anyhow I am quite happy I got into the rooting business, Already feel the different abilities and the uncovered raw code + new line of root supported apps waiting to be savaged by me.
Thank you for the help anyways (and the quick reply as well)
And I think the original ROOT tutorial post should have a step which is a "back-up step" which explains how to back up most of the info before rooting, and the ability to back up with root.
Cheers!
Hi gents. I am trying to restore my phone to a recent clockworkmod recovery backup but several of the latest backups won't restore the downloaded programs of the time, nor do they seem to be restoring the general Android settings. Older backups are working perfectly, however, and I can't think of any significant or unusual changes I made to the system that could stop it working properly. Also, I have been backing up and restoring the same way.
I'm using Revolutionary CWM v4.0.1.4, HTC ICS 4.0.4 (w/ older HBOOT for S-OFF), HTC Desire S.
I find this very strange, and unsettling that clockworkmod's backup may be unreliable. Has anybody experienced this?
Another thing, I have also been backing up the entire SD card data in the later backups (just copying to computer via card reader), thinking it may be best to restore backups with this data, but it seems to not make a difference whether this is done, or the data left as-is, or the card cleared. What's best practice in regard to the Android data that's put on the SD card?
My phone is up the creek. Help!?
i am having the same problem restoring CWM v4.0.1.4 backups on my galaxy s2: older backups restore perfectly and newer backups restore incompletely (some settings and SMSs, but no apps and no contacts).
i believe this is an issue with large nandroid images (containing >10,000 files) being unable to restore properly. this implies the backup itself is intact, luckily.
read this:
http://forum.xda-developers.com/showthread.php?t=1154892
to rebuild my phone i started by using unyaffs on my PC to access the files on the CWM nandroid image and restore my contacts, call logs, etc:
http://forum.xda-developers.com/showthread.php?t=1370349
you can find additional file locations listed in this thread:
http://forum.xda-developers.com/showthread.php?t=969650
i then used appextractor (available in the play store) to manually restore my apps and app data from the CWM nandroid image. some needed to be redownloaded.
it's a schlepp but better than losing everything entirely.
i've read that TWRP is a more reliable way to backup and restore. i'm certainly not using CWM again.
regarding your second question, the data on your internal and external SD card is irrelevant to the function of the OS, but some apps may rely on that data. for example large apps install on the SD card.
Thanks so much for that matey. I searched and searched but hadn't come across the 10,000 files limitation. How annoying!
Anyhoo, before I saw your post I ended up restoring the latest working backup and reinstalled apps, configured things, got the device mostly up to speed.. but there were some sentimental Messages I wanted to restore. I got that unyaffs exe, the 10,000 file fix one, and managed to extract the mmssms.db just fine..
But I soon learnt it seems to be a nightmare to get it back into ICS because the SQLite db is in full-vacuum mode or some such, meaning it is cached and rewritten to help avoid fragmentation or some such. So I overwrite the mmssms.db, set permissions and ownership, reboot, and it's overwritten with the current one. Argh. I tried clearing cache. Tried running SMS backup/restore apps while the 'good' mmssms.db was in place, but they seem to look at the db cache/temp data, not the mmssms.db.
For those facing my situation and want Messages from CWM/nandroid backups in ICS, I found a great tip just posted on one of the threads N-acetyltransferase linked;
RichAP said:
I did this yesterday using Titanium Backup for my SMS/MMS
You need to go into Preferences and check the *Migrate system data [Experimental] option
Then you go to More - Extract from nandroid backup
There it is listed as Dialer Storage
Click to expand...
Click to collapse
I had tried using TiBu to restore Messages from nandroid backup (no success), but Dialer Storage!? How annoying. Anyhoo it's sorted.
Then I quickly found a good alternative recovery and quickly bought the premium version of 4EXT. It's very good, but get this, it actually recovered my latest CWM recovery backup no drama, just like that! Ahh well I didn't need those hours of my life anyway. I've often thought the Desire S's codename Saga is rather apt.
In conclusion, TiBu looks damn good and 4EXT is a ripper. I'd recommend it however the Galaxy S2 support is apparently unconfirmed thus far. Maybe look into it nevertheless.
Thanks for the help dude.
what is the difference between a Online Nandroid backup and a titanium backup?
dady613 said:
what is the difference between a Online Nandroid backup and a titanium backup?
Click to expand...
Click to collapse
nanodroid backup referred to a full backup (kernel, rom, all apps, all settings) a completed backup of the entire OS.
Titanium backup is only used for application backup (included all the system apps, user download apps and all settings belong to the apps).
Both can be backup to local storage like internal memory or external sdcard and/or Online backup meaning to the cloud storage like Sugar sync, Asus cloud storage, Samsung cloud storage, dropbox...etc.. The advantage of Online backup is you can copy the backup down to your device from the cloud anywhere that have access to the internet.
thanks a lot buhohitr for the great and fast answer.
Usually nandroid backups are made through (a custom) recovery, which implies you need to reboot into recovery mode to perform the operation.
The one error in buhohitr's answer is that in this case, online means you don't have to reboot, the backup is performed while your android system is live and running. Less time spent offline (and on a phone, no missed calls...)
I was not aware this had been turned into an app, I knew only a version of the onandroid script. Will try that (on my unlocked phone, not my locked tablet).
quietseb said:
Usually nandroid backups are made through (a custom) recovery, which implies you need to reboot into recovery mode to perform the operation.
The one error in buhohitr's answer is that in this case, online means you don't have to reboot, the backup is performed while your android system is live and running. Less time spent offline (and on a phone, no missed calls...)
I was not aware this had been turned into an app, I knew only a version of the onandroid script. Will try that (on my unlocked phone, not my locked tablet).
Click to expand...
Click to collapse
You're correct if you referred to "Online" as a name of an app
Migrated to android about a year ago and still haven't been able to work out proper android backup.
I want to be able to mess with the system without fear of losing my data/configurations.
For example when installing Lollipop I encountered a lot of problems and every time I ****ed something up and had to reflash it I need to reinstall and reconfigure ABSOLUTELY EVERYTHING (Including redownloading about 13GB of music for google play music).
This has been a major letdown as I love tweaking and messing with the system getting it 100% the way I like it.
I've been able to get a basic functioning Lollipop installation but I want more.
Here's what I want to be able to backup to my PC (a 13GB backup on the device isn't very practical, especially when messing with the system):
My apps and all their settings (Including google play's downloaded music).
System settings.
This way and I can mess with the system and once I get it the way I want- restore the backup.
I know this might seem like a lazy google question but I am really at a lose.
I've scoured the internet looking for a solution, Tried numerous programs with no success.
Please if someone can guide me in the right direction I'll be forever grateful.
TWRP and/or Titanium Backup.
The backup via won't be 13 GB. It doesn't back up the /sdcard mount.
There are 3 backups you should do.
1) Nandroid backup from custom recovery such as TWRP
This backs up /system (where your ROM is), boot (where your kernel is) and /data (where all your user apps, user data and system data is). By user data, I mean app data. Not /sdcard as already mentioned.
This backup will be 3/4GB most likely if compressed and can be copied to your computer from /sdcard.
Nandroid backup will return your ROM and settings exactly to the point at which the backup was taken. Its a complete snapshot of everything you need.
2 ) titanium backup. This app is most useful when switching between ROMs or having had to wipe to resolve some compatibility issue. It takes a snap shot of each app, but its best just to snapshot user apps NOT system apps. Sometimes system data becomes incompatible and causes problems after an upgrade or ROM change. That's why we wipe when flashing new ROMs. You'll always be best setting up system settings (ringtones, alarms etc) manually but titanium takes the sting out of setting up your own user apps.
3) some kind of sdcard backup. Personally I use foldersync to backup my entire /sdcard whilst I sleep over WiFi to my NAS. Something like this would be extremely wise. If not, at least backup your photos using some cloud app.
Is there a way to perform a full backup via ADB?
Like a script or a line which creates a flashable full copy of one or all of the phones filesystems to PC.
Not(!)
adb backup -all
which backups merely system settings and not a migrate app which a) does not perform a full backup as it omits random files and b) works internal and neither TWRP backup-tool which only works internal.
andy01q said:
Is there a way to perform a full backup via ADB?
Like a script or a line which creates a flashable full copy of one or all of the phones filesystems to PC.
Not(!)
adb backup -all
which backups merely system settings and not a migrate app which a) does not perform a full backup as it omits random files and b) works internal and neither TWRP backup-tool which only works internal.
Click to expand...
Click to collapse
It can backup/restore your data, it can not backup your ROM/firmware, if you are not rooted, you will only be able to backup/restore user data, you will not be able to backup/restore any of your system apps/system app data, or any other part of system data and system settings, that will require rooting the device.
If you want a complete backup of the operating system and everything else on the device, you will need TWRP.
Sent from my SM-S767VL using Tapatalk
I am rooted. I am running TWRP.
I can't copy large amounts of data from inside the phone because something is buggy; this includes TWRPs backup-function. Copying works fine if and only if controlled by connected PC.
andy01q said:
I am rooted. I am running TWRP.
I can't copy large amounts of data from inside the phone because something is buggy; this includes TWRPs backup-function. Copying works fine if and only if controlled by connected PC.
Click to expand...
Click to collapse
Try flashing a newer or slightly older version of TWRP, it might fix the bugs that you are experiencing in TWRP. Is TWRP set to store your backups on external? Or is it set to store backups on your internal storage? Storing backups in internal takes up too much space, it's best to store backups on external.
As for adb...
https://forum.xda-developers.com/galaxy-nexus/guide-phone-backup-unlock-root-t1420351
There are several switches that can be used to be specific about what you want backed up and what you don't.
Sent from my SM-S767VL using Tapatalk
I updated to TWRP 3.4.0-0 (newest version) and backup from within TWRP still always fails between 20% and 30%. The last 30 to 40 backups failed and the only one that did finish to 100% is still broken and can't be read.
I would be thinking hardware-failure if not for the thing that copying anything from within the phone is super buggy, but copying when controlled from a PC works completely fine.
I guess I'll try the simple adb backup with some more parameters and then move on.
andy01q said:
I updated to TWRP 3.4.0-0 (newest version) and backup from within TWRP still always fails between 20% and 30%. The last 30 to 40 backups failed and the only one that did finish to 100% is still broken and can't be read.
I would be thinking hardware-failure if not for the thing that copying anything from within the phone is super buggy, but copying when controlled from a PC works completely fine.
I guess I'll try the simple adb backup with some more parameters and then move on.
Click to expand...
Click to collapse
How much free space do you have on internal storage? If there is less free space than your system, data and cache partitions combined, that may be why it is failing. Clear some space. I say this because when the backup is being created, the data that you want included in your backup is being read bit by bit from their partitions and then written into cache bit by bit and then it is read from cache bit for bit then it is written to its final storage location bit for bit. If there is not enough space to write it all into cache before writing it to its final location, it will fail.
The same rule applies when you move files from internal to external or vice versa, the same rule also applies when you are extracting files. It requires enough free space to write that data into cache before it goes where it is actually going.
Sent from my SM-S767VL using Tapatalk
Oh come on, I appreciate your efforts, but I felt being treated like a little child from your first answer and this tops it very much. Backup fails at 20-30% with 45GB of free internal space, external MicroSD removed and <10GB of data to backup. Aside from that I doubt that if a lack of free space causes the backup to fail, that this will not cause an error message to tell me what caused the fail.
By now I did the backups that I could backup:
That is I manually pulled all folders I deemed important.
Then I used "adb backup -apk -shared -all -system" which created a 7GB backup.
I also used the buggy migrate-app and ticked all boxes to get a 3GB backup.
And I used Googles cloud-service-backup.
In the end plenty of data and configurations were lost; worst is probably the Corona Warn App data as this can't be remedied.
andy01q said:
Oh come on, I appreciate your efforts, but I felt being treated like a little child from your first answer and this tops it very much. Backup fails at 20-30% with 45GB of free internal space, external MicroSD removed and <10GB of data to backup. Aside from that I doubt that if a lack of free space causes the backup to fail, that this will not cause an error message to tell me what caused the fail.
By now I did the backups that I could backup:
That is I manually pulled all folders I deemed important.
Then I used "adb backup -apk -shared -all -system" which created a 7GB backup.
I also used the buggy migrate-app and ticked all boxes to get a 3GB backup.
And I used Googles cloud-service-backup.
In the end plenty of data and configurations were lost; worst is probably the Corona Warn App data as this can't be remedied.
Click to expand...
Click to collapse
I can't help how you feel and you are entitled to feel however you wantl, but, at the end of the day, it is still only how you "feel", that doesn't make anything true or false. Just saying.
I didn't post anything to make you feel any kind of way, I was merely attempting to be as informative as I could because one can never gauge what someone else knows or understands before your interaction with them.
I've seen some other issues with TWRP backups failing and/or being unreadable, the solution is not a singular solution, there are several reasons why it fails and several fixes to solve it. Have you tried doing some google searches for:
"TWRP backups fail at 40%"
That pulls up some information that might apply or may not, you'll just have to peruse what you find to see if any of seems like it might be what you are experiencing and then try some of the solutions that others have tried. You might get lucky, you might not.
I'm more interested in helping you "find" an answer than whether I "know" the answer or "tell" you the answer.
Sent from my SM-S767VL using Tapatalk
Droidriven said:
Have you tried doing some google searches for:
"TWRP backups fail at 40%"
Click to expand...
Click to collapse
Yeah, I've done that for an hour or so before starting a thread - obviously b/c that has the chance for much quicker solutions and seems less work for me.
Also I did try several solutions like update TWRP, try smaller backups (the small ones do usually work) and so on. I think I did mention that I tried this over 40 times.
Some of the solutions suggest installing another ROM first, (apparently the ROM can cause Read-errors within TWRP?) but obviously I want the full backup before I try another ROM so I tried other methods to backup my data.
Also moving data via adb has worked completely without fail so far, so I wanted to try that path.
By now I can tell that TWRP full backup doesn't work on other ROMs either.
andy01q said:
Yeah, I've done that for an hour or so before starting a thread - obviously b/c that has the chance for much quicker solutions and seems less work for me.
Also I did try several solutions like update TWRP, try smaller backups (the small ones do usually work) and so on. I think I did mention that I tried this over 40 times.
Some of the solutions suggest installing another ROM first, (apparently the ROM can cause Read-errors within TWRP?) but obviously I want the full backup before I try another ROM so I tried other methods to backup my data.
Also moving data via adb has worked completely without fail so far, so I wanted to try that path.
By now I can tell that TWRP full backup doesn't work on other ROMs either.
Click to expand...
Click to collapse
Flash an older version or newer version of the exact same ROM as what you have but not doing any of the wipes, just flash ROM and that's it. This is called a "dirty flash", it will not wipe any of your data unless you specifically tell it to.
Alternatively, you could just do a backup via TWRP without including system and boot, just data and maybe cache(cache isn't necessary and might even cause issues if you restore it). Then, if you need to restore, flash the ROM the backup was created from then reboot to system, then boot back into TWRP then use "advanced restore" in TWRP to restore data and cache, this "should" give the same result as including system and boot in the TWRP backup along with data then restoring everything from that backup.
Also, can you post a copy of your recovery logs to some hosting site then link it here, someone here might be able to see something in the log that needs addressing, preferably the logs from the failed backup processes, if they errored in different ways, post the differences also. You should be able to find your logs if you haven't wiped the device.
Sent from my SM-S767VL using Tapatalk