I need to know if it's possible to pull old text messages off of a backup from an old phone. Before selling my HTC One S, I made an entire dump of it onto my computer.
Now I REALLY need some texts that might have been on the phone for a legal case I'm involved in. This is REALLY, REALLY, REALLY important. I don't know if it's possible to pull texts out of a backup made on a PC or not, but I REALLY need to see what texts are on there if nothing else at all.
I also used a couple different backup methods on the phone such as nandroid, titanium, Backup Everything, etc and would like to be able to go through those as well. Is there a way to emulate the phone so I can restore these backups? Is there another way to go through the data in them without having the phone?
ANY ANY ANY and ALL help greatly appreciated! Thanks!!
Install SMS backup from market
Sent from my HTC One S using xda app-developers app
flex420 said:
Install SMS backup from market
Sent from my HTC One S using xda app-developers app
Click to expand...
Click to collapse
With all due respect, did you read my post? lol
I don't have the phone anymore. All I have is a file structure dump I did onto my computer before I sold it.
Yes, you can do it. You need a linux host (or a mac) with sqlite3 installed.
If you have a nandroid, then you have a tar file of your data partition. You have to untar it somewhere.
If you have a raw image, you have to mount it as an ext4. (In that case, a mac won't do, and you need a linux box.)
either:
tar -xf data.tar
or:
mkdir data
mount -t ext4 -o loop data.img data
Your precious data is in the following file:
data/data/com.android.providers.telephony/databases/mmssms.db
You can:
cd data/data/com.android.providers.telephony/databases/
sqlite3 mmssms.db "select * from sms;"
Edit to add:
I'm sure you can extract the file in question from a .tar on windows, and I bet there are also tools to extract it from an ext4 image, but I can't point you at any.
Also, I doubt that texts you pull from a dump are good for legal use. You may have to get the data from your network provider.
touch of jobo said:
Yes, you can do it. You need a linux host (or a mac) with sqlite3 installed.
If you have a nandroid, then you have a tar file of your data partition. You have to untar it somewhere.
If you have a raw image, you have to mount it as an ext4. (In that case, a mac won't do, and you need a linux box.)
either:
tar -xf data.tar
or:
mkdir data
mount -t ext4 -o loop data.img data
Your precious data is in the following file:
data/data/com.android.providers.telephony/databases/mmssms.db
You can:
cd data/data/com.android.providers.telephony/databases/
sqlite3 mmssms.db "select * from sms;"
Edit to add:
I'm sure you can extract the file in question from a .tar on windows, and I bet there are also tools to extract it from an ext4 image, but I can't point you at any.
Also, I doubt that texts you pull from a dump are good for legal use. You may have to get the data from your network provider.
Click to expand...
Click to collapse
Awesome. You rule.
I actually have 3 backups, all 3 of which have a file in them called "data.ext4.tar", and I can use my beloved Backtrack 5 machine (it's just so HANDY! lol) to do this. No problem.
So, I'd just untar it (this would give me an ext4 file system, right?) then mount the resulting ext4 file system and go from there?
I'm trying to get the texts from this AND from the provider.
rmp5s said:
So, I'd just untar it (this would give me an ext4 file system, right?) then mount the resulting ext4 file system and go from there?
Click to expand...
Click to collapse
Untarring the .tar file will give you a folder structure. You dont need to mount anything then. Just navigate to that mmssms.db file and query it with sqlite.
-Jobo
Thanks for the help!
Sent from my Nexus 4 using xda app-developers app
Related
for those who do not know how to do this yet, follow directions given here.
Just took this info from the web. None of this except for the guide is written by me.
I am not responsible for anything please do not contact me if a green sludge starts attacking your neighborhood eating one person at a time.
************************************************************
************************************************************
************************************************************
http://code.google.com/p/unyaffs/downloads/list
made for linux, but if you know a bit of c, you can probably change it to work with windows, maybe I will do that if I have some time this week.
********OR********
http://jiggawatt.org/badc0de/android/unyaffs-x86-win.zip
or http://www.4shared.com/get/fgryPnPx/unyaffs-windows.html
for windows..... instructions below apply for both
if not included in the zips above, cygwin1.dll and unyaffs.exe are included as an attachment of this reply.
open up a command line, in windows vista/7 just press start and then type cmd press enter and change the directory to the folder with the unyaffs and the img in it. in xp press start>run>cmd... then the same
the usage is: unyaffs /path/to/img
example:
(if unyaffs.exe and system.img are both in c:\temp)
"c:\temp> unyaffs /temp/system.img"
note: the '/' is the way linux determines the pathname of a file and since it is built using cygwin(a file allowing for windows) it is preferrable that the path to the actual file is in unix POSIX format ie: / instead of \ between folders and files.
*********WARNING************
this will extract to directory that unyaffs binary is in.
That's so weird -- I was gonna post the same thing tonight. Thx for writing this up. It's odd that more people don't talk or ask about this tool!
I remember reading this stuff back when I got my G1, and tonight I was trolling the themes and apps section cause I couldnt sleep. (which I rarely look at)
I was baffled that this wasnt common knowledge. I figured if there was no threads about it that people just knew how and didnt have any problems with it. My mistake lol...
enjoy.
can you please explain what is it? what it does? thanks
ericizzy1 said:
can you please explain what is it? what it does? thanks
Click to expand...
Click to collapse
Umm, it expands nandroid img files not sure how else to explain it. Clockworkmod backups are in the img file format. This process allows you to extract those files to get stuff out of them
Sent from my SPH-D700 using XDA App
When you create a backup in Clockwork recovery, it leaves you with a folder like this:
2010-11-18.08.52.49
As you can prob figure out, I made that backup on 11/18 at 8:52am
Inside that folder are .img files, such as: data.img and system.img
Those files basically contain your data (like apps you've installed from market) and system (..system apps that might've come with the phone).
In the orig post, he's explaining how to extract the files from these large .img files.
To give an example, I had a couple of corrupted clockwork backups, and wanted to ONLY pull out some database files (like text messages and call logs) which were not corrupted. Even tho the backup was messed up, I could still extract the files I needed by using the method above, then copied them manually to my phone.
decalex said:
When you create a backup in Clockwork recovery, it leaves you with a folder like this:
2010-11-18.08.52.49
As you can prob figure out, I made that backup on 11/18 at 8:52am
Inside that folder are .img files, such as: data.img and system.img
Those files basically contain your data (like apps you've installed from market) and system (..system apps that might've come with the phone).
In the orig post, he's explaining how to extract the files from these large .img files.
To give an example, I had a couple of corrupted clockwork backups, and wanted to ONLY pull out some database files (like text messages and call logs) which were not corrupted. Even tho the backup was messed up, I could still extract the files I needed by using the method above, then copied them manually to my phone.
Click to expand...
Click to collapse
thx
Sent from my SPH-D700 using XDA App
smeyerhuky said:
Umm, it expands nandroid img files not sure how else to explain it. Clockworkmod backups are in the img file format. This process allows you to extract those files to get stuff out of them
Sent from my SPH-D700 using XDA App
Click to expand...
Click to collapse
Right, like if you wanted a file or a few files from clockworkmod img backup, then this process would give you an extended file system, other than what you would normally use to extract the img files. Just trying to understand this so, please correct me if im wrong.... I've been trying to get more knowledge of this, so im still learning.
I prolly sound like a complete idiot, lol...
Sweet! Thanks for this man. I'd been looking for how to do this for some time and kept ending up learning about emulation or some other tangential topic. This is the basic functionality I was looking for. Thanks for posting!
Np. I'm working on writing some stuff on top of the source code that should allow you to browse and pull just one our more file
Its been a but since I looked theory source code so it might take a while... Maybe in between semesters?
Sent from my SPH-D700 using XDA App
Dode you saved my as** thx big thx
Great, now how can I re-compress back to .img after I've made the changes that I want?
EDIT: Nevermind. I just made a zip of the new system folder and flashed it in CWM.
This worked great for me.
Thanks,
Bud
Man, I do not know how to thank you. I was able to recover some important stuff off of my cwm recovery backup. THANKS!!
Can Someone adb Pull The Triumph System Files And Send Them To Me? I Will IM My Email Address To You If Neccesary.
barnito at androidforums posted a stock 039 nandroid backup
http://androidforums.com/3036500-post178.html
it has system.img in there
Hello
I am here too, just can't post links yet, I backed up my Triumph without powering it on....
How do i extract the files from the system.img
souder14 said:
How do i extract the files from the system.img
Click to expand...
Click to collapse
http://forum.xda-developers.com/showthread.php?t=961164
zurui said:
http://forum.xda-developers.com/showthread.php?t=961164
Click to expand...
Click to collapse
The system.img extraction keeps freezing and stopping could someone just adb pull their system folder and zip it then send it to me?
souder14 said:
The system.img extraction keeps freezing and stopping could someone just adb pull their system folder and zip it then send it to me?
Click to expand...
Click to collapse
I've got dd dumps of the entire MT "disk" and individual dd dumps of each partition. I also have tars of each file system.
I took the dd "images" prior to the first boot into the OS, by holding volume-down+power. That put me into the strange read-write boot loader mode. Then I connected the phone via USB to a linux box. Unfortunately, Ubuntu auto-mounted the phone "disk" and all the file systems which no doubt updated some file system timestamps, but I didn't change anything. I unmounted the file systems, took my dd images then mounted them again and tar'd them up. Then unmounted and rebooted the phone and activated it.
Unfortunately, I doubt I have enough karma to upload yet. Perhaps I can put them somewhere for you to download directly? PM me for the location. Sorry offer open to souder14 only, or someone else with enough karma to post them to xda.
Did you find an upload location? If so please share it.
Thanks
srmuc69
I read a thread and either I didnt understand or it didnt work as I understood it. I want to be able to make a system dump of anything that comes out in case I want to import apps or things. Do you always have to use ADB or terminal to extract files from a .tar or .md5 file? So you always have to extract to your sdcard from what I read? Just a little confirmation on if thats how I need to do it or the correct way please.
Root explorer? Don't know much just a thought
Sent from my SCH-I510 using xda premium
blueis300 said:
I read a thread and either I didnt understand or it didnt work as I understood it. I want to be able to make a system dump of anything that comes out in case I want to import apps or things. Do you always have to use ADB or terminal to extract files from a .tar or .md5 file? So you always have to extract to your sdcard from what I read? Just a little confirmation on if thats how I need to do it or the correct way please.
Click to expand...
Click to collapse
you have to explain better what you want to do...you can extract/make roms from odin files without having to flash it...you just use dsxidia's kitchen (requires linux or cygwin with windows) and run the odin through it
to *make* an odin of your current set up for backup, imnuts has a guide in development somewhere, but you would need linux to make the tar (you can extract with adb in windows though)
or to just make a system pull
if adb is universal
cd "whatever folder u want it in"
adb pull /system/
and let it do its thing
to then make a flashable zip from it you would have to have a working update-script and proper file structure within the zip
nitsuj17 said:
you have to explain better what you want to do...you can extract/make roms from odin files without having to flash it...you just use dsxidia's kitchen (requires linux or cygwin with windows) and run the odin through it
to *make* an odin of your current set up for backup, imnuts has a guide in development somewhere, but you would need linux to make the tar (you can extract with adb in windows though)
or to just make a system pull
if adb is universal
cd "whatever folder u want it in"
adb pull /system/
and let it do its thing
to then make a flashable zip from it you would have to have a working update-script and proper file structure within the zip
Click to expand...
Click to collapse
Do you always have to use ADB or terminal to extract files from a .tar or .md5 file? So you always have to extract to your sdcard from what I read?
Yes I was basically asking how to extract all the files from a .tar or .md5 file. I want to be able to look at each separate main file and pull whatever I want from there. So one option is to use ADB and pull to sd card. I don't use linux so I guess this is my only option?
dsxidia's kitchen (requires linux or cygwin with windows)
blueis300 said:
Do you always have to use ADB or terminal to extract files from a .tar or .md5 file? So you always have to extract to your sdcard from what I read?
Yes I was basically asking how to extract all the files from a .tar or .md5 file. I want to be able to look at each separate main file and pull whatever I want from there. So one option is to use ADB and pull to sd card. I don't use linux so I guess this is my only option?
dsxidia's kitchen (requires linux or cygwin with windows)
Click to expand...
Click to collapse
the only 2 ways i can think of atm to extract from a tar are this:
push the tar with odin/heimdall to ur phone then pull /system with adb or with terminal emulator to ur sdcard
or use dsxidia's kitchen to extract it on the computer w/o flashing it
(well u can take apart the tar in linux other ways too, but it ll do it in a similar way to the kitchen and thats easier)
Yeah thats what I was thinking. Thanks.
Hello! Before posting, i search this forum but not found nothing understandable. I wont to modify .ftf file of sony xperia e single sim, edit some apps and repack in fleshable ftf again, but i have some problems. Ok i'm doo next:
1. Extract flashtool via winrar.
2. Copy system.sin to flashtool directory and extract with sin editor, and get 2 files system.ext4 and system.partinfo, but with flashtool only possible to extract yaffs2 files. 3. Here is big problem, i tryed menny softwares for opening ext4 images in windows, but only ext2 exploer can modify this files, but totaly inaccesible with screen readers like as jaws.
4. I found some tutoryals for mounting in linux but i can not understand linux commands as well, for example did i must know exactly syze of system partition to mount ext4 image, maby exist some gui program for linux from wich can mount end edit ext4 images?
Finally, does possible making flashable img or ftf file for sonny phones for custom roms? For example, easyer to me is flashing custom roms via fastboot, because in some situations for cm recovery i need sited assistance.
Thanks, and best regards.
iksi said:
Hello! Before posting, i search this forum but not found nothing understandable. I wont to modify .ftf file of sony xperia e single sim, edit some apps and repack in fleshable ftf again, but i have some problems. Ok i'm doo next:
1. Extract flashtool via winrar.
2. Copy system.sin to flashtool directory and extract with sin editor, and get 2 files system.ext4 and system.partinfo, but with flashtool only possible to extract yaffs2 files. 3. Here is big problem, i tryed menny softwares for opening ext4 images in windows, but only ext2 exploer can modify this files, but totaly inaccesible with screen readers like as jaws.
4. I found some tutoryals for mounting in linux but i can not understand linux commands as well, for example did i must know exactly syze of system partition to mount ext4 image, maby exist some gui program for linux from wich can mount end edit ext4 images?
Finally, does possible making flashable img or ftf file for sonny phones for custom roms? For example, easyer to me is flashing custom roms via fastboot, because in some situations for cm recovery i need sited assistance.
Thanks, and best regards.
Click to expand...
Click to collapse
I have looked around to repack an extacted system.sin but never found how to do it. To dump the ext4 img just use ext2 explorer after it has finished extracting & loading click save in the toolbar and choose the folder where you want to save it.
gavster26 said:
I have looked around to repack an extacted system.sin but never found how to do it. To dump the ext4 img just use ext2 explorer after it has finished extracting & loading click save in the toolbar and choose the folder where you want to save it.
Click to expand...
Click to collapse
hello! I can't found ext2 explorer, i only found ext2explore, but this software is totaly unusable for me, don't works with screen readers.
P. s. whats your problem when you try making system.sin?
iksi said:
hello! I can't found ext2 explorer, i only found ext2explore, but this software is totaly unusable for me, don't works with screen readers.
P. s. whats your problem when you try making system.sin?
Click to expand...
Click to collapse
There's no tutorial on how to build one or a tool. Not for windows anyway.
NOTE THIS IS NOT "UNABLE TO BACK UP" THIS IS "UNABLE TO RESTORE"
Lots of people on this forum, and lots of people on the internet have errors trying to back up because of this dreaded error 255. But, for me, the backup was fine, but now I can't restore. This is a huge problem because I just tried out a custom ROM (which turned out to be a huge fail), and I want to go back to stock.
So I switched back to the other slot (A/B ftw), wiped all data, formatted SD card, set up the OS to have the same lock pattern as the backup, transferred the backup onto the phone and tried to restore, and, well, fuck.
Code:
[RESTORE STARTED]
Restore folder: '/data/media/0/TWRP/2020-11-08--22-06-46 PRE-RR'
Skipping Digest check based on user setting.
Calculating restore details...
Restoring 4 partitions...
Total restore size is 11276MB
Restoring Vendor Image...
[Vendor Image done (3 seconds)]
Restoring Boot...
[Boot done (0 seconds)]
Wiping Data (excl. storage)
Wiping data without wiping /data/media ...
Done.
Restoring Data (excl. storage)...
extractTarFork() process ended with ERROR: 255
It failed, and there is absolutely nothing I can do about it. TWRP is using buggy commands to extract the backup and I don't know what's triggering the bug.
As a result, right now all my data is in limbo. TWRP cannot restore this backup as-is. I can't just go back in time and use `pm remove-user 999` or `pm remove-user 10` because the backup has already been created. Any solution that has to be done before the backup phase is inaccessible to me now.
My hope is that some smart person here can help me find a way to get this backup back onto the phone. Right now the internet does not have a solution to this problem, no matter how many naive users spam discussion boards with "pm remove-user 999 totally worked". Serious answers only.
Relevant section in the recovery log:
Code:
==> extracting: //data/vendor_de/10/ (mode 40771, directory)
Cannot find key for 10
error looking up proper e4crypt policy for '//data/vendor_de/10/' - 1DE10
tar_extract_file(): failed to extract //data/vendor_de/10/ !!!
I:Unable to extract tar archive '/data/media/0/TWRP/2020-11-08--22-06-46 PRE-RR/data.ext4.win000'
Error during restore process.
I:Error extracting '/data/media/0/TWRP/2020-11-08--22-06-46 PRE-RR/data.ext4.win000' in thread ID 0
I:Error extracting split archive.
Error during restore process.
pigz: write error code 32
pigz: abort: write error on <stdout>
extractTarFork() process ended with ERROR: 255
I suppose the solution is to make it "find a key for 10", but how? Do I just need to somehow recreate the 10 user before restoring the backup?
EDIT NOTICE: THIS DIDN'T WORK, DON'T MAKE IT WORSE BY TRYING
ORIGINAL POST IS BELOW
Alright. Well, I found a solution. That was quick.
The error is triggered by the backup containing a user that isn't present during the restore, I think. One solution is (theoretically) to create the missing user... another solution is to remove the missing user from the backup.
Turns out, backups are not so hard to modify, even in their compressed form. However, it is a little bit unintuitive at first.
You can't use the `tar` command to modify the backup, at all. TWRP's tar doesn't even support the --delete flag, nor does the BSD tar command (what ships with macOS). Even after installing GNU tar, it still refuses to work on a compressed archive (probably for the better).
You either need a Windows system with WinRAR/7zip (allows opening/modifying compressed files without extracting them), or a macOS/Linux system with FUSE and archivemount. If you're on Windows, you will have to manually search every single folder for 10 or 999 folders and delete every single one. If you're on macOS/Linux, the process is a bit more technical but also a bit more automated. And in both cases you will have to do this many times, and it can easily take half an hour or more and many restore attempts.
First and foremost, you need the recovery log to find out where the process failed. Every time it fails you will need to go to Advanced > Copy Log, copy them to the SD card, then use the flipping horrible Android File Transfer program to transfer the recovery.log to your PC, then use the cat/tail command or a text editor of your choice to read the log file.
Somewhere around the end of that file you will find something that looks like this:
Code:
==> extracting: //data/misc_de/10/ (mode 41771, directory)
Cannot find key for 10
error looking up proper e4crypt policy for '//data/misc_de/10/' - 1DE10
tar_extract_file(): failed to extract //data/misc_de/10/ !!!
I:Unable to extract tar archive '/data/media/0/TWRP/2020-11-08--22-06-46 PRE-RR/data.ext4.win003'
This contains all the info you need: the archive where the tar command gave up, and the file it gave up on.
Destroy that file with extreme prejudice.
Make a mount point with `mkdir mnt`, and mount the archive with `sudo archivemount data.ext4.win003 mnt`. (Of course, substitute data.ext4.win003 with the culprit archive file.)
If you're like me and have this problem with 10 folders, or 999 folders, you can find every single one of its kind with `sudo find mnt -name 10`. You can then use `sudo rm -rf` to delete every one of them. If you're brave, try `sudo find data -name 10 -exec sudo rm -rf \{\} \;`. I'm not responsible for any data losses incurred by using powerful commands that don't prompt you for confirmation.
Notice how I'm repeatedly using `sudo` for this: FUSE is kind of quirky and so is the kernel about multi-user filesystems. If you don't use sudo to mount the archive, then the user won't have permission to represent files it doesn't own, even if root is the one asking. You'll get weird "No such file or directory" errors for files and directories that are contained in parent directories' file listings. If you don't use sudo for the find or rm commands, you won't have permission to look inside folders or delete folders. Really, do everything with sudo because it's the only way to do this correctly.
Once you're done scrubbing the problem files away (be careful, do not use a file explorer of any kind, especially macOS's, which generally craps .DS_Store files all over your filesystem), unmount and save the archive with `sudo umount mnt`.
The next part is very important because recompressing files is not instant.
You will need to run `watch -n 1 du -hs data.ext4.win003` and WAIT.
Watch the file size slowly grow; only continue once it has stayed the exact same for at least one minute. Yes, it can definitely stop for 30 seconds at a time and then resume. You will need to give it time to recompress. It's also probably normal for the recompressed file to be about half the size of the initial file; that's just an artifact of TWRP not using very good compression to begin with. archivemount will create a backup of the archive with .orig appended, so you can try again if your backup ends up getting corrupted. (I just created an entirely separate copy of the backup to use archivemount on)
Of course, if you're on Windows you can skip all this and simply delete the problem file from WinRAR/7zip, and it'll be gone instantly and you can just proceed to copy the archive over to your device and retry the backup. You will have to do all the searching manually though, or perhaps you could try your luck with WSL. (Does WSL support FUSE?)
Expect this process to fail multiple times, but all at different points; whenever it fails at a different file, you know it got past the one you just removed. all you need to do is repeat this process over and over, slowly fixing your backup until it restores properly. This is what needs to be done if you can't reproduce the event that created that user in the first place. I'm sure if the user is important, it'll get created again.
Also, looks like user 10 is what happens when Google Play updates Device Personalization Services. Is this the generic "updates for system apps" user or is it specifically for that app?
EDIT: But updating that app doesn't create user 10. Interesting...
EDIT 2: User 10 is the guest user!! All I needed to do was create user 10 and that allowed me to restore my backup!
My post above is still good to see what I tried at first but it actually bricked my device until I factory reset it again, so :/
LoganDark said:
Also, looks like user 10 is what happens when Google Play updates Device Personalization Services. Is this the generic "updates for system apps" user or is it specifically for that app?
EDIT: But updating that app doesn't create user 10. Interesting...
EDIT 2: User 10 is the guest user!! All I needed to do was create user 10 and that allowed me to restore my backup!
My post above is still good to see what I tried at first but it actually bricked my device until I factory reset it again, so :/
Click to expand...
Click to collapse
Glad you managed to get your device back!
I had the same problem on another phone. The reason I got the error was because I tried to restore a decrypted backup on an encrypted device. After choosing the option "Format" in TWRP and typing 'yes' to confirm the decryption process, I was able to restore my backup.
Im having the same error in my realme xt...
sir..please help me.. I have a twrp backup of system image vendor image data boot recovery persist but im unable to recover because it shows an "error extract tar fork process ended up with error 255".during backup i had a pincode for screenlock.
i also have a backup of my internal storage in twrp but im not able to restore because of this error
LoganDark said:
Also, looks like user 10 is what happens when Google Play updates Device Personalization Services. Is this the generic "updates for system apps" user or is it specifically for that app?
EDIT: But updating that app doesn't create user 10. Interesting...
EDIT 2: User 10 is the guest user!! All I needed to do was create user 10 and that allowed me to restore my backup!
My post above is still good to see what I tried at first but it actually bricked my device until I factory reset it again, so :/
Click to expand...
Click to collapse
Hi. Did you create user 10 in Source, or in destination?
How did you do that in source?
If in destination, then is it possible to do it via TWRP?
EDIT: disregard. Mine user in the error path is 0 :-( '//data/system_de/0/'
just use option format data when entering wipe selection
erosman23 said:
just use option format data when entering wipe selection
Click to expand...
Click to collapse
doesnt work
redmi k20 davinci