As it seems to be a fairly common question, here is a guide on how to extract your clockworkmod recovery images.
This can come handy if you only want to extract certain apps, or files.
Put your Clockworkmod images in a folder on your computer.
Download this and put it in there, too.
Extract it in there.
Open a cmd-prompt in this folder.
Use this command to extract your data:
Code:
unyaffs.exe img_name_to_extract.img
Wait. the tool will display "end of image" when it is done.
Done. The data should be in the same directory as the tool is.
If you want to restore your contacts:
Extract the data.img
Your contacts will be in:
Code:
\data\com.android.providers.contacts\databases
I have not written the program used in this guide, just found it on my harddrive.
thanks much. so helpful
helped me to pull out contacts
I first had to download and install cgwin1.dll from here http://www.dll-files.com/dllindex/dll-files.shtml?cygwin1 and place it in the windows directory to get this to work. But after proceeding I get a "broken image file" when trying this on recovery.img and same thing with boot.img.
I am simply trying to recover some data from this nandroid backup specifically pictures in the sdcard/DCIM directory but I have no idea where it is. Is it located somewhere in the data.ext4.rar or system.ext4.rar files? If so I have looked through those and could not find a DCIM directory of any kind. Would anyone be kind enough to direct me?
Thanks
JDubbed said:
I first had to download and install cgwin1.dll from here http://www.dll-files.com/dllindex/dll-files.shtml?cygwin1 and place it in the windows directory to get this to work. But after proceeding I get a "broken image file" when trying this on recovery.img and same thing with boot.img.
I am simply trying to recover some data from this nandroid backup specifically pictures in the sdcard/DCIM directory but I have no idea where it is. Is it located somewhere in the data.ext4.rar or system.ext4.rar files? If so I have looked through those and could not find a DCIM directory of any kind. Would anyone be kind enough to direct me?
Thanks
Click to expand...
Click to collapse
You won't be able to do that. The recovery does not backup the sdcard.
Thank you very much ,it help me much
Cheers buddy! Needed this to extract my data backup
I am not sure if I post at the right place but I would like to share this to everyone as I understand how hard it is to find a solution that works. With this guide, you will get to keep the original WhatsApp message information (such as thumbnail, link, emoticon, etc.) and work like it is a restore file that backup from itself.
There will be three main phase you need to perform. You can get all required file in the attachment. Make sure you backup your end result before jump into the next phase.
Phase A: Decrypt and obtain msgstore.db file
This is the easiest method I can find and NO ROOT is required to perform this. All you need is to follow the instruction below.
INSTRUCTIONS:
Extract "WhatsAppKeyExtract.zip" (from attachment) on your computer maintaining the directory structure.
Browse to the extracted folder and click on "WhatsAppKeyExtract.bat".
Enable USB Debugging on your device.
Connect your device via USB**, unlock your screen and wait for "Full backup" to appear.
Leave the password field blank and tap on "Back up my data".
The "extracted" folder will now contain your "whatsapp.key", "msgstore.db" and "wa.db".
**If you have never used USB Debugging before, you may also need to verify the fingerprint.
(Credited to TripCode for providing this application! Here is the link to original thread.)
For my case, I have three backup files that named msgstore.db.crypt, msgstore.db.crypt7 and msgstore.db.crypt8. Although I know there might be other easier method to achieve this, I choose to restore each file (by uninstall and install WhatsApp) and run the above method to get my decrypted file. Hence I can be sure that all decrypted file will come in the same structure (I will show you what this mean later) and prevent any error that may cause by that.
Phase B: Edit and prepare database for merge
This will be a bit tricky. But in general, you MUST make sure that you have no overlapping message history between your files.
INSTRUCTION:
Download SQLite Database Browser and install on your computer.
Open the database you want to edit.
Click on [Browse Data] tab.
In Table drop down menu, click on [messages] (You will see all your WhatsApp message here).
Use it like you are using Excel, select any row that is overlapping (EXCEPT the row with many [-1] value) and click on [Delete Record], then click on [Write Changes] (this means you have save the change to the database).
Keep doing this for your remaining file and close all windows of SQLite Database Browser when you are done.
(Thanks ztedd for posting his own method where I get inspired from. This is the link to his original post.)
For my case, the number of column of all my file when display in "messages" table is different (newer WhatsApp database file have more data type). This is the reason I restore and run backup again to make sure all my file have the same structure. Also, a quick tips to figure out which row is overlapping or duplicate is export the messages table to CSV file and open it with Excel. You will need to copy the first few column of each database and paste them together, then use the feature that come with Excel such as Filter and Remove Duplicate to identify which row number you need to remove from actual database file.
IMPORTANT - Never edit your database on excel and import it back to your original database. This will remove all thumbnail, link, emoticon and etc. and show you some weird message when you restore back to your device.
Phase C: Merge database file
This method only can merge two file at once. You may need to do this several time depending how many database you want to merge. I suggest you start from the oldest to newest.
INSTRUCTION:
Rename the older file to old.db.
Rename the newer file to new.db
Create a new folder called Merge and copy the file old.db, new.db and merjapp.jar into it.
Run Command Prompt on your computer (or click on Start Menu and search for cmd)
In Command Prompt, type each of the following separately and press Enter:
Code:
cd desktop
cd merge
You now should see [C:\Users\<UserName>\Desktop\Merge>] in cmd window.
Now type the following and press Enter:
Code:
java.exe -jar merjapp.jar -old old.db -new new.db
You should see a "Done!" is displayed and a new file named output.db will be created in the Merge folder.
If you have other database to merge, repeat the steps above by rename the output.db to old.db.
Once you are done, rename the output.db to msgstore.db and it is ready to restore into WhatsApp.
(Credited to namnoops for this merge application! This is the link to original thread.)
For my case, I have to run this steps twice to merge 3 databases. I get error the first time I run this because some messages is overlapping and this is the reason for Phase B to edit my database. Once I got my merged database, I renamed it to msgstore.db (without encrypt again) and have no problem restore back to my device. I also copy and paste all my backed up WhatsApp folder to my device (except the file in Database folder) for the picture in WhatsApp to work.
Disclaimer: I do not know any coding behind all these so if you have any questions I will try to answer but the best bet is to go back to the original thread.
Below is some other method that I found online but can't figure out how to do it. You may try it at your own risk.
Additional Method:
sch3m4's wforensic Method
Alternate sch3m4's wforensic Method
Reserve for later use.
Reserve for later use, again.
How could i restore the merged database file ?
Xperia000sola said:
How could i restore the merged database file ?
Click to expand...
Click to collapse
I think you should overwrite new merged DB uninstall whatsapp then reinstall it
Thank you! I' ve tried the method in the thread of the merge tool, but your procedure did the trick for me =)
i cannot use it
help us with video please
big fail after attemped retrieve msgstore.db
when tested edit under sqlitebrowser, save change then exit. copy backup the "msgstore.db" file to my device original place and set permission as before. run whatsapp then all my message on list has been gone. no idea wut happen. even though i copy back my original msgstore.db to same place. so **** and make me down becoz i forgot to backup the list b4 make this test.
i use whatsapp ver 2.12.367. and dun add or del row under sqlitebrowser, only retrieve some text on same row, but why i copy back the original msgstore.db also fail? lost all message
While it might not be a lot of fun, it's quite easy to merge those databases using this method
I have used the Android WhatCrypt-app to decrypt my databases on the smartphone, copied both unencrypted ones to my PC and then trimmed the newer one down.
After merging, the only problem was getting Whatsapp to accept my database - the first attempt resulted in Whatsapp deleting all my messages.
My solution was to "Clear Data" on Whatsapp and simply drop the newly created database in Whatsapp's folder inside /data.
It accepted the new file and created the missing files by itself.
Now I have finally got all those messages in my Whatsapp
Thank you very much!
Hey,
I have a problem : everytime I try to do the last step e.g. merge the old.db and new.db via cmd - i get the error message " -E- Database old.db cannot be found! " . I renamed the old database to "old.db" still it doesnt work? What I am doing wrong?
Is my old.db file corrupted? I can read it with Whatsapp Viewer tool.. So it seems to work.
And there shouldnt be any overlapping messages - I have an old database and a fresh new one without any old backups.
(my old phone broke and i just jused the new one for the incoming messages)
Help
Hi,
I just wanted to ask, I have the msgsstore.db.crypt8 File from my old whatsapp back on older device but I am unable to copy it to the database folder in my new device.
Help Please
Guys I have 2 backups, 1 crypt9 and 1 crypt12. I decrypted and merged both succesfully but when I try to restore my messages through Whatsapp it tells me "Cannor restore your messagges......." . What can be the problem? Whatsappviewer open without problems outout.db, so I think it is not corrupted. There may be some overlapping msgs? How can I notice them? I whatched the table column received_timestamp and I didn't see any equal row. However the column timestamp has multiple equal column, can be this my problem?
Thank in advance
Now I'm on phase B, and with the DB Browser for SQLite, I opened (tab Browse Data) the msgstore file in the "extracted" folder and then selected messages In Table drop down menu.
I didn't not understand what to do next.
DLz88 said:
Now I'm on phase B, and with the DB Browser for SQLite, I opened (tab Browse Data) the msgstore file in the "extracted" folder and then selected messages In Table drop down menu.
I didn't not understand what to do next.
Click to expand...
Click to collapse
You would need to delete the records you do not want and then save the changes. In any case, what is the end result you are trying to achieve? Do you wish to import the merged database back to WhatsApp? Are the databases from separate accounts? If you intend to write back msgstore.db to /data/data/com.whatsapp/databases? How are you going to do that without root? If you do intend to use your merged database with WhatsApp then you may run into difficulties. Things have changed a lot since this guide was written.
I have previous crypted (crypt8) backup files from whatsapp database (same account) and I want to merge those old messages with others whatsapp messages I have on my LG G5. Basically my galaxy s4 died, I managed to rescue the whatsapp database and now I want those messages in my new LG G5 (no backup on whatsapp servers because it was setted off by mistake). The folder is inside the main storage/WhatsApp/Database Also, I want all the records and excel told me that there are not any duplicates values.
Don't work on Nexus 5X, show message like strong AES encryption. :'(
Guys, I have FINALLY found a way to merge two differten WhatsApp backup files. I'll post the guide asap (have to translate it). Is different from the one on the first page for the first two parts.
How to merge two different WhatsApp backup files:
We have two different WhatsApp backup file (doesn't matter if encrypted by different methods, for example, I had a crypt9 file and one crypt8) and we want to combine them into one so we can read them with WhatsApp.
Requirements: OS: Windows Vista, Windows 7 or Windows 8
Programs you need:
1) WhatsApp Xtract: http://forum.xda-developers.com/showthread.php?t=2770982
2) SQLite Database Browser: https://github.com/sqlitebrowser/sql....2.0-win32.exe
3) merjapp (merjapp.jar) https://github.com/namnoops/merjapp/releases
4) JDK http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
5) Java: https://www.java.com/en/download/
6) ADB (Android Debug Bridge) Drivers - If not installed: http://developer.android.com/sdk/index.html
Enable USB Debugg: Settings \ Developer options \ USB debugging (to activate them go to Settings-> About phone-> Software Information-> click many times on the Build number).
Before proceeding further, I suggest to temporarily remove all security protocols such as pin, lock screen with a sequence or other method and scanning fingerprints. In order to avoid the forced backup password.
I also suggest to make the extension of the files, visible (on windows, go to Tools-> Folder Options-> View-> uncheck "Hide extensions for known files").
First phase: decrypt
Extract the "WhatsAppKeyExtract.zip" program in a PC folder, making sure to maintain the folder structure.
a) Start "WhatsAppKeyExtract.bat"
b) Once started, connect your phone via USB, unlock the screen.
c) Wait until the screen that tells you to back up your data appears.
d) Leave the password field blank and click "Back up my data".
e) When finished, inside the "extracted" folder (located inside the WhatsAppKeyExtractor folder) you will find the files "whatsapp.key", "msgstore.db" and "wa.db". (That's if the latest versions of whatsapp).
At this point I recommend to save these files in a folder, calling it "new backup."
Now we must do the same process with the old backup, then:
Uninstall Whatsapp.
Remove all backups from the folder Whatsapp\databases (inside the phone).
Put back in the old backup file ( "msgstore").
Reinstall whatsapp, and let it reopen old messages (restore backup). When done, remove the connection data from the phone (both wifi, and UMTS).
Redo the same procedure as before, from point a) to e).
Save these new files "msgstore.db" and "wa.db" in a folder, this time calling it "old backup."
The first part is finished.
Second phase: organizing the chat
Before you start, create backup copies of previously decrypted file (you never know).
Open SQLiteBrowsers, position yourself in BrowseData tab and select Open Database. Choose the old backup file msgstore.db (that previously we have decrypted).
Click the lower Tab, select "messages".
With the scroll menu, go down to the last message. Here, count the "_id" number of the last message, for example "65422"
Open the new backup file (the new msgstore.db we had decrypted) always with the SQLite Database Browser program. By the same process: open the inferior tab and select messages. Highlight entirely the first line that has the fields, for the most part, containing the value "-1", and delete the line with the Delete Record command.
Make a note now of the number of the first message, under "_id" for example "12"
Now go in the tab "Execute SQL" and write the following string: UPDATE messages Set_id = (65422 + 1 + _id - 12) (put the numbers that you had marked earlier, in this case 65422 and 12). Click ExecuteSQL ( "Play" button)
Close SQLiteBrowers.
End of the second phase.
Third phase: merge
Create a new folder on your desktop called "merge".
Rename the old backup file (the msgstore.db file that is inside the "old backup" folder) that we had decrypted, in old (so we have the file old.db).
Rename the new backup file (the msgstore.db file that is inside the "new backup" folder) that we had decrypted and on which we had executed the SQLiteBrowser string, in new (so we have the file new.db).
Copy and paste the two files in the directory "merge".
Copy merjapp (merjapp.jar) in the "merge" folder.
Now run the command prompt as an administrator.
Enter the following commands:
cd C: \ Users \ [username] (press "ENTER")
cd desktop (press "ENTER")
cd merge (press "ENTER")
java.exe -jar merjapp.jar-old old.db -new new.db (press "ENTER")
It must appear "Done!"
At this point in the "merge" folder there must be a file called output (output.db file)
Copy it in desktop and rename it msgstore (so we have the msgstore.db file).
Open the phone's internal memory, go into WhatsApp \ Database folder, and delete all the files that are in there.
Copy "msgstore" (the msgstore.db file that was named output) in the folder Whatsapp \ Database.
Uninstall and reinstall WhatsApp (obviously choosing to restore backups).
END
DLz88 said:
Guys, I have FINALLY found a way to merge two differten WhatsApp backup files. I'll post the guide asap (have to translate it). Is different from the one on the first page for the first two parts.
Click to expand...
Click to collapse
Many thanks for finding the time to post your updated working method. I'm sure your contribution will come in handy for many users.
Hi. With this tutorial can I merge 2 backups from different numbers? Basically I would like to move the conversations from my second number to the first number I have.
Thanks.
foxmolder1985 said:
Hi. With this tutorial can I merge 2 backups from different numbers? Basically I would like to move the conversations from my second number to the first number I have.
Thanks.
Click to expand...
Click to collapse
I don't know if my guide can works with backups from two different numbers. You can always try it and let us know
So yesterday My phone, Sony Xperia S, decided to go into bootloop and not boot at all, no matter what I did. Tried flashing stock PRF, stock firmware, some stock based firmware, but it would get stuck in bootloop. The reason for this being I was not formatting the data partition, which I needed to preserve since it had all My sms, contacts, call logs and other app data. Luckily, I had a working recovery(TWRP) and was able to take a backup. Then using I was able to recover My contacts, sms, call logs and app data. After a lot of Google searches, I was finally able to work it out. The process is as follows:
Needed software:
Code:
yaffs-mmssmsdb-calls-extractor.zip from [URL="http://forum.xda-developers.com/showthread.php?t=1585957"]here[/URL].
Firefox Browser
Firefox addon of SQLite manager : [URL="https://addons.mozilla.org/en-US/firefox/addon/sqlite-manager"]link[/URL].
Excel macro to convert .db contacts to .csv : [URL="http://thydzik.com/downloads/convert-contacts2.db-data-to-vcard-vcf.xlsm"]link[/URL].
Software for csv to vcard conversion (Windows) : [URL="http://www.csvtovcard.com/"]link[/URL].
CSV to vcard conversion (online) : [URL="http://labs.brotherli.ch/vcfconvert/"]link[/URL].
SMS Backup & Restore android app : [URL="https://play.google.com/store/apps/details?id=com.riteshsahu.SMSBackupRestore"]link[/URL].
Call Log Backup and Restore app : [URL="https://play.google.com/store/apps/details?id=com.riteshsahu.CallLogBackupRestore"]link[/URL].
The procedure is a lot crude as of yet, I would update as more universal and convenient options are found.
I know there is Titanium backup app which can restore contact, sms etc from nandroid backups, but You need a paid version of the app.
First we need to extract the data partition of nandroid
Rename the data.ext4.win to data.ext4.tar.gz . If Your files are like data.ext4.win000, data.ext4.win001, then rename each of them to data.ext4.tar.gz in separate folders.
On windows, now You can directly extract them using WinRaR or 7zip application, although You don’t need to extract it entirely.
On linux, You can do so using the following command. It is recommended to store the file in a separate folder as the file gets extracted directly.
Code:
tar –xf data.ext4.tar.gz
Now we need to find two files contacts2.db and mmssms.db . The contacts2.db files is for contacts as well as the call logs and the mmssms.db is for messages. The contacts2.db is constant across most ROM’s and OEM firmware apps. The mmssms.db file may vary as per app and ROM’s. So what You need to do is look for a .db file inside the SMS app You have been using with the firmware/ROM.
We will restore the SMS and Call logs first.
Currently it is limited to Windows OS only.
Extract the contents of yaffs-mmssmsdb-calls-extractor.zip to a folder.
Copy the mmssms.db file and contacts2.db file to the above folder.
Now open a command prompt at location of the above folder. This can be done by opening the folder, right clicking using shift key pressed in blank space. You get the “Open command window here” option.
In it type:
Code:
extract -s mmssms.db
It will ask You to enter a name for the output .xml file, You can press enter to let it give a default name. This will convert Your sms to a .xml file .
Then type:
Code:
extract –c contacts2.db
It will also ask You to enter a name for the output .xml file, You can press enter to let it give a default name. This will convert Your call logs to a .xml file .
Copy both of these files to Your newly formatted phone and then use the two before mentioned apps (Call Log backup& Restore and SMS Backup & Restore) to restore Your SMS and call logs.
Now retrieving contacts. It is a bit lengthy, but simple process.
Start firefox browser. In it, after You have installed the addon do following to enable the addon
Go to options (3 horizontal lines).
Select “Customize” and drag the “SQLite Manager” to the blue bordered box.
Now open options and select SQLite Manager. In it select Database>Connect Database. In the opened dialog box, select All Files(*.*). Then select Your contacts2.db .
From the left side tree structure in the SQLite Manager window, right click on “data” and select “Export Table”.
Now in right side part of the window, You will get CSV tab and SQL tab. Under CSV tab, simply press OK button without changing any other options.
It will ask You to save Your .csv file. Save it.
Now if You try to open this file directly, You will notice that file is not properly arranged.
To arrange it properly, open the convert-contacts2.db-data-to-vcard-vcf.xlsm file. If You don’t know how to enable Macros don the following:
It will show You a warning “Security Warning Macros have been disabled. Options..”
Press the Options button. Select “Enable this content” and press ok.
Now Press the office button present at left top corner. In it select “Excel Options”.
Select Customize tab from left side. Then from “Choose commands from:” drop down menu, select Macros.
Now add convertDatatoContacts and writeContactstovCard to right pane from left pane by selecting them and pressing add. Press OK button.
Now as You can see two white square icons have been added at top right corner next to save undo redo options.
Select the first white square icon. Select “Enable Macros” button. Now select Your .csv file in the opened dialog box.
Now the file must have opened in a separate excel. The format should be 4 rows with they being First Name, Last Name, Phone 1 and Phone 2.
Then select Save As>Excel Workbook>Save Type as CSV(Comma delimited).csv . Press ok and yes on the two warnings generated.
At the left bottom, right click on “contacts” sheet and press delete. Save and close the file.
Run CSV to vCard software and open the above saved .csv file. Here You can save all contacts as single vCard file or separate files.
In the “Properties” section on the window, in front of “Phone 1” and “Phone 2” select “Mobile Phone” from options. Press “Convert” button.
Alternative to above two steps, You can also convert .csv to vcard using the website.
Now a .vcf file will be created where the .csv file was stored.
Now simply copy the .vcf file to phone and import using the phone contacts app!
Done!
Credits and sources:
@abbot2
[TOOL] yaffs extractor, mmssms.db & contacts2.db converter
http://thydzik.com/export-android-contacts-contacts2-db-to-vcard-vcf-on-windows/
Reserved.
Awesome concept
Thanks for posting this! Im working on a sprint S4 that is bootlooping. ive extracted all data and am attempting to recover contacts with your method. I am getting an error with the yaffs extract.exe that im hoping you could give me some direction on.
**EDIT**
I was never successful using yaffs (dependency issue perhaps) but i downloaded firefox and was able to extract everything with the sqlite extension. THANK YOU!!!! This tool is one of those things you dont need often, but when you do it's a godsend!
when i issue the contacts extraction command i get this output
C:\Users\PartSwap\Desktop\DB_xtract>extract -c contacts2.db
Read 0 calls
Failed to extract calls: list index out of range
contacts2.db is present in the yaffs folder.
Any ideas/suggestions would be greatly appreciated!
This doesn't work anymore, just puts 0 in all of the phone number fields, you can even see the numbers before hand, just jumbled up, so this script needs to be updated.
Stillworks July 2020
This worked in July 2020 to restore SMS messages (I didn't need contacts). I tried to restore selectively from a full OrangeFox (OF) NANDroid backup (a full restore didn't work, encryption/password issues) with Titanium Backup (TB), but this didn't work (maybe I simply didn't know what to restore?). I tried the old Nandroid Manager's facility to restore SMS from a NANDdroid backup, but it hung.
I then successfully extracted mmssms.db as described here and further extracted the xml file with the yaff extractor. TB's "Restore data from XML" hung, but SMS Backup & Restore worked fine.
Xiaomi Redmi Note 7 (lavender), all software current as of July 2020: xiaomi.eu 20-07-23 weekly ROM, TB 8.4.0.2, others downloaded today. A useful way to find the SMS file if it has a different name than mmssms.db: after extracting from the tar.gz, search the extracted files for a string that only appears in your SMS (I used "bekonscot").
Incidentally, a note on Titanium Backup, which can restore either from its own files or a NANDroid backup: the directory for the NANDroid file is hard-wired to the directories used by TWRP and CWM. I use the current OrangeFox instead of TWRP, and it uses a different directory. But the OF backup is compatible with TWRP; moving it to /storage/emulated/0/TWRP/BACKUPS/ works.
HTH
I used a Note 4 (Android 6) full nandroid to Note9 (Android 10). For contacts I just copied the .db to the new location (the directories where .db are located might not be the same) and rebooted - it worked (on rooted), but this will remove any contacts already made, replacing them with the backup.
SMS and Call logs were restored as described, but I used a newer version of the app (which combines both SMS and logs restoration), searched the backup archives with ZArchiver, and I ran the python2 version of the converter (https://github.com/abbot/android-restore-tools) on TermuxArch (meaning that, unlike the Windows version in the tutorial, this should work on any OS supporting python2, allowing me to do all steps on my phone).
Also, in the directories where. db files are located there are big .db-wal files, too, which might contain some data that aren't in the .db yet, so you might need to VACUUM with sqlite3 (https://stackoverflow.com/questions...f-sqlite-3-7-wal-file-into-main-database-file). I did it on TermuxArch as well, but the size of the db didn't change much, while the wal disappeared, so I don't know if that did anything.
P.S. Sqlite3 wouldn't open db at all just on Termux for some reason, opened on TermuxArch. Python2 also exists for Termux, didn't try but should work.