Related
the problem:
i receive daily around 50 sms's (very soon will be twice more), with just a line in a subject -so basically i need something to export the content from those XX sms's - into a singe txt file, or maybe a excel sheet (even better), not the copy/forward/paste way i do - takes some time :/
.. and the question is: did anyone saw/used something like that? cause the most programs i checked are for backup, and save all the data (tel, date...etc.) in separate files, biut i only need the contents of all sms's saved together into a single file.
prehaps i might need some sms-parser, but didnt saw a product like that.
..so, anyone can help or i'll have to do that copy/paste for for the rest of my life ?
thanks in advance.
I am using pim backup for my persoanl information backups specialy for the contacts. I have question I my PPC broken or stolen and I dont have another PPC then how do i view my contacts on my desktop PC.
solved. I was taking backup in binary mode thats why I wasnt able to import data in excel.
Thanks every one for reading.
azfar said:
solved. I was taking backup in binary mode thats why I wasnt able to import data in excel.
Thanks every one for reading.
Click to expand...
Click to collapse
Well,for the benefit of the community at large always give full details of the solution,what you did and how you did it,it helps many others to solve and benefit from it.This forum believes in "Help & Support"
Its very simple.
Just uncheck binary backup while taking backup then go to backup archive rename its extension from pib to zip, extract it with any zip software, rename the contacts file from .csc to .csv extention.
Open excel, import data and choose semicolumn as delimerter, finish.
YUP !
I know,its very simple,I know the workaround
Just wanted to point out to you that always explain how you solved your problem,as to help others,who are seeking for the same solution and might be wondering how to solve it.
when a new thread is started with a view to seek help,it must have ample info and workaround to finish it solving that problem,with keeping in mind that many cud benefit from it.
dear all!!!
really it's so hard to restore many pimbackup.pib files on a device...
i have been searching for an extracor, or kind of emulator that alow me to view the backuped file on my pc...
thankyou
best regards...
moiisse
If I remember correctly, you can rename the .PIB file as a .ZIP and then look at it with your "favourite archive tool".... Within the ZIP/PIB file there are loads of CSV format files for each of the messages/contacts/phone logs etc.
The only problem may/might/will be if you did the PIM backup in binary format; your best bet would be to post on the PIMbackup thread (here) and see if Dotfred or anyone else can help
Cheers,
Mark.
Dear Mark
I have tried this trick but the files I got were not .CSV one some other extension files. I am afraid if I am doing anything wrong.
The extension is not .CSV, but they are in CSV format. If you rename them to .CSV and open them with Microsoft Excel or Openoffice Calc (or whatever), they can be imported as semicolon delimited files (so strictly speaking, they're SSV - "Semicolon Separated Values" rather than CSV )
Hope that helps,
Mark.
That was a great tip!
Any idea how to import these into an Android Phone - just got an HTC Hero today. Took a backup of my existing contacts, sms etc from WM6.5 (HTC Touch HD) and could see the .pib (renamed as .zip) and individual files (which I renamed as .csv)..
Guess there might be a simple trick to restore them as well in HTC Hero?
I think I may have found something........
sam_htc_touch said:
Any idea how to import these into an Android Phone - just got an HTC Hero today. Took a backup of my existing contacts, sms etc from WM6.5 (HTC Touch HD) and could see the .pib (renamed as .zip) and individual files (which I renamed as .csv)..
Guess there might be a simple trick to restore them as well in HTC Hero?
Click to expand...
Click to collapse
I was about to say I have no idea and I don't really care (about Android phones that is, not your problem getting your messages onto your Hero!), but then.......
I found this - Sprite Software's FREE (at the moment) migration software - which looks like the answer to your prayers
I've seen a report from someone who used it - apparently, it's pretty slow, but as it's a "one-time" transfer operation.....who cares how long "free" takes!!!???
Let us know how you get on - and post something in the Android-related forums so that other users can benefit.....
Cheers,
Mark.
(At times like this, I wish I had a Paypal "donate" button.....this gem of wisdom is surely worth a buck/quid/euro or two ....Ho hum!!!)
Excellent! That seems to be promising!
Just tried using that and tried to restore only the sms on my android, however it gave an error. Have emailed sprite software the relevant log file, lets see if they come up with the reason of failure. Guess it might have something to do with the threaded view on my win mob 6.5
PS : Paypal donate button shudn't be that difficult to get
And yes just thought that I would mention about the contacts - its actually very easy - just get them into csv file using the trick that you mentioned above and then import the csv file in gmail contacts! that's it! gmail automatically syncs the contacts thereafter!
Mark Crouch said:
I was about to say I have no idea and I don't really care (about Android phones that is, not your problem getting your messages onto your Hero!), but then.......
I found this - Sprite Software's FREE (at the moment) migration software - which looks like the answer to your prayers
I've seen a report from someone who used it - apparently, it's pretty slow, but as it's a "one-time" transfer operation.....who cares how long "free" takes!!!???
Let us know how you get on - and post something in the Android-related forums so that other users can benefit.....
Cheers,
Mark.
(At times like this, I wish I had a Paypal "donate" button.....this gem of wisdom is surely worth a buck/quid/euro or two ....Ho hum!!!)
Click to expand...
Click to collapse
Hi Mark (Am I Noob to Android)
It seems like, you have solved the issue here. I did have one question for you:
What if you allready had a PIM back of your old phone. Then the phone you have sold it. But you still the the PIM backup file on your SD card. Can you still go on to follow and start from 2 step and then step 3.
Or is the another way of doing this then?
Many Thanks.
yes u should be able too as long as the PIM file is on your SD card. you can just directly transfer it onto your PC, then rename it to a ZIP file and proceed as instructed above.
sam_htc_touch said:
Any idea how to import these into an Android Phone - just got an HTC Hero today. Took a backup of my existing contacts, sms etc from WM6.5 (HTC Touch HD) and could see the .pib (renamed as .zip) and individual files (which I renamed as .csv)..
Guess there might be a simple trick to restore them as well in HTC Hero?
Click to expand...
Click to collapse
tried it..
i just took the contacts.csv file ,tried to extract with gmail...
then
An unknown error occurred while importing your contacts.Learn more
.....oohhhh.....really needed help
Hello to everyone!
I had an HTC Touch HD and after a serious accident it's officially dead! I had a backup of contacts and messages to my sd card and i did what is described. I renamed it to .zip and i extracted it to my pc.. The result i get is files .pba, .pbl, .pbc, .pbx, .pbd and .pbt
Does anyone know what is all that and how i can use them in android phone????
please help me!
isovitis33 said:
Hello to everyone!
I had an HTC Touch HD and after a serious accident it's officially dead! I had a backup of contacts and messages to my sd card and i did what is described. I renamed it to .zip and i extracted it to my pc.. The result i get is files .pba, .pbl, .pbc, .pbx, .pbd and .pbt
Does anyone know what is all that and how i can use them in android phone????
please help me!
Click to expand...
Click to collapse
that is what I got also, I tried importing data to excel (with / without renaming to csv or ssv) but the text was not understandable in any language so I did not even proceed in selecting the separator and actually import the data.
Mr Black'd said:
that is what I got also, I tried importing data to excel (with / without renaming to csv or ssv) but the text was not understandable in any language so I did not even proceed in selecting the separator and actually import the data.
Click to expand...
Click to collapse
Come on guys read a little more on the web, if you would have done so you would have figured out you have the binary backup files, not the text file, so what you need to do is convert. Now if you press the search button for the forum you would have found this interesting link:
http://forum.xda-developers.com/showthread.php?t=424641
the title says: 'Convert PIM Backup binary file to text without using windows mobile device'
(considering yours broke like mine)
You can read there that you can convert it to a text file using an emulator on your pc and subsequentlyfollow the steps that were described here...
First of all what makes you think that none has bothered to use search button or at least Google the issue?
Second and last ok you have proved us stupid.
Anyway thanks for the link even though a late reply cause personally I have solved my issues.
Sent from my HTC HD2 using XDA app.
So a while ago I grabbed a Motorola G2, made a complete physical memory dump to file “Full_Phone_Backup.image” I also have a .vhd which will not mount, but I remember very clearly extracting the contacts list from it.
I didn’t keep records of how I went about doing it, but I’m sure it was some busybox or cygwin and I’m very sure the file contains all the partitions. Don’t know what happened to .vhd, but I also can’t make a new one either, so bad luck I guess. Anyway, now I need to extract SMS messages from it, and so I turned to Foremost. With the following in a foremost-db.conf:
Code:
db n 4000000 \x53\x51\x4c\x69\x74\x65\x20\x66\x6f\x72\x6d\x61\x74\x20\x33\x00
I hoped to get a good rundown of the databases and extract from there. Now, I don’t need deleted messages or anything, I need to find the SMS messages that would be there as if the phone was on right now. Well, that and search them.
So, Foremost carved a whooping 36 GiB of .db files, which I think suggests that the file header is actually not for SQLite 3 file as much as for a part of SQLite3 file, and one file may contain many. Secondary evidence of that is that massive number of files contain the same data shifted a more or less uniform number of lines (like an entire block shifted 1k lines down across 10 files before it completely disappears).
Anyway, from the carved DB files I got meaningful e-mail messages, from the carved photos and videos, I got meaningful pictures (ones which would be on this phone), so I am sure data is there. Problem is, I could not find the messages I’m looking for. I was unable to find a single SMS message, I was unable to browse any of the recovered databases with sqlite database browsers, and the most useful thing I was able to do was to use Ransack in windows to search for relevant text in those recovered files.
What am I doing wrong that I’m not finding SMS messages or any relevant text in this mess?
KYKYLLIKA said:
So a while ago I grabbed a Motorola G2, made a complete physical memory dump to file “Full_Phone_Backup.image” I also have a .vhd which will not mount, but I remember very clearly extracting the contacts list from it.
I didn’t keep records of how I went about doing it, but I’m sure it was some busybox or cygwin and I’m very sure the file contains all the partitions. Don’t know what happened to .vhd, but I also can’t make a new one either, so bad luck I guess. Anyway, now I need to extract SMS messages from it, and so I turned to Foremost. With the following in a foremost-db.conf:
Code:
dbn 4000000\x53\x51\x4c\x69\x74\x65\x20\x66\x6f\x72\x6d\x61\x74\x20\x33\x00
I hoped to get a good rundown of the databases and extract from there. Now, I don’t need deleted messages or anything, I need to find the SMS messages that would be there as if the phone was on right now. Well, that and search them.
So, Foremost carved a whooping 36 GiB of .db files, which I think suggests that the file header is actually not for SQLite 3 file as much as for a part of SQLite3 file, and one file may contain many. Secondary evidence of that is that massive number of files contain the same data shifted a more or less uniform number of lines (like an entire block shifted 1k lines down across 10 files before it completely disappears).
Anyway, from the carved DB files I got meaningful e-mail messages, from the carved photos and videos, I got meaningful pictures (ones which would be on this phone), so I am sure data is there. Problem is, I could not find the messages I’m looking for. I was unable to find a single SMS message, I was unable to browse any of the recovered databases with sqlite database browsers, and the most useful thing I was able to do was to use Ransack in windows to search for relevant text in those recovered files.
What am I doing wrong that I’m not finding SMS messages or any relevant text in this mess?
Click to expand...
Click to collapse
You can't get into the phone and use SMS backup app or PC to backup your SMS? Can you back them up to your Google account and then recover them from there?
There are several ways to recover SMS from a device, is this method the only one you've tried?
Are you recovering your SMS or someone else's?
Sent from my SM-S903VL using Tapatalk
KYKYLLIKA said:
So a while ago I grabbed a Motorola G2, made a complete physical memory dump to file “Full_Phone_Backup.image”
.
.
and I’m very sure the file contains all the partitions.
Click to expand...
Click to collapse
How did you make this Backup.image? In my question over here I dreamt of something like 'dd if=/dev/sda of=/image.bak' for a really complete backup. Did you make your complete memory dump in such a way and would i be possible to write it back to the phone?
Thanks!
Droidriven said:
You can't get into the phone and use SMS backup app or PC to backup your SMS? Can you back them up to your Google account and then recover them from there?
There are several ways to recover SMS from a device, is this method the only one you've tried?
Are you recovering your SMS or someone else's?
Sent from my SM-S903VL using Tapatalk
Click to expand...
Click to collapse
This is my sister’s phone. The backup is from 16 months back or so. I can’t go search the SMS it has now, because updates been around since then, including a new version of android, and all that. It does not have the old messages.
What I tried is a sqlite forensics utility called “sqlite forensics reporter”, but no luck with that either. Piriform recuva did not work, and I was unable to mount it as a virtual hard drive or find a part of it that I could mount as a virtual hard drive.
andy_ross said:
How did you make this Backup.image? In my question over here I dreamt of something like 'dd if=/dev/sda of=/image.bak' for a really complete backup. Did you make your complete memory dump in such a way and would i be possible to write it back to the phone?
Thanks!
Click to expand...
Click to collapse
This is entirely possible. However, I cannot find the boot record in there or a partition table, which is very strange. I do remember dumping all the partitions in that file, though. It’s been over a year now, so details kind of gone fuzzy. I am sure I could write it back to a phone and use like that, but I don’t have a suitable surrogate phone or a virtual machine to try that on. I will not do it on the device itself, seeing as how it’s in use and all that. I just want to find the text of some messages.
Goal: pass a whatsapp chat history backup (.crypt12) from one device to another with different telephone (whatsapp account) numbers.
Update: I succeeded with ultimate goal to move the chat history to a different device with a different phone number, but I failed with the re-encryption.
Encryption
First, I believed that the /data/data/com.whatsapp/files/key file might have been a leftover from an old version of Whatsapp on the old device, because I could not find the file. It turned out that it was generated later on the new device, after I finished my experiments. I’m not sure what triggers its creation.
Furthermore I was not able to decipher header and footer of the crypt12 backup file. I believe that a message authentication code (MAC) is part of it and something related to the Whatsapp account number (telephone number), because the app was quick in determining if a backup file is a restore candidate without decrypting, I’ld say.
How I managed to transfer the chat history:
Short version: get root on both devices and move /data/data/com.whatsapp/databases/msgstore.db over to the new device.
Long version: root on the new device will be less problematic, I guess. For the old device I used the fishy Kingroot app. It looks very professional and seems to download exploit code from a huge database for many devices. I got instant root access without flashing anything, but to be honest I don’t trust it regarding what else it might be doing...
Kill both Whatsapp apps before reading the msgstore.db file from the old and writing to the new device. I also removed the msgstore.db-* files on the new device. Sqlite might detect itself that the new database does not fit those helper files, but if they are not even there, they will be recreated correctly without any doubt.
Also set the permissions and ownership of that file to what it would be on the new device (not the old). My biggest oversight were the SELinux security attributes stored in the extended file attributes (XA). It made me believe that Whatsapp is verifying the database content and rejects it, but in reality it eats it just fine, as long as it gets proper access to it.
The XA tools I had available on the phones apparently show all the extended file attributes with “getfattr -d filename”, while e.g. on a standard Linux you need “getfattr -dm- filename” to get them all, not just the user.* domain. „ls -Z” shows the SELinux security context, which is a specific part of the extended file attributes, and in this case the only part I had set.
As with the file ownership, check which context is usually set on the database file on the new device and set it accordingly (“setfattr”). My old device did not even have SELinux and no extended file attributes were set there.
Other findings:
As the restore kicks only in during initial whatsapp setup, I’ve cleared data on whatsapp a lot. Constant reactivating the same phone number will trigger a hold-off on the whatsapp servers, delaying activation-SMS or -call.
But a seemingly corrupt or missing msgstore.db file trigger a restore as well, so it is possible to feed backups to Whatsapp without constant reactivation. Whatsapp failed with the restore if the SELinux context was wrong on the to-be-replaced database file, I believe. During my trials I just deleted the database (if I recall correctly) and the artificially triggered restored worked out.
Below this line is stuff I tried before:
At the moment I try to plant the old backup on whatsapp for decryption. To get to the point of restore, I force close whatsapp, clear app data, open whatsapp, give it no permissions and activate a number. At this point it asks for permissions to find restore files: I deny “contacts”, but allow for “media/files”. Here it consistently finds the most recent _local_ backup from the _same_ whatsapp account. At this point I force close the app and clear the app cache.
Restarting whatsapp brings it to the backup screen where it finds the local backup for the _same_ device.
At the moment I’m investigating which backups it picks up. For this I use backups from the same device/account as well (known to be working).
/data/data/com.whatsapp/files/key
This file is non-existent on the new device. If I drop a key file from a different account, it seems to get ignored. Whatsapp still finds the native backup (not one matching the key), and it successfully restores it.
/data/data/com.whatsapp/shared_prefs/keystore.xml
This file already exists, but only with “client_static_keypair”. If I replace that entry with the one from the old device, whatsapp will still find the native backup, but it will fail to restore it. The restore has to be skipped and whatsapp triggers a reactivation of the phone number, but if you enter the same new number again, whatsapp accepts it without SMS/call.
→this seems to be the encryption key to be used by whatsapp.
I can confirm the following crypt12 decryption code to be working:
https://gist.github.com/nlitsme/b079f351eb1bf9c3d356ce988bb6afdc
https://github.com/EliteAndroidApps/WhatsApp-Crypt12-Decrypter
They both require the backup file and the “key” file. The latter has a check where it compares a component from the key file with the backup 1:1. In the code this is called “t1” and “t2” which should match. So far I have backups with three different t1/t2. The original backup, with a matching key; the backups from the new account on the new devices, without a key; and the backups from the new device/account, where I mixed the string from the keystore.xml file in.
The “key” file is not generated by the latest Whatsapp anymore, as it seems. Maybe the encryption/decryption key is generated on-the-fly from the keystore.xml data. If this is true, then a new activation of my new telephone number would make these backups unreadable.
Created by author: 2018-04-19
Last edit by author: 2018-05-01
siemer said:
Goal: pass a whatsapp chat history backup (.crypt12) from one device to another with different telephone (whatsapp account) numbers.
Update: I succeeded with ultimate goal to move the chat history to a different device with a different phone number, but I failed with the re-encryption.
Encryption
First, I believed that the /data/data/com.whatsapp/files/key file might have been a leftover from an old version of Whatsapp on the old device, because I could not find the file. It turned out that it was generated later on the new device, after I finished my experiments. I’m not sure what triggers its creation.
Furthermore I was not able to decipher header and footer of the crypt12 backup file. I believe that a message authentication code (MAC) is part of it and something related to the Whatsapp account number (telephone number), because the app was quick in determining if a backup file is a restore candidate without decrypting, I’ld say.
How I managed to transfer the chat history:
Short version: get root on both devices and move /data/data/com.whatsapp/databases/msgstore.db over to the new device.
Long version: root on the new device will be less problematic, I guess. For the old device I used the fishy Kingroot app. It looks very professional and seems to download exploit code from a huge database for many devices. I got instant root access without flashing anything, but to be honest I don’t trust it regarding what else it might be doing...
Kill both Whatsapp apps before reading the msgstore.db file from the old and writing to the new device. I also removed the msgstore.db-* files on the new device. Sqlite might detect itself that the new database does not fit those helper files, but if they are not even there, they will be recreated correctly without any doubt.
Also set the permissions and ownership of that file to what it would be on the new device (not the old). My biggest oversight were the SELinux security attributes stored in the extended file attributes (XA). It made me believe that Whatsapp is verifying the database content and rejects it, but in reality it eats it just fine, as long as it gets proper access to it.
The XA tools I had available on the phones apparently show all the extended file attributes with “getfattr -d filename”, while e.g. on a standard Linux you need “getfattr -dm- filename” to get them all, not just the user.* domain. „ls -Z” shows the SELinux security context, which is a specific part of the extended file attributes, and in this case the only part I had set.
As with the file ownership, check which context is usually set on the database file on the new device and set it accordingly (“setfattr”). My old device did not even have SELinux and no extended file attributes were set there.
Other findings:
As the restore kicks only in during initial whatsapp setup, I’ve cleared data on whatsapp a lot. Constant reactivating the same phone number will trigger a hold-off on the whatsapp servers, delaying activation-SMS or -call.
But a seemingly corrupt or missing msgstore.db file trigger a restore as well, so it is possible to feed backups to Whatsapp without constant reactivation. Whatsapp failed with the restore if the SELinux context was wrong on the to-be-replaced database file, I believe. During my trials I just deleted the database (if I recall correctly) and the artificially triggered restored worked out.
Below this line is stuff I tried before:
At the moment I try to plant the old backup on whatsapp for decryption. To get to the point of restore, I force close whatsapp, clear app data, open whatsapp, give it no permissions and activate a number. At this point it asks for permissions to find restore files: I deny “contacts”, but allow for “media/files”. Here it consistently finds the most recent _local_ backup from the _same_ whatsapp account. At this point I force close the app and clear the app cache.
Restarting whatsapp brings it to the backup screen where it finds the local backup for the _same_ device.
At the moment I’m investigating which backups it picks up. For this I use backups from the same device/account as well (known to be working).
/data/data/com.whatsapp/files/key
This file is non-existent on the new device. If I drop a key file from a different account, it seems to get ignored. Whatsapp still finds the native backup (not one matching the key), and it successfully restores it.
/data/data/com.whatsapp/shared_prefs/keystore.xml
This file already exists, but only with “client_static_keypair”. If I replace that entry with the one from the old device, whatsapp will still find the native backup, but it will fail to restore it. The restore has to be skipped and whatsapp triggers a reactivation of the phone number, but if you enter the same new number again, whatsapp accepts it without SMS/call.
→this seems to be the encryption key to be used by whatsapp.
I can confirm the following crypt12 decryption code to be working:
https://gist.github.com/nlitsme/b079f351eb1bf9c3d356ce988bb6afdc
https://github.com/EliteAndroidApps/WhatsApp-Crypt12-Decrypter
They both require the backup file and the “key” file. The latter has a check where it compares a component from the key file with the backup 1:1. In the code this is called “t1” and “t2” which should match. So far I have backups with three different t1/t2. The original backup, with a matching key; the backups from the new account on the new devices, without a key; and the backups from the new device/account, where I mixed the string from the keystore.xml file in.
The “key” file is not generated by the latest Whatsapp anymore, as it seems. Maybe the encryption/decryption key is generated on-the-fly from the keystore.xml data. If this is true, then a new activation of my new telephone number would make these backups unreadable.
Created by author: 2018-04-19
Last edit by author: 2018-05-01
Click to expand...
Click to collapse
I'm trying to do the same thing as you; it is going mostly smooth with permissions and user/group, but I'm having some issues changing the SELinux extended security attributes. I must change mine from
Code:
u:object_r:app_data_file:s0
to
Code:
u:object_r:app_data_file:s0:c512,c768
but I can't really find enough guidance online... If you could help, I'd be really grateful
Hi
My whatsapp backup was corrupted, I extracted it from Google Drive
Then .dump into a sql file, fixed the errors
Recompile into a repair db, which works because it can be viewed on the WhatsappViewer.exe
Since the db is not encrypted, how do I restore it in my own phone?
Can msgstore.db be placed inside /data/data/com.whatsapp/databases/ of my own phone? Will it work without the msgstore.db being crypted with crypt12?
Nite.Achilles said:
Hi
My whatsapp backup was corrupted, I extracted it from Google Drive
Then .dump into a sql file, fixed the errors
Recompile into a repair db, which works because it can be viewed on the WhatsappViewer.exe
Since the db is not encrypted, how do I restore it in my own phone?
Can msgstore.db be placed inside /data/data/com.whatsapp/databases/ of my own phone? Will it work without the msgstore.db being crypted with crypt12?
Click to expand...
Click to collapse
Hi, bro. Would you mind telling me how you fixed errors in the sql file? I have one corrupted database and Im not able to see it in WA Viewer. Thanks in advance.
IvanN8458 said:
Hi, bro. Would you mind telling me how you fixed errors in the sql file? I have one corrupted database and Im not able to see it in WA Viewer. Thanks in advance.
Click to expand...
Click to collapse
Hi I used this
reference from here https://andreas-mausch.de/whatsapp-viewer/
echo .dump | sqlite3 msgstore.db > temp.sql
echo .quit | sqlite3 -init temp.sql repaired.db
Nite.Achilles said:
Hi I used this
reference from here https://andreas-mausch.de/whatsapp-viewer/
echo .dump | sqlite3 msgstore.db > temp.sql
echo .quit | sqlite3 -init temp.sql repaired.db
Click to expand...
Click to collapse
Thank you very much, bro. I'll try that and hope it can fix it. Have a nice one.
Hi,
had any of you success with the restored db, to get it reintegrated into WhatsApp?
siemer said:
Goal: pass a whatsapp chat history backup (.crypt12) from one device to another with different telephone (whatsapp account) numbers.
Update: I succeeded with ultimate goal to move the chat history to a different device with a different phone number, but I failed with the re-encryption.
Encryption
First, I believed that the /data/data/com.whatsapp/files/key file might have been a leftover from an old version of Whatsapp on the old device, because I could not find the file. It turned out that it was generated later on the new device, after I finished my experiments. I’m not sure what triggers its creation.
Furthermore I was not able to decipher header and footer of the crypt12 backup file. I believe that a message authentication code (MAC) is part of it and something related to the Whatsapp account number (telephone number), because the app was quick in determining if a backup file is a restore candidate without decrypting, I’ld say.
How I managed to transfer the chat history:
Short version: get root on both devices and move /data/data/com.whatsapp/databases/msgstore.db over to the new device.
Long version: root on the new device will be less problematic, I guess. For the old device I used the fishy Kingroot app. It looks very professional and seems to download exploit code from a huge database for many devices. I got instant root access without flashing anything, but to be honest I don’t trust it regarding what else it might be doing...
Kill both Whatsapp apps before reading the msgstore.db file from the old and writing to the new device. I also removed the msgstore.db-* files on the new device. Sqlite might detect itself that the new database does not fit those helper files, but if they are not even there, they will be recreated correctly without any doubt.
Also set the permissions and ownership of that file to what it would be on the new device (not the old). My biggest oversight were the SELinux security attributes stored in the extended file attributes (XA). It made me believe that Whatsapp is verifying the database content and rejects it, but in reality it eats it just fine, as long as it gets proper access to it.
The XA tools I had available on the phones apparently show all the extended file attributes with “getfattr -d filename”, while e.g. on a standard Linux you need “getfattr -dm- filename” to get them all, not just the user.* domain. „ls -Z” shows the SELinux security context, which is a specific part of the extended file attributes, and in this case the only part I had set.
As with the file ownership, check which context is usually set on the database file on the new device and set it accordingly (“setfattr”). My old device did not even have SELinux and no extended file attributes were set there.
Other findings:
As the restore kicks only in during initial whatsapp setup, I’ve cleared data on whatsapp a lot. Constant reactivating the same phone number will trigger a hold-off on the whatsapp servers, delaying activation-SMS or -call.
But a seemingly corrupt or missing msgstore.db file trigger a restore as well, so it is possible to feed backups to Whatsapp without constant reactivation. Whatsapp failed with the restore if the SELinux context was wrong on the to-be-replaced database file, I believe. During my trials I just deleted the database (if I recall correctly) and the artificially triggered restored worked out.
Below this line is stuff I tried before:
At the moment I try to plant the old backup on whatsapp for decryption. To get to the point of restore, I force close whatsapp, clear app data, open whatsapp, give it no permissions and activate a number. At this point it asks for permissions to find restore files: I deny “contacts”, but allow for “media/files”. Here it consistently finds the most recent _local_ backup from the _same_ whatsapp account. At this point I force close the app and clear the app cache.
Restarting whatsapp brings it to the backup screen where it finds the local backup for the _same_ device.
At the moment I’m investigating which backups it picks up. For this I use backups from the same device/account as well (known to be working).
/data/data/com.whatsapp/files/key
This file is non-existent on the new device. If I drop a key file from a different account, it seems to get ignored. Whatsapp still finds the native backup (not one matching the key), and it successfully restores it.
/data/data/com.whatsapp/shared_prefs/keystore.xml
This file already exists, but only with “client_static_keypair”. If I replace that entry with the one from the old device, whatsapp will still find the native backup, but it will fail to restore it. The restore has to be skipped and whatsapp triggers a reactivation of the phone number, but if you enter the same new number again, whatsapp accepts it without SMS/call.
→this seems to be the encryption key to be used by whatsapp.
I can confirm the following crypt12 decryption code to be working:
https://gist.github.com/nlitsme/b079f351eb1bf9c3d356ce988bb6afdc
https://github.com/EliteAndroidApps/WhatsApp-Crypt12-Decrypter
They both require the backup file and the “key” file. The latter has a check where it compares a component from the key file with the backup 1:1. In the code this is called “t1” and “t2” which should match. So far I have backups with three different t1/t2. The original backup, with a matching key; the backups from the new account on the new devices, without a key; and the backups from the new device/account, where I mixed the string from the keystore.xml file in.
The “key” file is not generated by the latest Whatsapp anymore, as it seems. Maybe the encryption/decryption key is generated on-the-fly from the keystore.xml data. If this is true, then a new activation of my new telephone number would make these backups unreadable.
Created by author: 2018-04-19
Last edit by author: 2018-05-01
Click to expand...
Click to collapse
Those extended attributes ... I would never have suspected that if it weren't for you, thank you so much !
so, does it work ?
Nite.Achilles said:
Hi
My whatsapp backup was corrupted, I extracted it from Google Drive
Then .dump into a sql file, fixed the errors
Recompile into a repair db, which works because it can be viewed on the WhatsappViewer.exe
Since the db is not encrypted, how do I restore it in my own phone?
Can msgstore.db be placed inside /data/data/com.whatsapp/databases/ of my own phone? Will it work without the msgstore.db being crypted with crypt12?
Click to expand...
Click to collapse
Hi,
So how you fix it ultimately?
Hi, I am having the same problem as with restoring my Whatsapp history and wondered if any kind soul can help me to restore my Whatsapp history?
What I had done :
I had my S8 rooted. The whole phone was wipe when it was rooted. I hope this step is correct.
My encrypted backup (crypt12 and crypt14) was saved already before I rooted and I also need a backup of my phone (just not the application data because the Dr Fone software mentioned that my S8 needed to be rooted before I can backup my application data)
But I couldnt find the folders mentioned here :
/data/data/com.whatsapp/databases/msgstore.db
Can anyone guide me pls?
XMatrix2099 said:
Hi, I am having the same problem as with restoring my Whatsapp history and wondered if any kind soul can help me to restore my Whatsapp history?
What I had done :
I had my S8 rooted. The whole phone was wipe when it was rooted. I hope this step is correct.
My encrypted backup (crypt12 and crypt14) was saved already before I rooted and I also need a backup of my phone (just not the application data because the Dr Fone software mentioned that my S8 needed to be rooted before I can backup my application data)
But I couldnt find the folders mentioned here :
/data/data/com.whatsapp/databases/msgstore.db
Can anyone guide me pls?
Click to expand...
Click to collapse
Hello, since you rooted your phone it wiped out all data.
So you msgstore.db is actually crypt12 and crypt14 data.
Don't root unless you have all your items.
It wipes the key as well.
I do think key don't exist anymore and login happens in keystore.xml with static client