Have you ever wanted to
- have a useful backup of your whatsapp chats?
- be able to read your whatsapp chats again using a backup file?
- delete the grown-too-big message history of whatsapp without losing the possibility to read the old conversations?
- read older messages without the need of endlessly pressing the annoying "load older messages" button?
- be able to search old messages?
- ...?
Here is a little Tool to display whatsapp chats on your computer, using the database files from your android installation of whatsapp.
It's called WhatsApp Xtract and was created by Fabio Sangiacomo and me.
Preview of the result:
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
So here is WhatsApp Xtract v2.3
Released on April 25th, 2018
Tested with Whatsapp (Android) 2.18.46
Tested with Whatsapp (iPhone) 2.5.1
CHANGELOG:
v2.3 (updated by ztedd - Apr 25, 2018)
- fixed image preview to work with new Whatsapp database format
- some minor fixes
v2.2 (updated by ztedd - Nov 17, 2012)
- now supports new emoji smileys
- (Android Version) hotfix for TypeError in b64encode
- (Android Version) decoded file won't be deleted even if it can't be opened
v2.1 (updated by Fabio Sangiacomo and ztedd - May 7th, 2012)
- improved install pyCrypto.bat
- added easy drag and drop possibility with whatsapp_xtract_drag'n'drop_database(s)_here.bat
- (Android Version) added support to fix corrupted android whatsapp database (needs sqlite3, for windows sqlite3.exe is contained in the archive)
- (Android Version) removed wrong extraction of owner in android version
- (Iphone Version) information from Z_METADATA table will be printed to shell
- (Iphone Version) fixed bug in support of older Iphone whatsapp databases
V2.0 (updated by Fabio Sangiacomo and ztedd - Apr 28, 2012)
- supports WhatsApp DBs coming from both Android and iPhone platforms
- (Android Version) wa.db is optional
- (Android Version) now also crypted msgstore.db.crypt from the SD card can be imported
- chat list is sorted by the last sent message
- fixed some bugs (e.g. that the script didn't work with python 3)
------------------------
Wish list:
Feel free to take the code and improve it. So maybe we could include some code from this different version which is able to export separate htmls for each chat:
https://github.com/BurakDev/hotoloti/issues/10
------------------------
How to use:
(instrucciones en español / deutsche Anleitung - old)
You need a copy of the whatsapp database and (if no root) get your key.
a) if your Android phone is rooted:
get these files:
/data/data/com.whatsapp/databases/msgstore.db and wa.db
(for this you need root access. detailed instructions in the bottom of this file.
The advantage of having wa.db is that the corresponding contact names of phone numbers will be displayed.)
b) if you have a normal Android (not rooted):
you need this to get the key:
[Tool] WhatsApp Key/DB Extractor | CRYPT6-12 | NON-ROOT
https://forum.xda-developers.com/showthread.php?t=2770982
then you need the .crypt12 database from /sdcard/WhatsApp/Databases/
then you need a tool like Whatsapp Viewer to decrypt the database file:
Whatsapp Viewer
http://andreas-mausch.de/whatsapp-viewer/
https://forum.xda-developers.com/showthread.php?t=2719741
The decrypted database can already be viewed in Whatsapp Viewer. If you want the output style from Whatsapp xtract then you can continue with the decrypted file.
c) On IPhone, get this file (don't know if it still works in 2018):
net.whatsapp.WhatsApp/Documents/ChatStorage.sqlite
(You can use an Iphone Backup Tool to get the file, e.g. I-Twin or Iphone Backup Extractor. Make sure to create an unencrypted backup with Itunes, as these tools can't handle encrypted backups. Another possibility are forensic tools like UFED Physical Analyzer.)
Extract this archive (Whatsapp_Xtract....zip) to a certain folder on your computer, e.g. C:\WhatsApp.
Copy the database(s) to e.g. C:\WhatsApp (on Android, you simply copy the whole folder WhatsApp on SD card to your computer e.g. to C:\WhatsApp and then copy the database file from C:\WhatsApp\Databases to C:\WhatsApp)
You need Python
https://www.python.org/downloads/
It's available for Windows, Linux and Mac. However in my HowTo, I focus on Windows.
Now run whatsapp_xtract_android.bat or whatsapp_xtract_iphone.bat
OR simply drag and drop the database file(s) to whatsapp_xtract_drag'n'drop_database(s)_here.bat
OR whatsapp_xtract_console.bat and then manually specify the input file with one of these commands:
COMMAND LINE OPTIONS:
For Android DB:
python whatsapp_xtract.py msgstore.db -w wa.db
OR (if wa.db is unavailable)
python whatsapp_xtract.py msgstore.db
For iPhone DB: (-w option is ignored)
python whatsapp_xtract.py ChatStorage.sqlite
Click to expand...
Click to collapse
Once finished, your browser will open and show the chats.
The resulting file size of the .html file will be slightly bigger than the size of the original .db.crypt12 database.
Attachment:
Whatsapp Xtract 2.3 2018-04-25 (~ 1,8 MB)
contains:
!README.txt
bplist.py
convert_smileys_python_2.py
SetEnv.exe
sqlite3.exe
whatsapp_xtract.py
whatsapp_xtract_android.bat
whatsapp_xtract_iphone.bat
whatsapp_xtract_console.bat
whatsapp_xtract_drag'n'drop_database(s)_here.bat
folder: data
samples of needed input files and Media folder
also available here:
https://www.xup.in/dl,18324514/Whatsapp_Xtract_2.3_2018-04-25.zip/
IF SOMETHING DOESN'T WORK:
1. First check if you already have the newest download (check full name of the .zip file: is it the same as in mentioned in this thread?). Maybe your error already has been fixed.
2. Read the post Some additional information and read the last pages of this thread to see if your problem is already being discussed.
Use the Search inside this thread function!
3. If not or if it doesn't help, post your error with a detailed description (and error message) in the thread.
Changelog, Recent Bugfixes, Feature Requests
CHANGELOG:
v2.3 (updated by ztedd - Apr 25, 2018)
- fixed image preview to work with new Whatsapp database format
- updated Readme, refering to Whatsapp Viewer (which is able to decrypt the .crypt12 files)
- some minor fixes
v2.2 (updated by ztedd - Nov 17, 2012)
- now supports new emoji smileys
- (Android Version) hotfix for TypeError in b64encode
- (Android Version) decoded file won't be deleted even if it can't be opened
v2.1-bugsfixed-4 (2012-05-16)
- fixed .bat files again to support script execution from each external directory
- fixed .bat files if error "python" not found occurs (if you get that error, run !install pyCrypto.bat first; rightclick on it and choose "run as administrator")
- updated compatibility for newer android whatsapp versions to show media files with thumbnails correctly
v2.1 (updated by Fabio Sangiacomo and ztedd - May 7th, 2012)
- improved install pyCrypto.bat
- added easy drag and drop possibility with whatsapp_xtract_drag'n'drop_database(s)_here.bat
- (Android Version) added support to fix corrupted android whatsapp database (needs sqlite3, for windows sqlite3.exe is contained in the archive)
- (Android Version) removed wrong extraction of owner in android version
- (Iphone Version) information from Z_METADATA table will be printed to shell
- (Iphone Version) fixed bug in support of older Iphone whatsapp databases
v2.0-bugsfixed-10
file: Whatsapp_Xtract_V2.0_2012-05-02-3.zip
- On errors, now the window will not close but show you an error message
- Now installation of PyCrypto is indeed optional. You only need it if you want to import a crypted db.
- added some advanced instructions in the Readme on how to use this script with the command line (also added a .bat that will open the command prompt and show the help message of the script)
- changed error management to let the script continue on errors like these:
* 'NoneType' object is not subscriptable in line 1028
* 'NoneType' object is not subscriptable in line 1126
The errors will be displayed in the message table and in the DOS Box, but the script will go on
- fixed syntax error in line 216
- fixed error that it tries to name output file "N/A" which is not possible because of the "/"
- fixed error of handling timestamps in Iphone database
- fixed " AttributeError: 'int' object has no attribute 'split' "
- changed parsing of sqlite databases, columns are now accessed by name and not by number (e.g., msgs["ZTEXT"] instead of msgs[10])
- included Iphone support for images, videos, vcard and (beta) gps. Fixed bug of opening a wrong output file in browser.
- changed install pyCrypto.bat
V2.0 (updated by Fabio Sangiacomo and ztedd - Apr 28, 2012)
- supports WhatsApp DBs coming from both Android and iPhone platforms
- (Android Version) wa.db is optional
- (Android Version) now also crypted msgstore.db.crypt from the SD card can be imported
- chat list is sorted by the last sent message
- fixed some bugs (e.g. that the script didn't work with python 3)
V1.3 (updated by ztedd - Apr 17, 2012)
- corrected linking of offline files (now linking according to media file size)
V1.2 (updated by ztedd - Apr 5, 2012)
- media files also linked to offline files
- corrected hyperlinks
V1.1 (updated by ztedd - Apr 5, 2012)
- changed database structure, Android only
- show contact names
- show smileys
- show images
- link / popup for images, video, audio, gps
- clickable links
V1.0 (created by Fabio Sangiacomo - Dec 10, 2011)
- first release, iPhone only:
it takes in input the file "ChatStorage.sqlite",
extracts chat sessions and the bare text
- sortable js allows table sorting to make chat sessions easily readable
FEATURE REQUESTS:
These features below are quite outdated. For example, conversion/migration from Iphone / Windows Phone to Android was made possible by paid apps (Wazzapmigrator / WinWazzapmigrator).
Anyway, if you want to contribute, please take my source file, put a repo on github and go on!
- Blackberry Support: Step 1 - How is it possible to get the unencrypted messagestore.db file from Blackberry Whatsapp? (valek07); Step 2 - get a sample db file (see how to create sample db) and include Blackberry support
- Nokia Support: Step 1 - How to get the database file from Nokia? (oscargpes) Step 2 - Get a sample db file (see how to create sample db) and include Nokia support
- Windows Phone Support (beastie_carrion): Step 1 - How to get the database file from Windows Phone? Step 2 - Get a sample db file (see how to create sample db) Step 3 - include Windows Phone support
- create an android app for this, ideally offering "export to googlemail" function as in SMS Backup + (nomad4ever)
/ conversion of whatsapp to sms xml (ofmb)
- conversion between Android and Iphone database:
-- Android -> Iphone: ark0n3 is working on a Java tool called WhatsAppMigrator, here you can download WhatsAppMigrator
-- Iphone -> Android: no solution yet
- merge multiple backup files (darKpoiSonend) (you can use the tool SQLite Compare as a workaround, see howto in this post by ztedd)
- extract deleted messages (sometimes deleted messages remain in the database file but can't be accessed with normal Sqlite commands anymore) (Rauken)
- include Iphone support for audio, gps, gps of a place (like a restaurant), group messages and change of group name -> we would need an Iphone sample database containing these message types!
Some Additional Information
General advice on how to backup Whatsapp
Some general advice on how to backup Whatsapp and get the database file:
Android:
- In Whatsapp go to settings - more - Backup Chats
- Copy the folder "Whatsapp" on the SD card to your backup location (e.g., PC)
- (ideally also) use the app Titanium Backup to backup the full whatsapp application together with its data, copy the backup from the folder "TitaniumBackup" on the SD card to your backup location (e.g., PC)
- Use this tool Whatsapp Backup Extractor (download in this thread) to show the chats in a friendly readable format. The necessary files "wa.db" and "msgstore.db" you will find inside the Titanium Backup archive "com.whatsapp-[Date]-[some digits].tar.gz", alternatively (without Titanium Backup) you can use the msgstore.db.crypt file from the folder Whatsapp/Databases on the SD card.
Iphone:
- use Itunes to create an unencrypted Backup
- use an Iphone Backup Tool to get the file net.whatsapp.WhatsApp/Documents/ChatStorage.sqlite, e.g. I-Twin or Iphone Backup Extractor. Make sure to create an unencrypted backup with Itunes, as these tools can't handle encrypted backups. Another possibility are forensic tools like UFED Physical Analyzer.)
Blackberry:
- sync your blackberry with desktop manager and then copy the messagestore.db file from SD
- however, it seems that this file is encrypted? Currently we don't know how to get the unencrypted messagestore.db file
- Blackberry not supported yet!
Nokia:
- not known yet
- Nokia not supported yet!
CHECK IF DATABASE FILE CAN BE OPENED / EDIT CHAT MESSAGES INSIDE DATABASE
You may want to open the decrypted whatsapp database (ChatStorage.sqlite, msgstore.db or msgstore.plain.db) without this tool, e.g. if something goes wrong to check if it can be opened at all or to edit chat messages.
For this use SQLite Database Browser ( http://sqlitebrowser.sourceforge.net/ ).
However it is much more confusing and the messages are ordered by date, not by conversations. Also you won't see the smileys and media files...
IF DATABASE CAN'T BE OPENED: REPAIR DATABASE
Update: This function is now included in Whatsap Xtract 2.1.
If you get an error like this: Error: file is encrypted or is not a database and you tried to open the unencrypted database with SQLite Database Browser but it can't be opened with it, then try the following:
Open the database with Editor, can you see it starts with "SQLite format 3"? Then you may try to repair the database.
Thanks to hape70 for the instructions on how to repair (I translated it from German to English):
- download sqlite3.exe from http://www.sqlite.org/download.html
- start whatsapp_xtract_console.bat and then type:
- echo .dump | sqlite3.exe msgstore.db > Temp.sql
- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db
- python whatsapp_xtract.py msgstore_new.db
(make sure to replace the underlined msgstore.db with the name of the database that is unencrypted but can't be opened, sometimes it will be msgstore.plain.db)
Click to expand...
Click to collapse
MEDIA FILES
If you want to watch the videos, audios and images, you can click on the thumbnails and media links. Popups should open displaying the media.
However, online media files are available only for the last ~ 3 weeks.
But you still can open the offline media files, they are linked as well.
For this it is necessary to at first copy the folder "Media" from /sdcard/Whatsapp (Android) or net.whatsapp.WhatsApp (Iphone) to the certain folder of your computer where this tool is installed. Then run the script. Now the media files should be correctly linked.
HOW TO MERGE MULTIPLE DATABASE FILES WITH SQLite Compare
Merging of multiple backup files is still on the feature request list, but you can use the tool SQLite Compare as a workaround, see howto in this post by ztedd.
Worked here!
Thanks a lot!
Glad you can use it!
any way to ADD more emjoyies ... if OK ... how ??
abumhmmd1 said:
any way to ADD more emjoyies ... if OK ... how ??
Click to expand...
Click to collapse
What do you mean? Do you want to be able to post more emoticons in the android whatsapp app?
i did what is said above, but when i use xtract.bat nothing happen,
just a cmd popup and close but nothing else open
edit: my files are .db.crypt
msgstore.db <--> msgstore.db.crypt
jasaad said:
i did what is said above, but when i use xtract.bat nothing happen,
just a cmd popup and close but nothing else open
edit: my files are .db.crypt
Click to expand...
Click to collapse
It won't work with the .db.crypt files from the SD card. You need the unencrypted msgstore.db that you can get from the internal memory, location:
/data/data/com.whatsapp/databases
I guess you will need a rooted android to access this.
I updated the Readme to show 3 different ways how to do this, e.g. using Titanium Backup.
Update 2012-04-28: new version 2.0 supports android whatsapp .db.crypt files!
ztedd said:
Have you ever wanted to
- have a useful backup of your whatsapp chats?
- be able to read your whatsapp chats again using a backup file?
- delete the grown-too-big message history of whatsapp without losing the possibility to read the old conversations?
- read older messages without the need of endlessly pressing the annoying "load older messages" button?
- be able to search old messages?
- ...?
Here is a little Tool to display whatsapp chats on your computer, using the database files from your android installation of whatsapp.
It's called WhatsApp Extractor and was created by Fabio Sangiacomo on Dec 10, 2011.
blog.digital-forensics.it/2011/12/whatsapp-xtract.html
code.google.com/p/hotoloti/
Preview of the result (old version):
I found the script and realized it didn't work anymore, so I updated it and along that opportunity added some improvements.
So here is WhatsApp Extractor for Android v1.2
Updated on April 5, 2012
Tested with Whatsapp (Android) 2.7.5613
Changelog:
V1.2
- media files also linked to offline files
- corrected hyperlinks
V1.1
- updated database structure, old version didn't work anymore
- show contact names
- show smileys
- show images
- link / popup for images, video, audio, gps
- clickable links
How to use:
Root your android phone or use any other method to get access to the folder
/data/data/com.whatsapp/databases
copy the files "msgstore.db" and "wa.db" to a certain folder of your computer
You can do this by
- using the App RootExplorer (first copy to SD, then mount phone to computer)
- or using adb: open cmd and type "adb pull /data/data/com.whatsapp/databases/msgstore.db C:\Whatsapp" (replace C:\Whatsapp by the location of the certain folder of your computer)
- or using the app Titanium Backup. Use Titanium Backup to backup the full whatsapp application together with its data, copy the backup from the folder "TitaniumBackup" on the SD card to your PC, then extract the files "wa.db" and "msgstore.db" that you will find inside the Titanium Backup archive "com.whatsapp-[Date]-[some digits].tar.gz" to the certain folder.
Download and Install Python python.org/getit/
Extract the attached archive Whatsapp Extractor for Android V1.1 2012-04-05.zip (whatsapp_xtract.py, whatsapp_xtract.bat, folders emoji and sort-table) to the certain folder and run whatsapp_xtract.bat (on windows, other OS different)
Once finished, your browser will open and show you your chats.
The resulting file size of the .htm file will be slightly bigger than the size of the .db database.
Attachment:
Whatsapp Extractor for Android V1.2 2012-04-05.zip (~ 500 KB)
contains:
whatsapp_xtract.py
whatsapp_xtract.bat
folder: emoji
folder: sort-table
also available here:
freakshare.com/files/im8s5y35/Whatsapp-Extractor-for-Android-V1.2-2012-04-05.zip.html
Click to expand...
Click to collapse
I don't suppose there is a video tut to go with this is there? I am finding it hard to follow.. Thanks
For a while now I'm looking for such a tool, for my iPhone. I know this is a Android forum but I was still wondering if it would be very difficult to "adjust" this tool to work for the ChatStorage.sqlite database in iOS?
Whatsapp Extract messages from Backup for Whatsapp on Iphone / iOS
Mayn said:
For a while now I'm looking for such a tool, for my iPhone. I know this is a Android forum but I was still wondering if it would be very difficult to "adjust" this tool to work for the ChatStorage.sqlite database in iOS?
Click to expand...
Click to collapse
Yes, there is a possibility. The "old" version of this tool works with Whatsapp for Iphone:
http://blog.digital-forensics.it/2011/12/whatsapp-xtract.html
http://code.google.com/p/hotoloti/
However, it doesn't support the new features yet, that's to say displaying smileys, images, videos, and so on.
Clair.f said:
I don't suppose there is a video tut to go with this is there? I am finding it hard to follow.. Thanks
Click to expand...
Click to collapse
Where exactly can't you follow? At which step?
New version of Whatsapp Backup Extractor: v1.3 Updated April 17, 2012
As of today, a new version is available.
V1.3
by Android Fan on April 17, 2012
- corrected linking of offline files (now linking according to media file size)
Download:
http://freakshare.com/files/yw32d30x/Whatsapp-Extractor-for-Android-V1.3-2012-04-17.zip.html
http://forum.xda-developers.com/attachment.php?attachmentid=1009665&d=1334683158
ztedd said:
Yes, there is a possibility. The "old" version of this tool works with Whatsapp for Iphone:
http://blog.digital-forensics.it/2011/12/whatsapp-xtract.html
http://code.google.com/p/hotoloti/
However, it doesn't support the new features yet, that's to say displaying smileys, images, videos, and so on.
Click to expand...
Click to collapse
The developer Fabio Sangiacomo told me that he is working on a new version of the Iphone Whatsapp Backup Extractor. This will be compatible as well with Iphone and Android Version of Whatsapp databases and will have all the features of my Android Version (e.g., displaying smileys, images, and so on).
ztedd said:
The developer Fabio Sangiacomo told me that he is working on a new version of the Iphone Whatsapp Backup Extractor. This will be compatible as well with Iphone and Android Version of Whatsapp databases and will have all the features of my Android Version (e.g., displaying smileys, images, and so on).
Click to expand...
Click to collapse
That is good news!
I just read that the developer of iTwin is also working on implementing Facebook and WhatsApp output in his (Windows) program.
Mayn said:
That is good news!
I just read that the developer of iTwin is also working on implementing Facebook and WhatsApp output in his (Windows) program.
Click to expand...
Click to collapse
Ah, interesting.
Have you tried the 1.0 version of Whatsapp Extractor with your iPhone whatsapp backup files?
I'm about to. But maybe I'll wait till Fabio has come up with his new version.
How to open and convert a crypted whatsapp database backup (msgstore.db.crypt)
CRYPTED WHATSAPP DATABASE
If you only have a backup in the crypted format from /sdcard/WhatsApp/Databases/msgstore.db.crypt then you need to decrypt the database at first.
At the moment, this can only be done by reinstalling whatsapp. If the backup is present in the mentioned folder on the SD, then you will be asked to import the backup. However, the backup can be from a totally different phone number / whatsapp account.
After installing and importing the backup, access the database files as explained above (1. and 2.).
You don't need an android phone for this, can use Bluestacks Emulator for Windows. Make sure to root Bluestacks, using this guide:
http://www.androidfilehost.com/main...ueStacks Beta/HowTo/HowTo Root BlueStacks.pdf
Here it isn't working i have the unencrypted file, placed in the same folder as whatsapp_xtract.py and .bat and three folders, the bat appears for a second and it pop away.
What i'm doing wrong?
Edit: Getting error: This file is encrypted or not a database.
But it isn't a db.crypt file.
Hello, recently I looked into LG backup and restore application using static analysis.
Note: this is based on version lg bnr 8.0.26 from 2020-02-13, tested with 7.0.20 from 2008-12-31
Here is the extract tool:
https://github.com/Mysak0CZ/LBFtool
And here is the technical part (you don't need to read this, if you just want to extract your data):
The file contains 2 headers (one of them is referred to as "new")
These are XML documents encrypted using AES/ECB/PKCS5Padding + SHA256 and null IV with static password
Code:
cd562f61-5399-3978-ac76-7c54b0508010
And they look like this:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<BackUp>
<Common>
<Name>LGBackup_200217_1</Name> <!-- Name of the backup file -->
<Id>Randomly generated UUID</Id> <!-- Id of the backup file -->
<IsNew>True</IsNew> <!-- IsNew is same in new and old header -->
<Version>1.00</Version>
<Attr Name="product_model">LM-X525</Attr>
<Attr Name="os_version">9</Attr>
<!-- And other attributes including google email -->
</Common>
<BackupItem Category="settings_DB" StartOffset="59392" DataSize="136626" />
<BackupItem Category="MEDIA_DB" StartOffset="901169" DataSize="78393344" FileName="external.db" />
<!-- Other backuped items -->
</BackUp>
The "new" header has more information than original header, but same structure.
The attribute "IsNew" signals the presence of the new header. (and is True in both files)
All numbers inside file are in Big-endian.
And this is the structure of the file (decimal position):
Code:
0: Unknown - 0x00 for me
1: Int32 - size of the new header
5: Int64 - offset to the new header (usually 13)
13 ... EOF-33: data
EOF-33: 20 bytes - SHA1 hash of the password (if present, otherwise still data)
EOF-13: Int64 - offset to the (old) header
EOF-5: Int32 - size of the (old) header
EOF-1: Bool - Lock byte - the file is (0x01) or isn't (0x00) protected by password
The password is only checked during import, not used to encrypt data (so to remove password, just set the last byte to 0)
Every "BackupItem" entry has a "Category".
Special case is Category="APPLICATION" with this structure:
Code:
<BackupItem Category="APPLICATION">
<APPLICATION Name="com.lge.ime" StartOffset="19221586" DataSize="2385983" Package="com.lge.ime">
<Attr Name="com.lge.ime">com.lge.ime</Attr>
</APPLICATION>
<APPLICATION Name="org.exmple.app" StartOffset="1429652009" DataSize="3420160" FileName="org.exmple.app.tar" Package="org.exmple.app">
<Attr Name="org.exmple.app">org.exmple.app</Attr>
</APPLICATION>
<!-- ... -->
</BackupItem>
All applications are tar files that contain 2 folders:
app - contains the apk file
data - application's internal data
All other items have:
StartOffset - Offset inside file
DataSize - Self explanatory
FileName - Not all have this, but most do; it is the original name of the backuped file.
The files are not compressed nor encrypted.
Footnote: This is my first post on XDA, so sorry for any mistakes I made (including my english) and feel free to correct me/give me advice.
Mysak0CZ said:
Hello, recently I looked into LG backup and restore application using static analysis.
Note: this is based on version lg bnr 8.0.26 from 2020-02-13, tested with 7.0.20 from 2008-12-31
Here is the extract tool:
And here is the technical part (you don't need to read this, if you just want to extract your data):
The file contains 2 headers (one of them is referred to as "new")
These are XML documents encrypted using AES/ECB/PKCS5Padding + SHA256 and null IV with static password
And they look like this:
The "new" header has more information than original header, but same structure.
The attribute "IsNew" signals the presence of the new header. (and is True in both files)
All numbers inside file are in Big-endian.
And this is the structure of the file (decimal position):
The password is only checked during import, not used to encrypt data (so to remove password, just set the last byte to 0)
Every "BackupItem" entry has a "Category".
Special case is Category="APPLICATION" with this structure:
All applications are tar files that contain 2 folders:
app - contains the apk file
data - application's internal data
All other items have:
StartOffset - Offset inside file
DataSize - Self explanatory
FileName - Not all have this, but most do; it is the original name of the backuped file.
The files are not compressed nor encrypted.
Footnote: This is my first post on XDA, so sorry for any mistakes I made (including my english) and feel free to correct me/give me advice.
Click to expand...
Click to collapse
Nice job dude! just a the quick recommendation: I would suggest that you put your code on gitlab or GitHub so it will become easier for others to contribute
Sent from my OnePlus 6T using XDA Labs
Hi. I'm trying to use this tool but I don't know how to use it. I followed the instructions and downloaded Python. but none of the other steps are working for me. I have my backup on my PC and am trying to extract my files into normal formats as I wish to move away from the LG platform. Thank you.
---------- Post added at 07:20 AM ---------- Previous post was at 06:57 AM ----------
Nevermind, I found a work around.
For anyone else in the future wondering: If you select backup with the app on the phone and backup to the internal memory, it will create .pdf files of all text/images in internal storage>>Quickmemo+. This doesn't help to retrieve the audio or video recordings, but at least that way you can retrieve your text files/images.
Tried script on old backup, failed with error.
Mysak0CZ said:
Hello, recently I looked into LG backup and restore application using static analysis.
Note: this is based on version lg bnr 8.0.26 from 2020-02-13, tested with 7.0.20 from 2008-12-31
Here is the extract tool:
And here is the technical part (you don't need to read this, if you just want to extract your data):
The file contains 2 headers (one of them is referred to as "new")
These are XML documents encrypted using AES/ECB/PKCS5Padding + SHA256 and null IV with static password
Code:
cd562f61-5399-3978-ac76-7c54b0508010
...
[/CODE]
All applications are tar files that contain 2 folders:
app - contains the apk file
data - application's internal data
All other items have:
StartOffset - Offset inside file
DataSize - Self explanatory
FileName - Not all have this, but most do; it is the original name of the backuped file.
The files are not compressed nor encrypted.
Footnote: This is my first post on XDA, so sorry for any mistakes I made (including my english) and feel free to correct me/give me advice.
Click to expand...
Click to collapse
My first post as well. Thanks for the script. I ran it on an old backup I was hoping to extract contacts from. See excerpt with error below.
C:\Temp\LG G4\Backup>python extract.py LGBackup_190218.lbf
[+] Backup size: 597864605
Parsing...
Reading header
[*] Header start offset: 597792768
[*] Header length: 71824
[+] Header read OK
Reading header (new)
[*] Header start offset: 7021235167904623918
Traceback (most recent call last):
File "extract.py", line 125, in <module>
root = readHeader(True)
File "extract.py", line 99, in readHeader
size = getBackupInfoSize(isNew)
File "extract.py", line 88, in getBackupInfoSize
raise Exception("Invalid header length.")
Exception: Invalid header length.
C:\Temp\LG G4\Backup>
Click to expand...
Click to collapse
I'm a complete newbie with Python, haven't done any coding in over a decade. Where do I go from here?
Ninja_Toad said:
My first post as well. Thanks for the script. I ran it on an old backup I was hoping to extract contacts from. See excerpt with error below.
I'm a complete newbie with Python, haven't done any coding in over a decade. Where do I go from here?
Click to expand...
Click to collapse
Hi! Your backup seem to be at least partially corrupted. (The new header position seems to start at 7021235167904623918, however the file is only 597864605 bytes long). Fortunatly there are two headers in the backup and the old one seems to be ok (You got "[+] Header read OK" for the old header). So you can try ignoring the "new" header.
To ignore the faulty header just set the relevant "FORCE_*_HEADER" to True in the script header. In your case just set "FORCE_OLD_HEADER = True".
Also note, that extracting contacts in readable format is beyond this script's current scope. This can however be done manually. After you get your data successfully extracted, look for export/data/contacts_DB file. This is a TAR file in which you need to find file "data/data/user/0/com.android.providers.contacts/databases/contacts2.db". This is the contact database as your phone stores it. To actually extract the contacts, you will need to find a way to convert the database to some other format (I have't looked into how the contacts are stored, you will need to do your own googling)
Files extracted but...
hello guys,
I also managed to extract my files using the hack mentioned above (force old headers to TRUE) since by using the default script i was also getting a similar error as above.
However now all of my files (pictures in particular) are corrupted and i cannot seem to be able to open them using several editors.
Anyone else had the same issue?
Thanks in advance!
serlal said:
hello guys,
I also managed to extract my files using the hack mentioned above (force old headers to TRUE) since by using the default script i was also getting a similar error as above.
However now all of my files (pictures in particular) are corrupted and i cannot seem to be able to open them using several editors.
Anyone else had the same issue?
Thanks in advance!
Click to expand...
Click to collapse
Yes I have exactly same issue as you have. All exported files cannot be read.
Hello @tking2000 and @serlal, this script was written based on versions that are available to me. There are two possibilities of what is happening:
1) The file was badly corruped, however the old header stayed itact. (This is the likely case, if you are trying this tool because your phone refused to restore the backup)
2) The file was made by version that does something differently and the script doesn't account for it.
In the first case there is nothing anyone can do (short of trying some corrupted file recovery software hoping the corruption is only minor)
In the second case, you can try sending me a PM with following information, so I can have a look and confirm whether this is truly the case (and improve the script)
1) The script's output
2a) The <Version> and <Attr Name="settingVersionName"> from the <Common> part of the header
2b) If you aren't concerned about privacy or it is just easier for you: just send the whole header file
3) The version of the com.lge.bnr app (LG Mobile Switch); this is not necessary, but may help me
Feel free to censor anything you are sending with [CENSORED] (like your email or filenames)
I have no experience with python, but I have one of these lbf files and I'm trying to figure out how to make it work and all I get is couldn't get library pycryptodome. How do I get Python 3.8.3 to get this library?
Multipart backups
Hello everyone,
I just finished an update, that should enable the tool to extract backups that have multiple files.
If you received "Exception: Invalid header length." in the past and your backup has multiple files, try the script now; your backups are likely to extract correctly now.
This is the likely case for: @Ninja_Toad @serlal and @tking2000
This was made possible thanks to @cheesyzombie who sent me the data I asked for.
@kamikazkid, You don't need to know Python to use this tool, the easiest way is to enter the commands in quotes on the github page into the console (To open the console Shift+Right click in the folder with backups and choose "Open command window here").
Just:
Download the zip on the github page (Under "Code" button)
Make sure the extract.py and your backup are in the same folder
Follow the usage, skip the 4th step and replace xxxxxx to match your backup name.
I figured it out, it was something with windows 10 and the version of python installation I used. Microsoft garbaging things up as usual, thanks for the help though. Thanks, I recovered my files from my old phone successfully.
I have no experience withc++, but I have one of these lbf files and I'm trying to figure out how to make it work, it is a great this forum
Is there a way to import the _DB files (contacts, messages, call logs etc.) back to a phone?
I asked before:
https://forum.xda-developers.com/lg-g3/general/restore-backup-lbf-lineage-17-t4172483#post83644281
Thank you so much for this! I was able to extract the Messaging data from an LG G8 ThinQ. Does anybody know the easiest way to view that data on my PC?
Hi, you can follow these steps to extract data fron _db files:
1) Send the _db files like contacts_db, messages_db to any android phone
2) Double tap on the file and open it using File Manager (which is by default on Mi phones)
3) The file will now open as a folder. Continue opening the folders until you reach to the last file.
4) The last file is the database file. It won't open directly. Select the file and extract the file.
5) Now transfer the file to computer and you can now view it using SQLite DB Browser if its a contacts file or you can use any VMG reader if its a messages file.
Hope this helps you.
Hi @Mysak0CZ, just follow your instructions on GitHub but when I run the command on Command Prompt, nothing happens. Just like this:
F:\>python -m pip install pycryptodome
F:\>python extract.py LGBackup-201031.lbf
When I enter nothing happened. Kindly help to advise for this.
Thank you
Problem in powershell
Mysak0CZ said:
Hi! Your backup seem to be at least partially corrupted. (The new header position seems to start at 7021235167904623918, however the file is only 597864605 bytes long). Fortunatly there are two headers in the backup and the old one seems to be ok (You got "[+] Header read OK" for the old header). So you can try ignoring the "new" header.
To ignore the faulty header just set the relevant "FORCE_*_HEADER" to True in the script header. In your case just set "FORCE_OLD_HEADER = True".
Also note, that extracting contacts in readable format is beyond this script's current scope. This can however be done manually. After you get your data successfully extracted, look for export/data/contacts_DB file. This is a TAR file in which you need to find file "data/data/user/0/com.android.providers.contacts/databases/contacts2.db". This is the contact database as your phone stores it. To actually extract the contacts, you will need to find a way to convert the database to some other format (I have't looked into how the contacts are stored, you will need to do your own googling)
Click to expand...
Click to collapse
Hi, I am trying to extract via your method but in the power shell in sending the command "python extract.py LGBackup_1x0x07.lbf" it does not execute and brings me back the initial path line. Do you have solutions in this regard? Thanks in advance
Ps. I use Win10
Worked so well. I only needed to recover images and audio recordings after realizing that LG backup tool won't restore on other brand phones.
Thank you!!
Mysak0CZ said:
Hello, recently I looked into LG backup and restore application using static analysis.
Note: this is based on version lg bnr 8.0.26 from 2020-02-13, tested with 7.0.20 from 2008-12-31
Here is the extract tool:
https://github.com/Mysak0CZ/LBFtool
And here is the technical part (you don't need to read this, if you just want to extract your data):
The file contains 2 headers (one of them is referred to as "new")
These are XML documents encrypted using AES/ECB/PKCS5Padding + SHA256 and null IV with static password
Code:
cd562f61-5399-3978-ac76-7c54b0508010
And they look like this:
Code:
<?xml version="1.0" encoding="UTF-8"?>
<BackUp>
<Common>
<Name>LGBackup_200217_1</Name> <!-- Name of the backup file -->
<Id>Randomly generated UUID</Id> <!-- Id of the backup file -->
<IsNew>True</IsNew> <!-- IsNew is same in new and old header -->
<Version>1.00</Version>
<Attr Name="product_model">LM-X525</Attr>
<Attr Name="os_version">9</Attr>
<!-- And other attributes including google email -->
</Common>
<BackupItem Category="settings_DB" StartOffset="59392" DataSize="136626" />
<BackupItem Category="MEDIA_DB" StartOffset="901169" DataSize="78393344" FileName="external.db" />
<!-- Other backuped items -->
</BackUp>
The "new" header has more information than original header, but same structure.
The attribute "IsNew" signals the presence of the new header. (and is True in both files)
All numbers inside file are in Big-endian.
And this is the structure of the file (decimal position):
Code:
0: Unknown - 0x00 for me
1: Int32 - size of the new header
5: Int64 - offset to the new header (usually 13)
13 ... EOF-33: data
EOF-33: 20 bytes - SHA1 hash of the password (if present, otherwise still data)
EOF-13: Int64 - offset to the (old) header
EOF-5: Int32 - size of the (old) header
EOF-1: Bool - Lock byte - the file is (0x01) or isn't (0x00) protected by password
The password is only checked during import, not used to encrypt data (so to remove password, just set the last byte to 0)
Every "BackupItem" entry has a "Category".
Special case is Category="APPLICATION" with this structure:
Code:
<BackupItem Category="APPLICATION">
<APPLICATION Name="com.lge.ime" StartOffset="19221586" DataSize="2385983" Package="com.lge.ime">
<Attr Name="com.lge.ime">com.lge.ime</Attr>
</APPLICATION>
<APPLICATION Name="org.exmple.app" StartOffset="1429652009" DataSize="3420160" FileName="org.exmple.app.tar" Package="org.exmple.app">
<Attr Name="org.exmple.app">org.exmple.app</Attr>
</APPLICATION>
<!-- ... -->
</BackupItem>
All applications are tar files that contain 2 folders:
app - contains the apk file
data - application's internal data
All other items have:
StartOffset - Offset inside file
DataSize - Self explanatory
FileName - Not all have this, but most do; it is the original name of the backuped file.
The files are not compressed nor encrypted.
Footnote: This is my first post on XDA, so sorry for any mistakes I made (including my english) and feel free to correct me/give me advice.
Click to expand...
Click to collapse
ok I am so lost. And here is the technical part (you don't need to read this, if you just want to extract your data):
that statement would make it appear easy. well the link only takes you to the code, there is no tool and nothing with "new" as header. so ive tried deciphering what to do for hours now and i cant figure npthing out. rooted and flashed my lgv20 and did a backup with lg bridge beforehand. trying to restore and phone connects to computer fine but doesnt regster in lg bridge so that is why i need to extract .lbf file. can someone make this easier?
hello..it ended up successful.but there is no jpeg or mp4 file on DCIM folder..how can i restore them?should i do somthing with the script? i don't know pyton