Database Info

expand a nandroid img file

for those who do not know how to do this yet, follow directions given here.
Just took this info from the web. None of this except for the guide is written by me.
I am not responsible for anything please do not contact me if a green sludge starts attacking your neighborhood eating one person at a time.
************************************************************
************************************************************
************************************************************
http://code.google.com/p/unyaffs/downloads/list
made for linux, but if you know a bit of c, you can probably change it to work with windows, maybe I will do that if I have some time this week.
********OR********
http://jiggawatt.org/badc0de/android/unyaffs-x86-win.zip
or http://www.4shared.com/get/fgryPnPx/unyaffs-windows.html
for windows..... instructions below apply for both
if not included in the zips above, cygwin1.dll and unyaffs.exe are included as an attachment of this reply.
open up a command line, in windows vista/7 just press start and then type cmd press enter and change the directory to the folder with the unyaffs and the img in it. in xp press start>run>cmd... then the same
the usage is: unyaffs /path/to/img
example:
(if unyaffs.exe and system.img are both in c:\temp)
"c:\temp> unyaffs /temp/system.img"
note: the '/' is the way linux determines the pathname of a file and since it is built using cygwin(a file allowing for windows) it is preferrable that the path to the actual file is in unix POSIX format ie: / instead of \ between folders and files.
*********WARNING************
this will extract to directory that unyaffs binary is in.

That's so weird -- I was gonna post the same thing tonight. Thx for writing this up. It's odd that more people don't talk or ask about this tool!

I remember reading this stuff back when I got my G1, and tonight I was trolling the themes and apps section cause I couldnt sleep. (which I rarely look at)
I was baffled that this wasnt common knowledge. I figured if there was no threads about it that people just knew how and didnt have any problems with it. My mistake lol...
enjoy.

can you please explain what is it? what it does? thanks

ericizzy1 said:
can you please explain what is it? what it does? thanks
Click to expand...
Click to collapse
Umm, it expands nandroid img files not sure how else to explain it. Clockworkmod backups are in the img file format. This process allows you to extract those files to get stuff out of them
Sent from my SPH-D700 using XDA App

When you create a backup in Clockwork recovery, it leaves you with a folder like this:
2010-11-18.08.52.49
As you can prob figure out, I made that backup on 11/18 at 8:52am
Inside that folder are .img files, such as: data.img and system.img
Those files basically contain your data (like apps you've installed from market) and system (..system apps that might've come with the phone).
In the orig post, he's explaining how to extract the files from these large .img files.
To give an example, I had a couple of corrupted clockwork backups, and wanted to ONLY pull out some database files (like text messages and call logs) which were not corrupted. Even tho the backup was messed up, I could still extract the files I needed by using the method above, then copied them manually to my phone.

decalex said:
When you create a backup in Clockwork recovery, it leaves you with a folder like this:
2010-11-18.08.52.49
As you can prob figure out, I made that backup on 11/18 at 8:52am
Inside that folder are .img files, such as: data.img and system.img
Those files basically contain your data (like apps you've installed from market) and system (..system apps that might've come with the phone).
In the orig post, he's explaining how to extract the files from these large .img files.
To give an example, I had a couple of corrupted clockwork backups, and wanted to ONLY pull out some database files (like text messages and call logs) which were not corrupted. Even tho the backup was messed up, I could still extract the files I needed by using the method above, then copied them manually to my phone.
Click to expand...
Click to collapse
thx
Sent from my SPH-D700 using XDA App

smeyerhuky said:
Umm, it expands nandroid img files not sure how else to explain it. Clockworkmod backups are in the img file format. This process allows you to extract those files to get stuff out of them
Sent from my SPH-D700 using XDA App
Click to expand...
Click to collapse
Right, like if you wanted a file or a few files from clockworkmod img backup, then this process would give you an extended file system, other than what you would normally use to extract the img files. Just trying to understand this so, please correct me if im wrong.... I've been trying to get more knowledge of this, so im still learning.
I prolly sound like a complete idiot, lol...

Sweet! Thanks for this man. I'd been looking for how to do this for some time and kept ending up learning about emulation or some other tangential topic. This is the basic functionality I was looking for. Thanks for posting!

Np. I'm working on writing some stuff on top of the source code that should allow you to browse and pull just one our more file
Its been a but since I looked theory source code so it might take a while... Maybe in between semesters?
Sent from my SPH-D700 using XDA App

Dode you saved my as** thx big thx

Great, now how can I re-compress back to .img after I've made the changes that I want?
EDIT: Nevermind. I just made a zip of the new system folder and flashed it in CWM.

This worked great for me.
Thanks,
Bud

Man, I do not know how to thank you. I was able to recover some important stuff off of my cwm recovery backup. THANKS!!

[UTILITY][10-8-2012] APK Manager for Mac OS X, Intel only, v3.0

So almost everyone here should be pretty familiar with android, and that means you've probably heard of (and maybe even used yourself) the windows tool APK Manager by fellow XDA member Daneshm90.
Early last year I started to port the windows 4.9 version to mac using the outdated linux script for a base. And I had some early success at the time with simply using macports to install sox, optipng, etc. And then my computer crashed (double drive failure in July, 2011) and I gave up for the time being. Recently though I was excited to find that someone else tried to do the same thing, XDA member MAD Industries released a very basic port for mac.
Sadly though, a quick diff of his script and the outdated linux version shows that it was still woefully behind the windows version. And so I started working (once again) on my own port of APK Manager for Mac OS X, with my goal to not only bring this current to the windows feature set, but exceed it in many ways.
I've built in 32/64-bit architecture checking, and compiled and include the required missing binaries to re-enable optimizing png images and ogg files. I'm also including a .terminal file for easily configuring the default display, and a copy of my NEW favorite fixed-width font, Bitstream Vera Sans Mono.
And now some notes, and requests:
PLEASE note: this is currently INTEL only, but it supports both 32-bit and 64-bit all-in-one with architecture checking built-in to the script.
PLEASE note: if your computer is running OS X 10.7.X, "Lion" then it does not include a java runtime by default, please download it here first: http://support.apple.com/kb/DL1421
PLEASE note: due to the length of some of the strings used in the menus, etc., in order to not need to scroll your terminal window, you must adjust your default columns and rows settings. You can set this automatically by using the include "JocelynCustom.terminal" file, and you can also find simple instructions to set this yourself in the README.txt
PLEASE read the included readme for installation and very basic (for right now) usage instructions. I welcome any and all feedback on improvements for the readme file.
PLEASE report any and all bugs or generally wonky behavior, preferrably by posting here in this thread, or by posting a comment back on my blog.
LASTLY, please if you re-post this anywhere else, including blogs, other forums, etc. I would greatly appreciate if you would at least give me credit, and link back to my blog (http://girlintroverted.wordpress.com/) or at the very least, link back here to this post.
WHEN UPGRADING, DO NOT COPY OVER THE CONTENTS OF THE "other" directory or the root directory unless you absolutely know what you're doing.
IMPORTANT: Please see the second post in this thread for update notes, issues, concerns, etc. thanks
Download apk manager v2.2.5 here: http://www.mediafire.com/?o4idx4en9ldnwx4
Download migration tool v1.4 here: http://www.mediafire.com/?hbhftv9195tk86t
v3.0 BETA INFORMATION
May 11, 2012 - beta version of v3.0 is online now on github, if you want to help test it, I suggest backing up your old apk_manager directory entirely, and then going here, and reading the simple install instructions:
https://github.com/jocelynmallon/apkmanager
The basic install is pretty simple though: open terminal, cd into whatever directory you want the apkmanager folder to be created in, and run:
cd into whatever directory you want the apkmanager folder to be created in, and run:
Code:
git clone git://github.com/jocelynmallon/apkmanager.git
launch APKManager.app and follow all the instructions.
Also, git is utterly retarded when it comes to the OSX 'icon' resource files (for custom folder icons) so by default, the folder icon isn't included with 3.0+, but you can download it and apply it yourself still. Link is on the wiki here:
https://github.com/jocelynmallon/apkmanager/wiki/Basic-Information
and thank you very much to anyone that helps test this!
Cheers everyone

IMPORTANT!
5-11-2012 - v3.0 BETA notes
Some notes about the new v3.0 beta released today.
First, please backup your entire old apk_manager directory before you install/clone the 3.0 git repo. My OCD got the better of me when it came to releasing 3.0, and I ended up trashing my entire git repo/history (from v1.0-2.2.5) and starting a new history when I was ~90% through the 3.0 dev process. Because of that, your old directory will cause problems, DO NOT CLONE/INSTALL 3.0b into your old 1.X/2.X directory.
Second, for those comfortable with the command line that really want to help test APK Manager, there are four 'hidden' menu options in most menus (main menu, debug/settings menu, auto-updates menu, signing menu, clean menu) that turn on a variety of debugging options.
96) turns on 'trace mode' (set -x) for the entire script, once the main loop starts. This will make the script basically unusable for people, please only use this if you can reproduce a bug, and are saving the entire terminal output to submit an issue/error log.
97) turns on 'verbose mode' (set -v) for the entire script, once the main loop starts. This will cause some display issues, but the script will remain mostly usable.
98) turns on 'error mode' (set -e) for the entire script, once the main loop starts. This basically causes the script to exit if any condition/test/command fails. Not really useful on its own, but very useful in combination with option 96 or 97.
99) turns on a very (and I do mean VERY) simple debug information display between main apk manager header and most menus. This shows the current PID, the last exit/return code (Currently non-functional) and the last directory change.
So, if you can reproduce any bugs, please close apk manager, and delete your log.txt. Then open APK Manager again, and on the main menu, choose option 98, then next choose option 96. Then reproduce the bug, and copy/paste the entire terminal output into a log on pastebin.com, gist, etc., and send it to me. And of course, if you feel confident debugging and fixing it yourself, then fork the apkmanager repo, and send me a pull request with your changes.
4-14-2012 IMPORTANT!
An update to the issue below. This has been resolved in version 2.1 of APK Manager, and should not present any issues moving forward. However, because the old user settings and private keys were stored locally, inside the apk_manager directory, in order to save these before upgrading, I've created a migration utility/script, that needs to be run before you upgrade.
The link to this migration utility is in the original post in this thread, and the tool is also included in the APK Manager disk image starting with versions 2.1.1.
To run this tool:
mount the "Migration Utility.dmg"
Copy Migrate.app to your old root apk_manager directory.
Copy Migrate.sh to your old apk_manager/other directory.
From your old root apk_manager directory, run Migrate.app
Verify that the script completed without any errors
OPTIONAL - for those paranoid, you can double check that it worked manually. Execute the following commands in a terminal:
Code:
cd $HOME
ls -la | grep .apkmanager
cd .apkmanager
ls -la
4-13-2012 IMPORTANT!
AFFECTS all versions of apk manager prior to v2.1 (forthcoming release as of 4/13/2012.)
If you use any advanced signing functionality, you must backup your existing private key(s) and keystore(s) before upgrading to a new version of APK Manager.
Forthcoming version 2.1 will have a separate, automated, one-time backup & upgrade script to run, and all future versions will have automatic backup of key(s) and keystore(s) built in.
By default they get created and saved in the apk_manager/other/.keystores directory. This is a hidden directory by default. You can do one of several different things to unhide/view this directory, and backup your keystore(s).
I'm outlining one simple method here.
Enable 'show hidden files' in Finder. From a terminal copy and paste the following two lines:
Code:
defaults write com.apple.finder AppleShowAllFiles TRUE
killall Finder
Finder will restart after those two lines, browse to your apk_manager/other directory and make sure you see the .keystores folder (it should be dimmed compared to regular folders.)
Copy this folder and all contents somewhere safe (desktop, user home directory, user backup folder, dropbox, etc.)
Proceed with your upgrade (usually just delete the old apk_manager folder and copy over the new one, you might need to manually move project folders, etc. too)
Copy/Paste your saved .keystores directory into your new apk_manager/other folder
Test advanced signing functionality to ensure everything still works.
Once you've verified everything is working, run the following code to turn off viewing all files/folders in finder:
Code:
defaults write com.apple.finder AppleShowAllFiles FALSE
killall Finder

CHANGELOG
10-8-2012
fixed a couple bugs that were introduced with some changes to homebrew and pngcrush
updated smali.jar and baksmali.jar to version 1.4.0
fixed a few various typos
updated the built-in pngout downloader to download the newest version (May 30, 2012)
changed the "view changelog" option on the debug menu to use "git log" if you used homebrew & git to install
added the ability to forcibly reset the "home" directory by holding the command/apple key when launching APKManager.app
removed all the little "b/beta" tags I could find since this seems good enough for a "real" 3.0 release finally
4-28-2012
v2.2.5 uploaded! - Fixed "adb log" function. I have no idea how or when I broke this, but it was yet another boneheaded mistake.
4-28-2012
v2.2.4 uploaded! - Fixed a bug in the "batch sign with private key" option that would cause the signing process to fail if the key and keystore password weren't identical.
4-24-2012
v2.2.3 uploaded! - I'm retarded and somehow failed to include zipalign in every release. Fixed.
Added zipalign path information to debug/binary info screen.
Replaced all instances/uses of 'which' with 'command -v' instead.
Fixed an obscure bug that could cause infinite loop in the function to check and set png optimization tool.
4-20-2012
v2.2.2 uploaded! - Updated 32-bit optipng to v0.7.1 (Somehow I forgot to update the 32-bit binary when I last updated the 64-bit version)
Minor changes to the way debug/binary version info is scraped and displayed. I personally think this new method is cleaner, and more consistent.
Fixed bug in binary display screen (debug menu, option 2) whereby pressing "any key" did not actually close the screen and return to debug menu.
Updated other/Migrate.sh to dynamically pull the APKManager version banner from whichever other/Script.sh version is present, when its run. This should mean the end of unnecessary updates to Migration Utility.dmg, purely to bump the APK Manager version information.
4-17-2012
v2.2.1 uploaded! - Updated pngcrush to 1.7.27 (yep, that's it, lol)
4-16-2012
v2.2 uploaded! - Integrated color scheme selection into one single script, and removed the separate 'black text on light backgrounds' script file since its no longer needed.
Added new option to the "Clean" menu to reset/change color scheme.
Fixed a few obscure bugs with the apktool selection menu.
Added the actual apktool jar file in use to the debug menu information.
Changed the way the default apktool.jar symlink is created in order to prevent APK Manager from ever linking 'mod' versions by default.
Added a new custom icon for the apk_manager folder (CC by-nc-sa) Jocelyn Mallon, 2012
4-15-2012
v2.1.2 uploaded! - Fixed a bug causing decompile/compile errors due to old framework files not being deleted when changing apktool versions.
Added 2 new apktool versions, an upgraded 1.4.2 version, and 1.4.3 ICS Mod. For most instances, 1.4.3 is still the best option to use.
4-14-2012
v2.1.1 uploaded! - Fix brain-dead mistake regarding user settings & private key migration.
Packaged "Migration Utility.dmg" with APK Manager for easier upgrading from previous versions.
4-14-2012
v2.1 uploaded! - Migrated user settings and private keys to new location ($HOME/.apkmanager) to ensure preservation when upgrading.
Bumped copyright/CC info in various files to 2012
4-12-2012
v2.0.1 uploaded! - Quick hotfix for directory check/creation for advanced signing options.
4-12-2012
v2.0 uploaded! - Re-packaged into a DMG (mac disk image) file to alleviate potential unzip/setup errors.
Updated a lot of the various binaries to latest versions (adb, sox, optipng, pngcrush, etc.)
Added a new option to view java source, this has a lot of potential issues/caveats so please read the CHANGELOG.txt for more information.
Re-worked and (hopefully) simplified debug menu for the millionth time.
Added Chocolat.app to list of text editors/viewers http://chocolatapp.com/
Lots of stuff I probably missed, please read README.txt and CHANGELOG.txt for full details.
10-18-2011
v1.2.1 uploaded! - Mostly bugfixes for a few really stupid bugs I introduced in 1.2. If you had problems running APK Manager because of "missing programs" or "not in PATH" errors (from a clean install) it was my crappy coding skills, and it should be fixed now.
I honestly don't think I added any new features, just bug fixes and code cleanups.
10-10-2011
v1.2 uploaded! - Added another png tool option, pngout. Unfortunately, due to the licensing restrictions, it is illegal to redistribute the actual pngout binary, so instead, APK Manager will download and install the binary upon first attempted use of pngout. This means that if you never use pngout, it will never be downloaded.
Completely re-worked the adblog.txt function, now completely automated, and, with much clearer instructions.
Added two additional text editor/log viewer options: sublime text 2, and vico. Just like all the rest, they both require command line support to be installed in order to function.
Cleaned up adb shell, ddms, and draw9patch options so that they all now automatically close the new tabs that they open, and when necessary, will kill and re-start adb.
1.2 might be the last release. I had fun learning how to write shell script for this, but the overwhelming lack of interest in this project has already started to make it no fun for me. And well, its already basically good enough for everyday use, there's just not much else to keep adding.
10-8-2011
v1.1 uploaded! Please see Changelog.txt for full details.
switched some code back to a "portable" syntax and changed back to /bin/sh env in case users have a login shell other than bash set.
cleaned up and re-arranged debug menu again, it should be a lot less cluttered, and easier to use overall now.
fixed a few bugs in the startup check, hopefully nobody ever encountered them, they were pretty embarrassing, lol.
added option to choose between optipng and pngcrush for png optimization functions, and the setting is persistent between launches of APK Manager.

Good work man. Let me know if you'd like to collaborate or borrow ideas. I wrote Android-Utility and it is currently on v1.0.2 and development is active.
Tommy
Android-Utility Linky

tommytomatoe said:
Good work man. Let me know if you'd like to collaborate or borrow ideas. I wrote Android-Utility and it is currently on v1.0.2 and development is active.
Tommy
Android-Utility Linky
Click to expand...
Click to collapse
Hi Tommy,
Thanks for the compliment and the offer to work together. I'm not really sure how much else I'm going to add, but I did take a nod from your tool and add the option to choose between optipng and pngcrush. Mostly I added that just because optipng hasn't been updated in forever, and pngcrush just had an update last month.
So I compiled both 32-bit and 64-bit specific binaries of the newest pngcrush (v1.7.17) and included them with v1.1 of APK Manager.
Anyway, your tool is pretty amazing, I'm rather intimidated by how many features you have, lol.
Thanks again, cheers

s0niqu3 said:
Hi Tommy,
Thanks for the compliment and the offer to work together. I'm not really sure how much else I'm going to add, but I did take a nod from your tool and add the option to choose between optipng and pngcrush. Mostly I added that just because optipng hasn't been updated in forever, and pngcrush just had an update last month.
So I compiled both 32-bit and 64-bit specific binaries of the newest pngcrush (v1.7.17) and included them with v1.1 of APK Manager.
Anyway, your tool is pretty amazing, I'm rather intimidated by how many features you have, lol.
Thanks again, cheers
Click to expand...
Click to collapse
Back at you man. Yeah last week I went crazy. I went from version 1.0.2 to version 1.0.7 in three days. There's always something new that can be added.
Don't be intimidated! AU and apkmanager each have their places. I aimed to create a tool that encompasses everything to do with Android while apk manager focuses on modifying apks. Surprisingly however while I wrote the tool for mac users, I've had mote Linux users for au. But the mac community needed some love and attention, and now it has two tools to choose from
tapatalk signature here. lovely.

version 1.2 released, this might be the last release, I can't think of anything else to add really, and well, nobody really cares either, lol.

great work

1.2.1 uploaded with a few bugfixes, please re-download, and I'm really sorry I didn't catch them sooner until someone messaged me today.

Wow, this is amazing! So glad I don't need to boot into WinXP just to use apkmanager. I just downloaded the newest version, followed all instructions, and everything is working perfectly. Thanks for including all the extras such as terminal settings. Great for semi-noobs like myself.

Can't wait to try this tool out. Thanks for all the hard work in putting it together.

this not working anymore? Says applet version unsupported

Nhialor said:
this not working anymore? Says applet version unsupported
Click to expand...
Click to collapse
Yes same here

Hey,
Sorry, I'm pretty much stopped doing any work on android, but if you can describe the error in more detail, I'll look into it and try and fix the problem.
Sorry about that.

s0niqu3 said:
Hey,
Sorry, I'm pretty much stopped doing any work on android, but if you can describe the error in more detail, I'll look into it and try and fix the problem.
Sorry about that.
Click to expand...
Click to collapse
OK cool.
So it's not possible to start the setup file, because it's unsupported and starting the apk manager directly is also not possible because it has to be in the root directory
Gesendet von meinem Desire S mit Tapatalk

Kolo39 said:
OK cool.
So it's not possible to start the setup file, because it's unsupported and starting the apk manager directly is also not possible because it has to be in the root directory
Gesendet von meinem Desire S mit Tapatalk
Click to expand...
Click to collapse
Hi again,
Ok, so first, you can set the root directory manually if you open up the APKManager.command file with a text editor.
Look for the following:
Code:
# Set path to use this script in .command mode
# ENTER the POSIX path to your root APK Manager
# Directory between the quotation marks
MAINDIR=""
and inside the MAINDIR="" paste the full path to your apk_manager directory
for example, mine looks like:
Code:
MAINDIR="/Volumes/LappyHD/Android/apk_manager"
then you need to manually make the entire directory executable, so open a terminal window, and type:
Code:
chmod -R ug+x /Volumes/LappyHD/Android/apk_manager
But replace the path with the correct path for your setup (the same one you used in the .command file.)
Of course, that's not the solution for everyone, and I'd like to figure out why the setup script isn't working. I tried the setup script myself today on 10.7.3, x86_64 and it still works fine for me, so can you post some information about your mac, version information, java version installed, etc.?
sorry again about this, but hopefully I can get this fixed asap.
cheers,

s0niqu3 said:
Hi again,
Ok, so first, you can set the root directory manually if you open up the APKManager.command file with a text editor.
Look for the following:
Code:
# Set path to use this script in .command mode
# ENTER the POSIX path to your root APK Manager
# Directory between the quotation marks
MAINDIR=""
and inside the MAINDIR="" paste the full path to your apk_manager directory
for example, mine looks like:
Code:
MAINDIR="/Volumes/LappyHD/Android/apk_manager"
then you need to manually make the entire directory executable, so open a terminal window, and type:
Code:
chmod -R ug+x /Volumes/LappyHD/Android/apk_manager
But replace the path with the correct path for your setup (the same one you used in the .command file.)
Of course, that's not the solution for everyone, and I'd like to figure out why the setup script isn't working. I tried the setup script myself today on 10.7.3, x86_64 and it still works fine for me, so can you post some information about your mac, version information, java version installed, etc.?
sorry again about this, but hopefully I can get this fixed asap.
cheers,
Click to expand...
Click to collapse
I try this in the next days
My Mac Book Pro end 2011 has all updates installed (Lion 10.7.3)
---------- Post added at 05:17 PM ---------- Previous post was at 05:07 PM ----------
Ahh okay I fixed it!!!
The Unarchiver was not able to extract all files from the zip. So files doesn't work. But by using OSX Unarchiver everything works
Thanks a lot for this port!!

Kolo39 said:
I try this in the next days
My Mac Book Pro end 2011 has all updates installed (Lion 10.7.3)
---------- Post added at 05:17 PM ---------- Previous post was at 05:07 PM ----------
Ahh okay I fixed it!!!
The Unarchiver was not able to extract all files from the zip. So files doesn't work. But by using OSX Unarchiver everything works
Thanks a lot for this port!!
Click to expand...
Click to collapse
Hi again,
I'm glad you got it figured out. I think I'll start packaging it as a DMG file instead of a zip, and just made a custom background image to explain about copying the apk_manager folder somewhere.
I've also been working on updating it a little bit the past couple days, and will have a new version soon.
Cheers!

Uploaded version 2.0 update!
This adds a couple new features, some bug fixes, and re-packages the entire thing in a disk image, to hopefully alleviate those unzip/setup errors.
Please see OP for download link, and second post for changelog.

OP update with v2.0.1, hotfix for advanced signing options, please download the newest version.

[TOOL] Merjapp - merge whatsapp databases

If you ever lost your phone due to an accident, or just had to use a temporary phone for some reason...
And you're one of those who hates to lose history, even if you know you hardly look at it...
This tool is for you.
Pre-requisites:
Neither of your phones where your whatsapp is installed need be rooted.
You must enable USB debugging on both devices
You must have Android SDK, we will be using the Android Debug Bridge (ADB)
Have Android Backup Extractor ready, we will use it.
Instructions:
Connect the phone which holds your old database to your computer, and enable USB debugging.
Use ADB to make a backup of the whatsapp database
Code:
adb.exe backup -f whatsapp.ab com.whatsapp
Use 'Android Backup Extractor' to extract whatsapp.ab to whatsapp.tar
Open whatsapp.tar with 7zip (or any other compression utility) and extract the file called 'msgstore.db'
Rename the file to 'old.db'
Repeat the above steps for your new phone, but rename the file to 'new.db'
Run merjapp in the following fashion:
Code:
java.exe -jar merjapp.jar -old old.db -new new.db
You can also use the -simulation mode if you like, it will not create any output database, and you will be able to read the log created in the working directory.
If you want extra debug information you can use the -debug mode.
If everything went correctly and "Done!' is displayed, rename output.db (which is your merged database) to msgstore.db
Connect your new phone to your computer, open the Whatsapp/Databases directory on the internal SD card and backup its contents to some other location.
Copy msgstore.db created in the previous steps to this directory
Clear whatsapp's data from your new phone and force stop it.
Open whatsapp and when prompted, elect to restore from backup, and voila!
Download
Download merjapp v141201 (runnable jar file)

Backup
Backup

Just registered to say thank you. It works!
Official whatsapp method wasn't working for me: "unable to recover chat history". So your tool saved me. Thanks!

namnoops said:
If you ever lost your phone due to an accident, or just had to use a temporary phone for some reason...
And you're one of those who hates to lose history, even if you know you hardly look at it...
This tool is for you.
Pre-requisites:
Neither of your phones where your whatsapp is installed need be rooted.
You must enable USB debugging on both devices
You must have Android SDK, we will be using the Android Debug Bridge (ADB)
Have Android Backup Extractor ready, we will use it.
Instructions:
Connect the phone which holds your old database to your computer, and enable USB debugging.
Use ADB to make a backup of the whatsapp database
Code:
adb.exe backup -f whatsapp.ab com.whatsapp
Use 'Android Backup Extractor' to extract whatsapp.ab to whatsapp.tar
Open whatsapp.tar with 7zip (or any other compression utility) and extract the file called 'msgstore.db'
Rename the file to 'old.db'
Repeat the above steps for your new phone, but rename the file to 'new.db'
Make back-ups of both old.db and new.db, as merjapp will make changes in them
Run merjapp in the following fashion:
Code:
java.exe -jar merjapp.jar -old old.db -new new.db
You can also use the -simulation mode if you like, it will not change your databases and you will be able to read the log created in the working directory.
If you want extra debug information you can use the -debug mode.
If everything went correctly and "Done!' is displayed, rename old.db (which is your merged database now) to msgstore.db
Connect your new phone to your computer, open the Whatsapp/Databases directory on the internal SD card and move its contents to some other location
Copy msgstore.db created in the previous steps to this directory
Clear whatsapp's data from your new phone and force stop it.
Open whatsapp and when prompted, elect to restore from backup, and voila!
Download
Download merjapp v141005 (runnable jar file)
Click to expand...
Click to collapse
A pretty neat tool!
I was trying to merge two databases. It seemed that everything went okay and all with the "Done!" message. However, there seems to be no changes made two any of the database. Their filesizes remains the same for old.db and new.db.
I tried loaded the old.db anyway since you said that is the merged one. However, I am getting only the old one.
Please advise.

And you didn't use the '-simulation' flag by chance, right?
Can you attach the log file?

Hi,
I think I have done everything as instructed above. But the phone doesn't seem to recognize the new mgstore.db backup file.
I notice that the new file isn't crypt7 as the original mgstore.db file from the internal drive. Is that causing my problems? How do I fix it?
Greetz

clevernam said:
mgstore.db
Click to expand...
Click to collapse
Did you make a typo with the file like you did here? It should be msgstore.db, not mgstore.db.
No, it doesn't have to be encrypted in order for whatsapp to recognize it.

namnoops said:
Did you make a typo with the file like you did here? It should be msgstore.db, not mgstore.db.
No, it doesn't have to be encrypted in order for whatsapp to recognize it.
Click to expand...
Click to collapse
Nope, that's not it, just checked.
Do i have to put it into the msgstore.tar file? Where i extracted new and old from?
Greetz (thanks for helping btw., if this works, its a realy big help to me!!)

Nope, you just have to place msgstore.db inside the 'whatsapp' directory on your internal SD card (I'd make sure it's the only file there).
Then clear data on whatsapp, force close it, and run it.
Is that exactly what you're doing?

Multiple Databases
Can this app merge multiple databases...
Like if i want to merge 3-4 old databases into one? is it possible? if either one of these databases have same data, will that be duplicated?

It can merge more than one, but that's under the assumption they're mutually exclusive (in terms of messages - i.e., no message appears twice)
So if you used one phone (1), started with a clean whatsapp on a second phone (2), then another clean whatsapp on a third (3), and so on...
You'll merge (1) as 'old' and (2) as 'new', then use the output as 'old' and merge it with (3) as 'new', etc.

namnoops said:
Nope, you just have to place msgstore.db inside the 'whatsapp' directory on your internal SD card (I'd make sure it's the only file there).
Then clear data on whatsapp, force close it, and run it.
Is that exactly what you're doing?
Click to expand...
Click to collapse
Yes, thats exactly what i did. Any other idea what could be wrong?

I'm sorry - it should be under WhatsApp/Databases like the opening post says...

namnoops said:
I'm sorry - it should be under WhatsApp/Databases like the opening post says...
Click to expand...
Click to collapse
That's what i figured, because I was already thinking about all the other files with dates like msgstore-2014-10-12.1.db.crypt7 that i took out of the Database folder.

namnoops said:
I'm sorry - it should be under WhatsApp/Databases like the opening post says...
Click to expand...
Click to collapse
The output file, that I renamed to msgstore.db has "gear wheels" as an icon, but the msgstore file from the database of the phone has no icon at all. Is that somehow significant? Does that maybe lead you to my mistacke?

The fact the the output file has the 'gear wheel' icon just means Windows detects it as a database, I think it's quite meaningless...
The other files it didn't give this icon are *.db? Or are you referring to the encrypted databases? They encrypted ones probably won't have this icon.
It seems to me you're doing everything right:
Renaming the file to 'msgstore.db'
Put it in your internal SD card under WhatsApp/Databases
Making sure there's no other file there except this file
Force closing whatsapp, clearing data, opening the app...
So you're doing all this, and when whatsapp loads it initializes from scratch and doesn't tell you a backup has been found?
It just goes on to initialize an empty database as though you just installed whatsapp for the first time?

namnoops said:
The fact the the output file has the 'gear wheel' icon just means Windows detects it as a database, I think it's quite meaningless...
The other files it didn't give this icon are *.db? Or are you referring to the encrypted databases? They encrypted ones probably won't have this icon.
It seems to me you're doing everything right:
Renaming the file to 'msgstore.db'
Put it in your internal SD card under WhatsApp/Databases
Making sure there's no other file there except this file
Force closing whatsapp, clearing data, opening the app...
So you're doing all this, and when whatsapp loads it initializes from scratch and doesn't tell you a backup has been found?
It just goes on to initialize an empty database as though you just installed whatsapp for the first time?
Click to expand...
Click to collapse
Precisley. Nothing comes, when I reinstall whatsapp.
It is not like when I put the normal backup data back in the databse folder. Then whatsapp realises that there is a backup file in the internal memory and gives me the option to restore the history. Not so with the merged file.
Btw. I have no sd card. I connect the phone with usb. I figured that it is an unessencial diffrence? Am I wrong? My phone btw. is Motorola Moto G. If that helps
I'll check, if the old an new are mutually exclusive. I think in the progress I got a message on the new and old one. What would happen in that case? Would your program do the job, but then creat a bad "output" file?
If the only problem was, that I messed up the historys, I'm sorry, namnoops.

Are you using the latest version of whatsapp?

iclumyc Doctrine
namnoops said:
Are you using the latest version of whatsapp?
Click to expand...
Click to collapse
Well if I didn't then, now I do. And it still doesn't work

I checked if there are any intersecting messages... It turns out there weren't any.
@namnoops: Does the oldest message in the new history have to be more recent then any old message from the old history? I mean: Is it a problem for merjapp if the timelines intersect in any way, although there are no double messages?
Thanks for your help in advance

Add APK - repack system.img

I was looking for a similar topic but I did not find one. Refers to problems with running applications added to "system.img".
I bought UMIDIG S for my father-in-law. It is based on a mediatec chipset. Unfortunately, soft is tragic, and there is no full translation, so I decided to bury it a bit in "system.img".
I can easily install the file in Linux and make changes in it. I can delete applications, I can make changes in configuration files and "buikd.prop".
Unfortunately, all APK files were uploaded to "system.img" even though I set permissions 755 for directories, and 644 for directories. Although the files are root: root, after uploading such "system.img" none of these applications works for the phone. The system sees them and tries to load. Unfortunately, applications hang because of errors. The system informs that the application hangs, and does not load its window. The icons of these applications and the names do not appear either (applications have green robocik as icon, and domain name, e.g. com.android.clock ....)
I am asking for advice. How to add APK files to the unzipped "system.img" so that after packing and uploading to the phone there were no errors ???

Do you have found how add app in system.img?

It's all around, but you can replace apps with others. In the linux system, unpack the img, and mount them. Then you can freely rename files as well as move files within the .img mount point
I chose the applications or other files I did not need, and moved them to the directories (which I named my applications). If the application needed libraries, you also had to get files in a similar way, and place them in arm or arm64 directories.
Then prepared files "stuffed" with data, using the command DD. That is, dd if = (source file) of = (the recipient file at the system.img mount point).

Thanks for reply, so this only on Linux? For Windows there is something?

Markosv76 said:
Thanks for reply, so this only on Linux? For Windows there is something?
Click to expand...
Click to collapse
I do not know, I do not use Windows. Certainly you can in a similar way from BSD systems, probably from Android and MacOS. I read something that Microsoft can somehow support linux shell, but I do not know the details. You can always use some distribution that works with a pendrive.

Thanks, I will try again

Markosv76 said:
Thanks, I will try again
Click to expand...
Click to collapse
You can use Virtual Box to run a Linux distro inside Windows or you can try using Cygwin.
Sent from my LGL84VL using Tapatalk

jaroslawstrauchmann said:
I was looking for a similar topic but I did not find one. Refers to problems with running applications added to "system.img".
I bought UMIDIG S for my father-in-law. It is based on a mediatec chipset. Unfortunately, soft is tragic, and there is no full translation, so I decided to bury it a bit in "system.img".
I can easily install the file in Linux and make changes in it. I can delete applications, I can make changes in configuration files and "buikd.prop".
Unfortunately, all APK files were uploaded to "system.img" even though I set permissions 755 for directories, and 644 for directories. Although the files are root: root, after uploading such "system.img" none of these applications works for the phone. The system sees them and tries to load. Unfortunately, applications hang because of errors. The system informs that the application hangs, and does not load its window. The icons of these applications and the names do not appear either (applications have green robocik as icon, and domain name, e.g. com.android.clock ....)
I am asking for advice. How to add APK files to the unzipped "system.img" so that after packing and uploading to the phone there were no errors ???
Click to expand...
Click to collapse
did you find a solution to this? please reply if yes, I am having the same problem

Help RESTORING backup, "extractTarFork() process ended with ERROR: 255"

NOTE THIS IS NOT "UNABLE TO BACK UP" THIS IS "UNABLE TO RESTORE"
Lots of people on this forum, and lots of people on the internet have errors trying to back up because of this dreaded error 255. But, for me, the backup was fine, but now I can't restore. This is a huge problem because I just tried out a custom ROM (which turned out to be a huge fail), and I want to go back to stock.
So I switched back to the other slot (A/B ftw), wiped all data, formatted SD card, set up the OS to have the same lock pattern as the backup, transferred the backup onto the phone and tried to restore, and, well, fuck.
Code:
[RESTORE STARTED]
Restore folder: '/data/media/0/TWRP/2020-11-08--22-06-46 PRE-RR'
Skipping Digest check based on user setting.
Calculating restore details...
Restoring 4 partitions...
Total restore size is 11276MB
Restoring Vendor Image...
[Vendor Image done (3 seconds)]
Restoring Boot...
[Boot done (0 seconds)]
Wiping Data (excl. storage)
Wiping data without wiping /data/media ...
Done.
Restoring Data (excl. storage)...
extractTarFork() process ended with ERROR: 255
It failed, and there is absolutely nothing I can do about it. TWRP is using buggy commands to extract the backup and I don't know what's triggering the bug.
As a result, right now all my data is in limbo. TWRP cannot restore this backup as-is. I can't just go back in time and use `pm remove-user 999` or `pm remove-user 10` because the backup has already been created. Any solution that has to be done before the backup phase is inaccessible to me now.
My hope is that some smart person here can help me find a way to get this backup back onto the phone. Right now the internet does not have a solution to this problem, no matter how many naive users spam discussion boards with "pm remove-user 999 totally worked". Serious answers only.
Relevant section in the recovery log:
Code:
==> extracting: //data/vendor_de/10/ (mode 40771, directory)
Cannot find key for 10
error looking up proper e4crypt policy for '//data/vendor_de/10/' - 1DE10
tar_extract_file(): failed to extract //data/vendor_de/10/ !!!
I:Unable to extract tar archive '/data/media/0/TWRP/2020-11-08--22-06-46 PRE-RR/data.ext4.win000'
Error during restore process.
I:Error extracting '/data/media/0/TWRP/2020-11-08--22-06-46 PRE-RR/data.ext4.win000' in thread ID 0
I:Error extracting split archive.
Error during restore process.
pigz: write error code 32
pigz: abort: write error on <stdout>
extractTarFork() process ended with ERROR: 255
I suppose the solution is to make it "find a key for 10", but how? Do I just need to somehow recreate the 10 user before restoring the backup?

EDIT NOTICE: THIS DIDN'T WORK, DON'T MAKE IT WORSE BY TRYING
ORIGINAL POST IS BELOW
Alright. Well, I found a solution. That was quick.
The error is triggered by the backup containing a user that isn't present during the restore, I think. One solution is (theoretically) to create the missing user... another solution is to remove the missing user from the backup.
Turns out, backups are not so hard to modify, even in their compressed form. However, it is a little bit unintuitive at first.
You can't use the `tar` command to modify the backup, at all. TWRP's tar doesn't even support the --delete flag, nor does the BSD tar command (what ships with macOS). Even after installing GNU tar, it still refuses to work on a compressed archive (probably for the better).
You either need a Windows system with WinRAR/7zip (allows opening/modifying compressed files without extracting them), or a macOS/Linux system with FUSE and archivemount. If you're on Windows, you will have to manually search every single folder for 10 or 999 folders and delete every single one. If you're on macOS/Linux, the process is a bit more technical but also a bit more automated. And in both cases you will have to do this many times, and it can easily take half an hour or more and many restore attempts.
First and foremost, you need the recovery log to find out where the process failed. Every time it fails you will need to go to Advanced > Copy Log, copy them to the SD card, then use the flipping horrible Android File Transfer program to transfer the recovery.log to your PC, then use the cat/tail command or a text editor of your choice to read the log file.
Somewhere around the end of that file you will find something that looks like this:
Code:
==> extracting: //data/misc_de/10/ (mode 41771, directory)
Cannot find key for 10
error looking up proper e4crypt policy for '//data/misc_de/10/' - 1DE10
tar_extract_file(): failed to extract //data/misc_de/10/ !!!
I:Unable to extract tar archive '/data/media/0/TWRP/2020-11-08--22-06-46 PRE-RR/data.ext4.win003'
This contains all the info you need: the archive where the tar command gave up, and the file it gave up on.
Destroy that file with extreme prejudice.
Make a mount point with `mkdir mnt`, and mount the archive with `sudo archivemount data.ext4.win003 mnt`. (Of course, substitute data.ext4.win003 with the culprit archive file.)
If you're like me and have this problem with 10 folders, or 999 folders, you can find every single one of its kind with `sudo find mnt -name 10`. You can then use `sudo rm -rf` to delete every one of them. If you're brave, try `sudo find data -name 10 -exec sudo rm -rf \{\} \;`. I'm not responsible for any data losses incurred by using powerful commands that don't prompt you for confirmation.
Notice how I'm repeatedly using `sudo` for this: FUSE is kind of quirky and so is the kernel about multi-user filesystems. If you don't use sudo to mount the archive, then the user won't have permission to represent files it doesn't own, even if root is the one asking. You'll get weird "No such file or directory" errors for files and directories that are contained in parent directories' file listings. If you don't use sudo for the find or rm commands, you won't have permission to look inside folders or delete folders. Really, do everything with sudo because it's the only way to do this correctly.
Once you're done scrubbing the problem files away (be careful, do not use a file explorer of any kind, especially macOS's, which generally craps .DS_Store files all over your filesystem), unmount and save the archive with `sudo umount mnt`.
The next part is very important because recompressing files is not instant.
You will need to run `watch -n 1 du -hs data.ext4.win003` and WAIT.
Watch the file size slowly grow; only continue once it has stayed the exact same for at least one minute. Yes, it can definitely stop for 30 seconds at a time and then resume. You will need to give it time to recompress. It's also probably normal for the recompressed file to be about half the size of the initial file; that's just an artifact of TWRP not using very good compression to begin with. archivemount will create a backup of the archive with .orig appended, so you can try again if your backup ends up getting corrupted. (I just created an entirely separate copy of the backup to use archivemount on)
Of course, if you're on Windows you can skip all this and simply delete the problem file from WinRAR/7zip, and it'll be gone instantly and you can just proceed to copy the archive over to your device and retry the backup. You will have to do all the searching manually though, or perhaps you could try your luck with WSL. (Does WSL support FUSE?)
Expect this process to fail multiple times, but all at different points; whenever it fails at a different file, you know it got past the one you just removed. all you need to do is repeat this process over and over, slowly fixing your backup until it restores properly. This is what needs to be done if you can't reproduce the event that created that user in the first place. I'm sure if the user is important, it'll get created again.

Also, looks like user 10 is what happens when Google Play updates Device Personalization Services. Is this the generic "updates for system apps" user or is it specifically for that app?
EDIT: But updating that app doesn't create user 10. Interesting...
EDIT 2: User 10 is the guest user!! All I needed to do was create user 10 and that allowed me to restore my backup!
My post above is still good to see what I tried at first but it actually bricked my device until I factory reset it again, so :/

LoganDark said:
Also, looks like user 10 is what happens when Google Play updates Device Personalization Services. Is this the generic "updates for system apps" user or is it specifically for that app?
EDIT: But updating that app doesn't create user 10. Interesting...
EDIT 2: User 10 is the guest user!! All I needed to do was create user 10 and that allowed me to restore my backup!
My post above is still good to see what I tried at first but it actually bricked my device until I factory reset it again, so :/
Click to expand...
Click to collapse
Glad you managed to get your device back!
I had the same problem on another phone. The reason I got the error was because I tried to restore a decrypted backup on an encrypted device. After choosing the option "Format" in TWRP and typing 'yes' to confirm the decryption process, I was able to restore my backup.

Im having the same error in my realme xt...
sir..please help me.. I have a twrp backup of system image vendor image data boot recovery persist but im unable to recover because it shows an "error extract tar fork process ended up with error 255".during backup i had a pincode for screenlock.
i also have a backup of my internal storage in twrp but im not able to restore because of this error

LoganDark said:
Also, looks like user 10 is what happens when Google Play updates Device Personalization Services. Is this the generic "updates for system apps" user or is it specifically for that app?
EDIT: But updating that app doesn't create user 10. Interesting...
EDIT 2: User 10 is the guest user!! All I needed to do was create user 10 and that allowed me to restore my backup!
My post above is still good to see what I tried at first but it actually bricked my device until I factory reset it again, so :/
Click to expand...
Click to collapse
Hi. Did you create user 10 in Source, or in destination?
How did you do that in source?
If in destination, then is it possible to do it via TWRP?
EDIT: disregard. Mine user in the error path is 0 :-( '//data/system_de/0/'

just use option format data when entering wipe selection

erosman23 said:
just use option format data when entering wipe selection
Click to expand...
Click to collapse
doesnt work
redmi k20 davinci