Related
This script is based off of devsk's switchrom.sh found here: http://forum.xda-developers.com/showthread.php?t=541183. If you are unfamiliar with switchrom, you should probably read that thread first.
Requirements:
1) cyanogen's 1.4 recovery image, Amon_Ra's recovery image, or any other that is based off of the 1.4 recovery image.
2) an extended partition isn't required but makes it easier.
Installation:
rename bart-*.txt to bart.sh
copy bart.sh to /system/sd/
for example:
Code:
mv /sdcard/download/bart-0.9.8.txt /system/sd/bart.sh
note: if using Amon_Ra's 1.2.3 recovery, bart-0.9.2 is already included. Either type 'utility' into the console or invoke bart without sh, file extension, or path. For example: "bart -s myrom" instead of "sh /system/sd/bart.sh -s myrom".
Optional: Create config file named bart.config and place in /system/sd/
Features:
ability to store/restore ROMs based on name
store/restore all made default (nandroid and entire ext partition)
ability to customize the store/restore process
The main difference between the different restores is what data is deleted before the restoration process, not what is restored. The ext backup is kept in a single tar/tar.gz file and is extracted after deleting whatever the user chose to delete.
-r: deletes entire ext partition (rm -rf *)
-b -r: deletes apps folders and dalvik cache (rm -rf app* dalv*)
-a -r: deletes apps_s and dalvik cache (rm -rf app_* dalv*)
-n -r: deletes nothing from /system/sd
This version stores backups in /sdcard/bart. If you want your old switchrom backups available to you, just copy or move them there.
Code:
mkdir /sdcard/bart
mv /sdcard/switchROM/* /sdcard/bart/
Usage:
sh /system/sd/bart.sh [--noninteractive] [options] -s|--store [rom_name]
sh /system/sd/bart.sh [--noninteractive] [options] -r|--restore [rom_name]
sh /system/sd/bart.sh [--noninteractive] -d|--delete [rom_name]
sh /system/sd/bart.sh -l
Options:
-a|--apps_s
-b|--base, nandroid apps dalvik_cache
-c|--compress, only used with -s
-e|--ext_only
-h|--help
-n|--nandroid_only
-v|--version
--verbose
--nocache
--norecovery
--noboot
--nodata
--nosystem
--noninteractive
--reboot, only used with -r or -s
--shutdown, only used with -r or -s
Always specify options as separate words"
e.g. -c -r instead of -cr. It is required!
-r and -s need to be last or followed by rom_name.
--noninteractive needs to be first.
Don't use spaces or special characters in rom_name.
Compress will take longer but image will be smaller.
Examples:
store a complete backup, prompt for rom_name (nandroid and entire ext partition)
Code:
sh /system/sd/bart.sh -s
store nandroid, /system/sd/apps*, and /system/sd/dalv* (original switchrom behavior)
Code:
sh /system/sd/bart.sh -b -s
store nandroid only and name it mycy_nand
Code:
sh /system/sd/bart.sh -n -s mycy_nand
store complete ext partition only
Code:
sh /system/sd/bart.sh -e -s
store /system/sd/apps* and system/sd/dalv* only
Code:
sh /system/sd/bart.sh -b -e -s
store nandroid, /system/sd/apps_* and system/sd/dav* only
Code:
sh /system/sd/bart.sh -n -a -s
restore complete backup (nandroid and entire ext partition)
Code:
sh /system/sd/bart.sh -r
restore nandroid, apps folder, and dalvik-cache (original switchrom behavior)
Code:
sh /system/sd/bart.sh -b -r
restore entire ext partition only
Code:
sh /system/sd/bart.sh -e -r
restore /system/sd/app* and /system/sd/dalv* (not nandroid, only apps* dalv* folders cleared first,rest of ext partition left intact)
Code:
sh /system/sd/bart.sh -b -e -r
delete a backed up rom
Code:
sh /system/sd/bart.sh -d
list available roms
Code:
sh /system/sd/bart.sh -l
Changelog:
bart-0.9.1.sh: fixed bug where old nandroid backups were retained when same name was used.
bart-0.9.2.sh: added battery level check before restoring to avoid it crapping out midway during restore process.
bart-0.9.3.sh: added ability to send nandroid norecovery, noboot, nodata, and nosystem flags. Added verbose mode option for nandroid backup/restore.
bart-0.9.4.sh: added ability to delete ext-backup or nandroid separately. Added extra output for verbose mode.
bart-0.9.5.sh: added non-interactive mode
bart-0.9.6.sh: added option to backup/restore app_s, dalvik-cache, and nandroid
bart-0.9.7.sh: no longer tries to mount ext partition when -n option is given.
bart-0.9.8.sh: checks whether or not it is in recovery mode, increased verbosity for verbose mode, code cleanup.
bart-1.0.0.sh: ability to reboot/shutdown after store/restore, minor code cleanup
bart-1.0.1.sh: removed recovery mode check since its not always accurate
bart-1.1.0.sh: can now source a user config file.
bart-1.1.1.sh: fixed the shbang
bart-1.2.0.sh: added ability to backup/restore cache.
bart-1.3.0.sh: cache backup is now default (cache option is now --nocache). Now works with recovery's that mount ext at either /sd-ext or /system/sd.
bart-1.3.1.sh: fix to properly source bart.config on sd-ext as well (/sd-ext/bart.config or /system/sd/bart.config)
Much thanks to Devsk.
_______________________________
Of course I'll take your money.
Or you could donate to others that helped me. Such as:
cyanogen, Meltus, Amon_Ra, rgv151, Rogro82, Loccy, manup456, and many more.
Use switchrom quite often (between cyanogen, xrom, and jacxhero). I am going to upgrade to this though.
Will I be able to just remove the old switchrom script and restore my backups made with switchrom? Or would I need to restore them with the switchrom script and then back them up with this script?
I just really hope cyanogen will integrate this into his recovery image soon. It works wonders. The only thing that bothers me is that I am lazy and dont like having to type to mount system/sd and what not
Edit: Nevermind see in the OP if they are moved to /sdcard/bart it will work, dont know how I missed that
rondey- said:
Use switchrom quite often (between cyanogen, xrom, and jacxhero). I am going to upgrade to this though.
Will I be able to just remove the old switchrom script and restore my backups made with switchrom? Or would I need to restore them with the switchrom script and then back them up with this script?
I just really hope cyanogen will integrate this into his recovery image soon. It works wonders. The only thing that bothers me is that I am lazy and dont like having to type to mount system/sd and what not
Click to expand...
Click to collapse
this sounds to me alot like switchrom on steroids lol... good work.
just posted bart-0.9.1. Fixes a bug where old nandroid backups are retained when same rom name is used.
IDK why, but removing the last * in the following command seemed to work better (got an error including the last *).
Code:
mkdir /sdcard/bart
mv /sdcard/switchROM/* /sdcard/bart/
overground said:
IDK why, but removing the last * in the following command seemed to work better (got an error including the last *).
Code:
mkdir /sdcard/bart
mv /sdcard/switchROM/* /sdcard/bart/
Click to expand...
Click to collapse
My bad. Thanks.
dumfuq said:
just posted bart-0.9.1. Fixes a bug where old nandroid backups are retained when same rom name is used.
Click to expand...
Click to collapse
Was this per my request in the switchrom thread? If so sweet, if not then rockon anyways cuz the damn issue is fixed, I looked one day and I had 100mb left on my sd card, the switchrom folder was like 6GB cuz old bkup NAND imgs that stacked up over a month of bouncing between CM and hero
ESKIMOn00b said:
Was this per my request in the switchrom thread? If so sweet, if not then rockon anyways cuz the damn issue is fixed, I looked one day and I had 100mb left on my sd card, the switchrom folder was like 6GB cuz old bkup NAND imgs that stacked up over a month of bouncing between CM and hero
Click to expand...
Click to collapse
Yup. You instigated that one. Figured it was kinda a big issue for people who are always juggling space (like myself).
So now if I were to be using cyanogen for three days and for whatever reason I switch to xrom, if I was gonna back up the cyanogen rom I don't need to delete my previous backup?
rondey- said:
So now if I were to be using cyanogen for three days and for whatever reason I switch to xrom, if I was gonna back up the cyanogen rom I don't need to delete my previous backup?
Click to expand...
Click to collapse
If you use the same name to store a rom, yes it will overwrite the previous backup with said name. The ext-backup.tar will be overwritten and the nandroid backup will be deleted before the new one is made.
works great, thanks!
Used it to switch back and forth between CM 4.1.11.1 and JacHEROSki today and it worked like a champ. Nice job!
Thank you sir!, any way you can point me in the direction of modifying the script so the default is just ap* dal*, versus the whole ext partition, have several files and other things in the ext partiton that I don't need older version over writing my current version by accident.... basically restore the way devsk's script backed up ext side, do I just move the variable for the condition from the conditional statement and recreate it as global and set it equal to 1? 0? I'm pretty good at c, java, and a few other languages so its not totally greek, just a little unfamilar.
thanks bro
Good Work!
I have an suggestion. I use switchrom all the time. Every time I back up, the whole of the ext is backed up every time..
I would like it if only the dalvik-cache and the app_s alone is backed up and restored, and the app and app-private is either left alone, OR restored from a central location..
This would enable me to switch between ROM's, but keep the up to date apps in their present state...
I never wipe the app and app-private when switching between ROM's so that any app updates I have previously performed, remain as they are.. The issue arises when I use switchROM.sh to restore an old build, this will restore my apps to that point in time, and I have to re-update them...
Just a thought... I was considering doing this myself, but a couple of attempts failed, so I will leave it to the PRO's to figure out..
ESKIMOn00b said:
Thank you sir!, any way you can point me in the direction of modifying the script so the default is just ap* dal*, versus the whole ext partition, have several files and other things in the ext partiton that I don't need older version over writing my current version by accident.... basically restore the way devsk's script backed up ext side, do I just move the variable for the condition from the conditional statement and recreate it as global and set it equal to 1? 0? I'm pretty good at c, java, and a few other languages so its not totally greek, just a little unfamilar.
thanks bro
Click to expand...
Click to collapse
If you throw the -b switch in there (base), it will only store/restore app* and dalv* folders to/from your EXT.
A naming convention I use is CMx.xx.x09-21-09 [store all]
CMx.xx.x09-21-09NB[nandroid-base]
CMx.xx.x09-21-09EB[EXT-base]...and so on...
I tried using BART and this is what I get:
Code:
/system/sd # sh bart.sh -s
sh bart.sh -s
Please entere name for ROM.
test
test
Storing ROM test, proceed? (y/n) y
y
Storing app data...
tar: empty archive
Error occurred during storing of app data...
tar operation failed.
Do you have enough space on the /sdcard?
I have more than 2.5GB left on my SD card, my ext partition is 300MB, switchrom is working just fine, so what do you think my problem is?
pascanu said:
I tried using BART and this is what I get:
Code:
/system/sd # sh bart.sh -s
sh bart.sh -s
Please entere name for ROM.
test
test
Storing ROM test, proceed? (y/n) y
y
Storing app data...
tar: empty archive
Error occurred during storing of app data...
tar operation failed.
Do you have enough space on the /sdcard?
I have more than 2.5GB left on my SD card, my ext partition is 300MB, switchrom is working just fine, so what do you think my problem is?
Click to expand...
Click to collapse
Are you running it through the recovery console? Also I'm not sure if it is a prerequisite to create /sdcard/bart folder prior to running the script. It says to in the OP, so that's what I did. I know switchrom created it for you. Try this in recovery:
Code:
mount /system/sd
sh /system/sd/bart.sh [-flags][romname]
pascanu said:
I tried using BART and this is what I get:
Code:
/system/sd # sh bart.sh -s
sh bart.sh -s
Please entere name for ROM.
test
test
Storing ROM test, proceed? (y/n) y
y
Storing app data...
tar: empty archive
Error occurred during storing of app data...
tar operation failed.
Do you have enough space on the /sdcard?
I have more than 2.5GB left on my SD card, my ext partition is 300MB, switchrom is working just fine, so what do you think my problem is?
Click to expand...
Click to collapse
I have been unable to recreate this.
1) Can you try from the phone itself (in recovery of course) not using adb?
2) Are you positive there is space left? `busybox df -h /sdcard`
script works flawlessly, thank you soo much for making life |------------------| that much easier.
hopefully this can be added into the next recovery image once cyanogen gets some time together for it. he's been tweaking the **** outa his other roms. lol
madunix said:
hopefully this can be added into the next recovery image once cyanogen gets some time together for it. he's been tweaking the **** outa his other roms. lol
Click to expand...
Click to collapse
I already talked to him about it, he said he's gonna work on a new recovery this week.
First-Boot Install System
I have searched Far and wide for something like this since i first put out SleeperROM in November and always come up empty.
So with the latest release, i decided it was finally time to do it myself.
All you have to do is, using the following package as a template either on its own or in your ROM, make sure your batch folder with the .apk's to install are in /data/.firstboot/
Why
Some apps like QuickPic, ConnectBot, TinyFlashlight, Flash, Google Goggles and others that rely on linked libs don't like to simply be copied to their install dir because many won't install their own libs unless the PackageManager does it and/or they won't add themselves to the packages list (like QuickPic). The old solution is to include the lib either in the /data/data/appdir/lib with the rom install OR in /system/lib but this is quite wasteful especially in the case of big apps like Flash where including the libs separately from the app effectively doubles the space taken up on the rom by that single app since the apk still contains the lib files within.
So the solution is to install on first boot by including the apps in a batch folder for the script to process.
How it works
What it does is run from the init scripts, as one of the last scripts to run, it waits until the Android core system is up (checks to be sure by waiting for the SystemUI process is running then waits for an additional 10 seconds)
Then runs /data/.firstboot.sh, which is where you should put your first boot routines. Included in this script is the batch app installer which looks for the apps in /data/.firstboot/ and stores temporary app backups in /cache/tmp -- so be mindful that on MTD roms, the cache partition is smaller so may have to modify the $tmp variable to wherever you want to store temporaries
After installing, it sleeps for a proportional number of seconds (5 seconds per app installed) to ensure there is no race condition between installs or the final permissions_fix, zipalign and tmp cleanup. The /data/.firstboot.sh script removes itself when everything is complete.
The remaining components are kept in there for additional use by the /etc/init.d/Y02firstboot script in the future, so then all that needs to be put on the phone is a new /data/.firstboot/ dir and replacement /data/.firstboot.sh to run a batch of updates.
The Apps MUST be named by their package name.
I.e. Titanium Backup MUST be named com.keramidas.TitaniumBackup.apk
the file name must correspond with the name of the data dir in /data/data/ the script is lazy in that way, i didn't feel like keeping a file manifest of installs, instead just like to drop in apps to install.
The installer
consists of the following components:
- /system/etc/init.d/Y02firstboot
- /system/xbin/rsync
- /system/xbin/fix_permissions
- /system/xbin/batch_zipalign
- /system/xbin/sleeperlog (echos, logcats, and writes a log of activity to /sdcard/sleeperlog.txt)
- /data/.firstboot.sh
- /data/.firstboot/ (batch app directory)
- NOT INCLUDED, there must be a busybox bin somewhere in $PATH - this is not a problem for most roms
See the package link for a ready to use first-boot installer [CWM] flashable. Just drop your apks into /fs/data/.firstboot/ for a batch app updater
Download the template/usable package
DOWNLOAD MOD_CWM-FirstBoot-20120112.zip
File size: 341.07 KB / MD5 23d88c349b8d2fa3cd2f9958ae99a1f6
THE MAIN COMPONENTS:
/system/etc/init.d/Y02firstboot
Code:
#!/system/bin/sh
# execute post-install script on First boot
# 01022012 SENSEISIMPLE
SLEEP=3
FBSCR="/data/.firstboot.sh"
BB="busybox"
pg () {
$BB ps | $BB grep "[email protected]" | $BB grep -v "$( echo $BB grep [email protected] )"
}
if [ -f "$FBSCR" ]; then
#install apps on first boot after system services have started
sleeperlog "Found $FBLOC"
sleeperlog "Waiting for system"
$BB chmod 0755 $FBSCR
while : ; do
if pg systemui; then
$BB sleep 10
sleeperlog "system loaded."
log -p i -t boot "Executing $FBSCR script"
sleeperlog "Running FirstBoot init"
$FBSCR
break
fi
sleeperlog "WAITING FOR SYSTEM SERVICE: sleeping for $SLEEP s..."
$BB sleep $SLEEP
done
fi
/data/.firstboot.sh
Code:
#!/system/bin/sh
#
# 20120107 - SENSEISIMPLE
#
log -p i -t init:firstboot "INIT.firstboot BEGIN: USER SCRIPT $FBLOC.sh"
sleeperlog "FirstBoot Script OK TO RUN"
FBLOC="/data/.firstboot"
tmp="/cache/tmp/firstboot"
bak="$tmp/bak/data/data/"
BB="busybox"
RM="$BB rm"
CP="$BB cp"
MV="$BB mv"
MKDIR="$BB mkdir"
LS="$BB ls"
CHMOD="$BB chmod"
SLEEP="$BB sleep"
GREP="$BB grep"
pg () {
$BB ps | $BB grep "[email protected]" | $BB grep -v "$( echo $BB grep [email protected] )"
}
$CHMOD 0777 /data
#install apps on first boot
if [ -d $FBLOC ]; then
sleeperlog "Found $FBLOC"
if [ "$($LS $FBLOC)" ]; then
cd $FBLOC
$MKDIR -p $bak
for pkg in *.apk; do
log -p i -t init:firstboot "INIT.firstboot INSTALLING: $pkg"
sleeperlog "PREPARING: $pkg"
pkgname=${pkg%.*}
sleeperlog "BACKING UP APP DATA - $pkgname"
#back up data, delete the original data dir to prevent install errors (pm isn't very good at what it does)
if [ -d /data/data/$pkgname ]; then
$CP -a /data/data/$pkgname $bak/$pkgname
$RM -rf /data/data/$pkgname
fi
#WAIT, then install, then WAIT SOME MORE
$SLEEP 2
sleeperlog "INSTALLING $pkgname"
pm install -r $FBLOC/$pkg &
$SLEEP 10
#REIntegrate application data
if [ -d "$bak/$pkgname" ]; then
sleeperlog "\nSYNCING APP DATA \n\n$(/system/xbin/rsync -auv --exclude=lib $bak/$pkgname/ /data/data/$pkgname/)\n"
fi
i=$((i+1))
#Move the install .apk to tmp
if $LS /data/app | $GREP "$pkgname"; then
sleeperlog "Package appears to have installed."
$MV "$pkg" $tmp/
fi
#If the firstboot batch dir is empty, delete it now
! [ "$($LS $FBLOC)" ] && $RM -rf $FBLOC
done
#WAIT for [#ofapps x 5 seconds each] to avoid a race condition
waitsecs=$(( i * 5 ))
sleeperlog "Waiting for ${waitsecs}s before Fixing Permissions"
$SLEEP $waitsecs
sleeperlog "Fixing Permissions $(/system/xbin/fix_permissions)"
sleeperlog "Running batch zipalign \n\n $(/system/xbin/zipalign)"
sleeperlog "Clearing tmp $tmp"
fi
fi
sleeperlog "FIRSTBOOT SCRIPT COMPLETE"
log -p i -t init:firstboot "INIT.firstboot SELF DESTRUCT FIRSTBOOTSCRIPT"
if ! [ "$($LS $FBLOC)" ]; then
$MV $0 $tmp/.firstboot
# COMMENT THIS OUT FOR DEBUGGING, TO NOT REMOVE THE TMP DIR
$RM -r $tmp
echo ""
fi
echo -e "#\n#COMPLETED ON $(date)\n#" >> $0
sleeperlog "FirstBoot Apoptosis"
log -p i -t init:firstboot "INIT.firstboot END: USER SCRIPT $FBLOC.sh"
THANKS
CyanogenMod team for the Fix_permissions script
DarkyROM for the Batch Zipalign script
Saving this seat . . .
does this work with a bml rom?
^^^^In theory, it would work with an NTFS rom, if one existed - different filesystems don't [usually] create any changes on the surface... if you try to copy a file from a partition of any file system to a partition of any other file system, you don't have to explicitly tell the system the fs types involved. I haven't looked through the entire script, but if there are any commands that must specify the partition type, it would be a simple matter of changing any occurances of EXT4 to YAFFS2, etc.
This could be a great way to make a clean reinstall with all of your apps intact (minus app data, do a separate backup with your backup app of choice) - first, backup your installed apps by copying them to the appropriate directory, then do a full wipe and install, with the script automatically reinstalling your apps on first boot...
EDIT: I just looked through the script more closely. First, it appears that data is backed up. Second, I can confirm that the standard, non-file system-specific linux commands are used for file operations (cp to copy, etc), so this script shouldn't need any adjustment to accomodate different file systems
Sent from my SPH-D700 using XDA App
Is it possible for someone to develop an application to backup your personal apps and then running a script in clockwork or on first boot that does the same thing as this, alllowing for personal apps to be in the rom directly after flashing. I understand that sometimes apps may not be compatible, but can randomroms rdu script be modified to choose to do this?
Sent From My Cyan4g
Good looking script SenseiSimple!!
Okay, I'm trying to use this script, but you said to put the apps in fs/data/firstboot/ but where is "fs"? I found the "firstboot.sh" file, but I'm sure I can't put the apk's in there...
In loving memory of my son "Jeffrey Ryan Giles" 11/17/1992 to 11/25/2011 :'(
sniperkill said:
Okay, I'm trying to use this script, but you said to put the apps in fs/data/firstboot/ but where is "fs"? I found the "firstboot.sh" file, but I'm sure I can't put the apk's in there...
In loving memory of my son "Jeffrey Ryan Giles" 11/17/1992 to 11/25/2011 :'(
Click to expand...
Click to collapse
If you look at the folder structure inside the SleeperROM zip, you'll see he uses an fs directory to hold his system and data folders for installation. If you're developing a ROM, you can adapt the script to point to wherever your data folder is (most other ROMs just have system and data in the root directory of the zip, so the path would just be data/firstboot/).
EDIT: no need to look in SleeperROM zip, the firstboot zip in the OP has the same folder structure. It just doesn't seem to have the firstboot directory inside data. You'll need to add that.
Sent from my SPH-D700 using XDA App
bbelos said:
If you look at the folder structure inside the SleeperROM zip, you'll see he uses an fs directory to hold his system and data folders for installation. If you're developing a ROM, you can adapt the script to point to wherever your data folder is (most other ROMs just have system and data in the root directory of the zip, so the path would just be data/firstboot/).
EDIT: no need to look in SleeperROM zip, the firstboot zip in the OP has the same folder structure. It just doesn't seem to have the firstboot directory inside data. You'll need to add that.
Sent from my SPH-D700 using XDA App
Click to expand...
Click to collapse
Thanks for the reply buddy, so what I THINK your saying is to create a folder called "first boot" in my data folder?
In loving memory of my son" Jeffrey Ryan Giles" 11/17/1992 to 11/25/2011 - RIP :'(
sniperkill said:
Thanks for the reply buddy, so what I THINK your saying is to create a folder called "first boot" in my data folder?
In loving memory of my son" Jeffrey Ryan Giles" 11/17/1992 to 11/25/2011 - RIP :'(
Click to expand...
Click to collapse
Are you trying to do this in a ROM or on the phone? Sorry if that's a dumb question, but I just want to be sure we're on the same page.
Sent from my SPH-D700 using XDA App
sniperkill said:
Thanks for the reply buddy, so what I THINK your saying is to create a folder called "first boot" in my data folder?
In loving memory of my son" Jeffrey Ryan Giles" 11/17/1992 to 11/25/2011 - RIP :'(
Click to expand...
Click to collapse
bbelos said:
Are you trying to do this in a ROM or on the phone? Sorry if that's a dumb question, but I just want to be sure we're on the same page.
Sent from my SPH-D700 using XDA App
Click to expand...
Click to collapse
i hadn't noticed my zip routine apparently skips empty .folders
in the zip, there SHOULD have been /fs/data/.firstboot into which you place whatever apk files named according to their actual package name i.e. com.keramidas.TitaniumBackup.apk for Titanium Backup, com.alensw.PicFolder.apk for QuickPic and so on...
you can find out this info in Titanium Backup by finding the app in the list and tapping its name to show the data store location.
it HAS to correspond to the actual package name in order to properly back up the data files because the package manager binary as run from a script won't overwrite the data dir so the old data has to be synced back into the new install (rsync)
I am currently refactoring the script to reduce the number of dependencies... the system will then consist of the init.d script, zipalign and rsync bins, and the /data/.firstboot.sh, /data/.firstboot dir all in one place rather than scattered throughout the system.
irule9000 said:
Is it possible for someone to develop an application to backup your personal apps and then running a script in clockwork or on first boot that does the same thing as this, alllowing for personal apps to be in the rom directly after flashing. I understand that sometimes apps may not be compatible, but can randomroms rdu script be modified to choose to do this?
Sent From My Cyan4g
Click to expand...
Click to collapse
it would be possible to create an app to do this, and i considered it because of the need to make sure the android system is fully loaded for package manager to work, but given the requirement for full root access to start running before the system is up without having to worry about android permissions or getting SuperUser approval it's easier/more effective to do it as part of the system init in a script.
as far as randomking's rdu script, they would be compatible, but they exist separately as in one shouldn't interfere with the other, because the firstboot script runs long after install once the system is up and running.
you could modify the rdu setup to include/exclude the /data/.firstboot dir and .firstboot.sh script based on the selected config, since the init script does nothing if it doesn't see the /data/.firstboot.sh file, and the .firstboot.sh script does nothing if it doesn't see the .firstboot dir
SenseiSimple said:
it would be possible to create an app to do this, and i considered it because of the need to make sure the android system is fully loaded for package manager to work, but given the requirement for full root access to start running before the system is up without having to worry about android permissions or getting SuperUser approval it's easier/more effective to do it as part of the system init in a script.
as far as randomking's rdu script, they would be compatible, but they exist separately as in one shouldn't interfere with the other, because the firstboot script runs long after install once the system is up and running.
you could modify the rdu setup to include/exclude the /data/.firstboot dir and .firstboot.sh script based on the selected config, since the init script does nothing if it doesn't see the /data/.firstboot.sh file, and the .firstboot.sh script does nothing if it doesn't see the .firstboot dir
Click to expand...
Click to collapse
Expanding on the rdu script idea, how about a prop variable in the config to specify a user apps directory on the sdcard (best if standardized)? The user can store the apks to be installed after a clean flash in this directory. The rdu script would be modified to copy those apks to the firstboot directory. Although how to handle data? Maybe another script to be run before flashing that checks the apks in the sdcard folder, and copies the current data folder to the sdcard. This data is then restored somehow with the firstboot script. This all can be in addition to the firstboot apps included with the rom.
Sent from my SPH-D700 using XDA App
app
I honestly have no developing experience. To clarify, I was I simply asking if the app could back up the users personal apps/data to the specified folders, to make this script fool proof. Then when a rom is flashed the randomromkings rdu script which currently decides on lite roms or full roms could be modified to either install apps or not install apps depending on the rom compatibility. Finally data is wiped, rom flashes, and your script runs on boot allowing the apps to all be there. Expanding on that if the ROM allows apps to be re installed via your script, through the modified rdu, your script could be moved from one directory to another allowing the process to happen and at the end moves itself back to the original directory. This means that the end user who has no clue how to do anything but flash roms (like myself, but I want to learn) has done nothing accept backing up their apps. I know this is all hypothetical, but would this be possible, and also I have another idea that is probably impossible. If a script could be written so that the apps are backed up when the rom flashes, then the rom automatically does a factory restore and wipes the various caches, meaning that less people are going to have force close issues or other avoidable bugs because they didnt wipe. thanks for all your hard work
Clear Me Here !!
You said that the script (.firstboot.sh) removes itself, but does it remove Y02firstboot script from init.d?? I don't think so, also, there are no disadvantages of this after first boot, as it won't find .firsboot.sh and skip the operation, but why shud it execute unneccesarily everytime??
@OP--Plz. tell me if i am right or wrong.......If right, plz. add a function to remove that script from init.d. However files in /xbin are OK though, so no need to remove them............
Everybody is writing and making this appear as complicated as possible! So let me clarify what I believe the idea is here. First of all, the 'RDU script' I assume you're referring to is simply a flash-able zip to set you phone for a lite or full installation. It sets a config file. I use it in my rom, and I also currently use that config file to set several other features upon installation, and changeable at anytime one of my themes is flashed.
RDU was simply meant to jump start the idea of developers making installations more uniform. Which has happened to various extents, for example, I've used this first-boot install to solve my quickpic(I love the app) and flash pre-install problems. It also fixes usb tether, vlingo, terminal, etc.
SO, what are we saying here? We'd like to be able to backup our apps to the sd card, as many apps can do for us. Then, we'd like to either:
A. Build a script into a rom to restore these apps prior to or upon installation.
or
B. Build a separate script which would be run AFTER a rom installation to restore these apps from CWM.
Yes?
P.S. I see this is very late to the game, didn't realize this thread was being revived...
Still. Neat idea, sorry I hadn't noticed it yet.
+1 Thanks
After integrating this into my custom ROM, my phone (Galaxy S 4G) would only install the first .apk file in the .firstboot directory. After I removed the code which tells it to backup/restore the /data/data directories, it worked fine. I won't need that code since the ROM does a full wipe of /data every time anyway, but I'm not sure why it doesnt work when it's there. I'm not well versed enough with Java's syntax yet to comprehend why.
Also, your first post says that it should record logs to \sdcard\sleeperlog.txt but the script tries to record it to \sdcard\sleeperromlog.txt and neither one of those files actually appear on the sd card.
Edit: I had to remove the check to make sure the .firstboot directory is empty before deleting it, but it works fine on my Galaxy S 4G ROM as long as there are no /data/data directories for the programs I am installing.
does anyone still have a template of this? The link is down.
Edit: nevermind I don't need it anymore.
From now on, Android rom can be easy to pack. You don't need to worry about the permissions of the files or the symlinks anymore.
What does this package do:
1. Automatically find and recognize the img with patterned name in the same folder of this package.
2. Automatically wipe and flash the system partition when system.img is found.
3. Automatically flash the addon pack without wiping the system partition when addon.*.img is found.
4. Automatically backup efs and efs_gsm partition, on Samsung Duo sim card phones.
5. Be able to flash the unmountable partitions like boot, recovery, modem, etc.
How to pack:
1. Make you rom on on the Phone or Linux. On the Phone? Yes, you can test your patches on your phone.
2. No matter on the Phone or on Linux, cd /system/ or cd <your system folder>, check the permissions and symlinks. Usually I don't check cause I modify directly on a system I dumped from my phone. Yes a dump of system partition is a good idea.
3. You have two optitions: 1. Use the modified dumped system partition dump as system.img. This usually generate a big img to flash, but will flash faster. 2. If you use extracted folder or want to use small files, cd to the system folder in shell and make the folder to a tarball with the command: 'tar --lzma -cf <your tar file> * .[!.]*'. Why use * .[!.]*? This will put the .* files and folders in the tarball.
4. Then rename the dumped image or the tarball to system.img and put it in the same folder of the FLASHER.
5. Want to make an addon? Just make the name like addon.<your name>.img. Don't for get the permissions and ownership of the files and folders.
Future plan:
1. Make a packer/dumper for the system partition.
Github:
https://github.com/maxfu/universal_android_flasher
I will edit this post and post some pictures later.
I needed to restore my Whatsapp messages without having a proper whatsapp backup, in cloud or local, just a TWRP data partition backup (without userdata, thanks to encryption and some other unrelated problems), so, having only data/data/com.whatsapp directory with the unencrypted databases and files to work with, the usual solutions discussed here weren't an option. In my case, fully replacing the com.whatsapp folder and changing its permissions, as some other sources recommend, resulted in Whatsapp crashing for reasons I ignore. Such info was on the right track and after trying all kind of stuff I finally was able to restore my messages. My steps were as follows, hopefully it works for you too:
Requirements:
A backup of /data/data/com.whatsapp, it should have the databases and files subdirectories.
Rooted Android phone with Whatsapp uninstalled.
Basic terminal usage knowledge. (Should be possible to do it with a decent enough file browser but I won't cover it, feel free to adapt it)
If you need to manually extract com.whatsapp from a TWRP backup:
Locate the different data.ext4.win### files in the TWRP backup.
Rename them adding .tar at the end of the name and open them with Winrar/7zip.
If the .tar contains another data.ext4.win###, extract it and repeat the process.
Once you have access to the actual contents inside the files, locate the file which has the data/data/com.whatsapp directory.
Extract that com.whatsapp directory.
Once you have your copy of com.whatsapp:
Copy your backup of com.whatsapp directory to your phone (I will use /sdcard/com.whatsapp as example).
Install Whatsapp but DO NOT RUN IT. Try to use the same version. (Probably works between versions to some extent)
Install an Android terminal emulator and launch it. (Tested with termux)
Elevate, enter /data/data and show the permissions of the installed Whatsapp com.whatsapp directory:
Code:
$ su
# cd /data/data
# ls -ld com.whatsapp
You'll see something similar to:
Code:
drwx------ 10 u0_a117 u0_a117 4096 2020-03-09 17:28 com.whatsapp
Take note of the user/group, will have u0_a# format. (In my case will be u0_a117)
Enter into that directory:
Code:
# cd com.whatsapp
Copy files and databases directories from your transferred com.whatsapp copy in step 1 to your install location. (Don't forget the final space and dot)
Code:
# cp -R /sdcard/com.whatsapp/databases .
# cp -R /sdcard/com.whatsapp/files .
Change both directories owner and group to the one seen in step 5.
Code:
# chown -R u0_a117:u0_a117 databases
# chown -R u0_a117:u0_a117 files
Check the permissions to be 100% sure that the user/group are correct.
Code:
# ls -lad *es
Launch Whatsapp.
Whatsapp will ask you to configure your phone number. Once you're done, you'll see your messages right there.
If Whatsapp crashes after launching it, you wrote a wrong user/group when changing permissions. Whatsapp overwrites the contents of com.whatsapp before crashing, so you'll need to reinstall Whatsapp and start again. The user/group will be a different one each time you install it.
My system was buggy, not being possible to boot and backup via TitaniumBackup or similar..
I tried your method and worked perfectly via OrangeFox Recovery (with the terminal and the included file manager that facilitates copy/paste operations)
Thank you very much!!
Thank you for this tutorial. I have the same difficulty as the first respondent, in that my miui system is buggy and constantly restarting so no time to do a chat backup with the normal procedure. I also have orange fox recovery so I shall try to follow your instructions to the detail. Hopefully all will go well; I shall post back later with the results.
Yaagod said:
My system was buggy, not being possible to boot and backup via TitaniumBackup or similar..
I tried your method and worked perfectly via OrangeFox Recovery (with the terminal and the included file manager that facilitates copy/paste operations)
Thank you very much!!
Click to expand...
Click to collapse
Did you introduce any variation or it worked straight off?
One thing that worries me is checking all the permissions
Thanks a lot! I'd just drop my device and it's screen broke, and I was able to "transplant" my WhatsApp installation to another device just fine!
I was trying something similar to this long ago (but also copying some whatsapp-related files from /data/user/0/) and it didn't work, maybe they were related to authentication? Anyway, if it works I don't care
Very nice guide.
you can skip the chown part and use
Code:
cp -rp databases files /data/local/tmp/
to keep the permissions and ownership.
And i think that
Code:
pm clear com.whatsapp
will do the work, you don't need to uninstall if the app opens after the installation.
.
It worked! You are great!
Thanks a lot
does this still work, iv tried exact but whatsapp keeps closing
I just want to say a big THANK YOU. I can confirm that it's still working.
As I have no root on my phone I have used the internal Terminal of TWRP Recovery instead of an app and executed the commands there. They were running without problems (except the command "su" of course) and WhatsApp recognized my previous data.
Custom ROM died yesterday, not able to do backups, recovery had problems too, so I managed to find some Whatsapp inside a TWRP Backup.
I'm unable to work with terminal, can't change directories, maybe because of Android 13. But with total commander app I'm able to move file and database folder and rearrange owner, group and permissions. at least I'm pretty sure to get it right. whatsapp still crashes.
Using a Xiaomi Mi11 (rooted). Recently upgraded to MIUI 13 which is based on Android 12. Was going to do my usual TWRP backup before bringing the phone in to fix some minor problems with sticky volume keys, when I realized (!) my TWRP cannot even mount the /data partition, let alone decrypting and doing any backup.
So I started reading up on TWRP developments, and realized TWRP for now has lost its ability to see anything under /data if your phone is on Android 12.
Never a fan of things like Titanium backup where the backup is done on an app-by-app basis, so a lot of of settings like magisk modules / phone behavior, etc etc cannot be retained (at least that was my impression of it when I briefly tried those solutions). So when I decided to bring my phone in for repair anyway, I went ahead and wiped the phone clean, and had to live with losing 10 day's worth of my data - 10 days because fortunately I did a backup just before I upgraded from MIUI 12 to MIUI 13 10 days ago... (yeah could have done a lot of manual work to salvage some of the data before I wiped it clean, but I didn't bother with the tedious processes).
So I now have a fixed phone, no more sticky buttons, and restored my nandroid backup with the older MIUI 12 system (android 11 based), and not even considering moving back to MIUI 13 until there is a feasible way to do a TOTAL backup of the /data partition, in others words a nandroid backup on Android 12....
Question - is there any feasible method of doing a Nandroid Backup on an Android 12 system, with or without TWRP?
Thank you !!!
A NANDroid-backup is the bitwise 1:1 copy of existing Android system.
If phone's Anndroid OS is rooted then you always can launch a NANDroid-backup.
This can get achieved by pure ADB commands what of course requires ADB is enabled on phone.
xXx yYy said:
A NANDroid-backup is the bitwise 1:1 copy of existing Android system.
If phone's Anndroid OS is rooted then you always can launch a NANDroid-backup.
This can get achieved by pure ADB commands what of course requires ADB is enabled on phone.
Click to expand...
Click to collapse
Could you elaborate?
I can picture this issue -
if you do "adb shell" to enter terminal (or plain adb pull?) while your phone is switched on, a lot of files are being locked and/or being modified while the phone OS is running so how can someone just take a snapshot of everything under /data even with proper adb commands?
And if you go to recovery mode first, well at the present time no TWRP can access the data partition it seems. So again even with the appropriate adb commands, no copying is possible....?
Any clarification appreciated !
You would run
Code:
adb wait-for-device
adb root & adb shell "stop"
adb shell "mount -o rw,remount /data"
: run the backup command here
adb shell "start" & adb unroot
xXx yYy said:
You would run
Code:
adb wait-for-device
adb root & adb shell "stop"
adb shell "mount -o rw,remount /data"
: run the backup command here
adb shell "start" & adb unroot
Click to expand...
Click to collapse
Dear Android export @xXx yYy - wow ! This looks really promising !!
I just did a quick test by going straight to adb shell, "su", then "stop". My phone screen totally went blank, and I was amazed ! This is awesome !!! "start" and after a while the phone boots up again.
I then tried "top" while the phone is stopped. It seems to still have a few android related processes running, so I am not 100% sure if the whole system has been frozen. But you obviously know what you are talking about, and I have faith in you.
(by the way, I cannot "adb root", seems like after doing a quick search I will need to make my phone think it is a development build by patching the adbd daemon first on my phone.. suggestions on what to do appreciated)
You have just made me decide to spend the coming hours to test the following. Let me know if I should skip any of the steps below because you know it works so I don't need to waste time to validate:
1. Do a proper backup with TWRP first in case I screw up anything
2. start a terminal session with adb shell
3. "su", "stop"
4. "cd /data"
5. "tar -cvpzf /data/backup.tar.gz -C /data"
(If no error, this should be my nandroid backup...?)
6. flash phone and wipe everything clean, so it is back to brand new status, non-rooted
7. reboot phone, see if it is starting new as if I have just bought the phone
8. root the phone, then try and "stop", "delete everything under /data except /data/media", "delete everything under /data/media", "copy backup.tar.gz back to /data", "tar -xvzf /data/backup.tar.gz -C /"
9. If phone works and is back to the state immediately before backup, then restore successful
Take note that
Code:
adb root
is giving root access to adb ( adbd - read: adb daemon )
what has nothing to do with giving root access to current Android user with following shell command
Code:
adb shell "su"
Also take note that Android services aren't located in /data partition, the partition you want to back up.
With @xXx yYy 's help I think I am getting somewhere.
So essentially a "stop" command in android will stop Zygote (i.e. the mother of all app processes if I am not mistaken). Once you have stopped Zygote, I believe you are then free to make a duplicate of the entire /data environment.
So far that's exactly what I have done. Created a tar.gz file with a size of around 40GB. I believe I am halfway there in my quest to do Nandroid without TWRP, but what I still need to try is to restore the tar file after factory resetting the phone. Will be a time consuming process (as obviously I will also need to have a tried-and-true real backup created first in case I screw something up... I am doing everything on my main phone that I actually use everyday), so I will continue my experiment in the coming days.
One question I have already encountered however - I still cannot do "adb root", which would have allowed me to directly create the backup tar file AND pipe it to my PC all in one go. So far I have had to tar all within the phone, which means space will be a constraint, and it is more time consuming creating the backup file THEN think of a way to transfer that file out of the phone. Already posted a question here asking for help, and if anyone knows of a good way to get adbd to grant adb root request, please let me know.
Above all else, once I have a working method, and I have polished the process, I will be happy to share. I suspect many others are also yearning for a good backup / restore procedure on Android 12.
one can't backup /data partition this way, because tar is just a toybox applet not cabable of preserving secontext. get a gnu tar binary (for example from opengapps installer), set mount namespaces to global, set selinux permissive (if kernel allows it, important) and run from su
/storage/1234-5678 is exFAT and has enough free disk space
Code:
tar --selinux --xattrs --numeric-owner -vcpPf /storage/1234-5678/data.tar /data
/storage/1234-5678 is vfat or less free disk space
Code:
tar --selinux --xattrs --numeric-owner -vcpP /data | gzip | split -a 1 -b 1024m - /storage/1234-5678/data.tar.gz.
another approach would be loop mount some file and busybox cp -avc everything where the -c flag is responsible for secontext (proper busybox required)
--numeric-owner flag is recommended if you are planning to extract it on linux PC later
you could also exec-out straight to PC if no MicroSD Card available, but requires gzip or other compressed stream, otherwise windows will mess up linefeed with carriage return and render your file unreadable
Code:
adb exec-out "su -c 'tar --selinux --xattrs --numeric-owner -vcpP /data | gzip'" > data.tar.gz
restoring .tar.gz from TWRP is absolutely possible, it's just that TWRP can't handle encrypted userdata partition (yet)
Code:
cat /external_sd/data.tar.gz.* | gzip -d | tar --selinux --xattrs -vxpPC /
(where tar must called with full path to binary like /cache/tar or /tmp/tar, or unlink /sbin/tar applet and place binary /sbin, or just rename it gtar)
Note: bitwise 1:1 copy of apps is not possible/sufficient if you factory reset your device, because apps might save data in TEE TrustZone (which is flushed on factory reset)
Hi @seemebreakthis, very interesting discussion on Android 12 backup!
Did you reach a workable solution with this?
Since we can restore most apps from a Google backup, it seems the real issue is to recover the user settings etc. after the Google restore.
Interested in this. Any success thus far?
aIecxs said:
one can't backup /data partition this way, because tar is just a toybox applet not cabable of preserving secontext. get a gnu tar binary (for example from opengapps installer), set mount namespaces to global, set selinux permissive (if kernel allows it, important) and run from su
/storage/1234-5678 is exFAT and has enough free disk space
Code:
tar --selinux --xattrs --numeric-owner -vcpPf /storage/1234-5678/data.tar /data
/storage/1234-5678 is vfat or less free disk space
Code:
tar --selinux --xattrs --numeric-owner -vcpP /data | gzip | split -a 1 -b 1024m - /storage/1234-5678/data.tar.gz.
another approach would be loop mount some file and busybox cp -avc everything where the -c flag is responsible for secontext (proper busybox required)
--numeric-owner flag is recommended if you are planning to extract it on linux PC later
you could also exec-out straight to PC if no MicroSD Card available, but requires gzip or other compressed stream, otherwise windows will mess up linefeed with carriage return and render your file unreadable
Code:
adb exec-out "su -c 'tar --selinux --xattrs --numeric-owner -vcpP /data | gzip'" > data.tar.gz
restoring .tar.gz from TWRP is absolutely possible, it's just that TWRP can't handle encrypted userdata partition (yet)
Code:
cat /external_sd/data.tar.gz.* | gzip -d | tar --selinux --xattrs -vxpPC /
(where tar must called with full path to binary like /cache/tar or /tmp/tar, or unlink /sbin/tar applet and place binary /sbin, or just rename it gtar)
Note: bitwise 1:1 copy of apps is not possible/sufficient if you factory reset your device, because apps might save data in TEE TrustZone (which is flushed on factory reset)
Click to expand...
Click to collapse
You are a genius! That's excactly what i was searching for! Thank you!
This is a bit beyond me.. though I'm looking for a full ROM backup on Android 12. Does this work?
TWRP 3.7.0 is for Android 12 including Encryption Support (except for Samsung)