Database Info

RC30 can forcibly update your phone

Code:
long now = SystemClock.elapsedRealtime();
long nextPrompt = getNextPromptTime(intent, now);
if (nextPrompt == 0) {
Log.i(TAG, "Installing overdue OTA update without prompting");
installUpdate();
return;
}
Basically, getNextPromptTime determines the prompt interval and number of prompts to show. So Google can potentially never prompt and just install without asking permission.
If you want to keep root access, I highly recommend doing the following to disable OTA updates:
Go to a root prompt.
Type the following:
# mount -oremount,rw /dev/block/mtdblock3 /system
# cd system
# cd etc
# cd security
# mv otacerts.zip otacerts.zip.bak
# mv cacerts.bks cacerts.bks.bak
That should disable cert checking by disabling (renaming) all the certs on the device. Someone please verify, but I'm pretty sure those are the certs that are checked.
Alternatively, you can follow the instructions to mod your recovery and boot images:
http://forum.xda-developers.com/showthread.php?t=443041
I have confirmed that leaving a shroot/suroot laying around will not work, as Google formats the system partition and resets all permissions upon an update:
Code:
format SYSTEM:
set_perm 0 3004 02755 SYSTEM:bin/ping
In addition, it is not possible to put the shroot/suroot in another partition besides system, as those are all reverted to noexec or nosuid upon reboot.
Code:
/dev/block/mtdblock5 /data yaffs2 rw,nosuid,nodev 0 0
/dev/block/mtdblock4 /cache yaffs2 rw,nosuid,nodev 0 0
/dev/block/mmcblk0 /sdcard vfat rw,dirsync,nosuid,nodev,noexec,uid=1000,gid=1000,fmask=0711,dmask=0700,codepage=cp437,iocharset=iso8859-1,utf8 0 0
Anyone know where the underlying update code is? I've only seen the Java front end (/packages/apps/Updater).

How do I know if I did this right?
Keep in mind I know next to nothing about Linux.

Ive done it to mine and have yet to see an update

Might want to move telnetd to something, so an app doesnt call it and get root, if you dont want the patch. Also copy sh to something so you can get root, and remove the permissions on sh
Not sure if suid on sh is needed for anything, so beware...

moving telnetd is not necessary, moving cacerts is dumb
Moving telnetd is not necessary, because telnetd does not become root, nor is it setuid root. The bug was that a root shell was listening on the keyboard (so if you type reboot on your phone while the android is pulsing or after you boot - the phone reboots)
Moving the cacerts will make ALL SSL stuff to work incorrectly because you no longer have a list of trusted root authorities.
The only good long-term solution is to reflash your recovery... etc

hmm
the updates are downloaded to /cache what if we touch "nameofupdatefile" or lock down /cache completely and that may prevent updates

or
or we could create a script that kills the update app, deletes any zip files every couple minutes in /cache

My phone gave me an update notification, I immediately killed the com.android.update process then cd /cache ; rm signed-RC30-from-RC29-fat.1582cace.zip then sudo busybox touch signed-RC30-from-RC29-fat.1582cace.zip then rebooted, and so far no more update notification or update process started

Did it, I will inform you what happened....

for some reason signed-RC30-from-RC29-fat.1582cace.zip got deleted, possibly ln -s /dev/null signed-RC30-from-RC29-fat.1582cace.zip will work

RyeBrye said:
Moving telnetd is not necessary, because telnetd does not become root, nor is it setuid root. The bug was that a root shell was listening on the keyboard (so if you type reboot on your phone while the android is pulsing or after you boot - the phone reboots)
Moving the cacerts will make ALL SSL stuff to work incorrectly because you no longer have a list of trusted root authorities.
The only good long-term solution is to reflash your recovery... etc
Click to expand...
Click to collapse
The recovery and boot images both get replaced by patches.
Incidentally, I haven't been prompted yet to update. Has anyone else who followed my instructions gotten prompted? I'll be able to work on a patched update when I get back to work, I've got several G1s at my disposal.

@koush
I did the modifications you mentioned at the begin of the thread, now I want to update to RC30 with the jesus method. Is it necessary to undo the changes I made with your method ? If yes, what do I have to do exactly ?
thanks a lot

PAO1908 said:
@koush
I did the modifications you mentioned at the begin of the thread, now I want to update to RC30 with the jesus method. Is it necessary to undo the changes I made with your method ? If yes, what do I have to do exactly ?
thanks a lot
Click to expand...
Click to collapse
You shouldn't have to, since Jesus has a full update, but you should anyways just to be safe.
Basically, just move the files back:
# mount -oremount,rw /dev/block/mtdblock3 /system
# cd system
# cd etc
# cd security
# mv otacerts.zip.bak otacerts.zip
# mv cacerts.bks.bak cacerts.bks

thanks koush, I will try it !

Why just don't change the update-script of an update to remove any unwanted lines, like formatting root? Why doesn't it work?

Koush said:
You shouldn't have to, since Jesus has a full update, but you should anyways just to be safe.
Click to expand...
Click to collapse
I prefer "JF" as the shorthand for my nick. Just FYI . I'm not Spanish, and I'm not "The Man".

all worked ok, thanks JF
JesusFreke said:
I prefer "JF" as the shorthand for my nick. Just FYI . I'm not Spanish, and I'm not "The Man".
Click to expand...
Click to collapse

[HOW-TO] ROM-HACKING: init.rc ext2-auto-mount / ROM Signing / ROM Kitchen

AS MENTIONED IN THE INTRODUCTION TEXT THIS HAS ONLY BEEN TESTED ON AMON RA ROM 1.6.2 BUT SHOULD REALLY WORK ON ANY ROM THAT HAS NO EXT2 AUTO-MOUNT. AND YEAH THIS WHOLE PROCESS HAS BEEN DONE ON A 32a BOARD. FOR THOSE THAT TRY THIS ON OTHER ROMS LET ME KNOW HOW IT GOES.
I've searched and shuffled through the entire forum and made inquiries to ROM authors without much light being shed on this issue. I doubt I am the only one who has been looking for a way of doing this so I decided to do a small HOW-TO. Here I will explain step by step as to how you can implement a script to be part of your ROM that will auto mount an ext2 partition on boot up if such partition is present. I have included all the tools I've used in order to pull this off, and as the title suggests this has only been done on Amon Ra's latest 1.6.2 ROM. In order to follow these instructions you are expected to allready have set up an adb enviroment on your linux box and for the signing process to work you must have sun-java present, the gnu java wont work. And of course a microSD card with an ext2 partition
1. Download install.sh to your home directory
Code:
wget http://www.grindhouse.no/androidtools/install.sh
chmod a+x install.sh
2. Now execute the install.sh script which will create a directory to work in and download a tool and script package and unpack it.
Code:
./install.sh
When the install.sh script is done you need to move the mkbootimg preferebly to your tools directory of your SDK.
Code:
mv toolstomove/mkbootimg <path/to/sdk/tools/mkbootimg>
3. Unpack the RA1.6.2 ROM into a directory in your home dir. In this HOW-TO we will use directory name "ra1.6.2" as an example through out the entire process.
4. Copy the boot.img from ra1.6.2 to the ROM-cooker dir
Code:
cp $HOME/ra1.6.2/boot.img $HOME/ROM-cooker/boot.img
cd $HOME/ROM-cooker
5. Use unpack.pl to extract the ramdisk from the boot image. I've modified the script a little so it automates the entire process and decompresses the ramdisk to a directory
Code:
./unpack boot.img
6. Now you can either replace the init.rc file here with the one I've included in this package or you can add these lines by yourself. In wich case do the following
Code:
cd boot.img-ramdisk
pico init.rc
Press CTRL+w and then CTRL+t and input 27. hit enter. This will take you to line 27 of init.rc so you can add a line right before the init process remounts the rootfs in read-only mode. Add following line:
Code:
mkdir /sdext2 0771 system system
Now scroll down to the end of the init.rc file and add the following:
Code:
service mountsdext2 /system/bin/mountsd
user root
group root
oneshot
7. You have now edited (or replaced) your init.rc file and prepared it to execute a script on boot that will detect an ext2 partition and boot it if there is one to be found. Now you have to make the mountsd script a part of the ROM. Do the following:
Code:
cd $HOME/ROM-cooker
mv toolstomove/mountsd $HOME/ra1.6.2/system/bin/mountsd
rm -rf toolstomove
8. Now that the init.rc file is sorted out and mountsd has been placed in /system/bin of the ROM so it is time to re-pack the boot.img:
Code:
cd $HOME/ROM-cooker
./repack boot.img-kernel boot.img-ramdisk boot.img
rm $HOME/ra1.6.2/boot.img
mv boot.img $HOME/ra1.6.2/boot.img
9. Your ROM now has a new boot image with an updated init.rc and the /system/bin dir has the script needed to auto-mount the microsd ext2. Now you must re-zip the ROM and sign it. Do the following:
Code:
cd $HOME/ra1.6.2
zip -r update.zip *
mv update.zip $HOME/ROM-cooker/update.zip
cd $HOME/ROM-cooker
./sign.pl update.zip
10. The ROM is now signed and you now have a file called update-signed.zip. Connect the phone to your computer and execute thus:
Code:
./push update-signed.zip
11. Now you are ready to flash the modified ROM which will auto-mount an ext2 partition on your microSD. There is no need to wipe before flashing. If you have no prior experience with ROM flashing or whatever just backup your current install. If you're using OpenHOME or anything similar, nothing will be changed or damaged but if you're using MontAlbert's themes with the ROM you will have to flash them again after flashing this modified ROM.
Code:
adb reboot recovery
12. Flash from choose zip and of course choose update-signed.zip. Reboot. After the system boots up again you can now check whats what with either one of the commands:
Code:
[email protected]:~$ adb shell mount | grep sdext2
/dev/block/mmcblk0p2 on /sdext2 type ext2 (rw,noatime,nodiratime,errors=continue)
[email protected]:~/boot$ adb shell busybox df -h | grep sdext2
/dev/block/mmcblk0p2 893.7M 13.0K 846.0M 0% /sdext2
13. Voila! Your RA 1.6.2 ROM now detects and mounts your microSD ext2 partition on boot. Woohoo?
I hope the HOW-TO was easy reading and that you have succeeded in hacking up your ROM. I know that certain ROMs have this as a built-in function but Amon Ra's does not. But since alot of people including myself use his ROM because of the high speed and stability I thought I should contribute to his project and add a cool (and missed?) function to it.
Mind you that you can use the ROM-cooker set to further adjust and hack up the ROM as you see fit. Happy learning!

Very nice!
Now the question many people will ask : why would you automount ext2 if you don't use apps2sd ?

I personally have ubuntu on my ext2 And besides this approach can be used for a number of things, people who have had the need, or wanted to experiment with init.rc doing things on boot, the mountsd script can easily be altered to do what ever needed.
For me its been a learning curve finding these things out, so by sharing it I may spare some people breaking their backs over this whole init.rc thing. people may want to modify init.rc for whatever reason, so I'm sure people wont have a problem finding a way of putting this to use, and its a subject that isnt all that covered on the forum .. and hey .. at least they get a rom kitchen out of the whole shabang

Very interesting! Thank you.
I used your unpack-program to unpack a recovery-image. It seems to work fine. What I am trying to do is change the state the recovery-image returns the phone to. Would it be possible to just replace your mountsd-script with, for example, a script that installs apps? Or is there a better way to do what Im trying to achieve?
Cheers,
edit: I noticed that on the emulator it is sufficient to just place an apk-file in "data/app" to get it installed. Could it be possible that this is all I need a script to do? :O or could I hurt my poor phone by doing so you think?

sandis84 said:
edit: I noticed that on the emulator it is sufficient to just place an apk-file in "data/app" to get it installed. Could it be possible that this is all I need a script to do? :O or could I hurt my poor phone by doing so you think?
Click to expand...
Click to collapse
That's indeed all you need to do.

Hi!
So I tried to create a signed update.zip, but it failed. It didnt create a "update-script"-file, so my device refused to install it. I wrote my own "update-script"-file, but then it complained "no digest" for the file. How do I solve this?

post the contents of your script people might see whats up

so is this all on linux?
also where are the script files for your tutorial
thanks for the time to put together

sitimber said:
so is this all on linux?
also where are the script files for your tutorial
thanks for the time to put together
Click to expand...
Click to collapse
Says where its at in the first line : )
Code:
wget http://www.grindhouse.no/androidtools/install.sh
But now that I checked, I have to apologize, I see I have a missed payment with my hosting, I'll fix that within the day. Also sorry I havent been answering the few questions here I've been afk cause of surgery.

sitimber said:
post the contents of your script people might see whats up
Click to expand...
Click to collapse
well, I looked in another "update-script" file and found this:
assert compatible_with("0.2") == "true"
assert getprop("ro.product.device") == "dream" || getprop("ro.build.product") == "dream"
show_progress 0.5 0
write_radio_image PACKAGE:radio.img
show_progress 0.5 10
Click to expand...
Click to collapse
So I figured that nothing was essential other then the line "write_radio_image PACKAGE:radio.img". Also ofcourse I made sure it contained the name of my image-file instead of "radio.img". This gave me the "no digest" message, so now I feel unsure on how to create a working update.zip.
edit:
SOLVED! How silly of me. When you sign the update, a hash of each file is put in manifest.mf. Since I added the update-script after signing the file, ofcourse the digest(hash) was missing. Now everything works alot better and I can proceed... until I get stuck again
Cheers,
edit2:
Just to get a better understanding, what exactly does each line do here? Or where can I read about this?
Code:
service mountsdext2 /system/bin/mountsd
user root
group root
oneshot
edit3:
Ok, so I have experimentet, but I still dont manage to solve those last steps. I tried to edit init.rc and just add "mkdir /testdir 0000 system system" where the other directories were created. I then repacked it, zipped it, signed it, put it on my sdcard, started up a custom recovery, installed the update and rebooted. Everything seems to work fine. But when I start adb and check around, I dont see the "testdir"-directory. Also when I check in init.rc my line is gone. Do you guys have an idea of where I went wrong?

sitimber said:
so is this all on linux?
also where are the script files for your tutorial
thanks for the time to put together
Click to expand...
Click to collapse
it doesnot necesarily have to be linux ...you can also do it in windows using cygwin and dsxda's android rom kitchen

[HOW TO] Create ROMs (So you can stop PMing me ;) )

Note: This guide was written for Froyo
Well to start, the Samsung leaked firmwares (many of them, at least) are located here:
http://www.samfirmware.com/WEBPROTECT-i9000.htm
You unpack that with WinRAR, and you'll get a modem.bin, factory.rfs, zImage, cache.rfs, and maybe some other crap. The modem.bin and zImage are useless as they're the modem and kernel, which would technically work on a Captivate but is less than ideal. For example the screen will be rotated 90 degrees incorrectly (Captivate kernels based on i9000 firmwares that have the problems fixed are frequently called reoriented kernels for this reason). The speaker will only work if headphones are plugged in (and headphones only work when they're not plugged in, which is a bit of a problem).
The cache.rfs has the CSC folder, mostly, which you can use but it's also not ideal. If there's ever an sbl.bin or boot.bin those need to be deleted as they're locked bootloader files. The leaked firmware for the Tab had these and people got locked into Samsung firmwares from flashing them. Don't know if that was ever fixed or not.
Then there's factory.rfs. You can open up the .rfs in Magic ISO and extract the /system files.
From there the system apps have to be deodexed, (which takes the .apk files and their corresponding .odex file and puts it into one single .apk file). This allows the .apk to be modified without worrying about the .odex becoming incompatible. The /system/framework contains a bunch of .jar files and two .apks that have to be deodexed also.
From there you can begin to remove the useless junk. The Samsung Apps, the FM Radio (doesn't work), the Book Reader app, Voice Recorder, etc. Some things you're stuck with, the Contacts and the Dialer.
In this particular firmware base (JPY) there's a couple things that are messed up that need to be addressed. The wifi firmware needs to be rolled back to an older one to improve battery life, the gallery app needs to be swapped out to prevent a battery drain problem, there's a camera-related driver file that needs to be swapped out so the Camera will work without the Email app being present. Things like that.
The .apks can be decompiled to edit them (for example Contacts, Dialer, etc. have "Video Call" buttons that need to be edited out in the .xml files that are contained). This can de done with APK Manager, or directly with APK Tool. Either way it comes down to changing things in Notepad++. I've got a guy named ChanceM that I talk to in IRC... he's great with this part and helped with a lot of different things. Particularly the 4 lock screen mod, and he keeps a battery mod maintained for people that like that sort of thing.
As far as theming them, you can either copy over images in 7-zip, or decompile them with APK Manager. Depends on what you need to do. To change colors of text, what the text says, layouts, etc. you need to decompile it and look at the .xml files. If it's a .png that doesn't end in .9.png then you can just copy it over as you see fit.
If the .png ends in .9.png, there's a special 1 pixel border around the image that only appears after it's been decompiled. It's either black or transparent and that governs the stretching of the image. This is called 9patch data. If you simply copy over an image without decompiling it, decompiling it afterwards will not work correctly and you'll have to recreate the image with that one pixel border.
As far as what the 9patch data actually tells the system, I'll differ to this picture I found on developer.android.com:
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
This is used on buttons and things, so that one image can be used to create multiple buttons with different sizes. For example:
Beyond the theming of the system apps and the (tw)framework-res.apk (these two are used to draw things like the status bar and menus), there's also some heavy duty modifying to the android.policy.jar. This changes the Power Menu such that Reboot/Recovery/Download are added to the bottom of the standard options. This gives easy access to those modes without the use of the SDK. The fix for the home haptic button is also in android.policy.jar. The lock screens are all inside android.policy.jar.
Unlike .apks, it's a bit more difficult to modify the .jar files. You've got to open the .jar in 7-zip or some such, and then extract a classes.dex file into any old folder. You'll also need a baksmali.jar and smali.jar inside that folder too. Then (after you've installed the Java SDK, Android SDK, and added them to the Windows PATH file) you must enter into a Command Prompt the following:
java -jar baksmali.jar -o classout/ classes.dex
Click to expand...
Click to collapse
This creates a classout folder that contains an entire hierarchy of .smali files ready to be edited in Notepad++. Not as easy as .xml edits, because these are written in straight up code and can be thousands of lines long for each .smali. However, once you're done, the following will convert it back into a classes.dex file, which can replace the original using 7-zip:
java -Xmx512M -jar smali.jar classout/ -o classes.dex
Click to expand...
Click to collapse
From there you can add any apps and tweaks you like. A custom kernel, a modem you like, and you're almost done.
You've still got to install the files for busybox, root, etc. Those go in /system/xbin.
Once you've got it down to a useable state you've got to write an update-script to actually install it. This is what mine is:
show_progress 1 0
Click to expand...
Click to collapse
Sets the progress bar equal to zero.
format SYSTEM:
format CACHE:
format DATADATA:
format DATA:
delete_recursive SDCARD:.android_secure
delete_recursive SDCARD:Voodoo
delete_recursive SDCARD:Android
delete_recursive DBDATA:
Click to expand...
Click to collapse
Out with the old...
copy_dir PACKAGE:system SYSTEM:
copy_dir PACKAGE:data DATA:
copy_dir PACKAGE:updates TMP:/updates
copy_dir PACKAGE:sdcard SDCARD:
Click to expand...
Click to collapse
...and in with the new.
symlink toolbox SYSTEM:bin/cat
symlink toolbox SYSTEM:bin/chmod
symlink toolbox SYSTEM:bin/chown
symlink toolbox SYSTEM:bin/cmp
symlink toolbox SYSTEM:bin/date
symlink toolbox SYSTEM:bin/dd
symlink toolbox SYSTEM:bin/df
symlink toolbox SYSTEM:bin/dmesg
symlink toolbox SYSTEM:bin/getevent
symlink toolbox SYSTEM:bin/getprop
symlink toolbox SYSTEM:bin/hd
symlink toolbox SYSTEM:bin/id
symlink toolbox SYSTEM:bin/ifconfig
symlink toolbox SYSTEM:bin/iftop
symlink toolbox SYSTEM:bin/insmod
symlink toolbox SYSTEM:bin/ioctl
symlink toolbox SYSTEM:bin/ionice
symlink toolbox SYSTEM:bin/kill
symlink toolbox SYSTEM:bin/ln
symlink toolbox SYSTEM:bin/log
symlink toolbox SYSTEM:bin/ls
symlink toolbox SYSTEM:bin/lsmod
symlink toolbox SYSTEM:bin/mkdir
symlink toolbox SYSTEM:bin/mount
symlink toolbox SYSTEM:bin/mv
symlink toolbox SYSTEM:bin/nandread
symlink toolbox SYSTEM:bin/netstat
symlink toolbox SYSTEM:bin/newfs_msdos
symlink toolbox SYSTEM:bin/notify
symlink toolbox SYSTEM:bin/printenv
symlink toolbox SYSTEM:bin/ps
symlink toolbox SYSTEM:bin/reboot
symlink toolbox SYSTEM:bin/renice
symlink toolbox SYSTEM:bin/rm
symlink toolbox SYSTEM:bin/rmdir
symlink toolbox SYSTEM:bin/rmmod
symlink toolbox SYSTEM:bin/route
symlink toolbox SYSTEM:bin/schedtop
symlink toolbox SYSTEM:bin/sendevent
symlink toolbox SYSTEM:bin/setconsole
symlink toolbox SYSTEM:bin/setprop
symlink toolbox SYSTEM:bin/sleep
symlink toolbox SYSTEM:bin/smd
symlink toolbox SYSTEM:bin/start
symlink toolbox SYSTEM:bin/stop
symlink toolbox SYSTEM:bin/sync
symlink toolbox SYSTEM:bin/top
symlink toolbox SYSTEM:bin/umount
symlink toolbox SYSTEM:bin/vmstat
symlink toolbox SYSTEM:bin/watchprops
symlink toolbox SYSTEM:bin/wipe
symlink /system/xbin/su SYSTEM:bin/su
Click to expand...
Click to collapse
This creates symlinks for various commands that you or the system might need.
set_perm_recursive 0 0 0755 0644 SYSTEM:
set_perm_recursive 0 2000 0755 0755 SYSTEM:bin
set_perm_recursive 0 0 0755 0755 SYSTEM:etc
set_perm_recursive 1002 1002 0755 0440 SYSTEM:etc/bluetooth
set_perm_recursive 0 0 0777 0777 SYSTEM:etc/init.d
set_perm 0 3003 02755 SYSTEM:bin/netcfg
set_perm 0 3004 02755 SYSTEM:bin/ping
set_perm 0 0 0755 SYSTEM:etc/bluetooth
set_perm 1002 1002 0440 SYSTEM:etc/dbus.conf
set_perm 1014 2000 0550 SYSTEM:etc/dhcpcd/dhcpcd-run-hooks
set_perm 0 2000 0550 SYSTEM:etc/init.goldfish.sh
set_perm 0 0 04755 SYSTEM:xbin/su
set_perm 0 0 04755 SYSTEM:xbin/busybox
set_perm 1000 1000 0771 DATA:app
set_perm 0 0 755 TMP:/updates/redbend_ua
Click to expand...
Click to collapse
This sets the permissions on the files and folders to what's necessary.
run_program /system/xbin/busybox --install -s /system/xbin
Click to expand...
Click to collapse
This will install busybox.
run_program /tmp/updates/redbend_ua restore /tmp/updates/modem.bin /dev/block/bml12
Click to expand...
Click to collapse
This installs the modem.
run_program /tmp/updates/redbend_ua restore /tmp/updates/zImage /dev/block/bml7
Click to expand...
Click to collapse
This installs the kernel.
show_progress 1 10
Click to expand...
Click to collapse
That's a followup command to the original command for the progress bar.
-----
That pretty much covers the basic stuff. I tried to keep it as short as possible.

Nice job. This should help alot of people.
And like you said. Stop "some" of the PMs for you...

Glad to see this all put together. Gives everyone a better idea of all the hard work you guys do.
Sent from my SGH-I897 using XDA App

I agree. It took me a while to figure it out. And this is so much more detailed. Great job Mikey.

IamSonoma said:
I agree. It took me a while to figure it out. And this is so much more detailed. Great job Mikey.
Click to expand...
Click to collapse
10charchar

Two things....
1)THIS NEEDS TO BE STICKIED
2) Oh just wait for it..... wait.....wait... for the DEV section to get cluttered with an ass-ton of freshly released, half-working, quadrant scores of 3 million, CWM flashable brick.zip, works of art
Thank you Mikey for the detailed write up.

vunuts said:
Two things....
1)THIS NEEDS TO BE STICKIED
2) Oh just wait for it..... wait.....wait... for the DEV section to get cluttered with an ass-ton of freshly released, half-working, quadrant scores of 3 million, CWM flashable brick.zip, works of art
Thank you Mikey for the detailed write up.
Click to expand...
Click to collapse
No don't curse it.although we all know is bound to happen. No good deed goes unpunished
Sent from my SGH-I897 using XDA App

crystalhand said:
No don't curse it.although we all know is bound to happen. No good deed goes unpunished
Sent from my SGH-I897 using XDA App
Click to expand...
Click to collapse
I just figured I might as well throw it out there so, BOOM, it's here on the front page of this thread, so hopefully people see this and don't go into the DEV section and decide to flash one of the 46 new rom's that might pop up in the next week or so.
You notice how quiet this thread is?
People are up to something.... And I don't like it lol.

vunuts said:
You notice how quiet this thread is?
People are up to something.... And I don't like it lol.
Click to expand...
Click to collapse

vunuts said:
I just figured I might as well throw it out there so, BOOM, it's here on the front page of this thread, so hopefully people see this and don't go into the DEV section and decide to flash one of the 46 new rom's that might pop up in the next week or so.
You notice how quiet this thread is?
People are up to something.... And I don't like it lol.
Click to expand...
Click to collapse
Can't be any worse than some of the cut and paste rom building that already occurs.
Nice that you took the time to write this up mikey.

MikeyMike01 said:
Click to expand...
Click to collapse
^^^^Yea, something like that^^^^
I thought most of the cut and paste rom's got locked? I know that the ports from different galaxy s phones are legit, I just thought it was the people building in the kitchen and posting as their own got shut down.
Eh, whatever though. People will do what they do, more power to them.

vunuts said:
I thought most of the cut and paste rom's got locked?
Click to expand...
Click to collapse
Some peoples release dates been a little too... convenient.
Sent from my Captivate.

MikeyMike01 said:
Some peoples release dates been a little too... convenient.
Sent from my Captivate.
Click to expand...
Click to collapse
I for one may never post mine. Not to be rude or anything. There are currently many roms to choose from. Most are well done.
For me it is just a learning experience. I managed to get the HTC Kitchen working. After figuring out a few quirks. My custom rom is JS3. I figure I could help out more if I understood the steps from beginning to end.
I could know I could just use someone else but there is no personal gain out of copying someone's else stuff if you can't do it yourself.
Thank you Mikey for thinking of others and taking the time especially for the individuals who want to learn.

IamSonoma said:
Thank you Mikey for thinking of others and taking the time especially for the individuals who want to learn.
Click to expand...
Click to collapse
There's almost nothing on Samsung ROM...ing. It's all HTC/AOSP/etc.

U R the MAN
MIKEY_MIKE my humble hat off 2 U

Mike this is a great post, I really appreciate the info. This will help answer a question I've been pondering a while- can the default tab in the dialer be changed (say to "Call logs" or "Favorites")?
I'm guessing if I decompile the dialer APK I may be able to find such a setting in its XML.
If anyone already knows the answer to this feel free to let me know.

This definitely shows how much work it takes to make a ROM. Too time consuming for me to try this anytime soon. I've only customized.a theme, a mashup of many different ones. I thank all the devs though for their hardwork.
Sent from my Captivate running Cezar's Continuum v.2.0 with hardcore's k12q kernel.

tzabka said:
MIKEY_MIKE my humble hat off 2 U
Click to expand...
Click to collapse
10charararar

Woot I got a sticky!

Congrats...well deserved!

[BETA][TOOL] adb_bak2computer [MAC & LINUX][UPDATED: 11/15/2012]

BETA TESTERS NEEDED!!
DESCRIPTION: adb_bak2computer.sh
- tool (set of scripts) to backup d2vzw (SCH-I535) partitions directly to a users computer.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
PROGRESS:
- Beta5 - Now in two flavors, Mac and Linux!!
- Fully functioning tarballs are attached to the OP (as always)
TO-DO/KNOWN BUGS
- Re-wrote the code so it actually works on Linux now. (See the _Linux tar)
- All known bugs fixed. Unless someone points one out.
- To Do: Add functionality to wrap backups into a flashable zip (it'll be a big zip, but its one file to keep track of instead of blobs or sets of img files) This should be the first restore option I explore.
- To Do: Automate a standard backup (include: System, Boot, Data)
- To Do: Automate a set of backups that are compatible with TWRP. Why TWRP, cause I like it, and the differential backups that CWM does will be problematic for this tool. (I may need some help on this one.)
WARNING USE AT YOUR OWN RISK
I am not responsible if this tool destroys your soul, or your phone, or it makes a rainbow rhinoceros shoot out your bum.
NOTE: We all know the risks of doing stuff like this by now right? But the warning is still par for the course I guess. Is there a standard warning file I could link to, and make it all fancy and legal? Kinda like creative commons license only creative warning instead? Moving on...
PREREQUISITES
YOUR DEVICE MUST BE ROOTED
Linux or Mac operating system (maybe Cygwin might work, but I haven't tested it)
If you are on Linux you must have xterm installed (gnome-terminal or konsole alone won't work)
Must have the android SDK installed, and "platform-tools" must be in your $PATH
Process Viewer, "pv", must be installed
NOTE: To check if you have Process Viewer installed run pv --version in your terminal, if its not there download from here and install.
INSTRUCTIONS:
Download the appropriate tar for your OS
Extract it to a folder named "adb_bak2computer" in your $HOME directory.
Chmod everything so its all executable (chmod -Rf 777 ~/adb_bak2computer ).
Make sure your SGS3 is plugged into your machine via USB, and that you have USB debugging enabled.
If you are on CM10 or some other AOSP based ROM check the preference that keeps the screen on while plugged in.
If you are on a TouchWiz ROM use something like Wake Lock - PowerManager and force a full wakelock so your device doesn't go to sleep and or lock out adb debugging.
Run adb_bak2computer.sh from the terminal.
Pick a number
ALL CREDIT GOES TO:
das7982 - for ODIN guide here:
http://forum.xda-developers.com/showpost.php?p=28876440&postcount=1
scandiun - for "nandroid" backup directly to computer guide here:
http://forum.xda-developers.com/showpost.php?p=29862574&postcount=1
(and whomever they credited in their guides as well)
Feel free to fork, download, mod, make it all pretty, whatever you want. Its all open for sharing.
REPOS:
http://github.com/ALQI/adb_bak2computer-Mac
http://github.com/ALQI/adb_bak2computer-Linux
PM me if you want to help me with this tool, or just want to have a nice cup a tea.
Ta,
ALQI
REFERENCE THREADS:
El Grande Partition Table Reference -- E.V.A
Unlock Bootloaders -- AdamOutler
Android Kitchen -- dsixda

Look's kinda cool. Can't wait till this is finished!

Why not just use "rsync for android" from the market?
Sent from my Galaxy Note 10.1 using Tapatalk 2

blulite said:
Why not just use "rsync for android" from the market?
Sent from my Galaxy Note 10.1 using Tapatalk 2
Click to expand...
Click to collapse
That's for file based backups, which is great and I love rsync and ssh, but its not what I'm looking for. I want full on partition images going directly from my phone to my laptop (hopefully at USB2.0 speeds). So far the only way that I have seen that can do this is with scandiun's guide here:
http://forum.xda-developers.com/showpost.php?p=29862574&postcount=1
It works. I've tried scandiun's original directions and it does produce viable iamges. What I want to do is automate/simplify that process.
So far, I haven't had a chance to figure out how to get adb to play nice with spawning subprocesses in bash scripts.
like :
Code:
#!/bin/bash
function1(){
echo "function is running"
adb forward tcp:5555 tcp:5555
adb shell /system/xbin/busybox nc -l -p 5555 -e /system/xbin/busybox dd bs=4096 if=/dev/block/mmcblk0p17
}
function2(){
echo "function2 has to finish execution before either function can exit"
adb forward tcp:5555 tcp:5555; cd ./Cache_Test; nc 127.0.0.1 5555 | pv -i 0.5 > ./mmcblk0p17_cache.img
wait
echo "function2 is done"
}
function0(){
(function1)&
(function2)&
wait
exit
}
function0
wait
exit
This code doesn't work though. And I'm pretty sure its because adb needs actual separate terminals (not subprocesses) to run the two instances of adb forward.
This means I need a portable way to spawn new terminals. I've tried xterm -e [command] and that kinda works on linux but not on mac and I doubt it would on cygwin/windows. I may have to re-write this in python, but I'd rather use bash cause not everyone has python installed or is comfortable using it. Also, I'm lazy and I don't want to have to re-write what seems to be perfectly viable code.
Ta,
ALQI
FYI - I thought the cache partition would be a small safe partition to play around with, but its like 800MB on my device. I'll pick another for my next test/example.

Eureka (I think)
Ok, so I have a set of scripts now that can do what I need. That is execute in new terminal windows and play nice with adb.
This script (which can be .sh file), calls the other two .command files and actually backs up my cache partition.
Code:
#!/bin/bash
cd ./adb_bak2computer
open ./cache_bak1.command
sleep 2
open ./cache_bak2.command
wait
echo "cache should be backed up."
Here's the first command file:
Code:
#!/bin/bash
cache_bak1(){
echo "cache_bak1 is running"
adb forward tcp:5555 tcp:5555
sleep 2
echo "/system/xbin/busybox nc -l -p 5555 -e /system/xbin/busybox dd bs=4096 if=/dev/block/mmcblk0p17" | adb shell
wait
}
cache_bak1
Here's the second one:
Code:
#!/bin/bash
cache_bak2(){
cd ./adb_bak2computer
echo "cache_bak2 is running"
adb forward tcp:5555 tcp:5555
sleep 2
nc 127.0.0.1 5555 | pv -i 0.5 > ./mmcblk0p17_cache.img
wait $!
echo "cache_bak2 is done"
}
cache_bak2
For some reason I to pipe in an echo of the command I wanted to run in "adb shell".
Also, depending on your terminal settings, the windows for the two command files will stay open even when they are done. To fix that, just make sure your preferred terminal emulator is set to close when a process is complete.
I'd still like to get subprocesses (from post #4) to work but its stating to look like adb no likey.
I'll try and have the rest of the code for all the paritions up within the next few days, I still have some things to figure out though. I can use a menu to seperate the dd instances or loop/wait for the pid's of each instance to quit before moving on to the next partition, I'm on the fence as to whether I want everything to run consecutively or to give users the option to pick each partition they want to back up. Maybe I can do both, but that will take longer for me to finish.
Please excuse the crap typing as I'm doing this all in my "spare" time, which means I should be sleeping.
Ta,
ALQI

Why not just make scripts pull each partition? Adb can simply pull them......
Sent from my SCH-I535 using xda app-developers app

tonu42 said:
Why not just make scripts pull each partition? Adb can simply pull them......
Sent from my SCH-I535 using xda app-developers app
Click to expand...
Click to collapse
IIRC, adb pull is for files only. It can't pull full partition images.
I'm getting full partition images that can be (hopefully) used for Odin restoration and maybe nandroid or cwm restore.
Thanks for the input.
Ta,
ALQI
Sent from my SCH-I535 using xda app-developers app

ALPHA Version
OP is updated with fully functional alpha release of the tool.
Ta,
ALQI

Going to have to check this out over the weekend when I can sit down and read first.
Thank you for providing us with an alternative backup solution.
Sent from my SCH-I535 using Tapatalk 2

ok, so backing up the data block is HUGENORMOUSBIG
maybe mmcblk0p15 is including the internal sdcard because of that /data sub folder link to the sdcard?
I dunno for now, I'm going on 45mins and 6Gigs. I'll have to take a look in teh morning

Just wanted to let people know that this is not abandoned.
I only have one last hurdle. And that's getting an img file for /data without backing up /data/media (which is essentialy your internal sdcard). I can't really do a block dump cause mmcblk0p15 (am I even close there) is all userdata including /data/media.
Any ideas would be helpful? (Hint Hint)
Ta,
ALQI
Sent from my SCH-I535 using xda app-developers app

update ready to roll for Beta testers
alquimista said:
Just wanted to let people know that this is not abandoned.
I only have one last hurdle. And that's getting an img file for /data without backing up /data/media (which is essentialy your internal sdcard). I can't really do a block dump cause mmcblk0p15 (am I even close there) is all userdata including /data/media.
Any ideas would be helpful? (Hint Hint)
Ta,
ALQI
Sent from my SCH-I535 using xda app-developers app
Click to expand...
Click to collapse
I got /data backup working (yay me).
OP updated!!

alquimista said:
- Thread title updated with your American way of noting the date
Click to expand...
Click to collapse
My American way of noting the date is an ISO standard... YYYY-MM-DD

OP updated with new beta2 version of the tool.
Github commited as well.
ALL mmc blocks are set for backup now, check in the "CRAZY BACKUP OPTIONS" menu for the crazy backup options.
Next up:
Fix the echo [command] | adb shell scripts not exiting without CTRL+C
Option to have the basic backups turned into an "update".zip
Ta,
ALQI

Beta3
OP Updated with Beta3
Android side adb hang bug is fixed. Yay me.
GitHub updated as well.
Ta,
ALQI

HAAAAALLP!!!
I'm struggling a bit with creating a flashable zip with the files I get using my tool.
Basically I can pick whatever partition I want, but lets just say I want a standard backup and I've run my tool pulling the following:
Code:
boot.emmc.img
cache.ext4.img
data.ext4.tar
recovery.emmc.img
system.ext4.img
Actually, ican name them whatever I want and use whatever I want, so lets simplify that even greater and just go with:
Code:
boot.img
data.tar
system.img
The updater-binary and accompanying updater-script is pretty straight forward for the boot.img and system.img, the script would have something like:
Code:
write_raw_image PACKAGE:system.img SYSTEM:
write_raw_image PACKAGE:boot.img BOOT:
That's a bit of an over simplification depending on the updater-binary I wind up using, but its the right idea (some of the wording might change a bit but the "write_raw_image" part is whats important).
But what do I do with data.tar? Can I unpack that tar directly to "DATA:"? Something like:
Code:
package_extract_dir PACKAGE:data DATA:
I don't know if the updater-binary supports this with a tar??
I could unpack the tar file before creating the zip, but then I have to set all the permissions, which should have been preserved in the tar. Or can I just write a little script to go inside the zip that will essentially "tar -xf /tmp/data.tar /data" and unpack the tarball while in recovery?
I dunno, any ideas??
This would be alot easier if I could workout a way to dd a data.img without including the internal sdcard.
Ta,
ALQI

Beta4
OP Updated with Beta4
Github as well
Added funtionality to backup internal and external SDCards.
Ta,
ALQI

Important: Mac only
I just realized that this code probably won't work on Linux.
I'm on a Mac for most of my work, so I've been making all this on a Mac (OS X Lion 10.7). The "open" command probably won't work on linux or Cygwin.
I just need to add a quick check for for the OS and then use "xterm" instead of "open" for the .command files.
Actually, since I'm doing alot of cleanup and such, I will probably remove file extensions for most of the scripts.
Anyway, sorry for any confusion. I haven't been awake for most of this. But its definately a great big FAIL on my part. Well at least until I fix it.
Ta,
ALQI

BETA5 - Now works on Linux!!
OP Updated (as well as new repos) for new linux version.
Had to seperate out a version for linux cause Macs are dumb and can't open new terminal windows without the "open" command.
Tars are attached to the OP!!

JB 4.2 Nested /0 Fix Script

Hey-
I was wondering if I could get some help and feedback on a script I conceived to help 4.2 multi-user tree structure to work with pre 4.2 backups (Device, ROMs, Recoveries, Nandroids, Titaniums). Can one of you guru's (_that, becomingx, etc.) chime in and let me know if this seems logical and a good solution, what syntax errors I have, and partially test the theory and code as I am still on 4.1 and locked.
Assuming you have no stock /*/0 directories this script finds all instances of /*/0 (created from the 4.2 multi-user tree structure). /* is "the parent directory" and /*/0 is the "nested directory". First it backs up the parent directory then copies the nested directory (overwrites) into their parent directory. It also combines the backup and the parent directory in the parent directory at the end. This script assumes the user to be single-nested and after running it allows pre-4.2 backups to find their data and targets for restoration. Maybe its OK to combine before the restoration, or after, or not at all? Please give me your feedback on this and don't beat me up too hard as this is my first script with a variable and an array.
Code:
#!/system/bin/sh
nests=( grep ! /*/0 )
for NEST in "${nests[@]}"
do
mkdir /Removable/MicroSD/$NEST_bak
cp -r /$NEST /Removable/MicroSD/$NEST_bak
cp -rf /$NEST/0 /$NEST
#[Restore Pre-4.2 backups]
cp /Removable/MicroSD/$NEST_bak/*.* /$NEST
#[Backup in 4.2]
rm -rf /Removable/MicroSD/$NEST_bak
done
[Edit] Yup, this script needs some serious help. The first line doesn't even work. But when you type it as a command from the prompt it does what I'm looking to capture. How can I put this output in an array?
Code:
grep ! /*/0
Why after running the code above does my prompt which was "[email protected]: #" now show "1|[email protected]: #"? When I was trying to debug the script I even got a 2 and 3|[email protected]: #" Are these some kind of grep shells?

I've now read your post twice, and I still have no idea what you want to do - neither from the script itself nor from the textual description. The confusion starts with the fact that I have no "0" directories on my MicroSD card.
The first step in software development is always to clearly define the problem that should be solved by the software. As long as I don't understand what you want to do, I can't help you with the script - except maybe a hint: "grep" searches for text inside files, "find" searches for files or directories.
Why after running the code above does my prompt which was "[email protected]: #" now show "1|[email protected]: #"? When I was trying to debug the script I even got a 2 and 3|[email protected]: #" Are these some kind of grep shells?
Click to expand...
Click to collapse
The number is the last exit code from the previous command. It indicates that something went wrong.

_that said:
I've now read your post twice, and I still have no idea what you want to do - neither from the script itself nor from the textual description. The confusion starts with the fact that I have no "0" directories on my MicroSD card.
Click to expand...
Click to collapse
No, but you do have some in `/storage/emulated' and `/mnt/shell/emulated' , which I *think* might be what elfaure is referring to (confirm?).
The first step in software development is always to clearly define the problem that should be solved by the software. As long as I don't understand what you want to do, I can't help you with the script - except maybe a hint: "grep" searches for text inside files, "find" searches for files or directories.
Click to expand...
Click to collapse
True words indeed. Although, just telling someone what you want to do and having them show you how to do it, or having them do it for you, is never as much fun as trying to do it for yourself first.
elfaure said:
Hey-
I was wondering if I could get some help and feedback on a script I conceived to help 4.2 multi-user tree structure to work with pre 4.2 backups (Device, ROMs, Recoveries, Nandroids, Titaniums). Can one of you guru's (_that, becomingx, etc.) chime in and let me know if this seems logical and a good solution, what syntax errors I have, and partially test the theory and code as I am still on 4.1 and locked.
Assuming you have no stock /*/0 directories this script finds all instances of /*/0 (created from the 4.2 multi-user tree structure). /* is "the parent directory" and /*/0 is the "nested directory". First it backs up the parent directory then copies the nested directory (overwrites) into their parent directory. It also combines the backup and the parent directory in the parent directory at the end. This script assumes the user to be single-nested and after running it allows pre-4.2 backups to find their data and targets for restoration. Maybe its OK to combine before the restoration, or after, or not at all? Please give me your feedback on this and don't beat me up too hard as this is my first script with a variable and an array.
Code:
#!/system/bin/sh
nests=( grep ! /*/0 )
for NEST in "${nests[@]}"
do
mkdir /Removable/MicroSD/$NEST_bak
cp -r /$NEST /Removable/MicroSD/$NEST_bak
cp -rf /$NEST/0 /$NEST
#[Restore Pre-4.2 backups]
cp /Removable/MicroSD/$NEST_bak/*.* /$NEST
#[Backup in 4.2]
rm -rf /Removable/MicroSD/$NEST_bak
done
Click to expand...
Click to collapse
This is indeed a bit confusing. First I need to know what you're expecting `(grep ! /*/0)' to evaluate to before I can comment on the rest of the code.
[Edit] Yup, this script needs some serious help. The first line doesn't even work. But when you type it as a command from the prompt it does what I'm looking to capture.
Click to expand...
Click to collapse
To capture the output of a command into a variable, you need a dollar sign in front of the opening parenthesis, i.e.:
Code:
nests=$(grep ! /*/0)
But, as _that mentioned, the `grep' call probably isn't the right one to use here. `find' can surely do what you want, but requires busybox to be installed and usually needs to have a bunch of options supplied to it to make sure it's giving you exactly what you want and expect. A simple `ls' command might be all that's needed.
How can I put this output in an array?
Click to expand...
Click to collapse
I, and most other people based on the code I've seen, don't use arrays much in shell scripts. Usually a value separated list is used instead, where the separator value is a space, tab, or newline. Take the example:
Code:
ROOT_LISTING=$(ls /)
for OBJ in $ROOT_LISTING; do
...
done
When the variable $ROOT_LISTING gets expanded at execution time it winds up looking like:
Code:
for OBJ in Removable acct cache [snipped for brevity]; do
and so the `for' loop will iterate through each value after the `in' keyword that is seperated from the next one by a space, tab, or newline. (If we have files, directories, etc. with spaces (possible), tabs (unlikely), or newlines (also unlikely) in the name, we have to be trickier, but I won't get into that here. )
Hopefully this helps. If there's anything else technical that I can yammer on about for the betterment of your understanding, just holler. :good:

Hi Guys-
Thanks for the reply. What service! I simply call for the two guys I need most and like rubbing a magic lamp my genies appear.
When _that reads it twice and still has no clue what you are trying to do you know you need some serious schooling. That's why I'm here - to listen twice as often as I speak and learn something useful.
So here's what I am trying to do. JB 4.2 implemented a radical depart from previous Android versions to the directory tree structure for multi-users as you know, similar to the way Windows uses user profiles to separate users desktops, accessible installed apps, etc. This new structure created /0 subdirectories for certain directories like /sdcard and /sdcard/0 (/data/media and /data/media/0, /clockworkmod and /clockworkmod/0) and others. Playing with grep I discovered I could find all instances of any directory containing a /0 subdirectory (pair). "Find" seems a better approach than "grep" and works exactly the same with no fancy options required.
Code:
find */0
For these directory pairs I'll call them from the above output, the data that was in /directory in 4.1 is now located and pointed to in /directory/0 in 4.2. I figured if I could find all these pairs, and copy /directory/0 into /directory (after backing up /directory to SD) then 4.1 backups looking for data in /directory could now find their data and targets. Then I could copy the SD backup of /directory into /directory (now overwritten by /directory/0) to merge everything into a single folder. But after thinking about this again, this is incorrect. What I meant to do at this step is copy all /0 subdirectories of a given /directory/0 into that folder [ /directory/0/0 into /directory/0 and directory/0/0/0 into /directory/0 and /directory/0/0/0/0... into /directory/0 ] to combine all these sub-nested directories into a single /0 directory. Deep subnesting causes by crack flashing custom ROMs over and over on JB 4.2. This is the concept and I'm trying to implement a script to do it.
So I will attempt to replace grep with find and the array with a variable value space separated list and see what I come up with. Becomingx, you are right about wanting to use my brain a bit to figure it out on my own, even if it sends me down a one-way road to a dead end. If I learned something in the process then its all good and I can always ask for directions at the locked gate if I arrive there, but even with 4 flat tires and a broken axle I'm still moving (forward hopefully) as long as there's still road to travel on.
Thks

So you want to write a script to downgrade from 4.2 to 4.1?

Test new avitar
How do you delete a post? Under Edit/Delete I only find Edit??

_that said:
So you want to write a script to downgrade from 4.2 to 4.1?
Click to expand...
Click to collapse
No, I want to write a script that lets you restore a pre 4.2 backup (Nandroid, Titanium, Recovery, ROM) on 4.2 without having to move/copy files.

elfaure said:
No, I want to write a script that lets you restore a pre 4.2 backup (Nandroid, Titanium, Recovery, ROM) on 4.2 without having to move/copy files.
Click to expand...
Click to collapse
Are you sure that this is not already handled by TWRP, or by TB?
Recovery and ROM themselves are not affected by the /0 move in /data anyway.

Thanks becomingx, getting somewhere now but I still have problems.
When I type this at the terminal I get what i am looking for as output
Code:
su
cd /
find */0
Output: (These are test directories I created)
data/0
sdcard/0
sdcard/0/0
But when I run this script, the above output is repeated 3 times? Why is that?? One for each file system /(rootfs), /data, and /sdcard? How can I rewrite the script to produce the same output as from the terminal?
Code:
#!/system/bin/sh
cd /
NEST=$(find */0)
for OBJ in $NEST; do
echo "$NEST"
done

_that said:
Are you sure that this is not already handled by TWRP, or by TB?
Recovery and ROM themselves are not affected by the /0 move in /data anyway.
Click to expand...
Click to collapse
Long story short, yes if you had all the latest and greatest installed before you upgraded to 4.2 then its a transparant issue. But if you're already "nested" because you used an older custom recovery and factory reset a few times to crack flash some some 4.2 ROMs or need to recover to a previous Android version then there is a use for this script. If I can learn how to script variables and help some nested folks out at the same time then _that's what I call a win-win. I'm not on 4.2 nor do I plan on going there anytime soon so this won't help me out less what I learn by doing it, although it is interesting and challenging to try to implement an automated solution for. Plus I'm like you guys, I just really enjoy rooted Android and helping people with it when I can (within my very limited but expanding skill set).
By the way, what is the name and /data path to that special file?
****************************************************************************************************
http://androidforums.com/verizon-ga...t/649940-4-2-sdcard-sdcard-0-observation.html
http://androidforums.com/verizon-ga...y-bean-roms-edited-3-24-13-a.html#post5796630
Quoted from Androidspin.com -
"With Android 4.2, Google introduced multiple users as a new feature. In order to accommodate multiple users, Google is now giving each user a their own folder for storage. If you upgraded to 4.2 from 4.1, then the 4.2 ROM will look for a certain file in /data to determine whether it needs to migrate all of your files to the new multi-user data structure. By default, 4.2 migrates all of /data/media to /data/media/0.
A problem arose though with custom recoveries. A custom recovery retains the /data/media folder during a factory reset. When you factory reset and then boot a 4.2 ROM again, the 4.2 ROM will migrate everything in /data/media again. It will migrate your files every time you factory reset. This multiple migration is what resulted in some people having their files moved to /sdcard/0 or even /sdcard/0/0 etc.
In TWRP 2.3.2.0 we have corrected this problem by ensuring that we do not delete the special file during a factory reset. However, if ended up having your files upgraded you will need to move or merge them back into /sdcard. Also, if you have moved your TWRP folder from /data/media/0 to /data/media so that you could restore backups while using prior TWRP versions, you may now need to move the TWRP folder back into /data/media/0.
As a special note, if you restore a backup to a prior version of Android, you may have to move your files out of /data/media/0 and into /data/media to be able to see them again."

elfaure said:
TBut when I run this script, the above output is repeated 3 times? Why is that?? One for each file system /(rootfs), /data, and /sdcard? How can I write the script to produce the same output as from the terminal?
Code:
#!/system/bin/sh
cd /
NEST=$(find */0)
for OBJ in $NEST; do
echo "$NEST"
done
Click to expand...
Click to collapse
Read your script - for each item in $NEST you output all of $NEST instead of the single item $OBJ.

_that said:
Read your script - for each item in $NEST you output all of $NEST instead of the single item $OBJ.
Click to expand...
Click to collapse
Ah ha. Lost sight of the fact that a multi-object variable really behaves like an array and you can call its objects as independent variables (syntax wise; I thought that was what I was doing with echo $NEST). Variables in a variable. That's why I started with an array. Thanks.
Code:
#!/system/bin/sh
cd /
NEST=$(find */0)
for OBJ in $NEST; do
echo "$OBJ"
done
Output:
data/0
sdcard/0
sdcard/0/0
New Questions:
How can I strip off all the "/0's" from each object variable and store it in a new mutil-variable until there are no more "/0's to strip off and consolidate duplicates? The desired result here would be VAR=$(data, sdcard).
If two or more object variables have the same root directory name, how can I select only the object variable with the most /0's between two variables (sdcard/0 and sdcard/0/0; I would want only the latter) then store those object variable in a new variable including all other object variables? The desired result here would be VAR1=$(data/0, sdcard/0/0)

Code:
#!/system/bin/sh
nests=( grep ! /*/0 )
for NEST in "${nests[@]}"
do
mkdir /Removable/MicroSD/$NEST_bak
cp -r /$NEST /Removable/MicroSD/$NEST_bak
cp -rf /$NEST/0 /$NEST
#[Restore Pre-4.2 backups]
cp /Removable/MicroSD/$NEST_bak/*.* /$NEST
#[Backup in 4.2]
rm -rf /Removable/MicroSD/$NEST_bak
done
becomingx said:
To capture the output of a command into a variable, you need a dollar sign in front of the opening parenthesis, i.e.:
Code:
nests=$(grep ! /*/0)
[snipped]
I, and most other people based on the code I've seen, don't use arrays much in shell scripts. Usually a value separated list is used instead...[snipped]
Click to expand...
Click to collapse
"nests" in my original code here refers to an array not a variable. Assuming that, is the syntax correct (for reference)? I like the multi-item(object) variable value separated list you showed me and _that corrected my use of and will use that instead for this script.

elfaure said:
For these directory pairs I'll call them from the above output, the data that was in /directory in 4.1 is now located and pointed to in /directory/0 in 4.2. I figured if I could find all these pairs, and copy /directory/0 into /directory (after backing up /directory to SD) then 4.1 backups looking for data in /directory could now find their data and targets. Then I could copy the SD backup of /directory into /directory (now overwritten by /directory/0) to merge everything into a single folder. But after thinking about this again, this is incorrect. What I meant to do at this step is copy all /0 subdirectories of a given /directory/0 into that folder [ /directory/0/0 into /directory/0 and directory/0/0/0 into /directory/0 and /directory/0/0/0/0... into /directory/0 ] to combine all these sub-nested directories into a single /0 directory. Deep subnesting causes by crack flashing custom ROMs over and over on JB 4.2. This is the concept and I'm trying to implement a script to do it.
Click to expand...
Click to collapse
Wow! This is very messy, no wonder you're trying to automate the cleanup.
So I will attempt to replace grep with find and the array with a variable value space separated list and see what I come up with. Becomingx, you are right about wanting to use my brain a bit to figure it out on my own, even if it sends me down a one-way road to a dead end. If I learned something in the process then its all good and I can always ask for directions at the locked gate if I arrive there, but even with 4 flat tires and a broken axle I'm still moving (forward hopefully) as long as there's still road to travel on.
Click to expand...
Click to collapse
Love the analogy here :laugh:
elfaure said:
By the way, what is the name and /data path to that special file?
Click to expand...
Click to collapse
Looks to be `/data/.layout_version' [ref].
elfaure said:
Code:
#!/system/bin/sh
cd /
NEST=$(find */0)
for OBJ in $NEST; do
echo "$OBJ"
done
Output:
data/0
sdcard/0
sdcard/0/0
New Questions:
How can I strip off all the "/0's" from each object variable and store it in a new mutil-variable until there are no more "/0's to strip off and consolidate duplicates? The desired result here would be VAR=$(data, sdcard).
Click to expand...
Click to collapse
If we're going full steam ahead with the requirement of Busybox being installed, then this is pretty easy:
Code:
VAR="$(find */0 | sed -e 's!/0!!g' | sort -u)"
Sed is our Stream EDitor. The option `-e' tells sed that what follows is a sed script.
The script itself (s!/0!!g) breaks down like this:
* `s' means we are doing a substitution.
* The exclamation point (!) is just a delimeter; any character can be used. Typically a forward slash (/) is used, but since directory paths have forward slashes in them, we go with something else to make it easier to read. Otherwise we'd have to escape the forward slashes and it would look like this: s/\/0//g
* `/0' is what we are looking to match.
* Another exclamation point (!) indicates we are done with our "search" pattern and next begins our "replace" pattern.
* Since we are just looking to just get rid of all occurences of `/0', our replacement pattern is literally nothing.
* Another exclamation point (!) to indicate the end of the replacement pattern.
* Finally, `g' (for global) tells sed to keep applying the substitution until it no longer matches. This means a path like `/dir/0/0/0' gets each `/0' stripped away one at a time until there are none left.
The last command sorts our listing so that the `-u' (for unique) option can be applied to it, which removes duplicate lines.
If two or more object variables have the same root directory name, how can I select only the object variable with the most /0's between two variables (sdcard/0 and sdcard/0/0; I would want only the latter) then store those object variable in a new variable including all other object variables? The desired result here would be VAR1=$(data/0, sdcard/0/0)
Click to expand...
Click to collapse
The easiest way to do this is just check for the presence of a `0' dir at each level. If one exists then there's another level of nesting below us and we ignore the current level:
Code:
VAR1=""
for i in $(find */0); do
if [ ! -d $i/0 ]; then
VAR1="$VAR1 $i"
fi
done

Wow! Thanks!! But now I'm suffering from cognitive dissonance (dis-sed-ance). I knew it would require either find, grep, awk, or sed but using those commands without having any grip on regex is nearly impossible. Can you point me to a good resource for learning Android regex or is any GNU/Linux resource valid? Is this a good one?
http://www.linux.org/article/view/introduction-to-regular-expressions-within-a-shell

elfaure said:
I'm not on 4.2 nor do I plan on going there anytime soon so this won't help me out less what I learn by doing it
Click to expand...
Click to collapse
So does that mean you don't have any "0" directories in your /sdcard right now, and you are doing all this just for fun?
At least, after reading your additional material, I am now beginning to understand the problem.
However I don't understand how you get any useful output from "find */0" - I get either "No such file or directory", or a listing of thousands of files, depending on where I start this.
elfaure said:
Can you point me to a good resource for learning Android regex or is any GNU/Linux resource valid?
Click to expand...
Click to collapse
The nice thing with regular expressions is that the basics are the same everywhere.
But I don't know if you even need such powerful tools... Correct me if I am wrong, but your script should convert this structure:
/data/media/0
/data/media/0/stuff1
/data/media/0/0
/data/media/0/0/stuff2
/data/media/0/0/0
/data/media/0/0/0/stuff3
to
/data/media/0
/data/media/0/stuff1
/data/media/0/stuff2
/data/media/0/stuff3
?
So you'd basically need to run this inside /data/media/0:
1. If there is no subdirectory named "0", there is nothing to do, and we can exit
2. If there is a subdirectory named "0", move everything from inside one level out
2a. how do you want to handle conflicts, if the file or directory you want to move outside already exists in the parent directory?
3. go to step 1

[Reply to _that's last post. Getting lazy, I'm at lunch...]
Here is my output from
Code:
find */0
[email protected]:/ $ su
[email protected]:/ # find */0
data/0
sdcard/0
sdcard/0/0
[email protected]:/ # exit
[email protected]:/ $ find */0
data/0
find: data/0: Permission denied
sdcard/0
sdcard/0/0
1|[email protected]:/ $
I'm not sure about the structure I am looking for until I get some output from this code run on a 4.2 device with nested /0's so I can see all of them and their variations. I also want to see what happens to subsequent user directories nested starting with /data/media/10 (second user) and /data/media/11 (third user) and how they nest in comparison.
The only /0 directories that I have on my device were created by me to test this script. I am still on 4.1 and this is a 4.2 only issue so as you state its "just for fun" or rather "for the fun of learning" but at least its not a "hello world" script that does nothing useful. I'm hoping to automate someone's cleanup after installing 4.2 with this, maybe my own at some point, if required.
Here's some more info about the issue:
http://teamw.in/DataMedia
ps-I just discovered the equivalent to the stock browser "about:debug" for Chrome is "about:flags" which brings up an extensive list of experimental settings. I enabled all that were GPU and speed related and its a marked improvement (it feels 25-75% faster). With a 100MB Chrome cache I can have about 20 tabs open at the same time now and its still fast and snappy.

Not sure if this will work or not, but if you want to try to retain single user mode in 4.2 you can try this:
http://androidforums.com/verizon-ga...y-bean-roms-edited-3-24-13-a.html#post5806236
If anyone has a copy of the .layout_version file, please post it or a link to it so I can check it out.
Thks

becomingx said:
Wow! This is very messy, no wonder you're trying to automate the cleanup.
Love the analogy here :laugh:
Looks to be `/data/.layout_version' [ref].
If we're going full steam ahead with the requirement of Busybox being installed, then this is pretty easy:
Code:
VAR="$(find */0 | sed -e 's!/0!!g' | sort -u)"
Sed is our Stream EDitor. The option `-e' tells sed that what follows is a sed script.
The script itself (s!/0!!g) breaks down like this:
* `s' means we are doing a substitution.
* The exclamation point (!) is just a delimeter; any character can be used. Typically a forward slash (/) is used, but since directory paths have forward slashes in them, we go with something else to make it easier to read. Otherwise we'd have to escape the forward slashes and it would look like this: s/\/0//g
* `/0' is what we are looking to match.
* Another exclamation point (!) indicates we are done with our "search" pattern and next begins our "replace" pattern.
* Since we are just looking to just get rid of all occurences of `/0', our replacement pattern is literally nothing.
* Another exclamation point (!) to indicate the end of the replacement pattern.
* Finally, `g' (for global) tells sed to keep applying the substitution until it no longer matches. This means a path like `/dir/0/0/0' gets each `/0' stripped away one at a time until there are none left.
The last command sorts our listing so that the `-u' (for unique) option can be applied to it, which removes duplicate lines.
The easiest way to do this is just check for the presence of a `0' dir at each level. If one exists then there's another level of nesting below us and we ignore the current level:
Code:
VAR1=""
for i in $(find */0); do
if [ ! -d $i/0 ]; then
VAR1="$VAR1 $i"
fi
done
Click to expand...
Click to collapse
Thanks again for your help and explanations. I find it amazing what a few (not so) simple lines of code can do with a complex task. Learning regex will take me until next year so for now I will go blindly forward with my plan. The next step would be to use BOTH of these variables in some call statement like [ ] to automate the process of backing up and moving data around. And combining it all into a script here it is below. Is the syntax in the main block I added correct?
Code:
#!/system/bin/sh
VAR="$(find */0 | sed -e 's!/0!!g' | sort -u)"
VAR1=""
for i in $(find */0); do
if [ ! -d $i/0 ]; then
VAR1="$VAR1 $i"
fi
done
for OBJ in $VAR and OBJ1 in $VAR1; do
#Backup $OBJ to SD
mkdir /Removable/MicroSD/$OBJ_BAK
cp -r /$OBJ /Removable/MicroSD/$OBJ_BAK
#Over write $OBJ with $OBJ1
cp -rf /$OBJ1 /$OBJ
#Merge backup and $OBJ
cp /Removable/MicroSD/$OBJ_BAK/*.* /$OBJ
#Remove $OBJ backup
rm -rf /Removable/MicroSD/$OBJ_BAK
done

elfaure said:
Wow! Thanks!! But now I'm suffering from cognitive dissonance (dis-sed-ance). I knew it would require either find, grep, awk, or sed but using those commands without having any grip on regex is nearly impossible. Can you point me to a good resource for learning Android regex or is any GNU/Linux resource valid? Is this a good one?
http://www.linux.org/article/view/introduction-to-regular-expressions-within-a-shell
Click to expand...
Click to collapse
lolz. Any resource on regexes should be applicable, just know that there are some small differences between the three main dialects (basic, extended, and perl). If you've got money to spend, Mastering Regular Expressions is considered *the* book to own on the subject. Also good is this pocket reference for sed and awk, which is a bit cheaper. I've used my copy so much the spine has completely gone out and it's just a collection of loose pages inside the cover now!
---------- Post added at 11:29 AM ---------- Previous post was at 11:00 AM ----------
_that said:
So does that mean you don't have any "0" directories in your /sdcard right now, and you are doing all this just for fun?
At least, after reading your additional material, I am now beginning to understand the problem.
However I don't understand how you get any useful output from "find */0" - I get either "No such file or directory", or a listing of thousands of files, depending on where I start this.
Click to expand...
Click to collapse
This is what I was hinting at back in post #3.
Using `find' like this, the `*/0' will get expanded by the shell if there happens to exist at least one `SOMEDIR/0', which will result in `find' printing out every file/directory/etc. that exists under any and every `SOMEDIR/0'. Otherwise, if none exist, it will get passed to the `find' command as literally `*/0', which is where the "No such file or directory" comes from.
The proper way to do this is using a `find' option like `-name', to limit scope of what gets returned. Again, though, one still has to be careful because we could still get search matches that we don't want. Something like this might be a good starting point:
Code:
find -name '0' -type d
@elfaure: Try populating your self-created `0' directories with some files and directories, the output of `find */0' should become a bit surprising.
But I don't know if you even need such powerful tools... Correct me if I am wrong, but your script should convert this structure:
/data/media/0
/data/media/0/stuff1
/data/media/0/0
/data/media/0/0/stuff2
/data/media/0/0/0
/data/media/0/0/0/stuff3
to
/data/media/0
/data/media/0/stuff1
/data/media/0/stuff2
/data/media/0/stuff3
?
So you'd basically need to run this inside /data/media/0:
1. If there is no subdirectory named "0", there is nothing to do, and we can exit
2. If there is a subdirectory named "0", move everything from inside one level out
2a. how do you want to handle conflicts, if the file or directory you want to move outside already exists in the parent directory?
3. go to step 1
Click to expand...
Click to collapse
Seems simple enough, but need a little time to think about it...
---------- Post added at 11:31 AM ---------- Previous post was at 11:29 AM ----------
elfaure said:
If anyone has a copy of the .layout_version file, please post it or a link to it so I can check it out.
Thks
Click to expand...
Click to collapse
Mine just contains the number "2", nothing else, not even a newline.
---------- Post added at 11:43 AM ---------- Previous post was at 11:31 AM ----------
elfaure said:
Thanks again for your help and explanations. I find it amazing what a few (not so) simple lines of code can do with a complex task. Learning regex will take me until next year so for now I will go blindly forward with my plan. The next step would be to use BOTH of these variables in some call statement like [ ] to automate the process of backing up and moving data around. And combining it all into a script here it is below. Is the syntax in the main block I added correct?
Code:
#!/system/bin/sh
VAR="$(find */0 | sed -e 's!/0!!g' | sort -u)"
VAR1=""
for i in $(find */0); do
if [ ! -d $i/0 ]; then
VAR1="$VAR1 $i"
fi
done
for OBJ in $VAR and OBJ1 in $VAR1; do
#Backup $OBJ to SD
mkdir /Removable/MicroSD/$OBJ_BAK
cp -r /$OBJ /Removable/MicroSD/$OBJ_BAK
#Over write $OBJ with $OBJ1
cp -rf /$OBJ1 /$OBJ
#Merge backup and $OBJ
cp /Removable/MicroSD/$OBJ_BAK/*.* /$OBJ
#Remove $OBJ backup
rm -rf /Removable/MicroSD/$OBJ_BAK
done
Click to expand...
Click to collapse
Negatory, good buddy. Everthing after the `in' keyword till the semicolon gets treated as part of the (one and only) list. This would be a good place for two parallel arrays, but I suspect that might be overkill. Need a little time to think...
P.S. Never go blindly forward when working as user `root'. Much too dangerous.