JB 4.2 Nested /0 Fix Script - Asus Transformer TF700

Hey-
I was wondering if I could get some help and feedback on a script I conceived to help 4.2 multi-user tree structure to work with pre 4.2 backups (Device, ROMs, Recoveries, Nandroids, Titaniums). Can one of you guru's (_that, becomingx, etc.) chime in and let me know if this seems logical and a good solution, what syntax errors I have, and partially test the theory and code as I am still on 4.1 and locked.
Assuming you have no stock /*/0 directories this script finds all instances of /*/0 (created from the 4.2 multi-user tree structure). /* is "the parent directory" and /*/0 is the "nested directory". First it backs up the parent directory then copies the nested directory (overwrites) into their parent directory. It also combines the backup and the parent directory in the parent directory at the end. This script assumes the user to be single-nested and after running it allows pre-4.2 backups to find their data and targets for restoration. Maybe its OK to combine before the restoration, or after, or not at all? Please give me your feedback on this and don't beat me up too hard as this is my first script with a variable and an array.
Code:
#!/system/bin/sh
nests=( grep ! /*/0 )
for NEST in "${nests[@]}"
do
mkdir /Removable/MicroSD/$NEST_bak
cp -r /$NEST /Removable/MicroSD/$NEST_bak
cp -rf /$NEST/0 /$NEST
#[Restore Pre-4.2 backups]
cp /Removable/MicroSD/$NEST_bak/*.* /$NEST
#[Backup in 4.2]
rm -rf /Removable/MicroSD/$NEST_bak
done
[Edit] Yup, this script needs some serious help. The first line doesn't even work. But when you type it as a command from the prompt it does what I'm looking to capture. How can I put this output in an array?
Code:
grep ! /*/0
Why after running the code above does my prompt which was "[email protected]: #" now show "1|[email protected]: #"? When I was trying to debug the script I even got a 2 and 3|[email protected]: #" Are these some kind of grep shells?

I've now read your post twice, and I still have no idea what you want to do - neither from the script itself nor from the textual description. The confusion starts with the fact that I have no "0" directories on my MicroSD card.
The first step in software development is always to clearly define the problem that should be solved by the software. As long as I don't understand what you want to do, I can't help you with the script - except maybe a hint: "grep" searches for text inside files, "find" searches for files or directories.
Why after running the code above does my prompt which was "[email protected]: #" now show "1|[email protected]: #"? When I was trying to debug the script I even got a 2 and 3|[email protected]: #" Are these some kind of grep shells?
Click to expand...
Click to collapse
The number is the last exit code from the previous command. It indicates that something went wrong.

_that said:
I've now read your post twice, and I still have no idea what you want to do - neither from the script itself nor from the textual description. The confusion starts with the fact that I have no "0" directories on my MicroSD card.
Click to expand...
Click to collapse
No, but you do have some in `/storage/emulated' and `/mnt/shell/emulated' , which I *think* might be what elfaure is referring to (confirm?).
The first step in software development is always to clearly define the problem that should be solved by the software. As long as I don't understand what you want to do, I can't help you with the script - except maybe a hint: "grep" searches for text inside files, "find" searches for files or directories.
Click to expand...
Click to collapse
True words indeed. Although, just telling someone what you want to do and having them show you how to do it, or having them do it for you, is never as much fun as trying to do it for yourself first.
elfaure said:
Hey-
I was wondering if I could get some help and feedback on a script I conceived to help 4.2 multi-user tree structure to work with pre 4.2 backups (Device, ROMs, Recoveries, Nandroids, Titaniums). Can one of you guru's (_that, becomingx, etc.) chime in and let me know if this seems logical and a good solution, what syntax errors I have, and partially test the theory and code as I am still on 4.1 and locked.
Assuming you have no stock /*/0 directories this script finds all instances of /*/0 (created from the 4.2 multi-user tree structure). /* is "the parent directory" and /*/0 is the "nested directory". First it backs up the parent directory then copies the nested directory (overwrites) into their parent directory. It also combines the backup and the parent directory in the parent directory at the end. This script assumes the user to be single-nested and after running it allows pre-4.2 backups to find their data and targets for restoration. Maybe its OK to combine before the restoration, or after, or not at all? Please give me your feedback on this and don't beat me up too hard as this is my first script with a variable and an array.
Code:
#!/system/bin/sh
nests=( grep ! /*/0 )
for NEST in "${nests[@]}"
do
mkdir /Removable/MicroSD/$NEST_bak
cp -r /$NEST /Removable/MicroSD/$NEST_bak
cp -rf /$NEST/0 /$NEST
#[Restore Pre-4.2 backups]
cp /Removable/MicroSD/$NEST_bak/*.* /$NEST
#[Backup in 4.2]
rm -rf /Removable/MicroSD/$NEST_bak
done
Click to expand...
Click to collapse
This is indeed a bit confusing. First I need to know what you're expecting `(grep ! /*/0)' to evaluate to before I can comment on the rest of the code.
[Edit] Yup, this script needs some serious help. The first line doesn't even work. But when you type it as a command from the prompt it does what I'm looking to capture.
Click to expand...
Click to collapse
To capture the output of a command into a variable, you need a dollar sign in front of the opening parenthesis, i.e.:
Code:
nests=$(grep ! /*/0)
But, as _that mentioned, the `grep' call probably isn't the right one to use here. `find' can surely do what you want, but requires busybox to be installed and usually needs to have a bunch of options supplied to it to make sure it's giving you exactly what you want and expect. A simple `ls' command might be all that's needed.
How can I put this output in an array?
Click to expand...
Click to collapse
I, and most other people based on the code I've seen, don't use arrays much in shell scripts. Usually a value separated list is used instead, where the separator value is a space, tab, or newline. Take the example:
Code:
ROOT_LISTING=$(ls /)
for OBJ in $ROOT_LISTING; do
...
done
When the variable $ROOT_LISTING gets expanded at execution time it winds up looking like:
Code:
for OBJ in Removable acct cache [snipped for brevity]; do
and so the `for' loop will iterate through each value after the `in' keyword that is seperated from the next one by a space, tab, or newline. (If we have files, directories, etc. with spaces (possible), tabs (unlikely), or newlines (also unlikely) in the name, we have to be trickier, but I won't get into that here. )
Hopefully this helps. If there's anything else technical that I can yammer on about for the betterment of your understanding, just holler. :good:

Hi Guys-
Thanks for the reply. What service! I simply call for the two guys I need most and like rubbing a magic lamp my genies appear.
When _that reads it twice and still has no clue what you are trying to do you know you need some serious schooling. That's why I'm here - to listen twice as often as I speak and learn something useful.
So here's what I am trying to do. JB 4.2 implemented a radical depart from previous Android versions to the directory tree structure for multi-users as you know, similar to the way Windows uses user profiles to separate users desktops, accessible installed apps, etc. This new structure created /0 subdirectories for certain directories like /sdcard and /sdcard/0 (/data/media and /data/media/0, /clockworkmod and /clockworkmod/0) and others. Playing with grep I discovered I could find all instances of any directory containing a /0 subdirectory (pair). "Find" seems a better approach than "grep" and works exactly the same with no fancy options required.
Code:
find */0
For these directory pairs I'll call them from the above output, the data that was in /directory in 4.1 is now located and pointed to in /directory/0 in 4.2. I figured if I could find all these pairs, and copy /directory/0 into /directory (after backing up /directory to SD) then 4.1 backups looking for data in /directory could now find their data and targets. Then I could copy the SD backup of /directory into /directory (now overwritten by /directory/0) to merge everything into a single folder. But after thinking about this again, this is incorrect. What I meant to do at this step is copy all /0 subdirectories of a given /directory/0 into that folder [ /directory/0/0 into /directory/0 and directory/0/0/0 into /directory/0 and /directory/0/0/0/0... into /directory/0 ] to combine all these sub-nested directories into a single /0 directory. Deep subnesting causes by crack flashing custom ROMs over and over on JB 4.2. This is the concept and I'm trying to implement a script to do it.
So I will attempt to replace grep with find and the array with a variable value space separated list and see what I come up with. Becomingx, you are right about wanting to use my brain a bit to figure it out on my own, even if it sends me down a one-way road to a dead end. If I learned something in the process then its all good and I can always ask for directions at the locked gate if I arrive there, but even with 4 flat tires and a broken axle I'm still moving (forward hopefully) as long as there's still road to travel on.
Thks

So you want to write a script to downgrade from 4.2 to 4.1?

Test new avitar
How do you delete a post? Under Edit/Delete I only find Edit??

_that said:
So you want to write a script to downgrade from 4.2 to 4.1?
Click to expand...
Click to collapse
No, I want to write a script that lets you restore a pre 4.2 backup (Nandroid, Titanium, Recovery, ROM) on 4.2 without having to move/copy files.

elfaure said:
No, I want to write a script that lets you restore a pre 4.2 backup (Nandroid, Titanium, Recovery, ROM) on 4.2 without having to move/copy files.
Click to expand...
Click to collapse
Are you sure that this is not already handled by TWRP, or by TB?
Recovery and ROM themselves are not affected by the /0 move in /data anyway.

Thanks becomingx, getting somewhere now but I still have problems.
When I type this at the terminal I get what i am looking for as output
Code:
su
cd /
find */0
Output: (These are test directories I created)
data/0
sdcard/0
sdcard/0/0
But when I run this script, the above output is repeated 3 times? Why is that?? One for each file system /(rootfs), /data, and /sdcard? How can I rewrite the script to produce the same output as from the terminal?
Code:
#!/system/bin/sh
cd /
NEST=$(find */0)
for OBJ in $NEST; do
echo "$NEST"
done

_that said:
Are you sure that this is not already handled by TWRP, or by TB?
Recovery and ROM themselves are not affected by the /0 move in /data anyway.
Click to expand...
Click to collapse
Long story short, yes if you had all the latest and greatest installed before you upgraded to 4.2 then its a transparant issue. But if you're already "nested" because you used an older custom recovery and factory reset a few times to crack flash some some 4.2 ROMs or need to recover to a previous Android version then there is a use for this script. If I can learn how to script variables and help some nested folks out at the same time then _that's what I call a win-win. I'm not on 4.2 nor do I plan on going there anytime soon so this won't help me out less what I learn by doing it, although it is interesting and challenging to try to implement an automated solution for. Plus I'm like you guys, I just really enjoy rooted Android and helping people with it when I can (within my very limited but expanding skill set).
By the way, what is the name and /data path to that special file?
****************************************************************************************************
http://androidforums.com/verizon-ga...t/649940-4-2-sdcard-sdcard-0-observation.html
http://androidforums.com/verizon-ga...y-bean-roms-edited-3-24-13-a.html#post5796630
Quoted from Androidspin.com -
"With Android 4.2, Google introduced multiple users as a new feature. In order to accommodate multiple users, Google is now giving each user a their own folder for storage. If you upgraded to 4.2 from 4.1, then the 4.2 ROM will look for a certain file in /data to determine whether it needs to migrate all of your files to the new multi-user data structure. By default, 4.2 migrates all of /data/media to /data/media/0.
A problem arose though with custom recoveries. A custom recovery retains the /data/media folder during a factory reset. When you factory reset and then boot a 4.2 ROM again, the 4.2 ROM will migrate everything in /data/media again. It will migrate your files every time you factory reset. This multiple migration is what resulted in some people having their files moved to /sdcard/0 or even /sdcard/0/0 etc.
In TWRP 2.3.2.0 we have corrected this problem by ensuring that we do not delete the special file during a factory reset. However, if ended up having your files upgraded you will need to move or merge them back into /sdcard. Also, if you have moved your TWRP folder from /data/media/0 to /data/media so that you could restore backups while using prior TWRP versions, you may now need to move the TWRP folder back into /data/media/0.
As a special note, if you restore a backup to a prior version of Android, you may have to move your files out of /data/media/0 and into /data/media to be able to see them again."

elfaure said:
TBut when I run this script, the above output is repeated 3 times? Why is that?? One for each file system /(rootfs), /data, and /sdcard? How can I write the script to produce the same output as from the terminal?
Code:
#!/system/bin/sh
cd /
NEST=$(find */0)
for OBJ in $NEST; do
echo "$NEST"
done
Click to expand...
Click to collapse
Read your script - for each item in $NEST you output all of $NEST instead of the single item $OBJ.

_that said:
Read your script - for each item in $NEST you output all of $NEST instead of the single item $OBJ.
Click to expand...
Click to collapse
Ah ha. Lost sight of the fact that a multi-object variable really behaves like an array and you can call its objects as independent variables (syntax wise; I thought that was what I was doing with echo $NEST). Variables in a variable. That's why I started with an array. Thanks.
Code:
#!/system/bin/sh
cd /
NEST=$(find */0)
for OBJ in $NEST; do
echo "$OBJ"
done
Output:
data/0
sdcard/0
sdcard/0/0
New Questions:
How can I strip off all the "/0's" from each object variable and store it in a new mutil-variable until there are no more "/0's to strip off and consolidate duplicates? The desired result here would be VAR=$(data, sdcard).
If two or more object variables have the same root directory name, how can I select only the object variable with the most /0's between two variables (sdcard/0 and sdcard/0/0; I would want only the latter) then store those object variable in a new variable including all other object variables? The desired result here would be VAR1=$(data/0, sdcard/0/0)

Code:
#!/system/bin/sh
nests=( grep ! /*/0 )
for NEST in "${nests[@]}"
do
mkdir /Removable/MicroSD/$NEST_bak
cp -r /$NEST /Removable/MicroSD/$NEST_bak
cp -rf /$NEST/0 /$NEST
#[Restore Pre-4.2 backups]
cp /Removable/MicroSD/$NEST_bak/*.* /$NEST
#[Backup in 4.2]
rm -rf /Removable/MicroSD/$NEST_bak
done
becomingx said:
To capture the output of a command into a variable, you need a dollar sign in front of the opening parenthesis, i.e.:
Code:
nests=$(grep ! /*/0)
[snipped]
I, and most other people based on the code I've seen, don't use arrays much in shell scripts. Usually a value separated list is used instead...[snipped]
Click to expand...
Click to collapse
"nests" in my original code here refers to an array not a variable. Assuming that, is the syntax correct (for reference)? I like the multi-item(object) variable value separated list you showed me and _that corrected my use of and will use that instead for this script.

elfaure said:
For these directory pairs I'll call them from the above output, the data that was in /directory in 4.1 is now located and pointed to in /directory/0 in 4.2. I figured if I could find all these pairs, and copy /directory/0 into /directory (after backing up /directory to SD) then 4.1 backups looking for data in /directory could now find their data and targets. Then I could copy the SD backup of /directory into /directory (now overwritten by /directory/0) to merge everything into a single folder. But after thinking about this again, this is incorrect. What I meant to do at this step is copy all /0 subdirectories of a given /directory/0 into that folder [ /directory/0/0 into /directory/0 and directory/0/0/0 into /directory/0 and /directory/0/0/0/0... into /directory/0 ] to combine all these sub-nested directories into a single /0 directory. Deep subnesting causes by crack flashing custom ROMs over and over on JB 4.2. This is the concept and I'm trying to implement a script to do it.
Click to expand...
Click to collapse
Wow! This is very messy, no wonder you're trying to automate the cleanup.
So I will attempt to replace grep with find and the array with a variable value space separated list and see what I come up with. Becomingx, you are right about wanting to use my brain a bit to figure it out on my own, even if it sends me down a one-way road to a dead end. If I learned something in the process then its all good and I can always ask for directions at the locked gate if I arrive there, but even with 4 flat tires and a broken axle I'm still moving (forward hopefully) as long as there's still road to travel on.
Click to expand...
Click to collapse
Love the analogy here :laugh:
elfaure said:
By the way, what is the name and /data path to that special file?
Click to expand...
Click to collapse
Looks to be `/data/.layout_version' [ref].
elfaure said:
Code:
#!/system/bin/sh
cd /
NEST=$(find */0)
for OBJ in $NEST; do
echo "$OBJ"
done
Output:
data/0
sdcard/0
sdcard/0/0
New Questions:
How can I strip off all the "/0's" from each object variable and store it in a new mutil-variable until there are no more "/0's to strip off and consolidate duplicates? The desired result here would be VAR=$(data, sdcard).
Click to expand...
Click to collapse
If we're going full steam ahead with the requirement of Busybox being installed, then this is pretty easy:
Code:
VAR="$(find */0 | sed -e 's!/0!!g' | sort -u)"
Sed is our Stream EDitor. The option `-e' tells sed that what follows is a sed script.
The script itself (s!/0!!g) breaks down like this:
* `s' means we are doing a substitution.
* The exclamation point (!) is just a delimeter; any character can be used. Typically a forward slash (/) is used, but since directory paths have forward slashes in them, we go with something else to make it easier to read. Otherwise we'd have to escape the forward slashes and it would look like this: s/\/0//g
* `/0' is what we are looking to match.
* Another exclamation point (!) indicates we are done with our "search" pattern and next begins our "replace" pattern.
* Since we are just looking to just get rid of all occurences of `/0', our replacement pattern is literally nothing.
* Another exclamation point (!) to indicate the end of the replacement pattern.
* Finally, `g' (for global) tells sed to keep applying the substitution until it no longer matches. This means a path like `/dir/0/0/0' gets each `/0' stripped away one at a time until there are none left.
The last command sorts our listing so that the `-u' (for unique) option can be applied to it, which removes duplicate lines.
If two or more object variables have the same root directory name, how can I select only the object variable with the most /0's between two variables (sdcard/0 and sdcard/0/0; I would want only the latter) then store those object variable in a new variable including all other object variables? The desired result here would be VAR1=$(data/0, sdcard/0/0)
Click to expand...
Click to collapse
The easiest way to do this is just check for the presence of a `0' dir at each level. If one exists then there's another level of nesting below us and we ignore the current level:
Code:
VAR1=""
for i in $(find */0); do
if [ ! -d $i/0 ]; then
VAR1="$VAR1 $i"
fi
done

Wow! Thanks!! But now I'm suffering from cognitive dissonance (dis-sed-ance). I knew it would require either find, grep, awk, or sed but using those commands without having any grip on regex is nearly impossible. Can you point me to a good resource for learning Android regex or is any GNU/Linux resource valid? Is this a good one?
http://www.linux.org/article/view/introduction-to-regular-expressions-within-a-shell

elfaure said:
I'm not on 4.2 nor do I plan on going there anytime soon so this won't help me out less what I learn by doing it
Click to expand...
Click to collapse
So does that mean you don't have any "0" directories in your /sdcard right now, and you are doing all this just for fun?
At least, after reading your additional material, I am now beginning to understand the problem.
However I don't understand how you get any useful output from "find */0" - I get either "No such file or directory", or a listing of thousands of files, depending on where I start this.
elfaure said:
Can you point me to a good resource for learning Android regex or is any GNU/Linux resource valid?
Click to expand...
Click to collapse
The nice thing with regular expressions is that the basics are the same everywhere.
But I don't know if you even need such powerful tools... Correct me if I am wrong, but your script should convert this structure:
/data/media/0
/data/media/0/stuff1
/data/media/0/0
/data/media/0/0/stuff2
/data/media/0/0/0
/data/media/0/0/0/stuff3
to
/data/media/0
/data/media/0/stuff1
/data/media/0/stuff2
/data/media/0/stuff3
?
So you'd basically need to run this inside /data/media/0:
1. If there is no subdirectory named "0", there is nothing to do, and we can exit
2. If there is a subdirectory named "0", move everything from inside one level out
2a. how do you want to handle conflicts, if the file or directory you want to move outside already exists in the parent directory?
3. go to step 1

[Reply to _that's last post. Getting lazy, I'm at lunch...]
Here is my output from
Code:
find */0
[email protected]:/ $ su
[email protected]:/ # find */0
data/0
sdcard/0
sdcard/0/0
[email protected]:/ # exit
[email protected]:/ $ find */0
data/0
find: data/0: Permission denied
sdcard/0
sdcard/0/0
1|[email protected]:/ $
I'm not sure about the structure I am looking for until I get some output from this code run on a 4.2 device with nested /0's so I can see all of them and their variations. I also want to see what happens to subsequent user directories nested starting with /data/media/10 (second user) and /data/media/11 (third user) and how they nest in comparison.
The only /0 directories that I have on my device were created by me to test this script. I am still on 4.1 and this is a 4.2 only issue so as you state its "just for fun" or rather "for the fun of learning" but at least its not a "hello world" script that does nothing useful. I'm hoping to automate someone's cleanup after installing 4.2 with this, maybe my own at some point, if required.
Here's some more info about the issue:
http://teamw.in/DataMedia
ps-I just discovered the equivalent to the stock browser "about:debug" for Chrome is "about:flags" which brings up an extensive list of experimental settings. I enabled all that were GPU and speed related and its a marked improvement (it feels 25-75% faster). With a 100MB Chrome cache I can have about 20 tabs open at the same time now and its still fast and snappy.

Not sure if this will work or not, but if you want to try to retain single user mode in 4.2 you can try this:
http://androidforums.com/verizon-ga...y-bean-roms-edited-3-24-13-a.html#post5806236
If anyone has a copy of the .layout_version file, please post it or a link to it so I can check it out.
Thks

becomingx said:
Wow! This is very messy, no wonder you're trying to automate the cleanup.
Love the analogy here :laugh:
Looks to be `/data/.layout_version' [ref].
If we're going full steam ahead with the requirement of Busybox being installed, then this is pretty easy:
Code:
VAR="$(find */0 | sed -e 's!/0!!g' | sort -u)"
Sed is our Stream EDitor. The option `-e' tells sed that what follows is a sed script.
The script itself (s!/0!!g) breaks down like this:
* `s' means we are doing a substitution.
* The exclamation point (!) is just a delimeter; any character can be used. Typically a forward slash (/) is used, but since directory paths have forward slashes in them, we go with something else to make it easier to read. Otherwise we'd have to escape the forward slashes and it would look like this: s/\/0//g
* `/0' is what we are looking to match.
* Another exclamation point (!) indicates we are done with our "search" pattern and next begins our "replace" pattern.
* Since we are just looking to just get rid of all occurences of `/0', our replacement pattern is literally nothing.
* Another exclamation point (!) to indicate the end of the replacement pattern.
* Finally, `g' (for global) tells sed to keep applying the substitution until it no longer matches. This means a path like `/dir/0/0/0' gets each `/0' stripped away one at a time until there are none left.
The last command sorts our listing so that the `-u' (for unique) option can be applied to it, which removes duplicate lines.
The easiest way to do this is just check for the presence of a `0' dir at each level. If one exists then there's another level of nesting below us and we ignore the current level:
Code:
VAR1=""
for i in $(find */0); do
if [ ! -d $i/0 ]; then
VAR1="$VAR1 $i"
fi
done
Click to expand...
Click to collapse
Thanks again for your help and explanations. I find it amazing what a few (not so) simple lines of code can do with a complex task. Learning regex will take me until next year so for now I will go blindly forward with my plan. The next step would be to use BOTH of these variables in some call statement like [ ] to automate the process of backing up and moving data around. And combining it all into a script here it is below. Is the syntax in the main block I added correct?
Code:
#!/system/bin/sh
VAR="$(find */0 | sed -e 's!/0!!g' | sort -u)"
VAR1=""
for i in $(find */0); do
if [ ! -d $i/0 ]; then
VAR1="$VAR1 $i"
fi
done
for OBJ in $VAR and OBJ1 in $VAR1; do
#Backup $OBJ to SD
mkdir /Removable/MicroSD/$OBJ_BAK
cp -r /$OBJ /Removable/MicroSD/$OBJ_BAK
#Over write $OBJ with $OBJ1
cp -rf /$OBJ1 /$OBJ
#Merge backup and $OBJ
cp /Removable/MicroSD/$OBJ_BAK/*.* /$OBJ
#Remove $OBJ backup
rm -rf /Removable/MicroSD/$OBJ_BAK
done

elfaure said:
Wow! Thanks!! But now I'm suffering from cognitive dissonance (dis-sed-ance). I knew it would require either find, grep, awk, or sed but using those commands without having any grip on regex is nearly impossible. Can you point me to a good resource for learning Android regex or is any GNU/Linux resource valid? Is this a good one?
http://www.linux.org/article/view/introduction-to-regular-expressions-within-a-shell
Click to expand...
Click to collapse
lolz. Any resource on regexes should be applicable, just know that there are some small differences between the three main dialects (basic, extended, and perl). If you've got money to spend, Mastering Regular Expressions is considered *the* book to own on the subject. Also good is this pocket reference for sed and awk, which is a bit cheaper. I've used my copy so much the spine has completely gone out and it's just a collection of loose pages inside the cover now!
---------- Post added at 11:29 AM ---------- Previous post was at 11:00 AM ----------
_that said:
So does that mean you don't have any "0" directories in your /sdcard right now, and you are doing all this just for fun?
At least, after reading your additional material, I am now beginning to understand the problem.
However I don't understand how you get any useful output from "find */0" - I get either "No such file or directory", or a listing of thousands of files, depending on where I start this.
Click to expand...
Click to collapse
This is what I was hinting at back in post #3.
Using `find' like this, the `*/0' will get expanded by the shell if there happens to exist at least one `SOMEDIR/0', which will result in `find' printing out every file/directory/etc. that exists under any and every `SOMEDIR/0'. Otherwise, if none exist, it will get passed to the `find' command as literally `*/0', which is where the "No such file or directory" comes from.
The proper way to do this is using a `find' option like `-name', to limit scope of what gets returned. Again, though, one still has to be careful because we could still get search matches that we don't want. Something like this might be a good starting point:
Code:
find -name '0' -type d
@elfaure: Try populating your self-created `0' directories with some files and directories, the output of `find */0' should become a bit surprising.
But I don't know if you even need such powerful tools... Correct me if I am wrong, but your script should convert this structure:
/data/media/0
/data/media/0/stuff1
/data/media/0/0
/data/media/0/0/stuff2
/data/media/0/0/0
/data/media/0/0/0/stuff3
to
/data/media/0
/data/media/0/stuff1
/data/media/0/stuff2
/data/media/0/stuff3
?
So you'd basically need to run this inside /data/media/0:
1. If there is no subdirectory named "0", there is nothing to do, and we can exit
2. If there is a subdirectory named "0", move everything from inside one level out
2a. how do you want to handle conflicts, if the file or directory you want to move outside already exists in the parent directory?
3. go to step 1
Click to expand...
Click to collapse
Seems simple enough, but need a little time to think about it...
---------- Post added at 11:31 AM ---------- Previous post was at 11:29 AM ----------
elfaure said:
If anyone has a copy of the .layout_version file, please post it or a link to it so I can check it out.
Thks
Click to expand...
Click to collapse
Mine just contains the number "2", nothing else, not even a newline.
---------- Post added at 11:43 AM ---------- Previous post was at 11:31 AM ----------
elfaure said:
Thanks again for your help and explanations. I find it amazing what a few (not so) simple lines of code can do with a complex task. Learning regex will take me until next year so for now I will go blindly forward with my plan. The next step would be to use BOTH of these variables in some call statement like [ ] to automate the process of backing up and moving data around. And combining it all into a script here it is below. Is the syntax in the main block I added correct?
Code:
#!/system/bin/sh
VAR="$(find */0 | sed -e 's!/0!!g' | sort -u)"
VAR1=""
for i in $(find */0); do
if [ ! -d $i/0 ]; then
VAR1="$VAR1 $i"
fi
done
for OBJ in $VAR and OBJ1 in $VAR1; do
#Backup $OBJ to SD
mkdir /Removable/MicroSD/$OBJ_BAK
cp -r /$OBJ /Removable/MicroSD/$OBJ_BAK
#Over write $OBJ with $OBJ1
cp -rf /$OBJ1 /$OBJ
#Merge backup and $OBJ
cp /Removable/MicroSD/$OBJ_BAK/*.* /$OBJ
#Remove $OBJ backup
rm -rf /Removable/MicroSD/$OBJ_BAK
done
Click to expand...
Click to collapse
Negatory, good buddy. Everthing after the `in' keyword till the semicolon gets treated as part of the (one and only) list. This would be a good place for two parallel arrays, but I suspect that might be overkill. Need a little time to think...
P.S. Never go blindly forward when working as user `root'. Much too dangerous.

Related

Possible [How To] Moving all caches to SD?

Hi i have rooted successfully recently and was wondering how to move my caches to the SDcard, i have search around and stumble upon these 2 script that completely relocate anything with a cache to the SDcard.
Being a total noob with unix command, i can't be sure how perfect these commands are and was wondering if anyone can improve upon it, or notice if there is any problem in applying it?
Lastly, i was wondering if there is a method to activate the code easily then typing it all out?
I have done some searches but none or no one commented on how useful or practical these commands are?
Some of the searches brought me to multiple threads, and i was wondering if my suggestion would make it easier for people to look for it later on.
Again thanks in advance, i don't mind being the first to try any of these commands but wanted a error check first to see the feasibility of such codes.
Thanks to the coders who prepared the 2 code below.
daeglo said:
What it does (allegedly):
Searches /data/data for any file containing the string 'cache' which is not already a simlink.
Moves files to the supplied new cache directory.
Creates simlinks in place of the old files to the new files.
What you need to do:
Save the attached file (recache.zip)
Extract the two shell files (recache, relocate)
Examine the files to be certain I am not malicious/stupid (the latter is quite possible)
Push the files to a directory in the default path on your phone (/system/xbin works for me)
Make the files executable (chmod 555 /system/xbin/{recache,relocate})
Run the command: recache <new_cache_directory>
Synopsis
recache <new_cache_directory>
Moves all files matching /data/data/*/*cache* to new_cache_directory and creates a symlink from the old location of the file.
new_cache_directory - optional - defaults to '/sdcard/cache'
relocate file new_directory
Moves the named file to new directory, encoding the original path in the new file name and creates a symlink from the original file location.
file - required - the file to move
new_directory - required - the directory to move the file to.
edit: emphasized the need to examine the files -- I make no claim of fitness or usefulness of this script for any reason. Many eyes make deep cracks small. If anybody sees a problem with my script, post the problem and PM me about it.
Click to expand...
Click to collapse
Fnorder said:
This one should catch everything that uses 'webviewCache' - provided it's already created it's cache directory. It also has a configurable list for those that hog space in other ways.
Code:
#!/system/xbin/bb/ash
TARGET="/sdcard/cache"
CACHE="
/data/data/com.imeem.gynoid/cache
/data/data/com.google.android.street/cache
/data/data/com.android.vending/cache
/data/data/com.android.browser/app_thumbnails
/data/data/*/cache/webviewCache
"
sdtotal=0;mmctotal=0;successtotal=0;failtotal=0
if [ ! -d "$TARGET" ]; then
mkdir $TARGET
fi
for i in $CACHE; do
local odir=$(echo "$i"|sed s,^/data/data/,,)
local app=$(echo "$odir"|cut -d / -f1)
if [ -d "$i" ]; then
if [ ! -L "$i" ]; then
let mmctotal++
echo -e "$app:\n\t* Moving '$odir'..."
mkdir -p "$TARGET/$odir"
if [ ! -d "$TARGET/$odir" ]; then
echo -e "\t- Error creating directory: $TARGET/$odir"
let failtotal++
else
rm -r "$i"
ln -s "$TARGET/$odir" "$i"
if [ -L "$i" ]; then
echo -e "\t+ Success"
let successtotal++
else
echo -e "\t- Failure! Please investigate $i"
let failtotal++
fi
fi
else
let sdtotal++
fi
fi
done
echo "Matched cache dirs: $sdtotal symlinked, $mmctotal on mmc.";
if [ "$sdtotal" ]; then
echo "Successful moves: $successtotal, Failures: $failtotal"
fi
Click to expand...
Click to collapse
No one willing to look at the codes?
if you are asking on how to use that code there..
1. needs to be saved with an .sh extention eg: recache.sh and located at /usr/local/bin or you can just leave it in the root and esecute it with ./recache path/to/where/u/want/it eg: /sdcard/cache
2. need to give ti permisions.. chmod 555 recache.sh
if thats what u are looking to know?
Nitro212 said:
if you are asking on how to use that code there..
1. needs to be saved with an .sh extention eg: recache.sh and located at /usr/local/bin or you can just leave it in the root and esecute it with ./recache path/to/where/u/want/it eg: /sdcard/cache
2. need to give ti permisions.. chmod 555 recache.sh
if thats what u are looking to know?
Click to expand...
Click to collapse
No actually i was wondering if the codes are correct, and both says they do the same thing however the codes seems different. Was wondering which one should i run, and is the code correct.
Since i am a noob in Linux, i can't be sure what is happening.
I have been looking around and every other cache to sdcard method, are limited to certain programs, the code quoted by me earlier finds any cache and put it into Sdcard automatically, however other then wondering if the codes is correct, i was wondering if there is any possible drawback? And if possible, is there a method so that i can easily clear my caches too?
So my request to sum it up would be :
1)Is the code correct, and will it run?
2)Which of the 2 code is practical?
3)Anydrawback when i transfer all known cache? (Since settings to have cahce right?)
4)Last but not least, a code to clear the caches on my SDcard?
naTTan said:
No actually i was wondering if the codes are correct, and both says they do the same thing however the codes seems different. Was wondering which one should i run, and is the code correct.
Since i am a noob in Linux, i can't be sure what is happening.
I have been looking around and every other cache to sdcard method, are limited to certain programs, the code quoted by me earlier finds any cache and put it into Sdcard automatically, however other then wondering if the codes is correct, i was wondering if there is any possible drawback? And if possible, is there a method so that i can easily clear my caches too?
So my request to sum it up would be :
1)Is the code correct, and will it run?
2)Which of the 2 code is practical?
3)Anydrawback when i transfer all known cache? (Since settings to have cahce right?)
4)Last but not least, a code to clear the caches on my SDcard?
Click to expand...
Click to collapse
basically he created a repeating loop for cache .. the line you are interested in is the first part:
CACHE="
/data/data/com.imeem.gynoid/cache
/data/data/com.google.android.street/cache
/data/data/com.android.vending/cache
/data/data/com.android.browser/app_thumbnails
/data/data/*/cache/webviewCache
"
these are the only cache files at which he's looking .. now the /*/ would potentially mean all programs using /cache/webviewCache under their folder would also be moved .. there are other script and method out there that do the same thing
me personally .. i would change the target to /system/sd/cache and put them on the EXT2 partition .. it also appears he is deleting the existing cache and starting over .. i have a script that copies the existing cache before creating the link
bottom line .. should work ok
Thx for your help but one uses webview cahce while the other uses cahce alone in their command any significant difference?
There are other scripts? Mine linking me? If it not too much of a trouble?
Thanks again for all your help/.

[DEV][SCRIPT] First-Boot App Install

First-Boot Install System​
I have searched Far and wide for something like this since i first put out SleeperROM in November and always come up empty.
So with the latest release, i decided it was finally time to do it myself.
All you have to do is, using the following package as a template either on its own or in your ROM, make sure your batch folder with the .apk's to install are in /data/.firstboot/
Why
Some apps like QuickPic, ConnectBot, TinyFlashlight, Flash, Google Goggles and others that rely on linked libs don't like to simply be copied to their install dir because many won't install their own libs unless the PackageManager does it and/or they won't add themselves to the packages list (like QuickPic). The old solution is to include the lib either in the /data/data/appdir/lib with the rom install OR in /system/lib but this is quite wasteful especially in the case of big apps like Flash where including the libs separately from the app effectively doubles the space taken up on the rom by that single app since the apk still contains the lib files within.
So the solution is to install on first boot by including the apps in a batch folder for the script to process.
How it works
What it does is run from the init scripts, as one of the last scripts to run, it waits until the Android core system is up (checks to be sure by waiting for the SystemUI process is running then waits for an additional 10 seconds)
Then runs /data/.firstboot.sh, which is where you should put your first boot routines. Included in this script is the batch app installer which looks for the apps in /data/.firstboot/ and stores temporary app backups in /cache/tmp -- so be mindful that on MTD roms, the cache partition is smaller so may have to modify the $tmp variable to wherever you want to store temporaries
After installing, it sleeps for a proportional number of seconds (5 seconds per app installed) to ensure there is no race condition between installs or the final permissions_fix, zipalign and tmp cleanup. The /data/.firstboot.sh script removes itself when everything is complete.
The remaining components are kept in there for additional use by the /etc/init.d/Y02firstboot script in the future, so then all that needs to be put on the phone is a new /data/.firstboot/ dir and replacement /data/.firstboot.sh to run a batch of updates.
The Apps MUST be named by their package name.
I.e. Titanium Backup MUST be named com.keramidas.TitaniumBackup.apk
the file name must correspond with the name of the data dir in /data/data/ the script is lazy in that way, i didn't feel like keeping a file manifest of installs, instead just like to drop in apps to install.
The installer
consists of the following components:
- /system/etc/init.d/Y02firstboot
- /system/xbin/rsync
- /system/xbin/fix_permissions
- /system/xbin/batch_zipalign
- /system/xbin/sleeperlog (echos, logcats, and writes a log of activity to /sdcard/sleeperlog.txt)
- /data/.firstboot.sh
- /data/.firstboot/ (batch app directory)
- NOT INCLUDED, there must be a busybox bin somewhere in $PATH - this is not a problem for most roms
See the package link for a ready to use first-boot installer [CWM] flashable. Just drop your apks into /fs/data/.firstboot/ for a batch app updater
Download the template/usable package
DOWNLOAD MOD_CWM-FirstBoot-20120112.zip
File size: 341.07 KB / MD5 23d88c349b8d2fa3cd2f9958ae99a1f6
​
THE MAIN COMPONENTS:
/system/etc/init.d/Y02firstboot
Code:
#!/system/bin/sh
# execute post-install script on First boot
# 01022012 SENSEISIMPLE
SLEEP=3
FBSCR="/data/.firstboot.sh"
BB="busybox"
pg () {
$BB ps | $BB grep "[email protected]" | $BB grep -v "$( echo $BB grep [email protected] )"
}
if [ -f "$FBSCR" ]; then
#install apps on first boot after system services have started
sleeperlog "Found $FBLOC"
sleeperlog "Waiting for system"
$BB chmod 0755 $FBSCR
while : ; do
if pg systemui; then
$BB sleep 10
sleeperlog "system loaded."
log -p i -t boot "Executing $FBSCR script"
sleeperlog "Running FirstBoot init"
$FBSCR
break
fi
sleeperlog "WAITING FOR SYSTEM SERVICE: sleeping for $SLEEP s..."
$BB sleep $SLEEP
done
fi
/data/.firstboot.sh
Code:
#!/system/bin/sh
#
# 20120107 - SENSEISIMPLE
#
log -p i -t init:firstboot "INIT.firstboot BEGIN: USER SCRIPT $FBLOC.sh"
sleeperlog "FirstBoot Script OK TO RUN"
FBLOC="/data/.firstboot"
tmp="/cache/tmp/firstboot"
bak="$tmp/bak/data/data/"
BB="busybox"
RM="$BB rm"
CP="$BB cp"
MV="$BB mv"
MKDIR="$BB mkdir"
LS="$BB ls"
CHMOD="$BB chmod"
SLEEP="$BB sleep"
GREP="$BB grep"
pg () {
$BB ps | $BB grep "[email protected]" | $BB grep -v "$( echo $BB grep [email protected] )"
}
$CHMOD 0777 /data
#install apps on first boot
if [ -d $FBLOC ]; then
sleeperlog "Found $FBLOC"
if [ "$($LS $FBLOC)" ]; then
cd $FBLOC
$MKDIR -p $bak
for pkg in *.apk; do
log -p i -t init:firstboot "INIT.firstboot INSTALLING: $pkg"
sleeperlog "PREPARING: $pkg"
pkgname=${pkg%.*}
sleeperlog "BACKING UP APP DATA - $pkgname"
#back up data, delete the original data dir to prevent install errors (pm isn't very good at what it does)
if [ -d /data/data/$pkgname ]; then
$CP -a /data/data/$pkgname $bak/$pkgname
$RM -rf /data/data/$pkgname
fi
#WAIT, then install, then WAIT SOME MORE
$SLEEP 2
sleeperlog "INSTALLING $pkgname"
pm install -r $FBLOC/$pkg &
$SLEEP 10
#REIntegrate application data
if [ -d "$bak/$pkgname" ]; then
sleeperlog "\nSYNCING APP DATA \n\n$(/system/xbin/rsync -auv --exclude=lib $bak/$pkgname/ /data/data/$pkgname/)\n"
fi
i=$((i+1))
#Move the install .apk to tmp
if $LS /data/app | $GREP "$pkgname"; then
sleeperlog "Package appears to have installed."
$MV "$pkg" $tmp/
fi
#If the firstboot batch dir is empty, delete it now
! [ "$($LS $FBLOC)" ] && $RM -rf $FBLOC
done
#WAIT for [#ofapps x 5 seconds each] to avoid a race condition
waitsecs=$(( i * 5 ))
sleeperlog "Waiting for ${waitsecs}s before Fixing Permissions"
$SLEEP $waitsecs
sleeperlog "Fixing Permissions $(/system/xbin/fix_permissions)"
sleeperlog "Running batch zipalign \n\n $(/system/xbin/zipalign)"
sleeperlog "Clearing tmp $tmp"
fi
fi
sleeperlog "FIRSTBOOT SCRIPT COMPLETE"
log -p i -t init:firstboot "INIT.firstboot SELF DESTRUCT FIRSTBOOTSCRIPT"
if ! [ "$($LS $FBLOC)" ]; then
$MV $0 $tmp/.firstboot
# COMMENT THIS OUT FOR DEBUGGING, TO NOT REMOVE THE TMP DIR
$RM -r $tmp
echo ""
fi
echo -e "#\n#COMPLETED ON $(date)\n#" >> $0
sleeperlog "FirstBoot Apoptosis"
log -p i -t init:firstboot "INIT.firstboot END: USER SCRIPT $FBLOC.sh"
THANKS
CyanogenMod team for the Fix_permissions script
DarkyROM for the Batch Zipalign script
Saving this seat . . .
does this work with a bml rom?
^^^^In theory, it would work with an NTFS rom, if one existed - different filesystems don't [usually] create any changes on the surface... if you try to copy a file from a partition of any file system to a partition of any other file system, you don't have to explicitly tell the system the fs types involved. I haven't looked through the entire script, but if there are any commands that must specify the partition type, it would be a simple matter of changing any occurances of EXT4 to YAFFS2, etc.
This could be a great way to make a clean reinstall with all of your apps intact (minus app data, do a separate backup with your backup app of choice) - first, backup your installed apps by copying them to the appropriate directory, then do a full wipe and install, with the script automatically reinstalling your apps on first boot...
EDIT: I just looked through the script more closely. First, it appears that data is backed up. Second, I can confirm that the standard, non-file system-specific linux commands are used for file operations (cp to copy, etc), so this script shouldn't need any adjustment to accomodate different file systems
Sent from my SPH-D700 using XDA App
Is it possible for someone to develop an application to backup your personal apps and then running a script in clockwork or on first boot that does the same thing as this, alllowing for personal apps to be in the rom directly after flashing. I understand that sometimes apps may not be compatible, but can randomroms rdu script be modified to choose to do this?
Sent From My Cyan4g
Good looking script SenseiSimple!!
Okay, I'm trying to use this script, but you said to put the apps in fs/data/firstboot/ but where is "fs"? I found the "firstboot.sh" file, but I'm sure I can't put the apk's in there...
In loving memory of my son "Jeffrey Ryan Giles" 11/17/1992 to 11/25/2011 :'(
sniperkill said:
Okay, I'm trying to use this script, but you said to put the apps in fs/data/firstboot/ but where is "fs"? I found the "firstboot.sh" file, but I'm sure I can't put the apk's in there...
In loving memory of my son "Jeffrey Ryan Giles" 11/17/1992 to 11/25/2011 :'(
Click to expand...
Click to collapse
If you look at the folder structure inside the SleeperROM zip, you'll see he uses an fs directory to hold his system and data folders for installation. If you're developing a ROM, you can adapt the script to point to wherever your data folder is (most other ROMs just have system and data in the root directory of the zip, so the path would just be data/firstboot/).
EDIT: no need to look in SleeperROM zip, the firstboot zip in the OP has the same folder structure. It just doesn't seem to have the firstboot directory inside data. You'll need to add that.
Sent from my SPH-D700 using XDA App
bbelos said:
If you look at the folder structure inside the SleeperROM zip, you'll see he uses an fs directory to hold his system and data folders for installation. If you're developing a ROM, you can adapt the script to point to wherever your data folder is (most other ROMs just have system and data in the root directory of the zip, so the path would just be data/firstboot/).
EDIT: no need to look in SleeperROM zip, the firstboot zip in the OP has the same folder structure. It just doesn't seem to have the firstboot directory inside data. You'll need to add that.
Sent from my SPH-D700 using XDA App
Click to expand...
Click to collapse
Thanks for the reply buddy, so what I THINK your saying is to create a folder called "first boot" in my data folder?
In loving memory of my son" Jeffrey Ryan Giles" 11/17/1992 to 11/25/2011 - RIP :'(
sniperkill said:
Thanks for the reply buddy, so what I THINK your saying is to create a folder called "first boot" in my data folder?
In loving memory of my son" Jeffrey Ryan Giles" 11/17/1992 to 11/25/2011 - RIP :'(
Click to expand...
Click to collapse
Are you trying to do this in a ROM or on the phone? Sorry if that's a dumb question, but I just want to be sure we're on the same page.
Sent from my SPH-D700 using XDA App
sniperkill said:
Thanks for the reply buddy, so what I THINK your saying is to create a folder called "first boot" in my data folder?
In loving memory of my son" Jeffrey Ryan Giles" 11/17/1992 to 11/25/2011 - RIP :'(
Click to expand...
Click to collapse
bbelos said:
Are you trying to do this in a ROM or on the phone? Sorry if that's a dumb question, but I just want to be sure we're on the same page.
Sent from my SPH-D700 using XDA App
Click to expand...
Click to collapse
i hadn't noticed my zip routine apparently skips empty .folders
in the zip, there SHOULD have been /fs/data/.firstboot into which you place whatever apk files named according to their actual package name i.e. com.keramidas.TitaniumBackup.apk for Titanium Backup, com.alensw.PicFolder.apk for QuickPic and so on...
you can find out this info in Titanium Backup by finding the app in the list and tapping its name to show the data store location.
it HAS to correspond to the actual package name in order to properly back up the data files because the package manager binary as run from a script won't overwrite the data dir so the old data has to be synced back into the new install (rsync)
I am currently refactoring the script to reduce the number of dependencies... the system will then consist of the init.d script, zipalign and rsync bins, and the /data/.firstboot.sh, /data/.firstboot dir all in one place rather than scattered throughout the system.
irule9000 said:
Is it possible for someone to develop an application to backup your personal apps and then running a script in clockwork or on first boot that does the same thing as this, alllowing for personal apps to be in the rom directly after flashing. I understand that sometimes apps may not be compatible, but can randomroms rdu script be modified to choose to do this?
Sent From My Cyan4g
Click to expand...
Click to collapse
it would be possible to create an app to do this, and i considered it because of the need to make sure the android system is fully loaded for package manager to work, but given the requirement for full root access to start running before the system is up without having to worry about android permissions or getting SuperUser approval it's easier/more effective to do it as part of the system init in a script.
as far as randomking's rdu script, they would be compatible, but they exist separately as in one shouldn't interfere with the other, because the firstboot script runs long after install once the system is up and running.
you could modify the rdu setup to include/exclude the /data/.firstboot dir and .firstboot.sh script based on the selected config, since the init script does nothing if it doesn't see the /data/.firstboot.sh file, and the .firstboot.sh script does nothing if it doesn't see the .firstboot dir
SenseiSimple said:
it would be possible to create an app to do this, and i considered it because of the need to make sure the android system is fully loaded for package manager to work, but given the requirement for full root access to start running before the system is up without having to worry about android permissions or getting SuperUser approval it's easier/more effective to do it as part of the system init in a script.
as far as randomking's rdu script, they would be compatible, but they exist separately as in one shouldn't interfere with the other, because the firstboot script runs long after install once the system is up and running.
you could modify the rdu setup to include/exclude the /data/.firstboot dir and .firstboot.sh script based on the selected config, since the init script does nothing if it doesn't see the /data/.firstboot.sh file, and the .firstboot.sh script does nothing if it doesn't see the .firstboot dir
Click to expand...
Click to collapse
Expanding on the rdu script idea, how about a prop variable in the config to specify a user apps directory on the sdcard (best if standardized)? The user can store the apks to be installed after a clean flash in this directory. The rdu script would be modified to copy those apks to the firstboot directory. Although how to handle data? Maybe another script to be run before flashing that checks the apks in the sdcard folder, and copies the current data folder to the sdcard. This data is then restored somehow with the firstboot script. This all can be in addition to the firstboot apps included with the rom.
Sent from my SPH-D700 using XDA App
app
I honestly have no developing experience. To clarify, I was I simply asking if the app could back up the users personal apps/data to the specified folders, to make this script fool proof. Then when a rom is flashed the randomromkings rdu script which currently decides on lite roms or full roms could be modified to either install apps or not install apps depending on the rom compatibility. Finally data is wiped, rom flashes, and your script runs on boot allowing the apps to all be there. Expanding on that if the ROM allows apps to be re installed via your script, through the modified rdu, your script could be moved from one directory to another allowing the process to happen and at the end moves itself back to the original directory. This means that the end user who has no clue how to do anything but flash roms (like myself, but I want to learn) has done nothing accept backing up their apps. I know this is all hypothetical, but would this be possible, and also I have another idea that is probably impossible. If a script could be written so that the apps are backed up when the rom flashes, then the rom automatically does a factory restore and wipes the various caches, meaning that less people are going to have force close issues or other avoidable bugs because they didnt wipe. thanks for all your hard work
Clear Me Here !!
You said that the script (.firstboot.sh) removes itself, but does it remove Y02firstboot script from init.d?? I don't think so, also, there are no disadvantages of this after first boot, as it won't find .firsboot.sh and skip the operation, but why shud it execute unneccesarily everytime??
@OP--Plz. tell me if i am right or wrong.......If right, plz. add a function to remove that script from init.d. However files in /xbin are OK though, so no need to remove them............
Everybody is writing and making this appear as complicated as possible! So let me clarify what I believe the idea is here. First of all, the 'RDU script' I assume you're referring to is simply a flash-able zip to set you phone for a lite or full installation. It sets a config file. I use it in my rom, and I also currently use that config file to set several other features upon installation, and changeable at anytime one of my themes is flashed.
RDU was simply meant to jump start the idea of developers making installations more uniform. Which has happened to various extents, for example, I've used this first-boot install to solve my quickpic(I love the app) and flash pre-install problems. It also fixes usb tether, vlingo, terminal, etc.
SO, what are we saying here? We'd like to be able to backup our apps to the sd card, as many apps can do for us. Then, we'd like to either:
A. Build a script into a rom to restore these apps prior to or upon installation.
or
B. Build a separate script which would be run AFTER a rom installation to restore these apps from CWM.
Yes?
P.S. I see this is very late to the game, didn't realize this thread was being revived...
Still. Neat idea, sorry I hadn't noticed it yet.
+1 Thanks
After integrating this into my custom ROM, my phone (Galaxy S 4G) would only install the first .apk file in the .firstboot directory. After I removed the code which tells it to backup/restore the /data/data directories, it worked fine. I won't need that code since the ROM does a full wipe of /data every time anyway, but I'm not sure why it doesnt work when it's there. I'm not well versed enough with Java's syntax yet to comprehend why.
Also, your first post says that it should record logs to \sdcard\sleeperlog.txt but the script tries to record it to \sdcard\sleeperromlog.txt and neither one of those files actually appear on the sd card.
Edit: I had to remove the check to make sure the .firstboot directory is empty before deleting it, but it works fine on my Galaxy S 4G ROM as long as there are no /data/data directories for the programs I am installing.
does anyone still have a template of this? The link is down.
Edit: nevermind I don't need it anymore.

adbsync.exe - Win32 command line syncer

Because of the incredible success of my adbsync.exe on the Nook Touch (19 downloads! )
I've decided to advertise it to the general Android public.
There are all sorts of GUI file managers that serve as a front end to ADB.
(Some of them appear to use shelling to ADB instead of using port 5037.)
I wanted a command line program for Win32 that could sync my devices before I go out the door.
It had to work with a Nook Touch (2.1) connected on WiFi and a Kyocera Rise (4.0) connected on USB, often at the same time.
As any program with "option-itis", it runs into an alphabet soup of arguments.
The good part is that you mostly just have to get it right once and stick it in a batch or makefile.
There are options for recursive and whether to create missing subdirs or files.
It can copy in either or both direction and even a 3rd destination for resolving discrepancies.
Code:
usage: adbsync [flags] hostroot androidroot [altroot]
/d device name
/h update host
/a update android
/s search subdirectories
/l list files: d (differences), h (host only), a (android only)
/i ignore files in pattern list
/o only files in pattern list
/v verbose
/q quiet
/1 use 1 second resolution (ext*)
/2 use 2 second resolution (vfat)
Examples:
Code:
[color=red]Update android[/color]
adbsync C:\Books "sdcard/My Files/Books" /a /v
[color=red]Sync both directions[/color]
adbsync C:\Notes sdcard/Notes /a /h /v
[color=red]Sync both directions but don't overwrite the PC, use another dir[/color]
adbsync C:\Notes sdcard/Notes C:\Fixme /a /h /v
[color=red]Sync both directions including subdirectories[/color]
adbsync C:\Stuff sdcard/Stuff /a /h /s /v
[color=red]Sync only existing files on android[/color]
adbsync C:\Reference sdcard/Reference /au /s /v
[color=red]Update host with photos shot on handy (specifying device by id)[/color]
adbsync /d7a12820e /h C:\Handy\Photos sdcard/DCIM/Camera
[color=red]Update android with photos, but not vacation ones[/color]
adbsync C:\Photos sdcard/Photos /o*.jpg;*.jpeg /ivacation* /a /v
Note: For many systems sdcard is just a link (which adbsync.exe will not follow).
Use mnt/sdcard
Download in the signature.
Looks nice, but the problem is that it is hard to determine whether or not it is useful...since it is not clear what it is doing. What do you mean when you say "sync"? Perhaps give an example when you need this and a screen shot or two...
E:V:A said:
Looks nice...
Click to expand...
Click to collapse
Thanks for the bump! :victory:
adbsync.exe compares the timestamps on files on the host and android.
If the flags allow it, it will copy the newer file over the older one.
If the alternate root is specified and it would have overwritten the host's older file it will instead write to that directory.
The post above gives lots of examples.
If files are copied over, they will be listed.
if the /v (verbose) flag is specified, it will list statistics.
There seems to be some problem with DDOS on Dev-Host.
Try the signature below now to download adbsync.exe
This is an indispensable utility for Windows users wanting a complete snapshot of their device + sdcard with their backups. And a great time saver versus standard adb pull. Highly recommended!
Glad to see this over in the main Android Development and Hacking section finally.
I've updated this a bit to correct for a few bugs.
There were a few anomalies if you tried to use the root of Android or the root of a Windows disk.
It will not try to copy links anymore.
If there is a permission problem with reading on the Android it will say so.
I should get around to adding options for ignoring errors, right now it stops.
Finally on your device /sdcard may be only a link.
Use the appropriate real path, probably /mnt/sdcard
Download in signature.
Not copying
I have empty folder in my PC named sdcard and i want to copy all my phone sdcard content to this folder. I used below comment but its not copying anything it just tells "A Android Dir only 33"
can you tell me what i am doing wrong?
adbsync /h /s /v "c:\Name\Name\Soft\Mobile_Soft\Nexus\sdcard\" "/mnt/shell/emulated/0"
mejack03 said:
Can you tell me what i am doing wrong?
Click to expand...
Click to collapse
Code:
adbsync /h[color=red][b]scu[/b][/color] /s /v "c:\Name\Name\Soft\Mobile_Soft\Nexus\sdcard\" "/mnt/shell/emulated/0"
The default action is not to create directories that do not already exist.
Code:
adbsync /?
I am trying to ignore an entire directory. I have tried /i/Dir /iDir* /i/Dir/* and all continue to copy files from the directory. I am using the example above:
adbsync /hscu /s /v /iDir "e:\Nexus\sdcard\" "/mnt/shell/emulated/0"
Yes, currently the include/only options operate on file names, not paths.
I'll look into this...
Thank you. It would be nice if a directory could be disabled. The one I have puts cache items with file names over 100 characters, and neither my win computer nor various cloud storage will take them. Wish I could still run linux, but alas, work software I need is win only.
I'm still seeing issues with the following command putting stuff in the root of the working drive:
Code:
adbsync /hscu /s /ld /v "C:\ADBsync\sdcard\" "/data/media" /i*catalog*
I end up with a bunch of stuff in C:\data\media\ but not everything. So bizarre. I've tried adding/removing trailing slashes from both paths, and all sorts of stuff.
Edit: Seems to be an issue syncing 0 length files (like .nomedia) as well. Not seeing them get created/updated anywhere on the host side, even though adbsync logs report they are.
Yes, the last version would not write zero length files.
That was a by-product of waiting for valid data before creating the file.
The new version (in the signature) addresses that.
For those that wanted ignoration :cyclops: of directories:
I left the current /i /o flags the way they were, that is, operating only on file names.
In the new version if you want a directory ignored, drop a file called .noadbsync in it.
Renate NST said:
Yes, the last version would not write zero length files.
That was a by-product of waiting for valid data before creating the file.
The new version (in the signature) addresses that.
For those that wanted ignoration :cyclops: of directories:
I left the current /i /o flags the way they were, that is, operating only on file names.
In the new version if you want a directory ignored, drop a file called .noadbsync in it.
Click to expand...
Click to collapse
Great idea! This new version is still pulling files to my C:\ though (as reported a few times previously).
Any ideas for that issue?
osm0sis said:
Great idea! This new version is still pulling files to my C:\ though (as reported a few times previously).
Any ideas for that issue?
Click to expand...
Click to collapse
Sorry, I've never seen this.
How many files does it do this to?
Are the filenames weird? Are the paths on the Android weird? Are the paths on the PC weird?
Does it say that it copied them?
Does the correct copy appear also?
I just did a sync like this with a few GB and didn't see anything wrong.
Renate NST said:
Sorry, I've never seen this.
How many files does it do this to?
Are the filenames weird? Are the paths on the Android weird? Are the paths on the PC weird?
Does it say that it copied them?
Does the correct copy appear also?
I just did a sync like this with a few GB and didn't see anything wrong.
Click to expand...
Click to collapse
I wouldn't consider C:\ADBsync\sdcard to be a weird path, nor are /data/media/0/ paths particularly weird, especially when booted into CWM recovery like I am (ie. no weird FUSE FS stuff going on, and no symlinks involved). It copies a lot of files, but not all of them. I'll PM you a dir /s of my sdcard directory before and after the sync, a dir /s of the files that show up at C:\data\media and a log of adbsync's output.
This bug fixed and having an option to delete files that are host-only (to make adbsync clean up after itself) are the only 2 things I feel are missing in adbsync.
Oops.
Yes, there was a problem copying updated Android files that existed on the host to the wrong directory.
That bug popped up in a restructuring of the code.
There is also an improvement for prompter error reporting when attempting to write to Android read-only or protected directories.
Download the new version in the signature.
Renate NST said:
Oops.
Yes, there was a problem copying updated Android files that existed on the host to the wrong directory.
That bug popped up in a restructuring of the code.
There is also an improvement for prompter error reporting when attempting to write to Android read-only or protected directories.
Download the new version in the signature.
Click to expand...
Click to collapse
Confirmed squashed!
Seeing another issue now. adbsync is ignoring some files it shouldn't on the host side. I have a backup in my C:\ADBsync\sdcard\clockworkmod\backup\ directory that isn't in /data/media/clockworkmod/backup/ and adbsync isn't showing it in the log.
The command is still:
Code:
adbsync /hscu /s /ld /v "C:\ADBsync\sdcard\" "/data/media" /i*catalog*
So that should include everything. It pulls new backups from the Android side directory just fine, but it seems to ignore the old ones in the host directory.
You're not syncing from the host to the android.
In this case when adbsync comes across directories on the host that are not on the android it does not even bother enumerating them.
We don't want to waste time traversing everything if we are not going to do anything with it anyway.
If you do want to cover everything in both directions, add /ascu
Hmm. I see your point. However, I don't want to sync in both directions, just have it log what's missing. If there was a proper list only option then that'd be doable with my workaround to delete removed files from the host side (ie. a true host-side sync).
Edit: I know that /ld without /a or /h will list only, but it still doesn't list every host-only difference, like those directories. Could you please make list more thorough and traverse everything, or at least add an option for this?
/ldt maybe?

Duplicate Binaries

I just made an alarming discovery. There are some duplicate binaries installed on our system. rm is one example. I found this out using the rm command. Here is what I found. It exists in BOTH /system/bin AND /system/xbin. If you need proof, I can show you that too.
*********************************************************
Busybox installs its binaries to /system/xbin. Your $PATH is constructed from the /system/etc/mkshrc file that gets executed at boot. The path construct outputs PATH=/system/bin:/system/xbin. Notice that /system/bin comes before /system/xbin in your PATH.
So then the problem with duplicate binaries is that when commands are run from the terminal (without a full path specified to the command call), they are executed from the first directory found in the $PATH which is /system/bin not /system/xbin.
I could do some more **** Tracy and find all the duplicate binaries but I have an easier solution. The solution (for me) is to edit my mkshrc file, and force /system/xbin to come before /system/bin in the $PATH construct. That way I'm not getting jipped with a full Busybox installed by using their less functional Android counterparts. I will post the edited file here in case anyone needs it.
[Edit] On second thought after reviewing some feedback, I will not post this file modification. It has other unintended consequences. Read. Oops, I thought this was posted in the Help & TS section.
elfaure said:
I just made an alarming discovery. There are some duplicate binaries installed on our system. rm is one example. I found this out using the rm command. Here is what I found. It exists in BOTH /system/bin AND /system/xbin.
Click to expand...
Click to collapse
Well, the one in /system/bin is part of stock Android, and the one in xbin is from busybox. No surprise here. In many cases it does not matter which one you actually call, but the versions from busybox are usually more feature-rich. If you need the busybox version, the easiest and safest way to ensure you get the correct one is to explicitly call "busybox rm".
If you modify your PATH, you risk writing scripts that don't work on other devices.
_that said:
If you modify your PATH, you risk writing scripts that don't work on other devices.
Click to expand...
Click to collapse
Please explain.
[Edit] I just discovered other unintended problems with changing my $PATH in mkshrc. Because I have multiple superusers installed [I like to switch between them sometimes; one in /system/bin (active) and the other in /system/xbin (other one)] and by doing that when I rebooted it changed my superuser app. Interesting. So I will not post up the file then - bad idea.
[Edit] It just gets worse. After doing that I now can't sign into my Google account, and I just got a suspicious sign-in email from Google. Really bad idea.
[Edit] Yup. Just verified that my Google account was disabled due to suspicious activity and I had to reset my password. That email was not spam.
elfaure said:
Please explain.
Click to expand...
Click to collapse
I think I get what _that is saying.
For the same reasons that rm was causing some unexpected behavior in your Brower2Ram script, the same thing could happen if a system binary is not as feature-rich as the busybox version. It just so happens for the case of rm that the system binary has more switches than busybox, but who can say for other binaries?
shardsx said:
[Snipped] It just so happens for the case of rm that the system binary has more switches than busybox, but who can say for other binaries?
Click to expand...
Click to collapse
I think you have that backwards. Busybox binary is more feature rich, has more switches, and is the one we want to use. That is why I am suggesting pointing the PATH to busybox (/system/xbin) rather than Android (/system/bin). The one that does NOT have the -f option is the Android one; the default one that gets called from a terminal or my script.
elfaure said:
Please explain.
Click to expand...
Click to collapse
Just think what could happen:
1. You have modified your PATH.
2. You write a script that works on your device.
3. You publish your script.
4. I try running your script and it calls the wrong binary because my PATH is not modified.
elfaure said:
I think you have that backwards. Busybox binary is more feature rich, has more switches, and is the one we want to use. That is why I am suggesting pointing the PATH to busybox (/system/xbin) rather than Android (/system/bin). The one that does NOT have the -f option is the Android one; the default one that gets called from a terminal or my script.
Click to expand...
Click to collapse
Oops, you're right!
_that said:
Just think what could happen:
1. You have modified your PATH.
2. You write a script that works on your device.
3. You publish your script.
4. I try running your script and it calls the wrong binary because my PATH is not modified.
Click to expand...
Click to collapse
Gotcha. This is what I didn't understand @shardsx how modifying my local path in any way could affect anyone else running my script but @_that clearly shows why above. If I can't test it on the user's binary then I'm only testing it for operation on MY device which would be different than YOUR device. So like you said @_that, I need to specifically call the busybox binary for the rm command if I want to use the -f option by either:
1. /system/xbin/rm
or
2. busybox rm
instead of just simply rm (which calls /system/bin/rm). Drama concludes with effective solution.
Find duplicate binaries script
Here is a script to find all the duplicate binaries between /system/bin and /system/xbin. There are quite a few.
[Edit] The attached script was REMOVED because it is WRONG. Once I fix the problems and @_that has approved of the corrections I will post it back up.
elfaure said:
Here is a script to find all the duplicate binaries between /system/bin and /system/xbin.
Click to expand...
Click to collapse
Nice, short and sweet ... and wrong. You should test your scripts before publishing.
Read this: http://en.wikipedia.org/wiki/Test_(Unix) to find how to fix it.
This line is wrong:
Code:
if [ "OBJ"="$XBIN" ]; then
The "[" command (yes, that's a command also called "test") does not understand "=". In this invocation it will simply return true if the string between [ and ] is not empty, which is always. Edit: It does understand " = " though.
Even if test did understand "=" in the normal way of comparing for equality it wouldn't work. First, you forgot the $ before OBJ - you want to compare with the current file name, not with the fixed name "OBJ". Then, if you compare the current $OBJ with the whole list of the files in xbin, it will always be false.
You don't even need the list of all files in xbin - you only need to check for each file in bin if an equally named file in xbin exists. "test" (or "[") has an operator for that.
a working solution:
Code:
for OBJ in $(ls /system/bin)
do
if [ -e /system/xbin/$OBJ ]; then
echo $OBJ
fi
done
_that said:
Nice, short and sweet ... and wrong. You should test your scripts before publishing.
Read this: http://en.wikipedia.org/wiki/Test_(Unix) to find how to fix it.
This line is wrong:
Code:
if [ "OBJ"="$XBIN" ]; then
The "[" command (yes, that's a command also called "test") does not understand "=". In this invocation it will simply return true if the string between [ and ] is not empty, which is always.
Even if test did understand "=" in the normal way of comparing for equality it wouldn't work. First, you forgot the $ before OBJ - you want to compare with the current file name, not with the fixed name "OBJ". Then, if you compare the current $OBJ with the whole list of the files in xbin, it will always be false.
You don't even need the list of all files in xbin - you only need to check for each file in bin if an equally named file in xbin exists. "test" (or "[") has an operator for that.
a working solution:
Code:
for OBJ in $(ls /system/bin)
do
if [ -e /system/xbin/$OBJ ]; then
echo $OBJ
fi
done
Click to expand...
Click to collapse
Ouch! Busy at day job now but I felt that slap all the way from the E. Coast. Is _that called SIP - slap over IP?.
Agreed, I'm still too much of a newb to be QCing my own work. Won't publish anything else without your approval. I will investigate and fix it but it seemed to work when I ran it. Maybe it just listed all the contents of /system/bin?
[Edit] I just wanted to say until I look this over that I did run it and the output looked reasonable for matching binaries. I did not check more than a few.
Note: I have not opened your solution because I am not yet convinced I need to or this structure is wrong. But historically speaking you've never been wrong so I expect to be wrong and learn something from this, but for now I just don't see your point, I believe these work, so I will hold my ground.
Here is a simple test example with identical structure. I see no difference (besides forgetting the $). Comment?
Code:
#!/bin/sh
string1="blah"
string2="blah"
if [ $string1=$string2 ]; then
echo "$string1 $string2"
fi
and here is the code block from my sniper method script after which you called for a cleanup. This code compared a single variable (like $OBJ) to a list variable (like $XBIN).
Code:
If [ -d "${CACHE_NAME1}"=${BROWSER_LIST} ];
then...
[Edit] I put the $ infront of OBJ and re-ran my script and it produces identical output as without it. So every file in /system/bin is getting echoed and the script does not work you are (always) correct. But the test example script I provided does work. Hmm...thats a single variable not a list. But my sniper method worked. Hmm...hmm...
Nor did this make any difference
Code:
if [ "${OBJ}"="${XBIN}" ]; then
Conclusion: My sniper method never worked as intended. It just mounted all of the browser list without ever making the match to all /cache dirs. But it seemed to work as intended. Again, I state that it is difficult to impossible to know when you are doing something wrong in the code when you get your desired output thru a flawed code block which further substantiates your point about needing review. Because then we convince ourselves something worked and try to use the same method again and again only later to find out the method is flawed.
I'm going to follow your links and read up a bit before I take the bait and peek at your solution.
elfaure said:
Here is a simple test example with identical structure. I see no difference (besides forgetting the $). Comment?
Code:
#!/bin/sh
string1="blah"
string2="blah"
if [ $string1=$string2 ]; then
echo "$string1 $string2"
fi
Click to expand...
Click to collapse
Try these variants and observe the result:
Code:
if [ a=a ]; then echo yes; fi
if [ a=b ]; then echo yes; fi
if [ a ]; then echo yes; fi
if [ ]; then echo yes; fi
(I actually did the same to find out what's going on)
elfaure said:
The one that does NOT have the -f option is the Android one; the default one that gets called from a terminal or my script.
Click to expand...
Click to collapse
This seems to have changed between 10.4.4.25 and 10.6.1.14.4 (below, respectively):
Code:
rm [-rR] <target>
Code:
Usage: rm [-rR] [-f] <target>
Welcome to the world of code portability.
---------- Post added at 07:01 AM ---------- Previous post was at 06:48 AM ----------
elfaure said:
Here is a simple test example with identical structure. I see no difference (besides forgetting the $). Comment?
Code:
#!/bin/sh
string1="blah"
string2="blah"
if [ $string1=$string2 ]; then
echo "$string1 $string2"
fi
Click to expand...
Click to collapse
1. You need a space between the comparison operator and the variable names for this to work like you expect.
2. Get into the habit of always quoting variable names when performing such tests (i.e. if [ "$string1" = "$string2" ]; then). Not doing so may bite you when you're not expecting it.
becomingx said:
1. You need a space between the comparison operator and the variable names for this to work like you expect.
2. Get into the habit of always quoting variable names when performing such tests (i.e. if [ "$string1" = "$string2" ]; then). Not doing so may bite you when you're not expecting it.
Click to expand...
Click to collapse
1. I started out with these spaces and tested the script with and without the spaces and it did not make any difference (output was the same) so I removed them thinking they weren't needed. But I did question this. I observed other examples that had this space and was wondering why they were there. Also the spaces between the [ ] and the arguements are required. I did actually know that [=test was a command (from prior readings).
2. Why is this important for variables with no special characters? I thought that was only a grouping thing for variables like "${OBJ}_Cache"
[Edit] Wiki: The quotation marks around $1 ensure that the test works properly even if the value of $1 is a null string. If the quotation marks are omitted and $1 is the empty string, the test command displays the error message "test: argument expected"
_that said:
Try these variants and observe the result:
Code:
if [ a=a ]; then echo yes; fi
if [ a=b ]; then echo yes; fi
if [ a ]; then echo yes; fi
if [ ]; then echo yes; fi
(I actually did the same to find out what's going on)
Click to expand...
Click to collapse
But you may need the spaces around the "=" and quotes around the variables that @becomingx points out for accurate results. As the variable names become more complex this may fail.
Code:
if [ "a" = "b" ]; then echo yes; fi
elfaure said:
2. Why is this important for variables with no special characters? I thought that was only a grouping thing for variables like "${OBJ}_Cache"
[Edit] Wiki: The quotation marks around $1 ensure that the test works properly even if the value of $1 is a null string. If the quotation marks are omitted and $1 is the empty string, the test command displays the error message "test: argument expected"
Click to expand...
Click to collapse
Not only for that reason, but also for things like spaces.
Code:
THIS="and that"
if [ $THIS == "and that" ]; then
...
fi
== `[: too many arguments'
becomingx said:
Not only for that reason, but also for things like spaces.
Code:
THIS="and that"
if [ $THIS == "and that" ]; then
...
fi
== `[: too many arguments'
Click to expand...
Click to collapse
So I followed @_that's link to the Wiki for [=test, and there is absolutely no discussion at all about using "=" or "==" for testing comparisons between variables, only strings. Why?? These examples and your Firefox code for B2R script use the equal operator for comparison and that is what i am trying to do here but I can't yet make it work.
I have failed to figure this out. Here is my test code citing due dilligence. I will now peek at @_that 's solution.
Code:
#!/system/bin/sh
#This script finds duplicate binaries in /system/bin and /system/xbin and prints them to the screen
BIN="$(ls /system/bin)"
#XBIN="$(ls /system/xbin)"
XBIN="$(find /system/xbin -iname "{$OBJ}")"
for OBJ in $BIN
do
#if [ "${OBJ}" = "${XBIN}" ]; then
#if [ "${OBJ}" ] && [ "${XBIN}" ]; then
if [ "${OBJ}" == "$XBIN" ]; then
echo $OBJ
fi
done
I have failed to figure this out. Here is my test code citing due dilligence. I will now peek at @_that 's solution.
[Edit] Short, sweet, simple, works, brilliant. What cereal do you eat in the morning @_that? I think I need to change mine.
Question: Is there any way to make this work with my original structure??? I need a comprehensive method for test matching a string variable to a list variable AND a list variable to another list variable. I would think the latter could be reduced to the former thru the use of a for-do loop "for OBJ in $LIST_VARIABLE1 do ...something to...$LIST_VARIABLE2".
Code:
#!/system/bin/sh
#This script finds duplicate binaries in /system/bin and /system/xbin and prints them to the screen
BIN="$(ls /system/bin)"
#XBIN="$(ls /system/xbin)"
XBIN="$(find /system/xbin -iname "{$OBJ}")"
for OBJ in $BIN
do
#if [ "${OBJ}" = "${XBIN}" ]; then
#if [ "${OBJ}" ] && [ "${XBIN}" ]; then
if [ "${OBJ}" == "$XBIN" ]; then
echo $OBJ
fi
done

[Shell][mksh][Scripting]::mksh-R50e-static-FULL+mkshrc_mods:::

:::BRIEF DESCRIPTION:::
mksh is the official, currently-developed shell used for Android. It is not an app nor a virus (lol) like some people seem to think, looking at a quick xda search. And as the core shell for android, it is used to shell superuser, so if you're rooted and get a message saying "mksh requests root", that basically just means that superuser is being run correctly and is using the default Android shell that is in every Android device. (GOOD)
If you get a message saying bash(or any other shell) requested root, and you're NOT in the terminal interactively. That most likely is NOT GOOD! Almost all bash scripts I've seen will run shelled with mksh, so be safe and shebang with mksh. You'll have better performance and features, and when you see bash requesting root, you'll know when your security is likely compromised.
::: mksh Resources/links/info :::
mksh author: mirabilos (on xda forums)
mirbsd mksh main page: https://www.mirbsd.org/mksh.htm
mksh man page: https://www.mirbsd.org/htman/i386/man1/mksh.htm
edited 15Aug2014
Now that that's out of the way, I've attached a few zips in the 2nd post. The first contains my personalized mkshrc (startup mksh script), the full original mkshrc it's based on, and a diff for anyone who's interested. **This file belongs at /system/etc/mkshrc.** The rest of this first post details what I've cooked up in my mkshrc (and fpath functions). The other zips' contents are described in the 2nd post.
Features scripted into mkshrc:
--sudo: basic su executor
e.g.:
Code:
sudo mount -o rw,remount -t f2fs /system /system
sudo cat <<< $'\n' >> /system/build.prop
sudo 'sysrw; for i in /system/*; do chown 0 "$i" ; done; sysro'
--suid: (set user id) Uses chpst to run a prog setsuid, this is broken in most busyboxes so I included one with a working chpst. Usage:::
e.g.:
Code:
suid 1000 sh
--Rebind tab completion: unjams clustered words:
e.g.: <[cursor]> is cursor position
Code:
bind -m -- '^I= ^B^[=^D'
<<EXAMPLE USAGE>>
ec<[cursor]>12345 + tab ---> echo 12345
--Rebinded "control + u" to "cut-to-beginning of line" like bash. (As mirabilos points out, this is ksh incompatible behaviour since "control + u" is "kill-line" in *ksh shells. This can be removed from the (/system)/etc/mkshrc shell startup file if it presents any problems in editing.) The exact binds are:
Code:
bind -- '^[W=kill-region'
bind -m -- '^U=^[ ^A^[W'
<<EXAMPLE USAGE>>
ec<[cursor]>12345 + ctrl + u ---> <[cursor]>12345
--TMPDIR: mounts a tmpdir at /tmp for all users and random generates subdirs for individual secured tmpdirs. This fixes here-docs, here-strings, and here-executions.
e.g.:
Code:
su << 'eof'
sysrw; chgrp 2000 /system/bin; sysro
! (( $? )) && print -- -changed them bin group-
eof
--Basic "help" command: Android lacks this and the doc is still under development I believe. I added all the basic info/command-forms for mksh bultins to a command help(aliased to h):
e.g.:
Code:
help mkn
> mknod [-m mode] name b|c major minor
> mknod [-m mode] name p
or
Code:
help
(all help is printed, grep from here if you want)
--Complicated-programmed and colorful PS1 prompt, it looks good and I thought it through. Programming shows mksh potential for rich shell programming. It auto-turns off xtrace within itself (to 3 lines) so that 50 lines don't get printed each return with set -x. mksh small (R43 mksh) will be autodetected and give a different white and black classic ksh prompt.
--ton of ultra lazy aliases, got my android scripting speed up to ~70% my normal pc speed.
--dot.mkshrc functions from official mkshrc like pushd, popd, dirs, more, hd, Lb64encode, Lb64decode, hash optimizations
The scripting in this gets pretty complex, especially the inherent mkshrc functions. It would make a good scripting learning tool to say the least. I know I've learned twice what I knew from bash scripting back 3 months ago.
I've been working on modding the mkshrc a while, so I appreciate any error reports if you give mine a try. Shouldn't be any problems though.
Modifying the mkshrc file / mksh shell has system-wide effects, so PLEASE nandroid backup as usual!!! This should be COMPATIBLE with all androids, but has only been tested on my Galaxy Nexus thus far. **The binary is compatible with androids using an ARM cpu, but the steps in the 3rd post will generate a binary for whatever ARCH you're running.**
EDIT: reprogrammed sudo with pexec, so it can do parallel executions. pexec, grep, and a full busybox are included in a zip with some other bins.
For Jack Palevich's terminal emulator or connectbot, telnet, etc, I use this as my su'd "shell" command under >>Options>>Shell:
Code:
/system/xbin/su --shell /system/bin/mksh --command exec /system/bin/pexec -z -15 --shell /system/bin/mksh -m '/system/bin/mksh'
What's safer though is to just use a non-su shell command:
Code:
/system/bin/mksh -
...and then use sudo for your su commands. This is more linux'y.
EDIT2 (big update): Got static mksh R50 compiled. Ended up booting ubuntu in Android to compile instead of cross-compiling. mksh's Build.sh compiler script works great. I just had to "sudo apt-get install gcc" and bam it built with "LDSTATIC=-static ./Build.sh".
EDIT3 - Changed main.c source mkshrc file to /system/etc/mkshrc, perfect static mksh R50 now.
EDIT4 - Changed compiler to klcc to get size way down, now at 196KB with -O2 LTO speed optimizations.
EDIT5 - Binary is updated to R50d, compiled with CFLAGS=-Os, and stripped to get size down to ~140KB. (This is even smaller than the R43 dynamic mksh binary that is currently used in roms.)
EDIT6 - Added a mksh trunk (latest and greatest) binary with the printf builtin compiled in. Use with caution. Btw, if you read all the way down here, then yes, these binaries work in lollipop since they're statically compiled.
:ownloads:: (#7 or #8 is the static binary by itself)
(1) mksh R50 source, binary, manual
(2) Highly compressed pack of extra bins/libs that are mentioned in my mkshrc (zsh, lz4c, grep, busybox, bash, nano, vim, curl, etc) and some other cool ones like macchanger and ed. Download #3 to extract on Android.
(3) 7za in case you need it to extract (2) on Android.
Code:
7za x name_of_archive.7z
(4) Classic unix text editor "ed" which mksh uses for edit functionality. (Copy to /system/bin along with mksh.)
(5) Official mkshrc (unzip and copy to /system/etc, set permissions to 0444)
(6)
My personalized mkshrc startup mksh shell script based on the official full mkshrc 2014/01/11 release. Described in first post. If you try this, make sure to symlink /system/xbin/busybox to /system/bin/busybox.ndk.
Official mkshrc 2014/01/11 release
"diff -ur /system/etc/dot.mkshrc--official.mkshrc mkshrc" (diff patch set from official mkshrc to my personalized mkshrc)
(7) mksh static binary! (Just unzip, "chmod +rx" it, and copy to /system/bin or run from anywhere, no libs required )
(8) mksh R50e with printf builtin (I know some people were looking for this)
::: Building mksh Static On Android (really Ubuntu though..):::
This is just for anyone who wants to try, and especially those without access to a PC.
Things we'll need besides your phone:
-"Android Terminal Emulator" app {get it from (http://jackpal.github.com/Android-Terminal-Emulator/downloads/Term.apk) or fdroid or gplay}
-"Complete Linux Installer" app {get it from (http://sourceforge.net/projects/linuxonandroid/?source=typ_redirect) or gplay or maybe fdroid}
-internet/wifi
-mksh source {download from (https://www.mirbsd.org/MirOS/dist/mir/mksh/mksh-R50.tgz) or link in first post or DL in second post}
1) We need to get Ubuntu or Debian booted for a sufficient build environment. I've used both on Android but I like the better stocked terminal in the Ubuntu images. I used the app Complete Linux Installer which is free and works beautifully, very simple and easy too. In the app you want to follow the short instructions to download an Ubuntu image, rename it to ubuntu.img, and place it in a folder named ubuntu in /sdcard. Then hit menu in the app and click the terminal screen icon that says "Launch". An Ubuntu terminal will now open in Android Terminal Emulator. Super quick and easy.
2) Let's download GCC and ed so we can build and test with zero errors.
Code:
apt-get install -y gcc ed
3) Now the cool thing about this chroot Ubuntu environment is that we still have access to the sdcard to transfer files between Android environment and the chroot jail. Extract your downloaded mksh source to your Ubuntu home with something like:
Code:
cd
tar -xf /sdcard/Download/mksh*.tgz
cd mksh
4) Since we're building this for Android, the mkshrc file is at /system/etc/mkshrc instead of ~/.mkshrc, so we can change this with:
Code:
sed -i 's|~/.mkshrc|/system/etc/mkshrc|' ./main.c
Now we can build mksh statically:
Code:
chmod +x Build.sh
MKSHRC_PATH=/system/etc/mkshrc LDSTATIC=-static ./Build.sh -r -c lto && ./test.sh
Let the build script crank out the binary for a couple minutes and run the test suite. There should only be an error on a "mkshrc" test since we changed it for android. The "-r -c lto" flags/opt aren't needed but recommended.
Edit: (size optimization): Strip the compiled mksh binary to make it super small (~150KB with a CC="klcc" or CC="diet gcc" compile).
Code:
strip --strip-unneeded ./mksh
5) Now let's copy it to /system/bin to replace your old one.
Code:
cp ./mksh /sdcard
(Open a new terminal tab to get into Android Environment)
sysrw
cp -f /sdcard/mksh /system/bin
chmod 0555 /system/bin/mksh
sysro
.. and done. Run some scripts and enjoy your static mksh!
*** OPTIONAL EXTRA STEPS TO USE klcc or other compiler ***
In step (1), also do
Code:
apt-get install -y libklibc-dev
to get the klibc development tools, and then...
In step (4), this is how you specify to use klcc (aka build against klibc using the klcc wrapper):
Code:
CC="$(which klcc)" MKSHRC_PATH=/system/etc/mkshrc LDSTATIC=-static ./Build.sh -r -c lto && ./test.sh
:::Compile mksh in Android (for ARM):::
1) Download the attached cross-compiler and mksh source (https://www.mirbsd.org/MirOS/dist/mir/mksh/mksh-R50.tgz). Copy them to /data/media. If you don't have 7za, download that as well from the previous post but place it in /system/bin and "chmod +x" it
2) Extract contents:
Code:
cd /data/media
7za x -so 2014-10-01__simple-cross-compiler-armv4tl.stripped.tar.7z | tar -xf -
tar -xf mksh*.tgz
3) Compile the source with:
Code:
cd mksh
chmod +x Build.sh
LDFLAGS=-static CC=../simple-cross-compiler-armv4tl.stripped/bin/*gcc ./Build.sh -r -c lto
../simple-cross-compiler-armv4tl.stripped/bin/*strip -s mksh
4) Replace old mksh and mkshrc
Code:
mount -o rw,remount /system
cp mksh /system/bin
cp dot.mkshrc /system/etc/mkshrc
chmod 0555 /system/bin/mksh
chmod 0444 /system/etc/mkshrc
mount -o ro,remount /system
@7175 : Hello! Look like a bunch of hard work there. Very nice.
1. Would you have any clue to what's going on HERE?
2. Would you know if I can just install over my old mksh binary to get this working, or do I need "everything"?
3. And since I've heard that /system/bin/mksh will soon be permanently only /system/bin/sh perhaps this need to be changed?
Hey E.VA. thanks for checking out the thread. If you want to just test out R48 mksh, just copy(from zip root directory) the lib*.so from /data/media/shell to /system/lib and then copy /data/media/shell/mksh to /system/bin (and chmod 0555 /system/bin/mksh; chmod 0444 /system/lib/*.so). Then you'll have your whole system running full mksh R48.
As far as the ssh problem with android, this definitely becomes grey-zone material to my knowledge, which is limited in this area. I'm pretty sure android has udev amputated so it lacks a fully working /dev/tty. This could probably be patched over though with some terminal work, mknod action. I've gotten ssh working under "terminal ide", but haven't used the stock ssh stuff in /system/bin. Now that you mention this though, I will try and get stock ssh going as well as openssh which had a similar problem last time I tried (thx for mentioning your support thread, I'll be checking that out).
Here's what I searched to shed a little light on that ssh /dev/tty issue: duckduckgo.com/html5/?q=ssh+%27dev%2Ftty%27+error.
It looks like you've done a ton of searching though.
For su commands, I would try using, e.g.,
Code:
su exec [i]command[/i]
...opposed to "su -c", or better e.g.,
Code:
su -c exec /system/bin/sh -c '[i]command[/i]'
...and see if that gives any different results.
I just know from experience "su -c" hasn't always cut it.
*edit: Couple other things worth trying:
1) preappend command with environment variable: global TTY=/dev/pts/0 ...
2) use the -T option when calling mksh, i,e. ssh ... -e mksh -T /dev/pts/0 ...
@7175 : Hey thanks for looking into this. I will definitely try your suggestions.
However, I've already tried the -T flag to ssh, but never by specifying the pts to use, as it should be dynamically allocated by the ptmx. (As it could be already in use, by something else, like ATE.) Also, what does "global" do?
You also mentioned "stock ssh" in /system/bin. I've never seen that. Perhaps that could be a better solution, unless its' even more crippled. Does it also support sshd (i.e. ssh -D)?
In the mean time, is there any particular reason why you decided to dynamically link mksh? (I assumed it would have been more portable with static linking?)
I see that you did a huge amount of work on that mkshrc script. Looks crazy (in a good way)!
Finally, and aside, I also noticed that when I start mksh from ssh session, there are some aliases in there, that is not specified in the mkshrc, nor the .profile of my SSH server, so where the heck do they come from?
Code:
[SIZE=2][email protected]:home # alias
autoload='typeset -fu'
functions='typeset -f'
hash='alias -t'
history='fc -l'
integer='typeset -i'
local=typeset
login='exec login'
nameref='typeset -n'
nohup='nohup '
r='fc -e -'
source='PATH=$PATH:. command .'
type='whence -v'[/SIZE]
Perhaps they're hardcoded or sourced from somewhere? Have you seen this?
E:V:A said:
@7175 : Hey thanks for looking into this. I will definitely try your suggestions.
However, I've already tried the -T flag to ssh, but never by specifying the pts to use, as it should be dynamically allocated by the ptmx. (As it could be already in use, by something else, like ATE.) Also, what does "global" do?
You also mentioned "stock ssh" in /system/bin. I've never seen that. Perhaps that could be a better solution, unless its' even more crippled. Does it also support sshd (i.e. ssh -D)?
In the mean time, is there any particular reason why you decided to dynamically link mksh? (I assumed it would have been more portable with static linking?)
I see that you did a huge amount of work on that mkshrc script. Looks crazy (in a good way)!
Finally, and aside, I also noticed that when I start mksh from ssh session, there are some aliases in there, that is not specified in the mkshrc, nor the .profile of my SSH server, so where the heck do they come from?
Code:
[SIZE=2][email protected]:home # alias
autoload='typeset -fu'
functions='typeset -f'
hash='alias -t'
history='fc -l'
integer='typeset -i'
local=typeset
login='exec login'
nameref='typeset -n'
nohup='nohup '
r='fc -e -'
source='PATH=$PATH:. command .'
type='whence -v'[/SIZE]
Perhaps they're hardcoded or sourced from somewhere? Have you seen this?
Click to expand...
Click to collapse
Thanks man lol it's a big rc. It was 50KB before I converted a bunch of the functions to fpath functions in /etc/fpath and stripped most of the comments. It should give a pretty quick shell in terminal though with the ulimit and renicing tweaks. The PS1 prompt is pretty awesome, probably the best thing in the mkshrc I included.
"global" is a function just like "typeset" or "declare" in bash. The difference is that global prevents an accessed variable from being localized/ different than the global value in a function. This is a mksh function, bash will use "declare -g". There are a couple of example usage in the mkshrc, like with the set -x spam fix for the command prompt.
I checked and stock ssh in /system/bin has the "-D" flag. I would also check rolo's zshaolin zsh project which comes with some afterstock ssh stuff, ssh keygen, ssh keysearch, different ssh etc.
Yeah I just symlinked sh to save space really and for portability to try out other shells like bash and zsh(this works!) to run the system. Recently I switched to using a copy of mksh as sh since "chattr +i" wasn't protecting the symlink from being over-linked.
The aliases you posted are in fact hard-coded. If you do "strings /system/bin/mksh | grep -C7 autoload" or something similar, you'll see them in the binary. I always wondered where those came from too until I read through the mksh manual over a couple months, that's some pretty dense reading lol. mksh is a superb shell though, so it's definitely well worth it.
Sorry I can't be of more help with the ssh /dev/tty problem. I will keep my eyes open though, and try not to be so lazy using telnet for all my remote connections. I mean that's basically using the internet without a condom lol.
(*Also, just something I should mention about the install zip contents: not all the /etc/fpath functions are finished yet. I still gotta tweak the ssh ports from /etc/ssh and finish my xargs. Most should work though and are safe and won't interfer with shell usage, even if autoloaded.)
@mirabilos :
Hi, why did you choose to hardcode the aliases (shown in post#6) into the mksh binary?
Also, do you have a later version than the R48 Android binary above?
The "r" alias is particularly disturbing as it is masking the native toolbox use of "r" to read a memory location.
7175 said:
with the ulimit and renicing tweaks.
Click to expand...
Click to collapse
Why do they need to be tweaked?
I checked and stock ssh in /system/bin has the "-D" flag. I would also check rolo's zshaolin zsh project which comes with some afterstock ssh stuff, ssh keygen, ssh keysearch, different ssh etc.
Click to expand...
Click to collapse
Sorry my bad, I was sleepy and mistakenly understood "-D" as running SSH in daemon mode, when in fact it does the opposite and prevents it. Seem like his links are dead or I'm just looking in the wrong place.
Recently I switched to using a copy of mksh as sh since "chattr +i" wasn't protecting the symlink from being over-linked.
Click to expand...
Click to collapse
Not sure what you're saying here...
Sorry I can't be of more help with the ssh /dev/tty problem. I will keep my eyes open though, and try not to be so lazy using telnet for all my remote connections. .
Click to expand...
Click to collapse
Thanks, but if telnet works with tty/pts jobcontrol and vi etc, then I'm fine with that for now. I tried your command variations above and got this:
Code:
[SIZE=2]#ssh -2 [email protected] -p 2222 -T su -c exec /system/bin/sh -i # This worked!
#ssh -2 [email protected] -p 2222 -T su -c exec /tmp-mksh/tmp-mksh -T /dev/pts/0
/tmp-mksh/tmp-mksh: chvt: can't find tty /dev/pts/0
#ssh -2 [email protected] -p 2222 -T su -c exec /tmp-mksh/tmp-mksh -T /dev/ptmx
/tmp-mksh/tmp-mksh: chvt: new shell is potentially insecure, can't revoke /dev/ptmx
[/SIZE]
The fist one is good to get direct root prompt, the second fails, since pts/0 has not been created yet, and if it has, it fails with permission issue, since another process is trying to steal the pts. The reason why the third one fails, is very cryptic and I don't understand the error message and neither the code generating it.
Do you think one could use the ATE app to get a working root shell by the use of a more complex ssh command? Any suggestions how that would look?
Finally, I'm a bit worried about replacing the libc.so dependency, since the whole system depend upon it, and would probably not boot if an incompatible libc.so is used. This is why I asked about a static binary. I'd much rather have a bloated mksh than having to add library files. (We have plenty of memory and space these days.)
Regarding compiling, I would recommend using Linaro a build, rather than the crappy and often broken/outdated Code Sourcery junk.
Check this thread:
[LINARO GCC] Cross Compiler Toolchains [Linaro GCC 4.9 2014.07, 4.8 and 4.7 2014.04][19/07/2014]
Cheers!
7175 said:
--Better tab completion: unjams clustered words:
--Added "control + u" to "cut-to-beginning of line" like bash.
Click to expand...
Click to collapse
Hi! As the author of mksh, I am hereby explicitly requesting you to add a suffix to the shell version, similar to what the PLD Linux guys do here, because this changes the behaviour of the shell in an incompatible way. (Just add your own string there after a space, the @distro@ is just a placeholder that gets replaced with the PLD Linux package version.)
Do note that ^U in Korn Shell (and others) is “delete entire line”.
7175 said:
--Very complicated-programmed and colorful PS1 prompt, it looks good and I thought it through. Programming shows mksh potential for rich shell programming. It auto-turns off xtrace within itself so that 50 lines don't get printed each return with set -x. mksh small will be autodetected and give a different white and black classic ksh prompt.
Click to expand...
Click to collapse
Interesting idea, I think I will pick up the “set +x” thing.
Do you mind sharing a patchset of all your changes, so I can incorporate those I feel would be good to have in general?
7175 said:
EDIT: It appears the libc required for full mksh isnt always friendly
Click to expand...
Click to collapse
Just link the shell statically, then.
E:V:A said:
1. Would you have any clue to what's going on HERE?
Click to expand...
Click to collapse
You are annoying me. I've told you times and times again to disable SEAndroid, or to fix its SELinux policies.
E:V:A said:
2. Would you know if I can just install over my old mksh binary to get this working, or do I need "everything"?
Click to expand...
Click to collapse
Define “everything”. mksh in AOSP is /system/bin/mksh plus mkshrc.
E:V:A said:
3. And since I've heard that /system/bin/mksh will soon be permanently only /system/bin/sh perhaps this need to be changed?
Click to expand...
Click to collapse
I'm working with enh of Google on that. I would like for mksh to be installed as /system/bin/mksh and /system/bin/sh to be a link/copy of it, always. He said he'd agree I could change that back.
E:V:A said:
Hi, why did you choose to hardcode the aliases (shown in post#6) into the mksh binary?
Click to expand...
Click to collapse
These are part of the shell, and have always been, even in pdksh times.
E:V:A said:
Also, do you have a later version than the R48 Android binary above?
Click to expand...
Click to collapse
I do not deal in binaries normally. I'm somewhat working on updating AOSP to mksh CVS HEAD, in tandem with enh from Google, but when I had the time for that, it appears they changed the build system requirements. But it will be there.
It's not as if a newer mksh version would fix your SELinux problem, though…
E:V:A said:
The "r" alias is particularly disturbing as it is masking the native toolbox use of "r" to read a memory location.
Click to expand...
Click to collapse
The “r” alias is especially required for the Emacs command line editing mode. Interesting that toolbox uses it too. Thanks for bringing this to my attention, I will ask enh to change “r” in toolbox.
@E:V:A : Glad the first execution you mentioned worked. I would try messing with quotes/double quotes around the shell command, i.e. ssh ... -T su -c 'exec ...' or try ssh ... -T su -c exec "...", as something to test out further. This can make a lot of difference.
The ulimit shell function can control all sorts of performance related stuff, priority, max niceness, etc. You can see all values with "ulimit -a".
I understand your hesitance about the libc.so replacement. I would keep a copy of the old one and twrp backup before trying out a new one. I get what you're saying about statically linked binaries. I just fixed my i7 desktop that had a blown power supply, so I'll be looking into this. Thanks for the linaro link!
@mirabilos : Thanks for taking a look at this and providing us with mksh! I updated my OP with the specifics about the rebinds for "control I" and "control U" that you mentioned, and I added a suffix to the beginning comments of the file. Something like:
Code:
# $MirOS: src/bin/mksh/dot.mkshrc,v 1.88 2014/01/11 18:09:39 tg Exp $Android_mod by [email protected] 2014/08/04$
Not sure if that's what you wanted me to add in. I included an extra zip to download in the 2nd post that contains the official mkshrc 2014/01/11, my personal mkshrc, and a "diff -ur" patch (also attached to this post).
I will look into getting a statically linked mksh compiled, that sounds like something really handy to keep around.
edit: Here's basically what I added to the beginning and end of PS1 to trim down its "set -x" verbosity to 3 lines. I'd like to get it down more but haven't figured it out.
Code:
PS1=$'\001\r'; PS1+='${|
local e=$?
[[ -o -x ]] && set +x && global -x XTR_DBG=1 || global -x XTR_DBG=0
...
...
...
(( XTR_DBG )) && set -x
return ${e}
} '
7175 said:
@mirabilos : Thanks for taking a look at this and providing us with mksh!
Click to expand...
Click to collapse
You’re welcome!
7175 said:
Not sure if that's what you wanted me to add in.
Click to expand...
Click to collapse
Ah, so you did not patch the mksh binary, only the mkshrc file?
But then, that’s okay as you wrote, sure.
7175 said:
I included an extra zip to download in the 2nd post that contains the official mkshrc 2014/01/11, my personal mkshrc, and a "diff -ur" patch (also attached to this post).
Click to expand...
Click to collapse
OK, thanks. Will have a look at it within the next weeks (August is going to be a busy month for me).
I’m assuming you are okay with me adding some of your changes back to main mksh?
7175 said:
I will look into getting a statically linked mksh compiled, that sounds like something really handy to keep around.
Click to expand...
Click to collapse
Indeed, it is.
The first mksh run on Android actually was statically compiled against µClibc, though that’s cheating. (Also, dietlibc and klibc are better than µClibc). I think it should work against Bionic, too.
7175 said:
edit: Here's basically what I added to the beginning and end of PS1 to trim down its "set -x" verbosity to 3 lines. I'd like to get it down more but haven't figured it out.
Click to expand...
Click to collapse
I don’t think it works in less, considering you have to save the previous $?. There are a few handy tricks around. I tried these four:
#1 is:
Code:
local e=$?
[[ -o -x ]]; local x=$?; set +x
(( e )) && REPLY+="$e|"
[…]
(( x )) || set -x
return $e
This is about what you did. #2 is:
Code:
local e=$? x=$([[ -o -x ]]; echo $?); set +x
(( e )) && REPLY+="$e|"
[…]
(( x )) || set -x
return $e
This only looks different (I’d prefer #1 over this.) #3 is:
Code:
set +x $? $([[ -o -x ]]; echo $?)
(( $1 )) && REPLY+="$1|"
[…]
(( $2 )) || set -x
return $1
This reduces visual clutter. #4 is:
Code:
set +x $? $([[ -o -x ]]; echo $?)
(( $1 )) && REPLY+="$1|"
[…]
(( $2 )) && return $1
(( $1 )) && set -x && return $1
set -x
This reduces visual clutter even more, in that, when $? is 0, one line less is shown. But it makes the number of lines shown be not always the same. YMMV on whether to use this, #3 or #1.
This could be made even nicer *iff* mksh had the EXIT trap when leaving a function. The manpage says it’s TODO. This part of the manpage is from 1999 or older. (I started mksh in late 2002, early 2003.) Go figure… But now I have a reason to add it ☺
Another thing you could do is “set +o inherit-xtrace”, which makes functions not run with “set -x” when the main body runs with it. But wait, I see now that it does not affect ${|…;} blocks. This is a bug, which I’ll fix ☻
@mirabilos:
You Wrote:
1. Would you have any clue to what's going on HERE?
Click to expand...
Click to collapse
You are annoying me. I've told you times and times again to disable SEAndroid, or to fix its SELinux policies.
Click to expand...
Click to collapse
Well, you can be as annoyed as you like. First of all, that question was not aimed at you, and second, as I have also said repeatedly, I have already tried using SELinux in Permissive mode and it doesn't work either. In addition, your suggestion to "fix its SELinux policies" is rather ridiculous as there are no tools available to do so. At least nothing out of the box and no available binaries anywhere, and surely no instructions how to do so. So if you know how to do it, why don't you just explain for us how to do so. (My policies are updated to latest according Androids policy update tools.)
In addition there is no such thing as "disable SEAndroid". It's either in Permissive or Enforced mode. That is, unless you flash a new custom FW, which is out of the question. In addition, ATE works just fine, in Enforcing mode.
So believe me, I am far more annoyed by Googles lack of SELinux/SEAndroid developer support, than you will ever be from me asking these questions.
Thanks for explaining and helping with everything else above.
@7175:
The ulimit shell function can control all sorts of performance related stuff, priority, max niceness, etc. You can see all values with "ulimit -a".
Click to expand...
Click to collapse
Yes, I know about all those things, but I was asking why you think they need to be tweaked? The defaults, seem to work just fine.
@mirabilos : Thank you for your detailed replies, and I would be more than happy to contribute if you see anything from the content I posted.
It is interesting to see the other possibilities for turning off xtrace within the PS1. I didn't even realize you could set positional parameters after the set -x... I think I like that one the most. It just seems more array-like and streamlined, although I guess the same number of variables is still used. I look forward to seeing what updates you'll have on the exit trap and localization. This is something I'd like to toy around with.
I took your advice and set out to get a mksh static built. I ended up compiling with GCC, running "LDSTATIC=-static ./Build.sh" in the mksh source directory. I'm pretty new to cross compiling, so I didn't really get that far or try that hard before chrooting to a fresh Ubuntu terminal on my Nexus, installing GCC, and compiling there. The only thing I changed in the source code was ${ENV:-~/.mkshrc} to ${ENV:-/system/etc/mkshrc} in the "main.c" source file. I can include this in the OP though as you mentioned before. (edit: just saw this as a compiler option with Build.sh)
Edit: I see the option now in the Build.sh for changing the mkshrc path. Also, got R50 rebuilt static with klibc. Damn that got the size way down from using glibc, 684K to 204K.
R50e is released
R50e is released -- see https://www.mirbsd.org/mksh.htm#clog
Hi @7175, good stuffs here man :good:
I've been using your modded mkshrc, and I can see a hell of a lot of work you put into it. Looks really cool. Plus it's really a good place to learn for someone like me, I definitely had learnt a lot for the past few days. Tons of aliases too, and I'm getting used to using it now. I really appreciate your stuffs here, hope to see more of it
alanthehat said:
R50e is released -- see https://www.mirbsd.org/mksh.htm#clog
Click to expand...
Click to collapse
Thanks for the heads up. Updated the binaries posted in the 2nd OP to mksh R50e.
Reinkaos said:
Hi @7175, good stuffs here man :good:
I've been using your modded mkshrc, and I can see a hell of a lot of work you put into it. Looks really cool. Plus it's really a good place to learn for someone like me, I definitely had learnt a lot for the past few days. Tons of aliases too, and I'm getting used to using it now. I really appreciate your stuffs here, hope to see more of it
Click to expand...
Click to collapse
Thanks for the feedback man! Glad it's been a good learning experience like it has been for me.
7175 said:
Thanks for the feedback man! Glad it's been a good learning experience like it has been for me.
Click to expand...
Click to collapse
Well, thanks for the updated binary! Anyway, anything you can recommend me to read other than the man page in OP? And I have one more question, what's the difference between the busybox (I got one from your other thread) and yours busybox.ndk?
While I’m the upstream developer of mksh, hacking it has also been a good learning experience for me.
So it is for a friend of mine, who has taken some fundamental script frameworks I wrote, and extended it to an impressive management system, learning to code good shell while doing so.
I’m always happy when my stuff helps other people, and they share their joy! ☺
By the way: new mksh release announcements are sent over the miros-mksh mailing list (send an eMail to postmaster at mirbsd dot either org or de to subscribe).
---------- Post added at 08:40 PM ---------- Previous post was at 08:34 PM ----------
Well, we have https://www.mirbsd.org/ksh-chan.htm which collects lots of Korn Shell-related resources, although most of it is for AT&T ksh93 (the “original” Korn Shell), much also applies to mksh, though the Android OS has some special handling (especially caused by the different user-space tooling) obviously.
The shell itself behaves exactly the same across *all* supported platforms though, so (once you have set $TMPDIR to something writable for your current user) you can write Pure mksh code exactly like on Unix or Cygwin or the Macintosh or even Haiku.
mirabilos said:
While I’m the upstream developer of mksh, hacking it has also been a good learning experience for me.
So it is for a friend of mine, who has taken some fundamental script frameworks I wrote, and extended it to an impressive management system, learning to code good shell while doing so.
I’m always happy when my stuff helps other people, and they share their joy!
By the way: new mksh release announcements are sent over the miros-mksh mailing list (send an eMail to postmaster at mirbsd dot either org or de to subscribe).
---------- Post added at 08:40 PM ---------- Previous post was at 08:34 PM ----------
Well, we have https://www.mirbsd.org/ksh-chan.htm which collects lots of Korn Shell-related resources, although most of it is for AT&T ksh93 (the “original” Korn Shell), much also applies to mksh, though the Android OS has some special handling (especially caused by the different user-space tooling) obviously.
The shell itself behaves exactly the same across *all* supported platforms though, so (once you have set $TMPDIR to something writable for your current user) you can write Pure mksh code exactly like on Unix or Cygwin or the Macintosh or even Haiku.
Click to expand...
Click to collapse
Thanks for the link, and for your works on mksh too! I'm really new to shell, I'm not even that familiar with unix/linux either. But I always have been interested in shell scripting, so I'm glad that I stumbled upon this thread
Whew, that's a lot of stuff in that link you gave, definitely gonna make use of it, thanks again!

Categories

Resources