My thread and project are inactive now. Please accept my apologies for this inconvenience.
ast.ls
Blackout
ast.np
Blackout
ast.pm
Blackout
ast.ops
Blackout
ast.settings
Blackout
ast.storage
Blackout
2nd Feedback
Thanks for the dedicated thread. If I may, some more nitpicking about rudimentary things.
ast.install has hard coded file names of ast scripts, that's not very portable:
l_Scripts=( "ast.ls" "ast.np" "ast.pm" "ast.ops" "ast.settings" "ast.storage" "astcore" )
I have packages ( "ast.ls" "ast.net" "ast.pm" "ast.ops" "ast.set" "ast.sto" "astcore" )
so more generic pattern for handling like ast* may be considered. In case of renaming, ast.install could have a file renaming template (ast.rename) with say key value rows for generic renaming procedure before starting the actual transfer/installation. Another thing it could try to perform chmod 755 after installation.
EDIT: Looks like you have it but it doesn't work as system is read-only for say /system/xbin installation directory. So installation can be performed either by hand from a file manager with root access or TWRP recovery (install script for easy TWRP install may be a solution).
Do you have any comparative performance observations regarding background network access?
2a. The interesting are the influence of size of the blacklist on system/apps behaviour (fluency, speed, memory usage or possibly noticed negative consequences re some specific apps). By default whitelist contains less than a dozen of items, blacklist is empty. What packages would you consider safe to remove from the whitelist, like LG weather widget? Will 300 blacklisted items be completely practical to have?
2b. Battery consumption changes and practical benefits measured re having hundreds of packages blacklisted vs normally empty blacklist.
2c. I remember you mentioned a FW app you installing on the end of device restoration process, what functionality separate FW app provides for your use case, or is that just an app like AdAway?
hotcell said:
Thanks for the dedicated thread. If I may, some more nitpicking about rudimentary things.
ast.install has hard coded file names of ast scripts, that's not very portable:
l_Scripts=( "ast.ls" "ast.np" "ast.pm" "ast.ops" "ast.settings" "ast.storage" "astcore" )
I have packages ( "ast.ls" "ast.net" "ast.pm" "ast.ops" "ast.set" "ast.sto" "astcore" )
so more generic pattern for handling like ast* may be considered. In case of renaming, ast.install could have a file renaming template (ast.rename) with say key value rows for generic renaming procedure before starting the actual transfer/installation. Another thing it could try to perform chmod 755 after installation.
EDIT: Looks like you have it but it doesn't work as system is read-only for say /system/xbin installation directory. So installation can be performed either by hand from a file manager with root access or TWRP recovery (install script for easy TWRP install may be a solution).
Do you have any comparative performance observations regarding background network access?
2a. The interesting are the influence of size of the blacklist on system/apps behaviour (fluency, speed, memory usage or possibly noticed negative consequences re some specific apps). By default whitelist contains less than a dozen of items, blacklist is empty. What packages would you consider safe to remove from the whitelist, like LG weather widget? Will 300 blacklisted items be completely practical to have?
2b. Battery consumption changes and practical benefits measured re having hundreds of packages blacklisted vs normally empty blacklist.
2c. I remember you mentioned a FW app you installing on the end of device restoration process, what functionality separate FW app provides for your use case, or is that just an app like AdAway?
Click to expand...
Click to collapse
I "liked" your post because your nitpicking is only indication of your care and liking AST scripts, I respect that.
Now, onto addressing your 2nd phase of nitpicking, mildly jocking. As I mentioned previously, perhaps in Info Bank thread, once I get a question, in order not to repeat myself I would cover it, thoroughly, in its relevant post/tool. Please read post #3 for everything I could share on Netpolicy.
Naming issues: Please rename/copy ast.install to ast.setup or anything you wish then update the array list to your liking. This is a quick solution for anyone with the same issue. The beauty of AST is you can modify its scripts in any way you wish in a text editor, on any platforms. Please kindky, don't ask me and provide me with the solution when is not practical. I'm referring to your copying idea instead of hardcoded list of scripts. I know what I'm doing in this instance.
ast.install automatically mounts /system to 'rw' if it was 'ro'. If it was 'rw' already it never set it back to 'ro' to avoid causing problems to the process which set it to 'rw' before ast.install was executed. There are times, /system cannot be mounted as 'rw'. I can guess why, but don't wish go into details for now. A reboot often solves the problem. ast.install will show an error message if it fails but you didn't mention that in your post. So I have no idea what you have complained about at the end.
You have mentioned TWRP before as well. In order to avoid confusion to other users, AST has nothing to offer in TWRP and I don't know why you think it does. I have installed AST to /system/xbin on two LG V20 phones with the same script with no issues. If it is easier to do this manually please do by all means. When you mention installing AST from TWRP, you would give the wrong impression to users, as if AST is an overly complicated thing.
I don't have the time nor the patience to measure battery consumption. Battery consumption is very subjective, as you know. However, I would appreciate it if you like to share your future findings with the community in here.
I'm sorry, I don't recall what was exchanged in reference to 2.c.
Since I know you like AST, how about sharing something with the community about how it made things easier for you? That is a feedback too. No one would take my word for it because they think I'm on my way to become a millionaire.
3rd Feedback
xdav20 said:
I "liked" your post because your nitpicking is only indication of your care and liking AST scripts, I respect that..
Click to expand...
Click to collapse
I never do posts asking or hoping for likes. That would be rather strange assumption. Regarding your obviously negative connotation here, it would be more appropriate if you simply remove the "like" you made.
Now, onto addressing your 2nd phase of nitpicking, mildly jocking. As I mentioned previously, perhaps in Info Bank thread, once I get a question, in order not to repeat myself I would cover it, thoroughly, in its relevant post/tool. Please read post #3 for everything I could share on Netpolicy.
Click to expand...
Click to collapse
Thank you for more detailed explanation. I have to note that you added the explanations in your post #3 after I asked the relevant net policy questions. Either way, IMHO your statement "I think the performance loss or gain is debatable, totally dependant on how Android internally implemented it, and, frankly, this is something I don't wish to get heavily involved with. Please kindly, keep me out of future debates." is not very encouraging. To me it seems that you aren't interested in the discussion of benefits in performance or possible battery savings here, thus logically negating the major possible purpose of using your tool from users perspective. That's strange indeed, but I do respect your wish, so that's it.
Naming issues: Please rename/copy ast.install to ast.setup or anything you wish then update the array list to your liking. This is a quick solution for anyone with the same issue. The beauty of AST is you can modify its scripts in any way you wish in a text editor, on any platforms. Please kindky, don't ask me and provide me with the solution when is not practical. I'm referring to your copying idea instead of hardcoded list of scripts. I know what I'm doing in this instance.
Click to expand...
Click to collapse
Here you are contradicting yourself. As you previously mentioned, you made a deliberate effort to ensure any script can be renamed and still maintain its integrity thus boosting flexibility of usage even more. But here you are basically insisting such move would not be practical. In previous post I suggested you to maintain this flexibility by introducing renaming templates file, so updating the tool will be a simple seemless step/command, without manual intervention required, regardless of a chosen naming scheme. Well, again, it's your call. Perhaps you'd reconsider, just like you did with the inner AST folder in the distribution archive (Thanks BTW for that!).
ast.install automatically mounts /system to 'rw' if it was 'ro'. If it was 'rw' already it never set it back to 'ro' to avoid causing problems to the process which set it to 'rw' before ast.install was executed. There are times, /system cannot be mounted as 'rw'. I can guess why, but don't wish go into details for now. A reboot often solves the problem. ast.install will show an error message if it fails but you didn't mention that in your post. So I have no idea what you have complained about at the end.
Click to expand...
Click to collapse
Indeed I mentioned this due to failure to obtain the expected result on execution of ast.install. Note this script can't be run from ext SD card due to the obvious permission / media format issue. But, normally the /system is in RO state, so even if jackterm has required root permission, the ast.install still would fail when run from internal storage:
elsa:/storage/emulated/0/Download/ast #sh ast.install /system/xbin
cp: /system/xbin/ast.ls: Read-only file system
chmod: chmod '/system/xbin/ast.ls' to 100755: Read-only file system
cp: /system/xbin/ast.net: Read-only file system
chmod: chmod '/system/xbin/ast.net' to 100755: Read-only file system
cp: /system/xbin/ast.pm: Read-only file system
chmod: chmod '/system/xbin/ast.pm' to 100755: Read-only file system
cp: /system/xbin/ast.ops: Read-only file system
chmod: chmod '/system/xbin/ast.ops' to 100755: Read-only file system
cp: /system/xbin/ast.set: Read-only file system
chmod: chmod '/system/xbin/ast.set' to 100755: Read-only file system
cp: /system/xbin/ast.sto: Read-only file system
chmod: chmod '/system/xbin/ast.sto' to 100755: Read-only file system
cp: /system/xbin/astcore: Read-only file system
chmod: chmod '/system/xbin/astcore' to 100755: Read-only file system
I) ast.install: Installation completed.
elsa:/storage/emulated/0/Download/ast #
Why and how reboot supposed to resolve that?
EDIT: Looks like it works fine when performed right after reboot! It would be nice if install script would try to perform a mount, cp, chmod and report about the failure. Instead of showing the (fake) message "Installation completed." the hint to reboot and retry would be much more useful.
You have mentioned TWRP before as well. In order to avoid confusion to other users, AST has nothing to offer in TWRP and I don't know why you think it does. I have installed AST to /system/xbin on two LG V20 phones with the same script with no issues. If it is easier to do this manually please do by all means. When you mention installing AST from TWRP, you would give the wrong impression to users, as if AST is an overly complicated thing.
Click to expand...
Click to collapse
It has nothing to do with wrong impressions. All I tried to suggest was an attempt to provide a path to easy and working solution to install/update AST and have a workaround for system RO state issue as described above. One still will have to ensure the system partition is mounted in TWRP before doing AST install/update. Sure if ast.install or its exec environment can be fixed, there would be no need to reboot to recovery or perform manual steps with a file manager.
I don't have the time nor the patience to measure battery consumption. Battery consumption is very subjective, as you know. However, I would appreciate it if you like to share your future findings with the community in here.
Click to expand...
Click to collapse
If and when I'll proceed with net policy customisation and start to have some comarative data on battery consumption, I'll try to post here without involving you to the subject as per your wish to keep you out of future debates.
@hotcell
I have attached a modified version of ast.install that would double check if 'mount' command remounted /system rw or not. If the script still fails to copy your scripts, without displaying an error message, then, and previously, the problem has nothing to do with my script. I haven't developed 'mount' binary nor know why on your particular system things doesn't work normally. There is no 'fake' message, because of what happens the script assumes everything went well. If the scripts were previously copied to the target location, there is no easy way to know if the copy was successful or not. For such non-critical job, doing more is waste of time.
"ast.storage -lb online" displays the mounted partitions. The very last data is the current read/write status of the partition. After a reboot, could you run the above command to see what it prints about /system partition before and after running ast.install please.
I'm a tool maker, I'm not going to tell anyone where you can use my tool. I can provide guidance how the tool should be used, for productivity purposes only. What thing you have forgotten was, ast.np is a helper tool for Android Netpolicy tool. Without ast.np it is, literally, impossible to use it due to the input format it takes. Knowing how over analytical you can be, I made it clear I wish to stay away from any future debates myself. I have not stated, no one else can debate about it on my thread. I'm sorry, if you took my statement personally and I encourage you to use my thread to discuss anything related to AST. I would response, if I had to.
I thought for a while, how can I prove to @hotcell I wasn't being sarcastic when I "liked" his post? My proof is at the very end of that post when stated: "Since I know you like AST, how about sharing something with the community about how it made things easier for you?" If I didn't mean it, I wouldn't have stated a factual thing, I would had said; "If you like AST then why don't you...". Since I meant what I said without a menacing connotation, I stand by my previous decision. However, I do share your sentiment about the "like" system in place.
I have a serious issue with you or anyone else who think I have to accommodate to their specific needs in the same release that is meant for the public and no matter how accommodating I have been to you, you still have an approach that I don't feel comfortable with it. I will response equally to anyone who would want to dedicate terms to me. Once you renamed the scripts, I do not have to change the installer with your suggestion method, period. Have you thought, in the same directory users might have scripts or files starting with the "ast" prefix? That was the reason I hard-coded the script names to avoid potential disasters.
Edit: Noticed your edited comment about everything started working after a reboot. I do not like to discuss about something, specially in a public forum, when I do not have factual information. I'm normally good at observing patterns. I'm purely guessing, the reason behind the failure of mount command is to do with how certain file managers that support root handle mounting /system partition. Of course, there is always more than one reasons to such conditions.
Here is what you can do to see if that is the caase or not. Do everything as you would do after a reboot. Use the file managers you use and access the /system partition and copy something over to force the file manager to make the /system partition rw. Exit the file manager and then try the installer script again. Repeat the same thing, this time don't use the same file managers and after the system has been up and running for awhile, try the installer script again. What did you observe?
4th Feedback
Yes, you were right. After writing to the /system with MiXplorer file manager, the ast.install script fails. See screenshots: first made befre using MiX on /system, after that script worked; the second pic shows the ast.install.sh failure after using MiX, please note the (fake because of actual failure to mount and errors reported before) success report on the last line; the trird pic was made after the failed install. Also, with Root explorer, it asks to remount the system partition as R-W for operation, after that the ast.install(.sh) fails exactly as shown on the middle pic.
xdav20 said:
There is no 'fake' message, because of what happens the script assumes everything went well. If the scripts were previously copied to the target location, there is no easy way to know if the copy was successful or not.
Click to expand...
Click to collapse
The easiest way to know is probably to try to write an empty file into installation dir and check for its existance. Another way would be checking file timestamp.
I have a serious issue with you or anyone else who think I have to accommodate to their specific needs in the same release that is meant for the public and no matter how accommodating I have been to you, you still have an approach that I don't feel comfortable with it. I will response equally to anyone who would want to dedicate terms to me. Once you renamed the scripts, I do not have to change the installer with your suggestion method, period. Have you thought, in the same directory users might have scripts or files starting with the "ast" prefix? That was the reason I hard-coded the script names to avoid potential disasters.
Click to expand...
Click to collapse
Of course you don't have to accomodate or appease anyone. If maintaining renaming flexibility and extending that to install script is not you goal AST tool may benefit in terms of flexibility, just forget about this. And yes, I thought about ast prefix oversimplicity. That is the exact reason why I suggested ast.rename file template where one can explicitly define source-destination name pairs. So having a single fixed named file name for such template is all needed to avoid any possible clash.
hotcell said:
Yes, you were right. After writing to the /system with MiXplorer file manager, the ast.install script fails. See screenshots: first made befre using MiX on /system, after that script worked; the second pic shows the ast.install.sh failure after using MiX, please note the (fake because of actual failure to mount and errors reported before) success report on the last line; the trird pic was made after the failed install. Also, with Root explorer, it asks to remount the system partition as R-W for operation, after that the ast.install(.sh) fails exactly as shown on the middle pic.
The easiest way to know is probably to try to write an empty file into installation dir and check for its existance. Another way would be checking file timestamp.
Of course you don't have to accomodate or appease anyone. If maintaining renaming flexibility and extending that to install script is not you goal AST tool may benefit in terms of flexibility, just forget about this. And yes, I thought about ast prefix oversimplicity. That is the exact reason why I suggested ast.rename file template where one can explicitly define source-destination name pairs. So having a single fixed named file name for such template is all needed to avoid any possible clash.
Click to expand...
Click to collapse
Thank you for the quick response. Of course, if my script fails, so as everything else from command line. Now that we established my guess is an actual fact, I have another immediate concern.
Thanks to your screenshots, I have noticed storage.ast -lb command is not displaying the partition fs type and partition read&write status of /system and /userdata and there were errors. I installed Mixplorer and navigated to xbin and created a file. Went back to terminal and used ast.storage -lb online again and it reported everything correctly. You either have a corrupt fs table or there is a condition that I haven't seen before. To make sure, I would like to ask you to send me the following files by running these commands please:
Code:
cat /proc/mounts > mounts
cat /proc/self/mountinfo > mountinfo
cat /proc/self/mountstats > mountstats
Please make sure you have used Mixplorer or the usual things you do that may cause the problem first.
Edit: After a reboot, please run the ast.storage -lb online again and please report back if there were errors. I'm guessing if you do, there is something wrong on your end.
Disclaimer: At the time of publication of this article, I have not found a way to remedy the problem with the application mentioned in this article. If you did, please notify me to make the appropriate changes to this article.
This article came to existence after exchanging messages with @hotcell to which my further investigation revealed a problem with MIXplorer app. I understand MIXplorer has a thread on XDA so there is a chance the developer to be contacted to rectify the problem.
The problem is after /system partition is remounted 'rw' (Read & Write) MIXplorer does not set the partition back to 'ro' (Read-only) even after the application is closed explicitly. This can leave you extremely vulnerable in case a malicious app awaits for such opportunity. Fortunately, after a reboot all partitions will be set back to their default status. However, users might not reboot their phones for days.
Root Explorer from SpeedSoftware has a mount toggle (rw,ro) button for each partition you use which it gives you the control to change the status at will. Then I tested ES File Explorer, as soon as I closed the app the /system was remounted back to 'ro'.
Thanks to ast.storage -lb command I could easily tell what was going on. My advice to you is, to either contact the developer of MIXplorer and request the partitions to be remounted as 'ro' upon exiting the app or similarly to provide a toggle button. In the mean while, I wouldn't use MIXplorer on a device that is connected to the internet.
5th
@xdav20
Although irrelevant in the thread common context, please see the attachment below (as it looks like you disabled PM functionality).
I see your AST gives formatting errors, but practically I can't see any problems with my system. Also, looks like if /system left R-W mounted by a file manager or other app, AST has no way to proceed. As you noticed about re-mounting on-the-fly from UI, reboot can be avoided by remounting the /system as RO in Root Explorer. Also, another decent file manager X-plorer doesn't need an (clean) exit as it will re-mount the system as RO after FS operation finished.
hotcell said:
@xdav20
I see your AST gives formatting errors, but practically I can't see any problems with my system. Also, looks like if /system left R-W mounted by a file manager or other app, AST has no way to proceed..
Click to expand...
Click to collapse
Thank you for the files. I will start looking into them.
I have already stated, two posts up, ast.storage's -lb command works fine on my phone even with MIXplorer installed and /system remounted rw. While I'm trying to establish if your phone has a problem or not, I have to be here correcting your incorrect statement. Also, I'm wondering why you never reported the error messages before when you did about other things?
Edit: I have attached a screenshot to prove everything works. I deliberately set the /system to rw.
6th
xdav20 said:
Thank you for the files. I will start looking into them.
I have already stated, two posts up, ast.storage's -lb command works fine on my phone even with MIXplorer installed and /system remounted rw. While I'm trying to establish if your phone has a problem or not, I have to be here correcting your incorrect statement. Also, I'm wondering why you never reported the error messages before when you did about other things?
Click to expand...
Click to collapse
Perhaps my statement is incorrect, I've no idea. But how would you explain that after Root Explorer toggling to RO state ast.install works perfectly, what Root Explorer can do that ast.install can't?
For the last question there is an easy answer: When I reported about "fake installation success" clause, it was obvious the copy failed, screenshots were attached. Timing to report wasn't/isn't critical for me. Also, looks like you are using SuperSU while I've Magisk, maybe there are some diffs/settings related?
EDIT: After playing with renaming ast.net in /system/xbin via X-plore, Root Explorer and MiX more:
1. X-plore always know how to rename the file
2. Root Explorer knows and warns when in RO state, but fails to rename after subsequent renamings in other FMs. Toggling RW->RO>RW restores renaming capability.
3. MiX fails to rename first.
hotcell said:
Perhaps my statement is incorrect, I've no idea. But how would you explain that after Root Explorer toggling to RO state ast.install works perfectly, what Root Explorer can do that ast.install can't?
For the last question there is an easy answer: When I reported about "fake installation success" clause, it was obvious the copy failed, screenshots were attached. Timing to report wasn't/isn't critical for me. Also, looks like you are using SuperSU while I've Magisk, maybe there are some diffs/settings related?
EDIT: After playing with renaming ast.net in /system/xbin via X-plore, Root Explorer and MiX more:
1. X-plore always know how to rename the file
2. Root Explorer knows and warns when in RO state, but fails to rename after subsequent renamings in other FMs. Toggling RW->RO>RW restores renaming capability.
3. MiX fails to rename first.
Click to expand...
Click to collapse
Found the cause, I think I have the solution, and I have to leave to my dentist appointment soon. I will release a fix as soon as I can. There is nothing wrong with your device but with what you use that commonly cause problems to everyone else's (not literally) code. Thanks again for the files. The future fix will not fix the mount issue, mount issue is a common issue in Android because so many processes use /system partition and bound to be conflicts.
7th
xdav20 said:
Found the cause, I think I have the solution, and I have to leave to my dentist appointment soon. I will release a fix as soon as I can. There is nothing wrong with your device but with what you use that commonly cause problems to everyone else's (not literally) code. Thanks again for the files. The future fix will not fix the mount issue, mount issue is a common issue in Android because so many processes uses /system partition and bound to be conflicts.
Click to expand...
Click to collapse
Thanks, although I have no idea what you are going to fix (as it seems the issue is in mounting)
Anyway, I thought you can try to implement an algo like this:
1. check if /system looks mounted as RW, remember the state for later restoration, if not RW, try to re-mount
2. create am empty file named as current time in installation dir
3. try to check if it exists. if yes, delete it and set flag ready=true and proceed to #5.
4. if flag tried is not set (false), try to make double re-mount RW>RO>RW, set flag tried=true, return to #2.
5. if flag ready=false, return error message asking to either reboot or mount /system as RW and exit
6. if ast.rename is absent from install dir, perform scripts copy, otherwise the same with renaming according to the template
7. set chmod on script files as required. Done.
@hotcell
I have attached a pre-release version of AST to this post. This release contains support to handle multiple mountpints on online blocks. Android by default has one single mountpoint per block. However, components/su such as Magisk creates additional mountpoints, one on /dev/block/sda14 and two on /dev/block/sda18. AST can now handle multiple mountpoints on any online blocks, referring to -lb command. Thank you again for the files and your assistance in resolving this issue.
As you have guessed by now, Magisk was the cause of errors and I'm guessing the source of your /system mount issues. Now, that '-lb' command can show correct information on your device, you can keep your eyes on read and write status of both mountpoints on /dev/block/sda14. Please kindly provide a screenshot of '-lb online' so I can make sure everything was okay before releasing it on OP. Since you have installed AST in /system/xbin, please do not run anything locally without specifying its full parh in command line. I'm updating OP about this point shortly.
One minor matter, I have to respectfully ask you to refrain from referencing to AST scripts as your renamed versions. I got confused when you referenced ast.np to ast.net. For anyone following our conversations it can workout confusing too. At this point of time, I don't wish to make a rule and publish it on OP. I hope you can understand the point I raised. Thank you for your understanding.
Edit:01
Added two screenshots. Magisk-Partitions image shows the online partitions with Magisk being installed (based on your device data files) and Default-Partitions image shows a typical device without Magisk? Have you spotted the additional entries?