Related
Well, I was going to hold off until all of the vendors had new releases posted, but now that the cat is out of the bag and the evildoers have sufficient information to figure out what got fixed:
[size=+1]Current Superuser/SuperSU releases have security holes that allow any application to execute commands as root without the user's permission (even apps with no permissions). Please upgrade immediately to SuperSU >= v1.69 or another patched release.[/size]
This is expected to impact the vast majority of rooted devices and custom ROMs.
Details follow:
[size=+2]Superuser unsanitized environment vulnerability on Android <= 4.2.x[/size]
Vulnerable releases of several common Android Superuser packages may allow malicious Android applications to execute arbitrary commands as root without notifying the device owner:
ChainsDD Superuser (current releases, including v3.1.3)
CyanogenMod/ClockWorkMod/Koush Superuser (current releases, including v1.0.2.1)
Chainfire SuperSU prior to v1.69
The majority of third-party ROMs include one of these packages.
On a rooted Android <= 4.2.x device, /system/xbin/su is a setuid root binary which performs a number of privilege checks in order to determine whether the operation requested by the caller should be allowed. In the course of its normal duties, and prior to making the allow/deny decision, /system/xbin/su invokes external programs under a privileged UID, typically root (0) or system (1000):
/system/bin/log, to record activity to logcat
/system/bin/am, to send intents to the Superuser Java app
/system/bin/sh, to execute the /system/bin/am wrapper script
/system/bin/app_process, the Dalvik VM
The user who invokes /system/xbin/su may have the ability to manipulate the environment variables, file descriptors, signals, rlimits, tty/stdin/stdout/stderr, and possibly other items belonging to any of these subprocesses. At least two vulnerabilities are readily apparent:
- On ClockWorkMod Superuser, /system/xbin/su does not set PATH to a known-good value, so a malicious user could trick /system/bin/am into using a trojaned app_process binary:
Code:
echo -e '#!/system/bin/sh\nexport PATH=/system/bin:$PATH\ntouch /data/trojan.out\nexec $0 "[email protected]"' > app_process ; chmod 755 app_process
PATH=`pwd`:$PATH su -c 'true'
The PATH vulnerability is being tracked under CVE-2013-6768.
- Other environment variables could be used to affect the behavior of the (moderately complex) subprocesses. For instance, manipulation of BOOTCLASSPATH could cause a malicious .jar file to be loaded into the privileged Dalvik VM instance. All three Superuser implementations allowed Dalvik's BOOTCLASSPATH to be supplied by the attacker.
The BOOTCLASSPATH vulnerability is being tracked under CVE-2013-6774.
[size=+2]Android Superuser shell character escape vulnerability[/size]
Vulnerable releases of two common Android Superuser packages may allow malicious Android applications to execute arbitrary commands as root, either without prompting the user or after the user has denied the request:
CyanogenMod/ClockWorkMod/Koush Superuser (current releases, including v1.0.2.1)
Chainfire SuperSU prior to v1.69
The majority of recent third-party ROMs include one of these packages. Older ROMs may use the ChainsDD Superuser package, which is not affected but is no longer maintained.
On a rooted Android <= 4.2.x device, /system/xbin/su is a setuid root binary which performs a number of privilege checks in order to determine whether the operation requested by the caller should be allowed. If any of these checks fail, the denial is recorded by broadcasting an intent to the Superuser app through the Android Activity Manager binary, /system/bin/am. /system/bin/am is invoked as root, and user-supplied arguments to the "su" command can be included on the "am" command line.
On a rooted Android >= 4.3 device, due to changes in Android's security model, /system/xbin/su functions as an unprivileged client which connects to a "su daemon" started early in the boot process. The client passes the request over a UNIX socket, and the daemon reads the caller's credentials using SO_PEERCRED. As described above, /system/bin/am is called (now from the daemon) to communicate with the app that implements the user interface.
If the user invokes "su -c 'COMMAND'" and the request is denied (or approved), ClockWorkMod Superuser constructs a command line to pass to a root shell:
Code:
snprintf(user_result_command, sizeof(user_result_command), "exec /system/bin/am " ACTION_RESULT " --ei binary_version %d --es from_name '%s' --es desired_name '%s' --ei uid %d --ei desired_uid %d --es command '%s' --es action %s --user %d",
VERSION_CODE,
ctx->from.name, ctx->to.name,
ctx->from.uid, ctx->to.uid, get_command(&ctx->to),
policy == ALLOW ? "allow" : "deny", ctx->user.android_user_id);
get_command() would return "COMMAND", unescaped, through "/system/bin/sh -c". By adding shell metacharacters to the command, the root subshell can be tricked into running arbitrary command lines as root:
Code:
su -c "'&touch /data/abc;'"
Upon denial by the operator, "touch /data/abc" will be executed with root privileges. The Superuser variant of this problem is being tracked under CVE-2013-6769.
SuperSU prior to v1.69 removes quote and backslash characters from the string passed to /system/bin/sh, but backticks or $() can be used instead for the same effect:
Code:
su -c '`touch /data/abc`'
su -c '$(touch /data/abc)'
The SuperSU variant of this problem is being tracked under CVE-2013-6775.
ChainsDD Superuser v3.1.3 does not appear to pass the user-supplied input on the /system/bin/am command line.
[size=+2]Superuser "su --daemon" vulnerability on Android >= 4.3[/size]
Current releases of the CyanogenMod/ClockWorkMod/Koush Superuser package may allow restricted local users to execute arbitrary commands as root in certain, non-default device configurations.
Android 4.3 introduced the concept of "restricted profiles," created through the Settings -> Users menu. A restricted profile can be configured to allow access to only a minimal set of applications, and has extremely limited abilities to change settings on the device. This is often used to enforce parental controls, or to protect shared devices set up in public places. The OS requires an unlock code to be entered in order to access the owner's profile to administer the system.
/system/xbin/su is a setuid root executable, and any user may invoke it in client mode ("su -c 'foo'" or just "su"), or in daemon mode ("su --daemon"). In either mode of operation, the user who invokes this program has the ability to manipulate its environment variables, file descriptors, signals, rlimits, tty/stdin/stdout/stderr, and possibly other items. By adding new entries at the front of the PATH for commonly-executed root commands, then re-invoking "su --daemon", an attacker may be able to hijack legitimate root sessions subsequently started by other applications on the device.
"su --daemon" is normally started up very early in the boot process, as root, from /init.superuser.rc (CM) or from /system/etc/install-recovery.sh (other ROMs). The fact that unprivileged users are allowed to restart the daemon later, under EUID 0, appears to be an oversight.
Successful exploitation requires a number of conditions to be met:
- The attacker must have ADB shell access, e.g. over USB. This is disabled by default, and normally restricted to trusted ADB clients whose RSA key fingerprints have been accepted by the device administrator. Root access via ADB (i.e. Settings -> Developer Options -> Root access -> Apps and ADB) is not required. Note that ADB shell access is typically considered a security risk, even in the absence of this problem.
- The attacker must have a way to assume a non-shell (non-2000), suid-capable Linux UID in order to prevent /system/xbin/su from creating infinitely recursive connections to itself through the daemon client UID check in main(). One way to do this would involve uploading an app with the "debuggable" flag and using /system/bin/run-as to assume this UID. "adb install" can probably used for this purpose. However, due to a bug in Android 4.3's "run-as" implementation[1], this does not currently work. This bug was fixed in Android 4.4, so CM11 will probably be able to satisfy this requirement.
- The device owner must have granted root permissions to one or more applications via Superuser. The restricted profile does not need to be able to run this app from the launcher.
Sample exploit:
The restricted local user can reboot the tablet, run "adb shell" when the boot animation shows up, then invoke the following commands:
Code:
echo -e '#!/system/bin/sh\nexport PATH=/system/bin:$PATH\ntouch /data/trojan.out\nexec $0 "[email protected]"' > /data/local/tmp/trojan
chmod 755 /data/local/tmp/trojan
for x in id ls cp cat touch chmod chown iptables dmesg; do ln -s trojan /data/local/tmp/$x ; done
PATH=/data/local/tmp:$PATH setsid run-as.422 my.debuggable.package /system/xbin/su --daemon &
(Note the use of "run-as.422" as a proxy for a working Android 4.3 run-as binary, and the installation of "my.debuggable.package" with the debuggable flag set.)
At this point the USB cable may be disconnected.
The next time a root application successfully passes the Superuser check and invokes one of the trojaned shell commands, /data/local/tmp/trojan will be executed under UID 0.
An ideal candidate for exploitation is a package which runs privileged commands on boot, e.g. AdBlock Plus or AFWall+, as this allows for instant access. Another possibility is to hijack an app which the device's operator runs frequently, such as Titanium Backup.
Note that this can NOT be exploited by malicious applications, as zygote-spawned processes (apps) always access /system in nosuid mode[2] on Android 4.3+. The ADB shell was used as the attack vector as it is not subject to this restriction.
ChainsDD Superuser v3.1.3 does not have an Android 4.3+ client/server mode at all, and SuperSU aborts if an existing "daemonsu" instance is already bound to the abstract @"eu.chainfire.supersu" socket.
Proposed resolution: on Android 4.3 and higher, install all Superuser-related binaries with mode 0755 (setuid bit unset).
This problem is being tracked under CVE-2013-6770.
[1] https://code.google.com/p/android/issues/detail?id=58373
[2] http://source.android.com/devices/tech/security/enhancements43.html
Did you report that to @Chainfire?
SecUpwN said:
Did you report that to @Chainfire?
Click to expand...
Click to collapse
Yes, he's been very responsive.
I contacted all three developers last Saturday, and posted the advisory after there was enough public information available to deduce what the problems were.
In case you're curious, there's been some additional discussion about exploiting ChainsDD Superuser on BUGTRAQ.
Is there a way we can patch this maybe using xposed framework
milojoseph said:
Is there a way we can patch this
Click to expand...
Click to collapse
There are new releases of SuperSU and CWM Superuser posted:
https://play.google.com/store/apps/details?id=eu.chainfire.supersu&hl=en
http://forum.xda-developers.com/showthread.php?t=1538053
https://play.google.com/store/apps/details?id=com.koushikdutta.superuser&hl=en
I haven't seen any updates to ChainsDD Superuser, and AFAICT the project is no longer maintained.
maybe using xposed framework
Click to expand...
Click to collapse
Xposed is useful for patching Java programs, but /system/xbin/su is compiled C code. So the techniques used by Xposed would not apply to this case.
cernekee said:
Xposed is useful for patching Java programs, but /system/xbin/su is compiled C code. So the techniques used by Xposed would not apply to this case.
Click to expand...
Click to collapse
There's always Substrate, that can be used even for patching native code, but still in this case not applicable I guess.
Where you able to find any patch to fix them?
thank you for sharing ...
I am getting this message in lollipop "zygote has been granted superuser permission" i accidentally allowed it root access thinking it was link2sd. Could it be malware? There is a nameless app in my supersu under name "zygote". i didn't installed anything outside from playstore. My supersu version is 2.78
diabolicalprophecy said:
I am getting this message in lollipop "zygote has been granted superuser permission" i accidentally allowed it root access thinking it was link2sd. Could it be malware? There is a nameless app in my supersu under name "zygote". i didn't installed anything outside from playstore. My supersu version is 2.78
Click to expand...
Click to collapse
Did you get an answer for this? I have the same issue on 4.4.4
Vankog said:
Did you get an answer for this? I have the same issue on 4.4.4
Click to expand...
Click to collapse
No I didn't, I reflashed the rom and it solved the problem.
I am using a custom executable to create a chroot from a user process, the su -c command is not working for me and even if I fixed it forwarding everything through the daemon wont work because the caller needs a pipe to the stdio of the process inside the chroot.
Android 4.3 introduces selinux and a restriction that is separate from selinux(which is disabled on my testing device) that prevents android apps calling setuid binarys from having the setuid bit function.
Android 4.3 Security Enhancements
How could I disable this?
I am able to compile my own kernel if that is required but I would prefer something that wont take as long.
I share here my script for repeating a wifi signal. Working on Lineage OS based ROM 8.1 and Qualcom(msm). This one is currently working in a debian chroot, I will post the Android shell version here soon. I would enjoy any feedback/suggestions.
Sources....
Mostly an update of this......
https://android.stackexchange.com/q...bridging-wi-fi-with-access-point?noredirect=1
I also barrowed pieces and took reference from create_ap....
https://github.com/oblique/create_ap
The main reason for using the chroot was access to iwconfig. The Android shell script hangs when scripting iw(as iw warns about in dialogs)
I use Debian's hostapd/dnmasq, the iptables binary is copied from Android system, it hangs otherwise.
It needs work, nl80211 driver complains, however it's working, both 2.4(hw_mode=g channel 11 in hostapd conf) and 5GHz(hw_mode=a channel 157) the end goal here is to dedicate device as repeater. My current setup will kill the Android zygote/led backlight at boot after wifi connects and It will run for over a day this way non stop with CPU governor being set to interactive every 15mins on a loop to prevent phone from sleeping.
Latest on github...
https://github.com/mrbox24/android_wifi_repeater
android-backup-project
The android backup project provides a set of scripts and tools to backup and/or restore applications installed on Android devices.
This is not adb backup which didn't work for my requirements as
it doesn't backup applications if those are requesting not to be backed up
it does not support restoring to different devices quite well
Google warns that it might be discontinued in future versions of Android
Motivation
I never was really happy about the possibilities to backup / restore Android devices. Especially when you are developing with (different) Android devices, you wish you were able to transfer "configurations" from one device to a different one. Or you would like to revert to an older version of an app or ...
The same is valid when you change your "main" device. The Google mechanism to set up a new device from another one or from a backup works quite well, but has almost the same deficits as adb backup: it doesn't restore all of your apps and data.
Also TWRP or other custom recovery implementation do not really help to get out of this situation. It's fine for creating a backup and restoring it later on the same device but there is no support for switching devices. In addition Android often changes architecture so that TWRP support for a specific device is not guaranteed and at the time of writing there is not even TWRP support for Android 10 yet.
There is another very interesting project related to backup and migration on XDA (https://forum.xda-developers.com/android/apps-games/app-migrate-custom-rom-migration-tool-t3862763). Unfortunately it relays on TWRP, too, and has still some issues to be solved.
Last but not least there is Titanium Backup which is available for a very long time now. It works quite well but the free version is very limited in functionality and provides only the possibility to store the backup on the device itself which is a kind of contradiction for a backup.
Some time ago I started already a similar project (https://github.com/AndDiSa/ART) to manage backup and restore of Android devices remotely. Unfortunately it never finished and due to architectural changes of Android it would need a complete workover so that I decided to start over from scratch
Precondition
adb installed and in execution path
Ubuntu Linux (other should work, too)
a rooted Android device (either by adbd unsecure or by e.g. Magisk / SuperSU)
Usage
Theses scripts kind of do that TiBackup is doing, but controlling the backup- and restore process remotely.
backup_apps.sh
./backup_apps.sh [--system-apps]
This script creates a backup of all user (and system) applications and their data. The backup files will be stored in a directory newly created named by the device and the current date.
restore_apps.sh
./restore_apps.sh [<directory_name>]
Ths script restores the apps and their data of a previous backup created by backup_apps.sh Either the directory is identified automatically by the device connected and the current date or you can pass a directory name as parameter. In that case all the apps and data found in the given directory will be restored.
If you want to restore only a part of them, please copy them into a different directory and give that directory as a parameter to the script.
Important
Keep in mind that restoring all apps may cause issues as some have unique IDs that end up causing problems if you use the same unique ID on different devices.
You also may want to consider only restoring apps that are missing from google play or apps that sadly decide to prevent backups of their (i.e. your) data.
full-backup.sh
./full-backup.sh [--data-backup][--no-data-backup][--media-backup][--no-media-backup][--image-backup][--no-image-backup]
This script does a full backup of (different parts of) the /data partition. There are several options to control the behaviour.
--data-backup will backup /data without /data/media and /data/mediadrm
--media-backup will backup /data/media and data/mediadrm
--image-backup will backup the whole partition which is mounted as /data as a 1:1 copy
As it cannot be guaranteed that during the backup process there will be no single modification on the /data partition it's quite common to get a checksum validation failure after the backup is finished.
full-restore.sh
./full-restore.sh [--data-backup][--no-data-backup][--media-backup][--no-media-backup][--image-backup][--no-image-backup] <directory_name>
This script is restoring data previously created by full-backup.sh It accepts the same parameters as full-backup.sh and takes in addition the name of the directory from where the backup shall be restored.
Important
Be very careful in unsing this method, especially when you are restoring /data It will overwrite everything on the /data partition and this may cause major issues up to the point that your device becomes no longer usable. In that case you probably need to do a full wipe to get it working again.
restore-single-partition.sh
./restore-single-partition.sh <path-to-image-to-be-restored> <name-of-the-partition-on-device>
This script is restoring a single partition image previously created by backup-partitions.sh back to the device. The image can be either in compressed or plain format. As parameters you need to give the path to the partition image and the name of the partition as is is configured on the device (see: /dev/block/by-name/<name>).
Important
This command is in experimental state and may be very dangerous as it can fully brick your device. Be very careful in using this method, you have been warned(!).
Downloads:
android-backup-project-v2.1.zip (md5: b0c71d602963696f7c372d3c2222ae75)
android-backup-project-v2.0.zip (md5: 14be02aa3e2a458ba7d7456dc152dad8)
android-backup-project-v1.6.zip (md5: 00f8dc8630b63fe1398ea61a436c2a0f)
android-backup-project-v1.5.zip (md5: 9cb1183d385f854c1f4b7858855cde58)
android-backup-project-v1.4.zip (md5: 9ad78f33a86e117a72058a336b34ab28)
android-backup-project-v1.3.1.zip (md5: 737a7d3483d98616f650d51eea033619)
android-backup-project-v1.3.zip (md5: 3bb11d4da02754a82f8614a53aacd4df)
android-backup-project-v1.2.zip (md5: 27f365663a97b3c7451938d24ffb7779)
android-backup-project-v1.1.zip (md5: c9806edb32b68109c2fc8e087927ac7a)
android-backup-project-v1.zip (md5: b754cf72fe10c89cdb8e4cb8e94f9e0a)
Special thanks
Credit goes to Raphael Moll who initiated a similar project some time ago and Marc Merlin who improved it to work with Android O. I took some ideas and inspiration from these projects and from my first trial I started years ago. The current implementation does not have much in common with neither of those versions.
Especially I would like to thank @topjohnwu for his great Magisk project and @osm0sis for his great collection of scripts and tools.
XDA:DevDB Information
Android Backup Project, Tool/Utility for all devices (see above for details)
Contributors
AndDiSa
Source Code: https://github.com/AndDiSa/android_backup_project/
Version Information
Status: Stable
Current Stable Version: V1.0
Stable Release Date: 2019-12-08
Created 2019-12-08
Last Updated 2020-01-19
Missing Permissions
Magisk Canary in combination with newer Android versions are restricting permissions even on rooted devices more and more. Due to that not all actions required to do a backup / restore are allowed any longer. To circumvent this restrictions there is a new Magisk Module available which enhances the permissions so that the backup / restore scripts are working.
If you are observing permission issues using the backup / restore scripts please install the AndroidBackupRestore Module in Magisk.
Edit: Latest Magisk Canary (>=20425) is working again without the need for additional changes
Reserved
I there and thanks for the work.
I tried to use your script and I am stuck at "Checking root access"
I am using Magisk. Shell is autorized in #Superuser.
In command line "su" is working see #
Here is my terminal screen
[email protected]~
$ bash full-backup.sh --data-backup --media-backup --image-backup
~/busybox-ndk ~
Déjà à jour.
~
Waiting for device...
Devices detected:
List of devices attached
da279704 device
Checking for root access...
[email protected]~
$ adb shell
gemini:/ $ su
gemini:/ #
Click to expand...
Click to collapse
@cocolol What Android version / device are you using? I did my tests with e.g. a Magisk rooted Pixel 3a on Android 10 without any issues.
Can you try from your command line the following:
adb shell su root -c whoami
and if this is blocked the following:
adb shell su root whoami
probably for your device / configuration
adb shell su whoami
is working
updated to version 1.1 as there was a quote mismatch in one of the scripts
Can you backup and restore just user apps?
str8str said:
Can you backup and restore just user apps?
Click to expand...
Click to collapse
If you use backup_apps.sh without passing the --system-apps option ... but on Android 10 most likely you will still backup (some) system apps. If there is need for that one could add a filter ...
@andisa thank you for your prompt reply and sorry for my late answer
I updated script to 1.1 and still be blocked.
Thus "adb shell su root -c whoami" is working answer is "root"
But other commands didn't work (see log below)
My aim is to jump from PE 9 to PE 10 on xiaomi MI5 and MI8 (dipper) with all my apps.
I came here from Migrate app forum which is not working for me since it is TWRP dependend.
[email protected] ~
$ bash full-backup.sh --data-backup --media-backup --image-backup
~/busybox-ndk ~
Déjà à jour.
~
Waiting for device...
Devices detected:
List of devices attached
da279704 device
Checking for root access...
[email protected] ~
$ adb shell su root -c whoami
root
[email protected] ~
$ adb shell su root whoami
[email protected] ~
$ adb shell su whoami
Click to expand...
Click to collapse
@cocolol ok, that's just as it should be, i.e. if you are rooted with Magisk
adb shell su root -c
prefix should work. Question is: why it is blocked within the script.
Edit: I expect that the script is blocked while checking a sane /data partition. Can you try to remove that part from the script? It doesn't harm and we can prove that my expectations are correct. It's sufficient the remove / comment out the checkForCleanData line.
I got an error: "Unrecognised architecture armv7l".
After adding that to line 82 in functions.sh (and switching flaky usb port) it went through.
Edit:
Device gohan (BQ Aquaris X5 plus)
OS LineageOS 14.1-20191115 (unofficial)
Host Manjaro, bash v5.0.11 (ah well, I had to install PV command)
Edit: fixed device name
Keule-T said:
I got an error: "Unrecognised architecture armv7l".
After adding that to line 82 in functions.sh (and switching flaky usb port) it went through.
Edit:
Device gohan (BQ Aquaris X5 Pro)
OS LineageOS 14.1-20191115 (unofficial)
Host Manjaro, bash v5.0.11 (ah well, I had to install PV command)
Click to expand...
Click to collapse
@Keule-T Thank you very much for the info. I'll add that this change into the next version.
Would be great if others could report issues / success, too, so we can improve the scripts to be able to run on (almost) every device.
AndDiSa said:
Would be great if others could report issues / success, too, so we can improve the scripts to be able to run on (almost) every device.
Click to expand...
Click to collapse
Well, that was a disturbing experience. I downloaded and un-zipped the v1.1 on Arch Linux, opened a terminal from within that extracted folder, and ran the command ./backup_apps.sh with no additional arguments.
The terminal reported this:
Checking for root access...
Determining architecture...
Pushing busybox to device...
busybox-ndk/busybox-arm64: 1 file pushed. 26.0 MB/s (1478208 bytes in 0.054s)
### Creating dir qcom_2019-12-21_QQ1C.191205.016.A1
~/Downloads/android-backup-project-v1.1/qcom_2019-12-21_QQ1C.191205.016.A1 ~/Downloads/android-backup-project-v1.1
Then the terminal listed about 70 lines of .apk files, and then disaster happened. These are the last few lines of what the terminal reported:
## Stop Runtime
## Pull apps
package:/data/app/com.touchtype.swiftkey-80xpz-vz5Aii7U6vdFVHnQ==/base.apk=com.touchtype.swiftkey
/data/app/com.touchtype.swiftkey-80xpz-vz5Aii7U6vdFVHnQ==/base.apk
/data/app/com.touchtype.swiftkey-80xpz-vz5Aii7U6vdFVHnQ==
com.touchtype.swiftkey
./backup_apps.sh: line 55: pv: command not found
Then after that, complete silence. I mean COMPLETE. My phone did a Force Close and powered off. Wouldn't reboot back into system, just looped. I had to do a a Nandroid restore to get back into my phone again. Not good....
OnePlus 3T rooted with Magisk 20.1 and running MSM Xtended XQ v4.0 Android Q rom.
I like the concept, but I think I'll wait until you get it a bit more developed before I do a beta test again... Maybe you can make use of the above information to further your project.
---------- Post added at 12:12 PM ---------- Previous post was at 12:11 PM ----------
Forgot to add that the only thing that made it into the directory your script created on my PC was a file named
app_com.touchtype.swiftkey.tar.gz with a filesize of 0 . That's all that was there.
@Oblias thank you very much for your report. I am very sorry for the mess but it's exactly those reports which help really a lot.
pv is a program which helps logging the communication between the device and the pc and I did not have in mind that it probably is not installed on a Unix system. I'll add a check for the presence of the important programs so that there is a stable basis before starting the scripts.
The normal behaviour of the script is as follows:
- shutdown runtime (to prevent modifications during the backup process)
- backup apps and data
- restart runtime
So a "warm" restart of the device is normal after the backup script finishes. I am only wondering why you had to restore a nandroid ...
AndDiSa said:
@Oblias thank you very much for your report. I am very sorry for the mess but it's exactly those reports which help really a lot.
pv is a program which helps logging the communication between the device and the pc and I did not have in mind that it probably is not installed on a Unix system. I'll add a check for the presence of the important programs so that there is a stable basis before starting the scripts.
The normal behaviour of the script is as follows:
- shutdown runtime (to prevent modifications during the backup process)
- backup apps and data
- restart runtime
So a "warm" restart of the device is normal after the backup script finishes. I am only wondering why you had to restore a nandroid ...
Click to expand...
Click to collapse
Well, you said in the OP that one of the things needed for these scripts to run was to have "Ubuntu Linux (other should work, too)" installed. Arch Linux and Ubuntu Linux both use the Linux kernel, so I thought that would be alright. Which leaves me a bit confused as to why you said that PV is not installed on a Unix system??? But anyways.......
As far as the normal warm restart is concerned, perhaps something got messed up in the command when the script halted?? I'm not all that good with scripts, so I don't know. Also, perhaps whatever made the phone do a poweroff instead of a reboot, also corrupted some system file? Just guessing, of course... But it's not biggie really. Anyone who tries anything like this had better have a backup on hand, just in case any sort of unpredictable event happens.
So, let me know about the PV command issue, and I'll give it another try, if you'd like.
---------- Post added at 01:27 PM ---------- Previous post was at 01:24 PM ----------
Also, as an added thought, when you get a chance, perhaps a small FAQ in the OP as to what the normal behavior is when running these scripts. For example, letting folks know that their phone will do a reboot on the script completion, and things of that nature.... Just a thought....
---------- Post added at 01:31 PM ---------- Previous post was at 01:27 PM ----------
UPDATE: I also just checked my repos, and pv is available to be installed, it just wasn't installed during a 'normal' install of Arch Linux (if there is such a thing).
So, if I install pv, are there any other commands I should check for to see if I have installed before trying the script again?
@Oblias beside those "normal" programs like git, tar, gzip, ... which are normally always installed there is no additional program needed. But nevertheless I agree that probably some additional compatibility checks should be done especially as there are different versions of the programs existing which take probably also different parameters ...
AndDiSa said:
@Oblias beside those "normal" programs like git, tar, gzip, ... which are normally always installed there is no additional program needed. But nevertheless I agree that probably some additional compatibility checks should be done especially as there are different versions of the programs existing which take probably also different parameters ...
Click to expand...
Click to collapse
Makes sense. Alright then, I'll wait until you check things out, and once you have posted back, I'll give it another try. I don't mind some beta testing..
AndDiSa said:
@cocolol
Edit: I expect that the script is blocked while checking a sane /data partition. Can you try to remove that part from the script? It doesn't harm and we can prove that my expectations are correct. It's sufficient the remove / comment out the checkForCleanData line.
Click to expand...
Click to collapse
After some investigation script was blocked on line 29 of functions.sh file on this condition "if [[ $($AMAGISK whoami) == "root" ]]"
I dont know why since when I replace this line with "$AMAGISK whoami" I get "root" as answer
Thus I replace "checkRootType" directly with "AS=$AMAGISK"
But I get the same with next function
Determining architecture...
Unrecognized architecture aarch64
Click to expand...
Click to collapse
Seems to be does'nt get the answers same as problem above
I am using cygwin maybe it come from this.
I replaced == with =~ and it works but for the case condition I found nothing
@cocolol I suppose cygwin has some issues with evaluating the expressions as aarch64 is supported. Probably also here you need to change the comparison ... If you know the type, you can also execute the code for the matching architecture.
Uploaded version 1.2 of the tool with the following improvements:
- support for armv7l architecture
- checking prerequisites needed by the tool, e.g. adb, pv, ...
Disable Tracking Services
This is a set of scripts that disables privacy invasive tracking components in all user installed apps (service/broadcast receiver/activity/content provider).
Root permission is required to run the scripts.
How it works:
UpdateExodus.sh downloads tracker names from Exodus Privacy.
DisableTracking.sh dumps a list of components of all installed apps, compare it against the list of trackers name, and disable those components (using the 'pm disable' command) if they match.
Download: Google Drive link
Check instructions.txt for details on how to use the scripts.
The latest version is 1.8.
You are recommended to backup your apps before running the scripts.
You are also recommended to do a full nandroid backup before running the scripts.
I hope you will find this useful
The script was originally based on the one here: https://forum.xda-developers.com/android/development/script-disable-fk-services-trackers-apps-t4074427
Instructions
Prerequisites:
Root permission is required.
You also need these extra binaries installed in a directory of $PATH (such as /system/xbin):
-aapt
-jq
You can download these binaries from the Termux app ( https://play.google.com/store/apps/details?id=com.termux )
You will also need busybox installed. Search for how to install it.
Method 1 (automatic) (experimental):
Run InstallReq.sh in Termux *as a normal user* (not as root)
i.e. run this command: bash /path/to/script/InstallReq.sh
It will check if aapt, jq, curl or sqlite3 are missing and automatically download them, then copy them to /system/xbin.
Method 2 (manual):
Run these commands in Termux:
pkg install aapt
pkg install jq
Copy them from /data/data/com.termux/files/usr/bin to a directory of $PATH
Then give them execution permission, e.g. "chmod +x /system/xbin/aapt"
To see what directories are in $PATH, run this command: "echo $PATH"
Note: if you've placed the binaries in /sbin, or you have installed Magisk,
the binaries will be deleted every reboot. You'll have to copy them again.
Note: aapt and jq must be downloaded from Termux and the Termux app must remain installed,
since the dependencies are stored in Termux's data folder.
------------------------------------------------------------
Usage:
-All scripts and txt files must be in the same directory as DisableTrackers.sh
You need a terminal emulator to run these scripts.
Personally I use "Terminal Emulator for Android" ( https://play.google.com/store/apps/details?id=jackpal.androidterm )
This terminal emulator lets you add launcher shortcuts. Quite convenient in my opinion.
Termux may also work, but I haven't tested it.
For first time usage, run UpdateExodus.sh first.
i.e. run this command: su -c sh /path/to/script/UpdateExodus.sh
If the script runs successfully, a MergedTrackers.txt will be created.
Then you can run DisableTrackers.sh AS SU to start disabling trackers. (root permission is required)
i.e. run this command: su -c sh /path/to/script/DisableTrackers.sh
By default, only third party apps are checked.
If you want to check for trackers in system apps as well, add 'sys' as the parameter.
i.e. run this command: su -c sh /path/to/script/DisableTrackers.sh sys
Not really recommended since it could theoretically mess up Google Play Services, though I can't confirm since I don't have it installed on my phone.
Note: if you've used version 1.4 or below and don't want the script to process all apps again, run CreateAppDB.sh.
It will create a ProcessedApps.db with records of all currently installed apps.
------------------------------------------------------------
Explanations:
UpdateExodus.sh will download a tracker list from Exodus Privacy, and merge it with CustomTrackers.txt (if it exists)
You can add your own trackers to CustomTrackers.txt. You may also write comments in this format: #(comment here)
DisableTracking.sh will dump a list of components (service/broadcast receiver/activity/content provider) of all installed apps,
compare it against the list of trackers in MergedTrackers.txt, and disable those components if they match.
It will save a list of already processed apps to a database file in the script's directory,
so they will not be processed twice when you run the script again (like maybe you've installed a new app and you want to disable its trackers by running the script).
Apps with trackers will be logged to AppsWithTrackers.txt.
exodustrackers.json is the raw list downloaded from Exodus Privacy.
MergedTrackers.txt contains the merged results of exodustrackers_extracted.txt and CustomTrackers.txt.
ProcessedApps.db contains a list of already processed apps. They will be skipped in the next run of DisableTrackers.sh.
whitelist.txt is a list of apps that will be skipped.
Update log
Version 1.5:
-DisableTracking.sh now uses a database to store processed app names and versions.
This means if an app was updated, it will check for trackers in the updated app again.
-If there was a deleted app, DisableTracking.sh will now delete its record from ProcessedApps.db automatically. No more manual deletion.
-Note: if you've used version 1.4 or below and don't want the script to process all apps again, run CreateAppDB.sh.
It will create a ProcessedApps.db with records of all currently installed apps.
Version 1.6:
-DisableTracking.sh now supports checking for trackers in system apps. Check the usage section for how to use.
-Improved logic of DisableTracking.sh to reduce problems. Thanks to hyborian for pointing them out.
-Added checks for root permission and presence of extra binaries.
-The new script 'InstallReq.sh' will automatically install missing extra binaries from Termux.
Check the prerequisites section for how to use.
-UpdateExodus.sh will no long generate exodustrackers_extracted.txt. The content is stored in memory instead of a file.
-MergeExodus.sh is no longer needed. The functions are...merged into UpdateExodus.sh.
Version 1.7:
-DisableTracking.sh now shows the app name as seen in the launcher when processing an app (will be empty if the app name is not in Engish)
-DisableTracking.sh will skip apps in whitelist.txt.
-UpdateExodus.sh now shows new tracker names that were not previously present in the old MergedTrackers.txt.
Version 1.8:
-Small fix in DisableTracking.sh that stops it from creating duplicated records in ProcessedApps.db
(in line 165, grep -qx is chnaged to grep -q)
If you've used version 1.6 or above, you are recommended to delete ProcessedApps.db located in the script's directory and run CreateAppDB.sh to regenerate a ProcessedApps.db, although the existing ProcessedApps.db with duplicated records may still work.
reserved 3
Thanks, fantastic tool
del
Hello I got a little issue with your tools ( it says to i need to verify my internet connection but I have wifi on. I have tried in mobile data still same ...
(When I open the web site manually it work but during the script execution I have this issue any other method?
Random-Tester said:
Hello I got a little issue with your tools ( it says to i need to verify my internet connection but I have wifi on. I have tried in mobile data still same ...
(When I open the web site manually it work but during the script execution I have this issue any other method?
Click to expand...
Click to collapse
Ok try running the script in attachment (remove .txt in the name), copy/screenshot the output and post it here.
steveglowplunk said:
Ok try running the script in attachment (remove .txt in the name), copy/screenshot the output and post it here.
Click to expand...
Click to collapse
Hi thnx for fast answers !! Unfortunately nothing new
I got this in terminal :
$ su
:/data/data/com.termux/files/home # /system/bin/UpdateExodus.sh
curl not found
wget not found
jq is a tracked alias for /sbin/jq
Checking connection with https://etip.exodus-privacy.eu.org/trackers/export
/system/bin/UpdateExodus.sh[26]: curl: inaccessible or not found
Network check status is empty
Unable to connect with https://etip.exodus-privacy.eu.org/trackers/export. Please check you internet connection.
No changes done.
Script will exit now.
Random-Tester said:
Hi thnx for fast answers !! Unfortunately nothing new
I got this in terminal :
$ su
:/data/data/com.termux/files/home # /system/bin/UpdateExodus.sh
curl not found
wget not found
jq is a tracked alias for /sbin/jq
Checking connection with https://etip.exodus-privacy.eu.org/trackers/export
/system/bin/UpdateExodus.sh[26]: curl: inaccessible or not found
Network check status is empty
Unable to connect with https://etip.exodus-privacy.eu.org/trackers/export. Please check you internet connection.
No changes done.
Script will exit now.
Click to expand...
Click to collapse
Yes, the purpose of the script was to find out missing binaries, and it says you're missing curl and wget. This could be because you don't have busybox installed (wget is included in busybox) and you're running stock rom (curl should be included in a custom rom such as LineageOS).
What you can do is download those binaries from Termux. (Use the command 'pkg install curl', same for wget), and copy them to /sbin, just like what you'd do with jq.
curl is used for checking connection status. wget is used for downloading the trackers json file.
After executing everything perfectly, I ran to a weird instance where everytime it scans an app, a command sqlite3: inaccessible or not found was shown. I was wondering if this shows because I performed disable f*ck services script first then this script? are they supposed to do the same? does that mean the apps I'm scanning right now already have their services/receivers disabled thats why disabletracking.sh can't detect them? thank you!
phantom146 said:
After executing everything perfectly, I ran to a weird instance where everytime it scans an app, a command sqlite3: inaccessible or not found was shown. I was wondering if this shows because I performed disable f*ck services script first then this script? are they supposed to do the same? does that mean the apps I'm scanning right now already have their services/receivers disabled thats why disabletracking.sh can't detect them? thank you!
Click to expand...
Click to collapse
That means sqlite3 is not installed (I thought it's supposed to be included in Android?). Verify it by running 'type sqlite3' in the terminal.
Maybe you can try downloading sqlite (without the 3) from Termux with the 'pkg install' command, then copy the downloaded binaries to /sbin (similar to what you'd do with jq). Or you can use version 1.4 for now.
sqlite3 is used for writing a database that stores processed apps names and versions, so that they will be skipped in the next run. The script will still disable tracking components that haven't been disabled already.
steveglowplunk said:
That means sqlite3 is not installed (I thought it's supposed to be included in Android?). Verify it by running 'type sqlite3' in the terminal.
Maybe you can try downloading sqlite (without the 3) from Termux with the 'pkg install' command, then copy the downloaded binaries to /sbin (similar to what you'd do with jq). Or you can use version 1.4 for now.
sqlite3 is used for writing a database that stores processed apps names and versions, so that they will be skipped in the next run. The script will still disable tracking components that haven't been disabled already.
Click to expand...
Click to collapse
This worked well. Thank you.
Do you plan on implementing something for sqlite, jq and aapt to stick on sbin for the next update?
Also, some of the apps I'm scanning shows:
Exception occured while executing:
java.lang.illegalArgumentException: component class bugsnag.com etc. Etc. Does not exist in bla bla app.
I assume exodus detected that a certain app has trackers and tries to disable it but since I used disablefohkservice script before this, I guess it was already disabled and disabletracking.sh can't find it now?
phantom146 said:
This worked well. Thank you.
Do you plan on implementing something for sqlite, jq and aapt to stick on sbin for the next update?
Also, some of the apps I'm scanning shows:
Exception occured while executing:
java.lang.illegalArgumentException: component class bugsnag.com etc. Etc. Does not exist in bla bla app.
I assume exodus detected that a certain app has trackers and tries to disable it but since I used disablefohkservice script before this, I guess it was already disabled and disabletracking.sh can't find it now?
Click to expand...
Click to collapse
The error you mentioned is normal. That component cannot be toggled.
Also, please try running the script attached in Termux as a normal user (root permission will be required half-way through the script) by running 'bash /path/to/script/InstallReq.sh' . It will check if aapt, jq, curl or sqlite3 are missing and automatically download them, then copy them to /system/xbin. (binaries there should retain across reboots). You are recommended to backup at least your system partition in a recovery. Although tested on my own phone and it shouldn't overwrite anything important in /system, I'm not responsible for any damage caused.
Edit: small change to the script to avoid problems with checking curl
Updated script to version 1.6. Check update log for details.
phantom146 said:
Also, some of the apps I'm scanning shows:
Exception occured while executing:
java.lang.illegalArgumentException: component class bugsnag.com etc. Etc. Does not exist in bla bla app.
Click to expand...
Click to collapse
I just added a 2> /dev/null to the disable line so I don't see it ...
---------- Post added at 10:10 AM ---------- Previous post was at 10:09 AM ----------
steveglowplunk said:
Updated script to version 1.6. Check update log for details.
Click to expand...
Click to collapse
Great, will try it ...
All good :good:, I just added code to delete old app versions from processed db, and to recreate DBFullList variable again after the sqlite changes.
hyborian said:
All good :good:, I just added code to delete old app versions from processed db, and to recreate DBFullList variable again after the sqlite changes.
Click to expand...
Click to collapse
Um... That doesn't sound necessary?
The version field in the database is updated after processing an updated app. There shouldn't be old app versions left behind?
Also, DBFullList doesn't need to be updated after processing an app. If an app's name doesn't exist in DBFullList, then it must be a new app. If it exists in DBFullList and was processed, then it must be an updated app. Not updating the DBFullList variable after processing each apps shouldn't affect the functions.
steveglowplunk said:
Um... That doesn't sound necessary?
The version field in the database is updated after processing an updated app. There shouldn't be old app versions left behind?
Also, DBFullList doesn't need to be updated after processing an app. If an app's name doesn't exist in DBFullList, then it must be a new app. If it exists in DBFullList and was processed, then it must be an updated app. Not updating the DBFullList variable after processing each apps shouldn't affect the functions.
Click to expand...
Click to collapse
Lol, you are right, I was looking at my old processed db which had apps with different versions. I should just have erased it and started fresh.
Anyway, it was nice practice for me. learned something while doing it when I couldn't grep the darn version number of Google Play Store (Version: 20.2.09-all [0] [PR] 311811225 (82020900) in DBFullList. I finally realized it had metacharacters [ ] so changed the grep from -qx to -qFx, and all ok now; otherwise, it will get processed again even if it was already done.
Thanks for the script, really great, beats manually doing it using MyAndroidPro or Blocker.
hyborian said:
Lol, you are right, I was looking at my old processed db which had apps with different versions. I should just have erased it and started fresh.
Anyway, it was nice practice for me. learned something while doing it when I couldn't grep the darn version number of Google Play Store (Version: 20.2.09-all [0] [PR] 311811225 (82020900) in DBFullList. I finally realized it had metacharacters [ ] so changed the grep from -qx to -qFx, and all ok now; otherwise, it will get processed again even if it was already done.
Thanks for the script, really great, beats manually doing it using MyAndroidPro or Blocker.
Click to expand...
Click to collapse
Ah I see. I'll add that -F option in the next update.