What is OpenPDroid?
OpenPDroid is a set of modifications to the Android framework and libraries which allows fine-tuning of the data which applications are able to retrieve about your device, your account, your messages, and more. Specifically, it is a Privacy service provider (using the PDroid 1.51 interface) forked from CollegeDev's PDroid 2.0, which is itself an extension of Syvat's PDroid.
A word to the wise
I'm just adding this for those who are just looking at OpenPDroid, or are trying to work out how to get involved.
Mateor, wbedard and I are currently in the process of moving repositories etc to a github 'organisation' in order to make it much easier for users to know which are the latest patches, where to report issues, etc. I expect this will be done soon (in the next day or so), and I'll post more information then.
Thanks for your suggestions on ways of managing repositories, patches, etc.
What does it do?
When configured using either FFU5y's PDroid Manager or CollegeDev's PDroid 2.0 App (up to v1.52), OpenPDroid intervenes in API interactions by apps (e.g. when an app attempts to retrieve your location, phone number, or contacts) and provides either real data, empty data, fake data, or random data, depending on the user setting. Unlike apps which actually remove permissions, OpenPDroid does not actually modify the permissions that an app has and so is much less likely to cause the app to crash as a result of data access being denied.
Note that CollegeDev's PDroid 2.0 App does not officially support OpenPDroid, so please don't contact CollegeDev for support if you're having problems with PDroid 2.0 App with OpenPDroid. You can post here, and we may be able to help you.
PDroid Manager does officially support OpenPDroid.
The complete list of data and functions controlled by OpenPDroid can be found at the end of this post, in What (specifically) can I control with OpenPDroid?, but at present it is identical to that offered by PDroid 2.0
For a list of issues which have been identified, please see below the Known Issues.
What makes OpenPDroid different to CollegeDev's PDroid 2.0 or Syvat's PDroid
There are a few differentiators, some technical, and some are philosophical or pragmatic. Both are important
Technical
Two major security issues are fixed, which allow apps to bypass many or all PDroid or PDroid 2.0 settings. (We are in the process of notifying CollegeDev of these issues, and sample code will not be released for at least a week to give him a chance to fix the problems in PDroid 2.0. Fixes have been added to PDroid in auto-patcher, so if you are using PDroid please consider re-patching your ROM).
It is available for Android 4.2.1
Database access threading has been rewritten, as the implementation actually caused bottlenecks and didn't protect from simultaneous read-writes (which is generally the goal of this type of code)
A bug which was causing 'null pointer exceptions' to occur (but not a crash) is fixed.
Other code tidy-up or restructuring which doesn't change functionality
Non-technical
The intention of OpenPDroid is to have a PDroid version which is developed in the open, and which welcomes (although doesn't necessarily always adopt) user input - especially code. This means that the source for OpenPDroid versions in progress is on a public repository (Github at the moment), that changes are put into the Github as they are made, and that other people can contribute code to it. We also hope that by having engaging people more in the development process, there will be more eyes trying to understand the code, and so security-related issues will be identified and address more efficiently.
We also want to make the discussion/decision making processes for what is in and what is out transparent, so that users can understand why some things are included and others are not, and actually influence the decision-making process.
How do I install it?
There are two 'supported' ways to install OpenPDroid:
mateor's auto-patcher (available for OS X, Linux and Windows) is the recommended way if you are not familiar with building from source, and have a supported ROM (CM10, CM10.1, AOKP Jellybean 4.1.2, AOKP Jellybean 4.2.1, AOSP 4.1.2, and AOSP 4.2.1).
Patch and build a ROM from source (see How do I compile a ROM with OpenPDroid). Patches are provided for: CM10, CM10.1, AOKP jb, AOKP jb-mr1, AOSP 4.1.2, AOSP 4.2.1
If you are using a stock ROM, then someone may have implemented OpenPDroid into that ROM - and if they have, they deserve huge credit because it is a difficult and unpleasant job.
Currently we are not aware of any ROMs which include OpenPDroid: if you are aware of one, please notify us and we'll list it here.
There are several reasons your preferred ROM may not currently be supported. First, adding support for a ROM in auto-patcher requires building the ROM with the OpenPDroid patches. We are only able to do this if the source is available for the ROM. (Note: strictly speaking, it is possible to build patches for ROMs where source is not available, as for stock ROMs. However, for non-stock ROMs the rate of change makes this fairly unmaintainable).
Second, there are a great many ROMs out there and it is not possible for us to build for all these ROMs. However, if you are able to patch source and build, then you may be able to build a version of your preferred ROM with OpenPDroid support - and even better, contribute patches for your preferred ROM to auto-patcher. For more details about contributing patches to auto-patcher, please contact Mateor.
How do I get the source?
The source is available as patches, or in complete form, from the OpenPDroid repositories
The source-code patches from the OpenPDroidPatches repository: . Note that there are two branches: 4.2.1 and 4.1.2. These source patches presently work against numerous roms (including CM 10/10.1, AOKP 4.1.2/4.2.1, AOSP 4.1.2/4.2.1, and potentially others)
[*] Complete repositories for the modified sections of the system are likewise located in the [URL=https://github.com/OpenPDroid/]OpenPDroid github. There are several repositories and branches; see the section 'What are all these repos and branches?' for more details about the content of the github.
What is the licence?
PDroid was under under the GPL v3 licence (as shown on the Google Code page), and as a result all derivative works (e.g. OpenPDroid) are likewise under the GPL v3 (or later) licence. Thus, OpenPDroid is under the GPL v3 licence, with no additional special conditions. The only way a PDroid derivative can be under any other licence is if the author contacts Syvat, gets him to agree provide PDroid under another licence, and then the author extends it.
Who's work is it?
PDroid was original developed by Syvat (with contributions and ports from others such as Pastime1971 and Mateor), and then CollegeDev extended it to created PDroid 2.0 (previously called PDroid Addon, then PDroid Extension). PDroid 2.0 has been forked by FFU5y, Mateor and wbedard to create OpenPDroid. Thus, OpenPDroid is the product of the work of many.
The following list includes all the contributors I know of for any of these version of the core (in alphabetical order):
CollegeDev (PDroid 2.0)
FFU5y (OpenPDroid)
Mateor (Porting PDroid to 4.x, porting PDroid 2.0 to 4.1.2, OpenPDroid)
Pastime1971 (Porting PDroid to 4.x, porting PDroid 2.0 to 4.1.2)
Syvat (PDroid)
Wbedard (OpenPDroid)
How do I report issues?
The best way to report an issue is to lodge it in github, although you may want to also post in this thread to let others know that the issue exists (and has been lodged). Please ensure you provide details of the issue you are experiencing (i.e. under what circumstances does it happen, what do you observe, etc) and please include a logcat: see how to get a logcat.
If you don't have a Github account, you can post the issue here (and note that you haven't lodged it in Github) so one of the team can add the issue to Github.
I want to contribute
Great! The whole point of OpenPDroid is to encourage community contribution in the form of ideas, testing, code, etc.
Presently, there isn't really a lot (read: any) documentation about how OpenPDroid works. However, this will be forthcoming, as will an API specification for the Privacy service, which will allow you to write your own management application (like PDroid Manager or the PDroid 2.0 App) if you so desire.
Right now, these are the particular areas of the service which would benefit from developer/code contributions.
Transient caching of privacy settings read from the database, ideally with destruction of cache entries when they are too old or memory conditions are low
Refining the locking code to ensure that reads are not unduly held up by write locks (note: this isn't a problem *yet*, but will be with the inclusion of batch support. More details soon).
Other performance optimisations
Adding good quality code comments - mainly function descriptions, etc
Back-porting of OpenPDroid to pre-4.1.2, so users currently only able to use PDroid can move to OpenPDroid and get the new features.
There are many other things you can work on, including adding whole new options (i.e. security features). If you are planning on creating new security features, however, please lodge the enhancement in github first so it can be discussed. Not every feature which seems like a good idea will be a good fit for OpenPDroid, and it may save some time if a prospective feature can be refined before it is fully coded.
How do I compile a ROM with OpenPDroid
To build a ROM with OpenPDroid, you will need to patch the source code using the provided patches, and then build as you normally would.
To patch the source:
Download the most relevant of the source code patch packages attached to this post
Extract the patch files from the package
On the command line, go to the root of your Android source code: it will contain folders such as 'build', 'frameworks', 'vendor', etc.
Patch the source code with each patch file using the following command
Code:
patch -p1 < <patch_filename>
You can then build your ROM as you would normally.
If you have previously built the ROM in the folder you are using, or you have previously built PDroid 2.0, Syvat's original PDroid, etc it can be helpful to delete the following directories:
out/target/common/obj/JAVA_LIBRARIES/framework_intermediates
out/target/common/obj/JAVA_LIBRARIES/framework2_intermediates
out/target/common/obj/JAVA_LIBRARIES/telephony-common_intermediates
Not all of these directories will exist for all ROMs, so if you try to delete one and it is missing that is not cause for concern.
What (specifically) can I control with OpenPDroid?
Currently, access to the following data and functions can be controlled (note this list exactly matches PDroid 2.0):
Code:
[b]Data access:[/b]
Device ID (IMEI/MEID/ESN)
Phone/voicemail number
Sim Card Serial (ICCID)
Subscriber ID (IMSI)
Source number of incoming calls
Destination number of outgoing calls
Network location
GPS location
Authentication credentials
List of accounts, including identifiers (e.g. gmail e-mail address)
Contact list and details
Call log
Calendar
Read and send SMS
Read and send MMS
Record audio (not phone calls)
Camera
Browser bookmarks and history
System logs
Wifi Info, such as current access point, IP address
Network information (detailed network information such as connection state, IP address, etc)
Android ID (a unique installation ID for your Android installation)
SIM information (your phone operator and country)
Restrict access to iptables command
ICC Access (reading or writing SMS on the SIM or R-UIM)
[b]Actions:[/b]
Make phone calls
Start on Boot (or more accurately, get notified when the boot is complete)
Switch Wifi State (turn wifi on and off, change access points)
Switch network state (turn off or on mobile data)
[b]Other[/b]
Force online state (always tell the app that the device is online: only needed if denying Wifi info and/or Network info)
Known Issues
Thanks to those users who have posted bug details, we now have a list of known issues. I have tried to indicate which will be fixed in the next release, but it is tentative. These issues can also be found (and discussed) on the github issues list
Requests for tower location are always blocked (ETA unknown)
Device lags when GPS is active (ETA: next release; experimental fix in 'devel' branch of 4.2.1-based ROMs: same fix as the below camera lag issue)
Camera lags when OpenPDroid is installed (ETA: next release; experimental fix in 'devel' branch of 4.2.1-based ROMs)
What are all these repos and branches?
In the github account containing the OpenPDroid repositories, there are numerous repositories. Many of these are OpenPdroid related: but not quite all of them. In addition, not all of them are relevant to each ROM.
The main repositories to be aware of are:
platform_frameworks_base
This contains the framework/base/* code, which constitutes the most important guts of OpenPDroid in 4.1.2 and 4.2.1. Other projects sometimes use the name android_frameworks_base for this.
platform_frameworks_opt_telephony
This contains the framework/opt/telephony code, which was split out from framework/base/* in 4.2.1.
platform_build
This contains the build code, mostly makefiles, and is present in 4.1.2 and 4.2.1.
platform_libcore
This contains core operational code outside of the framework (e.g. executing shell tasks, etc).
platform_packages_apps_mms
This contains the code for the Mms app, for 4.1.2 and 4.2.1.
platform_packages_apps_mms
This contains the code for the Mms app, which is modified to affect the ability of apps to send and receive MMS.
platform_packages_apps_videoeditor
This contais the code for the VideoEditor app. The only change here is including 'framework2', which is only required for AOSP 4.1.2.
platform_system_core
This contais a range of important core code, and is modified only for AOSP 4.1.2 to include 'framework2.jar' in the paths included at boot time.
The branches used include two key naming components. First, the rom:
e.g.
cm-10.1* = CyanogenMod 10.1
cm-jellybean* = CyanogenMod 10, cutting edge
cm-jellybean-stable* = CyanogenMod 10, stable release
aokp-jb-mr1* = AOKP 4.2.1-based
aokp-jb* (without the mr1) = AOKP 4.1.2-based
jb-mr0* = AOSP 4.1.2
jb-mr1* = AOSP 4.2.1
pa-jellybean* = ParanoidAndroid (currently buggy)
then the OpenPDroid version:
-openpdroid = the current stable release of OpenPDroid
-openpdroid-devel = the current development line of OpenPDroid
For development, the jb-mr1-release-openpdroid-devel is where I do my initial development, and I think expand it to be on the other -devel branches once I've tested it a bit.
FAQ
How do I include PDroid Manager in the ROM I am building so it can update from Google Play: see here
How do I include PDroid Manager in the ROM I am building so it is compiled during build: see here: TamCore provided a fix for the problem he mentioned, and it has been merged.
How do I build using the bleeding-edge (devel) code: see here, although there is a bit more to it than that. I'll provide extra details when time permits, but the important stuff is in this thread.
I have a question or issue not covered in this post
We'll be adding more to this post in the next day or two. Please be patient =)
Note to mods
I wasn't sure if this belonged in this forum, or in 'Apps and Games' but since it is not trivial to install I figured this was a safer bet.
reserved
Thank you!!!! I'll test
Sent from my LG-P920 using xda app-developers app
Announcement: OpenPDroid build patches, as well as the entire history of the PDroid framework, have been rebuilt and restructured and pushed to our github page. There are build patches for all Android versions from 2.3 to current. You can also take a look at the commit history of the original PDroid framework by svyat, from initial commit to the point where he halted development, 1.32 for gingerbread.
Build patches are here.
PDroid project history is here.
OpenPdroid for Android 4.2.1/4.2.2
These are the current build patches for Android 4.2.1 and 4.2.2 and should generally reflect the current state of the jb-mr1-release-openpdroid branch. The latest commist are not always reflected in the build patches, bleeding-edge development is in the -devel branches.
We realize that the patch/branch situation is not perfect, it is work in progress.
These build patches should work for most roms. As of their creation (1/17/13) they applied cleanly to AOKP, AOSP, Cyanogenmod, ParanoidAndroid and others.
The current best advice as to how to apply/remove them is below. Some easier application methods are being discussed.
To Apply
Code:
cd ~/android/system/build; git checkout -b pdroid; patch -p1 < ~/openpdroid_4.2.1_build.patch
cd ~/android/system/libcore; git checkout -b pdroid; patch -p1 < openpdroid_4.2.1_libcore.patch
cd ~/android/system/packages/apps/Mms; git checkout -b pdroid; patch -p1 < openpdroid_4.2.1_Mms.patch
cd ~/android/system/frameworks/base; git checkout -b pdroid; patch -p1 < openpdroid_4.2.1_frameworks_base.patch
cd ~/android/system/frameworks/opt/telephony; git checkout -b pdroid; patch -p1 < ~/openpdroid_4.2.1_frameworks_opt_telephony.patch
cd ~/android/system; . build/envsetup.sh; brunch <DEVICE_TARGET>
To Remove
Code:
cd ~/android/system
rm -rf out/target/common/obj/JAVA_LIBRARIES/framework_intermediates \
out/host/common/obj/JAVA_LIBRARIES/layoutlib_intermediates \
out/target/common/obj/JAVA_LIBRARIES/telephony-common_intermediates \
out/target/common/obj/APPS/Mms_intermediates
[OR]
Code:
cd ~/android; make clobber
and to reset your source tree**
Code:
cd ~/android/system/build; git checkout . ; git clean -df
cd ~/android/system/libcore; git checkout . ; git clean -df
cd ~/android/system/frameworks/base; git checkout . ; git clean -df
cd ~/android/system/frameworks/opt/telephony; git checkout . ; git clean -df
cd ~/android/system/packages/apps/Mms; git checkout . ; git clean -df
cd ~/android/system; repo abandon pdroid
If OpenPDroid intrigues you, yet the thought of applying all these patches gives you pause, there is a tool that can attempt to patch the framework of an already compiled rom called The Auto-Patcher.
OpenPDroid is an open source fork of the PDroid framework, as written by Svyat and expanded by CollegeDev. We are always looking for contributors, come see us on XDA or submit a pull request.
I'm glad to see there's a more open PDroid project happening. I'm not sure what I'll be able to contribute, but I'll find something to help out with.
Looks like I'm going to have to spend some quality time with the CM10.1 codebase tonight.
Decimalman:
If you look in my github (http://github.com/wsot) you will find there are forks of all the modified parts of Android, and these include forks from cm-10.1 (branches called cm-10.1-openpdroid). You can pull directly from these repositories into your existing repositories if you would prefer to do that, rather than apply patches to the source.
For cm-10.1, the repositories you'll care about are:
* platform_frameworks_base
* platform_libcore
* platform_build
* platform_packages_apps_mms
* platform_opt_telephony_common
Each of these has a branch for cm-10.1-openpdroid with the changes included.
This is great work guys, thanks!
I will test it and if everything works, I will release the next extended CM10/FXP ROM with it.
Is there already a solution for including PDroid Manager and still being able to get updates from the market?
I have one question, did you fix this bug too?
Is there any plan to cooperate with CollegeDev?
It would be a pity if development for PDroid would be fragmented.
M66B said:
I will test it and if everything works, I will release the next extended CM10/FXP ROM with it.
Click to expand...
Click to collapse
Excellent!
M66B said:
Is there already a solution for including PDroid Manager and still being able to get updates from the market?
Click to expand...
Click to collapse
If you include PDroid Manager as-is from the PDroid Manager thread, then it has the same signature as the one in the market. Because of that, it should be able to update from the market.
M66B said:
I have one question, did you fix this bug too?
Click to expand...
Click to collapse
No, thanks for pointing that out. I've updated the source and pushed it to Github with that change. It'll be in the next release, which I don't imagine will be all that far away.
M66B said:
Is there any plan to cooperate with CollegeDev?
It would be a pity if development for PDroid would be fragmented.
Click to expand...
Click to collapse
We have not discussed with CollegeDev any kind of collaboration at this stage. Because both OpenPDroid and PDroid 2.0 core are GPL licensed, I imagine there will be code migration in both directions as things are fixed and improved.
The main reason that OpenPDroid has appeared (rather than us just making a 4.2.1 port of PDroid 2.0) is because long periods of development of PDroid 2.0 went on without the code being available, and so there was really no scope to collaborate.
The reason we are planning to produce an OpenPDroid API document is essentially so that there is a clear (and shared) idea of what can be expected from a minimal PDroid implementation, which both PDroid 2.0 and OpenPDroid cover at this stage. That way even if additional features are added (or someone creates a whole new PDroid implementation from scratch) there can be a shared interface to allow interoperability of management apps (like PDroid Manager).
I think there will be some 'fragmentation', but our hope is that there can be a common API between the versions that users will not have to suffer as a result, and can continue to use their preferred management app, and so forth.
Whether than will happen, I can't say yet.
Thank you! That's what I've been waiting for. I'll integrate this in OpenSensation2 asap
FFU5y said:
If you include PDroid Manager as-is from the PDroid Manager thread, then it has the same signature as the one in the market. Because of that, it should be able to update from the market.
Click to expand...
Click to collapse
Good! Could you make a 'latest' URL available somewhere (github?), so I can integrate it in my build process?
M66B said:
Good! Could you make a 'latest' URL available somewhere (github?), so I can integrate it in my build process?
Click to expand...
Click to collapse
Just added https://github.com/wsot/pdroid_manager_build - the sole purpose of which is now to have the latest PDroid Manager binary.
You will be able to get the file from: https://github.com/wsot/pdroid_manager_build/blob/master/PDroid_Manager_latest.apk?raw=true
FFU5y said:
Just added https://github.com/wsot/pdroid_manager_build - the sole purpose of which is now to have the latest PDroid Manager binary.
You will be able to get the file from: https://github.com/wsot/pdroid_manager_build/blob/master/PDroid_Manager_latest.apk?raw=true
Click to expand...
Click to collapse
Thanks, I will integrate this tonight in my build process and I will document here how I did that, so other developers can maybe do the same.
M66B said:
Thanks, I will integrate this tonight in my build process and I will document here how I did that, so other developers can maybe do the same.
Click to expand...
Click to collapse
That would be excellent! Let us know how it goes.
Thanks for improving pdroid!
Great work :good:
but the Problem I got is that I want to test it on my Nexus 4 with ParanoidAndroid...
when will it be possible ?
because i feel a little bit naked without pdroid and i don´t like to use LBE
seety said:
Thanks for improving pdroid!
Click to expand...
Click to collapse
Cheers. We're happy to be able to expand on the great work done by others.
seety said:
... I want to test it on my Nexus 4 with ParanoidAndroid...
when will it be possible ? ...
Click to expand...
Click to collapse
If you haven't already, I suggest you try to patch with auto_patcher: I believe that quite a lot of the unofficial PA builds will actually patch even though we haven't specifically compiled for them.
While we were preparing for release, the source for ParanoidAndroid was not available, which is why there are not specific patches for it. The source has since been made available, so I expect we will have specific patches for PA integrated into auto_patcher quite soon. Exactly how soon is a bit hard to say, but we are certainly working on it.
Hi
I tried to find PDroid Manager in Play Store but can't find it.
Did a Web install from Play Store & it says my device is not compatible with the app.
I'm on Samsung Galaxy S3 i9300.
Is it really incompatible? If no where can I get the app?
Thanks.
Edit: lol Found the app in OP
Ignore me
Thanks for the good work, I'm test compiling to my build now.
temasek said:
Hi
I tried to find PDroid Manager in Play Store but can't find it.
Did a Web install from Play Store & it says my device is not compatible with the app.
Click to expand...
Click to collapse
Actually, it is a really valid question. I haven't updated the PDroid Manager in Google Play to be marked as supporting 4.2.1 yet. This will be done within the next 24 hrs, and you'll then be able to update from Google Play as new releases come out.
temasek said:
Thanks for the good work, I'm test compiling to my build now.
Click to expand...
Click to collapse
I hope it all goes smoothly, but if not let us know.
FFU5y said:
Cheers. We're happy to be able to expand on the great work done by others.
If you haven't already, I suggest you try to patch with auto_patcher: I believe that quite a lot of the unofficial PA builds will actually patch even though we haven't specifically compiled for them.
While we were preparing for release, the source for ParanoidAndroid was not available, which is why there are not specific patches for it. The source has since been made available, so I expect we will have specific patches for PA integrated into auto_patcher quite soon. Exactly how soon is a bit hard to say, but we are certainly working on it.
Click to expand...
Click to collapse
Hey thanks for the quick answer!
I tired the auto_patcher several times sadly it doesn't work...
maybe these logs are interesting for you:
Try to patch with CM Patches / With AOSP Patches
Perhabs it work if you try to patch this rom, but i am not angry if you got no time for this.
thanks in advance
seety said:
Hey thanks for the quick answer!
I tired the auto_patcher several times sadly it doesn't work...
maybe these logs are interesting for you:
Try to patch with CM Patches / With AOSP Patches
Perhabs it work if you try to patch this rom, but i am not angry if you got no time for this.
thanks in advance
Click to expand...
Click to collapse
Thanks for the details. We will be prepping patches for PA as a high priority so don't worry, you shouldn't be waiting long
I've compiled successful OpenPDroid and booted up fine. So far so good.
I'm letting my users of CM10.1 i9300 Unofficial BUILD to try your wonderful work.
This is my changelog for my CM10.1 Unofficial build V34 for i9300
http://forum.xda-developers.com/showpost.php?p=36697391&postcount=6919
Again, thanks for your good work.
Introducing Eva GCC Toolchain
What is GCC?
The GNU Compiler Collection (GCC) is an optimizing compiler produced by the GNU Project supporting various programming languages, hardware architectures and operating systems. The Free Software Foundation (FSF) distributes GCC as free software under the GNU General Public License (GNU GPL). Major corporate contributors to GCC include Red Hat, IBM, SUSE, ARM, Google and Intel. GCC is a key component of the GNU toolchain and the standard compiler for most projects related to GNU and the Linux kernel. With ~15 million lines of code in 2019, GCC is one of the biggest open source programs in existence. It has played an important role in the growth of free software, as both a tool and an example.
Source: Wikipedia
Click to expand...
Click to collapse
What is LLVM Clang?
Clang is a compiler front end for the C, C++, Objective-C and Objective-C++ programming languages, as well as the OpenMP, OpenCL, RenderScript, CUDA and HIP frameworks. It uses the LLVM compiler infrastructure as its back end and has been part of the LLVM release cycle since LLVM 2.6.
It is designed to act as a drop-in replacement for the GNU Compiler Collection (GCC), supporting most of its compilation flags and unofficial language extensions. Its contributors include Apple, Microsoft, Google, ARM, Sony, Intel and Advanced Micro Devices (AMD). It is open-source software, with source code released under the University of Illinois/NCSA License, a permissive free software licence. Since v9.0.0, it was relicensed to the Apache License 2.0 with LLVM Exceptions.
Source: Wikipedia
Click to expand...
Click to collapse
IntroductionAndroid as a whole has now fully switched to LLVM Clang for both their Platform (AOSP) and Kernels. In fact Pixel Phones have been shipping with Clang built and optimised kernels since 2018! But are there any improvements with using clang over GCC. I'd say yes, because the GCC that AOSP used was ancient (GCC 4.9). Also LLVM Clang has proven to be faster in compilation than GCC. But is this speed worth the improvement in Kernels? Let's answer that question with EvaGCC Toolchain.
How is my toolchain different from Android GCC?As I've mentioned earlier, AOSP GCC is ancient (version 4.9) in terms of the present stable GCC release (10.2.x). EvaGCC is compiled straight from the Master branch, making it a Bleeding Edge C Compiler. It is built with LTO and disabled a lot of feature bloat that are unneccesary for kernel building to yield a very small binary size. To list the features:
Bare Metal GCC (This does not depend on the standard libc)
Built straight from the GNU GCC Master branch
Built with LTO and O3 optimisations
Disabled documentation (Smaller size of the toolchain)
Disabled decimal float, libffi, libmudflap, libquadmath, libstdcxx-pch, native language support
Statically linked GCC
Integrated LLVM LLD (For faster linking, optional)
Built twice weekly (Thanks to Github Actions!)
Where can I find these toolchains?Well you have two options, you can compile them yourself (by using my script) or you can download precompiled binaries!
To compile this toolchain by yourself (although I recommend that you use precompiled binaries, to avoid the hassle and time to compile the toolchain itself), to use my script to compile your toolchain, it has everything preconfigured for GCC setup and cloning. Although you'll have to setup your system for building GCC, you can refer to my README for system setup.
Note:
To obtain precompiled binaries (Highly recommended), head over to these links (according to your architechture and liking):
ARM Git Repository with Precompiled Binaries
ARM precompiled binaries in a compressed zip (direct download)
AARCH64 (ARM64) Git Repository with Precompiled Binaries
AARCH64 (ARM64) precompiled binaries in a compressed zip (direct download)
Note: If you're doing a git clone, use --depth=1 to avoid heavy transfers, because the repositories are bound get bigger with subsequent updates.
How do I use these toolchains for compiling my kernelYou can either append the toolchain dir into your PATH variable, or you can just pass it along with make with setting your CROSS_COMPILE argument. I usually use the latter one.
Since this is a bare metal compiler, the prefix differs from the normal AOSP or Linux GNU GCC. the prefix is:
Bash:
# Pass this to your CROSS_COMPILE argument if you have appended toolchain to your PATH
## for AARCH64 or ARM64
$ make CROSS_COMPILE=aarch64-elf- ... # "..." indicates rest of your args
## for ARM
$ make CROSS_COMPILE=arm-eabi- ...
# Passing to make when you haven't appended to PATH
## for AARCH64 or ARM64
$ make CROSS_COMPILE=<path to toolchain>/bin/aarch64-elf- ...
## for ARM
$ make CROSS_COMPILE=path to toolchain>/bin/arm-eabi- ...
Sources:
Everything here is OSS, including the script, my ci automation script.
GCC: https://gcc.gnu.org/git/?p=gcc.git or https://git.linaro.org/toolchain/gcc.git
Binutils: https://sourceware.org/git/binutils-gdb.git
GCC Build script: https://github.com/mvaisakh/gcc-build
LLVM (Used for LLD): https://github.com/llvm/llvm-project
GCC Version: 13.x
Binutils Version: 2.36.x
LLD Version: 16.x
Telegram Channel:
Eva GCC
Bleeding Edge Bare Metal GCC, primarily targeting Android kernels.
t.me
NOTE: According to SultanXDA, and I quote
GCC 10 updated its interprocedural optimizer's logic to have it make
more conservative inlining decisions, resulting in worse syscall and
hackbench performance compared to GCC 9.
Click to expand...
Click to collapse
This can be fixed with a patch that he himself provided:
gcc-inline-regressions-2.patch
GitHub Gist: instantly share code, notes, and snippets.
gist.github.com
and if that did not work for you, try applying this patch
gcc-inline-regressions-2.patch
GitHub Gist: instantly share code, notes, and snippets.
gist.github.com
Update 30-Jan-2021
GCC+LLD has been merged into the main branch of the build script. Now GCC+LLD would be updated twice every week (on Sundays and Thurdays). This wasn't done before as it was under testing, and so far it only fails under LTO kernel compilation (Due to lack of GCC Plugin for LLD and vice versa).
Update 07-April-2021
lld-integration trunk has been merged into gcc-master branch. For those who use LLD, should switch to gcc-master as the lld-integration branch is now deprecated and will be removed soon.
The size difference between the two isn't much (~86mb vs 125mb), so it makes sense to have a single branch for everything.
I recommend to use zipped archive toolchains or if you use git operations to clone toolchain binaries, I recommend using --depth=1 while cloning the toolchain to avoid huge binary size cloning.
Update 27-April-2021
GCC Version has been bumped to 12.x
Eva GCC now ships with
GCC: 12.x
LLD: 13.x
BinUtils: 2.36.x
Update 26-June-2021
Toolchain binaries have been stripped off of debugging and hence are much smaller than before, ~90MB shaved off!
Shallow clones shall be much faster than before!
Update 24-Nov-2021
LLD has been bumped to version 14.x
GCC is still on 12.x
Update 1-May-2022
GCC has been bumped to version 13.x
LLD is at 15.x
Pro Vaisakh
Yes yes super pro Vaisakh
Nice
It's kang time
Let me try great work
Oh pro iz here
Going to use it soon
god level pro work
Amazing job at collecting data and optimising the toolchain, looking forward to using this as default in my kernel builds!
Keep up the great work dude
great work sar, tysm!
A small update!
GCC+LLD has been merged into the main branch of the build script. Now GCC+LLD would be updated twice every week (on Sundays and Thurdays). This wasn't done before as it was under testing, and so far it only fails under LTO kernel compilation (Due to lack of GCC Plugin for LLD and vice versa).
If anyone faces any issues with the toolchain, please do let me know. I will try to investigate the issue and check accordingly if it's a toolchain issue or a kernel side issue.
Because being a cron built toolchain, it's necessary for people to report bugs as soon as possible.
I still monitor on my end, but it's always good to have a helping hand
Thanks @m_vaisakh,
Kernel 3.18.140 compiled with https://github.com/mvaisakh/gcc-arm64.git -b gcc-master works fine on Oreo. No errors thru whole process.
adeii said:
Thanks @m_vaisakh,
Kernel 3.18.140 compiled with https://github.com/mvaisakh/gcc-arm64.git -b gcc-master works fine on Oreo. No errors thru whole process.
Click to expand...
Click to collapse
How is the performance?
@nift4 also uses 3.18.140 and has it has improved everything in his case.
m_vaisakh said:
How is the performance?
@nift4 also uses 3.18.140 and has it has improved everything in his case.
Click to expand...
Click to collapse
*3.18.124 sadly
m_vaisakh said:
How is the performance?
Click to expand...
Click to collapse
Lets see what Antutu Benchmaker v8.4.5 said:
Same kernel source, same device, connected via USB, 10 minutes after boot:
With GCC 4.9: score 71192, HTML5 score: 13087
With EVA-GCC: ... 72392, HTML5: 14554 and kernel is smaller for 235520 bytes !
Update!
lld-integration trunk has been merged into gcc-master branch. For those who use LLD, should switch to gcc-master as the lld-integration branch is now deprecated and will be removed soon.
The size difference between the two isn't much (~86mb vs 125mb), so it makes sense to have a single branch for everything.
I recommend to use zipped archive toolchains or if you use git operations to clone toolchain binaries, I recommend using --depth=1 while cloning the toolchain to avoid huge binary size cloning.
Update!
GCC Version has been bumped to 12.x
Eva GCC now ships with
GCC: 12.x
LLD: 13.x
BinUtils: 2.36.x
m_vaisakh said:
Update!
GCC Version has been bumped to 12.x
Eva GCC now ships with
GCC: 12.x
LLD: 13.x
BinUtils: 2.36.x
Click to expand...
Click to collapse
Been using Eva GCC for over 3 months now and it's been very reliable