[TOOL][LINUX/MAC]One Step to Initialize the Android Build Environment - Miscellaneous Android Development

Journey through the build process
Android is based on the Linux Kernel. Read on Wikipedia .
Before we get started, I created a binary long ago and thought I'd share it which helps you quickly setup your environment. If you're new to this, select option 3 and it should setup the environment for you. make sure to run it with `sudo` perms something like
Bash:
sudo ./setupEnv
. This will basically install library dependencies and set CCACHE=1 also size of ccache to a max of 15GiB(recommended). It should set your swappiness to 10 so that kernel doesnt swap out memory and lesser IO access == faster compile times. Now that that's over. Let's get into business!
Before we begin you must know at least the slightest of :
1. Java
2. C/C++
3. Makefiles
Knowing these will help you debug the ROM as well as solve any compile issues.
Short description about the folders present:
ART: Default Runtime used in lollipop
Bionic - the C-runtime for Android. It's mostly on BSD-derived sources. In this folder you will find the source for the c-library, math and other core runtime libraries.
Bootable: Contains the source for recovery.
Build - the build system implementation including all the core make file templates.
Device - product specific code for different devices.
External - contains source code for all external open source projects such as SQLite, Freetype and webkit.
Frameworks - this folder is essential to Android since it contains the sources for the framework. Consists of native libraries and core Java classes required for android. Example(ListView,TextView) classes are present here.
Hardware - hardware related source code such as the Android hardware abstraction layer specification and implementation. Also contains RIL implementations.
Kernel- Contains the sources for the Android version of the Linux kernel.
Packages - contains the source code for the default applications
Prebuilt - contains files that are distributed in binary form for convenience. Examples include the cross compilations toolchains for different development machines.
System - source code files for the core Android system. That is the minimal Linux system that is started before the VM and any java based services are enabled. This includes the source code for the init process and the default init.rc script that provide the dynamic configuration of the platform.
Read more about it at http://elinux.org/Android-4.0.3_r1. It's ICS but still gives a understanding of what each folder contains.
setupEnv
You should be able to setup build environment and initialise repository needed through my binary.
The device tree
.Contains the necessary PRODUCT_PACKAGES specified by your makefile which suits your device.
.CPU architecture, Partition size everything is set here.
.A skeleton tree can be created using
Code:
./build/tools/device/mkvendor.sh vendor devicename pathtoboot.img
.Doing this will provide a prebuilt kernel and not an inline kernel build process. using a prebuilt kernel is deprecated since it might not work correctly with the ROM you're building
.Custom init files are set here(Different radio types)
.Custom overlays for your device is set here.
.Custom init scripts are set here
Lets examine each of these files:
BoardConfig.mk
This file contains vital architectual and build information about the architecture of your device's motherboard, CPU, and other hardware. Getting this file right is essential.
To get a basic recovery booting, a few parameters need to be set in this file.
The following parameters must be set properly in BoardConfig to compile a working recovery image:
TARGET_ARCH: this is the architecture of the device it is usually something like arm or omap3.
BOARD_KERNEL_CMDLINE: not all devices pass boot parameters however if your device does this must be filled out properly in order to boot successfully. You can find this information in the ramdisk.img. (You can learn more about configuring the integrated kernel build-from-source here.)
BOARD_KERNEL_PAGESIZE: the pagesize of the stock boot.img and must be set properly in order to boot. Typical values for this are 2048 and 4096 and this information can be extracted from the stock kernel.
BOARD_BOOTIMAGE_PARTITION_SIZE: the number of bytes allocated to the kernel image partition.
BOARD_RECOVERYIMAGE_PARTITION_SIZE: the number of bytes allocated to the recovery image partition.
BOARD_SYSTEMIMAGE_PARTITION_SIZE: the number of bytes allocated to the Android system filesystem partition.
BOARD_USERDATAIMAGE_PARTITION_SIZE: the number of bytes allocated to the Android data filesystem partition.
Note:
The above information can be obtained by multiplying the size from /proc/partitions or /proc/mtd by the block size, typically 1024.
BOARD_HAS_NO_SELECT_BUTTON: (optional), use this if your device needs to use its Power button to confirm selections in recovery.
BOARD_FORCE_RAMDISK_ADDRESS / BOARD_MKBOOTIMG_ARGS: (optional), use these to force a specific address for the ramdisk. This is usually needed on larger partitions in order for the ramdisk to be loaded properly where it's expected to exist. This value can be obtained from the stock kernel. The former is deprecated as of Android 4.2.x and the latter will now be used in 4.2.x and beyond.
device_[codename].mk
The device_codename.mk makefile contains instructions about which Android packages to build , and where to copy specific files and packages, or specific properties to set during your compilation .
This file can be used to copy vital files into the ramdisk at compilation time.
PRODUCT_COPY_FILES: used to copy files during compilation into the ramdisk, which will be located at $OUT/recovery/root.
Example:
$(LOCAL_PATH)/sbin/offmode_charging:recovery/root/sbin/offmode_charging \
This will copy the file offmode_charging binary into the sbin folder within the ramdisk.
PRODUCT_NAME / PRODUCT_DEVICE: used for setting the value of your codename. This is the name of the device you load with Lunch.
This is simply the prebuilt kernel image or a kernel you built yourself used to boot the device. The format of the kernel may be in a zImage or uImage , depending on the requirements of the architecture of your device.
cm.mk
You'll need to make a few changes to this file to integrate with the lunch , brunch , and breakfast commands, so that your device shows up on the list and builds properly. You'll also set some variables (see other devices) to indicate what size splash animation should be used, whether this is a tablet or phone, etc.
Some of these settings aren't used for building just the recovery, but you may as well set them now because once recovery is done and working, the settings here will be important.
Again, take a look at a similar device to yours to get an idea of what the settings here should be. It's fairly intuitive.
recovery.fstab
recovery.fstab defines the file system mount point , file system type , and block device for each of the partitions in your device. It works almost exactly like /etc/fstab in a standard Linux operating system.
Example:
/system ext4 /dev/block/mmcblk0p32
This sets the block device at mmcblk0p32 to be mounted on /system as filesystem type ext4
All mountpoints should exist in this file and it is crucial this information be correct or else very bad things can happen, such as a recovery flash writing to the wrong location.
Note:
The filesystem type datamedia can be used for internal sdcards as well as setting the block device to /dev/null.
vendorsetup.sh
vendorsetup.sh is called when setupenv.sh is run. It is used to add non-standard lunch combos to the lunch menu.
To add your device to the lunch menu:
add_lunch_combo cm_<codename>-userdebug
Then build a test recovery image
wiki.cyanogenmod.org/w/Doc:_porting_intro
Vendor tree.
.Contains proprietary files/libraries
Examples: Vendor RIL, camera libraries
Once you have all this set up, you need to
Bash:
cd pathToSource
. build/envsetup.sh
lunch device_lunch_name
make recoveryimage/bacon/bootimage/Package/framework/library...
# Output generated in the out/target/product/devicename/
Download executable here : https://github.com/MasterAwesome/environment_setup/blob/master/Release/setupEnv?raw=true
I have added only Cyanogenmod and AOSP to my executable for creating a repository, for others someone fork my repo and create a pull request, if its suitable i'll merge and upload a newer version.
AOSP Guide : https://source.android.com/source/building.html
Cyanogenmod Docs : http://wiki.cyanogenmod.org/w/Development

Debugging
Debugging
This can be done using an IDE or by logcats.
Option 1(logcats) :
These provide logs which are created by android.util.log which is used in the Java Classes. They are used to log information as well as errors at runtime. Logcats and dmesg help debug most of the issues.
Option 2(IDE):
Setup a particular IDE(Eclipse or IntelliJ Idea)
Intellij: http://wiki.cyanogenmod.org/w/Doc:_import_to_intellij, https://shuhaowu.com/blog/setting_up_intellij_with_aosp_development.html
Eclipse: http://wiki.cyanogenmod.org/w/Doc:_eclipse
Once you've done that
1)https://software.intel.com/en-us/articles/android-system-level-javac-code-debugging
OR
Once you've imported your source code, you can add breakpoints to your modifications(if any) and then Remote debug it
Code:
Right-click "Remote Java Application", select "New".
Pick a name, i.e. "android-debug" or anything you like.
Set the "Project" to your android project name.
Keep the Host set to "localhost", but change Port to 8700.
Click the "Debug" button and you should be all set.
Then go to DDMS perspective and select a process you want to debug (select the process in the list of processes and then click on green bug icon).
Now you can switch to debug perspective.
.
I personally prefer eclipse for debugging maybe because I'm used to it, you should go with which ever is easier for you.

FAQ
FAQ
Will this tool work with all Linux distros?
No. It'll work with everything which uses apt-get(Debian based Distros){Ubuntu,Linux Mint,Elementary OS are some}
Once the tool completes installing and setting up what do I need to do?
Assuming you also initialised the repo using the tool, you could go to the folder and repo sync to start downloading the source code. Follow the build instructions after downloading(In post #1)
Why does this executable required elevated permissions
apt-get install can only install if elevated permissions are given, also, changing sysctl(/proc) also require `sudo` permissions.
How do I trust the tool with sudo permissions, can't that ruin my OS?
Since the executable uses system() calls, I've uploaded the source at GitHub and you can build it yourself. Instructions for building : README.md
Any other questions? post a reply and please dont quote the entire post.
Have a great day

Mac support is on the way...
Sent from my Moto G using XDA Free mobile app

.Mac support added.
.makefile added to provide a simpler build process
Using for Mac
.Download the source code.
.You need to install Xcode from App Store.
.type make make mac
./setupEnvMac to execute it.

Related

[Q] building ASOP generic ROM and applications

Hi.
I'm a veteran Android app developer but i am new to building roms, especially in the whole android build system.
It all started when trying to build the secured version of su.... i'm still unable to build it, the build system seems to ignore hte includes and claims there are undefined references to functions like get_property etc...
My main question is, i saw in the android build tree under /external that there are tcpdum, pcap and many nice to have applications i'll be happy to load on my device and compile for it (yes it's rooted)
My main steps in building (after checking out using repo) are:
run the . build/envsetup.sh script
run lunch generic-eng from the root of the srouce
run make
now my question is how do i make sure ALL the applications in the external folder are built ? should i change the build/code.mk to include their LOCAL_SRC name ? or is there a parameter i can pass to 'make' that should do the trick ?
Another question regarding the su itself is that it's Android.mk file did not contain LOCAL_MODULE_TAGS := sentence and i had to add it myself, anyone managed to successfully compile that file ?
Also anybody got some documentation regarding how to use the sqlite3 functions (or is it the std library and the documentation exist in the sqlite3 site ?)
10x
it seems to be a small problem with 2.3, once i checked out 2.2, compiled it using:
. build/envsetup.sh
launch generic-eng
make
all the binaries were built, since target was arm then all the binaries should be ok with the devices that use 2.2.
After the build i downloaded the su-binary from the git to $source_root/external/su-binary
changed the name in the Android.mk to su-binary so it wont collide with the system/external/su and run mm in the $source_root/external/su-binary dir.
Everything was build nicely.
I'll check the sqlite3 docs for refference but i'll appreciate anyone that knows the IBInder interface and the like that can explain the intnet send code (why not use am binary like in prev versions ? )

[DEV] How to compile from the source code for you Android

Greetings everyone...
As you might know: I'm the tech freak guy who compiled the GlibC and some Linux applications for HTC Desire.
As I have published my findings, I've recieved messages about how I'm compiling the applications given the source code. I try to help about this, but sadly I think half of what I say is not understandable without a proper guide. Ergo, this thread will suply this!
When?
Well, soon; because I'm still preparing it.
What will it include?
It'll be a basic guide about how to compile applications with your toolchain (Here is the guide about how to make a toolchain of yourself: How to make a toolchain). I'm planning to show it with GlibC compilation.
What's the extend of this help?
Frankly, compiling from the source sometimes can be quite messy and most of the time, a compilation/configuration line is never same with different applications. However, since most sources come with either Autotools or with a Makefile, most of the times, all you need is to change a few command line variable with a proper value.
I'm not planning to show you the code changes or "patchworks" they might be necessary, because this is quite application specific thing and I'm not an expert about this either. I'm just going to show you a guide about the compilation process.
What's the catch / scum of this help?
There is no catch in it. I just want to help curious nerdy guys like me )), so that we can learn from each other; or maybe improve each others work! I learned a lot from guys here, and I want to help to this community as much as I can as well.
Compilation Guide from Source code
Well, let's begin
We're going to compile the GlibC, with all internals to make it ready. After you prepare this, you might also check DoomLord's post here (http://forum.xda-developers.com/showthread.php?t=1041064) to make a recovery zip to flash this to your device. Make yourself comfortable now; because I'm going to explain everything we use here, thus this might be loong guide
1- Obtain the Sources
First, obtain the sources that you're going to compile. Check all the dependencies of the package and download them (and compile them) before. In our case, since we'll compile GlibC and all it needs is a working toolchain; we don't need to worry about this.
You can obtain GlibC source code from http://ftp.gnu.org/gnu/glibc/ address. Note that, since we're building GlibC for ARM devices and since GlibC seperated ARM support to "Ports" package, we need to download that as well. Make sure you download the same versions of Glibc and Glibc-Ports packages.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
2- Extract the Sources, and make a build directory
Extract the sources to some folder; say /home/<user>/Desktop/glibc . Also extract the glibc-ports package to some direcorty, say /home/<user>/Desktop/glibc-ports.
After this, rename the folder "glibc-ports" to just "ports" and then move it inside to the "glibc" folder. If you don't do this step, you'll have "ARM is unsupported" error in configuration step.
Since glibc cannot be compiled from the directory which sources are in it (a restriction made by the developers, to make sources unchanged and ready to be compiled again), we need to make another folder from which we're going to call the compilation tools. Let's say this directory is named as "glibc-build", also at /home/<user>/Desktop .
3- Start "configure"
Now, open a terminal emulator window ( you can use Ctrl+Alt+T keys under Ubuntu to easily open one ). Change into the glibc-build directory.
Code:
cd /home/<user>/Desktop/glibc-build
After this operation, we've to call the glibc's auto-configure script from this folder. Note that ".." is a "special folder name" which denotes the upper level directory (so basicly, we're changing into the upper level in directory tree - Desktop in our case - and then call something from a folder inside that)
Code:
../glibc-2.14/configure --help
When you run the above command, configure will give you the options you can use to configure this package for your needs. Nearly 90% of the time, checking the output of this command gives you the options you need to set to compile properly.
Code:
`configure' configures GNU C Library (see version.h) to adapt to many kinds of systems.
Usage: ../glibc-2.14/configure [OPTION]... [VAR=VALUE]...
To assign environment variables (e.g., CC, CFLAGS...), specify them as
VAR=VALUE. See below for descriptions of some of the useful variables.
Defaults for the options are specified in brackets.
Configuration:
-h, --help display this help and exit
--help=short display options specific to this package
--help=recursive display the short help of all the included packages
-V, --version display version information and exit
-q, --quiet, --silent do not print `checking ...' messages
--cache-file=FILE cache test results in FILE [disabled]
-C, --config-cache alias for `--cache-file=config.cache'
-n, --no-create do not create output files
--srcdir=DIR find the sources in DIR [configure dir or `..']
Installation directories:
--prefix=PREFIX install architecture-independent files in PREFIX
[/usr/local]
--exec-prefix=EPREFIX install architecture-dependent files in EPREFIX
[PREFIX]
By default, `make install' will install all the files in
`/usr/local/bin', `/usr/local/lib' etc. You can specify
an installation prefix other than `/usr/local' using `--prefix',
for instance `--prefix=$HOME'.
For better control, use the options below.
Fine tuning of the installation directories:
--bindir=DIR user executables [EPREFIX/bin]
--sbindir=DIR system admin executables [EPREFIX/sbin]
--libexecdir=DIR program executables [EPREFIX/libexec]
--sysconfdir=DIR read-only single-machine data [PREFIX/etc]
--sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com]
--localstatedir=DIR modifiable single-machine data [PREFIX/var]
--libdir=DIR object code libraries [EPREFIX/lib]
--includedir=DIR C header files [PREFIX/include]
--oldincludedir=DIR C header files for non-gcc [/usr/include]
--datarootdir=DIR read-only arch.-independent data root [PREFIX/share]
--datadir=DIR read-only architecture-independent data [DATAROOTDIR]
--infodir=DIR info documentation [DATAROOTDIR/info]
--localedir=DIR locale-dependent data [DATAROOTDIR/locale]
--mandir=DIR man documentation [DATAROOTDIR/man]
--docdir=DIR documentation root [DATAROOTDIR/doc/c-library]
--htmldir=DIR html documentation [DOCDIR]
--dvidir=DIR dvi documentation [DOCDIR]
--pdfdir=DIR pdf documentation [DOCDIR]
--psdir=DIR ps documentation [DOCDIR]
System types:
--build=BUILD configure for building on BUILD [guessed]
--host=HOST cross-compile to build programs to run on HOST [BUILD]
Optional Features:
--disable-option-checking ignore unrecognized --enable/--with options
--disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
--enable-FEATURE[=ARG] include FEATURE [ARG=yes]
--disable-sanity-checks really do not use threads (should not be used except
in special situations) [default=yes]
--enable-check-abi do "make check-abi" in "make check" (no/warn/yes)
[default=no]
--enable-shared build shared library [default=yes if GNU ld & ELF]
--enable-profile build profiled library [default=no]
--enable-omitfp build undebuggable optimized library [default=no]
--enable-bounded build with runtime bounds checking [default=no]
--disable-versioning do not include versioning information in the library
objects [default=yes if supported]
--enable-oldest-abi=ABI configure the oldest ABI supported [e.g. 2.2]
[default=glibc default]
--enable-stackguard-randomization
initialize __stack_chk_guard canary with a random
number at program start
--enable-add-ons[=DIRS...]
configure and build add-ons in DIR1,DIR2,... search
for add-ons if no parameter given
--disable-hidden-plt do not hide internal function calls to avoid PLT
--enable-bind-now disable lazy relocations in DSOs
--enable-static-nss build static NSS modules [default=no]
--disable-force-install don't force installation of files from this package,
even if they are older than the installed files
--enable-kernel=VERSION compile for compatibility with kernel not older than
VERSION
--enable-all-warnings enable all useful warnings gcc can issue
--enable-multi-arch enable single DSO with optimizations for multiple
architectures
--enable-experimental-malloc
enable experimental malloc features
--enable-nss-crypt enable libcrypt to use nss
Optional Packages:
--with-PACKAGE[=ARG] use PACKAGE [ARG=yes]
--without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no)
--with-gd=DIR find libgd include dir and library with prefix DIR
--with-gd-include=DIR find libgd include files in DIR
--with-gd-lib=DIR find libgd library files in DIR
--with-fp if using floating-point hardware [default=yes]
--with-binutils=PATH specify location of binutils (as and ld)
--with-elf if using the ELF object format
--with-selinux if building with SELinux support
--with-xcoff if using the XCOFF object format
--without-cvs if CVS should not be used
--with-headers=PATH location of system headers to use (for example
/usr/src/linux/include) [default=compiler default]
--with-tls enable support for TLS
--without-__thread do not use TLS features even when supporting them
--with-cpu=CPU select code for CPU variant
Some influential environment variables:
CC C compiler command
CFLAGS C compiler flags
LDFLAGS linker flags, e.g. -L<lib dir> if you have libraries in a
nonstandard directory <lib dir>
LIBS libraries to pass to the linker, e.g. -l<library>
CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if
you have headers in a nonstandard directory <include dir>
CPP C preprocessor
CXX C++ compiler command
CXXFLAGS C++ compiler flags
Use these variables to override the choices made by `configure' or to help
it to find libraries and programs with nonstandard names/locations.
Report bugs to <glibc>.
GNU C Library home page: <http://www.gnu.org/software/c-library/>.
General help using GNU software: <http://www.gnu.org/gethelp/>.
The help is quite straightforward, and all explanations are already done. The most important options for our case here is "prefix" and "host". Also, for compilation of GlibC, we need to give "with-headers", "with-tls" and "with-ports" proper values.
"Prefix" is a very important option. This options specifies where the installed binaries and libraries "think" they are. Some binaries and libs. are quite flexible: they don't care about where they are, thus the value of prefix isn't much of importance, but most of the time, it is. Omitting a prefix value here will make PREFIX value of "/usr/local" - which is quite dangerous because it may screw your own system (/usr/local is usually where your own PC applications are - most probably you cannot screw this folder if you're not a super-user but, well, anyway...); so always change this value to something proper when you're cross compiling.
Also, make sure that this path exists in your target device. If you set a prefix, say /cache, and then copy the stuff at, say, /data; it's highly probable that your applications will search for it's config files at /cache, and will complain that they are not found - since they're at /data. Since AFAIK, there is no way to change this in most application after compilation of software, you have to recompile the program to fix such errors (or move the application to the prefix it's set).
I usually use /data as prefix, because I install the stuff to there.
Host is another very important option. If you don't give this parameter, auto-configure will think you're compiling this application for the PC you're using. We've to give "host" the machine specification we're compiling the app for. For ARM based devices, this is mostly something line "arm-xxx-linux" or "arm-linux" or "arm-xxxx-linux-gnueabi" or such.
How do you found this out? Well, simple: If you followed my crostool-ng tutorial and used the same options as I did; your host name will be in "arm-xxxxxx-linux-gnueabi" format. "xxxxx" here is what your vendor name is. If you set a vendor value when making the toolchain, you must write it here. If you left it empty, xxxxx will be "unknown".
The other way to check this is your toolchain's name of gcc command: If your ARM GCC toolchain name is "arm-myvendor-linux-gnueabi-gcc" then, your host name is "arm-myvendor-linux-gnueabi".
Most of the times, there are other options that you must give. As for an example, here is what I generally use to compile glibC:
Code:
../glibc-2.14/configure --host=arm-msm-linux-gnueabi --prefix=/data --with-tls --with-ports="nptl, ports"
I use host as "arm-msm-linux-gnueabi" because this is my cross compile target name.
We set "with-tls" option here: it's for GlibC to support Thread Local Storage. This is a programming scheme that allows threads to have their own storage area; and we give this configuration here to give GlibC a support for this. More info about TLS can be found online in programming wiki's.
We also set "with-ports" because we're also installing some extra plugins for our GlibC install. NPTL is an advanced threading library that supports many advanced features like TLS and such. There is also a "linuxthreads" but it's older. New versions of GlibC is not shipped with "linuxthreads" either. The other port, "ports" is for ARM support.
You must also give the kernel source directory here with a parameter of "with-headers" if you're using some other toolchain and your kernel sources are in some other place. Thanks to crostool-ng that it moves all the headers to the toolchain folder and makes them automatically reachable, so you can omit this parameter.
Well, when you give the command and supply the options and press ENTER, it shall start configuring the package:
When there is a missing dependency or option, it shall generally give you a warning at this step and terminate the configuration. You must then find the solution of this error (install the necessary dependency etc.) and then try to configure the package again - as I said, this is usually not the case for GlibC .
3- Start "make"
After configuration process is done without errors, you can pass to the compilation phase. If you did configuration properly, you don't have to use special parameters and such for most packages. Just issue the command "make" and wait
Code:
make
Once this is completed without errors, you're nearly ready for the shipment Note that, if you get any errors in this process, you should definitely google it; since errors in this level is quite application and build environment (your machines configs. etc.) specific. The errors in this phase is usually needs some tricks to fix, and they are usually not so easy to fix
3- Start "make install"
Once the compilation is successful, you can install the application to the "PREFIX" folder you specified whilst configuring the package. In order to do that, just issue:
Code:
make install
However, issuing just this command isn't what we want generally: Since we've cross compiled the application, we're not interested in installing that to our PC! We want it to be somewhere we can easily access, so that we can distribute easily right?
For most packages, three important variables are necessary to make this. Which variable is heeded is highly dependent of package, however it's DESTDIR usually used.
DESTDIR: The variable usually used in makefiles. When you set DESTDIR while giving "make install", the application becomes installed to DESTDIR/PREFIX folder, instead of just PREFIX folder without changing the prefix info inside the application so it makes it easier for us to distribute the package.
install_root : GlibC uses this variable, instead of DESTDIR. I didn't see any other packages using this.
INSTALL_TOP: OpenSSL and Lua packages use this variable instead of DESTDIR.
Which of these variable is used cannot be identified either without examining the Makefile, or without testing it
How are you going to give it a value? Simple: you just assign the variable a value, after giving "make install" command:
Code:
make install DESTDIR=/home/<user>/myapp
For GlibC, this becomes:
Code:
make install install_root=/home/<user>/glibc
Which makes glibc installed to /home/<user>/glibc folder.
You can take the files from this folder and make a flashable zip; and send it to your device!
5- NOTES
- Be careful about the PREFIX value, you should install the files to this place in your device as well.
- Most of the times, you don't need the "include" folder - this folder keeps the header files for your packages, so that other applications may be compiled with them. Since we're not compiling applications in our device, we don't need headers. Compiled applications don't use headers anymore.
- Most of the times, .a files under the "lib" folder is unnecessary - these are static libraries that are usually used when applications are being compiled. Since we don't compile stuff at our device, we don't need them - they are usually quite big too! Be careful though: some packages don't offer .so files (which are dynamic libraries for applications to use dynamically) - it might be necessary to keep .a files then.
- .la files are needed for application compilation with libraries, you can erase them as well.
- pkgconfig can be erased in distribution packages. PKGConfig is a tool for compilation that automatically parses the files in pkgconfig dirs to give necessary compilation parameters with dynamic libraries to the applications at compile time. Like we don't need headers, we don't need those files.
Well, that's it. I hope I could be of some help. See you next time and happy Android'ing!
< Reserved for other stuff for guide, like hints.. >
Maybe also include a precompiled toolchain for download?
Can do that, but first need to compile a "static" toolchain for this. Actually, this seems like a good idea
Let me work on this next
Nice...big thx...i love such guide´s
with kind regards...Alex
theGanymedes said:
Can do that, but first need to compile a "static" toolchain for this. Actually, this seems like a good idea
Let me work on this next
Click to expand...
Click to collapse
Tried that: The output is huge, and even though the toolchain works without any need to any library, it still needs the libraries to be compiled in order to compile the other binaries - makes sense, since you need the libraries to compile against them anyway
So isn't it possible to package your toolchain which contains gcc, initial glibc and libraries, so that we can download and use them right away? It would save a lot of time and would be a far better option than the ndk.
Sent from my HTC Desire using Tapatalk
Droidzone said:
So isn't it possible to package your toolchain which contains gcc, initial glibc and libraries, so that we can download and use them right away? It would save a lot of time and would be a far better option than the ndk.
Sent from my HTC Desire using Tapatalk
Click to expand...
Click to collapse
That's possible, but only if you're also using Ubuntu 10.10 or 10.04 and a 64-bit machine
For some reason - guess library version differences - toolchains done in 11.10 was not working at 10.04; so I had to redo it, for instance. This technique you say might or might not work - there is no guarantee in it.
Ah well.. I'm using a 32 bit Ubuntu 11.10 on a 64 bit machine. Anyway I have the toolchain I compiled on this one with your help..But if you remember I took a couple of days to get it done, even when I had your excellent help..Newcomers might find it difficult I guess. But if it werent for your toolchain compilation guide in the glibc thread, it'd have been next to impossible!
Droidzone said:
Ah well.. I'm using a 32 bit Ubuntu 11.10 on a 64 bit machine. Anyway I have the toolchain I compiled on this one with your help..But if you remember I took a couple of days to get it done, even when I had your excellent help..Newcomers might find it difficult I guess. But if it werent for your toolchain compilation guide in the glibc thread, it'd have been next to impossible!
Click to expand...
Click to collapse
Thanks Actually, it took me 3 days to make a bootstrap GCC and 8 days to make a Stage 1 GCC when I first began this I could never compile GlibC with those, because of the messy patches needed
I'm never claiming the process to be easy; but believe me: making a toolchain is the most irritating part. Most of the other programs do compile and run just fine, once their dependencies met. Only "very massive projects" like Xorg gives headaches time to time: because normal users do never try to compile those stuff from the source; and because of that, the developers don't try to make the process error-free.
Still, "grep" and knowledge of C makes wonders time to time
Droidzone said:
Maybe also include a precompiled toolchain for download?
Click to expand...
Click to collapse
I have a precompiled toolchain for android, it is in alpha stage with work still ongoing, and needs 2Go of storage. See: http://forum.xda-developers.com/showthread.php?p=43256170#post43256170

[GUIDE] Understanding the Android Source Code

Hello XDA,
I have been going through many guides over XDA, there were many that taught you how to compile from source, but I didn't found a single guide that could explain how Android source code works or what are so many folders, files and commands for, so here it is, a lengthy yet informative guide for aspiring devs. And yeah, I'm not a dev, just another guy who learns by exploring many things and likes to share it.
Click to expand...
Click to collapse
Note - I own an HTC Explorer(Pico), so many things that are written *might* be exclusive to it, or others with some changes.
Describing the Android Source Code Folders -​
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
I'll take CM11 as an example to explain this.
Now the folders, in alphabetical order-
1. abi - This folder contains a sub folder called cpp which actually contains many C++ files linked to many places.
2. android Remember this?
Code:
repo init -u git://github.com/CyanogenMod/android.git
Yes, it's that android.git folder.
Have a look at this also.
3. art - Yeah, it is the folder that deals with the compilation of the latest android ART runtime. If you're looking into source directories of some other android versions, you won't fins it obviously.
4. bionic - Bionic is mainly a port of the BSD C library to our Linux kernel with the following additions/changes:
- No support for locales.
- No support for wide chars (i.e. multi-byte characters).
- its own smallish implementation of pthreads based on Linux futexes.
- Support for x86, ARM and ARM thumb CPU instruction sets and kernel interfaces.
5. bootable - Boot and startup related code. Some of it is legacy, the fastboot protocol info could be interesting since it is implemented by boot loaders in a number of devices such as the Nexus ones.
6. build - The main entry point of the build system resides here - envsetup.sh, if you follow the instruction in source.android.com you will see that the first step before you do anything to build Android is to use the command source build/envsetup.sh
The script will check few things to make sure all the needed application available in the local machine. It also setup the devices that can be built, which is extracted from the directory device.
7. cts - the compatability tests. The test suite to ensure that a build complies with the Android specification.
8. dalvik - This is the folder responsible for the compilation of the Dalvik runtime for the Android devices.
*Have a look at the difference between the two(art and dalvik) folders and you'll have an idea of how things work in that case*
art -
dalvik -
9. Development - This directory contains application that are not part of the deployed app in the OS. There are some useful application such as widget builder, etc
10. Device - It contains the device specific configurations for many devices.
Note - Many people ask me what, the folders like 'common' and 'generic are for, so here -
> common - This directory contains gps information and also a script that allows you to extract proprietary binary files from your phone to be part of the build process.(You can try to have a look at your device's device tree and then the cm.mk file, where you could find relations of these files. In my case, it shows like this:
Code:
# Include GSM stuff
$(call inherit-product, vendor/cm/config/gsm.mk)
-and-
# Inherit some common cyanogenmod stuff.
$(call inherit-product, device/common/gps/gps_eu_supl.mk)
> generic - This directory contains the generic device configuration that is called ‘goldfish’. This is the device classification used when building the emulator.
> Google - This directory contains the Android Accessories Kit code. It contains a demokit Android app that allows you to control the ADK board. The ADK firmware can be check out here http://code.google.com/p/microbridge/. There is a good article about this here.
> sample - This directory contains a full example of writing your own Android platform shared library, without changing the Android framework. It also shows how to write JNI code for incorporating native code into the library, and a client application that uses the library. This example is ONLY for people working with the open source platform to create a system image that will be delivered on a device which will include a custom library as shown here. It can not be used to create a third party shared library, which is not currently supported in Android.
11. docs - I contains an important sub-folder called source.android.com. Contains tutorials, references, and miscellaneous information relating to the Android Open Source Project (AOSP). The current iteration of this site is fully static HTML (notably lacking in javascript and doxygen content), and is and/or was maintained by skyler (illustrious intern under Dan Morrill and assistant to the almighty JBQ).
12. external - This directory contains source code for all external open source projects such as SQLite, Freetype, webkit and webview.
13. frameworks - Ah, one of the most important directories. it contains the sources for the framework. Here you will find the implementation of key services such as the System Server with the Package- and Activity managers. A lot of the mapping between the java application APIs and the native libraries is also done here.
A special note on this one - I'd recommend new users to not to play with any file/folder inside the frameworks folder, maybe your ROM doesn't boots then.
14. hardware - Hardware related source code such as the Android hardware abstraction layer specification and implementation. This folder also contains the reference Radio Interface Layer(RIL - To communicate with the modem side) implementation.
15. Kernel - It's not a default folder in the source code, but it's a part of device configuration set-up. It contains the kernel source of your device.
16. libcore - I'll explain this one with the important folders inside this, since every folder performs a different function.
dalvik - DalvikVM runtime for Android
dom - Java test classes for DOM
expectations - Contains information about the test cases
include - Some C/C++ include files that used for Array and String handling
json - JSON based Java implementation
luni - Contains test source code for loading .jar and .dex files
support - Contains support class file for testing Dalvik
xml - XML pull and push implementation
17. libnativehelper - I have no idea on this one. If someone knows, share your knowledge. :good:
18. ndk - Contains build scripts and helper files for building the NDK
19. out(Everyone's favorite directory ) - The build output will be placed here after you run make. The folder structure is out/target/product/. In the default build for the emulator the output will be placed in out/target/product/generic. This is where you will find the images used by the emulator to start (or to be downloaded and flashed to a device if you are building for a hardware target).
20. packages - Standard Android application that are available as part of the AOSP - Camera, SMS, Dialer, Launcher, etc
21. pdk - I believe that 'pdk' is the Platform Development Kit, it's basically an SDK/set of tools that Google sends to OEMs to evaluate their framework ahead of each major Android upgrade since Android 4.1.
(Thanks to @yowanvista )
22. prebuilt - Contains files that are distributed in binary form for convenience. Examples include the cross compilations toolchains for different development machines.
23. sdk - This directory contains lots of apps that are not part of operating system. There are quite useful apps that developers can leverage on and can be enhanced further as part of the operating system.
24. system - Source code files for the core Android system. That is the minimal Linux system that is started before the Dalvik VM and any java based services are enabled. This includes the source code for the init process and the default init.rc script that provide the dynamic configuration of the platform.
25. tools - Some external important tools that help in compiling. Not sure though. :/
26. vendor - This directory contains vendors specific libraries. Most of the proprietary binary libraries from non-open source projects are stored here when building AOSP.
One thing - Beyond the above you also have the hidden .repo directory that contains the source for the repo utility. It also holds the manifest specifying what git repositories you want to track for this Android source project. If you have your own additions you could automatically track them by adding a local manifest here. For modifications of the platform framework there are some instructions available in the device/sample folder of the source code tree. That will show you how to add APIs to Android without having to modify the core framework.
Now, many people ask me the difference between the 'Breakfast', 'Lunch' and 'Brunch' command. So, here's what these commands are specific for:​
Breakfast
You may not ever use this command , but in order to explain brunch, we have to explain breakfast first. Breakfast is a function used to configure your build. It keeps track of a list of officially-supported devices to build for, and allows you to choose one.
If you do not put a device, the script will output a list of available devices to build for, and you can then choose yours. Breakfast then goes on to configure your build environment with the correct variables to create your device-specific rom.
Brunch
Defined simply, brunch is equivalent to
Code:
breakfast [device name] && mka bacon
This means that it sets up your build environment to be configured for your device, and then commences the build process. mka bacon is just CyanogenMods’s way of saying build the code for your device. It’s generally only used for officially supported devices (ones that you use can choose through the breakfast menu).
Lunch
This ones also pretty simple to explain. It’s used EXACTLY like breakfast, the only difference being the choices you have to build with it. Using lunch, you can choose non-official or non-standard builds for your device. This includes special debug versions and also allows you to build CyanogenMod for use on the Android Emulator. To build after running lunch, simply issue the command mka.
For other commands and help, I recommend you to read this article.
Few common Terms​Hey, do you ever find few terms on xda, related to development that go over your mind? Well, I recommend you to have a look at this post by Recognized Contributor @TheByteSmasher and give him the Thanks he deserves.
Understanding Android Makefile (Android.mk)​Well, I bet if go through many folders of the Source Code, you might find the file named Android.mk! Here are few important things regarding that -
These Android.mk files defines how to build that source code. There are well defined specific rules for Android.mk files. Let me summarize them.
Name: We need to define a name for our build (LOCAL_MODULE := )
Local Variables: All builds may have some local variables so to start a new build it is good to clear all local variables (include $(CLEAR_VARS))
Files: We need to write all files we want it to be build (LOCAL_SRC_FILES := main.c)
Tags: Define tags for build. (LOCAL_MODULE_TAGS := eng development)
Libraries: If build needs to be linked to other libraries, we need to define them (LOCAL_SHARED_LIBRARIES := cutils)
Template file: We can define whether our build is executable,library or something else by including template file (include $(BUILD_EXECUTABLE))
BUILD_EXECUTABLE, CLEAR_VARS, etc. variables which are the absolute address of the template files are defined in build/core/config.mk.
Now an example -
The following Android.mk builds a simple APK. The codes beginning with a has(#) are the comments to tell you what the lines actually means.
Code:
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
# Build all java files in the java subdirectory
LOCAL_SRC_FILES := $(call all-subdir-java-files)
# Name of the APK to build
LOCAL_PACKAGE_NAME := LocalPackage
# Tell it to build an APK
include $(BUILD_PACKAGE)
Last line in Android.mk file builds the APK file. Different source code types must be built differently so we can also use $(BUILD_EXECUTABLE), $(HOST_JAVA_LIBRARY), $(HOST_PREBUILT) etc. variables according to our source code. (Definitions like my-dir, all-subdir-java-files are in build/core/definitions.mk). We can add LOCAL_MODULE_TAGS variable to Android.mk file to determine that module to be installed in that source code built. Here are the some defined tags and their meanings.
Makefile tricks. ​
>Build helper functions
A whole bunch of build helper functions are defined in the file build/core/definitions.mk
Try grep define build/core/definitions.mk for an exhaustive list.
Here are some possibly interesting functions:
print-vars - shall all Makefile variables, for debugging
emit-line - output a line during building, to a file
dump-words-to-file - output a list of words to a file
copy-one-file - copy a file from one place to another
>Add a file directly to the output area
You can copy a file directly to the output area, without building anything, using the add-prebuilt-files function.
The following line, extracted from prebuilt/android-arm/gdbserver/Android.mk copies a list of files to the EXECUTABLES directory in the output area:
Code:
$(call add-prebuilt-files, EXECUTABLES, $(prebuilt_files))
Understanding the Device Tree​Note - This is by no means complete, and there will be omissions as have explained all this top of my head and copied pasted certain bits that I have here on my own device tree.
Android.mk - this will tell the build system to include and to build sources specifically for your device. I have explained a bit of it above as well.
AndroidBoard.mk - this is for the kernel, the build system uses that to drop the kernel image in place.
AndroidProducts.mk - specifies the appropriate device's make file, to use for building. i.e. device/htc/pico/pico.mk(In my case), This is device-specific as well.
device_codename.mk - Every device has a codename, and there is a file named as the codename of device. It specifies the properties and extras to copy over into the final output, in this case, it could be for example, pico.mk
BoardConfig.mk - This is the meat of it all, this is where compiler conditional flags are set, partition layouts, boot addresses, ramdisk size, and so on.
Android.mk
Code:
LOCAL_PATH := $(my-dir)
ifeq ($(TARGET_DEVICE),pico)
include $(call all-makefiles-under,$(LOCAL_PATH))
endif
^This is how the build will use that to build recovery, sensors, lights and camera (of course there will be more), its saying 'Yo Builder, go into each of the directories specified, and build the respective sources'
AndroidBoard.mk:
Code:
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
ALL_PREBUILT += $(INSTALLED_KERNEL_TARGET)
# include the non-open-source counterpart to this file
-include vendor/htc/pico/AndroidBoardVendor.mk
^ This one is pretty simple and tells the build system to go to the vendor tree of the device and include AndroidBoardVendor.mk for building.
AndroidProducts.mk
Code:
PRODUCT_MAKEFILES := \
$(LOCAL_DIR)/cm.mk \
$(LOCAL_DIR)/pico.mk
^Just specifying the makefiles of the device.
BoardConfig.mk
Now in this one, I'm taking a BoardConfig.mk file that I found over the internet with a suitable explanation of it. Credits to the post over stackoverflow for the valuable efforts.
Code:
LOCAL_PATH:= $(call my-dir)
TARGET_NO_BOOTLOADER := true
TARGET_PREBUILT_KERNEL := device/lg/gt540/kernel
TARGET_PREBUILT_RECOVERY_KERNEL := device/lg/gt540/recovery_kernel
# This will vary from device!
TARGET_BOARD_PLATFORM := msm7k
TARGET_ARCH_VARIANT := armv6-vfp
TARGET_CPU_ABI := armeabi
TARGET_CPU_ABI := armeabi-v6l
TARGET_CPU_ABI2 := armeabi
# OpenGL drivers config file path
BOARD_EGL_CFG := device/lg/gt540/egl.cfg
# Dependant, not to be taken literally!
BOARD_GLOBAL_CFLAGS += -DHAVE_FM_RADIO
# Dependant, not to be taken literally!
BOARD_KERNEL_BASE := 0x02600000
# this will be device specific, and by doing cat /proc/mtd will give you the correct sizes
BOARD_BOOTIMAGE_PARTITION_SIZE := 0x00480000
BOARD_RECOVERYIMAGE_PARTITION_SIZE := 0x00480000
BOARD_SYSTEMIMAGE_PARTITION_SIZE := 0x0cf80000
BOARD_USERDATAIMAGE_PARTITION_SIZE := 0x0d020000
BOARD_FLASH_BLOCK_SIZE := 131072
^That is an excerpt, notice how we specify kernel's base address, this is how the boot.img gets generated after compilation is done and yet again, gets dropped into out/target/product/lg/gt540/boot.img. Also, more importantly, we're telling the build system to use the target platform for cross-compiling the sources (*TARGET_BOARD_PLATFORM*/*TARGET_CPU_ABI*) There will be more information in there such as conditional flags to pass to the compiler, for an example. we specified the directive HAVE_FM_RADIO to tell it, when it comes to handling the source for the FM radio system, to conditionally compile parts of the source. Again, this is hardware specific and mileage will vary, also this applies to the address for boot. In a nutshell, this is saying 'Yo Builder, read the damn variables and remember them and apply them when cross-compiling those source files!'
Now that the internals of each of those Android build make-files are shown.
Now, onto the vendor/ part of it, in AOSP, simply, once again, correlation and corresponds with the device/ tree, as in continuing with this example, vendor/lg/gt540/ which gets picked up by the lunch. There's more make files in there but the general consensus is there's a directory called proprietary which contains the proprietary libs (due to close-source etc) that gets copied over. The copying over of the libraries gets specified in the file device-vendor-blobs.mk, in this case, gt540-vendor-blobs.mk.
Adding your Own Apps in the Source Code​The following part of the guide will tell you how to make your own apps to compile with the ROM and be a part of it.
1. Add the App's source code to packages/apps/name_of_app
2. Regardless of how you put the source in packages/apps/, assuming that the source for the app has an Android.mk Makefile, you can get it to automatically build and install the resulting file in your $OUT directory (and thus your .zip) by simply determining the name of the project, which is typically defined in Android.mk with this:
Code:
LOCAL_PACKAGE_NAME := PackageName
3. Add the project to device.mk (or whatever .mk) in the device folder.
Have a look at an example here
Compiling specific parts of a ROM​
To compile an app-
Code:
make app_name.apk -j4
Wait, there's some more things you must know for compiling a *specific app* ! Thanks to @thewisenerd head over to this post.
To compile other parts you also need to enter the folder name, here are some examples from /frameworks/base [Credits-XDA University Article]
Code:
make android.policy (the power menu and lockscreen)
make framework (the initial framework files)
make framework-res (the initial framework resources)
make services (the services.jar file)
More examples:
Code:
make sdk (builds the android sdk)
make modules (builds all modules)
make installclean (removes all staging directories, such as out/target/product/boardname/system)
make clean (removes the whole /out directory)
make recoveryimage (builds the recovery from /bootable/recovery, customizable if wanted!)
Configuring a new Product for the AOSP.
Again, I'd recommend you to have a look at this great document.
A detail information about BoardConfig.mk
Well, I believe that BoardConfig.mk is one of the most important files for the device tree. Now, here are few parameters of the file explained-
TARGET_ARCH: set to arm for almost all current Android devices.
BOARD_KERNEL_CMDLINE: not all devices pass boot parameters however if your device does this must be filled out properly in order to boot successfully.
BOARD_KERNEL_PAGESIZE: the pagesize of the stock boot.img and must be set properly in order to boot. Typical values for this are 2048 and 4096 and this information can be extracted from the stock kernel.
BOARD_BOOTIMAGE_PARTITION_SIZE: the number of bytes allocated to the kernel image partition.
BOARD_RECOVERYIMAGE_PARTITION_SIZE: the number of bytes allocated to the recovery image partition.
BOARD_SYSTEMIMAGE_PARTITION_SIZE: the number of bytes allocated to the Android system filesystem partition.
BOARD_USERDATAIMAGE_PARTITION_SIZE: the number of bytes allocated to the Android data filesystem partition.
^The above information can be gathered by multiplying the size from /proc/partitions by the block size, typically 1024.
BOARD_HAS_NO_SELECT_BUTTON: (optional), use this if your device needs to use its Power button to confirm selections in recovery.
BOARD_FORCE_RAMDISK_ADDRESS / BOARD_MKBOOTIMG_ARGS: (optional), use these to force a specific address for the ramdisk. This is usually needed on larger partitions in order for the ramdisk to be loaded properly where it's expected to exist. This value can be obtained from the stock kernel. The former is deprecated as of Android 4.2.x and the latter will now be used in 4.2.x and beyond.
Few *more* external links ​Few great docs that I found over the web-
One
Two
Three
Credits​>Firstly, My parents.
> @Red Devil, @galaxyfreak, @speed_bot, @Nihar.G and my Team for helping me with all my doubts and even answering the most supid questions that I asked them
> @corruptionfreeindia for the idea.
> Helpful and informative posts over-
CyanogenMod Wiki
elinux.org
StackOverflow
Got a query? Ask in the thread. Quote or mention me for quicker replies.
Thanks to @eagleeyetom for featuring it on the portal!
Understanding the Kernel Source Code​Credits for this- Linux.org
I'll basically explain the 'Linux Kernel Source' and since Android is based on the Linux, it's almost same for both.
Now the folders in order-
1. arch - This folder contains a Kconfig which sets up some settings for compiling the source code that belongs in this folder. Each supported processor architecture is in the corresponding folder. So, the source code for Alpha processors belong in the alpha folder. Keep in mind that as time goes on, some new processors will be supported, or some may be dropped.
2. block - This folder holds code for block-device drivers. Block devices are devices that accept and send data in blocks. Data blocks are chunks of data instead of a continual stream.
3. crypto - This folder contains the source code for many encryption algorithms. For example, “sha1_generic.c” is the file that contains the code for the sha1 encryption algorithm.
4. Documentation - This folder contains plain-text documents that provide information on the kernel and many of the files. If a developer needs information, they may be able to find the needed information in here.
See, this folder is just so awesome-
5. drivers - This directory contains the code for the drivers. A driver is software that controls a piece of hardware. For example, for a computer to understand the keyboard and make it usable, a keyboard driver is needed. Many folders exist in this folder. Each folder is named after each piece or type of hardware. For example, the bluetooth folder holds the code for bluetooth drivers. Other obvious drivers are scsi, usb, and firewire. Some drivers may be more difficult to find. For instance, joystick drivers are not in a joystick folder. Instead, they are under ./drivers/input/joystick. Keyboard and mouse drivers are also located in the input folder. The Macintosh folder contains code for hardware made by Apple. The xen folder contains code for the Xen hypervisor. A hypervisor is software or hardware that allows users to run multiple operating systems on a single computer.
6. firmware - The firmware folder contains code that allows the computer to read and understand signals from devices. For illustration, a webcam manages its own hardware, but the computer must understand the signals that the webcam is sending the computer. The Linux system will then use the vicam firmware to understand the webcam. Otherwise, without firmware, the Linux system does not know how to process the information that the webcam is sending. Also, the firmware helps the Linux system to send messages to the device. The Linux system could then tell the webcam to refocus or turnoff.
7. fs - 'fs' stands for File System. All of the code needed to understand and use filesystems is here. Inside this folder, each filesystem's code is in its own folder. For instance, the ext4 filesystem's code is in the ext4 folder. Within the fs folder, developers will see some files not in folders. These files handle filesystems overall. For example, mount.h would contain code for mounting filesystems. A filesystem is a structured way to store and manage files and directories on a storage device. Each filesystem has its own advantages and disadvantages. These are due to the programming of the filesystem. For illustration, the NTFS filesystem supports transparent compression (when enabled, files are automatically compressed without the user noticing). Most filesystems lack this feature, but they could only possess this ability if it is programmed into the files in the fs folder.
8. include - The include folder contains miscellaneous header files that the kernel uses. The name for the folder comes from the C command "include" that is used to import a header into C code upon compilation.
9. init - The init folder has code that deals with the startup of the kernel (INITiation). The main.c file is the core of the kernel. This is the main source code file the connects all of the other files.
10. ipc - IPC stands for Inter-Process Communication. This folder has the code that handles the communication layer between the kernel and processes. The kernel controls the hardware and programs can only ask the kernel to perform a task. Assume a user has a program that opens the DVD tray. The program does not open the tray directly. Instead, the program informs the kernel that the tray should be opened. Then, the kernel opens the tray by sending a signal to the hardware. This code also manages the kill signals. For illustration, when a system administrator opens a process manager to close a program that has locked-up, the signal to close the program is called a kill signal. The kernel receives the signal and then the kernel (depending on which type of kill signal) will ask the program to stop or the kernel will simply take the process out of the memory and CPU. Pipes used in the command-line are also used by the IPC. The pipes tell the kernel to place the output data on a physical page on in memory. The program or command receiving the data is given a pointer to the page on memory.
11. kernel - The code in this folder controls the kernel itself. For instance, if a debugger needed to trace an issue, the kernel would use code that originated from source files in this folder to inform the debugger of all of the actions that the kernel performs. There is also code here for keeping track of time. In the kernel folder is a directory titled "power". Some code in this folder provide the abilities for the computer to restart, power-off, and suspend.
12. lib - the library folder has the code for the kernel's library which is a set of files that that the kernel will need to reference.
13. mm - The Memory Management folder contains the code for managing the memory. Memory is not randomly placed on the RAM. Instead, the kernel places the data on the RAM carefully. The kernel does not overwrite any memory that is being used or that holds important data.
14. net - The network folder contains the code for network protocols. This includes code for IPv6 and Appletalk as well as protocols for Ethernet, wifi, bluetooth, etc. Also, the code for handling network bridges and DNS name resolution is in the net directory.
15. samples - This folder contains programming examples and modules that are being started. Assume a new module with a helpful feature is wanted, but no programmer has announced that they would work on the project. Well, these modules go here. This gives new kernel programmers a chance to help by going through this folder and picking a module they would like to help develop.
16. scripts - This folder has the scripts needed for compiling the kernel. It is best to not change anything in this folder. Otherwise, you may not be able to configure or make a kernel.
17. security - This folder has the code for the security of the kernel. It is important to protect the kernel from computer viruses and hackers. Otherwise, the Linux system(Android in this case ) can be damaged.
18. sound - This directory has sound driver code for sound/audio cards.
19. tools - This directory contains tools that interact with the kernel.
20. usr - The code in this folder creates vmlinuz and similar files after the kernel is compiled.
21. virt - This folder contains code for virtualization which allows users to run multiple operating systems at once. This is different from Xen (mentioned previously). With virtualization, the guest operating system is acting like any other application within the Linux operating system (host system). With a hypervisor like Xen, the two operating systems are managing the hardware together and the same time. In virtualization, the guest OS runs on top of the Linux kernel while in a hypervisor, there is no guest OS and all of the operating systems do not depend on each other.
PS- If I went wrong at any place, please do tell, I'm tend to make mistakes.
Reserved 2
I believe that 'pdk' is the Platform Development Kit, it's basically an SDK/set of tools that Google sends to OEMs to evaluate their framework ahead of each major Android upgrade since Android 4.1.
Good.
Explicit information:thumbup:
Sent from my GT-I9100 using Tapatalk 2
Just some more info (and a small correction...)
To make a particular app, I don't think
Code:
make name_of_app.apk -j'X'
is the right way to do it...
Search for the packages in "packages" directory, and open up the Android.mk file. You'd see a line that states "LOCAL_PACKAGE_NAME". You need to type in:
Code:
make <insert-value-of-local_package_name-here> -j'X'
In case you are doing this for a particular module/shared library, you'd be searching for "LOCAL_MODULE" or "LOCAL_SHARED_LIBRARY" defines.
Also, you could also always rebuild a particular directory, with the
Code:
mmm -B path/to/dir
eg:
Code:
mmm -b packages/apps/Settings
command. This is especially helpful in case of stuff like libaudio, liblights, etc which are device specific.
Also note that some of the defines and variables stick only till the Terminal's closed. There is a workaround for this, i.e. the 'screen' app, which allows you to save a terminal state. Well, that can wait for another day
GOOD ...GOOD...
guidence detailed and understandable
Thanks
great !!
but could you tell me about propietary files and vendor_blobs.mk ? and how to make it for device that using GB (for example) ?
Awesome !!
This is great.
Thank you.
Thanks for this guide op. Really nice one. :good:
Outstanding job on the guide!! Thank you very much sir. :good:
Thank you very much sir for the detailed explanation........
cleverior.ipul said:
great !!
but could you tell me about propietary files and vendor_blobs.mk ? and how to make it for device that using GB (for example) ?
Click to expand...
Click to collapse
Sorry for the wait but I'll answer this after I reach home.
Sent from my HTC Explorer A310e using XDA Premium 4 mobile app
cleverior.ipul said:
great !!
but could you tell me about propietary files and vendor_blobs.mk ? and how to make it for device that using GB (for example) ?
Click to expand...
Click to collapse
Okay, I'll tell this with an example.
Have a look at this vendor tree. This for Galaxy Nexus(Maguro).
Now, the important makefile, vendor-blobs.mk
In this case, it is:
Code:
# Copyright (C) 2010 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file is generated by device/samsung/maguro/extract-files.sh - DO NOT EDIT
# Prebuilt libraries that are needed to build open-source libraries
PRODUCT_COPY_FILES := \
vendor/samsung/maguro/proprietary/libsecril-client.so:obj/lib/libsecril-client.so
# All the blobs necessary for maguro
PRODUCT_COPY_FILES += \
vendor/samsung/maguro/proprietary/fRom:system/bin/fRom \
vendor/samsung/maguro/proprietary/libsecril-client.so:system/lib/libsecril-client.so \
vendor/samsung/maguro/proprietary/pvrsrvinit:system/vendor/bin/pvrsrvinit \
vendor/samsung/maguro/proprietary/sirfgps.conf:system/vendor/etc/sirfgps.conf \
vendor/samsung/maguro/proprietary/bcm4330.hcd:system/vendor/firmware/bcm4330.hcd \
vendor/samsung/maguro/proprietary/ducati-m3.bin:system/vendor/firmware/ducati-m3.bin \
vendor/samsung/maguro/proprietary/libpn544_fw.so:system/vendor/firmware/libpn544_fw.so \
vendor/samsung/maguro/proprietary/libEGL_POWERVR_SGX540_120.so:system/vendor/lib/egl/libEGL_POWERVR_SGX540_120.so \
vendor/samsung/maguro/proprietary/libGLESv1_CM_POWERVR_SGX540_120.so:system/vendor/lib/egl/libGLESv1_CM_POWERVR_SGX540_120.so \
vendor/samsung/maguro/proprietary/libGLESv2_POWERVR_SGX540_120.so:system/vendor/lib/egl/libGLESv2_POWERVR_SGX540_120.so \
vendor/samsung/maguro/proprietary/gps.omap4.so:system/vendor/lib/hw/gps.omap4.so \
vendor/samsung/maguro/proprietary/gralloc.omap4.so:system/vendor/lib/hw/gralloc.omap4.so \
vendor/samsung/maguro/proprietary/libglslcompiler.so:system/vendor/lib/libglslcompiler.so \
vendor/samsung/maguro/proprietary/libIMGegl.so:system/vendor/lib/libIMGegl.so \
vendor/samsung/maguro/proprietary/libinvensense_mpl.so:system/vendor/lib/libinvensense_mpl.so \
vendor/samsung/maguro/proprietary/libpvr2d.so:system/vendor/lib/libpvr2d.so \
vendor/samsung/maguro/proprietary/libpvrANDROID_WSEGL.so:system/vendor/lib/libpvrANDROID_WSEGL.so \
vendor/samsung/maguro/proprietary/libPVRScopeServices.so:system/vendor/lib/libPVRScopeServices.so \
vendor/samsung/maguro/proprietary/libsec-ril.so:system/vendor/lib/libsec-ril.so \
vendor/samsung/maguro/proprietary/libsrv_init.so:system/vendor/lib/libsrv_init.so \
vendor/samsung/maguro/proprietary/libsrv_um.so:system/vendor/lib/libsrv_um.so \
vendor/samsung/maguro/proprietary/libusc.so:system/vendor/lib/libusc.so
So, in the source code, the vendor tree is kept in /source/vendor/manufacturer/codename(vendor/samsung/maguro in this case) and the makefile parameters tell it to copy the proprietary files(that are the files in the vendor tree, for instance, open up the vendor tree's link I gave above) to certain direstories in the ROM's zip, in this case, for example, this line:
Code:
vendor/samsung/maguro/proprietary/gps.omap4.so:system/vendor/lib/hw/gps.omap4.so \
this tells the builder to copy the proprietary file gps.omap4.so to lib/hw folder in the system.
I hope you'll figure out the remaining and this cleared your doubts.
PS- If I went wrong at some place, correct me.
Great job! Congrats on getting featured on the portal!
The guide is great!
Great guide mate! Maybe you can teach us how to track build error's ?
smileydr0id said:
Great guide mate! Maybe you can teach us how to track build error's ?
Click to expand...
Click to collapse
That are mostly easy to track, few can be easily be fixed by seeing the error's line and jumping to that file, and if you are unable to fix, then Google or this thread will help you.
Awesome job man! Keep going
The libnativehelper folder,
README:
Support functions for Android's class libraries
These are VM-agnostic native functions that implement methods for system
class libraries. All code here:
- MUST not be associated with an android.* class (that code lives in
frameworks/base/).
- SHOULD be written in C rather than C++ where possible.
Some helper functions are defined in include/nativehelper/JNIHelp.h.​
:good:
This is Awesome!
Thanks #Superuser

[GUIDE] Learning about the Android Build Process

Introduction​The Android open-source project (AOSP) is quite complex and it can be hard to find a good way to get more familiar with it. I’m going to try a practical approach, by describing the relevant parts of the build process.
Note - I recommend reading my this guide before you through this one just in case you are actually gonna read and learn.
This guide explains the whole android build process right from envsetup.sh to the working and understanding of makefiles to Package complete!
The Beginning(envsetup.sh)​It all gets started with the
Code:
. build/envsetup.sh
command. What it basically does is that it adds a number of commands to your environment which you later use for building Android for your device.
The commands are listed below -
Code:
- lunch: lunch <product_name>-<build_variant>
- tapas: tapas [<App1> <App2> ...] [arm|x86|mips|armv5] [eng|userdebug|user]
- croot: Changes directory to the top of the tree.
- cout: Changes directory to out.
- m: Makes from the top of the tree.
- mm: Builds all of the modules in the current directory.
- mmp: Builds all of the modules in the current directory and pushes them to the device.
- mmm: Builds all of the modules in the supplied directories.
- mmmp: Builds all of the modules in the supplied directories and pushes them to the device.
- mma: Builds all of the modules in the current directory, and their dependencies.
- mmma: Builds all of the modules in the supplied directories, and their dependencies.
- cgrep: Greps on all local C/C++ files.
- jgrep: Greps on all local Java files.
- resgrep: Greps on all local res/*.xml files.
- godir: Go to the directory containing a file.
- cmremote: Add git remote for CM Gerrit Review.
- cmgerrit: A Git wrapper that fetches/pushes patch from/to CM Gerrit Review.
- cmrebase: Rebase a Gerrit change and push it again.
- aospremote: Add git remote for matching AOSP repository.
- cafremote: Add git remote for matching CodeAurora repository.
- mka: Builds using SCHED_BATCH on all processors.
- mkap: Builds the module(s) using mka and pushes them to the device.
- cmka: Cleans and builds using mka.
- repolastsync: Prints date and time of last repo sync.
- reposync: Parallel repo sync using ionice and SCHED_BATCH.
- repopick: Utility to fetch changes from Gerrit.
- installboot: Installs a boot.img to the connected device.
- installrecovery: Installs a recovery.img to the connected device.
It also scans for the 'vendorsetup.sh' files in your source and display them in this form -
Code:
including device/generic/armv7-a-neon/vendorsetup.sh
including device/generic/goldfish/vendorsetup.sh
including device/generic/mips/vendorsetup.sh
including device/generic/x86/vendorsetup.sh
including vendor/cm/vendorsetup.sh
including sdk/bash_completion/adb.bash
including vendor/cm/bash_completion/git.bash
including vendor/cm/bash_completion/repo.bash
The file is responsible for -
All the make device calling, make etc. commands are possible only after you execute the envsetup.sh file.
It also scans and does some magic stuff with the build/core directory.
It also displays choices of variant for the selected device, like 'userdebug', 'user', 'eng'.
It is also responsible for performing the most important step, it sets up the android build paths and and sets up the toolchains for the build process.
It also calls java and sets it parameters for the compilation.
It displays the default device configurations that are -
Code:
add_lunch_combo aosp_arm-eng
add_lunch_combo aosp_x86-eng
add_lunch_combo aosp_mips-eng
add_lunch_combo vbox_x86-eng
It calls the 'roomservice.py' file in build/tools folder which is responsible for finding and downloading the device sources from CyanogenMod's github if they don't exist.
The file also contains the CWM and TWRP commands to sideload the product files into the device.
It also gets bug reports from the device to the machine for the developers to work upon.
The lunch/brunch Command​Next thing is to select target using the lunch menu, which was added to your bash shell environment after sourcing envsetup.sh. After making your selection, the chosen product and variant is verified and environment variables are set, including:
export TARGET_PRODUCT=$product – The chosen product
export TARGET_BUILD_VARIANT=$variant – The chosen variant
export TARGET_BUILD_TYPE=release – Only release type is available. Use choosecombo if you want to select type.
export ANDROID_BUILD_TOP=$(gettop) – The build root directory.
export ANDROID_TOOLCHAIN=... – The toolchain directory for the prebuilt cross-compiler matching the target architecture
export PATH=... – Among other stuff, the prebuilt toolchain is added to PATH.
export ANDROID_PRODUCT_OUT=... – Absolute path to the target product out directory
export ANDROID_HOST_OUT=... – Absolute path to the host out directory
The usage is pretty simply, you can just type 'lunch' and you'll be prompted with the list of devices that you can build for with the sources present or you can just type 'lunch cm_<device_name>-<variant_type>'.
The lunch command actually fixes the target for the build process and is responsible for the paths getting locked and the device configurationto be used and applied to the source.
It actually gets to use only after there is a vendorsetup.sh file in your device tree which is responsible for adding the lunch combo(Which shows up in the list of devices when you simply run lunch) of your device.
The MAKE Command​The purpose of the make utility is to determine automatically which pieces of a large program need to be recompiled, and issue the commands to recompile them. The manual describes the GNU implementation of make, which was written by Richard Stallman and Roland McGrath. The following examples show C programs, since they are most common, but you can use make with any programming language whose compiler can be run with a shell command. In fact, make is not limited to programs. You can use it to describe any task where some files must be updated automatically from others whenever the others change.
To prepare to use make, you must write a file called the makefile that describes the relationships among files in your program, and the states the commands for updating each file. In a program, typically the executable file is updated from object files, which are in turn made by compiling source files.
Once a suitable makefile exists, each time you change some source files, this simple shell command:
Code:
make
suffices to perform all necessary recompilations. The make program uses the makefile data base and the last-modification times of the files to decide which of the files need to be updated. For each of those files, it issues the commands recorded in the data base.
make executes commands in the makefile to update one or more target names, where name is typically a program. If no -f option is present, make will look for the makefiles GNUmakefile, makefile, and Makefile, in that order.
Normally you should call your makefile either makefile or Makefile. (I recommend Makefile because it appears prominently near the beginning of a directory listing, right near other important files such as README.) The first name checked, GNUmakefile, is not recommended for most makefiles. You should use this name if you have a makefile that is specific to GNU make, and will not be understood by other versions of make. If makefile is `-', the standard input is read.
make updates a target if it depends on prerequisite files that have been modified since the target was last modified, or if the target does not exist.
Makefile Execution​
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
The make command executes the commands in the makefile line by line. As make executes each command, it writes the command to standard output (unless otherwise directed, for example, using the -s flag). A makefile must have a Tab in front of the commands on each line.
When a command is executed through the make command, it uses make's execution environment. This includes any macros from the command line to the make command and any environment variables specified in the MAKEFLAGS variable. The make command's environment variables overwrite any variables of the same name in the existing environment.
Note:
When the make command encounters a line beginning with the word include followed by another word that is the name of a makefile (for example, include depend), the make command attempts to open that file and process its contents as if the contents were displayed where the include line occurs. This behavior occurs only if the first noncomment line of the first makefile read by the make command is not the .POSIX target; otherwise, a syntax error occurs.
Comments: Comments begin with a # character, anywhere but in a shell command line, and continue to the end of the line.
Click to expand...
Click to collapse
Environment: The make command uses the MAKEFLAGS environment variable, if it exists.
Target Rules​
Target rules have the following format:
target[target...] : [prerequisite...] [;command]
<Tab>command
Multiple targets and prerequisites are separated by spaces. Any text that follows the ; (semicolon) and all of the subsequent lines that begin with a Tab character are considered commands to be used to update the target. A new target entry is started when a new line does not begin with a Tab or # character.
You can also look up at the Makefile Tutorial here http://mrbook.org/tutorials/make/ !
That was the most important part of the build process, where make files are resposible for compiling almost each of the files in the source and putting them together for the useful apps/binaries/libraries etc.
Build Tricks​Used from wiki.
Seeing the actual commands used to build the software
Use the "showcommands" target on your 'make' line:
$ make -j4 showcommands
This can be used in conjunction with another make target, to see the commands for that build. That is, 'showcommands' is not a target itself, but just a modifier for the specified build.
In the example above, the -j4 is unrelated to the showcommands option, and is used to execute 4 make sessions that run in parallel.
Make targets
Here is a list of different make targets you can use to build different parts of the system:
make sdk - build the tools that are part of an SDK (adb, fastboot, etc.)
make snod - build the system image from the current software binaries
make services
make runtime
make droid - make droid is the normal build.
make all - make everything, whether it is included in the product definition or not
make clean - remove all built files (prepare for a new build). Same as rm -rf out/<configuration>/
make modules - shows a list of submodules that can be built (List of all LOCAL_MODULE definitions)
make <local_module> - make a specific module (note that this is not the same as directory name. It is the LOCAL_MODULE definition in the Android.mk file)
make clean-<local_module> - clean a specific module
Helper macros and functions
There are some helper macros and functions that are installed when you source envsetup.sh. They are documented at the top of envesetup.sh, but here is information about a few of them:
croot - change directory to the top of the tree
Code:
m - execute 'make' from the top of the tree (even if your current directory is somewhere else)
mm - builds all of the modules in the current directory
mmm <dir1> ... - build all of the modules in the supplied directories
cgrep <pattern> - grep on all local C/C++ files
jgrep <pattern> - grep on all local Java files
resgrep <pattern> - grep on all local res/*.xml files
godir <filename> - go to the directory containing a file
Speeding up the build​You can use the '-j' option with make, to start multiple threads of make execution concurrently.
In my experience, you should specify about 2 more threads than you have processors on your machine. If you have 2 processors, use 'make -j4', If they are hyperthreaded (meaning you have 4 virtual processors), try 'make -j6.
You can also specify to use the 'ccache' compiler cache, which will speed up things once you have built things a first time. To do this, specify 'export USE_CCACHE=1' at your shell command line. (Note that ccache is included in the prebuilt section of the repository, and does not have to be installed on your host separately.)
Building only an individual program or module
If you use build/envsetup.sh, you can use some of the defined functions to build only a part of the tree. Use the 'mm' or 'mmm' commands to do this.
The 'mm' command makes stuff in the current directory (and sub-directories, I believe). With the 'mmm' command, you specify a directory or list of directories, and it builds those.
To install your changes, do 'make snod' from the top of tree. 'make snod' builds a new system image from current binaries.
Setting module-specific build parameters
Some code in Android system can be customized in the way they are built (separate from the build variant and release vs. debug options). You can set variables that control individual build options, either by setting them in the environment or by passing them directly to 'make' (or the 'm...' functions which call 'make'.)
For example, the 'init' program can be built with support for bootchart logging by setting the INIT_BOOTCHART variable. (See Using Bootchart on Android for why you might want to do this.)
You can accomplish either with:
$ touch system/init/init.c
$ export INIT_BOOTCHART=true
$ make
or
$ touch system/init/init.c
$ m INIT_BOOTCHART=true
At last, after makefiles optimize all the processes and build the device specific parts including binaries and libs and apps necessary for it to get booted, the 'system' folder and the 'boot.img' folder are prepared in the out/target/product/device. The META-INF folder is prepared at instance and the system and boot.img are packed into a zip file(whose name is also processed by the makefiles ) and md5 sum prepared. The flashable zip gets prepared only if you run the "brunch" command or "lunch + mka" command.
The Build Tricks aren't *for fun*. This stuff is always gonna help you in the long run!
Credits -
>Firstly, my parents.
>Google, for Android!
>All the people over xda who have written awesome guides and done great works(Because that's what inspires me)
> @galaxyfreak , @speed_bot, @Red Devil, @thewisenerd, @vishal_android freak and all others who helped me every time and answered every(sensible) question of mine.
>The CyanogenMod, AOKP and OmniROM team!
> All those whom I missed.
Have a Happy time learning! ​
Reserved.
Good work. Another guide to feature in portal (?) Tipped
Cheers,
AJ
Awesome. Tipped second. All the best
Sent from my Nexus 5 using Tapatalk
Great work.. Nice guide
Thanks
Sent from xt1033, India
Another great one man!
thanks very useful
Noobs become a developer, is it possible?!
Could I be a developer by following this tutorial incredible? By the way thank you very much for the guidance.
Very good guide. Well written and easy to understand.
Really good guide bro.
Easy to understand and very helpful, thanks for writing it.
AWSM guide!!!
superb guide thread, beautifully explained with diagrams:good:...
got to know some new build tricks u mentioned...
Good work..!
Tipped..
Nice guide... Informative... :good:
This is what I've always wanted :laugh::laugh::laugh: Thanks my friend!
TPD-21 said:
Could I be a developer by following this tutorial incredible? By the way thank you very much for the guidance.
Click to expand...
Click to collapse
Even with all the guides and RTFM sessions, you'd need to go on the path of self-learning (documentation, experimenting, etc), to get better
@v_superuser, Nice guide You always make things easier for n00bs, don't you?
thewisenerd said:
Even with all the guides and RTFM sessions, you'd need to go on the path of self-learning (documentation, experimenting, etc), to get better
@v_superuser, Nice guide You always make things easier for n00bs, don't you?
Click to expand...
Click to collapse
I know the feel, I was one some time back. Thanks Google. Haha!
Nice guide. Thanks for sharing!
Superb guide I believe this is going to be my guide to RoM development... Thanks :good:
Sent from an open source device
Bump!
Bumping this guide. Just simply excellent.
Still relavent
Thanks for the guide,
I waas doing a build for oreo, and came across this guide by google search.
This will get a mention and a link in my how to article
Some lines need an update for oreo and but its really good

[Guide] Building post 2017 android kernel from source

About this guide
I have been building kernels for many machine types and have been fiddling around with custom android kernels for my own phone but to my great frustration the rules changed quite a lot for my new phone. It seems every kernel after 2017 is A LOT harder to build and needs quite some fiddling around te get it to boot. The documentation is terrible the least on the steps and all howtos are all outdated by now. This guide shares with you the secrets to build yourself a modern post 2017 kernel from source. I only own a op7 at this moment but I am pretty sure the rules apply to all devices.
So why do we need to build our own kernel? Well that's simple. I have scrolled through many custom kernels missing a simple module that I need. In my case binfmt. Perhaps you need a specific device driver compiled into your kernel or you want to hack in/patch in some special functions that are only needed by you. Or you love a certain kernel but need that one little adjustment in the configs that this kernel doesn’t have and so on.
Lets get through it step by step but I do assume you know something about Linux, you know how to use the terminal and that you have some knowledge on building binaries from source. Ready? Ok lets get started.
These steps are required to be followed to be successful:
1. Set up a building environment
2. Obtain the kernel source
3. Download the proper tool-chain
4. Adjust configuration files to match your needs
5. Build the actual kernel image
6. Backup your current boot.img!!!
7. Trow the image into a anykernel zip and flash it.
8. Close your eyes prey and run the kernel.
Prerequisites
You will need:
A fairly fast computer with lets say 4 gigs of RAM and a decent CPU.
A workable linux distribution that you can boot into (I prefer ubuntu bionic at this moment).
The kernel sources for your device.
ROOT!!
Lots of spare time.
A attitude that makes you unstoppable.
1. Setting up the building environment
Ok lets first open a terminal. You will need the terminal a lot. As a matter of fact it will be your friend through this guide so better get used to it. Assuming you are on a ubuntu like distribution lets fetch the important packages first
Code:
$sudo apt install build-essential bizon flex git
This command will install some of the prerequisites you need to do anything. More packages will need to be installed on the way but at least you will be able to type $make
Now lets create a working directory. Go into your home folder and create a working directory which will later on contain your kernel sources and tool chain.
Code:
$cd
$mkdir [COLOR="MediumTurquoise"]name-of-your-working-folder[/COLOR]
2. Obtain the kernel source
This one is tricky as my experience have learned me the kernel sources supplied by my own vendor are usually badly documented and contain a few bugs here and there resulting in either build errors or a unbootable kernel. So usually I grab a custom kernel that has minimal changes to the stock kernel and boots by only using a single kernel file without extra kernel modules attached. The custom kernels usually have the nasty building bugs flawed out and thus your chances for success are higher. I used the exkernel in my case and out of respect for the hard work of flang2 I bought the app that came along.
To get the sources google around and find the sources on for example git and click the download or clone button there and copy the link of the depository.
Then simply clone the depository to your working directory so after changing to your working directory type
Code:
$git clone link-you-just-copied
Git will now automatically clone the entire kernel source into the source sub-directory. You can change the name of this directory to make you life easier.
3. Download the proper tool-chain
This is where I lost many days and weeks of my life. Where do we find the proper tool-chain for our kernel?
You basically need 3 compilers
clang
GCC
device-tree-compiler
There are many versions of clang and GCC and you will need to find the one that matches your kernel and builds for your cpu architecture. I will assume in this guide that your CPU is aarch64 or arm64 (both names are used at the same time).
To obtain clang head over to the following address:
"https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/"
now in the tags and branches you can find the version of android you are looking for. If your kernel is for android 9 then use the android 9 tag. Click on it and then click tgz which will provide you with a huge tarball.
Create a folder in your working directory named toolchain en extract the contents of the tarball into the sub-folder clang.
You can try any subversion of clang there is but I recommend to simply peek into the build.config files located in the root of your kernel source directory which usually specify the exact subversion of clang used.
Time to get GCC which is easier since there is only one version right now.
Head over to
"https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/"
and use the correct version here as well. In my case I picked the latest build. Then do the same as for clang but name the sub-folder GCC or whatever you please to do.
And then finally get yourself the device-tree-compiler
Head over to
"https://github.com/dgibson/dtc/tree/v1.4.4"
and clone this depository into ~/your-working-directory/toolchain/
and navigate to the subdir you cloned it into
then build by typing
Code:
$make
any missing package errors will need you to install them using
Code:
$sudo apt install missing-package-name
and do make again until the build completes.
If all went well your tool-chain and working directory will look like this:
Code:
/[COLOR="mediumturquoise"]current-working-dir[/COLOR]/[COLOR="mediumturquoise"]kernel-source-dir[/COLOR]/
|
/toolchain/
|
/clang/
/gcc/
/dtc/
4. Adjust configuration files to match your needs
Lets first grab the correct defconfig file and setup the build directory. This defconfig file contains the default kernel configuration parameters which you propably want to adjust.
Head over to ~/working-directory/kernel-source-dir/arch/arm64/configs/
see what you can find there. I found the elementalX_defconfig file which is the file I needed to build the Exkernel from scratch. If you are trying a stock kernel dive a little bit deeper into the vendor folder and see if you can find a good config file there.
Alternatively sometimes the config is hidden on your device itself. Have a look at
/proc/config.gz
the file inside the archive contains a valid defconfig for your device but beware your may need to open it with menuconfig first and save it again to make it workable for the compiler script.
Place the defconfig file you like to use into the configs folder.
(Skip the following step if your are going to build your kernel for the first time because we first want to see it work.)
Now head over to your kernel source dir and type the following command
Code:
$PATH="~/[COLOR="mediumturquoise"]working-dir[/COLOR]/toolchain/clang/[COLOR="mediumturquoise"]clang-r353983c[/COLOR]/bin:~/[COLOR="mediumturquoise"]working-dir[/COLOR]/toolchain/gcc/a[COLOR="mediumturquoise"]arch64-linux-android-4.9[/COLOR]/bin:${PATH}" \
make menuconfig O=out \
ARCH=arm64 \
CC=clang \
CLANG_TRIPLE=aarch64-linux-gnu- \
CROSS_COMPILE=aarch64-linux-android-
now remember to use the correct paths to your tool-chain. In this command I use clang-r353983c but you may use a different version. The same applies for linux-android-4.9.
I would like to thank Nathan Chance for supplying documentation on the clang tool-chain on his github.
If all well you will be presented with a menu in which you can adjust things. First open the defconfig file you wish to use and then save the file after you are done. Now REMEMBER! With this command the menuconfig will save your new file into the /kernel-source-dir/out folder.
(end of the skipped step)
Now it is time to prepare our kernel build using the following command (do this from your kernel source dir) and remember that this command will look in the /kernel-source-dir/arch/arm64/configs/ directory
$make O=out ARCH=arm64 name-of-your-defconfig
If all goes well you will see something like configuration written to .config
Everything is up and ready to go. Your hart will start pounding from this moment on
5. Build the actual kernel image
This is where things get complicated. I hope you have many years to live because this part consumes time, a lot of it.
type
Code:
$PATH="~/[COLOR="mediumturquoise"]working-dir[/COLOR]/toolchain/clang/[COLOR="mediumturquoise"]clang-r353983c[/COLOR]/bin:~/[COLOR="mediumturquoise"]working-dir[/COLOR]/toolchain/gcc/[COLOR="mediumturquoise"]aarch64-linux-android-4.9[/COLOR]/bin:${PATH}" \
make -j$(nproc --all) O=out \
ARCH=arm64 \
CC=clang \
DTC=~/[COLOR="mediumturquoise"]working-directory[/COLOR]/toolchain/dtc/dtc \
CLANG_TRIPLE=aarch64-linux-gnu- \
CROSS_COMPILE=aarch64-linux-android-
ready set GO!
This will depending on your machine take between forever and eternally. You will witness many warnings on the DTC build seems normal these days and a few warnings on the CC part. Most important is that no errors are thrown at you.
If all goes well you will see a normal exit status and you will have a “working” kernel image.
“Error?: Well that happens. Try a different build of clang check your command line. And if all that fails try to find out what is wrong in the source and that means digging through thousands of forum pages until you find out whats wrong. But using the google tools usually goes well.”
Almost there go and check
kernel-source-dir/out/arch/arm64/boot
and there should be a image-dtb or image.gz-dtb file depending on you settings.
That is your kernel image right there. The difference in size between image and image-dtb should not be huge. 10Megs in difference usually means your dtb is not good but trying is the only way to find out if it works.
6. Backup your current boot.img!!!
You know what to do here right? Do not skip this step unless you like bricked devices or want to reflash and lose your data and all that kind of stuff. Not sure what you are doing stop here or backup your entire device including system vendor etc.
7. Trow the image into a anykernel zip and flash it.
Ok something changed in the last few years. Unpacking repacking and booting using fastboot somehow gives me problems. Dm-verity errors and all kind of red screens. Signing the boot image results into new errors. Well this is how I did it.
Get yourself a anykernel zip file. I used the Exkernel.zip because it only contains a kernel image which I like. Open the zip in a good zip tool (I used ark) and replace the image-dtb file with the one you created. Place this new zip you created on a memory stick and then….
Flash it using twrp or any tool of choice.
8. Close your eyes prey and run the kernel.
Two things can happen.
1. Blank screen nothing happens. Only god can help you, repeat all the steps.
2. Your android starts booting. Start crying of joy
Check in your android if this is indeed the kernel you build. If so time to make some adjustments to your configs or happily enjoy your boosted phone.
Now please remember. If you plan to distribute your kernel that you do the correct steps of accrediting the original programmer and trow the source online. If you use a already custom kernel please respect the hard work of the maker and communicate your plans with him/her.

Categories

Resources