Database Info

[TUTORIAL/DEV]Build AOSP Android 2.3.7 for Allwinner A10 tablets (Teclast P76Ti)

Hi!
I haven't really found any kind of guides on how to build a complete AOSP android build from source (kernel and software) for Allwinner A10 tablets like the Ainol Novo 7 Advanced or the Teclast P76Ti.
I have a Teclast P76TI (rev4), and the following guide is only tested on such a device. As it's mostly based on the source drop of the Ainol Novo 7 Advanced, it should also work on that too.

Part 1
Part 1: Preparations
A: Preconditions
For building and creating a LiveSuite flashable package you will need the following:
For building the kernel and android you will need a 64 bit Linux machine. Ubuntu 10.04 64 bit is preferred for Gingerbread builds
For creating the LiveSuite flashable images, and actually flashing you will need a Windows machine. Windows 7 x64 works fine, as should Windows XP 32 bit too.
You will also need a working LiveSuite image file, for your tablet. Make sure that you can flash it to your device without problems!
Virtual machines are fine, if you have enough memory available. To build Android in a VM, you should give it at least 2GB of RAM. Alternatively if you're using Linux, you can create a Windows VM for the image creation or flashing part.
For the rest of the tutorial I will assume you are using a Linux Virtual Machine on a Windows box.
For building the kernel and AOSP you will need at least 35GB of free space in your linux machine (the more the better). To create the images you will need an additional 2GB of space on your Windows machine.
B: Getting the kernel and AOSP sources
First initialize your Linux machine according to this page: http://source.android.com/source/initializing.html
Here are the most important commands:
Code:
$ sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
$ sudo apt-get update
$ sudo apt-get install sun-java6-jdk
$ sudo apt-get install git-core gnupg flex bison gperf build-essential \
zip curl zlib1g-dev libc6-dev lib32ncurses5-dev ia32-libs \
x11proto-core-dev libx11-dev lib32readline5-dev lib32z-dev \
libgl1-mesa-dev g++-multilib mingw32 tofrodos python-markdown \
libxml2-utils xsltproc
$ mkdir ~/bin
$ PATH=~/bin:$PATH
$ curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > ~/bin/repo
$ chmod a+x ~/bin/repo
// Additonal commands for Ubuntu 10.10:
$ sudo ln -s /usr/lib32/mesa/libGL.so.1 /usr/lib32/mesa/libGL.so
// Additonal commands for Ubuntu 11.10:
$ sudo apt-get install libx11-dev:i386
Now create a directory on your linux machine. The path should contain no spaces in it. Inside this directory first pull the kernel source from the allwinner github page:
Code:
$ git clone https://github.com/allwinner/linux-2.6.36 lichee
Make sure that the kernel resides in the lichee directory
Now download AOSP Android. We will use Gingerbread version 2.3.7:
Code:
$ mkdir android
$ cd android
$ repo init -u https://android.googlesource.com/platform/manifest -b android-2.3.7_r1
$ repo sync
Allwinner uses a special init, which has a few additional commands that you have to download too:
Code:
$ cd system/core
$ git pull git://github.com/sztupy/allwinner_android_system_core.git
$ cd ../..
You will also need the device descriptors for the tablets:
Code:
$ cd device
$ git clone git://github.com/sztupy/android_device_softwinner.git softwinner
$ cd ..
C: ARM compiler
You might simply use a compiler that is supplied for AOSP, they are inside the android/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/ directory. Alternatively you can also download the ARM compiler of CodeSourcery. Get the arm-2009q3 / arm-none-linux-gnueabi version, and unpack it inside a directory.
If ready, go on to next step: building the kernel

Part 2: Building the kernel
Change to the directory of the kernel. Now edit the file scripts/build_sun4i_crane.sh
Change the line "export CROSS_COMPILE=" to point to the ARM compiler. For example if you extracted the CodeSourcery files to /opt then use:
Code:
export CROSS_COMPILE=/opt/arm-2009q3/bin/arm-none-linux-gnueabi-
The default .config file is more or less the same that is supplied with your tablet. On the Teclast P76TI you can actually extract the .config file used from /proc using "adb pull /proc/config.gz" Not sure about other vendors.
Now you have to compile. Issue the command:
Code:
./build.sh -p sun4i_crane
This will build the kernel and modules inside the the "output" directory.
Next step is to compile Android

Part3
Part 3: Building Android
Go to the android directory. Now setup android using:
Code:
$ source build/envsetup.sh
$ lunch
You will get simething like this:
Code:
You're building on Linux
Lunch menu... pick a combo:
1. generic-eng
2. simulator
3. full_passion-userdebug
4. full_crespo4g-userdebug
5. full_crespo-userdebug
6. crane_Ainol_Novo7A-eng
7. crane_Teclast_P76TI_REV4-eng
Which would you like? [generic-eng]
Here select your device (option 6 or 7 depending on your tablet)
After you have selected it issue a make command:
Code:
$ make
This will take a while. If you're using ubuntu 11.10 make might fail, but you can fix it using these fixes: http://groups.google.com/group/android-building/browse_thread/thread/3484e7797909d014
After the building is complete you have to create the image files. Run the following command:
Code:
$ ./device/softwinner/crane-tcp76ti_r4/mkfs.sh
(substitute tcp76ti_r4 with ainovo7a if you're using the other tablet)
The image files will be ready in the directory "out/target/product/crane-tcp76ti_r4/images/" or "out/target/product/crane-ainovo7a/images/"
Copy the three files from here (root.img, recovery.img, system.img), and the "bImage" file from the "output" directory of the kernel to your Windows machine.
The next step is creating the flashable image, and flashing your build

Part 4: Flashing
A: Initialize kitchen
First download my A10 flash kitchen from this site: http://android.sztupy.hu/dl/a10/a10_flash_kitchen_v1.zip
This is based on the ainol novo 7 soruce drop, but only contains the nescessary files, and has a few batch scripts to automate the process.
There is also a v2, wich also supports ICS firmwares: http://android.sztupy.hu/dl/a10/a10_flash_kitchen_v2.zip
First unzip the files to a directory. You will see a few directories and to commands here: extract_image.bat and create_image.bat
First we have to extract an official image for your tablet. Simply comply the image file to this directory, and rename it to "original.img". If done, run extract_image.bat. This command should extract your image to the _extract directory.
Next you need to extract the bootfs inside the _bootfs directory. You have multiple options on how to do this:
1. Get the bootfs from your device on-line using adb:
Code:
> cd _bootfs
> adb shell
$ cd /
$ mount -o remount,rw -t ext4 /dev/root /
$ mkdir bootfs
$ mount -t vfat /dev/block/nanda /bootfs
(Control-C: exit adb shell)
> adb pull /bootfs
2. Get the bootfs from the extracted image file using Linux:
A. Copy the RFSFAT16_BOOTFS_000000000.fex file to linux
B.
Code:
$ mkdir bootfs
$ sudo mount -o loop RFSFAT16_BOOTFS_000000000.fex bootfs
$ cp -a bootfs b2
$ sudo umount bootfs
C. Copy the contents of the b2 directory to Windows, inside the _bootfs directory of the kitchen
B: Create image
If you have initialized the kitchen, you have to copy the four files from linux (system.img, root.img, recovery.img and bImage) inside the _input directory. You will also need to rename the .img files to .fex (so they should be system.fex, root.fex, recovery.fex).
If you are done with this, then run create_image.bat. It will create an output.img file, that can be flashed to the device using LiveSuite.

Additional information for developers:
1. Difference between AOSP 2.3.4 and Novo7 android 2.3.4
Download the diff file from here: http://android.sztupy.hu/dl/a10/diff_ainovo_aosp.gz
Filelist of differences: http://android.sztupy.hu/dl/a10/diff_ainovo_aosp_filelist.txt

sztupy, did you figure out why eDragonEx and FSBuild have an apparently unused Lua library next to each? I was interested by it, as it might be possible that some parts of the apps are actually written in Lua (would it be possible? Lua in a native DLL?), what would help reversing it.

fonix232 said:
sztupy, did you figure out why eDragonEx and FSBuild have an apparently unused Lua library next to each? I was interested by it, as it might be possible that some parts of the apps are actually written in Lua (would it be possible? Lua in a native DLL?), what would help reversing it.
Click to expand...
Click to collapse
Yeah, saw that. Also their build script used "convert" on the image.cfg, but that file wasn't in lua anyway (except for the image parts part which seems to be a lua hash).
There are also a lot of compiled files that are compiled inside the image file and which are neither the bootfs nor android or the kernel. I don't yet know what they are for, but I think they are used by LiveSuite during the flashing.

sztupy said:
Yeah, saw that. Also their build script used "convert" on the image.cfg, but that file wasn't in lua anyway (except for the image parts part which seems to be a lua hash).
There are also a lot of compiled files that are compiled inside the image file and which are neither the bootfs nor android or the kernel. I don't yet know what they are for, but I think they are used by LiveSuite during the flashing.
Click to expand...
Click to collapse
I know most of the files from the flashing, they are the following:
- SYS_CONFIG is used as a command bunch for LiveSuite. It tells the app how to flash, what to flash, where, and it configures the device too (screen size, ram info, cpu info, etc)
- Boot0 and Boot1 bins are NAND bootloaders
- FED FES and FET tools are NAND flashing utilities, checksums, hardware scanner, and other tools used during flashing.
- '12345678' files are bootloaders, config files, and tools for SDMMC flashing, if there's a device with SDMMC internal instead of NAND, these are used
- Split seems to be some kind of ID string, I had no luck retrieving it's usage and meaning.
So you say, that the actual Lua parts are the image encryption parts of eDragonEx? Interesting, maybe I've missed that spot with my tool...
Might I ask if you have tried disassembling (and decompiling) unimg.exe? I had several problems with it, but that would greatly help understanding how the images are created. I've got a C# framework, with image config parser, etcetera etcetera, to be able to read and create images in a much more advanced environment (filtering user errors, having tools for everything (bootfs modding, script.bin reversal, etcetera), and creating a working image as a final result), and it only needs the image file format (and some of my work, to create a parser).
The problems with unimg were all about positive sp values, and as I'm not a big assembly programmer, I couldn't make out anything from that. Maybe you understand it a bit more
Illetve beszélhetünk egy kicsit magyarul is. Tabletrepublic-on írtam hogy vegyél fel MSNre és részletezem a különböző Crane SDK elemek funkcióit, működését, egyebeket, amit eddig sikerült kiderítenem az egészről.
Sajnos elég zavaros, mivel vagy négyféle csomagoló rendszer készíti a fileokat, és ezek közül csak egy működött rendesen (crane_pack.exe). Jó lenne megérteni ezt a file formátumot, hogy egy kicsit normálisabb módon hozhassam létre, különféle vacakolások nélkül.

fonix232 said:
I know most of the files from the flashing, they are the following:
- SYS_CONFIG is used as a command bunch for LiveSuite. It tells the app how to flash, what to flash, where, and it configures the device too (screen size, ram info, cpu info, etc)
- Boot0 and Boot1 bins are NAND bootloaders
- FED FES and FET tools are NAND flashing utilities, checksums, hardware scanner, and other tools used during flashing.
- '12345678' files are bootloaders, config files, and tools for SDMMC flashing, if there's a device with SDMMC internal instead of NAND, these are used
- Split seems to be some kind of ID string, I had no luck retrieving it's usage and meaning.
Click to expand...
Click to collapse
Thanks for these. Seems I was mostly right
So you say, that the actual Lua parts are the image encryption parts of eDragonEx? Interesting, maybe I've missed that spot with my tool...
Click to expand...
Click to collapse
I think lua is not really used anymore. It probably had more relevance back in the past.
Might I ask if you have tried disassembling (and decompiling) unimg.exe? I had several problems with it, but that would greatly help understanding how the images are created. I've got a C# framework, with image config parser, etcetera etcetera, to be able to read and create images in a much more advanced environment (filtering user errors, having tools for everything (bootfs modding, script.bin reversal, etcetera), and creating a working image as a final result), and it only needs the image file format (and some of my work, to create a parser).
The problems with unimg were all about positive sp values, and as I'm not a big assembly programmer, I couldn't make out anything from that. Maybe you understand it a bit more
Click to expand...
Click to collapse
No, haven't tried disassembling it yet. I was very glad that it worked, and that I could create a whole working build just from the sources. I know there are a lot of quirks, like if the extension of the file is not .fex, then it will encrypt(?) it, etc. I might try it, but currently I'm more interested in getting a working AOSP ICS on my tablet. Besides for disassembly I need to be in a special mood, which I'm not really in now
Illetve beszélhetünk egy kicsit magyarul is. Tabletrepublic-on írtam hogy vegyél fel MSNre és részletezem a különböző Crane SDK elemek funkcióit, működését, egyebeket, amit eddig sikerült kiderítenem az egészről.
Sajnos elég zavaros, mivel vagy négyféle csomagoló rendszer készíti a fileokat, és ezek közül csak egy működött rendesen (crane_pack.exe). Jó lenne megérteni ezt a file formátumot, hogy egy kicsit normálisabb módon hozhassam létre, különféle vacakolások nélkül.
Click to expand...
Click to collapse
Már egy jó ideje nem használok MSN-t. Skype/GTalk/email viszont van. Ha nem használsz olyanokat, akkor azért majd felrakom.

Made a diff between AOSP android 2.3.4 and the Novo 7 2.3.4 source drop. The list can be found at post 6: http://forum.xda-developers.com/showpost.php?p=22397984&postcount=6

Nice tutorial! Do you mind if I fork your Github repo and add the device tree for the Bmorn V11 to the lunch list?

sztupy said:
Made a diff between AOSP android 2.3.4 and the Novo 7 2.3.4 source drop. The list can be found at post 6: http://forum.xda-developers.com/showpost.php?p=22397984&postcount=6
Click to expand...
Click to collapse
Most of the things changed aren't even needed for the device - ril can be replaced with a local one (and suggested by Google to do so), just like recovery changes, in ICS we already have USB BT support enabler, so BT changes can be dropped, just like framework changes (they are for the softbuttons on the notification bar), camera and mediaplayer changes should be local too (in the device tree), so 99% of changes can be dropped.
Additions are different, some can be totally erased, and some are needed. From your github, I see that you've already began making a cleaned up, generic A10 tree, but I miss a few things - libsensor for one, stagefright, camera, and audio. The AOSP stock ALSA should work, if the proper audio config is placed in a ROM, but I have bad feelings about the missing libsensors source, and camera. Stagefright has a chance too to work, but camera definitely won't, and the used sensors aren't the common ones to be included.
Használok GTalk-ot is, ott is ugyanez a nicknevem, gmail utótaggal, szóval a szokásos

FezzFest said:
Nice tutorial! Do you mind if I fork your Github repo and add the device tree for the Bmorn V11 to the lunch list?
Click to expand...
Click to collapse
That's what github is for

fonix232 said:
Most of the things changed aren't even needed for the device - ril can be replaced with a local one (and suggested by Google to do so), just like recovery changes, in ICS we already have USB BT support enabler, so BT changes can be dropped, just like framework changes (they are for the softbuttons on the notification bar), camera and mediaplayer changes should be local too (in the device tree), so 99% of changes can be dropped.
Additions are different, some can be totally erased, and some are needed. From your github, I see that you've already began making a cleaned up, generic A10 tree, but I miss a few things - libsensor for one, stagefright, camera, and audio. The AOSP stock ALSA should work, if the proper audio config is placed in a ROM, but I have bad feelings about the missing libsensors source, and camera. Stagefright has a chance too to work, but camera definitely won't, and the used sensors aren't the common ones to be included.
Használok GTalk-ot is, ott is ugyanez a nicknevem, gmail utótaggal, szóval a szokásos
Click to expand...
Click to collapse
libsensor and stagefright are still there, as I could manage them to get compiled. Not that they work though. For the libcamera it depends on CedarX, which I couldn't manage to compile, that's why I removed it (for now). Besides CedarX unfortunately not "open-source", so in theory we couldn't use it either (well... not that I actually care about licence violations).
I'm still compiling ICS. Will put back libcamera and try to get the other hardware libs to work after I managed to get ICS to boot.

sztupy said:
libsensor and stagefright are still there, as I could manage them to get compiled. Not that they work though. For the libcamera it depends on CedarX, which I couldn't manage to compile, that's why I removed it (for now). Besides CedarX unfortunately not "open-source", so in theory we couldn't use it either (well... not that I actually care about licence violations).
I'm still compiling ICS. Will put back libcamera and try to get the other hardware libs to work after I managed to get ICS to boot.
Click to expand...
Click to collapse
Please be noted that ICS requires new stagefright, camera HAL, new GPU drivers, and so on.
ICS also should have some differences in the build tree, make a new branch for sure (as an example, it requires a device.mk and device_base.mk, both being the base containers without target definition, full_[devicename].mk for the actual full target, and cm.mk for CyanogenMod, what I'd suggest you to build).
I couldn't find any of the sources, but must have overlooked something. Will check it further.

hi, I'm the author of unimg(esxgx).
and your unimg is not up-to-date(the version still have bugs to lead to fail the packing process.
Here is the latest version (fix bugs but no virus alarm[compared with the former version]), and you can use it in the same way.
please update your file.
PS. yes, the unimg can unpack and pack all allwinner's firewares without diffculty, good luck.
sorry for my english.

fonix232 said:
Might I ask if you have tried disassembling (and decompiling) unimg.exe? I had several problems with it, but that would greatly help understanding how the images are created. I've got a C# framework, with image config parser, etcetera etcetera, to be able to read and create images in a much more advanced environment (filtering user errors, having tools for everything (bootfs modding, script.bin reversal, etcetera), and creating a working image as a final result), and it only needs the image file format (and some of my work, to create a parser).
The problems with unimg were all about positive sp values, and as I'm not a big assembly programmer, I couldn't make out anything from that. Maybe you understand it a bit more
Click to expand...
Click to collapse
I think you should use unimg.exe.
unimg is the only tool to unpack it correctly.
you know, allwinner didn't want me to release the tool in public last year(the tool is for sc9800[the former chip]). for some commercial reason, the offical toolchain of a10 only contains pack_tool.
so I released the tool with other tools(rootcr, rootpk,etc..) in a small group.
but some person posted it on the internet. so....
I developed the analysis tools in the form of many files(unimg, rootcr, rootpk, unimg2), because I want to keep each of the packing stages simple.
you can use a "bat" file / a GUI shell to communicate with each other, and that is what I expected.
I can't smoothly speak english, so I modified several times.

Thank you very much!
If I may ask, would it be possible to release the source code too?

fonix232 said:
Thank you very much!
If I may ask, would it be possible to release the source code too?
Click to expand...
Click to collapse
I cannot released yet, but I will release it in a proper time on github (I have been touching with allwinner company, so I have to consider many factors).

[GUIDE] How to compile Sidekick 4G Kernel [Ubuntu]

1) download source code from https://opensource.samsung.com/index.jsp. You are looking for source code for SGH-T839.
2) Get initramfs (Need to make a kernel package)
Extract it using extract boot http://www.mediafire.com/?lc12eceeh617b97.
This is why I am looking for a boot.img
extract it
Code:
tar -xvf extractboot.tar.gz
now move into directory with extract boot and copy your boot.img into here and do
Code:
./extractboot boot.img
3) Get mkboot tools
http://www.mediafire.com/?w06d1m6n1dgo4op
untar it by doing
Code:
tar -xvf $FILENAMEHERE
Add the bin directory to your path by moving to the bin directory and copying down the path then
Now you will add this to your path by editing your .bashrc file.
Go to your bashrc file
Code:
gedit ~/.bashrc
and adding this
Code:
PATH=$PATH:/FULLDIRECTORYYOUWROTEDOWN/
export PATH
4) Download the ARM toolchain
https://sourcery.mentor.com/sgpp/lite/arm/portal/package5385/public/arm-none-linux-gnueabi/arm-2009q3-67-arm-none-linux-gnueabi.bin
and
https://sourcery.mentor.com/sgpp/lite/arm/portal/package5355/public/arm-none-eabi/arm-2009q3-68-arm-none-eabi.bin
5) Install the ARM Toolchain
create the directory /opt/toolchains/arm-2009q3/
Code:
sudo mkdir /opt/toolchains/arm-2009q3/
then install the toolchain using /opt/toolchains/arm-2009q3/
as the install directory
Code:
sudo chmod +x arm-2009q3-68-arm-none-eabi.bin
sudo chmod +x arm-2009q3-67-arm-none-linux-gnueabi.bin
sudo ./arm-2009q3-67-arm-none-linux-gnueabi.bin -i console
sudo ./arm-2009q3-68-arm-none-eabi.bin -i console
6) Compile
Extract your source code and go to the directory Kernel and do the following
WARNING: MAKE SURE THERE ARE NO SPACES IN YOUR FILEPATH BECAUSE THE MAKEFILE DOESNT LIKE THEM.
Code:
make clean
make arch=arm sidekick_rev02_defconfig
make ARCH=arm HOSTCFLAGS="-g -O3" -j8 CROSS_COMPILE=/opt/toolchain/bin/arm-none-eabi-
Now copy any of the resulting compiled ko files into the initramfs file you have extracted and you should have what you need to package a kernel.

you forget initramfs

windxixi said:
you forget initramfs
Click to expand...
Click to collapse
yeah I kept it out so that somebody would post a boot.img then could do steps related to that. I am looking for a boot.img to get initramfs from

unpack zImage
Sent from my SGH-T839 using XDA
---------- Post added at 08:46 AM ---------- Previous post was at 07:49 AM ----------
and how to pack a boot.img?

A request to anyone building SK4G kernels.
Please disable the keystroke logging printk statements in the file:
Code:
drivers/input/keyboard/s3c-keypad.c
The lines look like this:
Code:
//printk("\nkey Pressed : key %d map %d\n",i, pdata->keycodes[i]);
and
Code:
//printk("\nkey Released : %d map %d\n",i,pdata->keycodes[i]);
It is possible to recover the actual keystrokes from the numerical codes those statements log, and the messages go into the dmesg buffer. So it's pretty easy to extract them and determine exactly what the user typed.
In the latest Samsung sources I've seen, those lines were already commented out. It makes sense to enable them while debugging a new ROM build, but please do disable them prior to building a kernel intended for general consumption.

Do you guys know if the available source code will produce a kernel that will work with kj2? the kernel version in SGH-T839_Opensource_Update1 looks to match up, but I compilied a zImage and it didn't boot on stock kj2. but I could very well be missing something.
I have done a small amount of kernel work on an HTC device, but I basically just used the Rom Kitchen to pack up my zImages with a boot.img-ramdisk to create a boot.img. I guess I might need a little more instruction for packing up a Samsung kernel. Is it also an option to just tar up the zImage and flash it with Odin/Heindall?
Thanks for this thread, and for any other advice!
Sent from my SGH-T839 using Tapatalk 2

In case it might be useful to someone else working on building a kernel.
The official and Bali_SK4G sources both seem to insist on building with debug symbols enabled for some of the modules. In particular, i was ending up with dhd.ko being 2.4 MB in size, where it should have been less than 400 KB.
The ideal case would be to determine why the debug symbols are being included -- commenting out the labelled debug options in the bcm4329 Makefile didn't accomplish this.
But a workaround is to strip the modules after the build is finished, before assembling the initramfs.

Has anyone been able to build a working zImage for KJ2 using Dr. Honk's Bali sources [1] and sduvick's KJ2 ramdisk files [2]?
I have been able to build a zImage of a reasonable size (6520 KB). But when I flash this to the KERNEL partition using heimdall, the device boot loops to the B&W Sidekick logo. It doesn't get far enough to show any adb log output.
I can then use the same heimdall flashing procedure to flash the Platypus Egg v1 zImage, or other KJ2-compatible zImage files, and the device boots and works properly.
So I'm trying to figure out what I'm doing wrong in building my zImage. If anyone has any advice I would be appreciative.
[1] https://github.com/drhonk/Bali_SK4G
[2] https://github.com/sduvick/SK4g_KJ2_Ramdisk

I got an updated Bali_SK4G kernel booting using the ramdisk from GenericGinger 2.0.
I have worked up some patches to disable the logging of keystrokes and other more trivial debug spew in dmesg. Also included is a Makefile patch that was required for the compile to complete with my toolchain.
https://carbon.flatlan.net/nxd/patches_Bali_SK4G_nxd.tar.bz2
md5sum: 5d14ac32de155cdca0fd82f14bc4ceca
These patches are GPL licensed, in compliance with the license for the Linux kernel itself. I make no guarantees about their suitability for any purpose. I grant permission to use them to anyone who would like to do so, so long as they comply with the GPL.
I'd like to make a compiled kernel available with these changes, but XDA's rules can be interpreted to mean that I must obtain permission from a series of upstream contributors, some of whom may not be reachable. Perhaps a moderator will clarify the parameters of the permission rule.

[DEV] [GUIDE] [LINUX] Comprehensive Guide to Cross-Compiling

[SIZE="+1"]What is a Cross-Compiler?[/SIZE]
A cross compiler is a compiler capable of creating executable code for a platform other than the one on which the compiler is running. Cross compiler tools are used to generate executables for embedded system or multiple platforms. It is used to compile for a platform upon which it is not feasible to do the compiling, like microcontrollers that don't support an operating system. From wikipedia
Click to expand...
Click to collapse
[SIZE="+1"]How is that connected with an Android?[/SIZE]
In order to create a native C/C++ binary for an Android, you must firstly compile the source code. Usually you can't do so on an Android itself due to lack of proper tools and environment, or hardware barriers, especially amount of RAM. This is why you should learn how to cross-compile, to create a binary on your PC, that your ARM-based Android will understand.
[SIZE="+1"]Why do I need it?[/SIZE]
You need to learn cross-compiling technique if you want to run native C/C++ programs on an Android. Actually, if you've already built your own custom ROM from AOSP sources (i.e. CyanogenMod), then you used cross-compiling tools and methods even without noticing .
Building an AOSP ROM is fairly easy, there's one command like brunch, which does the job. However, what if you want to compile a custom, not natively included binary? This is the purpose of this tutorial.
[SIZE="+1"]What I will learn from this guide?[/SIZE]
How to properly set C/C++ building environment
How to build a native C/C++ application for Android device
How to optimize native binaries for my device
[SIZE="+1"]Step 1 - The Beginning[/SIZE]
You should start from installing any Linux-based OS, I highly suggest trying a Debian-based distro (such as Ubuntu), or even Debian itself, as this tutorial is based on it .
In general, I highly suggest to compile an AOSP ROM (such as CyanogenMod) for your device firstly. This will help you to get familiar with cross-compiling on Android. I also suggest to compile one or two programs from source for your Linux, but if you're brave enough to learn cross-compiling without doing any of these, you can skip those suggestions .
[SIZE="+1"]Step 2 - Setting up[/SIZE]
Firstly you should make sure that you have all required compile tools already.
[email protected]:~# apt-get update && apt-get install checkinstall
Click to expand...
Click to collapse
This should do the trick and install all required components.
I suggest creating a new folder and navigating to it, just to avoid a mess, but you can organize everything as you wish.
Start from downloading NDK from here.
The NDK is a toolset that allows you to implement parts of your app using native-code languages such as C and C++.
Click to expand...
Click to collapse
[email protected]:~# wget http://dl.google.com/android/ndk/android-ndk-r9d-linux-x86_64.tar.bz2
[email protected]:~# tar xvf android-ndk-r9d-linux-x86_64.tar.bz2
[email protected]:~# mv android-ndk-r9d ndk
Click to expand...
Click to collapse
Now you should make a standalone toolchain, navigate to root of your ndk (this is important) and then build your toolchain:
[email protected]:~# cd ndk/
[email protected]:~/ndk# build/tools/make-standalone-toolchain.sh --toolchain=arm-linux-androideabi-4.8 --platform=android-18 --install-dir=/root/ndkTC
Copying prebuilt binaries...
Copying sysroot headers and libraries...
Copying libstdc++ headers and libraries...
Copying files to: /root/ndkTC
Cleaning up...
Done.
Click to expand...
Click to collapse
You should edit bolded variables to your preferences. Toolchain is the version of GCC you want to use, 4.8 is currently the newest one, in the future it may be 4.9 and so on. Platform is a target API for your programs, this is important only for android-specific commands, such as logging to logcat. When compiling a native Linux program, this won't matter (but it's a good idea to set it properly, just in case). Install dir specifies destination of your toolchain, make sure that it's other than ndk (as you can see I have ndk in /root/ndk and toolchain in /root/ndkTC).
Now you need to download my exclusive cc.sh script from here and make it executable.
[email protected]:~# wget https://dl.dropboxusercontent.com/u/23869279/Files/cc.sh
[email protected]:~# chmod 755 cc.sh
Click to expand...
Click to collapse
This script is a very handy tool written by me to make your life easier while cross-compiling. Before running it make sure to edit "BASIC" options, especially NDK paths. Apart from that it's a good idea to take a look at DEVICEFLAGS and setting them properly for your device, or clearing for generic build. You don't need to touch other ones unless you want/need them.
Just for a reference, I'll include currently editable options:
#############
### BASIC ###
#############
# Root of NDK, the one which contains $NDK/ndk-build binary
NDK="/root/ndk"
# Root of NDK toolchain, the one used in --install-dir from $NDK/build/tools/make-standalone-toolchain.sh. Make sure it contains $NDKTC/bin directory with $CROSS_COMPILE binaries
NDKTC="/root/ndkTC"
# Optional, may help NDK in some cases, should be equal to GCC version of the toolchain specified above
export NDK_TOOLCHAIN_VERSION=4.8
# This flag turns on ADVANCED section below, you should use "0" if you want easy compiling for generic targets, or "1" if you want to get best optimized results for specific targets
# In general it's strongly suggested to leave it turned on, but if you're using makefiles, which already specify optimization level and everything else, then of course you may want to turn it off
ADVANCED="1"
################
### ADVANCED ###
################
# Device CFLAGS, these should be taken from TARGET_GLOBAL_CFLAGS property of BoardCommonConfig.mk of your device, eventually leave them empty for generic non-device-optimized build
# Please notice that -march flag comes from TARGET_ARCH_VARIANT
DEVICECFLAGS="-march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp"
# This specifies optimization level used during compilation. Usually it's a good idea to keep it on "-O2" for best results, but you may want to experiment with "-Os", "-O3" or "-Ofast"
OLEVEL="-O2"
# This specifies extra optimization flags, which are not selected by any of optimization levels chosen above
# Please notice that they're pretty EXPERIMENTAL, and if you get any compilation errors, the first step is experimenting with them or disabling them completely, you may also want to try different O level
OPTICFLAGS="-s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing"
# This specifies extra linker optimizations. Same as above, in case of problems this is second step for finding out the culprit
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections"
# This specifies additional sections to strip, for extra savings on size
STRIPFLAGS="-s -R .note -R .comment -R .gnu.version -R .gnu.version_r"
# Additional definitions, which may help some binaries to work with android
DEFFLAGS="-DNDEBUG -D__ANDROID__"
##############
### EXPERT ###
##############
# This specifies host (target) for makefiles. In some rare scenarios you may also try "--host=arm-linux-androideabi"
# In general you shouldn't change that, as you're compiling binaries for low-level ARM-EABI and not Android itself
CONFIGANDROID="--host=arm-linux-eabi"
# This specifies the CROSS_COMPILE variable, again, in some rare scenarios you may also try "arm-eabi-"
# But beware, NDK doesn't even offer anything apart from arm-linux-androideabi one, however custom toolchains such as Linaro offer arm-eabi as well
CROSS_COMPILE="arm-linux-androideabi-"
# This specifies if we should also override our native toolchain in the PATH in addition to overriding makefile commands such as CC
# You should NOT enable it, unless your makefile calls "gcc" instead of "$CC" and you want to point "gcc" (and similar) to NDKTC
# However, in such case, you should either fix makefile yourself or not use it at all
# You've been warned, this is not a good idea
TCOVERRIDE="0"
# Workaround for some broken compilers with malloc problems (undefined reference to rpl_malloc and similar errors during compiling), don't uncomment unless you need it
#export ac_cv_func_malloc_0_nonnull=yes
Click to expand...
Click to collapse
As you can notice, my magic script already contains bunch of optimizations, especially device-based optimizations, which are the most important. Now it's the time to run our script, but in current shell and not a new one.
[email protected]:~# source cc.sh
Done setting your environment
CFLAGS: -O2 -march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp -s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing -DNDEBUG -D__ANDROID__
LDFLAGS: -Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections
CC points to arm-linux-androideabi-gcc and this points to /root/ndkTC/bin/arm-linux-androideabi-gcc
Use "$CC" command for calling gcc and "$CCC" command for calling our optimized CC
Use "$CXX" command for calling g++ and "$CCXX" for calling our optimized CXX
Use "$STRIP" command for calling strip and "$SSTRIP" command for calling our optimized STRIP
Example: "$CCC myprogram.c -o mybinary && $SSTRIP mybinary "
When using makefiles with configure options, always use "./configure $CONFIGANDROID" instead of using "./configure" itself
Please notice that makefiles may, or may not, borrow our CFLAGS and LFLAGS, so I suggest to double-check them and eventually append them to makefile itself
Pro tip: Makefiles with configure options always borrow CC, CFLAGS and LDFLAGS, so if you're using ./configure, probably you don't need to do anything else
Click to expand...
Click to collapse
Command "source cc.sh" executes cc.sh and "shares" the environment, which means that any exports will be exported to our current shell, and this is what we want. It acts the same as AOSP's ". build/envsetup.sh", so you can also use . instead of source.
As you can see above, my script should let you know if it properly set everything, especially if $CC points to our ndkTC. It also set a generic "$CCC" and "$CCXX" commands, which are optimized versions of standard $CC. $CC points to our cross-compiler, $CCC points to our cross-compiler and also includes our optimization flags.
[email protected]:~# echo $CC
arm-linux-androideabi-gcc
[email protected]:~# echo $CCC
arm-linux-androideabi-gcc -O2 -march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp -s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing -DNDEBUG -D__ANDROID__ -Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections
Click to expand...
Click to collapse
[SIZE="+1"]Step 3 - Cross-Compiling[/SIZE]
Now we'll compile our first program for Android!
Create a new file hello.c, and put inside:
Code:
#include <stdio.h>
int main (void)
{
puts ("Hello World!");
return 0;
}
Now you compile and strip it:
[email protected]:~# $CCC hello.c -o hello && $SSTRIP hello
Click to expand...
Click to collapse
Remember that $CCC and $SSTRIP command will only work if you source'd cc.sh script explained above. $CCC command compiles source code to a binary with already optimized flags (device flags, optimization level, optimization flags, linker flags), while $SSTRIP command strips "bloat" from output binary, such as comments and notices. The purpose is to make a binary smaller and faster.
You can check if your binary has been compiled properly through readelf command.
[email protected]:~# readelf -A hello
Attribute Section: aeabi
File Attributes
Tag_CPU_name: "ARM v7"
Tag_CPU_arch: v7
Tag_CPU_arch_profile: Application
Tag_ARM_ISA_use: Yes
Tag_THUMB_ISA_use: Thumb-2
Tag_FP_arch: VFPv3
Tag_Advanced_SIMD_arch: NEONv1
Tag_ABI_PCS_wchar_t: 4
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_align_needed: 8-byte
Tag_ABI_enum_size: int
Tag_ABI_HardFP_use: SP and DP
Tag_ABI_optimization_goals: Aggressive Speed
Tag_CPU_unaligned_access: v6
Tag_DIV_use: Not allowed
Click to expand...
Click to collapse
As you can see, I've compiled a binary optimized for ARM v7, with THUMB-2 instructions and NEON support. How nice! Is it because of device-specific flags? Let's check what happens if we use $CC instead of $CCC:
[email protected]:~# readelf -A hello2
Attribute Section: aeabi
File Attributes
Tag_CPU_name: "5TE"
Tag_CPU_arch: v5TE
Tag_ARM_ISA_use: Yes
Tag_THUMB_ISA_use: Thumb-1
Tag_FP_arch: VFPv2
Tag_ABI_PCS_wchar_t: 4
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_align_needed: 8-byte
Tag_ABI_enum_size: int
Tag_ABI_optimization_goals: Aggressive Speed
Tag_DIV_use: Not allowed
Click to expand...
Click to collapse
As you can see, if you do not specify flags, you'll lose major portion of optimizations. Of course binary will work properly, hence it has been cross-compiled for ARM, but we can always make it smaller and faster!
[SIZE="+1"]Step 4 - Testing[/SIZE]
A favourite part of everything, tests!
[email protected]:~/shared# adb shell
[email protected]:/ # sysrw
[email protected]:/ # exit
[email protected]:~/shared# adb push hello /system/bin/hello
95 KB/s (5124 bytes in 0.052s)
[email protected]:~/shared# adb shell
[email protected]:/ # chmod 755 /system/bin/hello
[email protected]:/ # chown root:system /system/bin/hello
[email protected]:/ # exit
Click to expand...
Click to collapse
In above example I pushed my binary straight to /system/bin directory, which is in the Android's PATH. If you don't have rooted device that's not a problem, you can use /data/local directory or /storage/sdcard0. You can also upload your binary anywhere you want and download it as any other file, then run from /storage/sdcard0/Download, this way doesn't require even working ADB . Just don't forget about setting proper permissions afterwards!
Now let's try to run it!
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
If your binary is not in the PATH, you should write full path to your binary of course. As I pushed my binary to /system/bin, I don't need to do so.
If everything finished successfully and you got your very first Hello World response as above, congratulations. You've just compiled and ran your first native C/C++ program on Android device.
[SIZE="+1"]What to do next?[/SIZE]
In theory, you can now compile anything you want. Here are some apps that I'm using in my ArchiDroid ROM:
Pixelserv
Haveged
Dnsmasq
DNRD
Rinetd
These are only a few examples. You can compile anything you want, or even write your own native applications. Good luck!

JustArchi said:
[SIZE=+1]What is a Cross-Compiler?[/SIZE]
[SIZE=+1]How is that connected with an Android?[/SIZE]
In order to create a native C/C++ binary for an Android, you must firstly compile the source code. Usually you can't do so on an Android itself due to lack of proper tools and environment, or hardware barriers, especially amount of RAM. This is why you should learn how to cross-compile, to create a binary on your PC, that your ARM-based Android will understand.
[SIZE=+1]Why do I need it?[/SIZE]
You need to learn cross-compiling technique if you want to run native C/C++ programs on an Android. Actually, if you've already built your own custom ROM from AOSP sources (i.e. CyanogenMod), then you used cross-compiling tools and methods even without noticing .
Building an AOSP ROM is fairly easy, there's one command like brunch, which does the job. However, what if you want to compile a custom, not natively included binary? This is the purpose of this tutorial.
[SIZE=+1]What I will learn from this guide?[/SIZE]
How to properly set C/C++ building environment
How to build a native C/C++ application for Android device
How to optimize native binaries for my device
[SIZE=+1]Step 1 - The Beginning[/SIZE]
You should start from installing any Linux-based OS, I highly suggest trying a Debian-based distro (such as Ubuntu), or even Debian itself, as this tutorial is based on it .
In general, I highly suggest to compile an AOSP ROM (such as CyanogenMod) for your device firstly. This will help you to get familiar with cross-compiling on Android. I also suggest to compile one or two programs from source for your Linux, but if you're brave enough to learn cross-compiling without doing any of these, you can skip those suggestions .
[SIZE=+1]Step 2 - Setting up[/SIZE]
Firstly you should make sure that you have all required compile tools already.
This should do the trick and install all required components.
I suggest creating a new folder and navigating to it, just to avoid a mess, but you can organize everything as you wish.
Start from downloading NDK from here.
Now you should make a standalone toolchain, navigate to root of your ndk (this is important) and then build your toolchain:
You should edit bolded variables to your preferences. Toolchain is the version of GCC you want to use, 4.8 is currently the newest one, in the future it may be 4.9 and so on. Platform is a target API for your programs, this is important only for android-specific commands, such as logging to logcat. When compiling a native Linux program, this won't matter (but it's a good idea to set it properly, just in case). Install dir specifies destination of your toolchain, make sure that it's other than ndk (as you can see I have ndk in /root/ndk and toolchain in /root/ndkTC).
Now you need to download my exclusive cc.sh script from here and make it executable.
This script is a very handy tool written by me to make your life easier while cross-compiling. Before running it make sure to edit "BASIC" options, especially NDK paths. Apart from that it's a good idea to take a look at DEVICEFLAGS and setting them properly for your device, or clearing for generic build. You don't need to touch other ones unless you want/need them.
Just for a reference, I'll include currently editable options:
As you can notice, my magic script already contains bunch of optimizations, especially device-based optimizations, which are the most important. Now it's the time to run our script, but in current shell and not a new one.
Command "source cc.sh" executes cc.sh and "shares" the environment, which means that any exports will be exported to our current shell, and this is what we want. It acts the same as AOSP's ". build/envsetup.sh", so you can also use . instead of source.
As you can see above, my script should let you know if it properly set everything, especially if $CC points to our ndkTC. It also set a generic "$CCC" and "$CCXX" commands, which are optimized versions of standard $CC. $CC points to our cross-compiler, $CCC points to our cross-compiler and also includes our optimization flags.
[SIZE=+1]Step 3 - Cross-Compiling[/SIZE]
Now we'll compile our first program for Android!
Create a new file hello.c, and put inside:
Code:
#include <stdio.h>
int main (void)
{
puts ("Hello World!");
return 0;
}
Now you compile and strip it:
Remember that $CCC and $SSTRIP command will only work if you source'd cc.sh script explained above. $CCC command compiles source code to a binary with already optimized flags (device flags, optimization level, optimization flags, linker flags), while $SSTRIP command strips "bloat" from output binary, such as comments and notices. The purpose is to make a binary smaller and faster.
You can check if your binary has been compiled properly through readelf command.
As you can see, I've compiled a binary optimized for ARM v7, with THUMB-2 instructions and NEON support. How nice! Is it because of device-specific flags? Let's check what happens if we use $CC instead of $CCC:
As you can see, if you do not specify flags, you'll lose major portion of optimizations. Of course binary will work properly, hence it has been cross-compiled for ARM, but we can always make it smaller and faster!
[SIZE=+1]Step 4 - Testing[/SIZE]
A favourite part of everything, tests!
In above example I pushed my binary straight to /system/bin directory, which is in the Android's PATH. If you don't have rooted device that's not a problem, you can use /data/local directory or /storage/sdcard0. You can also upload your binary anywhere you want and download it as any other file, then run from /storage/sdcard0/Download, this way doesn't require even working ADB . Just don't forget about setting proper permissions afterwards!
Now let's try to run it!
If your binary is not in the PATH, you should write full path to your binary of course. As I pushed my binary to /system/bin, I don't need to do so.
If everything finished successfully and you got your very first Hello World response as above, congratulations. You've just compiled and ran your first native C/C++ program on Android device.
[SIZE=+1]What to do next?[/SIZE]
In theory, you can now compile anything you want. Here are some apps that I'm using in my ArchiDroid ROM:
Pixelserv
Haveged
Dnsmasq
DNRD
Rinetd
These are only a few examples. You can compile anything you want, or even write your own native applications. Good luck!
Click to expand...
Click to collapse
[Mod Edit: Please don't quote the whole OP]
Fricking awesome. Worked perfect on my builduntu running in VirtualBox

dicksteele said:
Fricking awesome. Worked perfect on my builduntu running in VirtualBox
Click to expand...
Click to collapse
I'm very glad it worked for you .

Maybe you happen to know which packages checkinstall depends on? I want to run this on Arch - pun not intended - and pacman doesn't exactly talk with debs.
(Przy okazji, świetny tutorial c: )

Dragoon Aethis said:
Maybe you happen to know which packages checkinstall depends on? I want to run this on Arch - pun not intended - and pacman doesn't exactly talk with debs.
(Przy okazji, świetny tutorial c: )
Click to expand...
Click to collapse
Checkinstall makes sure that you have all required packages installed. You can achieve nearly the same by installing "build-essential, gcc, g++, make", and that should be enough I guess .

Also, big kudos to @willverduzco for featuring my guide on XDA portal!

I would like to see a guide for llvm/ clang.
Sent from my GT-I9000 using xda app-developers app

maybe a bit irrelevant... but i wanted to learn how to cross compile/port a binary (for example "applypatch") for cygwin... any link to guide will be helpful
Thank You

DerRomtester said:
I would like to see a guide for llvm/ clang.
Sent from my GT-I9000 using xda app-developers app
Click to expand...
Click to collapse
When making standalone toolchain you should use clang instead of gcc. You should also study my cc.sh script and adapt to your own. After that, steps are nearly the same.
EnerJon said:
maybe a bit irrelevant... but i wanted to learn how to cross compile/port a binary (for example "applypatch") for cygwin... any link to guide will be helpful
Thank You
Click to expand...
Click to collapse
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.

JustArchi said:
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.
Click to expand...
Click to collapse
Actually i was making a tool for windows to generate/apply OTA for Android ROMs... i wanted to compile/port "IMGDIFF2" and "applypatch" from android sources...

EnerJon said:
Actually i was making a tool for windows to generate/apply OTA for Android ROMs... i wanted to compile/port "IMGDIFF2" and "applypatch" from android sources...
Click to expand...
Click to collapse
Then you should find your sources for IMGDIFF2 and applypatch and compile from source for Android, just like example hello.c above.

@JustArchi I saw this guide mentioned on the portal and read through it. Very interesting stuff. Great work explaining. I've got several questions, however, perhaps you can elaborate on.
My primary PC OS is Gentoo Linux (I've been using it for 10 years), in patricular ~amd64 which is the equivalent of Debian unstable. In Gentoo, all packages are compiled from the sources. I have a very up to date complete toolchain already installed and functioning properly as part of the native package installation system which uses portage for maintaining and updating.
I've already compiled CM and AOSP for my device, but I can't for the life of me understand why when setting up my build environment using either Google or CM tools several much older versions of GCC and GLIBC are installed into my source repos and used to build the ROM when the prerequisites for building the environment already require a working toolchain on the host build box?
Isn't there a way to just use the native toolchain from the host? Ideally, I'd love to free up the space used by these extra compilers and libraries for sources instead. Additionally, since my toolchain is much newer (gcc-4.8.2, glibc-2.19, etc) and optimized for my hardware than these generic prebuilt binaries, my ROM builds would compile faster and more optimized if I could use it instead.
The big question I ask is would you know what I'd have to do to setup my native environment to build Android? I'd truly love to be able to get rid of these other toolchains and free up the space on my harddrive. Any help would be greatly appreciated. TIA

JustArchi said:
When making standalone toolchain you should use clang instead of gcc. You should also study my cc.sh script and adapt to your own. After that, steps are nearly the same.
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.
Click to expand...
Click to collapse
I try this. I would like to cross compile a kernel with clang. Hopefully i get it working.

Odysseus1962 said:
@JustArchi I saw this guide mentioned on the portal and read through it. Very interesting stuff. Great work explaining. I've got several questions, however, perhaps you can elaborate on.
My primary PC OS is Gentoo Linux (I've been using it for 10 years), in patricular ~amd64 which is the equivalent of Debian unstable. In Gentoo, all packages are compiled from the sources. I have a very up to date complete toolchain already installed and functioning properly as part of the native package installation system which uses portage for maintaining and updating.
I've already compiled CM and AOSP for my device, but I can't for the life of me understand why when setting up my build environment using either Google or CM tools several much older versions of GCC and GLIBC are installed into my source repos and used to build the ROM when the prerequisites for building the environment already require a working toolchain on the host build box?
Isn't there a way to just use the native toolchain from the host? Ideally, I'd love to free up the space used by these extra compilers and libraries for sources instead. Additionally, since my toolchain is much newer (gcc-4.8.2, glibc-2.19, etc) and optimized for my hardware than these generic prebuilt binaries, my ROM builds would compile faster and more optimized if I could use it instead.
The big question I ask is would you know what I'd have to do to setup my native environment to build Android? I'd truly love to be able to get rid of these other toolchains and free up the space on my harddrive. Any help would be greatly appreciated. TIA
Click to expand...
Click to collapse
You need special compiler capable of compiling for specific architecture, this is not the same as native GCC toolchain for amd64. When you're using native compiler, output is always designed for amd64 or i386, when using cross-compiler, output is always designed for ARM, or other specific architecture.

JustArchi said:
You need special compiler capable of compiling for specific architecture, this is not the same as native GCC toolchain for amd64. When you're using native compiler, output is always designed for amd64 or i386, when using cross-compiler, output is always designed for ARM, or other specific architecture.
Click to expand...
Click to collapse
Thanks for the quick response. I'm a bit disappointed, but I'm still wondering that there has to be some way for me to utilize the ARM toolchain I currently have installed to cross-compile from the sources a more updated optimized toolchain for me to build with. Unfortunately (for me), that Gentoo is more of a niche Linux distro so finding help in their forums for working with ARM is difficult. As it is, it took much effort and trial and error to setup my current configuration to build with since nearly everything on the net is geared towards Ubuntu / Debian (both of which I feel are loaded with useless cruft and dependencies for things I have never and will never use).
Anyhow thanks again for this great guide, and for your continued work here helping us all.
Ciao

Dropbox link is down
Inviato dal mio GT-I9300 utilizzando Tapatalk

Code:
#!/bin/bash
# _ _ _ _ _
# | |_ _ ___| |_ / \ _ __ ___| |__ (_)
# _ | | | | / __| __| / _ \ | '__/ __| '_ \| |
# | |_| | |_| \__ \ |_ / ___ \| | | (__| | | | |
# \___/ \__,_|___/\__/_/ \_\_| \___|_| |_|_|
#
# Copyright 2014 Łukasz "JustArchi" Domeradzki
# Contact: [email protected]
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#############
### BASIC ###
#############
# Root of NDK, the one which contains $NDK/ndk-build binary
NDK="/root/ndk"
# Root of NDK toolchain, the one used in --install-dir from $NDK/build/tools/make-standalone-toolchain.sh. Make sure it contains $NDKTC/bin directory with $CROSS_COMPILE binaries
NDKTC="/root/ndkTC"
# Optional, may help NDK in some cases, should be equal to GCC version of the toolchain specified above
export NDK_TOOLCHAIN_VERSION=4.8
# This flag turns on ADVANCED section below, you should use "0" if you want easy compiling for generic targets, or "1" if you want to get best optimized results for specific targets
# In general it's strongly suggested to leave it turned on, but if you're using makefiles, which already specify optimization level and everything else, then of course you may want to turn it off
ADVANCED="1"
################
### ADVANCED ###
################
# Device CFLAGS, these should be taken from TARGET_GLOBAL_CFLAGS property of BoardCommonConfig.mk of your device, eventually leave them empty for generic non-device-optimized build
# Please notice that -march flag comes from TARGET_ARCH_VARIANT
DEVICECFLAGS="-march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp"
# This specifies optimization level used during compilation. Usually it's a good idea to keep it on "-O2" for best results, but you may want to experiment with "-Os", "-O3" or "-Ofast"
OLEVEL="-O2"
# This specifies extra optimization flags, which are not selected by any of optimization levels chosen above
# Please notice that they're pretty EXPERIMENTAL, and if you get any compilation errors, the first step is experimenting with them or disabling them completely, you may also want to try different O level
OPTICFLAGS="-s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing"
# This specifies extra linker optimizations. Same as above, in case of problems this is second step for finding out the culprit
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections"
# This specifies additional sections to strip, for extra savings on size
STRIPFLAGS="-s -R .note -R .comment -R .gnu.version -R .gnu.version_r"
# Additional definitions, which may help some binaries to work with android
DEFFLAGS="-DNDEBUG -D__ANDROID__"
##############
### EXPERT ###
##############
# This specifies host (target) for makefiles. In some rare scenarios you may also try "--host=arm-linux-androideabi"
# In general you shouldn't change that, as you're compiling binaries for low-level ARM-EABI and not Android itself
CONFIGANDROID="--host=arm-linux-eabi"
# This specifies the CROSS_COMPILE variable, again, in some rare scenarios you may also try "arm-eabi-"
# But beware, NDK doesn't even offer anything apart from arm-linux-androideabi one, however custom toolchains such as Linaro offer arm-eabi as well
CROSS_COMPILE="arm-linux-androideabi-"
# This specifies if we should also override our native toolchain in the PATH in addition to overriding makefile commands such as CC
# You should NOT enable it, unless your makefile calls "gcc" instead of "$CC" and you want to point "gcc" (and similar) to NDKTC
# However, in such case, you should either fix makefile yourself or not use it at all
# You've been warned, this is not a good idea
TCOVERRIDE="0"
# Workaround for some broken compilers with malloc problems (undefined reference to rpl_malloc and similar errors during compiling), don't uncomment unless you need it
#export ac_cv_func_malloc_0_nonnull=yes
############
### CORE ###
############
# You shouldn't edit anything from now on
if [ "$ADVANCED" -ne 0 ]; then # If advanced is specified, we override flags used by makefiles with our optimized ones, of course if makefile allows that
export CFLAGS="$OLEVEL $DEVICECFLAGS $OPTICFLAGS $DEFFLAGS"
export LOCAL_CFLAGS="$CFLAGS"
export CXXFLAGS="$CFLAGS" # We use same flags for CXX as well
export LOCAL_CXXFLAGS="$CXXFLAGS"
export CPPFLAGS="$CPPFLAGS" # Yes, CPP is the same as CXX, because they're both used in different makefiles/compilers, unfortunately
export LOCAL_CPPFLAGS="$CPPFLAGS"
export LDFLAGS="$LDFLAGS"
export LOCAL_LDFLAGS="$LDFLAGS"
fi
if [ ! -z "$NDK" ] && [ "$(echo $PATH | grep -qi $NDK; echo $?)" -ne 0 ]; then # If NDK doesn't exist in the path (yet), prepend it
export PATH="$NDK:$PATH"
fi
if [ ! -z "$NDKTC" ] && [ "$(echo $PATH | grep -qi $NDKTC; echo $?)" -ne 0 ]; then # If NDKTC doesn't exist in the path (yet), prepend it
export PATH="$NDKTC/bin:$PATH"
fi
export CROSS_COMPILE="$CROSS_COMPILE" # All makefiles depend on CROSS_COMPILE variable, this is important to set"
export AS=${CROSS_COMPILE}as
export AR=${CROSS_COMPILE}ar
export CC=${CROSS_COMPILE}gcc
export CXX=${CROSS_COMPILE}g++
export CPP=${CROSS_COMPILE}cpp
export LD=${CROSS_COMPILE}ld
export NM=${CROSS_COMPILE}nm
export OBJCOPY=${CROSS_COMPILE}objcopy
export OBJDUMP=${CROSS_COMPILE}objdump
export READELF=${CROSS_COMPILE}readelf
export RANLIB=${CROSS_COMPILE}ranlib
export SIZE=${CROSS_COMPILE}size
export STRINGS=${CROSS_COMPILE}strings
export STRIP=${CROSS_COMPILE}strip
if [ "$TCOVERRIDE" -eq 1 ]; then # This is not a a good idea...
alias as="$AS"
alias ar="$AR"
alias gcc="$CC"
alias g++="$CXX"
alias cpp="$CPP"
alias ld="$LD"
alias nm="$NM"
alias objcopy="$OBJCOPY"
alias objdump="$OBJDUMP"
alias readelf="$READELF"
alias ranlib="$RANLIB"
alias size="$SIZE"
alias strings="$STRINGS"
alias strip="$STRIP"
fi
export CONFIGANDROID="$CONFIGANDROID"
export CCC="$CC $CFLAGS $LDFLAGS"
export CXX="$CXX $CXXFLAGS $LDFLAGS"
export SSTRIP="$STRIP $STRIPFLAGS"
echo "Done setting your environment"
echo
echo "CFLAGS: $CFLAGS"
echo "LDFLAGS: $LDFLAGS"
echo "CC points to $CC and this points to $(which "$CC")"
echo
echo "Use \"\$CC\" command for calling gcc and \"\$CCC\" command for calling our optimized CC"
echo "Use \"\$CXX\" command for calling g++ and \"\$CCXX\" for calling our optimized CXX"
echo "Use \"\$STRIP\" command for calling strip and \"\$SSTRIP\" command for calling our optimized STRIP"
echo
echo "Example: \"\$CCC myprogram.c -o mybinary && \$SSTRIP mybinary \""
echo
echo "When using makefiles with configure options, always use \"./configure \$CONFIGANDROID\" instead of using \"./configure\" itself"
echo "Please notice that makefiles may, or may not, borrow our CFLAGS and LFLAGS, so I suggest to double-check them and eventually append them to makefile itself"
echo "Pro tip: Makefiles with configure options always borrow CC, CFLAGS and LDFLAGS, so if you're using ./configure, probably you don't need to do anything else"
Temporary replacement for cc.sh, as dropbox will be up soon.

Hi!
Great info.
To cross compile some packages with autotools (./configure; make; make install) it's needed to export the SYSROOT path ($ndkTC/sysroot) and include the option --sysroot=$SYSROOT on CFLAGS. Some need too --with-sysroot=$SYSROOT as configure option. This way the configure script and linker can find the libraries.
If i'm building a library that must be used as dependence to other program I use to include --static to build a static library and --prefix=$SYSROOT/usr on configure options to install the lib on toolchain sysroot folder...
Thanks.

sfortier said:
Hi!
Great info.
To cross compile some packages with autotools (./configure; make; make install) it's needed to export the SYSROOT path ($ndkTC/sysroot) and include the option --sysroot=$SYSROOT on CFLAGS. Some need too --with-sysroot=$SYSROOT as configure option. This way the configure script and linker can find the libraries.
If i'm building a library that must be used as dependence to other program I use to include --static to build a static library and --prefix=$SYSROOT/usr on configure options to install the lib on toolchain sysroot folder...
Thanks.
Click to expand...
Click to collapse
Hey.
Nice to know, I'll update my script with that. Thanks!

My last attempt to cross compile something was qemu (i'm was thinking on run windows on my tablet... )
I needed to build glib, pixmap, libpng, zlib, libjpeg-turbo, libiconv, libffi, libintl. Now I have my toolchain with all these usefull static (I prefer static libs to simplify binary installation) libs installed!

[GUIDE][HOW_TO] BUILD a LINUX KERNEL FROM SOURCE [UBUNTU]

This is a general open source linux development thread!
Android's kernel is a derivative of linux's kernel. Its good to know how to build both of these kernels. You might be already familiar with building kernels for various devices from sources. So I have made a new thread for guiding people on how to compile linux kernel from source (example taken as ubuntu kernel).
Requirements:
Any linux os x64 bit(example here: ubuntu 14.04)
Git (sudo apt-get install git)
Minimum of 4GB RAM and some reasonable linux-swap
To get the currently running kernel image, type the following:
Code:
apt-get source linux-image-$(uname -r)
Now we need to obtain Ubuntu Kernel Sources from its repositories. Make a new directory and inside it, initialise the git and clone the repository.
Code:
git clone git://kernel.ubuntu.com/ubuntu/ubuntu-<release>.git
<release> : Type in the required source. It can be lucid, precise, trusty, utopic etc.
Setting up the build environment. There are lots of tools and packages that are very much essential for building a kernel. These tools can be downloaded as a whole bundle and can be installed easily. Here's the code to set it up:
Code:
sudo apt-get build-dep linux-image-$(uname -r)
NOTE: The above comand can be executed only after you obtain the currently running kernel image. I have already given the code to obtain it above.
Now, change directory to the root of the kernel and type the following:
Code:
chmod -R a+x *
The above code will set the required permissions for building and executing the kernel.
Now, run these two commands:
Code:
fakeroot debian/rules clean
fakeroot debian/rules editconfigs
The first command cleans up the code automatically.
The slightly tricky part is with the second line of the code. When you execute it, you will have to edit a series of menuconfigs. To make changes to the configuration file we need to edit the configuration file. The kernel developers have created a script to edit kernel configurations which has to be called through the debian/rules makefile, unfortunately you will have to go through all the flavors for this script to work properly. The script will ask you if you want to edit the particular configuration. You should not make changes to any of the configurations until you see your wanted flavour configuration
We have now covered about 70% of progress. The rest is building the kernel and testing it.
Building the kernel is quite easy. Change your working directory to the root of the kernel source tree and then type the following commands:
Code:
fakeroot debian/rules clean
fakeroot debian/rules binary-headers binary-generic
If the build is successful, a set of three .deb binary package files will be produced in the directory above the build root directory. For example after building a kernel with version "3.13.-0.35" on an amd64 system, these three .deb packages would be produced:
Code:
cd ..
ls *.deb
linux-headers-3.13.0-35_3.13.0-35.37_all.deb
linux-headers-3.13.0-35-generic_3.13.0-35.37_amd64.deb
linux-image-3.13.0-35-generic_3.13.0-35.37_amd64.deb
Testing the new kernel
Install the three-package set (on your build system, or on a different target system) with dpkg -i and then reboot:
Code:
sudo dpkg -i linux*3.13.0-35.37*.deb
sudo reboot
Guys, I hope I have made an easy tutorial. You are always welcome to ask doubts (even on PM). Thank You.

Specific Hardware/Architecture
Creating a new config
I’ll be using the method of creating a new flavour, this adds a bit more work but this way you can always compile the original kernels.
We’ll use the generic flavour as the base for our own flavour being i7, this extension needs to be in small caps.
Code:
cp debian.master/config/amd64/config.flavour.generic debian.master/config/amd64/config.flavour.i7
fakeroot debian/rules clean
debian/rules updateconfigs
To make changes to the configuration file we need to edit the configuration file. The kernel developers have created a script to edit kernel configurations which has to be called through the debian/rules makefile, unfortunately you will have to go through all the flavours for this script to work properly.
Code:
debian/rules editconfigs
The script will ask you if you want to edit the particular configuration. You should not make changes to any of the configurations until you see the i7 configuration
Code:
Do you want to edit config: amd64/config.flavour.i7? [Y/n]
Make your changes, save the configuration and then keep going until the script ends.
When you’re done, make a backup of the config flavor file.
Code:
cp debian.master/config/amd64/config.flavour.i7 ../.
Now we need to clean up the git tree in order to get ready for compilation.
Code:
git reset --hard
git clean -df
Getting ready for compilation
Because we are going to be creating a new flavour based on a existing flavour (generic in my case) we need to create some extra files. During compilation the process checks the previous release for some settings, as we’re creating a local flavour it doesn’t exist in the source, so we’re creating it.
To see the previous release we use:
Code:
ls debian.master/abi
cp debian.master/abi/3.0.0-12.20/amd64/generic debian.master/abi/3.0.0-12.20/amd64/i7
cp debian.master/abi/3.0.0-12.20/amd64/generic.modules debian.master/abi/3.0.0-12.20/amd64/i7.modules
Copy our flavored configuration file back.
Code:
cp ../config.flavour.i7 debian.master/config/amd64/
We need to edit some files:
File: debian.master/etc/getabis
Search for the line:
Code:
getall amd64 generic server virtual
Change it in:
Code:
getall amd64 generic server virtual i7
File: debian.master/rules.d/amd64.mk
Search for the line:
Code:
flavours = generic server virtual
Change it in:
Code:
flavours = generic server virtual i7
File: debian.master/control.d/vars.i7
This files does not exist and in order to make the compilation process aware of our own flavor we want to compile we need to create it.
Code:
cp debian.master/control.d/vars.generic debian.master/control.d/vars.i7
You can edit the file and make it your own.
Code:
arch="i386 amd64"
supported="i7 Processor"
target="Geared toward i7 desktop systems."
desc="x86/x86_64"
bootloader="grub-pc | grub-efi-amd64 | grub-efi-ia32 | grub | lilo (>= 19.1)"
provides="kvm-api-4, redhat-cluster-modules, ivtv-modules, ndiswrapper-modules-1.9"
We need to commit our changes in the git repository.
Code:
git add .
git commit -a -m "i7 Modifications"
The text after -m is the message you add to your commit.
Compilation
It’s finally time for compiling, to keep our newly created branch in pristine condition we will do the compilation in a separate branch. We keep our branch clean as this will help later on when we want to update our new branch to a newer kernel.
Code:
git checkout -b work
fakeroot debian/rules clean
All the packages will be created in the directory /d1/development/kernel/ubuntu/oneiric
Create independent packages:
Code:
skipabi=true skipmodule=true fakeroot debian/rules binary-indep
The above statement will create the following deb files:
Code:
linux-doc_3.0.0-13.21_all.deb
linux-headers-3.0.0-13_3.0.0-13.21_all.deb
linux-source-3.0.0_3.0.0-13.21_all.deb
linux-tools-common_3.0.0-13.21_all.deb
Create the tools package:
Code:
skipabi=true skipmodule=true fakeroot debian/rules binary-perarch
The above statement will create the following deb file:
Code:
linux-tools-3.0.0-13_3.0.0-13.21_amd64.deb
Create the flavour depended files:
Code:
skipabi=true skipmodule=true fakeroot debian/rules binary-i7
The above statement will create the following deb files:
Code:
linux-headers-3.0.0-13-i7_3.0.0-13.21_amd64.deb
linux-image-3.0.0-13-i7_3.0.0-13.21_amd64.deb
Installation
After the compilation is finished we’ll have the above packages in the parent directory.
To install the files
Code:
cd ..
sudo dpkg -i linux-headers-3.0.0-13-i7_3.0.0-13.21_amd64.deb linux-headers-3.0.0-13_3.0.0-13.21_all.deb linux-image-3.0.0-13-i7_3.0.0-13.21_amd64.deb
Check your bootloader if the newly installed Ubuntu kernel is the default one, for grub check the file /boot/grub/menu.lst or if you run grub2 check /boot/grub/grub.cfg

thx for your info

nice job mate..!! :good:
now i'm gonna try this..!!

Nice ,i can't say anything

faizauthar12 said:
Nice ,i can't say anything
Click to expand...
Click to collapse
Thank you for the great guide!!!

Nice thread. I'll try it at home

Thanks
Enviado de meu Moto G usando Tapatalk

Thx for the guide
tra_dax

Guide: Compile /system/bin binaries for your device from AOSP source code

Now tested up to downloading AOSP and make toolbox you should be all set
Please give thanks to this thread: https://forum.xda-developers.com/newreply.php?do=newreply&p=43622764
Warning: I hacked my way through this stuff a few weeks ago I am not an expert!
How to compile Android Open Source Code modules​
I don't compile C code on Windows machines I have no idea about that.
Notice
This guide is a quick and dirty how to make a module. It will not cover finalizing setting up the source codes for your device. It is only my goal to enable you to compile binaries such as grep, toolbox, dumpstate, dalvikvm, jack and etc.
===>] Setup Ubuntu 64bit [<===​Unplug that Windows drive, plug in a fresh hard drive and install Ubuntu latest/greatest. Ignore the recommendation to downgrade gnu make!, for now.
Open a terminal and issue these commands (warning ppa repository for OpenJDK 7 is said to have a security issue?, isn't being updated?.. whatevs it works)
Code:
sudo apt-get update
sudo apt-get upgrade
sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt-get install openjdk-7-jdk
sudo apt-get install openjdk-8-jdk
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386 lib32z1 libbz2-1.0:i386
sudo apt-get install git ccache automake lzop bison gperf build-essential zip curl zlib1g-dev zlib1g-dev:i386 g++-multilib python-networkx libxml2-utils bzip2 libbz2-dev libbz2-1.0 libghc-bzlib-dev squashfs-tools pngcrush schedtool dpkg-dev liblz4-tool make optipng
(choose Java 1.7 in the following way)
Code:
sudo update-alternatives --config java
(let me know if I missed anything please)
"Tried the Android SDK only it is missing too many things we need as a developer"
===>] Setup Android Studio SDK & NDK [<===​Installation Paths:
*** I install to /home/username/Android and /home/username/Android/Sdk and /home/username/Android/Sdk/ndk-bundle ***
NOTE: from here forward username will == droidvoider
Note: Android Studio IDE isn't necessary only the SDK & NDK are needed to compile AOSP.
Install Android Studio Proper: (don't worry about setting up paths we will cover that, just install it)
https://developer.android.com/studio/install.html
or
SDK Only:
Typically we install these things manually by creating the directory then just unzipping the files there.
https://developer.android.com/studio/index.html#linux-bundle (scroll down for sdk only)
Code:
mkdir /home/droidvoider/Android
mkdir /home/droidvoider/Android/Sdk
(then unzip the sdk zip to that directory. I recommend the file explorer copy/paste right click uncompress and done.)
https://dl.google.com/android/repository/tools_r25.2.3-linux.zip
Install NDK through the SDK Manger:
(from terminal 'studio.sh' and then configure, and then sdk manger --- if this is hard to figure out tell me I will elaborate)
or
Manually Install Native Development Kit -- 'c programming support'
Download the Native Development Kit from Google: https://developer.android.com/ndk/downloads/index.html
Code:
mkdir /home/droidvoider/Android/Sdk/ndk-bundle
Then just unzip the ndk files into the directory we created above.
===>] Setup your toolchain [<===​** This example is arm64-v8a aarch64 **
1. Navigate to /home/droidvoider/Android/Sdk/ndk-bundle/build/tools and then open a terminal "right click open area"
2. mkdir /home/droidvoider/toolchains
3. ./make_standalone_toolchain.py --arch arm64 --api 23 --stl=libc++ --install-dir /home/mm/toolchains/aarch64-linux-android-4.9
4. cd /home/droidvoider
5. gedit .bashrc and morph this in at the bottom.. (AND edit or replace the existing PATH variable)
DON'T just PASTE IN *my* $PATH export!! I included my entire path statements to show you.
Code:
export PATH=$PATH:/usr/local/android-studio/bin:/home/droidvoider/Android/Sdk/platform-tools:/home/droidvoider/Android/Sdk/ndk-bundle:/home/droidvoider/Android/Sdk/tools
I feel this is human readable, for example change Android_Build_Out to be on your desktop instead if you want.
Code:
export PATH=$PATH:/home/droidvoider/toolchains/aarch64-linux-android-4.9
export NDK=/home/droidvoider/Android/Sdk/ndk-bundle
export SYSROOT=$NDK/platforms/android-23/arch-arm64
export TARGET=aarch64-linux-android
export HOST=$TARGET
export BUILD=x86_64-linux
export ANDROID_NDK_BIN=/home/droidvoider/toolchains/aarch64-linux-android-4.9/bin
export CC=$ANDROID_NDK_BIN/aarch64-linux-android-gcc-4.9
export CPP=$ANDROID_NDK_BIN/aarch64-linux-android-g++
export AR=$ANDROID_NDK_BIN/aarch64-linux-android-ar
export OUT_DIR_COMMON_BASE=/home/droidvoider/Android_Build_Out
Note: You might want to setup an alternate toolchain also but this is all of the puzzle pieces.
===>] Google's version of this How To -- Just for reference [<===​https://source.android.com/source/requirements.html
https://source.android.com/source/initializing.html
===>] Install the repo tool [<===​https://source.android.com/source/downloading.html
(don't type repo init or repo sync --- I will be taking back over from there on the next page)
Added Repair Notes -- Not part of the install!
Have you accidentally installed or removed something you shouldn't have? (welcome to development, here try this before reinstall)
sudo apt-get clean
sudo apt-get update
sudo apt-get install -f
sudo dpkg -a --configure
sudo apt-get dist-upgrade
sudo apt-get install -f
sudo dpkg -a --configure

Selecting the correct AOSP branch and downloading it.
Tested up to downloading AOSP and make toolbox -- you should be all set
===>] Match your build number to it's AOSP sources [<===​preface: You can get this from your device if you're on the same build id as your the available source code from your vendor for your device. Otherwise you need to open the AP file from the firmware that matches those available sources to extract the system.img, to extract build.prop. I explain how to open a system.img file below under retrieving your hardware drivers. build.prop is in the main directory of system.img
(Many times the build number is the same. For me I believe all of MM builds are using this number.)
Assumes sources match current device, worked out true in my case
1. Plug in your device and get it connected. (DEVELOPER OPTIONS|USB DEBUGGING) and select allow on device
2. Retrieve the build number that matches the available sources for your device.
From your ubuntu terminal retrieve the build id using this command:
Code:
adb shell getprop | grep 'ro.build.id'
Yields something similar to this: [ro.build.id]: [MMB29K]
3. Match it up to the Nexus build numbers (This info is for AT&T Note 5 Marshmallow MMB29K, get your specific build number!)
https://source.android.com/source/build-numbers.html#source-code-tags-and-builds
MMB29K android-6.0.1_r1 Marshmallow Nexus 5, Nexus 5X, Nexus 6, Nexus 7 (flo/deb), Nexus 9 (volantis/volantisg)
===>] Bring down a specific AOSP source branch [<===​
4. Make a directory for the source code.
Code:
mkdir /home/droidvoider/Desktop/AOSP_Android_6.01_r1
5.
Code:
cd /home/droidvoider/Desktop/AOSP_Android_6.01_r1
6. Bring down the sources, this one is approximately 13 gigabytes
Code:
repo init --depth=1 -u https://android.googlesource.com/platform/manifest -b android-6.0.1_r1
repo sync
===>] I'm not sure the rest of this is needed [<===​For compiling toolbox the remainder wasn't needed.. But I have a large list of things to do so I can't test each item. If you can't compile a specific module continue reading.
===>] Merge Vendor sources & AOSP sources [<===​
7. Download the available sources for your device. In this example I downloaded PE6 Marshmallow sources for AT&T Note 5:
http://opensource.samsung.com/reception/receptionSub.do?method=sub&sub=F&searchValue=SM-N920A
8. Read the readme file from the sources platform zip to understand how to merge them into the AOSP sources. For the 2 Samsungs I've worked with the idea is to replace any directory that already exists. But if there is just one file such as core.mk only replace the one file. Then edit the .mk files as described in your readme. (and/or take info from cyanogen/lineagos) -- <I can help more, ask>
note: you probably don't need to take the configs from LineageOS and put them into your .mk files. However, if you do need to get more configs then you should get a big fat message when you type make 'modulename'. At first only edit .mk files as described by vendor device source readme file.
===>] Merge in Hardware drivers and etc [<===​possibly unnecessary depends what you're doing
9. Obtain a copy of the firmware for your device that matches the version of the source code you are able to download from your vendor.
for me that was Build Number: MMB29K.N920AUCU2BPE6 but your mileage will almost certainly vary!
10. Download https://github.com/anestisb/android-simg2img
11. Unzip it right in your download folder, open the folder and then 'open in terminal'
12. Make it and then move it a directory in your path. Warning: My command puts in in the Ubuntu default /bin folder.
Code:
make
sudo mv append2simg img2simg simg2img simg2simg simg_dump.py /bin
13. Uncompress the AP file from the matching firmware and extract the system.img into it's own directory
then select that folder, right click, open in terminal
Code:
simg2img system.img sys.raw
mkdir sys
sudo mount -t ext4 -o loop sys.raw sys/
14. A drive mounted, look on your task bar it should've wiggled too. Copy the etc and vendor folders into the main folder of the sources we are merging

===>] Listing and building modules [<===​Navigate to the folder where you download the sources "/home/droidvoider/Desktop/AOSP_Android_6.01_r1" and open in terminal.
Code:
make modules -- list the available modules
make <module name> -- builds a specific module
example: make dumpstate
description: Will build everything needed for dumpstate and place it in the folder we specified in our export (above step). The final build line will read install and detail the final output folder
Example successful output:
[CODE]
Install: /home/droidvoider/Android_Build_Out/Android_6.01_r1/target/product/generic/system/bin/dumpstate
===>] Android Build System, basic intro [<===​Notice: I built this how to to answer the same question from 3 people regarding working with toolbox and the dirtycow exploit. So I decided to give a direct example of using toolbox.c from farm-root
#ifdef
Our makefile is Android.mk and that's where we link things together. If you look at the Android.mk file for farm-root you will notice bridge.c is used 3 different times called different 'module' names. bridge_pull, bridge_push, bridge_pull_boot. Each of these will be binaries of those names.
Inside bridge.c you will see #ifdef FARM_PULL and then you will see #else and further you will see #endif which you may have noticed matches inside the Android.mk file for the bridge modules -DFARM_PULL -DFARM_BOOT <== Notice the double define on bridge_pull_boot
toolbox.c
toolbox.c is going to be the same way. You will need to copy shared.h and shared.c into the directory where toolbox.c resides. Then edit the Android.mk, in our example:
1. Navigate to this directory and open: system/core/toolbox/Android.mk
2. CTRL + F and search for "LOCAL_MODULE := toolbox"
3. Add: LOCAL_CFLAGS += -DFARM_PULL -DFARM_BOOT (in this example add one, both or even new ones you created)
4. Navigate to the main directory of the sources, you should see a Makefile and a build_64bit.sh
5. from terminal: make toolbox
Note: I think from here you can Google it out in a few minutes if that is not the case please let me know.

Working with C cross platform​Ubuntu is Linux based just like Android and this makes testing blocks of code extremely easy. You of course can't use Android headers and in some rare cases you can't test the code on Ubuntu at all but in most cases you can. When I want to design something for Android I open gedit and save it as a .c file. Then I compile it using gcc -o mycode mycode.c There's plenty of examples on using gcc with linux but just understand you can do it all. Then before too much work test it on Android. (helpful commands at end of post)
My advice really is to build out your small blocks of code on your linux box but then paste them into your Android program folder, edit your Android.mk, add it to your Makefile including your 'push' section so that you can simply type make push to test it.
I am in fact trying to encourage you to learn C and not so much trying to encourage you to hack things. But I know that interest/passion is what teaches, not my words and not someone else's curriculum. So in that spirit I will do my best to give examples to help you with 'whatever' it is you are passionate about. Let me know what's missing.
Don't forget to compile for Android first
Before you can test your code you will have compiled it using the cross compiler for Android. ndk-build, or the correct gcc cross compiler. (Personally I put the .c file into a directory with Android.mk and a Makefile then just type make to build it to Android)
see examples section I will add a couple examples.
Android Developer Bridge -- a developers tool
adb is included with the Android SDK along with some other tools. Some of those tools are fastboot for unlocking bootloaders and another way of flashing. There is monitor which is a cool tool for remotely viewing processes, logcat, memory dumps and etc.
But pointedly what we will use the most is simply adb.
Using adb to test your code on locked down Android systems
Shell has fairly high privileges, you may not be aware but you can execute binaries and bash scripts. We use /data/local/tmp/ for these things. You can create a directory, add or remove files, execute your binaries and even execute shell scripts using sh script.sh
ndk-build places the binary in libs/(arch type) .. For a quick test you can just open a terminal in that directory then:
Code:
adb push mybinary /data/local/tmp/
adb shell
cd data/local/tmp
chmod 777 mybinary
./mybinary

Added:
Examples of basic make files for Android.
happy coding
If you get an error​Please reissue the command but pipe the output to a file.
make toolbox > /home/droidvoider/Desktop/build_toolbox-output.txt
zip that up with your source code, including your customized header files and attach it to this thread.
puzzles are fun but I like all the pieces

droidvoider said:
Tested up to downloading AOSP and make toolbox -- you should be all set
===>] Match your build number to it's AOSP sources [<===​preface: You can get this from your device if you're on the same build id as your the available source code from your vendor for your device. Otherwise you need to open the AP file from the firmware that matches those available sources to extract the system.img, to extract build.prop. I explain how to open a system.img file below under retrieving your hardware drivers. build.prop is in the main directory of system.img
(Many times the build number is the same. For me I believe all of MM builds are using this number.)
Assumes sources match current device, worked out true in my case
1. Plug in your device and get it connected. (DEVELOPER OPTIONS|USB DEBUGGING) and select allow on device
2. Retrieve the build number that matches the available sources for your device.
From your ubuntu terminal retrieve the build id using this command:
Code:
adb shell getprop | grep 'ro.build.id'
Yields something similar to this: [ro.build.id]: [MMB29K]
3. Match it up to the Nexus build numbers (This info is for AT&T Note 5 Marshmallow MMB29K, get your specific build number!)
https://source.android.com/source/build-numbers.html#source-code-tags-and-builds
MMB29K android-6.0.1_r1 Marshmallow Nexus 5, Nexus 5X, Nexus 6, Nexus 7 (flo/deb), Nexus 9 (volantis/volantisg)
===>] Bring down a specific AOSP source branch [<===​
4. Make a directory for the source code.
Code:
mkdir /home/droidvoider/Desktop/AOSP_Android_6.01_r1
5.
Code:
cd /home/droidvoider/Desktop/AOSP_Android_6.01_r1
6. Bring down the sources, this one is approximately 13 gigabytes
Code:
repo init --depth=1 -u https://android.googlesource.com/platform/manifest -b android-6.0.1_r1
repo sync
===>] I'm not sure the rest of this is needed [<===​For compiling toolbox the remainder wasn't needed.. But I have a large list of things to do so I can't test each item. If you can't compile a specific module continue reading.
===>] Merge Vendor sources & AOSP sources [<===​
7. Download the available sources for your device. In this example I downloaded PE6 Marshmallow sources for AT&T Note 5:
http://opensource.samsung.com/reception/receptionSub.do?method=sub&sub=F&searchValue=SM-N920A
8. Read the readme file from the sources platform zip to understand how to merge them into the AOSP sources. For the 2 Samsungs I've worked with the idea is to replace any directory that already exists. But if there is just one file such as core.mk only replace the one file. Then edit the .mk files as described in your readme. (and/or take info from cyanogen/lineagos) -- <I can help more, ask>
note: you probably don't need to take the configs from LineageOS and put them into your .mk files. However, if you do need to get more configs then you should get a big fat message when you type make 'modulename'. At first only edit .mk files as described by vendor device source readme file.
===>] Merge in Hardware drivers and etc [<===​possibly unnecessary depends what you're doing
9. Obtain a copy of the firmware for your device that matches the version of the source code you are able to download from your vendor.
for me that was Build Number: MMB29K.N920AUCU2BPE6 but your mileage will almost certainly vary!
10. Download https://github.com/anestisb/android-simg2img
11. Unzip it right in your download folder, open the folder and then 'open in terminal'
12. Make it and then move it a directory in your path. Warning: My command puts in in the Ubuntu default /bin folder.
Code:
make
sudo mv append2simg img2simg simg2img simg2simg simg_dump.py /bin
13. Uncompress the AP file from the matching firmware and extract the system.img into it's own directory
then select that folder, right click, open in terminal
Code:
simg2img system.img sys.raw
mkdir sys
sudo mount -t ext4 -o loop sys.raw sys/
14. A drive mounted, look on your task bar it should've wiggled too. Copy the etc and vendor folders into the main folder of the sources we are merging
Click to expand...
Click to collapse
And where is exactly the main folder? Sorry, Im just confused

DigitalDoraemon said:
And where is exactly the main folder? Sorry, Im just confused
Click to expand...
Click to collapse
it's no problem this stuff isn't easy to just figure out on your own. remember to substitute droidvoider for your ubuntu user name
In this example my sources are on my desktop in a folder named Android_6.01_r1
Sources for toolbox for example:
/home/droidvoider/Desktop/Android_6.01_r1/system/core/toolbox/<sources will be here including Android.mk>
Script for modules, including toolbox
/home/droidvoider/Desktop/Android_6.01_r1/Makefile <--- this is our modules script, if you will
<open a terminal in the above folder then use that Makefile like so>
make toolbox <---- this will compile only what is needed to compile the module 'toolbox' (this takes a minute)
Out export folder we decided in ./home/droidvoider/bashrc
/home/droidvoider/Android_Build_Out/Android_6.01_r1/target/product/generic/system/bin

Anybody, please compile grep utility for arm and x86... Minimum Platform Version Android 4.0.3, API Level - 15
Thanks

Great & useful .