[GUIDE][HOW_TO] BUILD a LINUX KERNEL FROM SOURCE [UBUNTU] - General Topics

This is a general open source linux development thread!
Android's kernel is a derivative of linux's kernel. Its good to know how to build both of these kernels. You might be already familiar with building kernels for various devices from sources. So I have made a new thread for guiding people on how to compile linux kernel from source (example taken as ubuntu kernel).
Requirements:
Any linux os x64 bit(example here: ubuntu 14.04)
Git (sudo apt-get install git)
Minimum of 4GB RAM and some reasonable linux-swap
To get the currently running kernel image, type the following:
Code:
apt-get source linux-image-$(uname -r)
Now we need to obtain Ubuntu Kernel Sources from its repositories. Make a new directory and inside it, initialise the git and clone the repository.
Code:
git clone git://kernel.ubuntu.com/ubuntu/ubuntu-<release>.git
<release> : Type in the required source. It can be lucid, precise, trusty, utopic etc.
Setting up the build environment. There are lots of tools and packages that are very much essential for building a kernel. These tools can be downloaded as a whole bundle and can be installed easily. Here's the code to set it up:
Code:
sudo apt-get build-dep linux-image-$(uname -r)
NOTE: The above comand can be executed only after you obtain the currently running kernel image. I have already given the code to obtain it above.
Now, change directory to the root of the kernel and type the following:
Code:
chmod -R a+x *
The above code will set the required permissions for building and executing the kernel.
Now, run these two commands:
Code:
fakeroot debian/rules clean
fakeroot debian/rules editconfigs
The first command cleans up the code automatically.
The slightly tricky part is with the second line of the code. When you execute it, you will have to edit a series of menuconfigs. To make changes to the configuration file we need to edit the configuration file. The kernel developers have created a script to edit kernel configurations which has to be called through the debian/rules makefile, unfortunately you will have to go through all the flavors for this script to work properly. The script will ask you if you want to edit the particular configuration. You should not make changes to any of the configurations until you see your wanted flavour configuration
We have now covered about 70% of progress. The rest is building the kernel and testing it.
Building the kernel is quite easy. Change your working directory to the root of the kernel source tree and then type the following commands:
Code:
fakeroot debian/rules clean
fakeroot debian/rules binary-headers binary-generic
If the build is successful, a set of three .deb binary package files will be produced in the directory above the build root directory. For example after building a kernel with version "3.13.-0.35" on an amd64 system, these three .deb packages would be produced:
Code:
cd ..
ls *.deb
linux-headers-3.13.0-35_3.13.0-35.37_all.deb
linux-headers-3.13.0-35-generic_3.13.0-35.37_amd64.deb
linux-image-3.13.0-35-generic_3.13.0-35.37_amd64.deb
Testing the new kernel
Install the three-package set (on your build system, or on a different target system) with dpkg -i and then reboot:
Code:
sudo dpkg -i linux*3.13.0-35.37*.deb
sudo reboot
Guys, I hope I have made an easy tutorial. You are always welcome to ask doubts (even on PM). Thank You.

Specific Hardware/Architecture
Creating a new config
I’ll be using the method of creating a new flavour, this adds a bit more work but this way you can always compile the original kernels.
We’ll use the generic flavour as the base for our own flavour being i7, this extension needs to be in small caps.
Code:
cp debian.master/config/amd64/config.flavour.generic debian.master/config/amd64/config.flavour.i7
fakeroot debian/rules clean
debian/rules updateconfigs
To make changes to the configuration file we need to edit the configuration file. The kernel developers have created a script to edit kernel configurations which has to be called through the debian/rules makefile, unfortunately you will have to go through all the flavours for this script to work properly.
Code:
debian/rules editconfigs
The script will ask you if you want to edit the particular configuration. You should not make changes to any of the configurations until you see the i7 configuration
Code:
Do you want to edit config: amd64/config.flavour.i7? [Y/n]
Make your changes, save the configuration and then keep going until the script ends.
When you’re done, make a backup of the config flavor file.
Code:
cp debian.master/config/amd64/config.flavour.i7 ../.
Now we need to clean up the git tree in order to get ready for compilation.
Code:
git reset --hard
git clean -df
Getting ready for compilation
Because we are going to be creating a new flavour based on a existing flavour (generic in my case) we need to create some extra files. During compilation the process checks the previous release for some settings, as we’re creating a local flavour it doesn’t exist in the source, so we’re creating it.
To see the previous release we use:
Code:
ls debian.master/abi
cp debian.master/abi/3.0.0-12.20/amd64/generic debian.master/abi/3.0.0-12.20/amd64/i7
cp debian.master/abi/3.0.0-12.20/amd64/generic.modules debian.master/abi/3.0.0-12.20/amd64/i7.modules
Copy our flavored configuration file back.
Code:
cp ../config.flavour.i7 debian.master/config/amd64/
We need to edit some files:
File: debian.master/etc/getabis
Search for the line:
Code:
getall amd64 generic server virtual
Change it in:
Code:
getall amd64 generic server virtual i7
File: debian.master/rules.d/amd64.mk
Search for the line:
Code:
flavours = generic server virtual
Change it in:
Code:
flavours = generic server virtual i7
File: debian.master/control.d/vars.i7
This files does not exist and in order to make the compilation process aware of our own flavor we want to compile we need to create it.
Code:
cp debian.master/control.d/vars.generic debian.master/control.d/vars.i7
You can edit the file and make it your own.
Code:
arch="i386 amd64"
supported="i7 Processor"
target="Geared toward i7 desktop systems."
desc="x86/x86_64"
bootloader="grub-pc | grub-efi-amd64 | grub-efi-ia32 | grub | lilo (>= 19.1)"
provides="kvm-api-4, redhat-cluster-modules, ivtv-modules, ndiswrapper-modules-1.9"
We need to commit our changes in the git repository.
Code:
git add .
git commit -a -m "i7 Modifications"
The text after -m is the message you add to your commit.
Compilation
It’s finally time for compiling, to keep our newly created branch in pristine condition we will do the compilation in a separate branch. We keep our branch clean as this will help later on when we want to update our new branch to a newer kernel.
Code:
git checkout -b work
fakeroot debian/rules clean
All the packages will be created in the directory /d1/development/kernel/ubuntu/oneiric
Create independent packages:
Code:
skipabi=true skipmodule=true fakeroot debian/rules binary-indep
The above statement will create the following deb files:
Code:
linux-doc_3.0.0-13.21_all.deb
linux-headers-3.0.0-13_3.0.0-13.21_all.deb
linux-source-3.0.0_3.0.0-13.21_all.deb
linux-tools-common_3.0.0-13.21_all.deb
Create the tools package:
Code:
skipabi=true skipmodule=true fakeroot debian/rules binary-perarch
The above statement will create the following deb file:
Code:
linux-tools-3.0.0-13_3.0.0-13.21_amd64.deb
Create the flavour depended files:
Code:
skipabi=true skipmodule=true fakeroot debian/rules binary-i7
The above statement will create the following deb files:
Code:
linux-headers-3.0.0-13-i7_3.0.0-13.21_amd64.deb
linux-image-3.0.0-13-i7_3.0.0-13.21_amd64.deb
Installation
After the compilation is finished we’ll have the above packages in the parent directory.
To install the files
Code:
cd ..
sudo dpkg -i linux-headers-3.0.0-13-i7_3.0.0-13.21_amd64.deb linux-headers-3.0.0-13_3.0.0-13.21_all.deb linux-image-3.0.0-13-i7_3.0.0-13.21_amd64.deb
Check your bootloader if the newly installed Ubuntu kernel is the default one, for grub check the file /boot/grub/menu.lst or if you run grub2 check /boot/grub/grub.cfg

thx for your info

nice job mate..!! :good:
now i'm gonna try this..!!

Nice ,i can't say anything

faizauthar12 said:
Nice ,i can't say anything
Click to expand...
Click to collapse
Thank you for the great guide!!!

Nice thread. I'll try it at home

Thanks
Enviado de meu Moto G usando Tapatalk

Thx for the guide
tra_dax

Related

[DEV] Kernel development HOWTO and Interactive menu

I havent yet found a simple guide for compiling kernels. Some of them assume too much, and some are just outdated. So I thought I'd write my own for devs/budding devs. Here you go!
Note:
This is not a guide for newbies. It's a dev guide for devs.
Research before asking questions, please
For The Menu driven interactive kernel build script, see Post #31
I will be developing this guide as I go, so it will be incomplete initially, or lacking in detailed explanations.
Essentials:
Ubuntu Box (By this I mean a PC with a Ubuntu installation, not a live CD)
A toolchain-Either the Android NDK, or your own toolchain
HTC Desire GB/Froyo source from htcdevs.com, or sources from github
Familiarity with the linux shell and basic linux commands.
The will to learn
First things first,
1. Getting the sources
The HTC Desire source is available from two kinds of resources-you can either get it from htcdevs.com (official HTC Dev site), or from source code uploaded from someone else. For the purpose of this tutorial, I'll assume we're working on the official HTC GB source code. So download bravo_2.6.35_gb-mr.tar.gz from htcdevs.com.
2. Setting up the compilation box and preparing source code
2.1 Install some essential linux packages from the Linux terminal:
Code:
sudo apt-get install libncurses5-dev
2.2 Extract the source code
The file you downloaded is a tar archive (like a zip file), so you need to extract it to a convenient location. Let's hit the linux shell-open a terminal window in linux (Accessories->Terminal)
Type:
Let's go to our home directory:
Code:
cd ~/
Now, create the directories for our kernel compilation box.
Code:
mkdir -p ~/android/kernel
Now you need to copy the tar.gz file from wherever you downloaded it to, to this dir.
Extract the archive:
Code:
tar -xvf ~/android/kernel/bravo_2.6.35_gb-mr.tar.gz
cd ~/android/kernel/bravo_2.6.35_gb-mr
Now we can view the extracted files within the directory ~/android/kernel/bravo_2.6.35_gb-mr/
2.3 Set up the toolchain
A toolchain is a set of programs which allow you to compile source code (any source code, not just kernels). The toolchain is specific for the processor and hardware, so we need a toolchain specific for Android and especially the Desire. If you're a semiadvanced-pro user, you may consider compiling your own toolchain (See theGanymedes' guide for doing so). If compilation of kernels is all that you require, fortunately for you, there is an easy way-the Android NDK - v7 (latest as of now) is available here
Get the NDK for Linux - android-ndk-r7-linux-x86.tar.bz2
Code:
mkdir -p ~/android/ndk
Now copy the NDK file to ~/android/ndk
Whenever I say copy, you have to manually copy the file with any file manager. Nautilus comes with Ubuntu, and Dolphin with Kubuntu. You may also use the shell of course with
Code:
cp [sourcefile] [destination]
Extract it:
Code:
tar -jvxf android-ndk-r7-linux-x86.tar.bz2
Now add the path for your toolchain to the env variable:
Code:
gedit ~/.bashrc
At the end of the file, add this line:
Code:
PATH=$PATH:~/android/ndk/android-ndk-r7-linux-x86/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin
3. Setting up kernel parameters
Kernels are compiled with a program called gnu make, and use a set of configuration options specified within a file called Makefile.
A vital point to note is that kernels are compiled with a program called gcc (basically the gnu C compiler), and our NDK itself has its own optimized version of gcc. While compiling, we're actually cross compiling it (meaning compiling a binary package on a system which is different from the actual system which is meant to run it- you're compiling it on your PC while it's actually meant to run on your Desire)
This means that when you compile it, you have to make sure that you compile it with the NDK's version of gcc instead of the system version. Otherwise you end up with a kernel meant to run on your pc, duh! Specifying which gcc to use is by the CROSS_COMPILE variable. You can set it up with this command:
Code:
CROSS_COMPILE=arm-linux-androideabi-
Note the hyphen (-) at the end, and do not forget to include it! At compilation time, system will actually use this variable to find all the programs it needs. Eg: The path for gcc will become arm-linux-androideabi-gcc
We can compile kernels with many different options, like with ext4 support, or without; ext4 support as part of the kernel zImage (in which case it makes the kernel larger), or as a loadable module (of the form somename.ko, which is loaded at init.d/init.rc with the command insmod modulename.ko)
We specify the exact options we require with the help of a useful configuration program called menuconfig (which as the name suggests, is a menu for configuration of make options).
An important thing to note is that as far as kernel compilation is concerned, there are a vast amount of options to setup, and unless you're thorough with kernel compilation, you wont be able to set up the options correctly and get your kernel to boot. Fortunately for us, the kernel source already comes with a default set of parameters which can be easily set up.
Note that all make commands must be executed within the directory bravo_2.6.35_gb-mr. Let's go there now:
Code:
cd ~/android/kernel/bravo_2.6.35_gb-mr
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- bravo_defconfig
This produces a .config file (used by the menuconfig) containing essential parameters to produce a booting kernel for the Desire.
Note: There is a simpler way to get the basic .config file, and this is to get it from a running kernel built by someone else. You can extract the .config from a running kernel with these commands:
Code:
cd ~/android/kernel/bravo_2.6.35_gb-mr
adb pull /proc/config.gz
zcat config.gz > .config
Now we can open menuconfig and add anything we need in addition.
Code:
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- menuconfig
You can view the huge amount of options available in menuconfig.
You can add ext4 support for example (See image above)
Once you're done choosing options, you can exit menuconfig.
4. Compiling it
This is simple. The basic command is:
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- -j10
The -j10 specifies the number of jobs to execute per operation. I can usually go upto 50 on my Quad core CPU. Beware, this can bring a slow CPU to a crawl and freeze up linux itself.
During compilation, you will see all sorts of messages, which may include warnings too. In most cases, its safe to ignore warnings. If there are errors, the compilation will stop, and you will have to fix the issues.
5. Distributing your kernel to users
At the end of compilation, it generates files named zImage, and various .ko files.
You have to copy them from their default location to a zip file. The best way is to use my variant of koush's Anykernel, and copy the files to it. Then, you can zip the whole folder and lo and behold-you have your flashable kernel zip which you can distribute to others.
You can also remove the zImage and the modules from /system/lib/modules of any kernel zip available with you, and copy over your files to it, at the correct location.
So, let's say that you have extracted an existing kernel zip to the location ~/flashable
The file structure should be like this:
Code:
|-- kernel
| |-- dump_image
| |-- mkbootimg
| |-- mkbootimg.sh
| |-- unpackbootimg
| `-- zImage
|-- META-INF
| |-- CERT.RSA
| |-- CERT.SF
| |-- com
| | `-- google
| | `-- android
| | |-- update-binary
| | `-- updater-script
| `-- MANIFEST.MF
`-- system
`-- lib
`-- modules
`-- bcm4329.ko
8 directories, 11 files
I've included my flashable zip directory along with this post. Download file kernel_flashable.tar.bz2.zip to ~/
Code:
cd ~/
tar -jvxf kernel_flashable.tar.bz2.zip
This will create the directory structure outlined above.
Now after every compilation of the kernel, execute these commands from where you executed make:
Code:
cp arch/arm/boot/zImage ~/kernel_flashable
find . -name '*ko' -exec cp '{}' ~/kernel_flashable/system/lib/modules/ \;
cd ~/kernel_flashable
zip -r mykernel ./
This will create mykernel.zip at ~/kernel_flashable. You can distribute this to your users to flash. Make sure you edit updater-script before though
Common errors and other stuff
Ok, post #1 was simple stuff. Now, supposing you get errors while compiling. Post #2 is about that, and ups the level of knowledge a bit..
Some kernel compilation errors:
Treat warnings as errors-Solved by removing the string "-Werror" from all Makefiles of the file which failed to compile. Some people had said that the real error (Array out of bounds warning) was because of gcc optimizations. But putting -O2 to -O0 didnt do a thing.
No of jobs - ought not to exceed 50.
"warning: variable set but not used [-Wunused-but-set-variable]"-Look at KBUILD_CFLAGS in the main Makefile. Add -Wno-error=unused-but-set-variable to the existing set of flags.
Note the following from gcc manual:
-WerrorMake all warnings into hard errors. Source code which triggers warnings will be rejected.
-w Inhibit all warning messages. If you're familiar with C code and like to fix stuff, rather than ignoring potential bugs, use this only as a last resort- A 'brahmastram' (most powerful weapon in your time of gravest need) as the epics would say
-WerrorMake all warnings into errors.
-Werror=Make the specified warning into an error. The specifier for a warning is appended, for example -Werror=switch turns the warnings controlled by -Wswitch into errors. This switch takes a negative form, to be used to negate -Werror for specific warnings, for example -Wno-error=switch makes -Wswitch warnings not be errors, even when -Werror is in effect. You can use the -fdiagnostics-show-option option to have each controllable warning amended with the option which controls it, to determine what to use with this option.
So what I did to suppress errors was to add:
Code:
KBUILD_CFLAGS += -w
KBUILD_CFLAGS += -Wno-error=unused-but-set-variable
Though the -Wunused-but-set-variable is not a real issue in itself, it generates so much "noise" that you may miss actual make errors.
This is the error what I was talking about..
Code:
drivers/net/wireless/bcm4329_204/wl_iw.c: In function 'wl_iw_set_pmksa':
drivers/net/wireless/bcm4329_204/wl_iw.c:5075:5: error: array subscript is above array bounds [-Werror=array-bounds]
drivers/net/wireless/bcm4329_204/wl_iw.c:5078:5: error: array subscript is above array bounds [-Werror=array-bounds]
Solution:
Edit drivers/net/wireless/bcm4329_204/Makefile
Locate -Werror within DHDCFLAGS, and delete it.
Code:
DHDCFLAGS = -DLINUX -DBCMDRIVER -DBCMDONGLEHOST -DDHDTHREAD -DBCMWPA2 \
-DUNRELEASEDCHIP -Dlinux -DDHD_SDALIGN=64 -DMAX_HDR_READ=64 \
-DDHD_FIRSTREAD=64 -DDHD_GPL -DDHD_SCHED -DBDC -DTOE -DDHD_BCMEVENTS \
-DSHOW_EVENTS -DBCMSDIO -DDHD_GPL -DBCMLXSDMMC -DBCMPLATFORM_BUS \
-Wall -Wstrict-prototypes -Werror -DOOB_INTR_ONLY -DCUSTOMER_HW2 \
-DDHD_USE_STATIC_BUF -DMMC_SDIO_ABORT -DWLAN_PFN -DWLAN_PROTECT \
-DBCMWAPI_WPI \
This will prevent gcc from treating mere warnings as errors.
How to modify kernels by applying mods - Applying Kernel Patches
Ok, you have compiled a simple stock kernel. Now what? Would you like to add fixes/mods developed by other kernel devs? This post explains patches and how exactly to do this.
Patches to the kernel are applied via patch files. Patch files are simple text files generated by the linux diff program which takes two text files, compares them and writes the differences (hence called diff) to another text file which by convention has the extension .patch
Attached to this post is a patch containing my "Extended battery" fix with Sibere's battfix. I'll explain patching with this. Let's understand the patch file. Open it up in any text editor.
Code:
diff -rupN -X /home/droidzone/android/kernel/exclude.opts bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c
--- bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c 2011-08-25 13:16:53.000000000 +0530
+++ bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c 2011-11-06 16:43:21.544317342 +0530
@@ -118,8 +118,11 @@ PS. 0 or other battery ID use the same p
/* Battery ID = 1: HT-E/Formosa 1400mAh */
#define BATT_ID_A 1
#define BATT_FULL_MAH_A 1400
-
#define BATT_FULL_MAH_DEFAULT 1500
+#define BATT_FULL_MAH_CAMERONSINO 2400
+#define BATT_ID_CAMERONSINO
+#define BATT_TYPE 0
+
Note the first line:
Code:
diff -rupN -X /home/droidzone/android/kernel/exclude.opts bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c
diff -rupN basically describes the command that was used to generate this patch. The -u means that the patch file is something called a universal patch
bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c was the original file, and bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c was the target file or file which contains the mod..
How to apply patch files?
The command depends on where your current directory is. If you're in ~/android/kernel/bravo_2.6.35_gb-mr/ and your current directory contains the directory 'drivers', you can apply this patch with this command:
Code:
patch -p1<extended_battfix.patch
If you're within drivers, then you have to modify the command like this:
Code:
patch -p2<extended_battfix.patch
Hope you get the gist. Basically, as you move into the source tree, you have to increment the patch level by the number of directories you've moved down into. Very simple, isnt it?
Sharing and Collaborating - Using Github and Commits
Kernel compilation is a group effort (at least it ought to be). When different devs work on different parts of the code and create their own mods, development progresses. For this purpose, it is important that you share your code with other devs. The best way to do this to upload your sources to github.
First, create a github account.
Next you can view other devs' github sources and examine their commits. Commits are basically patches applies to the previous source uploaded. Github commits use the universal patch format and can be viewed directly, downloaded as patch files, and applied to your code. You can also choose to download the whole source tree uploaded by another dev and examine it.
Kernel Build Interactive Menu system
This saves quite a lot of time if you make kernels a lot..
See post #22
Ok, the basic guide is done, guys... If you have doubts, I'll try to clear them
yeah....yeah....yeah...so nice...big thx...will try this as soon as possible..
that is what i searchd so long
edit: rated with 5 stars
with kind regards
Thank you very much droidzone.
I was waiting for a n00b guide.
Tapatalking
good job droidzone
[+1] [ i like]
Added a Howto on how to apply kernel source patch files, to post #3
lol now i understand how patching works.. i write all this **** by myself.. lol
Midian666 said:
lol now i understand how patching works.. i write all this **** by myself.. lol
Click to expand...
Click to collapse
Ha ha.. that would not have been so easy
Droidzone said:
Added a Howto on how to apply kernel source patch files, to post #3
Click to expand...
Click to collapse
sorry for offtopic but nice again and you see many people thought like me with the how to..
with kind regards...Alex
Alex-V said:
sorry for offtopic but nice again and you see many people thought like me with the how to..
with kind regards...Alex
Click to expand...
Click to collapse
I like explaining stuff and sharing..
This guide was written specifically because of your request, and I have never forgotten how you helped when I was a newbie to development.. I wouldnt probably have started developing if not for good responses from Firerat and you.
Droidzone said:
I like explaining stuff and sharing..
This guide was written specifically because of your request, and I have never forgotten how you helped when I was a newbie to development.. I wouldnt probably have started developing if not for good responses from Firerat and you.
Click to expand...
Click to collapse
and now i learn from you lol thx
with kind regards..Alex
Fantastic guide!!!!!!!
Did some more work on the first post. It now includes a flashable zip template and instructions on how to easily create your own flashable zip after compilation is over.
maybe some improvments to your making a flashable zip.
i did this with my kernels.. it took the version infos from the config files.. and put it into a folder... after this u can make zip.
ive stolen this from manus source
Code:
localVersion=`cat .your-config | fgrep CONFIG_LOCALVERSION= | cut -f 2 -d= | sed s/\"//g`
linuxVersion=`cat .your-config | fgrep "Linux kernel version:" | cut -d: -f 2 | cut -c2-`
VERSION=$linuxVersion$localVersion
echo "Kernel version=$VERSION"
rm -rf flash/system/lib/modules/*
mkdir flash/system/lib/modules/$VERSION
mkdir flash/system/lib/modules/$VERSION/kernel
tar czf modules.tgz `find . -name '*.ko'`
cd flash/system/lib/modules/$VERSION/kernel
tar xzf ../../../../../../modules.tgz
cd - > /dev/null
rm modules.tgz
This is good..Actually when I generate kernels I test too many versions that I dont usually change the local version number in the menuconfig. Instead I use the date and time (including second) to name the kernel dir and kernel zip name...
Like this..
Code:
date_str=`date '+%d%m%y_%H%M%S'`
dirname="kernel_"$nameflag"_"$date_str
pckdir="$packagedir/$dirname"
mkdir $pckdir
lastfolder=$pckdir
cd $outdir/
echo
zipnoname="kbase_"$nameflag"_"$date_str
zipaddnoname="kmods_"$nameflag"_"$date_str
zipname=$zipnoname".zip"
zipaddname=$zipaddnoname".zip"
zip -r $zipnoname ./
mv $zipname $pckdir/
As you can see, its part of my script which does a lot of things..
But getting the localversion too is a good thing..I'd put it into a textfile in the zip which users can read..
Great guide. Thanks a lot
Sent from my HTC Desire using Tapatalk

[TUTORIAL/DEV]Build AOSP Android 2.3.7 for Allwinner A10 tablets (Teclast P76Ti)

Hi!
I haven't really found any kind of guides on how to build a complete AOSP android build from source (kernel and software) for Allwinner A10 tablets like the Ainol Novo 7 Advanced or the Teclast P76Ti.
I have a Teclast P76TI (rev4), and the following guide is only tested on such a device. As it's mostly based on the source drop of the Ainol Novo 7 Advanced, it should also work on that too.
Part 1
Part 1: Preparations
A: Preconditions
For building and creating a LiveSuite flashable package you will need the following:
For building the kernel and android you will need a 64 bit Linux machine. Ubuntu 10.04 64 bit is preferred for Gingerbread builds
For creating the LiveSuite flashable images, and actually flashing you will need a Windows machine. Windows 7 x64 works fine, as should Windows XP 32 bit too.
You will also need a working LiveSuite image file, for your tablet. Make sure that you can flash it to your device without problems!
Virtual machines are fine, if you have enough memory available. To build Android in a VM, you should give it at least 2GB of RAM. Alternatively if you're using Linux, you can create a Windows VM for the image creation or flashing part.
For the rest of the tutorial I will assume you are using a Linux Virtual Machine on a Windows box.
For building the kernel and AOSP you will need at least 35GB of free space in your linux machine (the more the better). To create the images you will need an additional 2GB of space on your Windows machine.
B: Getting the kernel and AOSP sources
First initialize your Linux machine according to this page: http://source.android.com/source/initializing.html
Here are the most important commands:
Code:
$ sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
$ sudo apt-get update
$ sudo apt-get install sun-java6-jdk
$ sudo apt-get install git-core gnupg flex bison gperf build-essential \
zip curl zlib1g-dev libc6-dev lib32ncurses5-dev ia32-libs \
x11proto-core-dev libx11-dev lib32readline5-dev lib32z-dev \
libgl1-mesa-dev g++-multilib mingw32 tofrodos python-markdown \
libxml2-utils xsltproc
$ mkdir ~/bin
$ PATH=~/bin:$PATH
$ curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > ~/bin/repo
$ chmod a+x ~/bin/repo
// Additonal commands for Ubuntu 10.10:
$ sudo ln -s /usr/lib32/mesa/libGL.so.1 /usr/lib32/mesa/libGL.so
// Additonal commands for Ubuntu 11.10:
$ sudo apt-get install libx11-dev:i386
Now create a directory on your linux machine. The path should contain no spaces in it. Inside this directory first pull the kernel source from the allwinner github page:
Code:
$ git clone https://github.com/allwinner/linux-2.6.36 lichee
Make sure that the kernel resides in the lichee directory
Now download AOSP Android. We will use Gingerbread version 2.3.7:
Code:
$ mkdir android
$ cd android
$ repo init -u https://android.googlesource.com/platform/manifest -b android-2.3.7_r1
$ repo sync
Allwinner uses a special init, which has a few additional commands that you have to download too:
Code:
$ cd system/core
$ git pull git://github.com/sztupy/allwinner_android_system_core.git
$ cd ../..
You will also need the device descriptors for the tablets:
Code:
$ cd device
$ git clone git://github.com/sztupy/android_device_softwinner.git softwinner
$ cd ..
C: ARM compiler
You might simply use a compiler that is supplied for AOSP, they are inside the android/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/ directory. Alternatively you can also download the ARM compiler of CodeSourcery. Get the arm-2009q3 / arm-none-linux-gnueabi version, and unpack it inside a directory.
If ready, go on to next step: building the kernel
Part 2: Building the kernel
Change to the directory of the kernel. Now edit the file scripts/build_sun4i_crane.sh
Change the line "export CROSS_COMPILE=" to point to the ARM compiler. For example if you extracted the CodeSourcery files to /opt then use:
Code:
export CROSS_COMPILE=/opt/arm-2009q3/bin/arm-none-linux-gnueabi-
The default .config file is more or less the same that is supplied with your tablet. On the Teclast P76TI you can actually extract the .config file used from /proc using "adb pull /proc/config.gz" Not sure about other vendors.
Now you have to compile. Issue the command:
Code:
./build.sh -p sun4i_crane
This will build the kernel and modules inside the the "output" directory.
Next step is to compile Android
Part3
Part 3: Building Android
Go to the android directory. Now setup android using:
Code:
$ source build/envsetup.sh
$ lunch
You will get simething like this:
Code:
You're building on Linux
Lunch menu... pick a combo:
1. generic-eng
2. simulator
3. full_passion-userdebug
4. full_crespo4g-userdebug
5. full_crespo-userdebug
6. crane_Ainol_Novo7A-eng
7. crane_Teclast_P76TI_REV4-eng
Which would you like? [generic-eng]
Here select your device (option 6 or 7 depending on your tablet)
After you have selected it issue a make command:
Code:
$ make
This will take a while. If you're using ubuntu 11.10 make might fail, but you can fix it using these fixes: http://groups.google.com/group/android-building/browse_thread/thread/3484e7797909d014
After the building is complete you have to create the image files. Run the following command:
Code:
$ ./device/softwinner/crane-tcp76ti_r4/mkfs.sh
(substitute tcp76ti_r4 with ainovo7a if you're using the other tablet)
The image files will be ready in the directory "out/target/product/crane-tcp76ti_r4/images/" or "out/target/product/crane-ainovo7a/images/"
Copy the three files from here (root.img, recovery.img, system.img), and the "bImage" file from the "output" directory of the kernel to your Windows machine.
The next step is creating the flashable image, and flashing your build
Part 4: Flashing
A: Initialize kitchen
First download my A10 flash kitchen from this site: http://android.sztupy.hu/dl/a10/a10_flash_kitchen_v1.zip
This is based on the ainol novo 7 soruce drop, but only contains the nescessary files, and has a few batch scripts to automate the process.
There is also a v2, wich also supports ICS firmwares: http://android.sztupy.hu/dl/a10/a10_flash_kitchen_v2.zip
First unzip the files to a directory. You will see a few directories and to commands here: extract_image.bat and create_image.bat
First we have to extract an official image for your tablet. Simply comply the image file to this directory, and rename it to "original.img". If done, run extract_image.bat. This command should extract your image to the _extract directory.
Next you need to extract the bootfs inside the _bootfs directory. You have multiple options on how to do this:
1. Get the bootfs from your device on-line using adb:
Code:
> cd _bootfs
> adb shell
$ cd /
$ mount -o remount,rw -t ext4 /dev/root /
$ mkdir bootfs
$ mount -t vfat /dev/block/nanda /bootfs
(Control-C: exit adb shell)
> adb pull /bootfs
2. Get the bootfs from the extracted image file using Linux:
A. Copy the RFSFAT16_BOOTFS_000000000.fex file to linux
B.
Code:
$ mkdir bootfs
$ sudo mount -o loop RFSFAT16_BOOTFS_000000000.fex bootfs
$ cp -a bootfs b2
$ sudo umount bootfs
C. Copy the contents of the b2 directory to Windows, inside the _bootfs directory of the kitchen
B: Create image
If you have initialized the kitchen, you have to copy the four files from linux (system.img, root.img, recovery.img and bImage) inside the _input directory. You will also need to rename the .img files to .fex (so they should be system.fex, root.fex, recovery.fex).
If you are done with this, then run create_image.bat. It will create an output.img file, that can be flashed to the device using LiveSuite.
Additional information for developers:
1. Difference between AOSP 2.3.4 and Novo7 android 2.3.4
Download the diff file from here: http://android.sztupy.hu/dl/a10/diff_ainovo_aosp.gz
Filelist of differences: http://android.sztupy.hu/dl/a10/diff_ainovo_aosp_filelist.txt
sztupy, did you figure out why eDragonEx and FSBuild have an apparently unused Lua library next to each? I was interested by it, as it might be possible that some parts of the apps are actually written in Lua (would it be possible? Lua in a native DLL?), what would help reversing it.
fonix232 said:
sztupy, did you figure out why eDragonEx and FSBuild have an apparently unused Lua library next to each? I was interested by it, as it might be possible that some parts of the apps are actually written in Lua (would it be possible? Lua in a native DLL?), what would help reversing it.
Click to expand...
Click to collapse
Yeah, saw that. Also their build script used "convert" on the image.cfg, but that file wasn't in lua anyway (except for the image parts part which seems to be a lua hash).
There are also a lot of compiled files that are compiled inside the image file and which are neither the bootfs nor android or the kernel. I don't yet know what they are for, but I think they are used by LiveSuite during the flashing.
sztupy said:
Yeah, saw that. Also their build script used "convert" on the image.cfg, but that file wasn't in lua anyway (except for the image parts part which seems to be a lua hash).
There are also a lot of compiled files that are compiled inside the image file and which are neither the bootfs nor android or the kernel. I don't yet know what they are for, but I think they are used by LiveSuite during the flashing.
Click to expand...
Click to collapse
I know most of the files from the flashing, they are the following:
- SYS_CONFIG is used as a command bunch for LiveSuite. It tells the app how to flash, what to flash, where, and it configures the device too (screen size, ram info, cpu info, etc)
- Boot0 and Boot1 bins are NAND bootloaders
- FED FES and FET tools are NAND flashing utilities, checksums, hardware scanner, and other tools used during flashing.
- '12345678' files are bootloaders, config files, and tools for SDMMC flashing, if there's a device with SDMMC internal instead of NAND, these are used
- Split seems to be some kind of ID string, I had no luck retrieving it's usage and meaning.
So you say, that the actual Lua parts are the image encryption parts of eDragonEx? Interesting, maybe I've missed that spot with my tool...
Might I ask if you have tried disassembling (and decompiling) unimg.exe? I had several problems with it, but that would greatly help understanding how the images are created. I've got a C# framework, with image config parser, etcetera etcetera, to be able to read and create images in a much more advanced environment (filtering user errors, having tools for everything (bootfs modding, script.bin reversal, etcetera), and creating a working image as a final result), and it only needs the image file format (and some of my work, to create a parser).
The problems with unimg were all about positive sp values, and as I'm not a big assembly programmer, I couldn't make out anything from that. Maybe you understand it a bit more
Illetve beszélhetünk egy kicsit magyarul is. Tabletrepublic-on írtam hogy vegyél fel MSNre és részletezem a különböző Crane SDK elemek funkcióit, működését, egyebeket, amit eddig sikerült kiderítenem az egészről.
Sajnos elég zavaros, mivel vagy négyféle csomagoló rendszer készíti a fileokat, és ezek közül csak egy működött rendesen (crane_pack.exe). Jó lenne megérteni ezt a file formátumot, hogy egy kicsit normálisabb módon hozhassam létre, különféle vacakolások nélkül.
fonix232 said:
I know most of the files from the flashing, they are the following:
- SYS_CONFIG is used as a command bunch for LiveSuite. It tells the app how to flash, what to flash, where, and it configures the device too (screen size, ram info, cpu info, etc)
- Boot0 and Boot1 bins are NAND bootloaders
- FED FES and FET tools are NAND flashing utilities, checksums, hardware scanner, and other tools used during flashing.
- '12345678' files are bootloaders, config files, and tools for SDMMC flashing, if there's a device with SDMMC internal instead of NAND, these are used
- Split seems to be some kind of ID string, I had no luck retrieving it's usage and meaning.
Click to expand...
Click to collapse
Thanks for these. Seems I was mostly right
So you say, that the actual Lua parts are the image encryption parts of eDragonEx? Interesting, maybe I've missed that spot with my tool...
Click to expand...
Click to collapse
I think lua is not really used anymore. It probably had more relevance back in the past.
Might I ask if you have tried disassembling (and decompiling) unimg.exe? I had several problems with it, but that would greatly help understanding how the images are created. I've got a C# framework, with image config parser, etcetera etcetera, to be able to read and create images in a much more advanced environment (filtering user errors, having tools for everything (bootfs modding, script.bin reversal, etcetera), and creating a working image as a final result), and it only needs the image file format (and some of my work, to create a parser).
The problems with unimg were all about positive sp values, and as I'm not a big assembly programmer, I couldn't make out anything from that. Maybe you understand it a bit more
Click to expand...
Click to collapse
No, haven't tried disassembling it yet. I was very glad that it worked, and that I could create a whole working build just from the sources. I know there are a lot of quirks, like if the extension of the file is not .fex, then it will encrypt(?) it, etc. I might try it, but currently I'm more interested in getting a working AOSP ICS on my tablet. Besides for disassembly I need to be in a special mood, which I'm not really in now
Illetve beszélhetünk egy kicsit magyarul is. Tabletrepublic-on írtam hogy vegyél fel MSNre és részletezem a különböző Crane SDK elemek funkcióit, működését, egyebeket, amit eddig sikerült kiderítenem az egészről.
Sajnos elég zavaros, mivel vagy négyféle csomagoló rendszer készíti a fileokat, és ezek közül csak egy működött rendesen (crane_pack.exe). Jó lenne megérteni ezt a file formátumot, hogy egy kicsit normálisabb módon hozhassam létre, különféle vacakolások nélkül.
Click to expand...
Click to collapse
Már egy jó ideje nem használok MSN-t. Skype/GTalk/email viszont van. Ha nem használsz olyanokat, akkor azért majd felrakom.
Made a diff between AOSP android 2.3.4 and the Novo 7 2.3.4 source drop. The list can be found at post 6: http://forum.xda-developers.com/showpost.php?p=22397984&postcount=6
Nice tutorial! Do you mind if I fork your Github repo and add the device tree for the Bmorn V11 to the lunch list?
sztupy said:
Made a diff between AOSP android 2.3.4 and the Novo 7 2.3.4 source drop. The list can be found at post 6: http://forum.xda-developers.com/showpost.php?p=22397984&postcount=6
Click to expand...
Click to collapse
Most of the things changed aren't even needed for the device - ril can be replaced with a local one (and suggested by Google to do so), just like recovery changes, in ICS we already have USB BT support enabler, so BT changes can be dropped, just like framework changes (they are for the softbuttons on the notification bar), camera and mediaplayer changes should be local too (in the device tree), so 99% of changes can be dropped.
Additions are different, some can be totally erased, and some are needed. From your github, I see that you've already began making a cleaned up, generic A10 tree, but I miss a few things - libsensor for one, stagefright, camera, and audio. The AOSP stock ALSA should work, if the proper audio config is placed in a ROM, but I have bad feelings about the missing libsensors source, and camera. Stagefright has a chance too to work, but camera definitely won't, and the used sensors aren't the common ones to be included.
Használok GTalk-ot is, ott is ugyanez a nicknevem, gmail utótaggal, szóval a szokásos
FezzFest said:
Nice tutorial! Do you mind if I fork your Github repo and add the device tree for the Bmorn V11 to the lunch list?
Click to expand...
Click to collapse
That's what github is for
fonix232 said:
Most of the things changed aren't even needed for the device - ril can be replaced with a local one (and suggested by Google to do so), just like recovery changes, in ICS we already have USB BT support enabler, so BT changes can be dropped, just like framework changes (they are for the softbuttons on the notification bar), camera and mediaplayer changes should be local too (in the device tree), so 99% of changes can be dropped.
Additions are different, some can be totally erased, and some are needed. From your github, I see that you've already began making a cleaned up, generic A10 tree, but I miss a few things - libsensor for one, stagefright, camera, and audio. The AOSP stock ALSA should work, if the proper audio config is placed in a ROM, but I have bad feelings about the missing libsensors source, and camera. Stagefright has a chance too to work, but camera definitely won't, and the used sensors aren't the common ones to be included.
Használok GTalk-ot is, ott is ugyanez a nicknevem, gmail utótaggal, szóval a szokásos
Click to expand...
Click to collapse
libsensor and stagefright are still there, as I could manage them to get compiled. Not that they work though. For the libcamera it depends on CedarX, which I couldn't manage to compile, that's why I removed it (for now). Besides CedarX unfortunately not "open-source", so in theory we couldn't use it either (well... not that I actually care about licence violations).
I'm still compiling ICS. Will put back libcamera and try to get the other hardware libs to work after I managed to get ICS to boot.
sztupy said:
libsensor and stagefright are still there, as I could manage them to get compiled. Not that they work though. For the libcamera it depends on CedarX, which I couldn't manage to compile, that's why I removed it (for now). Besides CedarX unfortunately not "open-source", so in theory we couldn't use it either (well... not that I actually care about licence violations).
I'm still compiling ICS. Will put back libcamera and try to get the other hardware libs to work after I managed to get ICS to boot.
Click to expand...
Click to collapse
Please be noted that ICS requires new stagefright, camera HAL, new GPU drivers, and so on.
ICS also should have some differences in the build tree, make a new branch for sure (as an example, it requires a device.mk and device_base.mk, both being the base containers without target definition, full_[devicename].mk for the actual full target, and cm.mk for CyanogenMod, what I'd suggest you to build).
I couldn't find any of the sources, but must have overlooked something. Will check it further.
hi, I'm the author of unimg(esxgx).
and your unimg is not up-to-date(the version still have bugs to lead to fail the packing process.
Here is the latest version (fix bugs but no virus alarm[compared with the former version]), and you can use it in the same way.
please update your file.
PS. yes, the unimg can unpack and pack all allwinner's firewares without diffculty, good luck.
sorry for my english.
fonix232 said:
Might I ask if you have tried disassembling (and decompiling) unimg.exe? I had several problems with it, but that would greatly help understanding how the images are created. I've got a C# framework, with image config parser, etcetera etcetera, to be able to read and create images in a much more advanced environment (filtering user errors, having tools for everything (bootfs modding, script.bin reversal, etcetera), and creating a working image as a final result), and it only needs the image file format (and some of my work, to create a parser).
The problems with unimg were all about positive sp values, and as I'm not a big assembly programmer, I couldn't make out anything from that. Maybe you understand it a bit more
Click to expand...
Click to collapse
I think you should use unimg.exe.
unimg is the only tool to unpack it correctly.
you know, allwinner didn't want me to release the tool in public last year(the tool is for sc9800[the former chip]). for some commercial reason, the offical toolchain of a10 only contains pack_tool.
so I released the tool with other tools(rootcr, rootpk,etc..) in a small group.
but some person posted it on the internet. so....
I developed the analysis tools in the form of many files(unimg, rootcr, rootpk, unimg2), because I want to keep each of the packing stages simple.
you can use a "bat" file / a GUI shell to communicate with each other, and that is what I expected.
I can't smoothly speak english, so I modified several times.
Thank you very much!
If I may ask, would it be possible to release the source code too?
fonix232 said:
Thank you very much!
If I may ask, would it be possible to release the source code too?
Click to expand...
Click to collapse
I cannot released yet, but I will release it in a proper time on github (I have been touching with allwinner company, so I have to consider many factors).

Guide: Compile /system/bin binaries for your device from AOSP source code

Now tested up to downloading AOSP and make toolbox you should be all set
Please give thanks to this thread: https://forum.xda-developers.com/newreply.php?do=newreply&p=43622764
Warning: I hacked my way through this stuff a few weeks ago I am not an expert!
How to compile Android Open Source Code modules​
I don't compile C code on Windows machines I have no idea about that.
Notice
This guide is a quick and dirty how to make a module. It will not cover finalizing setting up the source codes for your device. It is only my goal to enable you to compile binaries such as grep, toolbox, dumpstate, dalvikvm, jack and etc.
===>] Setup Ubuntu 64bit [<===​Unplug that Windows drive, plug in a fresh hard drive and install Ubuntu latest/greatest. Ignore the recommendation to downgrade gnu make!, for now.
Open a terminal and issue these commands (warning ppa repository for OpenJDK 7 is said to have a security issue?, isn't being updated?.. whatevs it works)
Code:
sudo apt-get update
sudo apt-get upgrade
sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt-get install openjdk-7-jdk
sudo apt-get install openjdk-8-jdk
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386 lib32z1 libbz2-1.0:i386
sudo apt-get install git ccache automake lzop bison gperf build-essential zip curl zlib1g-dev zlib1g-dev:i386 g++-multilib python-networkx libxml2-utils bzip2 libbz2-dev libbz2-1.0 libghc-bzlib-dev squashfs-tools pngcrush schedtool dpkg-dev liblz4-tool make optipng
(choose Java 1.7 in the following way)
Code:
sudo update-alternatives --config java
(let me know if I missed anything please)
"Tried the Android SDK only it is missing too many things we need as a developer"
===>] Setup Android Studio SDK & NDK [<===​Installation Paths:
*** I install to /home/username/Android and /home/username/Android/Sdk and /home/username/Android/Sdk/ndk-bundle ***
NOTE: from here forward username will == droidvoider
Note: Android Studio IDE isn't necessary only the SDK & NDK are needed to compile AOSP.
Install Android Studio Proper: (don't worry about setting up paths we will cover that, just install it)
https://developer.android.com/studio/install.html
or
SDK Only:
Typically we install these things manually by creating the directory then just unzipping the files there.
https://developer.android.com/studio/index.html#linux-bundle (scroll down for sdk only)
Code:
mkdir /home/droidvoider/Android
mkdir /home/droidvoider/Android/Sdk
(then unzip the sdk zip to that directory. I recommend the file explorer copy/paste right click uncompress and done.)
https://dl.google.com/android/repository/tools_r25.2.3-linux.zip
Install NDK through the SDK Manger:
(from terminal 'studio.sh' and then configure, and then sdk manger --- if this is hard to figure out tell me I will elaborate)
or
Manually Install Native Development Kit -- 'c programming support'
Download the Native Development Kit from Google: https://developer.android.com/ndk/downloads/index.html
Code:
mkdir /home/droidvoider/Android/Sdk/ndk-bundle
Then just unzip the ndk files into the directory we created above.
===>] Setup your toolchain [<===​** This example is arm64-v8a aarch64 **
1. Navigate to /home/droidvoider/Android/Sdk/ndk-bundle/build/tools and then open a terminal "right click open area"
2. mkdir /home/droidvoider/toolchains
3. ./make_standalone_toolchain.py --arch arm64 --api 23 --stl=libc++ --install-dir /home/mm/toolchains/aarch64-linux-android-4.9
4. cd /home/droidvoider
5. gedit .bashrc and morph this in at the bottom.. (AND edit or replace the existing PATH variable)
DON'T just PASTE IN *my* $PATH export!! I included my entire path statements to show you.
Code:
export PATH=$PATH:/usr/local/android-studio/bin:/home/droidvoider/Android/Sdk/platform-tools:/home/droidvoider/Android/Sdk/ndk-bundle:/home/droidvoider/Android/Sdk/tools
I feel this is human readable, for example change Android_Build_Out to be on your desktop instead if you want.
Code:
export PATH=$PATH:/home/droidvoider/toolchains/aarch64-linux-android-4.9
export NDK=/home/droidvoider/Android/Sdk/ndk-bundle
export SYSROOT=$NDK/platforms/android-23/arch-arm64
export TARGET=aarch64-linux-android
export HOST=$TARGET
export BUILD=x86_64-linux
export ANDROID_NDK_BIN=/home/droidvoider/toolchains/aarch64-linux-android-4.9/bin
export CC=$ANDROID_NDK_BIN/aarch64-linux-android-gcc-4.9
export CPP=$ANDROID_NDK_BIN/aarch64-linux-android-g++
export AR=$ANDROID_NDK_BIN/aarch64-linux-android-ar
export OUT_DIR_COMMON_BASE=/home/droidvoider/Android_Build_Out
Note: You might want to setup an alternate toolchain also but this is all of the puzzle pieces.
===>] Google's version of this How To -- Just for reference [<===​https://source.android.com/source/requirements.html
https://source.android.com/source/initializing.html
===>] Install the repo tool [<===​https://source.android.com/source/downloading.html
(don't type repo init or repo sync --- I will be taking back over from there on the next page)
Added Repair Notes -- Not part of the install!
Have you accidentally installed or removed something you shouldn't have? (welcome to development, here try this before reinstall)
sudo apt-get clean
sudo apt-get update
sudo apt-get install -f
sudo dpkg -a --configure
sudo apt-get dist-upgrade
sudo apt-get install -f
sudo dpkg -a --configure
Selecting the correct AOSP branch and downloading it.
Tested up to downloading AOSP and make toolbox -- you should be all set
===>] Match your build number to it's AOSP sources [<===​preface: You can get this from your device if you're on the same build id as your the available source code from your vendor for your device. Otherwise you need to open the AP file from the firmware that matches those available sources to extract the system.img, to extract build.prop. I explain how to open a system.img file below under retrieving your hardware drivers. build.prop is in the main directory of system.img
(Many times the build number is the same. For me I believe all of MM builds are using this number.)
Assumes sources match current device, worked out true in my case
1. Plug in your device and get it connected. (DEVELOPER OPTIONS|USB DEBUGGING) and select allow on device
2. Retrieve the build number that matches the available sources for your device.
From your ubuntu terminal retrieve the build id using this command:
Code:
adb shell getprop | grep 'ro.build.id'
Yields something similar to this: [ro.build.id]: [MMB29K]
3. Match it up to the Nexus build numbers (This info is for AT&T Note 5 Marshmallow MMB29K, get your specific build number!)
https://source.android.com/source/build-numbers.html#source-code-tags-and-builds
MMB29K android-6.0.1_r1 Marshmallow Nexus 5, Nexus 5X, Nexus 6, Nexus 7 (flo/deb), Nexus 9 (volantis/volantisg)
===>] Bring down a specific AOSP source branch [<===​
4. Make a directory for the source code.
Code:
mkdir /home/droidvoider/Desktop/AOSP_Android_6.01_r1
5.
Code:
cd /home/droidvoider/Desktop/AOSP_Android_6.01_r1
6. Bring down the sources, this one is approximately 13 gigabytes
Code:
repo init --depth=1 -u https://android.googlesource.com/platform/manifest -b android-6.0.1_r1
repo sync
===>] I'm not sure the rest of this is needed [<===​For compiling toolbox the remainder wasn't needed.. But I have a large list of things to do so I can't test each item. If you can't compile a specific module continue reading.
===>] Merge Vendor sources & AOSP sources [<===​
7. Download the available sources for your device. In this example I downloaded PE6 Marshmallow sources for AT&T Note 5:
http://opensource.samsung.com/reception/receptionSub.do?method=sub&sub=F&searchValue=SM-N920A
8. Read the readme file from the sources platform zip to understand how to merge them into the AOSP sources. For the 2 Samsungs I've worked with the idea is to replace any directory that already exists. But if there is just one file such as core.mk only replace the one file. Then edit the .mk files as described in your readme. (and/or take info from cyanogen/lineagos) -- <I can help more, ask>
note: you probably don't need to take the configs from LineageOS and put them into your .mk files. However, if you do need to get more configs then you should get a big fat message when you type make 'modulename'. At first only edit .mk files as described by vendor device source readme file.
===>] Merge in Hardware drivers and etc [<===​possibly unnecessary depends what you're doing
9. Obtain a copy of the firmware for your device that matches the version of the source code you are able to download from your vendor.
for me that was Build Number: MMB29K.N920AUCU2BPE6 but your mileage will almost certainly vary!
10. Download https://github.com/anestisb/android-simg2img
11. Unzip it right in your download folder, open the folder and then 'open in terminal'
12. Make it and then move it a directory in your path. Warning: My command puts in in the Ubuntu default /bin folder.
Code:
make
sudo mv append2simg img2simg simg2img simg2simg simg_dump.py /bin
13. Uncompress the AP file from the matching firmware and extract the system.img into it's own directory
then select that folder, right click, open in terminal
Code:
simg2img system.img sys.raw
mkdir sys
sudo mount -t ext4 -o loop sys.raw sys/
14. A drive mounted, look on your task bar it should've wiggled too. Copy the etc and vendor folders into the main folder of the sources we are merging
===>] Listing and building modules [<===​Navigate to the folder where you download the sources "/home/droidvoider/Desktop/AOSP_Android_6.01_r1" and open in terminal.
Code:
make modules -- list the available modules
make <module name> -- builds a specific module
example: make dumpstate
description: Will build everything needed for dumpstate and place it in the folder we specified in our export (above step). The final build line will read install and detail the final output folder
Example successful output:
[CODE]
Install: /home/droidvoider/Android_Build_Out/Android_6.01_r1/target/product/generic/system/bin/dumpstate
===>] Android Build System, basic intro [<===​Notice: I built this how to to answer the same question from 3 people regarding working with toolbox and the dirtycow exploit. So I decided to give a direct example of using toolbox.c from farm-root
#ifdef
Our makefile is Android.mk and that's where we link things together. If you look at the Android.mk file for farm-root you will notice bridge.c is used 3 different times called different 'module' names. bridge_pull, bridge_push, bridge_pull_boot. Each of these will be binaries of those names.
Inside bridge.c you will see #ifdef FARM_PULL and then you will see #else and further you will see #endif which you may have noticed matches inside the Android.mk file for the bridge modules -DFARM_PULL -DFARM_BOOT <== Notice the double define on bridge_pull_boot
toolbox.c
toolbox.c is going to be the same way. You will need to copy shared.h and shared.c into the directory where toolbox.c resides. Then edit the Android.mk, in our example:
1. Navigate to this directory and open: system/core/toolbox/Android.mk
2. CTRL + F and search for "LOCAL_MODULE := toolbox"
3. Add: LOCAL_CFLAGS += -DFARM_PULL -DFARM_BOOT (in this example add one, both or even new ones you created)
4. Navigate to the main directory of the sources, you should see a Makefile and a build_64bit.sh
5. from terminal: make toolbox
Note: I think from here you can Google it out in a few minutes if that is not the case please let me know.
Working with C cross platform​Ubuntu is Linux based just like Android and this makes testing blocks of code extremely easy. You of course can't use Android headers and in some rare cases you can't test the code on Ubuntu at all but in most cases you can. When I want to design something for Android I open gedit and save it as a .c file. Then I compile it using gcc -o mycode mycode.c There's plenty of examples on using gcc with linux but just understand you can do it all. Then before too much work test it on Android. (helpful commands at end of post)
My advice really is to build out your small blocks of code on your linux box but then paste them into your Android program folder, edit your Android.mk, add it to your Makefile including your 'push' section so that you can simply type make push to test it.
I am in fact trying to encourage you to learn C and not so much trying to encourage you to hack things. But I know that interest/passion is what teaches, not my words and not someone else's curriculum. So in that spirit I will do my best to give examples to help you with 'whatever' it is you are passionate about. Let me know what's missing.
Don't forget to compile for Android first
Before you can test your code you will have compiled it using the cross compiler for Android. ndk-build, or the correct gcc cross compiler. (Personally I put the .c file into a directory with Android.mk and a Makefile then just type make to build it to Android)
see examples section I will add a couple examples.
Android Developer Bridge -- a developers tool
adb is included with the Android SDK along with some other tools. Some of those tools are fastboot for unlocking bootloaders and another way of flashing. There is monitor which is a cool tool for remotely viewing processes, logcat, memory dumps and etc.
But pointedly what we will use the most is simply adb.
Using adb to test your code on locked down Android systems
Shell has fairly high privileges, you may not be aware but you can execute binaries and bash scripts. We use /data/local/tmp/ for these things. You can create a directory, add or remove files, execute your binaries and even execute shell scripts using sh script.sh
ndk-build places the binary in libs/(arch type) .. For a quick test you can just open a terminal in that directory then:
Code:
adb push mybinary /data/local/tmp/
adb shell
cd data/local/tmp
chmod 777 mybinary
./mybinary
Added:
Examples of basic make files for Android.
happy coding
If you get an error​Please reissue the command but pipe the output to a file.
make toolbox > /home/droidvoider/Desktop/build_toolbox-output.txt
zip that up with your source code, including your customized header files and attach it to this thread.
puzzles are fun but I like all the pieces
droidvoider said:
Tested up to downloading AOSP and make toolbox -- you should be all set
===>] Match your build number to it's AOSP sources [<===​preface: You can get this from your device if you're on the same build id as your the available source code from your vendor for your device. Otherwise you need to open the AP file from the firmware that matches those available sources to extract the system.img, to extract build.prop. I explain how to open a system.img file below under retrieving your hardware drivers. build.prop is in the main directory of system.img
(Many times the build number is the same. For me I believe all of MM builds are using this number.)
Assumes sources match current device, worked out true in my case
1. Plug in your device and get it connected. (DEVELOPER OPTIONS|USB DEBUGGING) and select allow on device
2. Retrieve the build number that matches the available sources for your device.
From your ubuntu terminal retrieve the build id using this command:
Code:
adb shell getprop | grep 'ro.build.id'
Yields something similar to this: [ro.build.id]: [MMB29K]
3. Match it up to the Nexus build numbers (This info is for AT&T Note 5 Marshmallow MMB29K, get your specific build number!)
https://source.android.com/source/build-numbers.html#source-code-tags-and-builds
MMB29K android-6.0.1_r1 Marshmallow Nexus 5, Nexus 5X, Nexus 6, Nexus 7 (flo/deb), Nexus 9 (volantis/volantisg)
===>] Bring down a specific AOSP source branch [<===​
4. Make a directory for the source code.
Code:
mkdir /home/droidvoider/Desktop/AOSP_Android_6.01_r1
5.
Code:
cd /home/droidvoider/Desktop/AOSP_Android_6.01_r1
6. Bring down the sources, this one is approximately 13 gigabytes
Code:
repo init --depth=1 -u https://android.googlesource.com/platform/manifest -b android-6.0.1_r1
repo sync
===>] I'm not sure the rest of this is needed [<===​For compiling toolbox the remainder wasn't needed.. But I have a large list of things to do so I can't test each item. If you can't compile a specific module continue reading.
===>] Merge Vendor sources & AOSP sources [<===​
7. Download the available sources for your device. In this example I downloaded PE6 Marshmallow sources for AT&T Note 5:
http://opensource.samsung.com/reception/receptionSub.do?method=sub&sub=F&searchValue=SM-N920A
8. Read the readme file from the sources platform zip to understand how to merge them into the AOSP sources. For the 2 Samsungs I've worked with the idea is to replace any directory that already exists. But if there is just one file such as core.mk only replace the one file. Then edit the .mk files as described in your readme. (and/or take info from cyanogen/lineagos) -- <I can help more, ask>
note: you probably don't need to take the configs from LineageOS and put them into your .mk files. However, if you do need to get more configs then you should get a big fat message when you type make 'modulename'. At first only edit .mk files as described by vendor device source readme file.
===>] Merge in Hardware drivers and etc [<===​possibly unnecessary depends what you're doing
9. Obtain a copy of the firmware for your device that matches the version of the source code you are able to download from your vendor.
for me that was Build Number: MMB29K.N920AUCU2BPE6 but your mileage will almost certainly vary!
10. Download https://github.com/anestisb/android-simg2img
11. Unzip it right in your download folder, open the folder and then 'open in terminal'
12. Make it and then move it a directory in your path. Warning: My command puts in in the Ubuntu default /bin folder.
Code:
make
sudo mv append2simg img2simg simg2img simg2simg simg_dump.py /bin
13. Uncompress the AP file from the matching firmware and extract the system.img into it's own directory
then select that folder, right click, open in terminal
Code:
simg2img system.img sys.raw
mkdir sys
sudo mount -t ext4 -o loop sys.raw sys/
14. A drive mounted, look on your task bar it should've wiggled too. Copy the etc and vendor folders into the main folder of the sources we are merging
Click to expand...
Click to collapse
And where is exactly the main folder? Sorry, Im just confused
DigitalDoraemon said:
And where is exactly the main folder? Sorry, Im just confused
Click to expand...
Click to collapse
it's no problem this stuff isn't easy to just figure out on your own. remember to substitute droidvoider for your ubuntu user name
In this example my sources are on my desktop in a folder named Android_6.01_r1
Sources for toolbox for example:
/home/droidvoider/Desktop/Android_6.01_r1/system/core/toolbox/<sources will be here including Android.mk>
Script for modules, including toolbox
/home/droidvoider/Desktop/Android_6.01_r1/Makefile <--- this is our modules script, if you will
<open a terminal in the above folder then use that Makefile like so>
make toolbox <---- this will compile only what is needed to compile the module 'toolbox' (this takes a minute)
Out export folder we decided in ./home/droidvoider/bashrc
/home/droidvoider/Android_Build_Out/Android_6.01_r1/target/product/generic/system/bin
Anybody, please compile grep utility for arm and x86... Minimum Platform Version Android 4.0.3, API Level - 15
Thanks
Great & useful .

Build specific parts of system

Hey guys!
I tried building LineageOS 15.1 from source. No issues here, works great!
But here is the problem: I now want to include an app I developed into the system build.
Earlier, I did this by adding the files to the BUILD_DIR/packages/apps/<App name> directory. But to include it in the system, I had to rebuild the ROM from scratch.
The app uses an Android.mk file and not Gradle.
Also, if I modify an existing source file (say Activity.java), then how would I proceed, short of rebuilding the ROM?
I wanted to know if there is a more efficient way to build a part of the system. Any help would be appreciated.
Thank You
EDIT:
Found the answer. For anyone who needs it:
Note: This applies only if you have already built the system once and haven't deleted the output files.
Open the Android.mk file in the source tree that you want to build.
Find a line LOCAL_MODULE := <module_name>
Go to the project root folder.
Enter the following lines in the terminal:
Code:
$ source build/envsetup.sh
$ breakfast <device_name>
$ mka <module_name>
Then, to build the flashable zip:
Code:
$ brunch <device_name>

A noob tries his hand at ROM development for the MiA1 tissot

The problem with reading the articles on XDA is that eventually you want to try out some of the things described in these pages. I am no exception. After reading the superb article https://forum.xda-developers.com/mi-a1/how-to/guide-build-rom-source-tissot-t3754723 I decided to try my hand at developing the ROM for the Xiaomi MiA1. This is my daily driver and that in itself was a strong reason for motivation.
I tried replicating the steps given in the article on my Dell Inspiron and immediately ran into issues. The basic requirements mentioned are a linux or Mac computer. So to start with I formatted my laptop. Removed windows 10 and installed Ubuntu's latest 18.04 LTS.
I believed I had the next set of requirements for custom ROM development which were
• A basic understanding of Linux based OS specifically Ubuntu.
• Ability to type and use the command line. Write or copy paste commands in the Ubuntu konsole.
• A basic understanding of programming concepts, read and update XML files, understand error messages.
• Fast internet access.
• Steady electricity supply.
• Lots and lots of patience.
This still did not solve my problem. The build would crash repeatedly. Then I read the https://source.android.com/setup/. A whole new set of requirement came up. The article said that ideally the build system should exceed these conditions
• 64 bit environment
• 100 GB of free hard disk space
• 16 GB RAM / swap space.
At home my son was about to buy a new PC and this made his old PC available. I upgrade the RAM and added a new hard disk. Now my hardware configuration was as under :
• Intel i5 processor
• 16 GB RAM
• 1 TB hard disk.
With a new hard disk I had to create a boot-able USB drive. This I achieved with Rufus. My old laptop which again had its Windows 10 installation was used to install Rufus based on the instructions i got from https://rufus.akeo.ie/
The latest version of Ubuntu I downloaded from https://www.ubuntu.com/download/desktop
Setting Ubuntu from scratch for someone used to the world of windows is not exactly easy. I found the tutorial at https://tutorials.ubuntu.com/tutorial/tutorial-install-ubuntu-desktop#0 extremely helpful.
I wrote down the steps I took in reaching the final goal of a Lineage ROM for my Mi A1.
Step 1: Setup Ubuntu - already completed
I chose the default settings as recommended in the setup.
Step 2: Installation of Java Development Kit
Once Ubuntu was up and running I opened a konsole window and typed the following commands
PHP:
sudo apt-get update
sudo apt-get install openjdk-8-jdk
Ubuntu did all the installing as I looked on.
Step 3: Installing required packages
PHP:
sudo apt-get install git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev libgl1-mesa-dev libxml2-utils xsltproc unzip
Step 4: Configure your smart phone access
I next installed the tools required to access the phone. This access would be required to later flash the phone or access its internals with ADB.
PHP:
sudo apt-get install android-tools-adb
sudo apt-get install fastboot
Step 5: To creating the work directories i typed the following commands. I was following the instructions from the above two articles alternately.
PHP:
mkdir -p ~/bin
mkdir -p ~//lineage
Step 6: Installing repo
I was informed that the Android source tree is located in a Git repository which is hosted by Google. The Git repository includes metadata for the Android source. Repo is a tool that makes it easier to work with Git .
PHP:
curl [url]https://storage.googleapis.com/git-repo-downloads/repo[/url] > ~/bin/repo
Once the repo was downloaded I had to make it executable
PHP:
chmod a+x ~/bin/repo
Step 7: The next step was to initialize the lineage source
This required to change to the directory created in step number 5 above. As someone coming from the world of windows I preferred to use the GUI and open the lineage folder created by using the mouse click. There right click brings up the option of ' Open in Terminal' . Those who prefer to type commands can use the below command
PHP:
cd ~//lineage
Once I had made sure that I was in the newly created lineage folder I moved ahead.
Step 8: Now i was required to configure my Git user
PHP:
git config --global user.name "Your Name"
git config --global user.email "[email protected]"
I replaced the text within the " " with my creadentials, which I had created on https://github.com/
Step 9: Now we moved on to initializing the repo
PHP:
repo init -u [url]https://github.com/LineageOS/android.git[/url] -b lineage-15.1
A successful initialization showed a message stating that Repo is initialized in your working directory. The lineage directory now contain a .repo folder. To check hidden folder I used control + H key.
Step 8: Download the source code
This is the step where you will realize the importance of having a fast internet connection. The source code size is about 30 – 35 GB. I have a 10 mbps line and at times the download speed shoots up to 20- 25 mbps!!!* Yet it takes me anything from 4 – 10 hours to download the entire code!
My suggestion is check the time when your local ISP download speeds are at its best. For me it is during the night. I set the download up at night and go to sleep!
The command to get the source code is :
PHP:
repo sync -c -f --force-sync --no-clone-bundle --no-tags --optimized-fetch –prune
Optional Step 8 a: Making a backup of your source code
I copied the entire folder and kept a copy on an external device. This is just in case there is any need for me to format my hard disk. While copying I do not copy the .Repo and .Cache folders on to my backup. To check hidden folders press control + H keys.
Step 9: Get the source code for your specific device.
This is where I was stuck numerous times. As I understood the code which was downloaded is generic in nature. For your particular device you would need to get files which are specific to your device.
The source for all these files is github. You are targeting three sets of folders – Device, Kernel and vendor.
The git clone commands given below are for the tissot or Xiaomi MiA1 phone. Similarly other phones have their locations on GitHub from where you can get files specific to them.
PHP:
git clone [url]https://github.com/TheScarastic/andr...msm8953-common[/url] -b lineage-15.1 device/xiaomi/msm8953
git clone [url]https://github.com/TheScarastic/andr..._xiaomi_tissot[/url] -b lineage-15.1 device/xiaomi/tissot
git clone [url]https://github.com/TheScarastic/prop..._vendor_xiaomi[/url] -b lineage-15.1 vendor/xiaomi
git clone [url]https://github.com/Tissot-Developmen..._xiaomi_tissot[/url] -b 8.1 kernel/xiaomi/msm8953
Step 10: Modify Caching to speed up the build process.
PHP:
export CCACHE_DIR=./.ccache
ccache -C
export USE_CCACHE=1
export CCACHE_COMPRESS=1
prebuilts/misc/linux-x86/ccache/ccache -M 50G
Step 11: Configure JACK
JACK is the java compiler and can cause crashes – believe me! A simple fix is this command which you type next
PHP:
export _JACK_VM_ARGS="-Dfile.encoding=UTF-8 -XX:+TieredCompilation -Xmx4G"
Step 12: Cleaning up the build folder
PHP:
make clean
Step 13: Initializing the build
PHP:
source build/envsetup.sh
Step 14: Starting the build
PHP:
croot
brunch tissot
Watching the build on the screen is an interesting phase. It can get a bit tedious. In my case it took about 3 hours to complete the build. There were numerous errors that were thrown up. Files were declared as missing. After a couple of failures I realized I could search for them and then paste the file in the appropriate folders.
The one good thing about the build errors is googling for them usually gets you the right help. There would definitely be someone in some part of the world who would have run into a similar issue.
Finally after a 3 hour wait I got my zip file which when i flashed through TWRP on my MiA1. The sheer joy of having a ROM you created on your phone is indeed beyond words. It took me about a week of sitting for about ten to fifteen hours to get the various combination right, but believe me when I tell you the results are worth it.
My thanks to arunassain https://forum.xda-developers.com/member.php?u=6768248 from whose article I was able to understand the basics. He helped me with responses to my queries when I was stuck.
I further picked up from the excellent articles under the title Establishing a build Environment at https://source.android.com/setup/build/initializing
A great video that helped me was available on the topic How to build your own custom Android ROM at https://www.androidauthority.com/build-custom-android-rom-720453/
To check out the ROM's built for the MiA1 please click here
Now coming to the question if I have understood everything that is to learn about ROM development, well the answer is no. This was just the first build by a noob of a ROM which somehow worked. I have been using the ROM on my daily driver for the past one day and it is working perfectly. Having said that I still need to understand more of what happens behind the scenes in this wonder world of Custom ROM development.
Don't tell me you thought that you could build a rom on x86 processor/OS
Give him a break, he admitted that he is new to this building ROM world, I can understand that some people can think that if a proc is powerful enough they can use it to build no matter if it's a x64 or x86.
Thanks for you story mate! And don't hesitate to ask dev on telegrams or G+ to explain you some what you don't understand :good: I'm pretty sure that would love to finally have someone like you, who is truly interested in this whole world

Categories

Resources