Database Info

[HOWTO] Running debian via chroot on OO

This tutorial is based on this one for G1: http://www.saurik.com/id/10
Hi,
I'll show you how to run debian in chroot on our optimus one. It wont affect android in any way, and they can run simultaneously, side by side. You are only running bash from debian, not its kernel or any other low-level part. But you can use debian's programs.
Requirements
* Linux on your pc or in virtual machine
You need debootstrap to make debian image, so you need debian distro (maybe in *buntu it is too, but I am not sure). You dont need GUI, bash is all you need.
* Busybox(-> rooted phone)
Most custom roms comes with busybox, I am not sure how about store ROM.
* Terminal emulator on phone and/or ADB
I suggest you to make sure you have adb, because you dont wanna write it all on small touch keyboard.
Some knowledge about linux systems
If you are following this tutorial only because you want to /flex on your friends and you dont know anything about linux, then you really _should_ stop reading this tutorial.
I also recommend you to have a2sd, so we can put debian image on its partition, when it is on sdcard main partition, it _may_ cause some problems with usb mass storage.
So, lets start.
1. Getting debian image
Boot into debian and get bootstrap tool:
Code:
apt-get install debootstrap
Now, create an empty disk image, I'd say minimum size is about 150mb, but I recommend you use more:
Code:
dd if=/dev/zero of=debian.img seek=*size_in_bytes* bs=1 count=1
You need to create ext filesystem in image with this tool(you can try it with "sudo" if you have problems):
Code:
mke2fs -F debian.img
Image is ready now, now lets mount it so we can install debian into it:
Code:
mkdir debian
mount -o loop debian.img debian
Debootstrap tool will do all the work for us. You can try to use squeeze instead of lenny or download it from your country's mirror. "--variant=minbase" argument will install just base system with apt-get and only few other packages. (This also may need sudo)
Code:
debootstrap --verbose --arch armel --foreign lenny debian http://ftp.cz.debian.org/debian
umount debian
2. Mount the image
Now, we will use ADB shell to mount the image and finish debian installation. Do not forget to write "su" before you start, we need root for this.
You need to decide where to put debian.img. I suggest you to put it on a2sd partition, which is system/sd, so :
Code:
cp /sdcard/debian.img /system/sd
export kit=/system/sd
Our mount point will be /data/local/mnt, so lets make some variable to make it easier:
Code:
export mnt=/data/local/mnt
busybox mkdir -p $mnt
We also need to set these enviroment variables, and we can also make alias for busybox so we dont have to write busybox all the time.
Code:
export PATH=/usr/bin:/usr/sbin:/bin:$PATH
export TERM=linux
export HOME=/root
alias _=busybox
Now we need to create loop device for image:
Code:
_ mknod /dev/loop0 b 7 0
And if nothing went wrong, you should be able to mount image now ("mount: no /etc/mtab" is normal) :
Code:
_ mount -o loop,noatime $kit/debian.img $mnt
3. Finalizing installation of debian
With mounted image, we can proceed to second stage of debian bootstrap:
Code:
_ chroot $mnt /debootstrap/debootstrap --second-stage
If you want install some packages with apt-get, you also need to add repo to sources.list. Of course, if you downloaded squeeze instead of lenny, you need to change "lenny" to "squeeze"
Code:
echo 'deb http://ftp.cz.debian.org/debian lenny main' >$mnt/etc/apt/sources.list
Set correct DNS servers - debian can't take them from Android.
Code:
echo 'nameserver 4.2.2.2' >$mnt/etc/resolv.conf
Done, installation over.
4. Running debian bash
To get to debian bash, use this command. You can try "cat /etc/issue.net" or "uname -a" so you can see its really debian.
Code:
_ chroot $mnt /bin/bash
But its not all yet, you need to mount some things for debian:
Code:
mount -t devpts devpts /dev/pts
mount -t proc proc /proc
mount -t sysfs sysfs /sys
Now link mtab to /proc/mounts. You need to do this only once.
Code:
rm -f /etc/mtab
ln -s /proc/mounts /etc/mtab
And if you want to use ssh server to log in into debian, you also should set root pass:
Code:
passwd root
5. Installing packages
First, you need to update packages list:
Code:
apt-get update
and now you can install whatever you want with "apt-get install"!
6. Finishing
I suggest you to write some .sh script so you can mount it all with one command, for example something like this: http://pastebin.com/VT6272d5
What packages have you tested?
sshd - works great, you can connect to phone via putty or ssh
mc - works, but phone screen is to small
gcc-avr - if you are programming for avr microcontrollers, you can compile it on your phone
netcat - telnet connection
git-core - works great
htop - you can see this on screnshot below
apache2 - works like a charm
xfce4, Xorg server and tightvncserver - working but slow, see screenshots
gnash - working but slow, see screenshots. This is the only option how to run flash content on optimus one xD
Iceweasel - if you are using vnc server, you can browse web pages
gcc, g++, autoreconf, cmake and whatever else is needed to build mangos - working
mysql-server - you have to set "user = root" in /etc/mysql/my.conf, but works
MaNGOS - http://getmangos.com/ takes long to build, but works! See this post and also this one.
..and maybe some more which I dont remember.
Hope this tutorial helped you

I'll try bootstrapping debian when I figure out what the hell is wrong with my system. I can't download files from ftp (but I can with wget and other progs. just not with pacman, debootstrap, and the like).

debian on optimus v
searched around to avoid multiple posts, and found this.
I've been running chroot debian on my V as well, squeeze then sid now going back to squeeze.
the main problem I've encountered was in the "debootstrap --second-stage", which generally froze at "I: configuring sysvinit". I thought that was from overloading the CPU, but turns out android init doesn't create /dev/initctl like debian does, so debootstrap hangs waiting for a response reading/writing /dev/initctl
I didn't have that issue using SL4A's shell instead of android terminal.
being a bit of a 'noid, I have used "mknod /dev/initctl p" in the chroot before running "debootstrap --second-stage" since reading of that solution. not sure if it's making much difference, because that didn't fix the issue before I found SL4A and was still using android terminal for shell functions.
I like xfce4 as a window manager in the VNC, it's small but still functional.
I did a bit of things different than saurik, but used his guide extensively, along with much googling for other deb->droid ports.
my chroot runs in the android root directory with bind-mounts since the custom kernel I like doesn't include unionfs. I also directly use an ext4 2nd partition on the SD rather than a loop-mounted filesystem image. (yes, ext4 journals will eventually frag my SD but the cost of another is worth the crash-resistance to me.)
my short-term goal is to mix inits between the two os's and run X through an android app frontend to avoid the framebuffer copy lag from the VNC. my 3 main reasons for debian on the phone are program development (works currently,) flash (actually gnash, and needs a working full-speed access to framebuffer, not working yet,) and emulators for J2ME and MAME (same boat as flash.)
so far trying to run X directly blanks my screen and forces a battery pull to get it back, so kinda giving up that route for the moment.

so installing Archlinux on O1 must be possible? how?

pymebrahimi said:
so installing Archlinux on O1 must be possible? how?
Click to expand...
Click to collapse
All must work, as long as they support running on ARM. AFAIK Arch doesn't have an (official) ARM port. You can try Arch Mobile though

This command fails me
Code:
_ mount -o loop,noatime $kit/debian.img $mnt
The error I get is no device or file exists.
Tried creating more number of loop devises too. Still it didn't work. I am using cyanogemmod 2.3.3 ROM.

onehomelist said:
This command fails me
Code:
_ mount -o loop,noatime $kit/debian.img $mnt
The error I get is no device or file exists.
Tried creating more number of loop devises too. Still it didn't work. I am using cyanogemmod 2.3.3 ROM.
Click to expand...
Click to collapse
What is the exact wording of the error?

The exact error that I get:
Code:
# busybox mknod /dev/loop0 b 7 0
# busybox mount -o loop,noatime $kit/debian.img $mnt
mount: can't setup loop device: No such file or directory

Instead of using $kit/debian.img, try manually pointing to the debian image.
For example, if you have it on the sdcard's debian folder:
Code:
/sdcard/debian/debian.img

That too didn't work.

make sure that debian image is where it should be. Also try to "ls /dev/" and look for loop0 - i had some problems with loop devices on mik's 2.3, but I did not really solved it, because I went back to froyo after while.

The loop0 device exists is /dev. As you have experienced same issue on the same ROM, I am convinced that its a ROM issue.

I just tried to install xorg, xfce4 & tightvncserver - everything work just fine, although its kinda slow

ext partition instead of loop filesystem
Tasssadar said:
I just tried to install xorg, xfce4 & tightvncserver - everything work just fine, although its kinda slow
Click to expand...
Click to collapse
I think it runs faster from a sdcard partition than a loop filesystem, if I remember right. I tested a loop-mounted preconstructed debian img from some tutorial for another device before debootstrapping my own.
partition your card with a second partition, ext2 if you're worried about journalling wearing out your sd, or ext4 if you're more worried about corrupting your precious debian filesystem in a crash or unclean umount. gparted in ubuntu seemed to work well for me.
resize your ntfs partition instead of deleting it or android may complain.
with your debian img mounted as a loop filesystem on a linux box at the same time as your new partition, you can copy everything from the img to the ext partition by:
Code:
cp -av /path/to/loopmountedimg/* /path/to/extpartition
if you replace the loop-mounting code in your boot script with a vold mount of your ext partition, android is supposed to umount the partition cleanly on powerdown or reboot.
on my optimus v, the mount looks like the following in my boot script (I'm posting the comments too for info)
# mount 2nd sdcard partition from vold, should prevent dirty unmount at reboot
# if your linux is on a different partition than 2, substitute that number for the 2 in .../vold/179:2 below
# first unmount it; if not already mounted, system will echo 'umount: can't forcibly umount /dev/block/vold/179:2: Invalid argument' but this isn't an error to worry about
Code:
umount -l /dev/block/vold/179:2
mount -o noatime,exec,suid /dev/block/vold/179:2 /$mnt
maybe someone here can use this info.
also, for anyone who wants to save some hours and bandwidth, I have a basic squeeze debootstrap uploaded to 4shared.
it shouldn't be missing any of the readme's or licensing docs, so it should be GPL ok AFAIK.
resolv.conf and sources.list are already configured for 3g access and the main Debian repository, so it's pretty much plug-n-play.
no extra packages. you'll need to apt-get tightvncserver and a window manager unless you just want the bash shell.
the first time you chroot in, you should use passwd to set a root password, and adduser to get a non-root account on there.
the only glitch is that I accidentally tar'd it into a compressed folder; so you can't just untar it into the ext partition, it needs unpacked with an archive manager. sorry!
here is the filesystem image
uncompressed, 9645 items, totalling 264.3 MB
compressed, 116.3 MB (121929580 Bytes)

Tried to put it on ext2 partition without image, its a bit faster but still very slow.
But i took some screenshots this time. You can see flash (well, gnash) player on optimus one working on last screenshot
And I have one question: is debian using its own libc or android's bionic?

Tasssadar said:
Tried to put it on ext2 partition without image, its a bit faster but still very slow.
Click to expand...
Click to collapse
guess I'm biased, first linux I used was on a 100MHz pentium 1, so this handheld flies by comparison.
But i took some screenshots this time. You can see flash (well, gnash) player on optimus one working on last screenshot
Click to expand...
Click to collapse
I could only get gnash to play ads. boo. froze up iceweasel every time I tried to open a flash video. what browser are you using? oh, man, it's iceweasel, isn't it?? also, chromium just refused to start on my optimus.
And I have one question: is debian using its own libc or android's bionic?
Click to expand...
Click to collapse
pretty sure it's standard debian libc. you could integrate the android toolchain if you were so motivated, probably.

bigsupersquid said:
I could only get gnash to play ads. boo. froze up iceweasel every time I tried to open a flash video. what browser are you using? oh, man, it's iceweasel, isn't it?? also, chromium just refused to start on my optimus.
Click to expand...
Click to collapse
Chromium does not work for me, too.
Well, it froze for me too, but after while it at least got to show what you see on the screenshot
bigsupersquid said:
pretty sure it's standard debian libc. you could integrate the android toolchain if you were so motivated, probably.
Click to expand...
Click to collapse
I was just thinking if this is possible, then normal android aplication could have libc included, if its needed.

Do you know MaNGOS? It is open-source server for World of Warcraft (for more info go to http://getmangos.com/). I just compiled it and I run it on my optimus one. And guess what - without world db, its really fast! I am extracting UDB to database right now, I will post some screenshots and results with full-featured db soon.
And thats why I love linux and android

I got no words to describe my eyes when reading that at the moment.

So, what you might wanna know if you wanna try it too:
All packages needed for mangos to compile are working good, except for mysql server: you need to set "user = root" in my.cnf - it is located in /etc/mysql/ - because you cant put any root rights to any other user than root, so mysql user cant create tcp socket.
You will most likely need swap file - optimus has plenty of RAM, but yet not enough. All you need is to compile kernel with CONFIG_SWAP=y and follow this tutorial: http://www.cyberciti.biz/faq/linux-add-a-swap-file-howto/. 512mb swap is enough. Just for the record, swap is working on optimus one, I think nobody tried it here yet.
Now as for the compilation of mangos, it is mostly working as it is, only problems are intel tbb library and g3dlite library.
TBB is not available for arm yet, so you need to compile without it. Use -DUSE_STD_MALLOC=0 in cmake line.
As for the g3dlite, things are little bit more tricky - you need it, and there are some asm() instructions in it, which are different for arm. I only commented them, because I dont know assembler: https://github.com/Tasssadar/mangos-arm/commit/67c442fa0c10e82b282b83a8fe485588f8786e12
I have no idea how it can even work without all that code in g3dlite, but it just does.
You can use scriptdev2 from its git, theres no problem in it.
Thats all, after that just "make" and get something good to read, because it will take really long. At least 5 hours for compilation and nearly hour and a half for linking of scriptdev2 and mangosd binary.
Creating of databases is just like on normal pc, so I just skip it...same goes for dbc files, maps and vmaps.
Loading of mangos takes around 4 minutes with full-featured db. mangos then takes around 180mb of ram and cpu usage is...surprisingly low, considering the fact that our cpu does not have hw float operations. I tried to clear Azjol Nerub instance (well, solo with gm character) and maximal cpu usage was 60%, when fighting a boss with lots of spawns. I could imagine some 5-man dungeon run would be just fine
Also, you will get slight lag when moving into new location, when phone loads map files from sdcard.
Thats all, see screenshots

[DEV] Kernel development HOWTO and Interactive menu

I havent yet found a simple guide for compiling kernels. Some of them assume too much, and some are just outdated. So I thought I'd write my own for devs/budding devs. Here you go!
Note:
This is not a guide for newbies. It's a dev guide for devs.
Research before asking questions, please
For The Menu driven interactive kernel build script, see Post #31
I will be developing this guide as I go, so it will be incomplete initially, or lacking in detailed explanations.
Essentials:
Ubuntu Box (By this I mean a PC with a Ubuntu installation, not a live CD)
A toolchain-Either the Android NDK, or your own toolchain
HTC Desire GB/Froyo source from htcdevs.com, or sources from github
Familiarity with the linux shell and basic linux commands.
The will to learn
First things first,
1. Getting the sources
The HTC Desire source is available from two kinds of resources-you can either get it from htcdevs.com (official HTC Dev site), or from source code uploaded from someone else. For the purpose of this tutorial, I'll assume we're working on the official HTC GB source code. So download bravo_2.6.35_gb-mr.tar.gz from htcdevs.com.
2. Setting up the compilation box and preparing source code
2.1 Install some essential linux packages from the Linux terminal:
Code:
sudo apt-get install libncurses5-dev
2.2 Extract the source code
The file you downloaded is a tar archive (like a zip file), so you need to extract it to a convenient location. Let's hit the linux shell-open a terminal window in linux (Accessories->Terminal)
Type:
Let's go to our home directory:
Code:
cd ~/
Now, create the directories for our kernel compilation box.
Code:
mkdir -p ~/android/kernel
Now you need to copy the tar.gz file from wherever you downloaded it to, to this dir.
Extract the archive:
Code:
tar -xvf ~/android/kernel/bravo_2.6.35_gb-mr.tar.gz
cd ~/android/kernel/bravo_2.6.35_gb-mr
Now we can view the extracted files within the directory ~/android/kernel/bravo_2.6.35_gb-mr/
2.3 Set up the toolchain
A toolchain is a set of programs which allow you to compile source code (any source code, not just kernels). The toolchain is specific for the processor and hardware, so we need a toolchain specific for Android and especially the Desire. If you're a semiadvanced-pro user, you may consider compiling your own toolchain (See theGanymedes' guide for doing so). If compilation of kernels is all that you require, fortunately for you, there is an easy way-the Android NDK - v7 (latest as of now) is available here
Get the NDK for Linux - android-ndk-r7-linux-x86.tar.bz2
Code:
mkdir -p ~/android/ndk
Now copy the NDK file to ~/android/ndk
Whenever I say copy, you have to manually copy the file with any file manager. Nautilus comes with Ubuntu, and Dolphin with Kubuntu. You may also use the shell of course with
Code:
cp [sourcefile] [destination]
Extract it:
Code:
tar -jvxf android-ndk-r7-linux-x86.tar.bz2
Now add the path for your toolchain to the env variable:
Code:
gedit ~/.bashrc
At the end of the file, add this line:
Code:
PATH=$PATH:~/android/ndk/android-ndk-r7-linux-x86/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin
3. Setting up kernel parameters
Kernels are compiled with a program called gnu make, and use a set of configuration options specified within a file called Makefile.
A vital point to note is that kernels are compiled with a program called gcc (basically the gnu C compiler), and our NDK itself has its own optimized version of gcc. While compiling, we're actually cross compiling it (meaning compiling a binary package on a system which is different from the actual system which is meant to run it- you're compiling it on your PC while it's actually meant to run on your Desire)
This means that when you compile it, you have to make sure that you compile it with the NDK's version of gcc instead of the system version. Otherwise you end up with a kernel meant to run on your pc, duh! Specifying which gcc to use is by the CROSS_COMPILE variable. You can set it up with this command:
Code:
CROSS_COMPILE=arm-linux-androideabi-
Note the hyphen (-) at the end, and do not forget to include it! At compilation time, system will actually use this variable to find all the programs it needs. Eg: The path for gcc will become arm-linux-androideabi-gcc
We can compile kernels with many different options, like with ext4 support, or without; ext4 support as part of the kernel zImage (in which case it makes the kernel larger), or as a loadable module (of the form somename.ko, which is loaded at init.d/init.rc with the command insmod modulename.ko)
We specify the exact options we require with the help of a useful configuration program called menuconfig (which as the name suggests, is a menu for configuration of make options).
An important thing to note is that as far as kernel compilation is concerned, there are a vast amount of options to setup, and unless you're thorough with kernel compilation, you wont be able to set up the options correctly and get your kernel to boot. Fortunately for us, the kernel source already comes with a default set of parameters which can be easily set up.
Note that all make commands must be executed within the directory bravo_2.6.35_gb-mr. Let's go there now:
Code:
cd ~/android/kernel/bravo_2.6.35_gb-mr
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- bravo_defconfig
This produces a .config file (used by the menuconfig) containing essential parameters to produce a booting kernel for the Desire.
Note: There is a simpler way to get the basic .config file, and this is to get it from a running kernel built by someone else. You can extract the .config from a running kernel with these commands:
Code:
cd ~/android/kernel/bravo_2.6.35_gb-mr
adb pull /proc/config.gz
zcat config.gz > .config
Now we can open menuconfig and add anything we need in addition.
Code:
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- menuconfig
You can view the huge amount of options available in menuconfig.
You can add ext4 support for example (See image above)
Once you're done choosing options, you can exit menuconfig.
4. Compiling it
This is simple. The basic command is:
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- -j10
The -j10 specifies the number of jobs to execute per operation. I can usually go upto 50 on my Quad core CPU. Beware, this can bring a slow CPU to a crawl and freeze up linux itself.
During compilation, you will see all sorts of messages, which may include warnings too. In most cases, its safe to ignore warnings. If there are errors, the compilation will stop, and you will have to fix the issues.
5. Distributing your kernel to users
At the end of compilation, it generates files named zImage, and various .ko files.
You have to copy them from their default location to a zip file. The best way is to use my variant of koush's Anykernel, and copy the files to it. Then, you can zip the whole folder and lo and behold-you have your flashable kernel zip which you can distribute to others.
You can also remove the zImage and the modules from /system/lib/modules of any kernel zip available with you, and copy over your files to it, at the correct location.
So, let's say that you have extracted an existing kernel zip to the location ~/flashable
The file structure should be like this:
Code:
|-- kernel
| |-- dump_image
| |-- mkbootimg
| |-- mkbootimg.sh
| |-- unpackbootimg
| `-- zImage
|-- META-INF
| |-- CERT.RSA
| |-- CERT.SF
| |-- com
| | `-- google
| | `-- android
| | |-- update-binary
| | `-- updater-script
| `-- MANIFEST.MF
`-- system
`-- lib
`-- modules
`-- bcm4329.ko
8 directories, 11 files
I've included my flashable zip directory along with this post. Download file kernel_flashable.tar.bz2.zip to ~/
Code:
cd ~/
tar -jvxf kernel_flashable.tar.bz2.zip
This will create the directory structure outlined above.
Now after every compilation of the kernel, execute these commands from where you executed make:
Code:
cp arch/arm/boot/zImage ~/kernel_flashable
find . -name '*ko' -exec cp '{}' ~/kernel_flashable/system/lib/modules/ \;
cd ~/kernel_flashable
zip -r mykernel ./
This will create mykernel.zip at ~/kernel_flashable. You can distribute this to your users to flash. Make sure you edit updater-script before though

Common errors and other stuff
Ok, post #1 was simple stuff. Now, supposing you get errors while compiling. Post #2 is about that, and ups the level of knowledge a bit..
Some kernel compilation errors:
Treat warnings as errors-Solved by removing the string "-Werror" from all Makefiles of the file which failed to compile. Some people had said that the real error (Array out of bounds warning) was because of gcc optimizations. But putting -O2 to -O0 didnt do a thing.
No of jobs - ought not to exceed 50.
"warning: variable set but not used [-Wunused-but-set-variable]"-Look at KBUILD_CFLAGS in the main Makefile. Add -Wno-error=unused-but-set-variable to the existing set of flags.
Note the following from gcc manual:
-WerrorMake all warnings into hard errors. Source code which triggers warnings will be rejected.
-w Inhibit all warning messages. If you're familiar with C code and like to fix stuff, rather than ignoring potential bugs, use this only as a last resort- A 'brahmastram' (most powerful weapon in your time of gravest need) as the epics would say
-WerrorMake all warnings into errors.
-Werror=Make the specified warning into an error. The specifier for a warning is appended, for example -Werror=switch turns the warnings controlled by -Wswitch into errors. This switch takes a negative form, to be used to negate -Werror for specific warnings, for example -Wno-error=switch makes -Wswitch warnings not be errors, even when -Werror is in effect. You can use the -fdiagnostics-show-option option to have each controllable warning amended with the option which controls it, to determine what to use with this option.
So what I did to suppress errors was to add:
Code:
KBUILD_CFLAGS += -w
KBUILD_CFLAGS += -Wno-error=unused-but-set-variable
Though the -Wunused-but-set-variable is not a real issue in itself, it generates so much "noise" that you may miss actual make errors.
This is the error what I was talking about..
Code:
drivers/net/wireless/bcm4329_204/wl_iw.c: In function 'wl_iw_set_pmksa':
drivers/net/wireless/bcm4329_204/wl_iw.c:5075:5: error: array subscript is above array bounds [-Werror=array-bounds]
drivers/net/wireless/bcm4329_204/wl_iw.c:5078:5: error: array subscript is above array bounds [-Werror=array-bounds]
Solution:
Edit drivers/net/wireless/bcm4329_204/Makefile
Locate -Werror within DHDCFLAGS, and delete it.
Code:
DHDCFLAGS = -DLINUX -DBCMDRIVER -DBCMDONGLEHOST -DDHDTHREAD -DBCMWPA2 \
-DUNRELEASEDCHIP -Dlinux -DDHD_SDALIGN=64 -DMAX_HDR_READ=64 \
-DDHD_FIRSTREAD=64 -DDHD_GPL -DDHD_SCHED -DBDC -DTOE -DDHD_BCMEVENTS \
-DSHOW_EVENTS -DBCMSDIO -DDHD_GPL -DBCMLXSDMMC -DBCMPLATFORM_BUS \
-Wall -Wstrict-prototypes -Werror -DOOB_INTR_ONLY -DCUSTOMER_HW2 \
-DDHD_USE_STATIC_BUF -DMMC_SDIO_ABORT -DWLAN_PFN -DWLAN_PROTECT \
-DBCMWAPI_WPI \
This will prevent gcc from treating mere warnings as errors.

How to modify kernels by applying mods - Applying Kernel Patches
Ok, you have compiled a simple stock kernel. Now what? Would you like to add fixes/mods developed by other kernel devs? This post explains patches and how exactly to do this.
Patches to the kernel are applied via patch files. Patch files are simple text files generated by the linux diff program which takes two text files, compares them and writes the differences (hence called diff) to another text file which by convention has the extension .patch
Attached to this post is a patch containing my "Extended battery" fix with Sibere's battfix. I'll explain patching with this. Let's understand the patch file. Open it up in any text editor.
Code:
diff -rupN -X /home/droidzone/android/kernel/exclude.opts bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c
--- bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c 2011-08-25 13:16:53.000000000 +0530
+++ bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c 2011-11-06 16:43:21.544317342 +0530
@@ -118,8 +118,11 @@ PS. 0 or other battery ID use the same p
/* Battery ID = 1: HT-E/Formosa 1400mAh */
#define BATT_ID_A 1
#define BATT_FULL_MAH_A 1400
-
#define BATT_FULL_MAH_DEFAULT 1500
+#define BATT_FULL_MAH_CAMERONSINO 2400
+#define BATT_ID_CAMERONSINO
+#define BATT_TYPE 0
+
Note the first line:
Code:
diff -rupN -X /home/droidzone/android/kernel/exclude.opts bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c
diff -rupN basically describes the command that was used to generate this patch. The -u means that the patch file is something called a universal patch
bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c was the original file, and bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c was the target file or file which contains the mod..
How to apply patch files?
The command depends on where your current directory is. If you're in ~/android/kernel/bravo_2.6.35_gb-mr/ and your current directory contains the directory 'drivers', you can apply this patch with this command:
Code:
patch -p1<extended_battfix.patch
If you're within drivers, then you have to modify the command like this:
Code:
patch -p2<extended_battfix.patch
Hope you get the gist. Basically, as you move into the source tree, you have to increment the patch level by the number of directories you've moved down into. Very simple, isnt it?

Sharing and Collaborating - Using Github and Commits
Kernel compilation is a group effort (at least it ought to be). When different devs work on different parts of the code and create their own mods, development progresses. For this purpose, it is important that you share your code with other devs. The best way to do this to upload your sources to github.
First, create a github account.
Next you can view other devs' github sources and examine their commits. Commits are basically patches applies to the previous source uploaded. Github commits use the universal patch format and can be viewed directly, downloaded as patch files, and applied to your code. You can also choose to download the whole source tree uploaded by another dev and examine it.

Kernel Build Interactive Menu system
This saves quite a lot of time if you make kernels a lot..
See post #22

Ok, the basic guide is done, guys... If you have doubts, I'll try to clear them

yeah....yeah....yeah...so nice...big thx...will try this as soon as possible..
that is what i searchd so long
edit: rated with 5 stars
with kind regards

Thank you very much droidzone.
I was waiting for a n00b guide.
Tapatalking

good job droidzone
[+1] [ i like]

Added a Howto on how to apply kernel source patch files, to post #3

lol now i understand how patching works.. i write all this **** by myself.. lol

Midian666 said:
lol now i understand how patching works.. i write all this **** by myself.. lol
Click to expand...
Click to collapse
Ha ha.. that would not have been so easy

Droidzone said:
Added a Howto on how to apply kernel source patch files, to post #3
Click to expand...
Click to collapse
sorry for offtopic but nice again and you see many people thought like me with the how to..
with kind regards...Alex

Alex-V said:
sorry for offtopic but nice again and you see many people thought like me with the how to..
with kind regards...Alex
Click to expand...
Click to collapse
I like explaining stuff and sharing..
This guide was written specifically because of your request, and I have never forgotten how you helped when I was a newbie to development.. I wouldnt probably have started developing if not for good responses from Firerat and you.

Droidzone said:
I like explaining stuff and sharing..
This guide was written specifically because of your request, and I have never forgotten how you helped when I was a newbie to development.. I wouldnt probably have started developing if not for good responses from Firerat and you.
Click to expand...
Click to collapse
and now i learn from you lol thx
with kind regards..Alex

Fantastic guide!!!!!!!

Did some more work on the first post. It now includes a flashable zip template and instructions on how to easily create your own flashable zip after compilation is over.

maybe some improvments to your making a flashable zip.
i did this with my kernels.. it took the version infos from the config files.. and put it into a folder... after this u can make zip.
ive stolen this from manus source
Code:
localVersion=`cat .your-config | fgrep CONFIG_LOCALVERSION= | cut -f 2 -d= | sed s/\"//g`
linuxVersion=`cat .your-config | fgrep "Linux kernel version:" | cut -d: -f 2 | cut -c2-`
VERSION=$linuxVersion$localVersion
echo "Kernel version=$VERSION"
rm -rf flash/system/lib/modules/*
mkdir flash/system/lib/modules/$VERSION
mkdir flash/system/lib/modules/$VERSION/kernel
tar czf modules.tgz `find . -name '*.ko'`
cd flash/system/lib/modules/$VERSION/kernel
tar xzf ../../../../../../modules.tgz
cd - > /dev/null
rm modules.tgz

This is good..Actually when I generate kernels I test too many versions that I dont usually change the local version number in the menuconfig. Instead I use the date and time (including second) to name the kernel dir and kernel zip name...
Like this..
Code:
date_str=`date '+%d%m%y_%H%M%S'`
dirname="kernel_"$nameflag"_"$date_str
pckdir="$packagedir/$dirname"
mkdir $pckdir
lastfolder=$pckdir
cd $outdir/
echo
zipnoname="kbase_"$nameflag"_"$date_str
zipaddnoname="kmods_"$nameflag"_"$date_str
zipname=$zipnoname".zip"
zipaddname=$zipaddnoname".zip"
zip -r $zipnoname ./
mv $zipname $pckdir/
As you can see, its part of my script which does a lot of things..
But getting the localversion too is a good thing..I'd put it into a textfile in the zip which users can read..

Great guide. Thanks a lot
Sent from my HTC Desire using Tapatalk

[TUTORIAL/DEV]Build AOSP Android 2.3.7 for Allwinner A10 tablets (Teclast P76Ti)

Hi!
I haven't really found any kind of guides on how to build a complete AOSP android build from source (kernel and software) for Allwinner A10 tablets like the Ainol Novo 7 Advanced or the Teclast P76Ti.
I have a Teclast P76TI (rev4), and the following guide is only tested on such a device. As it's mostly based on the source drop of the Ainol Novo 7 Advanced, it should also work on that too.

Part 1
Part 1: Preparations
A: Preconditions
For building and creating a LiveSuite flashable package you will need the following:
For building the kernel and android you will need a 64 bit Linux machine. Ubuntu 10.04 64 bit is preferred for Gingerbread builds
For creating the LiveSuite flashable images, and actually flashing you will need a Windows machine. Windows 7 x64 works fine, as should Windows XP 32 bit too.
You will also need a working LiveSuite image file, for your tablet. Make sure that you can flash it to your device without problems!
Virtual machines are fine, if you have enough memory available. To build Android in a VM, you should give it at least 2GB of RAM. Alternatively if you're using Linux, you can create a Windows VM for the image creation or flashing part.
For the rest of the tutorial I will assume you are using a Linux Virtual Machine on a Windows box.
For building the kernel and AOSP you will need at least 35GB of free space in your linux machine (the more the better). To create the images you will need an additional 2GB of space on your Windows machine.
B: Getting the kernel and AOSP sources
First initialize your Linux machine according to this page: http://source.android.com/source/initializing.html
Here are the most important commands:
Code:
$ sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
$ sudo apt-get update
$ sudo apt-get install sun-java6-jdk
$ sudo apt-get install git-core gnupg flex bison gperf build-essential \
zip curl zlib1g-dev libc6-dev lib32ncurses5-dev ia32-libs \
x11proto-core-dev libx11-dev lib32readline5-dev lib32z-dev \
libgl1-mesa-dev g++-multilib mingw32 tofrodos python-markdown \
libxml2-utils xsltproc
$ mkdir ~/bin
$ PATH=~/bin:$PATH
$ curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > ~/bin/repo
$ chmod a+x ~/bin/repo
// Additonal commands for Ubuntu 10.10:
$ sudo ln -s /usr/lib32/mesa/libGL.so.1 /usr/lib32/mesa/libGL.so
// Additonal commands for Ubuntu 11.10:
$ sudo apt-get install libx11-dev:i386
Now create a directory on your linux machine. The path should contain no spaces in it. Inside this directory first pull the kernel source from the allwinner github page:
Code:
$ git clone https://github.com/allwinner/linux-2.6.36 lichee
Make sure that the kernel resides in the lichee directory
Now download AOSP Android. We will use Gingerbread version 2.3.7:
Code:
$ mkdir android
$ cd android
$ repo init -u https://android.googlesource.com/platform/manifest -b android-2.3.7_r1
$ repo sync
Allwinner uses a special init, which has a few additional commands that you have to download too:
Code:
$ cd system/core
$ git pull git://github.com/sztupy/allwinner_android_system_core.git
$ cd ../..
You will also need the device descriptors for the tablets:
Code:
$ cd device
$ git clone git://github.com/sztupy/android_device_softwinner.git softwinner
$ cd ..
C: ARM compiler
You might simply use a compiler that is supplied for AOSP, they are inside the android/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/ directory. Alternatively you can also download the ARM compiler of CodeSourcery. Get the arm-2009q3 / arm-none-linux-gnueabi version, and unpack it inside a directory.
If ready, go on to next step: building the kernel

Part 2: Building the kernel
Change to the directory of the kernel. Now edit the file scripts/build_sun4i_crane.sh
Change the line "export CROSS_COMPILE=" to point to the ARM compiler. For example if you extracted the CodeSourcery files to /opt then use:
Code:
export CROSS_COMPILE=/opt/arm-2009q3/bin/arm-none-linux-gnueabi-
The default .config file is more or less the same that is supplied with your tablet. On the Teclast P76TI you can actually extract the .config file used from /proc using "adb pull /proc/config.gz" Not sure about other vendors.
Now you have to compile. Issue the command:
Code:
./build.sh -p sun4i_crane
This will build the kernel and modules inside the the "output" directory.
Next step is to compile Android

Part3
Part 3: Building Android
Go to the android directory. Now setup android using:
Code:
$ source build/envsetup.sh
$ lunch
You will get simething like this:
Code:
You're building on Linux
Lunch menu... pick a combo:
1. generic-eng
2. simulator
3. full_passion-userdebug
4. full_crespo4g-userdebug
5. full_crespo-userdebug
6. crane_Ainol_Novo7A-eng
7. crane_Teclast_P76TI_REV4-eng
Which would you like? [generic-eng]
Here select your device (option 6 or 7 depending on your tablet)
After you have selected it issue a make command:
Code:
$ make
This will take a while. If you're using ubuntu 11.10 make might fail, but you can fix it using these fixes: http://groups.google.com/group/android-building/browse_thread/thread/3484e7797909d014
After the building is complete you have to create the image files. Run the following command:
Code:
$ ./device/softwinner/crane-tcp76ti_r4/mkfs.sh
(substitute tcp76ti_r4 with ainovo7a if you're using the other tablet)
The image files will be ready in the directory "out/target/product/crane-tcp76ti_r4/images/" or "out/target/product/crane-ainovo7a/images/"
Copy the three files from here (root.img, recovery.img, system.img), and the "bImage" file from the "output" directory of the kernel to your Windows machine.
The next step is creating the flashable image, and flashing your build

Part 4: Flashing
A: Initialize kitchen
First download my A10 flash kitchen from this site: http://android.sztupy.hu/dl/a10/a10_flash_kitchen_v1.zip
This is based on the ainol novo 7 soruce drop, but only contains the nescessary files, and has a few batch scripts to automate the process.
There is also a v2, wich also supports ICS firmwares: http://android.sztupy.hu/dl/a10/a10_flash_kitchen_v2.zip
First unzip the files to a directory. You will see a few directories and to commands here: extract_image.bat and create_image.bat
First we have to extract an official image for your tablet. Simply comply the image file to this directory, and rename it to "original.img". If done, run extract_image.bat. This command should extract your image to the _extract directory.
Next you need to extract the bootfs inside the _bootfs directory. You have multiple options on how to do this:
1. Get the bootfs from your device on-line using adb:
Code:
> cd _bootfs
> adb shell
$ cd /
$ mount -o remount,rw -t ext4 /dev/root /
$ mkdir bootfs
$ mount -t vfat /dev/block/nanda /bootfs
(Control-C: exit adb shell)
> adb pull /bootfs
2. Get the bootfs from the extracted image file using Linux:
A. Copy the RFSFAT16_BOOTFS_000000000.fex file to linux
B.
Code:
$ mkdir bootfs
$ sudo mount -o loop RFSFAT16_BOOTFS_000000000.fex bootfs
$ cp -a bootfs b2
$ sudo umount bootfs
C. Copy the contents of the b2 directory to Windows, inside the _bootfs directory of the kitchen
B: Create image
If you have initialized the kitchen, you have to copy the four files from linux (system.img, root.img, recovery.img and bImage) inside the _input directory. You will also need to rename the .img files to .fex (so they should be system.fex, root.fex, recovery.fex).
If you are done with this, then run create_image.bat. It will create an output.img file, that can be flashed to the device using LiveSuite.

Additional information for developers:
1. Difference between AOSP 2.3.4 and Novo7 android 2.3.4
Download the diff file from here: http://android.sztupy.hu/dl/a10/diff_ainovo_aosp.gz
Filelist of differences: http://android.sztupy.hu/dl/a10/diff_ainovo_aosp_filelist.txt

sztupy, did you figure out why eDragonEx and FSBuild have an apparently unused Lua library next to each? I was interested by it, as it might be possible that some parts of the apps are actually written in Lua (would it be possible? Lua in a native DLL?), what would help reversing it.

fonix232 said:
sztupy, did you figure out why eDragonEx and FSBuild have an apparently unused Lua library next to each? I was interested by it, as it might be possible that some parts of the apps are actually written in Lua (would it be possible? Lua in a native DLL?), what would help reversing it.
Click to expand...
Click to collapse
Yeah, saw that. Also their build script used "convert" on the image.cfg, but that file wasn't in lua anyway (except for the image parts part which seems to be a lua hash).
There are also a lot of compiled files that are compiled inside the image file and which are neither the bootfs nor android or the kernel. I don't yet know what they are for, but I think they are used by LiveSuite during the flashing.

sztupy said:
Yeah, saw that. Also their build script used "convert" on the image.cfg, but that file wasn't in lua anyway (except for the image parts part which seems to be a lua hash).
There are also a lot of compiled files that are compiled inside the image file and which are neither the bootfs nor android or the kernel. I don't yet know what they are for, but I think they are used by LiveSuite during the flashing.
Click to expand...
Click to collapse
I know most of the files from the flashing, they are the following:
- SYS_CONFIG is used as a command bunch for LiveSuite. It tells the app how to flash, what to flash, where, and it configures the device too (screen size, ram info, cpu info, etc)
- Boot0 and Boot1 bins are NAND bootloaders
- FED FES and FET tools are NAND flashing utilities, checksums, hardware scanner, and other tools used during flashing.
- '12345678' files are bootloaders, config files, and tools for SDMMC flashing, if there's a device with SDMMC internal instead of NAND, these are used
- Split seems to be some kind of ID string, I had no luck retrieving it's usage and meaning.
So you say, that the actual Lua parts are the image encryption parts of eDragonEx? Interesting, maybe I've missed that spot with my tool...
Might I ask if you have tried disassembling (and decompiling) unimg.exe? I had several problems with it, but that would greatly help understanding how the images are created. I've got a C# framework, with image config parser, etcetera etcetera, to be able to read and create images in a much more advanced environment (filtering user errors, having tools for everything (bootfs modding, script.bin reversal, etcetera), and creating a working image as a final result), and it only needs the image file format (and some of my work, to create a parser).
The problems with unimg were all about positive sp values, and as I'm not a big assembly programmer, I couldn't make out anything from that. Maybe you understand it a bit more
Illetve beszélhetünk egy kicsit magyarul is. Tabletrepublic-on írtam hogy vegyél fel MSNre és részletezem a különböző Crane SDK elemek funkcióit, működését, egyebeket, amit eddig sikerült kiderítenem az egészről.
Sajnos elég zavaros, mivel vagy négyféle csomagoló rendszer készíti a fileokat, és ezek közül csak egy működött rendesen (crane_pack.exe). Jó lenne megérteni ezt a file formátumot, hogy egy kicsit normálisabb módon hozhassam létre, különféle vacakolások nélkül.

fonix232 said:
I know most of the files from the flashing, they are the following:
- SYS_CONFIG is used as a command bunch for LiveSuite. It tells the app how to flash, what to flash, where, and it configures the device too (screen size, ram info, cpu info, etc)
- Boot0 and Boot1 bins are NAND bootloaders
- FED FES and FET tools are NAND flashing utilities, checksums, hardware scanner, and other tools used during flashing.
- '12345678' files are bootloaders, config files, and tools for SDMMC flashing, if there's a device with SDMMC internal instead of NAND, these are used
- Split seems to be some kind of ID string, I had no luck retrieving it's usage and meaning.
Click to expand...
Click to collapse
Thanks for these. Seems I was mostly right
So you say, that the actual Lua parts are the image encryption parts of eDragonEx? Interesting, maybe I've missed that spot with my tool...
Click to expand...
Click to collapse
I think lua is not really used anymore. It probably had more relevance back in the past.
Might I ask if you have tried disassembling (and decompiling) unimg.exe? I had several problems with it, but that would greatly help understanding how the images are created. I've got a C# framework, with image config parser, etcetera etcetera, to be able to read and create images in a much more advanced environment (filtering user errors, having tools for everything (bootfs modding, script.bin reversal, etcetera), and creating a working image as a final result), and it only needs the image file format (and some of my work, to create a parser).
The problems with unimg were all about positive sp values, and as I'm not a big assembly programmer, I couldn't make out anything from that. Maybe you understand it a bit more
Click to expand...
Click to collapse
No, haven't tried disassembling it yet. I was very glad that it worked, and that I could create a whole working build just from the sources. I know there are a lot of quirks, like if the extension of the file is not .fex, then it will encrypt(?) it, etc. I might try it, but currently I'm more interested in getting a working AOSP ICS on my tablet. Besides for disassembly I need to be in a special mood, which I'm not really in now
Illetve beszélhetünk egy kicsit magyarul is. Tabletrepublic-on írtam hogy vegyél fel MSNre és részletezem a különböző Crane SDK elemek funkcióit, működését, egyebeket, amit eddig sikerült kiderítenem az egészről.
Sajnos elég zavaros, mivel vagy négyféle csomagoló rendszer készíti a fileokat, és ezek közül csak egy működött rendesen (crane_pack.exe). Jó lenne megérteni ezt a file formátumot, hogy egy kicsit normálisabb módon hozhassam létre, különféle vacakolások nélkül.
Click to expand...
Click to collapse
Már egy jó ideje nem használok MSN-t. Skype/GTalk/email viszont van. Ha nem használsz olyanokat, akkor azért majd felrakom.

Made a diff between AOSP android 2.3.4 and the Novo 7 2.3.4 source drop. The list can be found at post 6: http://forum.xda-developers.com/showpost.php?p=22397984&postcount=6

Nice tutorial! Do you mind if I fork your Github repo and add the device tree for the Bmorn V11 to the lunch list?

sztupy said:
Made a diff between AOSP android 2.3.4 and the Novo 7 2.3.4 source drop. The list can be found at post 6: http://forum.xda-developers.com/showpost.php?p=22397984&postcount=6
Click to expand...
Click to collapse
Most of the things changed aren't even needed for the device - ril can be replaced with a local one (and suggested by Google to do so), just like recovery changes, in ICS we already have USB BT support enabler, so BT changes can be dropped, just like framework changes (they are for the softbuttons on the notification bar), camera and mediaplayer changes should be local too (in the device tree), so 99% of changes can be dropped.
Additions are different, some can be totally erased, and some are needed. From your github, I see that you've already began making a cleaned up, generic A10 tree, but I miss a few things - libsensor for one, stagefright, camera, and audio. The AOSP stock ALSA should work, if the proper audio config is placed in a ROM, but I have bad feelings about the missing libsensors source, and camera. Stagefright has a chance too to work, but camera definitely won't, and the used sensors aren't the common ones to be included.
Használok GTalk-ot is, ott is ugyanez a nicknevem, gmail utótaggal, szóval a szokásos

FezzFest said:
Nice tutorial! Do you mind if I fork your Github repo and add the device tree for the Bmorn V11 to the lunch list?
Click to expand...
Click to collapse
That's what github is for

fonix232 said:
Most of the things changed aren't even needed for the device - ril can be replaced with a local one (and suggested by Google to do so), just like recovery changes, in ICS we already have USB BT support enabler, so BT changes can be dropped, just like framework changes (they are for the softbuttons on the notification bar), camera and mediaplayer changes should be local too (in the device tree), so 99% of changes can be dropped.
Additions are different, some can be totally erased, and some are needed. From your github, I see that you've already began making a cleaned up, generic A10 tree, but I miss a few things - libsensor for one, stagefright, camera, and audio. The AOSP stock ALSA should work, if the proper audio config is placed in a ROM, but I have bad feelings about the missing libsensors source, and camera. Stagefright has a chance too to work, but camera definitely won't, and the used sensors aren't the common ones to be included.
Használok GTalk-ot is, ott is ugyanez a nicknevem, gmail utótaggal, szóval a szokásos
Click to expand...
Click to collapse
libsensor and stagefright are still there, as I could manage them to get compiled. Not that they work though. For the libcamera it depends on CedarX, which I couldn't manage to compile, that's why I removed it (for now). Besides CedarX unfortunately not "open-source", so in theory we couldn't use it either (well... not that I actually care about licence violations).
I'm still compiling ICS. Will put back libcamera and try to get the other hardware libs to work after I managed to get ICS to boot.

sztupy said:
libsensor and stagefright are still there, as I could manage them to get compiled. Not that they work though. For the libcamera it depends on CedarX, which I couldn't manage to compile, that's why I removed it (for now). Besides CedarX unfortunately not "open-source", so in theory we couldn't use it either (well... not that I actually care about licence violations).
I'm still compiling ICS. Will put back libcamera and try to get the other hardware libs to work after I managed to get ICS to boot.
Click to expand...
Click to collapse
Please be noted that ICS requires new stagefright, camera HAL, new GPU drivers, and so on.
ICS also should have some differences in the build tree, make a new branch for sure (as an example, it requires a device.mk and device_base.mk, both being the base containers without target definition, full_[devicename].mk for the actual full target, and cm.mk for CyanogenMod, what I'd suggest you to build).
I couldn't find any of the sources, but must have overlooked something. Will check it further.

hi, I'm the author of unimg(esxgx).
and your unimg is not up-to-date(the version still have bugs to lead to fail the packing process.
Here is the latest version (fix bugs but no virus alarm[compared with the former version]), and you can use it in the same way.
please update your file.
PS. yes, the unimg can unpack and pack all allwinner's firewares without diffculty, good luck.
sorry for my english.

fonix232 said:
Might I ask if you have tried disassembling (and decompiling) unimg.exe? I had several problems with it, but that would greatly help understanding how the images are created. I've got a C# framework, with image config parser, etcetera etcetera, to be able to read and create images in a much more advanced environment (filtering user errors, having tools for everything (bootfs modding, script.bin reversal, etcetera), and creating a working image as a final result), and it only needs the image file format (and some of my work, to create a parser).
The problems with unimg were all about positive sp values, and as I'm not a big assembly programmer, I couldn't make out anything from that. Maybe you understand it a bit more
Click to expand...
Click to collapse
I think you should use unimg.exe.
unimg is the only tool to unpack it correctly.
you know, allwinner didn't want me to release the tool in public last year(the tool is for sc9800[the former chip]). for some commercial reason, the offical toolchain of a10 only contains pack_tool.
so I released the tool with other tools(rootcr, rootpk,etc..) in a small group.
but some person posted it on the internet. so....
I developed the analysis tools in the form of many files(unimg, rootcr, rootpk, unimg2), because I want to keep each of the packing stages simple.
you can use a "bat" file / a GUI shell to communicate with each other, and that is what I expected.
I can't smoothly speak english, so I modified several times.

Thank you very much!
If I may ask, would it be possible to release the source code too?

fonix232 said:
Thank you very much!
If I may ask, would it be possible to release the source code too?
Click to expand...
Click to collapse
I cannot released yet, but I will release it in a proper time on github (I have been touching with allwinner company, so I have to consider many factors).

[DEV] [GUIDE] [LINUX] Comprehensive Guide to Cross-Compiling

[SIZE="+1"]What is a Cross-Compiler?[/SIZE]
A cross compiler is a compiler capable of creating executable code for a platform other than the one on which the compiler is running. Cross compiler tools are used to generate executables for embedded system or multiple platforms. It is used to compile for a platform upon which it is not feasible to do the compiling, like microcontrollers that don't support an operating system. From wikipedia
Click to expand...
Click to collapse
[SIZE="+1"]How is that connected with an Android?[/SIZE]
In order to create a native C/C++ binary for an Android, you must firstly compile the source code. Usually you can't do so on an Android itself due to lack of proper tools and environment, or hardware barriers, especially amount of RAM. This is why you should learn how to cross-compile, to create a binary on your PC, that your ARM-based Android will understand.
[SIZE="+1"]Why do I need it?[/SIZE]
You need to learn cross-compiling technique if you want to run native C/C++ programs on an Android. Actually, if you've already built your own custom ROM from AOSP sources (i.e. CyanogenMod), then you used cross-compiling tools and methods even without noticing .
Building an AOSP ROM is fairly easy, there's one command like brunch, which does the job. However, what if you want to compile a custom, not natively included binary? This is the purpose of this tutorial.
[SIZE="+1"]What I will learn from this guide?[/SIZE]
How to properly set C/C++ building environment
How to build a native C/C++ application for Android device
How to optimize native binaries for my device
[SIZE="+1"]Step 1 - The Beginning[/SIZE]
You should start from installing any Linux-based OS, I highly suggest trying a Debian-based distro (such as Ubuntu), or even Debian itself, as this tutorial is based on it .
In general, I highly suggest to compile an AOSP ROM (such as CyanogenMod) for your device firstly. This will help you to get familiar with cross-compiling on Android. I also suggest to compile one or two programs from source for your Linux, but if you're brave enough to learn cross-compiling without doing any of these, you can skip those suggestions .
[SIZE="+1"]Step 2 - Setting up[/SIZE]
Firstly you should make sure that you have all required compile tools already.
[email protected]:~# apt-get update && apt-get install checkinstall
Click to expand...
Click to collapse
This should do the trick and install all required components.
I suggest creating a new folder and navigating to it, just to avoid a mess, but you can organize everything as you wish.
Start from downloading NDK from here.
The NDK is a toolset that allows you to implement parts of your app using native-code languages such as C and C++.
Click to expand...
Click to collapse
[email protected]:~# wget http://dl.google.com/android/ndk/android-ndk-r9d-linux-x86_64.tar.bz2
[email protected]:~# tar xvf android-ndk-r9d-linux-x86_64.tar.bz2
[email protected]:~# mv android-ndk-r9d ndk
Click to expand...
Click to collapse
Now you should make a standalone toolchain, navigate to root of your ndk (this is important) and then build your toolchain:
[email protected]:~# cd ndk/
[email protected]:~/ndk# build/tools/make-standalone-toolchain.sh --toolchain=arm-linux-androideabi-4.8 --platform=android-18 --install-dir=/root/ndkTC
Copying prebuilt binaries...
Copying sysroot headers and libraries...
Copying libstdc++ headers and libraries...
Copying files to: /root/ndkTC
Cleaning up...
Done.
Click to expand...
Click to collapse
You should edit bolded variables to your preferences. Toolchain is the version of GCC you want to use, 4.8 is currently the newest one, in the future it may be 4.9 and so on. Platform is a target API for your programs, this is important only for android-specific commands, such as logging to logcat. When compiling a native Linux program, this won't matter (but it's a good idea to set it properly, just in case). Install dir specifies destination of your toolchain, make sure that it's other than ndk (as you can see I have ndk in /root/ndk and toolchain in /root/ndkTC).
Now you need to download my exclusive cc.sh script from here and make it executable.
[email protected]:~# wget https://dl.dropboxusercontent.com/u/23869279/Files/cc.sh
[email protected]:~# chmod 755 cc.sh
Click to expand...
Click to collapse
This script is a very handy tool written by me to make your life easier while cross-compiling. Before running it make sure to edit "BASIC" options, especially NDK paths. Apart from that it's a good idea to take a look at DEVICEFLAGS and setting them properly for your device, or clearing for generic build. You don't need to touch other ones unless you want/need them.
Just for a reference, I'll include currently editable options:
#############
### BASIC ###
#############
# Root of NDK, the one which contains $NDK/ndk-build binary
NDK="/root/ndk"
# Root of NDK toolchain, the one used in --install-dir from $NDK/build/tools/make-standalone-toolchain.sh. Make sure it contains $NDKTC/bin directory with $CROSS_COMPILE binaries
NDKTC="/root/ndkTC"
# Optional, may help NDK in some cases, should be equal to GCC version of the toolchain specified above
export NDK_TOOLCHAIN_VERSION=4.8
# This flag turns on ADVANCED section below, you should use "0" if you want easy compiling for generic targets, or "1" if you want to get best optimized results for specific targets
# In general it's strongly suggested to leave it turned on, but if you're using makefiles, which already specify optimization level and everything else, then of course you may want to turn it off
ADVANCED="1"
################
### ADVANCED ###
################
# Device CFLAGS, these should be taken from TARGET_GLOBAL_CFLAGS property of BoardCommonConfig.mk of your device, eventually leave them empty for generic non-device-optimized build
# Please notice that -march flag comes from TARGET_ARCH_VARIANT
DEVICECFLAGS="-march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp"
# This specifies optimization level used during compilation. Usually it's a good idea to keep it on "-O2" for best results, but you may want to experiment with "-Os", "-O3" or "-Ofast"
OLEVEL="-O2"
# This specifies extra optimization flags, which are not selected by any of optimization levels chosen above
# Please notice that they're pretty EXPERIMENTAL, and if you get any compilation errors, the first step is experimenting with them or disabling them completely, you may also want to try different O level
OPTICFLAGS="-s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing"
# This specifies extra linker optimizations. Same as above, in case of problems this is second step for finding out the culprit
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections"
# This specifies additional sections to strip, for extra savings on size
STRIPFLAGS="-s -R .note -R .comment -R .gnu.version -R .gnu.version_r"
# Additional definitions, which may help some binaries to work with android
DEFFLAGS="-DNDEBUG -D__ANDROID__"
##############
### EXPERT ###
##############
# This specifies host (target) for makefiles. In some rare scenarios you may also try "--host=arm-linux-androideabi"
# In general you shouldn't change that, as you're compiling binaries for low-level ARM-EABI and not Android itself
CONFIGANDROID="--host=arm-linux-eabi"
# This specifies the CROSS_COMPILE variable, again, in some rare scenarios you may also try "arm-eabi-"
# But beware, NDK doesn't even offer anything apart from arm-linux-androideabi one, however custom toolchains such as Linaro offer arm-eabi as well
CROSS_COMPILE="arm-linux-androideabi-"
# This specifies if we should also override our native toolchain in the PATH in addition to overriding makefile commands such as CC
# You should NOT enable it, unless your makefile calls "gcc" instead of "$CC" and you want to point "gcc" (and similar) to NDKTC
# However, in such case, you should either fix makefile yourself or not use it at all
# You've been warned, this is not a good idea
TCOVERRIDE="0"
# Workaround for some broken compilers with malloc problems (undefined reference to rpl_malloc and similar errors during compiling), don't uncomment unless you need it
#export ac_cv_func_malloc_0_nonnull=yes
Click to expand...
Click to collapse
As you can notice, my magic script already contains bunch of optimizations, especially device-based optimizations, which are the most important. Now it's the time to run our script, but in current shell and not a new one.
[email protected]:~# source cc.sh
Done setting your environment
CFLAGS: -O2 -march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp -s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing -DNDEBUG -D__ANDROID__
LDFLAGS: -Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections
CC points to arm-linux-androideabi-gcc and this points to /root/ndkTC/bin/arm-linux-androideabi-gcc
Use "$CC" command for calling gcc and "$CCC" command for calling our optimized CC
Use "$CXX" command for calling g++ and "$CCXX" for calling our optimized CXX
Use "$STRIP" command for calling strip and "$SSTRIP" command for calling our optimized STRIP
Example: "$CCC myprogram.c -o mybinary && $SSTRIP mybinary "
When using makefiles with configure options, always use "./configure $CONFIGANDROID" instead of using "./configure" itself
Please notice that makefiles may, or may not, borrow our CFLAGS and LFLAGS, so I suggest to double-check them and eventually append them to makefile itself
Pro tip: Makefiles with configure options always borrow CC, CFLAGS and LDFLAGS, so if you're using ./configure, probably you don't need to do anything else
Click to expand...
Click to collapse
Command "source cc.sh" executes cc.sh and "shares" the environment, which means that any exports will be exported to our current shell, and this is what we want. It acts the same as AOSP's ". build/envsetup.sh", so you can also use . instead of source.
As you can see above, my script should let you know if it properly set everything, especially if $CC points to our ndkTC. It also set a generic "$CCC" and "$CCXX" commands, which are optimized versions of standard $CC. $CC points to our cross-compiler, $CCC points to our cross-compiler and also includes our optimization flags.
[email protected]:~# echo $CC
arm-linux-androideabi-gcc
[email protected]:~# echo $CCC
arm-linux-androideabi-gcc -O2 -march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp -s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing -DNDEBUG -D__ANDROID__ -Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections
Click to expand...
Click to collapse
[SIZE="+1"]Step 3 - Cross-Compiling[/SIZE]
Now we'll compile our first program for Android!
Create a new file hello.c, and put inside:
Code:
#include <stdio.h>
int main (void)
{
puts ("Hello World!");
return 0;
}
Now you compile and strip it:
[email protected]:~# $CCC hello.c -o hello && $SSTRIP hello
Click to expand...
Click to collapse
Remember that $CCC and $SSTRIP command will only work if you source'd cc.sh script explained above. $CCC command compiles source code to a binary with already optimized flags (device flags, optimization level, optimization flags, linker flags), while $SSTRIP command strips "bloat" from output binary, such as comments and notices. The purpose is to make a binary smaller and faster.
You can check if your binary has been compiled properly through readelf command.
[email protected]:~# readelf -A hello
Attribute Section: aeabi
File Attributes
Tag_CPU_name: "ARM v7"
Tag_CPU_arch: v7
Tag_CPU_arch_profile: Application
Tag_ARM_ISA_use: Yes
Tag_THUMB_ISA_use: Thumb-2
Tag_FP_arch: VFPv3
Tag_Advanced_SIMD_arch: NEONv1
Tag_ABI_PCS_wchar_t: 4
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_align_needed: 8-byte
Tag_ABI_enum_size: int
Tag_ABI_HardFP_use: SP and DP
Tag_ABI_optimization_goals: Aggressive Speed
Tag_CPU_unaligned_access: v6
Tag_DIV_use: Not allowed
Click to expand...
Click to collapse
As you can see, I've compiled a binary optimized for ARM v7, with THUMB-2 instructions and NEON support. How nice! Is it because of device-specific flags? Let's check what happens if we use $CC instead of $CCC:
[email protected]:~# readelf -A hello2
Attribute Section: aeabi
File Attributes
Tag_CPU_name: "5TE"
Tag_CPU_arch: v5TE
Tag_ARM_ISA_use: Yes
Tag_THUMB_ISA_use: Thumb-1
Tag_FP_arch: VFPv2
Tag_ABI_PCS_wchar_t: 4
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_align_needed: 8-byte
Tag_ABI_enum_size: int
Tag_ABI_optimization_goals: Aggressive Speed
Tag_DIV_use: Not allowed
Click to expand...
Click to collapse
As you can see, if you do not specify flags, you'll lose major portion of optimizations. Of course binary will work properly, hence it has been cross-compiled for ARM, but we can always make it smaller and faster!
[SIZE="+1"]Step 4 - Testing[/SIZE]
A favourite part of everything, tests!
[email protected]:~/shared# adb shell
[email protected]:/ # sysrw
[email protected]:/ # exit
[email protected]:~/shared# adb push hello /system/bin/hello
95 KB/s (5124 bytes in 0.052s)
[email protected]:~/shared# adb shell
[email protected]:/ # chmod 755 /system/bin/hello
[email protected]:/ # chown root:system /system/bin/hello
[email protected]:/ # exit
Click to expand...
Click to collapse
In above example I pushed my binary straight to /system/bin directory, which is in the Android's PATH. If you don't have rooted device that's not a problem, you can use /data/local directory or /storage/sdcard0. You can also upload your binary anywhere you want and download it as any other file, then run from /storage/sdcard0/Download, this way doesn't require even working ADB . Just don't forget about setting proper permissions afterwards!
Now let's try to run it!
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
If your binary is not in the PATH, you should write full path to your binary of course. As I pushed my binary to /system/bin, I don't need to do so.
If everything finished successfully and you got your very first Hello World response as above, congratulations. You've just compiled and ran your first native C/C++ program on Android device.
[SIZE="+1"]What to do next?[/SIZE]
In theory, you can now compile anything you want. Here are some apps that I'm using in my ArchiDroid ROM:
Pixelserv
Haveged
Dnsmasq
DNRD
Rinetd
These are only a few examples. You can compile anything you want, or even write your own native applications. Good luck!

JustArchi said:
[SIZE=+1]What is a Cross-Compiler?[/SIZE]
[SIZE=+1]How is that connected with an Android?[/SIZE]
In order to create a native C/C++ binary for an Android, you must firstly compile the source code. Usually you can't do so on an Android itself due to lack of proper tools and environment, or hardware barriers, especially amount of RAM. This is why you should learn how to cross-compile, to create a binary on your PC, that your ARM-based Android will understand.
[SIZE=+1]Why do I need it?[/SIZE]
You need to learn cross-compiling technique if you want to run native C/C++ programs on an Android. Actually, if you've already built your own custom ROM from AOSP sources (i.e. CyanogenMod), then you used cross-compiling tools and methods even without noticing .
Building an AOSP ROM is fairly easy, there's one command like brunch, which does the job. However, what if you want to compile a custom, not natively included binary? This is the purpose of this tutorial.
[SIZE=+1]What I will learn from this guide?[/SIZE]
How to properly set C/C++ building environment
How to build a native C/C++ application for Android device
How to optimize native binaries for my device
[SIZE=+1]Step 1 - The Beginning[/SIZE]
You should start from installing any Linux-based OS, I highly suggest trying a Debian-based distro (such as Ubuntu), or even Debian itself, as this tutorial is based on it .
In general, I highly suggest to compile an AOSP ROM (such as CyanogenMod) for your device firstly. This will help you to get familiar with cross-compiling on Android. I also suggest to compile one or two programs from source for your Linux, but if you're brave enough to learn cross-compiling without doing any of these, you can skip those suggestions .
[SIZE=+1]Step 2 - Setting up[/SIZE]
Firstly you should make sure that you have all required compile tools already.
This should do the trick and install all required components.
I suggest creating a new folder and navigating to it, just to avoid a mess, but you can organize everything as you wish.
Start from downloading NDK from here.
Now you should make a standalone toolchain, navigate to root of your ndk (this is important) and then build your toolchain:
You should edit bolded variables to your preferences. Toolchain is the version of GCC you want to use, 4.8 is currently the newest one, in the future it may be 4.9 and so on. Platform is a target API for your programs, this is important only for android-specific commands, such as logging to logcat. When compiling a native Linux program, this won't matter (but it's a good idea to set it properly, just in case). Install dir specifies destination of your toolchain, make sure that it's other than ndk (as you can see I have ndk in /root/ndk and toolchain in /root/ndkTC).
Now you need to download my exclusive cc.sh script from here and make it executable.
This script is a very handy tool written by me to make your life easier while cross-compiling. Before running it make sure to edit "BASIC" options, especially NDK paths. Apart from that it's a good idea to take a look at DEVICEFLAGS and setting them properly for your device, or clearing for generic build. You don't need to touch other ones unless you want/need them.
Just for a reference, I'll include currently editable options:
As you can notice, my magic script already contains bunch of optimizations, especially device-based optimizations, which are the most important. Now it's the time to run our script, but in current shell and not a new one.
Command "source cc.sh" executes cc.sh and "shares" the environment, which means that any exports will be exported to our current shell, and this is what we want. It acts the same as AOSP's ". build/envsetup.sh", so you can also use . instead of source.
As you can see above, my script should let you know if it properly set everything, especially if $CC points to our ndkTC. It also set a generic "$CCC" and "$CCXX" commands, which are optimized versions of standard $CC. $CC points to our cross-compiler, $CCC points to our cross-compiler and also includes our optimization flags.
[SIZE=+1]Step 3 - Cross-Compiling[/SIZE]
Now we'll compile our first program for Android!
Create a new file hello.c, and put inside:
Code:
#include <stdio.h>
int main (void)
{
puts ("Hello World!");
return 0;
}
Now you compile and strip it:
Remember that $CCC and $SSTRIP command will only work if you source'd cc.sh script explained above. $CCC command compiles source code to a binary with already optimized flags (device flags, optimization level, optimization flags, linker flags), while $SSTRIP command strips "bloat" from output binary, such as comments and notices. The purpose is to make a binary smaller and faster.
You can check if your binary has been compiled properly through readelf command.
As you can see, I've compiled a binary optimized for ARM v7, with THUMB-2 instructions and NEON support. How nice! Is it because of device-specific flags? Let's check what happens if we use $CC instead of $CCC:
As you can see, if you do not specify flags, you'll lose major portion of optimizations. Of course binary will work properly, hence it has been cross-compiled for ARM, but we can always make it smaller and faster!
[SIZE=+1]Step 4 - Testing[/SIZE]
A favourite part of everything, tests!
In above example I pushed my binary straight to /system/bin directory, which is in the Android's PATH. If you don't have rooted device that's not a problem, you can use /data/local directory or /storage/sdcard0. You can also upload your binary anywhere you want and download it as any other file, then run from /storage/sdcard0/Download, this way doesn't require even working ADB . Just don't forget about setting proper permissions afterwards!
Now let's try to run it!
If your binary is not in the PATH, you should write full path to your binary of course. As I pushed my binary to /system/bin, I don't need to do so.
If everything finished successfully and you got your very first Hello World response as above, congratulations. You've just compiled and ran your first native C/C++ program on Android device.
[SIZE=+1]What to do next?[/SIZE]
In theory, you can now compile anything you want. Here are some apps that I'm using in my ArchiDroid ROM:
Pixelserv
Haveged
Dnsmasq
DNRD
Rinetd
These are only a few examples. You can compile anything you want, or even write your own native applications. Good luck!
Click to expand...
Click to collapse
[Mod Edit: Please don't quote the whole OP]
Fricking awesome. Worked perfect on my builduntu running in VirtualBox

dicksteele said:
Fricking awesome. Worked perfect on my builduntu running in VirtualBox
Click to expand...
Click to collapse
I'm very glad it worked for you .

Maybe you happen to know which packages checkinstall depends on? I want to run this on Arch - pun not intended - and pacman doesn't exactly talk with debs.
(Przy okazji, świetny tutorial c: )

Dragoon Aethis said:
Maybe you happen to know which packages checkinstall depends on? I want to run this on Arch - pun not intended - and pacman doesn't exactly talk with debs.
(Przy okazji, świetny tutorial c: )
Click to expand...
Click to collapse
Checkinstall makes sure that you have all required packages installed. You can achieve nearly the same by installing "build-essential, gcc, g++, make", and that should be enough I guess .

Also, big kudos to @willverduzco for featuring my guide on XDA portal!

I would like to see a guide for llvm/ clang.
Sent from my GT-I9000 using xda app-developers app

maybe a bit irrelevant... but i wanted to learn how to cross compile/port a binary (for example "applypatch") for cygwin... any link to guide will be helpful
Thank You

DerRomtester said:
I would like to see a guide for llvm/ clang.
Sent from my GT-I9000 using xda app-developers app
Click to expand...
Click to collapse
When making standalone toolchain you should use clang instead of gcc. You should also study my cc.sh script and adapt to your own. After that, steps are nearly the same.
EnerJon said:
maybe a bit irrelevant... but i wanted to learn how to cross compile/port a binary (for example "applypatch") for cygwin... any link to guide will be helpful
Thank You
Click to expand...
Click to collapse
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.

JustArchi said:
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.
Click to expand...
Click to collapse
Actually i was making a tool for windows to generate/apply OTA for Android ROMs... i wanted to compile/port "IMGDIFF2" and "applypatch" from android sources...

EnerJon said:
Actually i was making a tool for windows to generate/apply OTA for Android ROMs... i wanted to compile/port "IMGDIFF2" and "applypatch" from android sources...
Click to expand...
Click to collapse
Then you should find your sources for IMGDIFF2 and applypatch and compile from source for Android, just like example hello.c above.

@JustArchi I saw this guide mentioned on the portal and read through it. Very interesting stuff. Great work explaining. I've got several questions, however, perhaps you can elaborate on.
My primary PC OS is Gentoo Linux (I've been using it for 10 years), in patricular ~amd64 which is the equivalent of Debian unstable. In Gentoo, all packages are compiled from the sources. I have a very up to date complete toolchain already installed and functioning properly as part of the native package installation system which uses portage for maintaining and updating.
I've already compiled CM and AOSP for my device, but I can't for the life of me understand why when setting up my build environment using either Google or CM tools several much older versions of GCC and GLIBC are installed into my source repos and used to build the ROM when the prerequisites for building the environment already require a working toolchain on the host build box?
Isn't there a way to just use the native toolchain from the host? Ideally, I'd love to free up the space used by these extra compilers and libraries for sources instead. Additionally, since my toolchain is much newer (gcc-4.8.2, glibc-2.19, etc) and optimized for my hardware than these generic prebuilt binaries, my ROM builds would compile faster and more optimized if I could use it instead.
The big question I ask is would you know what I'd have to do to setup my native environment to build Android? I'd truly love to be able to get rid of these other toolchains and free up the space on my harddrive. Any help would be greatly appreciated. TIA

JustArchi said:
When making standalone toolchain you should use clang instead of gcc. You should also study my cc.sh script and adapt to your own. After that, steps are nearly the same.
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.
Click to expand...
Click to collapse
I try this. I would like to cross compile a kernel with clang. Hopefully i get it working.

Odysseus1962 said:
@JustArchi I saw this guide mentioned on the portal and read through it. Very interesting stuff. Great work explaining. I've got several questions, however, perhaps you can elaborate on.
My primary PC OS is Gentoo Linux (I've been using it for 10 years), in patricular ~amd64 which is the equivalent of Debian unstable. In Gentoo, all packages are compiled from the sources. I have a very up to date complete toolchain already installed and functioning properly as part of the native package installation system which uses portage for maintaining and updating.
I've already compiled CM and AOSP for my device, but I can't for the life of me understand why when setting up my build environment using either Google or CM tools several much older versions of GCC and GLIBC are installed into my source repos and used to build the ROM when the prerequisites for building the environment already require a working toolchain on the host build box?
Isn't there a way to just use the native toolchain from the host? Ideally, I'd love to free up the space used by these extra compilers and libraries for sources instead. Additionally, since my toolchain is much newer (gcc-4.8.2, glibc-2.19, etc) and optimized for my hardware than these generic prebuilt binaries, my ROM builds would compile faster and more optimized if I could use it instead.
The big question I ask is would you know what I'd have to do to setup my native environment to build Android? I'd truly love to be able to get rid of these other toolchains and free up the space on my harddrive. Any help would be greatly appreciated. TIA
Click to expand...
Click to collapse
You need special compiler capable of compiling for specific architecture, this is not the same as native GCC toolchain for amd64. When you're using native compiler, output is always designed for amd64 or i386, when using cross-compiler, output is always designed for ARM, or other specific architecture.

JustArchi said:
You need special compiler capable of compiling for specific architecture, this is not the same as native GCC toolchain for amd64. When you're using native compiler, output is always designed for amd64 or i386, when using cross-compiler, output is always designed for ARM, or other specific architecture.
Click to expand...
Click to collapse
Thanks for the quick response. I'm a bit disappointed, but I'm still wondering that there has to be some way for me to utilize the ARM toolchain I currently have installed to cross-compile from the sources a more updated optimized toolchain for me to build with. Unfortunately (for me), that Gentoo is more of a niche Linux distro so finding help in their forums for working with ARM is difficult. As it is, it took much effort and trial and error to setup my current configuration to build with since nearly everything on the net is geared towards Ubuntu / Debian (both of which I feel are loaded with useless cruft and dependencies for things I have never and will never use).
Anyhow thanks again for this great guide, and for your continued work here helping us all.
Ciao

Dropbox link is down
Inviato dal mio GT-I9300 utilizzando Tapatalk

Code:
#!/bin/bash
# _ _ _ _ _
# | |_ _ ___| |_ / \ _ __ ___| |__ (_)
# _ | | | | / __| __| / _ \ | '__/ __| '_ \| |
# | |_| | |_| \__ \ |_ / ___ \| | | (__| | | | |
# \___/ \__,_|___/\__/_/ \_\_| \___|_| |_|_|
#
# Copyright 2014 Łukasz "JustArchi" Domeradzki
# Contact: [email protected]
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#############
### BASIC ###
#############
# Root of NDK, the one which contains $NDK/ndk-build binary
NDK="/root/ndk"
# Root of NDK toolchain, the one used in --install-dir from $NDK/build/tools/make-standalone-toolchain.sh. Make sure it contains $NDKTC/bin directory with $CROSS_COMPILE binaries
NDKTC="/root/ndkTC"
# Optional, may help NDK in some cases, should be equal to GCC version of the toolchain specified above
export NDK_TOOLCHAIN_VERSION=4.8
# This flag turns on ADVANCED section below, you should use "0" if you want easy compiling for generic targets, or "1" if you want to get best optimized results for specific targets
# In general it's strongly suggested to leave it turned on, but if you're using makefiles, which already specify optimization level and everything else, then of course you may want to turn it off
ADVANCED="1"
################
### ADVANCED ###
################
# Device CFLAGS, these should be taken from TARGET_GLOBAL_CFLAGS property of BoardCommonConfig.mk of your device, eventually leave them empty for generic non-device-optimized build
# Please notice that -march flag comes from TARGET_ARCH_VARIANT
DEVICECFLAGS="-march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp"
# This specifies optimization level used during compilation. Usually it's a good idea to keep it on "-O2" for best results, but you may want to experiment with "-Os", "-O3" or "-Ofast"
OLEVEL="-O2"
# This specifies extra optimization flags, which are not selected by any of optimization levels chosen above
# Please notice that they're pretty EXPERIMENTAL, and if you get any compilation errors, the first step is experimenting with them or disabling them completely, you may also want to try different O level
OPTICFLAGS="-s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing"
# This specifies extra linker optimizations. Same as above, in case of problems this is second step for finding out the culprit
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections"
# This specifies additional sections to strip, for extra savings on size
STRIPFLAGS="-s -R .note -R .comment -R .gnu.version -R .gnu.version_r"
# Additional definitions, which may help some binaries to work with android
DEFFLAGS="-DNDEBUG -D__ANDROID__"
##############
### EXPERT ###
##############
# This specifies host (target) for makefiles. In some rare scenarios you may also try "--host=arm-linux-androideabi"
# In general you shouldn't change that, as you're compiling binaries for low-level ARM-EABI and not Android itself
CONFIGANDROID="--host=arm-linux-eabi"
# This specifies the CROSS_COMPILE variable, again, in some rare scenarios you may also try "arm-eabi-"
# But beware, NDK doesn't even offer anything apart from arm-linux-androideabi one, however custom toolchains such as Linaro offer arm-eabi as well
CROSS_COMPILE="arm-linux-androideabi-"
# This specifies if we should also override our native toolchain in the PATH in addition to overriding makefile commands such as CC
# You should NOT enable it, unless your makefile calls "gcc" instead of "$CC" and you want to point "gcc" (and similar) to NDKTC
# However, in such case, you should either fix makefile yourself or not use it at all
# You've been warned, this is not a good idea
TCOVERRIDE="0"
# Workaround for some broken compilers with malloc problems (undefined reference to rpl_malloc and similar errors during compiling), don't uncomment unless you need it
#export ac_cv_func_malloc_0_nonnull=yes
############
### CORE ###
############
# You shouldn't edit anything from now on
if [ "$ADVANCED" -ne 0 ]; then # If advanced is specified, we override flags used by makefiles with our optimized ones, of course if makefile allows that
export CFLAGS="$OLEVEL $DEVICECFLAGS $OPTICFLAGS $DEFFLAGS"
export LOCAL_CFLAGS="$CFLAGS"
export CXXFLAGS="$CFLAGS" # We use same flags for CXX as well
export LOCAL_CXXFLAGS="$CXXFLAGS"
export CPPFLAGS="$CPPFLAGS" # Yes, CPP is the same as CXX, because they're both used in different makefiles/compilers, unfortunately
export LOCAL_CPPFLAGS="$CPPFLAGS"
export LDFLAGS="$LDFLAGS"
export LOCAL_LDFLAGS="$LDFLAGS"
fi
if [ ! -z "$NDK" ] && [ "$(echo $PATH | grep -qi $NDK; echo $?)" -ne 0 ]; then # If NDK doesn't exist in the path (yet), prepend it
export PATH="$NDK:$PATH"
fi
if [ ! -z "$NDKTC" ] && [ "$(echo $PATH | grep -qi $NDKTC; echo $?)" -ne 0 ]; then # If NDKTC doesn't exist in the path (yet), prepend it
export PATH="$NDKTC/bin:$PATH"
fi
export CROSS_COMPILE="$CROSS_COMPILE" # All makefiles depend on CROSS_COMPILE variable, this is important to set"
export AS=${CROSS_COMPILE}as
export AR=${CROSS_COMPILE}ar
export CC=${CROSS_COMPILE}gcc
export CXX=${CROSS_COMPILE}g++
export CPP=${CROSS_COMPILE}cpp
export LD=${CROSS_COMPILE}ld
export NM=${CROSS_COMPILE}nm
export OBJCOPY=${CROSS_COMPILE}objcopy
export OBJDUMP=${CROSS_COMPILE}objdump
export READELF=${CROSS_COMPILE}readelf
export RANLIB=${CROSS_COMPILE}ranlib
export SIZE=${CROSS_COMPILE}size
export STRINGS=${CROSS_COMPILE}strings
export STRIP=${CROSS_COMPILE}strip
if [ "$TCOVERRIDE" -eq 1 ]; then # This is not a a good idea...
alias as="$AS"
alias ar="$AR"
alias gcc="$CC"
alias g++="$CXX"
alias cpp="$CPP"
alias ld="$LD"
alias nm="$NM"
alias objcopy="$OBJCOPY"
alias objdump="$OBJDUMP"
alias readelf="$READELF"
alias ranlib="$RANLIB"
alias size="$SIZE"
alias strings="$STRINGS"
alias strip="$STRIP"
fi
export CONFIGANDROID="$CONFIGANDROID"
export CCC="$CC $CFLAGS $LDFLAGS"
export CXX="$CXX $CXXFLAGS $LDFLAGS"
export SSTRIP="$STRIP $STRIPFLAGS"
echo "Done setting your environment"
echo
echo "CFLAGS: $CFLAGS"
echo "LDFLAGS: $LDFLAGS"
echo "CC points to $CC and this points to $(which "$CC")"
echo
echo "Use \"\$CC\" command for calling gcc and \"\$CCC\" command for calling our optimized CC"
echo "Use \"\$CXX\" command for calling g++ and \"\$CCXX\" for calling our optimized CXX"
echo "Use \"\$STRIP\" command for calling strip and \"\$SSTRIP\" command for calling our optimized STRIP"
echo
echo "Example: \"\$CCC myprogram.c -o mybinary && \$SSTRIP mybinary \""
echo
echo "When using makefiles with configure options, always use \"./configure \$CONFIGANDROID\" instead of using \"./configure\" itself"
echo "Please notice that makefiles may, or may not, borrow our CFLAGS and LFLAGS, so I suggest to double-check them and eventually append them to makefile itself"
echo "Pro tip: Makefiles with configure options always borrow CC, CFLAGS and LDFLAGS, so if you're using ./configure, probably you don't need to do anything else"
Temporary replacement for cc.sh, as dropbox will be up soon.

Hi!
Great info.
To cross compile some packages with autotools (./configure; make; make install) it's needed to export the SYSROOT path ($ndkTC/sysroot) and include the option --sysroot=$SYSROOT on CFLAGS. Some need too --with-sysroot=$SYSROOT as configure option. This way the configure script and linker can find the libraries.
If i'm building a library that must be used as dependence to other program I use to include --static to build a static library and --prefix=$SYSROOT/usr on configure options to install the lib on toolchain sysroot folder...
Thanks.

sfortier said:
Hi!
Great info.
To cross compile some packages with autotools (./configure; make; make install) it's needed to export the SYSROOT path ($ndkTC/sysroot) and include the option --sysroot=$SYSROOT on CFLAGS. Some need too --with-sysroot=$SYSROOT as configure option. This way the configure script and linker can find the libraries.
If i'm building a library that must be used as dependence to other program I use to include --static to build a static library and --prefix=$SYSROOT/usr on configure options to install the lib on toolchain sysroot folder...
Thanks.
Click to expand...
Click to collapse
Hey.
Nice to know, I'll update my script with that. Thanks!

My last attempt to cross compile something was qemu (i'm was thinking on run windows on my tablet... )
I needed to build glib, pixmap, libpng, zlib, libjpeg-turbo, libiconv, libffi, libintl. Now I have my toolchain with all these usefull static (I prefer static libs to simplify binary installation) libs installed!

Android 10 cross compiling

Hey.
First off, I'm a newbie with this whole Android thing.
Here's what I'm trying to do:
In the past, I've used LiME to build a LKM for the acquisition of RAM on another system. This was simpler, because I would build a VM with the same kernel as the target machine I was going to capture RAM from.
Fast forward a few years, and I'm interested in capturing some Android memory. I've followed the instructions from this page to attempt to cross compile on an Ubuntu system for Android 10 in order to build the LKM for capturing memory on Android: hxxps://github.com/504ensicsLabs/LiME/tree/master/doc
What I've done:
Disclaimer: Although, I'm researching and learning as much as I can, there are too many gaps in my knowledge at the moment. My development experience is limited to using Python to automate stuff. That's about it.
I built an Ubuntu 20 VM and installed SDK, NDK and all the necessary libraries.
I've downloaded the Google msm kernel source code.
I stepped through the LiME cross compiling instructions.
I initially tried using NDK 21.1.6352462 only to realize that the command in the LiME instructions would not work because the abi files are no longer there so I downgraded my NDK to 18.1.5063045 to have some kind of baseline with the instructions.
I got all my environment parameters up, pulled the .config from the phone and tried the following command:
make ARCH=arm64 CROSS_COMPILE=$CC_PATH/arm-linux-androideabi- modules_prepare
I initially got this error: "Cannot use CONFIG_CC_STACKPROTECTOR_STRONG: -fstack-protector-strong not supported by compiler" so I edited the Makefile and commented out the line.
That got rid of that error message. I'm not concerned about stack overflows (security) for what I'm doing.
I then ran the same command again and got the following error: TypeError: unsupported operand type(s) for >>: 'builtin_function_or_method' and '_io.TextIOWrapper'. Did you mean "print(<message>, file=<output_stream>)"?
I edited the ./scripts/gcc-wrapper.py file so that the print statements followed Python 3.6+ syntax
That got rid of that error message and brought on the following error message which is where I'm stumped.
Makefile:496: *** "Clang with Android --target detected. Did you specify CLANG_TRIPLE?". Stop.
From my research, I see that google is now using Clang/LLVM to compile the Kernel as opposed to gcc. This is where I'm stumped and have some trouble grasping exactly what I should be doing to make this work.
How do I make this command: "make ARCH=arm64 CROSS_COMPILE=$CC_PATH/arm-linux-androideabi- modules_prepare" work OR how do I translate that command into something clang understands, can I still use gcc command and clang will understand and convert?
The following steps are where my challenges lie:
Next we have to prepare our kernel source for our module.
$ cd $KSRC_PATH
$ make ARCH=arm CROSS_COMPILE=$CC_PATH/arm-eabi- modules_prepare
PREPARING THE MODULE FOR COMPILATION
We need to create a Makefile to cross-compile our kernel module. A sample Makefile for cross-compiling is shipped with the LiME source. The contents of your Makefile should be similar to the following:
***Will this process be the same?
obj-m := lime.o
lime-objs := main.o tcp.o disk.o
KDIR := /path/to/kernel-source
PWD := $(shell pwd)
CCPATH := /path/to/android-ndk/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/
default:
$(MAKE) ARCH=arm CROSS_COMPILE=$(CCPATH)/arm-eabi- -C $(KDIR) M=$(PWD) modules
COMPILING THE MODULE
cd $LIME_SRC
make
===========
Thanks for listening.
Denis
P.S. My env variables
export SDK_PATH=/home/devop/Android/Sdk
export NDK_PATH=/home/devop/Android/Sdk/ndk/18.1.5063045
export KSRC_PATH=/home/devop/WORKING_DIRECTORY/private/msm-google
export CC_PATH=$NDK_PATH/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin
export LIME_SRC=/home/devop/LiME/LiME-master/src