[SIZE="+1"]What is a Cross-Compiler?[/SIZE]
A cross compiler is a compiler capable of creating executable code for a platform other than the one on which the compiler is running. Cross compiler tools are used to generate executables for embedded system or multiple platforms. It is used to compile for a platform upon which it is not feasible to do the compiling, like microcontrollers that don't support an operating system. From wikipedia
Click to expand...
Click to collapse
[SIZE="+1"]How is that connected with an Android?[/SIZE]
In order to create a native C/C++ binary for an Android, you must firstly compile the source code. Usually you can't do so on an Android itself due to lack of proper tools and environment, or hardware barriers, especially amount of RAM. This is why you should learn how to cross-compile, to create a binary on your PC, that your ARM-based Android will understand.
[SIZE="+1"]Why do I need it?[/SIZE]
You need to learn cross-compiling technique if you want to run native C/C++ programs on an Android. Actually, if you've already built your own custom ROM from AOSP sources (i.e. CyanogenMod), then you used cross-compiling tools and methods even without noticing .
Building an AOSP ROM is fairly easy, there's one command like brunch, which does the job. However, what if you want to compile a custom, not natively included binary? This is the purpose of this tutorial.
[SIZE="+1"]What I will learn from this guide?[/SIZE]
How to properly set C/C++ building environment
How to build a native C/C++ application for Android device
How to optimize native binaries for my device
[SIZE="+1"]Step 1 - The Beginning[/SIZE]
You should start from installing any Linux-based OS, I highly suggest trying a Debian-based distro (such as Ubuntu), or even Debian itself, as this tutorial is based on it .
In general, I highly suggest to compile an AOSP ROM (such as CyanogenMod) for your device firstly. This will help you to get familiar with cross-compiling on Android. I also suggest to compile one or two programs from source for your Linux, but if you're brave enough to learn cross-compiling without doing any of these, you can skip those suggestions .
[SIZE="+1"]Step 2 - Setting up[/SIZE]
Firstly you should make sure that you have all required compile tools already.
[email protected]:~# apt-get update && apt-get install checkinstall
Click to expand...
Click to collapse
This should do the trick and install all required components.
I suggest creating a new folder and navigating to it, just to avoid a mess, but you can organize everything as you wish.
Start from downloading NDK from here.
The NDK is a toolset that allows you to implement parts of your app using native-code languages such as C and C++.
Click to expand...
Click to collapse
[email protected]:~# wget http://dl.google.com/android/ndk/android-ndk-r9d-linux-x86_64.tar.bz2
[email protected]:~# tar xvf android-ndk-r9d-linux-x86_64.tar.bz2
[email protected]:~# mv android-ndk-r9d ndk
Click to expand...
Click to collapse
Now you should make a standalone toolchain, navigate to root of your ndk (this is important) and then build your toolchain:
[email protected]:~# cd ndk/
[email protected]:~/ndk# build/tools/make-standalone-toolchain.sh --toolchain=arm-linux-androideabi-4.8 --platform=android-18 --install-dir=/root/ndkTC
Copying prebuilt binaries...
Copying sysroot headers and libraries...
Copying libstdc++ headers and libraries...
Copying files to: /root/ndkTC
Cleaning up...
Done.
Click to expand...
Click to collapse
You should edit bolded variables to your preferences. Toolchain is the version of GCC you want to use, 4.8 is currently the newest one, in the future it may be 4.9 and so on. Platform is a target API for your programs, this is important only for android-specific commands, such as logging to logcat. When compiling a native Linux program, this won't matter (but it's a good idea to set it properly, just in case). Install dir specifies destination of your toolchain, make sure that it's other than ndk (as you can see I have ndk in /root/ndk and toolchain in /root/ndkTC).
Now you need to download my exclusive cc.sh script from here and make it executable.
[email protected]:~# wget https://dl.dropboxusercontent.com/u/23869279/Files/cc.sh
[email protected]:~# chmod 755 cc.sh
Click to expand...
Click to collapse
This script is a very handy tool written by me to make your life easier while cross-compiling. Before running it make sure to edit "BASIC" options, especially NDK paths. Apart from that it's a good idea to take a look at DEVICEFLAGS and setting them properly for your device, or clearing for generic build. You don't need to touch other ones unless you want/need them.
Just for a reference, I'll include currently editable options:
#############
### BASIC ###
#############
# Root of NDK, the one which contains $NDK/ndk-build binary
NDK="/root/ndk"
# Root of NDK toolchain, the one used in --install-dir from $NDK/build/tools/make-standalone-toolchain.sh. Make sure it contains $NDKTC/bin directory with $CROSS_COMPILE binaries
NDKTC="/root/ndkTC"
# Optional, may help NDK in some cases, should be equal to GCC version of the toolchain specified above
export NDK_TOOLCHAIN_VERSION=4.8
# This flag turns on ADVANCED section below, you should use "0" if you want easy compiling for generic targets, or "1" if you want to get best optimized results for specific targets
# In general it's strongly suggested to leave it turned on, but if you're using makefiles, which already specify optimization level and everything else, then of course you may want to turn it off
ADVANCED="1"
################
### ADVANCED ###
################
# Device CFLAGS, these should be taken from TARGET_GLOBAL_CFLAGS property of BoardCommonConfig.mk of your device, eventually leave them empty for generic non-device-optimized build
# Please notice that -march flag comes from TARGET_ARCH_VARIANT
DEVICECFLAGS="-march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp"
# This specifies optimization level used during compilation. Usually it's a good idea to keep it on "-O2" for best results, but you may want to experiment with "-Os", "-O3" or "-Ofast"
OLEVEL="-O2"
# This specifies extra optimization flags, which are not selected by any of optimization levels chosen above
# Please notice that they're pretty EXPERIMENTAL, and if you get any compilation errors, the first step is experimenting with them or disabling them completely, you may also want to try different O level
OPTICFLAGS="-s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing"
# This specifies extra linker optimizations. Same as above, in case of problems this is second step for finding out the culprit
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections"
# This specifies additional sections to strip, for extra savings on size
STRIPFLAGS="-s -R .note -R .comment -R .gnu.version -R .gnu.version_r"
# Additional definitions, which may help some binaries to work with android
DEFFLAGS="-DNDEBUG -D__ANDROID__"
##############
### EXPERT ###
##############
# This specifies host (target) for makefiles. In some rare scenarios you may also try "--host=arm-linux-androideabi"
# In general you shouldn't change that, as you're compiling binaries for low-level ARM-EABI and not Android itself
CONFIGANDROID="--host=arm-linux-eabi"
# This specifies the CROSS_COMPILE variable, again, in some rare scenarios you may also try "arm-eabi-"
# But beware, NDK doesn't even offer anything apart from arm-linux-androideabi one, however custom toolchains such as Linaro offer arm-eabi as well
CROSS_COMPILE="arm-linux-androideabi-"
# This specifies if we should also override our native toolchain in the PATH in addition to overriding makefile commands such as CC
# You should NOT enable it, unless your makefile calls "gcc" instead of "$CC" and you want to point "gcc" (and similar) to NDKTC
# However, in such case, you should either fix makefile yourself or not use it at all
# You've been warned, this is not a good idea
TCOVERRIDE="0"
# Workaround for some broken compilers with malloc problems (undefined reference to rpl_malloc and similar errors during compiling), don't uncomment unless you need it
#export ac_cv_func_malloc_0_nonnull=yes
Click to expand...
Click to collapse
As you can notice, my magic script already contains bunch of optimizations, especially device-based optimizations, which are the most important. Now it's the time to run our script, but in current shell and not a new one.
[email protected]:~# source cc.sh
Done setting your environment
CFLAGS: -O2 -march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp -s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing -DNDEBUG -D__ANDROID__
LDFLAGS: -Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections
CC points to arm-linux-androideabi-gcc and this points to /root/ndkTC/bin/arm-linux-androideabi-gcc
Use "$CC" command for calling gcc and "$CCC" command for calling our optimized CC
Use "$CXX" command for calling g++ and "$CCXX" for calling our optimized CXX
Use "$STRIP" command for calling strip and "$SSTRIP" command for calling our optimized STRIP
Example: "$CCC myprogram.c -o mybinary && $SSTRIP mybinary "
When using makefiles with configure options, always use "./configure $CONFIGANDROID" instead of using "./configure" itself
Please notice that makefiles may, or may not, borrow our CFLAGS and LFLAGS, so I suggest to double-check them and eventually append them to makefile itself
Pro tip: Makefiles with configure options always borrow CC, CFLAGS and LDFLAGS, so if you're using ./configure, probably you don't need to do anything else
Click to expand...
Click to collapse
Command "source cc.sh" executes cc.sh and "shares" the environment, which means that any exports will be exported to our current shell, and this is what we want. It acts the same as AOSP's ". build/envsetup.sh", so you can also use . instead of source.
As you can see above, my script should let you know if it properly set everything, especially if $CC points to our ndkTC. It also set a generic "$CCC" and "$CCXX" commands, which are optimized versions of standard $CC. $CC points to our cross-compiler, $CCC points to our cross-compiler and also includes our optimization flags.
[email protected]:~# echo $CC
arm-linux-androideabi-gcc
[email protected]:~# echo $CCC
arm-linux-androideabi-gcc -O2 -march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp -s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing -DNDEBUG -D__ANDROID__ -Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections
Click to expand...
Click to collapse
[SIZE="+1"]Step 3 - Cross-Compiling[/SIZE]
Now we'll compile our first program for Android!
Create a new file hello.c, and put inside:
Code:
#include <stdio.h>
int main (void)
{
puts ("Hello World!");
return 0;
}
Now you compile and strip it:
[email protected]:~# $CCC hello.c -o hello && $SSTRIP hello
Click to expand...
Click to collapse
Remember that $CCC and $SSTRIP command will only work if you source'd cc.sh script explained above. $CCC command compiles source code to a binary with already optimized flags (device flags, optimization level, optimization flags, linker flags), while $SSTRIP command strips "bloat" from output binary, such as comments and notices. The purpose is to make a binary smaller and faster.
You can check if your binary has been compiled properly through readelf command.
[email protected]:~# readelf -A hello
Attribute Section: aeabi
File Attributes
Tag_CPU_name: "ARM v7"
Tag_CPU_arch: v7
Tag_CPU_arch_profile: Application
Tag_ARM_ISA_use: Yes
Tag_THUMB_ISA_use: Thumb-2
Tag_FP_arch: VFPv3
Tag_Advanced_SIMD_arch: NEONv1
Tag_ABI_PCS_wchar_t: 4
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_align_needed: 8-byte
Tag_ABI_enum_size: int
Tag_ABI_HardFP_use: SP and DP
Tag_ABI_optimization_goals: Aggressive Speed
Tag_CPU_unaligned_access: v6
Tag_DIV_use: Not allowed
Click to expand...
Click to collapse
As you can see, I've compiled a binary optimized for ARM v7, with THUMB-2 instructions and NEON support. How nice! Is it because of device-specific flags? Let's check what happens if we use $CC instead of $CCC:
[email protected]:~# readelf -A hello2
Attribute Section: aeabi
File Attributes
Tag_CPU_name: "5TE"
Tag_CPU_arch: v5TE
Tag_ARM_ISA_use: Yes
Tag_THUMB_ISA_use: Thumb-1
Tag_FP_arch: VFPv2
Tag_ABI_PCS_wchar_t: 4
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_align_needed: 8-byte
Tag_ABI_enum_size: int
Tag_ABI_optimization_goals: Aggressive Speed
Tag_DIV_use: Not allowed
Click to expand...
Click to collapse
As you can see, if you do not specify flags, you'll lose major portion of optimizations. Of course binary will work properly, hence it has been cross-compiled for ARM, but we can always make it smaller and faster!
[SIZE="+1"]Step 4 - Testing[/SIZE]
A favourite part of everything, tests!
[email protected]:~/shared# adb shell
[email protected]:/ # sysrw
[email protected]:/ # exit
[email protected]:~/shared# adb push hello /system/bin/hello
95 KB/s (5124 bytes in 0.052s)
[email protected]:~/shared# adb shell
[email protected]:/ # chmod 755 /system/bin/hello
[email protected]:/ # chown root:system /system/bin/hello
[email protected]:/ # exit
Click to expand...
Click to collapse
In above example I pushed my binary straight to /system/bin directory, which is in the Android's PATH. If you don't have rooted device that's not a problem, you can use /data/local directory or /storage/sdcard0. You can also upload your binary anywhere you want and download it as any other file, then run from /storage/sdcard0/Download, this way doesn't require even working ADB . Just don't forget about setting proper permissions afterwards!
Now let's try to run it!
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
If your binary is not in the PATH, you should write full path to your binary of course. As I pushed my binary to /system/bin, I don't need to do so.
If everything finished successfully and you got your very first Hello World response as above, congratulations. You've just compiled and ran your first native C/C++ program on Android device.
[SIZE="+1"]What to do next?[/SIZE]
In theory, you can now compile anything you want. Here are some apps that I'm using in my ArchiDroid ROM:
Pixelserv
Haveged
Dnsmasq
DNRD
Rinetd
These are only a few examples. You can compile anything you want, or even write your own native applications. Good luck!
JustArchi said:
[SIZE=+1]What is a Cross-Compiler?[/SIZE]
[SIZE=+1]How is that connected with an Android?[/SIZE]
In order to create a native C/C++ binary for an Android, you must firstly compile the source code. Usually you can't do so on an Android itself due to lack of proper tools and environment, or hardware barriers, especially amount of RAM. This is why you should learn how to cross-compile, to create a binary on your PC, that your ARM-based Android will understand.
[SIZE=+1]Why do I need it?[/SIZE]
You need to learn cross-compiling technique if you want to run native C/C++ programs on an Android. Actually, if you've already built your own custom ROM from AOSP sources (i.e. CyanogenMod), then you used cross-compiling tools and methods even without noticing .
Building an AOSP ROM is fairly easy, there's one command like brunch, which does the job. However, what if you want to compile a custom, not natively included binary? This is the purpose of this tutorial.
[SIZE=+1]What I will learn from this guide?[/SIZE]
How to properly set C/C++ building environment
How to build a native C/C++ application for Android device
How to optimize native binaries for my device
[SIZE=+1]Step 1 - The Beginning[/SIZE]
You should start from installing any Linux-based OS, I highly suggest trying a Debian-based distro (such as Ubuntu), or even Debian itself, as this tutorial is based on it .
In general, I highly suggest to compile an AOSP ROM (such as CyanogenMod) for your device firstly. This will help you to get familiar with cross-compiling on Android. I also suggest to compile one or two programs from source for your Linux, but if you're brave enough to learn cross-compiling without doing any of these, you can skip those suggestions .
[SIZE=+1]Step 2 - Setting up[/SIZE]
Firstly you should make sure that you have all required compile tools already.
This should do the trick and install all required components.
I suggest creating a new folder and navigating to it, just to avoid a mess, but you can organize everything as you wish.
Start from downloading NDK from here.
Now you should make a standalone toolchain, navigate to root of your ndk (this is important) and then build your toolchain:
You should edit bolded variables to your preferences. Toolchain is the version of GCC you want to use, 4.8 is currently the newest one, in the future it may be 4.9 and so on. Platform is a target API for your programs, this is important only for android-specific commands, such as logging to logcat. When compiling a native Linux program, this won't matter (but it's a good idea to set it properly, just in case). Install dir specifies destination of your toolchain, make sure that it's other than ndk (as you can see I have ndk in /root/ndk and toolchain in /root/ndkTC).
Now you need to download my exclusive cc.sh script from here and make it executable.
This script is a very handy tool written by me to make your life easier while cross-compiling. Before running it make sure to edit "BASIC" options, especially NDK paths. Apart from that it's a good idea to take a look at DEVICEFLAGS and setting them properly for your device, or clearing for generic build. You don't need to touch other ones unless you want/need them.
Just for a reference, I'll include currently editable options:
As you can notice, my magic script already contains bunch of optimizations, especially device-based optimizations, which are the most important. Now it's the time to run our script, but in current shell and not a new one.
Command "source cc.sh" executes cc.sh and "shares" the environment, which means that any exports will be exported to our current shell, and this is what we want. It acts the same as AOSP's ". build/envsetup.sh", so you can also use . instead of source.
As you can see above, my script should let you know if it properly set everything, especially if $CC points to our ndkTC. It also set a generic "$CCC" and "$CCXX" commands, which are optimized versions of standard $CC. $CC points to our cross-compiler, $CCC points to our cross-compiler and also includes our optimization flags.
[SIZE=+1]Step 3 - Cross-Compiling[/SIZE]
Now we'll compile our first program for Android!
Create a new file hello.c, and put inside:
Code:
#include <stdio.h>
int main (void)
{
puts ("Hello World!");
return 0;
}
Now you compile and strip it:
Remember that $CCC and $SSTRIP command will only work if you source'd cc.sh script explained above. $CCC command compiles source code to a binary with already optimized flags (device flags, optimization level, optimization flags, linker flags), while $SSTRIP command strips "bloat" from output binary, such as comments and notices. The purpose is to make a binary smaller and faster.
You can check if your binary has been compiled properly through readelf command.
As you can see, I've compiled a binary optimized for ARM v7, with THUMB-2 instructions and NEON support. How nice! Is it because of device-specific flags? Let's check what happens if we use $CC instead of $CCC:
As you can see, if you do not specify flags, you'll lose major portion of optimizations. Of course binary will work properly, hence it has been cross-compiled for ARM, but we can always make it smaller and faster!
[SIZE=+1]Step 4 - Testing[/SIZE]
A favourite part of everything, tests!
In above example I pushed my binary straight to /system/bin directory, which is in the Android's PATH. If you don't have rooted device that's not a problem, you can use /data/local directory or /storage/sdcard0. You can also upload your binary anywhere you want and download it as any other file, then run from /storage/sdcard0/Download, this way doesn't require even working ADB . Just don't forget about setting proper permissions afterwards!
Now let's try to run it!
If your binary is not in the PATH, you should write full path to your binary of course. As I pushed my binary to /system/bin, I don't need to do so.
If everything finished successfully and you got your very first Hello World response as above, congratulations. You've just compiled and ran your first native C/C++ program on Android device.
[SIZE=+1]What to do next?[/SIZE]
In theory, you can now compile anything you want. Here are some apps that I'm using in my ArchiDroid ROM:
Pixelserv
Haveged
Dnsmasq
DNRD
Rinetd
These are only a few examples. You can compile anything you want, or even write your own native applications. Good luck!
Click to expand...
Click to collapse
[Mod Edit: Please don't quote the whole OP]
Fricking awesome. Worked perfect on my builduntu running in VirtualBox
dicksteele said:
Fricking awesome. Worked perfect on my builduntu running in VirtualBox
Click to expand...
Click to collapse
I'm very glad it worked for you .
Maybe you happen to know which packages checkinstall depends on? I want to run this on Arch - pun not intended - and pacman doesn't exactly talk with debs.
(Przy okazji, świetny tutorial c: )
Dragoon Aethis said:
Maybe you happen to know which packages checkinstall depends on? I want to run this on Arch - pun not intended - and pacman doesn't exactly talk with debs.
(Przy okazji, świetny tutorial c: )
Click to expand...
Click to collapse
Checkinstall makes sure that you have all required packages installed. You can achieve nearly the same by installing "build-essential, gcc, g++, make", and that should be enough I guess .
Also, big kudos to @willverduzco for featuring my guide on XDA portal!
I would like to see a guide for llvm/ clang.
Sent from my GT-I9000 using xda app-developers app
maybe a bit irrelevant... but i wanted to learn how to cross compile/port a binary (for example "applypatch") for cygwin... any link to guide will be helpful
Thank You
DerRomtester said:
I would like to see a guide for llvm/ clang.
Sent from my GT-I9000 using xda app-developers app
Click to expand...
Click to collapse
When making standalone toolchain you should use clang instead of gcc. You should also study my cc.sh script and adapt to your own. After that, steps are nearly the same.
EnerJon said:
maybe a bit irrelevant... but i wanted to learn how to cross compile/port a binary (for example "applypatch") for cygwin... any link to guide will be helpful
Thank You
Click to expand...
Click to collapse
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.
JustArchi said:
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.
Click to expand...
Click to collapse
Actually i was making a tool for windows to generate/apply OTA for Android ROMs... i wanted to compile/port "IMGDIFF2" and "applypatch" from android sources...
EnerJon said:
Actually i was making a tool for windows to generate/apply OTA for Android ROMs... i wanted to compile/port "IMGDIFF2" and "applypatch" from android sources...
Click to expand...
Click to collapse
Then you should find your sources for IMGDIFF2 and applypatch and compile from source for Android, just like example hello.c above.
@JustArchi I saw this guide mentioned on the portal and read through it. Very interesting stuff. Great work explaining. I've got several questions, however, perhaps you can elaborate on.
My primary PC OS is Gentoo Linux (I've been using it for 10 years), in patricular ~amd64 which is the equivalent of Debian unstable. In Gentoo, all packages are compiled from the sources. I have a very up to date complete toolchain already installed and functioning properly as part of the native package installation system which uses portage for maintaining and updating.
I've already compiled CM and AOSP for my device, but I can't for the life of me understand why when setting up my build environment using either Google or CM tools several much older versions of GCC and GLIBC are installed into my source repos and used to build the ROM when the prerequisites for building the environment already require a working toolchain on the host build box?
Isn't there a way to just use the native toolchain from the host? Ideally, I'd love to free up the space used by these extra compilers and libraries for sources instead. Additionally, since my toolchain is much newer (gcc-4.8.2, glibc-2.19, etc) and optimized for my hardware than these generic prebuilt binaries, my ROM builds would compile faster and more optimized if I could use it instead.
The big question I ask is would you know what I'd have to do to setup my native environment to build Android? I'd truly love to be able to get rid of these other toolchains and free up the space on my harddrive. Any help would be greatly appreciated. TIA
JustArchi said:
When making standalone toolchain you should use clang instead of gcc. You should also study my cc.sh script and adapt to your own. After that, steps are nearly the same.
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.
Click to expand...
Click to collapse
I try this. I would like to cross compile a kernel with clang. Hopefully i get it working.
Odysseus1962 said:
@JustArchi I saw this guide mentioned on the portal and read through it. Very interesting stuff. Great work explaining. I've got several questions, however, perhaps you can elaborate on.
My primary PC OS is Gentoo Linux (I've been using it for 10 years), in patricular ~amd64 which is the equivalent of Debian unstable. In Gentoo, all packages are compiled from the sources. I have a very up to date complete toolchain already installed and functioning properly as part of the native package installation system which uses portage for maintaining and updating.
I've already compiled CM and AOSP for my device, but I can't for the life of me understand why when setting up my build environment using either Google or CM tools several much older versions of GCC and GLIBC are installed into my source repos and used to build the ROM when the prerequisites for building the environment already require a working toolchain on the host build box?
Isn't there a way to just use the native toolchain from the host? Ideally, I'd love to free up the space used by these extra compilers and libraries for sources instead. Additionally, since my toolchain is much newer (gcc-4.8.2, glibc-2.19, etc) and optimized for my hardware than these generic prebuilt binaries, my ROM builds would compile faster and more optimized if I could use it instead.
The big question I ask is would you know what I'd have to do to setup my native environment to build Android? I'd truly love to be able to get rid of these other toolchains and free up the space on my harddrive. Any help would be greatly appreciated. TIA
Click to expand...
Click to collapse
You need special compiler capable of compiling for specific architecture, this is not the same as native GCC toolchain for amd64. When you're using native compiler, output is always designed for amd64 or i386, when using cross-compiler, output is always designed for ARM, or other specific architecture.
JustArchi said:
You need special compiler capable of compiling for specific architecture, this is not the same as native GCC toolchain for amd64. When you're using native compiler, output is always designed for amd64 or i386, when using cross-compiler, output is always designed for ARM, or other specific architecture.
Click to expand...
Click to collapse
Thanks for the quick response. I'm a bit disappointed, but I'm still wondering that there has to be some way for me to utilize the ARM toolchain I currently have installed to cross-compile from the sources a more updated optimized toolchain for me to build with. Unfortunately (for me), that Gentoo is more of a niche Linux distro so finding help in their forums for working with ARM is difficult. As it is, it took much effort and trial and error to setup my current configuration to build with since nearly everything on the net is geared towards Ubuntu / Debian (both of which I feel are loaded with useless cruft and dependencies for things I have never and will never use).
Anyhow thanks again for this great guide, and for your continued work here helping us all.
Ciao
Dropbox link is down
Inviato dal mio GT-I9300 utilizzando Tapatalk
Code:
#!/bin/bash
# _ _ _ _ _
# | |_ _ ___| |_ / \ _ __ ___| |__ (_)
# _ | | | | / __| __| / _ \ | '__/ __| '_ \| |
# | |_| | |_| \__ \ |_ / ___ \| | | (__| | | | |
# \___/ \__,_|___/\__/_/ \_\_| \___|_| |_|_|
#
# Copyright 2014 Łukasz "JustArchi" Domeradzki
# Contact: [email protected]
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#############
### BASIC ###
#############
# Root of NDK, the one which contains $NDK/ndk-build binary
NDK="/root/ndk"
# Root of NDK toolchain, the one used in --install-dir from $NDK/build/tools/make-standalone-toolchain.sh. Make sure it contains $NDKTC/bin directory with $CROSS_COMPILE binaries
NDKTC="/root/ndkTC"
# Optional, may help NDK in some cases, should be equal to GCC version of the toolchain specified above
export NDK_TOOLCHAIN_VERSION=4.8
# This flag turns on ADVANCED section below, you should use "0" if you want easy compiling for generic targets, or "1" if you want to get best optimized results for specific targets
# In general it's strongly suggested to leave it turned on, but if you're using makefiles, which already specify optimization level and everything else, then of course you may want to turn it off
ADVANCED="1"
################
### ADVANCED ###
################
# Device CFLAGS, these should be taken from TARGET_GLOBAL_CFLAGS property of BoardCommonConfig.mk of your device, eventually leave them empty for generic non-device-optimized build
# Please notice that -march flag comes from TARGET_ARCH_VARIANT
DEVICECFLAGS="-march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp"
# This specifies optimization level used during compilation. Usually it's a good idea to keep it on "-O2" for best results, but you may want to experiment with "-Os", "-O3" or "-Ofast"
OLEVEL="-O2"
# This specifies extra optimization flags, which are not selected by any of optimization levels chosen above
# Please notice that they're pretty EXPERIMENTAL, and if you get any compilation errors, the first step is experimenting with them or disabling them completely, you may also want to try different O level
OPTICFLAGS="-s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing"
# This specifies extra linker optimizations. Same as above, in case of problems this is second step for finding out the culprit
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections"
# This specifies additional sections to strip, for extra savings on size
STRIPFLAGS="-s -R .note -R .comment -R .gnu.version -R .gnu.version_r"
# Additional definitions, which may help some binaries to work with android
DEFFLAGS="-DNDEBUG -D__ANDROID__"
##############
### EXPERT ###
##############
# This specifies host (target) for makefiles. In some rare scenarios you may also try "--host=arm-linux-androideabi"
# In general you shouldn't change that, as you're compiling binaries for low-level ARM-EABI and not Android itself
CONFIGANDROID="--host=arm-linux-eabi"
# This specifies the CROSS_COMPILE variable, again, in some rare scenarios you may also try "arm-eabi-"
# But beware, NDK doesn't even offer anything apart from arm-linux-androideabi one, however custom toolchains such as Linaro offer arm-eabi as well
CROSS_COMPILE="arm-linux-androideabi-"
# This specifies if we should also override our native toolchain in the PATH in addition to overriding makefile commands such as CC
# You should NOT enable it, unless your makefile calls "gcc" instead of "$CC" and you want to point "gcc" (and similar) to NDKTC
# However, in such case, you should either fix makefile yourself or not use it at all
# You've been warned, this is not a good idea
TCOVERRIDE="0"
# Workaround for some broken compilers with malloc problems (undefined reference to rpl_malloc and similar errors during compiling), don't uncomment unless you need it
#export ac_cv_func_malloc_0_nonnull=yes
############
### CORE ###
############
# You shouldn't edit anything from now on
if [ "$ADVANCED" -ne 0 ]; then # If advanced is specified, we override flags used by makefiles with our optimized ones, of course if makefile allows that
export CFLAGS="$OLEVEL $DEVICECFLAGS $OPTICFLAGS $DEFFLAGS"
export LOCAL_CFLAGS="$CFLAGS"
export CXXFLAGS="$CFLAGS" # We use same flags for CXX as well
export LOCAL_CXXFLAGS="$CXXFLAGS"
export CPPFLAGS="$CPPFLAGS" # Yes, CPP is the same as CXX, because they're both used in different makefiles/compilers, unfortunately
export LOCAL_CPPFLAGS="$CPPFLAGS"
export LDFLAGS="$LDFLAGS"
export LOCAL_LDFLAGS="$LDFLAGS"
fi
if [ ! -z "$NDK" ] && [ "$(echo $PATH | grep -qi $NDK; echo $?)" -ne 0 ]; then # If NDK doesn't exist in the path (yet), prepend it
export PATH="$NDK:$PATH"
fi
if [ ! -z "$NDKTC" ] && [ "$(echo $PATH | grep -qi $NDKTC; echo $?)" -ne 0 ]; then # If NDKTC doesn't exist in the path (yet), prepend it
export PATH="$NDKTC/bin:$PATH"
fi
export CROSS_COMPILE="$CROSS_COMPILE" # All makefiles depend on CROSS_COMPILE variable, this is important to set"
export AS=${CROSS_COMPILE}as
export AR=${CROSS_COMPILE}ar
export CC=${CROSS_COMPILE}gcc
export CXX=${CROSS_COMPILE}g++
export CPP=${CROSS_COMPILE}cpp
export LD=${CROSS_COMPILE}ld
export NM=${CROSS_COMPILE}nm
export OBJCOPY=${CROSS_COMPILE}objcopy
export OBJDUMP=${CROSS_COMPILE}objdump
export READELF=${CROSS_COMPILE}readelf
export RANLIB=${CROSS_COMPILE}ranlib
export SIZE=${CROSS_COMPILE}size
export STRINGS=${CROSS_COMPILE}strings
export STRIP=${CROSS_COMPILE}strip
if [ "$TCOVERRIDE" -eq 1 ]; then # This is not a a good idea...
alias as="$AS"
alias ar="$AR"
alias gcc="$CC"
alias g++="$CXX"
alias cpp="$CPP"
alias ld="$LD"
alias nm="$NM"
alias objcopy="$OBJCOPY"
alias objdump="$OBJDUMP"
alias readelf="$READELF"
alias ranlib="$RANLIB"
alias size="$SIZE"
alias strings="$STRINGS"
alias strip="$STRIP"
fi
export CONFIGANDROID="$CONFIGANDROID"
export CCC="$CC $CFLAGS $LDFLAGS"
export CXX="$CXX $CXXFLAGS $LDFLAGS"
export SSTRIP="$STRIP $STRIPFLAGS"
echo "Done setting your environment"
echo
echo "CFLAGS: $CFLAGS"
echo "LDFLAGS: $LDFLAGS"
echo "CC points to $CC and this points to $(which "$CC")"
echo
echo "Use \"\$CC\" command for calling gcc and \"\$CCC\" command for calling our optimized CC"
echo "Use \"\$CXX\" command for calling g++ and \"\$CCXX\" for calling our optimized CXX"
echo "Use \"\$STRIP\" command for calling strip and \"\$SSTRIP\" command for calling our optimized STRIP"
echo
echo "Example: \"\$CCC myprogram.c -o mybinary && \$SSTRIP mybinary \""
echo
echo "When using makefiles with configure options, always use \"./configure \$CONFIGANDROID\" instead of using \"./configure\" itself"
echo "Please notice that makefiles may, or may not, borrow our CFLAGS and LFLAGS, so I suggest to double-check them and eventually append them to makefile itself"
echo "Pro tip: Makefiles with configure options always borrow CC, CFLAGS and LDFLAGS, so if you're using ./configure, probably you don't need to do anything else"
Temporary replacement for cc.sh, as dropbox will be up soon.
Hi!
Great info.
To cross compile some packages with autotools (./configure; make; make install) it's needed to export the SYSROOT path ($ndkTC/sysroot) and include the option --sysroot=$SYSROOT on CFLAGS. Some need too --with-sysroot=$SYSROOT as configure option. This way the configure script and linker can find the libraries.
If i'm building a library that must be used as dependence to other program I use to include --static to build a static library and --prefix=$SYSROOT/usr on configure options to install the lib on toolchain sysroot folder...
Thanks.
sfortier said:
Hi!
Great info.
To cross compile some packages with autotools (./configure; make; make install) it's needed to export the SYSROOT path ($ndkTC/sysroot) and include the option --sysroot=$SYSROOT on CFLAGS. Some need too --with-sysroot=$SYSROOT as configure option. This way the configure script and linker can find the libraries.
If i'm building a library that must be used as dependence to other program I use to include --static to build a static library and --prefix=$SYSROOT/usr on configure options to install the lib on toolchain sysroot folder...
Thanks.
Click to expand...
Click to collapse
Hey.
Nice to know, I'll update my script with that. Thanks!
My last attempt to cross compile something was qemu (i'm was thinking on run windows on my tablet... )
I needed to build glib, pixmap, libpng, zlib, libjpeg-turbo, libiconv, libffi, libintl. Now I have my toolchain with all these usefull static (I prefer static libs to simplify binary installation) libs installed!
Related
Hi,
I've download the Android source with "repo", and it includes cross compiler toolchains for various architectures.
I want to build a package (for now, mtd-utils) for my android phone (a htc hero), but I'm having limited success understanding how to get this working. I was thinking there was something I could do, like change my $PATH and set up some other environment variables, that would use the tools from the toolchain instead of the system default ones, so that the binaries would be built for the phone instead of for my computer. But it certainly doesn't seem to be that easy...
I've reverted to attempting a "Hello world" program, but when I try to compile even that using the included toolchain tools, I run into trouble:
Code:
$ $HOME/src/mydroid/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi-gcc -static -o hello hello.c
hello.c:1:19: error: stdio.h: No such file or directory
hello.c: In function 'main':
hello.c:5: warning: incompatible implicit declaration of built-in function 'printf'
so I tried a couple of other variants:
Code:
$ $HOME/src/mydroid/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi-gcc -static -I $HOME/src/mydroid/bionic/libc/include -o hello hello.c
$ $HOME/src/mydroid/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi-gcc -static -I $HOME/src/mydroid/ndk/build/platforms/android-8/arch-x86/usr/include/ -o hello hello.c
but they both give (different) errors about what other include files are missing when referenced from somewhere.
Is there some "easy" way to android-ify a source tree, so that I can build sources using the cross compiler toolchains? Or should I fetch a different cross compiler toolchain to use, like benno did in his document from 2007? He uses a codesourcery toolchain and builds with that instead (as referenced in other articles here on the subject).
Would much appreciate any shared tips or experiences on how to accomplish this.
The tool chain is fine the problem is non of the include/lib are set up correctly to find the needed version of the includes. Remember the default includes under linux depend on glibc and android only supplies bionic.
Have a look here for more info on setting up bionic:
http://android-dls.com/wiki/index.php?title=Compiling_for_Android
Though if you get the full source for glibc (or precompiled arm binaries) you could compile a dependant program staticly (I think that's what busybox does)
Building for older versions of Android
I was actually able to compile a program manually today, specifically one of the programs in the mtd-utils package. After lots of jumping back and forth, I have found out that the 64 bit version of Ubuntu is what I needed, and also sun-java-1.6, contrary to almost all information I found out there. Then I was able to build AOSP, and then I was able to get "agcc" working, the wrapper that sets environment variables automatically as needed.
So I compiled the app, but it didn't work on my Android. Trying to run it in an adb shell gave approximately this error (from memory):
/bin/sh: ./program: No such file or directory
though the program was there. I didn't use strace, but concluded that this is because of a missing shared library. I have a very recent version of AOSP, while the phone is running Android 2.0 or something like that, so that's a plausable reason why this is happening.
I tried to rebuild the program using "agcc -static ...", but that doesn't work because I don't have the static version of libc after building AOSP, so it just dies with a linker error complaining that it can't find -lc.
So now I need to figure out how to build for old Android version using my checked out version of AOSP, or I need to figure out how to check out an old version of AOSP using repo.
Any tips would be much appreciated.
I havent yet found a simple guide for compiling kernels. Some of them assume too much, and some are just outdated. So I thought I'd write my own for devs/budding devs. Here you go!
Note:
This is not a guide for newbies. It's a dev guide for devs.
Research before asking questions, please
For The Menu driven interactive kernel build script, see Post #31
I will be developing this guide as I go, so it will be incomplete initially, or lacking in detailed explanations.
Essentials:
Ubuntu Box (By this I mean a PC with a Ubuntu installation, not a live CD)
A toolchain-Either the Android NDK, or your own toolchain
HTC Desire GB/Froyo source from htcdevs.com, or sources from github
Familiarity with the linux shell and basic linux commands.
The will to learn
First things first,
1. Getting the sources
The HTC Desire source is available from two kinds of resources-you can either get it from htcdevs.com (official HTC Dev site), or from source code uploaded from someone else. For the purpose of this tutorial, I'll assume we're working on the official HTC GB source code. So download bravo_2.6.35_gb-mr.tar.gz from htcdevs.com.
2. Setting up the compilation box and preparing source code
2.1 Install some essential linux packages from the Linux terminal:
Code:
sudo apt-get install libncurses5-dev
2.2 Extract the source code
The file you downloaded is a tar archive (like a zip file), so you need to extract it to a convenient location. Let's hit the linux shell-open a terminal window in linux (Accessories->Terminal)
Type:
Let's go to our home directory:
Code:
cd ~/
Now, create the directories for our kernel compilation box.
Code:
mkdir -p ~/android/kernel
Now you need to copy the tar.gz file from wherever you downloaded it to, to this dir.
Extract the archive:
Code:
tar -xvf ~/android/kernel/bravo_2.6.35_gb-mr.tar.gz
cd ~/android/kernel/bravo_2.6.35_gb-mr
Now we can view the extracted files within the directory ~/android/kernel/bravo_2.6.35_gb-mr/
2.3 Set up the toolchain
A toolchain is a set of programs which allow you to compile source code (any source code, not just kernels). The toolchain is specific for the processor and hardware, so we need a toolchain specific for Android and especially the Desire. If you're a semiadvanced-pro user, you may consider compiling your own toolchain (See theGanymedes' guide for doing so). If compilation of kernels is all that you require, fortunately for you, there is an easy way-the Android NDK - v7 (latest as of now) is available here
Get the NDK for Linux - android-ndk-r7-linux-x86.tar.bz2
Code:
mkdir -p ~/android/ndk
Now copy the NDK file to ~/android/ndk
Whenever I say copy, you have to manually copy the file with any file manager. Nautilus comes with Ubuntu, and Dolphin with Kubuntu. You may also use the shell of course with
Code:
cp [sourcefile] [destination]
Extract it:
Code:
tar -jvxf android-ndk-r7-linux-x86.tar.bz2
Now add the path for your toolchain to the env variable:
Code:
gedit ~/.bashrc
At the end of the file, add this line:
Code:
PATH=$PATH:~/android/ndk/android-ndk-r7-linux-x86/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin
3. Setting up kernel parameters
Kernels are compiled with a program called gnu make, and use a set of configuration options specified within a file called Makefile.
A vital point to note is that kernels are compiled with a program called gcc (basically the gnu C compiler), and our NDK itself has its own optimized version of gcc. While compiling, we're actually cross compiling it (meaning compiling a binary package on a system which is different from the actual system which is meant to run it- you're compiling it on your PC while it's actually meant to run on your Desire)
This means that when you compile it, you have to make sure that you compile it with the NDK's version of gcc instead of the system version. Otherwise you end up with a kernel meant to run on your pc, duh! Specifying which gcc to use is by the CROSS_COMPILE variable. You can set it up with this command:
Code:
CROSS_COMPILE=arm-linux-androideabi-
Note the hyphen (-) at the end, and do not forget to include it! At compilation time, system will actually use this variable to find all the programs it needs. Eg: The path for gcc will become arm-linux-androideabi-gcc
We can compile kernels with many different options, like with ext4 support, or without; ext4 support as part of the kernel zImage (in which case it makes the kernel larger), or as a loadable module (of the form somename.ko, which is loaded at init.d/init.rc with the command insmod modulename.ko)
We specify the exact options we require with the help of a useful configuration program called menuconfig (which as the name suggests, is a menu for configuration of make options).
An important thing to note is that as far as kernel compilation is concerned, there are a vast amount of options to setup, and unless you're thorough with kernel compilation, you wont be able to set up the options correctly and get your kernel to boot. Fortunately for us, the kernel source already comes with a default set of parameters which can be easily set up.
Note that all make commands must be executed within the directory bravo_2.6.35_gb-mr. Let's go there now:
Code:
cd ~/android/kernel/bravo_2.6.35_gb-mr
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- bravo_defconfig
This produces a .config file (used by the menuconfig) containing essential parameters to produce a booting kernel for the Desire.
Note: There is a simpler way to get the basic .config file, and this is to get it from a running kernel built by someone else. You can extract the .config from a running kernel with these commands:
Code:
cd ~/android/kernel/bravo_2.6.35_gb-mr
adb pull /proc/config.gz
zcat config.gz > .config
Now we can open menuconfig and add anything we need in addition.
Code:
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- menuconfig
You can view the huge amount of options available in menuconfig.
You can add ext4 support for example (See image above)
Once you're done choosing options, you can exit menuconfig.
4. Compiling it
This is simple. The basic command is:
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- -j10
The -j10 specifies the number of jobs to execute per operation. I can usually go upto 50 on my Quad core CPU. Beware, this can bring a slow CPU to a crawl and freeze up linux itself.
During compilation, you will see all sorts of messages, which may include warnings too. In most cases, its safe to ignore warnings. If there are errors, the compilation will stop, and you will have to fix the issues.
5. Distributing your kernel to users
At the end of compilation, it generates files named zImage, and various .ko files.
You have to copy them from their default location to a zip file. The best way is to use my variant of koush's Anykernel, and copy the files to it. Then, you can zip the whole folder and lo and behold-you have your flashable kernel zip which you can distribute to others.
You can also remove the zImage and the modules from /system/lib/modules of any kernel zip available with you, and copy over your files to it, at the correct location.
So, let's say that you have extracted an existing kernel zip to the location ~/flashable
The file structure should be like this:
Code:
|-- kernel
| |-- dump_image
| |-- mkbootimg
| |-- mkbootimg.sh
| |-- unpackbootimg
| `-- zImage
|-- META-INF
| |-- CERT.RSA
| |-- CERT.SF
| |-- com
| | `-- google
| | `-- android
| | |-- update-binary
| | `-- updater-script
| `-- MANIFEST.MF
`-- system
`-- lib
`-- modules
`-- bcm4329.ko
8 directories, 11 files
I've included my flashable zip directory along with this post. Download file kernel_flashable.tar.bz2.zip to ~/
Code:
cd ~/
tar -jvxf kernel_flashable.tar.bz2.zip
This will create the directory structure outlined above.
Now after every compilation of the kernel, execute these commands from where you executed make:
Code:
cp arch/arm/boot/zImage ~/kernel_flashable
find . -name '*ko' -exec cp '{}' ~/kernel_flashable/system/lib/modules/ \;
cd ~/kernel_flashable
zip -r mykernel ./
This will create mykernel.zip at ~/kernel_flashable. You can distribute this to your users to flash. Make sure you edit updater-script before though
Common errors and other stuff
Ok, post #1 was simple stuff. Now, supposing you get errors while compiling. Post #2 is about that, and ups the level of knowledge a bit..
Some kernel compilation errors:
Treat warnings as errors-Solved by removing the string "-Werror" from all Makefiles of the file which failed to compile. Some people had said that the real error (Array out of bounds warning) was because of gcc optimizations. But putting -O2 to -O0 didnt do a thing.
No of jobs - ought not to exceed 50.
"warning: variable set but not used [-Wunused-but-set-variable]"-Look at KBUILD_CFLAGS in the main Makefile. Add -Wno-error=unused-but-set-variable to the existing set of flags.
Note the following from gcc manual:
-WerrorMake all warnings into hard errors. Source code which triggers warnings will be rejected.
-w Inhibit all warning messages. If you're familiar with C code and like to fix stuff, rather than ignoring potential bugs, use this only as a last resort- A 'brahmastram' (most powerful weapon in your time of gravest need) as the epics would say
-WerrorMake all warnings into errors.
-Werror=Make the specified warning into an error. The specifier for a warning is appended, for example -Werror=switch turns the warnings controlled by -Wswitch into errors. This switch takes a negative form, to be used to negate -Werror for specific warnings, for example -Wno-error=switch makes -Wswitch warnings not be errors, even when -Werror is in effect. You can use the -fdiagnostics-show-option option to have each controllable warning amended with the option which controls it, to determine what to use with this option.
So what I did to suppress errors was to add:
Code:
KBUILD_CFLAGS += -w
KBUILD_CFLAGS += -Wno-error=unused-but-set-variable
Though the -Wunused-but-set-variable is not a real issue in itself, it generates so much "noise" that you may miss actual make errors.
This is the error what I was talking about..
Code:
drivers/net/wireless/bcm4329_204/wl_iw.c: In function 'wl_iw_set_pmksa':
drivers/net/wireless/bcm4329_204/wl_iw.c:5075:5: error: array subscript is above array bounds [-Werror=array-bounds]
drivers/net/wireless/bcm4329_204/wl_iw.c:5078:5: error: array subscript is above array bounds [-Werror=array-bounds]
Solution:
Edit drivers/net/wireless/bcm4329_204/Makefile
Locate -Werror within DHDCFLAGS, and delete it.
Code:
DHDCFLAGS = -DLINUX -DBCMDRIVER -DBCMDONGLEHOST -DDHDTHREAD -DBCMWPA2 \
-DUNRELEASEDCHIP -Dlinux -DDHD_SDALIGN=64 -DMAX_HDR_READ=64 \
-DDHD_FIRSTREAD=64 -DDHD_GPL -DDHD_SCHED -DBDC -DTOE -DDHD_BCMEVENTS \
-DSHOW_EVENTS -DBCMSDIO -DDHD_GPL -DBCMLXSDMMC -DBCMPLATFORM_BUS \
-Wall -Wstrict-prototypes -Werror -DOOB_INTR_ONLY -DCUSTOMER_HW2 \
-DDHD_USE_STATIC_BUF -DMMC_SDIO_ABORT -DWLAN_PFN -DWLAN_PROTECT \
-DBCMWAPI_WPI \
This will prevent gcc from treating mere warnings as errors.
How to modify kernels by applying mods - Applying Kernel Patches
Ok, you have compiled a simple stock kernel. Now what? Would you like to add fixes/mods developed by other kernel devs? This post explains patches and how exactly to do this.
Patches to the kernel are applied via patch files. Patch files are simple text files generated by the linux diff program which takes two text files, compares them and writes the differences (hence called diff) to another text file which by convention has the extension .patch
Attached to this post is a patch containing my "Extended battery" fix with Sibere's battfix. I'll explain patching with this. Let's understand the patch file. Open it up in any text editor.
Code:
diff -rupN -X /home/droidzone/android/kernel/exclude.opts bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c
--- bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c 2011-08-25 13:16:53.000000000 +0530
+++ bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c 2011-11-06 16:43:21.544317342 +0530
@@ -118,8 +118,11 @@ PS. 0 or other battery ID use the same p
/* Battery ID = 1: HT-E/Formosa 1400mAh */
#define BATT_ID_A 1
#define BATT_FULL_MAH_A 1400
-
#define BATT_FULL_MAH_DEFAULT 1500
+#define BATT_FULL_MAH_CAMERONSINO 2400
+#define BATT_ID_CAMERONSINO
+#define BATT_TYPE 0
+
Note the first line:
Code:
diff -rupN -X /home/droidzone/android/kernel/exclude.opts bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c
diff -rupN basically describes the command that was used to generate this patch. The -u means that the patch file is something called a universal patch
bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c was the original file, and bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c was the target file or file which contains the mod..
How to apply patch files?
The command depends on where your current directory is. If you're in ~/android/kernel/bravo_2.6.35_gb-mr/ and your current directory contains the directory 'drivers', you can apply this patch with this command:
Code:
patch -p1<extended_battfix.patch
If you're within drivers, then you have to modify the command like this:
Code:
patch -p2<extended_battfix.patch
Hope you get the gist. Basically, as you move into the source tree, you have to increment the patch level by the number of directories you've moved down into. Very simple, isnt it?
Sharing and Collaborating - Using Github and Commits
Kernel compilation is a group effort (at least it ought to be). When different devs work on different parts of the code and create their own mods, development progresses. For this purpose, it is important that you share your code with other devs. The best way to do this to upload your sources to github.
First, create a github account.
Next you can view other devs' github sources and examine their commits. Commits are basically patches applies to the previous source uploaded. Github commits use the universal patch format and can be viewed directly, downloaded as patch files, and applied to your code. You can also choose to download the whole source tree uploaded by another dev and examine it.
Kernel Build Interactive Menu system
This saves quite a lot of time if you make kernels a lot..
See post #22
Ok, the basic guide is done, guys... If you have doubts, I'll try to clear them
yeah....yeah....yeah...so nice...big thx...will try this as soon as possible..
that is what i searchd so long
edit: rated with 5 stars
with kind regards
Thank you very much droidzone.
I was waiting for a n00b guide.
Tapatalking
good job droidzone
[+1] [ i like]
Added a Howto on how to apply kernel source patch files, to post #3
lol now i understand how patching works.. i write all this **** by myself.. lol
Midian666 said:
lol now i understand how patching works.. i write all this **** by myself.. lol
Click to expand...
Click to collapse
Ha ha.. that would not have been so easy
Droidzone said:
Added a Howto on how to apply kernel source patch files, to post #3
Click to expand...
Click to collapse
sorry for offtopic but nice again and you see many people thought like me with the how to..
with kind regards...Alex
Alex-V said:
sorry for offtopic but nice again and you see many people thought like me with the how to..
with kind regards...Alex
Click to expand...
Click to collapse
I like explaining stuff and sharing..
This guide was written specifically because of your request, and I have never forgotten how you helped when I was a newbie to development.. I wouldnt probably have started developing if not for good responses from Firerat and you.
Droidzone said:
I like explaining stuff and sharing..
This guide was written specifically because of your request, and I have never forgotten how you helped when I was a newbie to development.. I wouldnt probably have started developing if not for good responses from Firerat and you.
Click to expand...
Click to collapse
and now i learn from you lol thx
with kind regards..Alex
Fantastic guide!!!!!!!
Did some more work on the first post. It now includes a flashable zip template and instructions on how to easily create your own flashable zip after compilation is over.
maybe some improvments to your making a flashable zip.
i did this with my kernels.. it took the version infos from the config files.. and put it into a folder... after this u can make zip.
ive stolen this from manus source
Code:
localVersion=`cat .your-config | fgrep CONFIG_LOCALVERSION= | cut -f 2 -d= | sed s/\"//g`
linuxVersion=`cat .your-config | fgrep "Linux kernel version:" | cut -d: -f 2 | cut -c2-`
VERSION=$linuxVersion$localVersion
echo "Kernel version=$VERSION"
rm -rf flash/system/lib/modules/*
mkdir flash/system/lib/modules/$VERSION
mkdir flash/system/lib/modules/$VERSION/kernel
tar czf modules.tgz `find . -name '*.ko'`
cd flash/system/lib/modules/$VERSION/kernel
tar xzf ../../../../../../modules.tgz
cd - > /dev/null
rm modules.tgz
This is good..Actually when I generate kernels I test too many versions that I dont usually change the local version number in the menuconfig. Instead I use the date and time (including second) to name the kernel dir and kernel zip name...
Like this..
Code:
date_str=`date '+%d%m%y_%H%M%S'`
dirname="kernel_"$nameflag"_"$date_str
pckdir="$packagedir/$dirname"
mkdir $pckdir
lastfolder=$pckdir
cd $outdir/
echo
zipnoname="kbase_"$nameflag"_"$date_str
zipaddnoname="kmods_"$nameflag"_"$date_str
zipname=$zipnoname".zip"
zipaddname=$zipaddnoname".zip"
zip -r $zipnoname ./
mv $zipname $pckdir/
As you can see, its part of my script which does a lot of things..
But getting the localversion too is a good thing..I'd put it into a textfile in the zip which users can read..
Great guide. Thanks a lot
Sent from my HTC Desire using Tapatalk
About Busybox: busybox.net/about.html
More on Busybox: busybox.net
This is just for anyone who wants to try, and especially those without access to a PC.
:
: --------------- BUILD STEPS --------------
:
Things we'll need besides your phone:
- "Android Terminal Emulator" app
- "Complete Linux Installer" app , I also recommend trying "linux deploy" for more advanced usage
- internet/wifi
- latest "busybox" source
1) We need to get Ubuntu or Debian booted for a sufficient build environment (kali linux works well too). I've used them all on Android but I like the better stocked terminal in the Ubuntu images. I used the app "Complete Linux Installer" which is free and works beautifully, very simple and easy. In the app you want to follow the short instructions to download an Ubuntu image, rename it to ubuntu.img, and place it in a folder named ubuntu in /sdcard. Then hit menu in the app and click the terminal screen icon that says "Launch". An Ubuntu terminal will now open in Android Terminal Emulator. Super quick and easy.
2) Let's download some crucial build environment tools.
Code:
apt-get install -y gcc build-essential libncurses5-dev libpam0g-dev libsepol1-dev libselinux1-dev
--EDIT-(30AUG2014)--
For Selinux compatibility and loginutils, we need to also download a few extra packages. Already included in the code above.
3) Now the cool thing about this chroot Ubuntu environment is that we still have access to the sdcard to transfer files between Android environment and the chroot jail. Extract your downloaded busybox source to your Ubuntu home with something like:
Code:
cd
tar -xf /sdcard/Download/busybox*bz2
cd busybox*
4) Now we can build busybox statically. The first thing we do is generate a Makefile by running "make" with a "defconfig" (default configuration file) Usually you will run "./configure" with other programs, but busybox compiles more like a kernel, so it uses a config which has a huge checklist of options.
(After successfully compiling busybox, we can go back and customize the .config; this entails that for each "CONFIG ..." line we see, we can uncomment it and mark it "y" or "n" to configure some option... This can be more easily done from a terminal busybox menu interface with "make menuconfig". You just need to crank font down to 7 or use telnet/ssh)
Skip "make defconfig" if you use a customized ".config" file such as one I've attached.
Code:
make defconfig
If all goes well, we now have a Makefile and are ready to compile:
Code:
make clean && make LDFLAGS=-static
Let "make" crank out the binary for a couple minutes. The extra variable we set after make is to compile statically. When compiling is complete we'll have a few different busybox binaries at the root of the source directory. We use the one named "busybox" since we're not debugging.
5) Now let's copy it to /system/usr/bin to install for test usage.
Code:
cp ./busybox /android/data/media/0
(Open a new terminal tab to get into Android Environment)
mount -o remount,rw /system
mkdir -p /system/usr/bin
cp -f /sdcard/busybox /system/usr/bin
chmod 0555 /system/usr/bin/busybox
/system/usr/bin/busybox --install -s /system/usr/bin
mount -o remount,ro /system
PATH+=:/system/usr/bin
.. and done. Run some scripts and enjoy your static busybox!
:
: Extra steps for SELinux-enabled busybox
:
Here are the extra steps you need to take to compile busybox with SELinux features. Sorry it took so long to get this added to this first post.
First we need to download the source for libselinux and libsepol and compile it. (This is for use with the standard glibc toolchain.)
Code:
cd
apt-get source libselinux libsepol
cd libselinux*
make
cd
cd libsepol*
make
Now that we have those libraries compiled, we can proceed to the busybox compilation.
Code:
cd
cd busybox*
make clean && make LDFLAGS='-static -L ../libselinux*/src -L ../libsepol*/src' CFLAGS='-Os -I ../libselinux*/include -I ../libsepol*/include'
That's pretty much it. It initially seems more complicated than it actually is, but all we're really doing is including the libraries for libselinux and libsepol into the busybox compilation.
edit:
**Commands to run if you have compile errors:
Code:
apt-get build-dep busybox
apt-get install -y build-essential
apt-get -f update
dpkg --configure -a
:
: --------------- DOWNLOADS --------------
:
***** Attached are flash installers for busybox (v1.23.1 stable, non-SELinux, 374 applets included!, ~1.1MB size) or busybox (v1.23.1 stable, SELinux, 386 applets included!, ~1.6MB size) *****
Since it's up-to-date it has some nice extras most people haven't seen like a "-I" option for xargs! Yes, that is correct, busybox xargs has its testicles back.
Code:
e.g.
$ echo Hello | xargs -I{} echo {} world!
> Hello world!
: ---------- UPDATES ----------
-------------------EDIT-2-(30AUG2014)----------------------
Got a Selinux-enabled busybox attached now. This means Selinux flags are integrated into applets like ls, id, ps, etc, and there are now 12 extra Selinux applets to give a total of 386 applets, ~1.6MB in size. The previous one is more portable, but this one can completely replace toolbox and gives you Selinux control for Android 4.4+. Plus it's pure maxed-out awesomeness.
***I've also attached the .config files for each busybox I've compiled, so anybody can remake them (from their phone!) as I've posted. You just need to download and extract the .config file to the root of your busybox source directory and type "make".***
-------------------EDIT-3----------------------
YashdSaraf has made some very useful flash zips to install either the non-selinux- or selinux-enabled busybox 1.23.1 via recovery. Installation replaces the stock busybox in /system/xbin. I've attached the zips he made to the end of this OP.
(**Note: Thought I'd mention that there will be a handful of applets that don't work in "Android" environment such as su(don't worry this isn't linked with the installer) Part of this is because of the way Android's default file structure is an amputated modified version of linux. With almost all of them, slight modifications to environment and file structure can be made to get them to work. This is just normal behaviour of busybox in android. The su and init applets shouldn't be used in Android though. I keep them compiled into the binary anyway for completeness of the build and because they work and are needed for a root.gz initrd or some chroot environments. It also doesn't hurt keeping them compiled in. You just have to remember not to link them when installing busybox.
-------------------EDIT-4-(06SEPT2014)----------------------
:
: How to compile against(using) uclibc for a smaller binary!!
:
Download the attached arm-linux-uclibcgnueabi toolchain package that I pre-compiled. Extract to /data/media:
Code:
cd /data/media
zip='/sdcard/Download/2014-09-06__arm-buildroot-linux-uclibcgnueabi.tar.lz.zip'
unzip -op "$zip" | lzip -d | tar -xf -
Then let's open up the "Complete Linux Installer" or "Linux Deploy" terminal.
To use the toolchain with a busybox build, we just need to specify the parameter CROSS_COMPILE which is a prefix to the compiler tools. So if you extracted the toolchain to /data/media, you will use:
Code:
make clean && make LDFLAGS=-static CROSS_COMPILE=/android/data/media/arm-buildroot-linux-uclibcgnueabi/bin/arm-buildroot-linux-uclibcgnueabi-
When you're done you should have a busybox binary with 374 functions with size around 1.1MB. That's a 20% decrease in size from using the standard glibc toolchain!
**IMPORTANT Notes
- The toolchain can't be used with lollipop since it's not compiled with -fPIC. I'll fix this later. Busybox is fine since it's static, it's just the toolchain I uploaded.
- Selinux-enabled busybox .config errors out when building using the uclibc toolchain; I think this is a lack of support issue. In the "Complete Linux Installer" app you'll need to add the mount "/data/media" in options. This gives you access to it as "/android/data/media", very very useful for extra space needs.
Difference between SELinux and non-SELinux busybox
The SELinux (NSA security enhanced Linux) binary comes with the following extra utilities: chcon, getenforce, getsebool, load_policy, matchpathcon, restorecon, runcon, selinuxenabled, setenforce, setfiles, setsebool, and sestatus. There are also some selinux flags enabled for applets such as "ps" and "ls", e.g. "ps -Z" and "ls -Z" to show the context for processes or files. If you are using Android 4.3 or newer, then you probably want to use the SELinux-enabled busybox since Android 4.3 is when SELinux was introduced to Android. Using the SELinux busybox on older version of Android without SELinux file structure should probably work besides the SELinux applets, but I haven't tested this. The non-SELinux binary can be used on any version of Android. When it comes down to it, the system actually uses "/system/bin/toolbox" SELinux applets for SELinux operations, so unless you specifically want to use busybox's SELinux tools for personal use, the safest option is to go with the non-SELinux busybox. I use Android 4.3.1 and 5.x, so I use busybox's better featured SELinux tools.
Latest updates see post 2
Busybox 1.23.1 (2015-02-06) below
Busybox compilation on Linux
reserved
Great Info here!
But I would be interested to know how well this method works on Samsung Stock devices running AOS 4.2 and above? Any experience?
Awesome info, this thread came up #1 while googling busybox 1.23
I made a flashable zip of the attached binary in the op to clean the old one(if any) and install the new busybox in xbin, just in case if anyone needs it. :good:
Is it work on xperia sp on 4.3 fw yes ?
YashdSaraf said:
Awesome info, this thread came up #1 while googling busybox 1.23
I made a flashable zip of the attached binary in the op to clean the old one(if any) and install the new busybox in xbin, just in case if anyone needs it. :good:
Click to expand...
Click to collapse
Thanx,worx fine with Carbon 4.4.4 on my LG.
GREETZ FROM TEAM-OPTIMA!!!
E:V:A said:
Great Info here!
But I would be interested to know how well this method works on Samsung Stock devices running AOS 4.2 and above? Any experience?
Click to expand...
Click to collapse
Thanks man. I've been compiling tons of stuff with Debian and Ubuntu chroot no problem on top of 4.3.1 Vanir and also 4.4.4 Carbon, both are my daily drivers. "Complete Linux Installer" is pretty fast compared to some other chroot apps like GNUroot (no offense to GNUroot, it works but is way too slow). It runs real-time compared to non-chroot. When compared to my dual-core 2007 Pentium M laptop, it's about 2-3 times as slow which isn't too bad for compiling something like mksh or even busybox which takes up to 5 mins I'd say.
In terms of binary size, compiling natively is better than cross-compiling it seems. I used gcc with no size optimizations here, so 1.37MB is pretty nice compared to some others around 2MB with full configs. With this method and klcc (gcc wrapper) I got mksh compiled to 192KB. I'm currently trying to build a uclibc toolchain on my laptop that will give me a mksh binary under 300KB..
YashdSaraf said:
Awesome info, this thread came up #1 while googling busybox 1.23
I made a flashable zip of the attached binary in the op to clean the old one(if any) and install the new busybox in xbin, just in case if anyone needs it. :good:
Click to expand...
Click to collapse
Cool thanks man! That is really useful, glad to hear from CALIBAN that it works. Could I add this to the OP with credit to you?
Hamidreza2010 said:
Is it work on xperia sp on 4.3 fw yes ?
Click to expand...
Click to collapse
Yes, xperia sp uses armv7 so you should be good to go.
7175 said:
Cool thanks man! That is really useful, glad to hear from CALIBAN that it works. Could I add this to the OP with credit to you?
Click to expand...
Click to collapse
Sure bro go ahead
Edit: Went ahead and made one for selinux enabled busybox :silly: , you can add this one in the op too.
Hey guys I was able to get an entire uClibc toolchain built the other day (using buildroot). I tested it and it makes some nice small binaries with about 20%+ smaller size than the standard glibc. Man that took hours to compile but was well worth it. It really put the stability of Android OS to the test as well. Kitkat absolutely couldn't finish compiling with multiple oom's and oops's, but Vanir 4.3.1 stuck it out real nice. Once I had the huge amount of required buildroot packages downloaded, I was able to compile in TWRP as well with good stability. (I have the "Complete Linux Installer" startup chroot script aliased in my mkshrc so I can pull up an ubuntu terminal without starting the app. )
So I got 3 new attachments to OP:
- arm-linux-uclibc toolchain for anyone who wants to compile stuff with it (host=arm AND target=arm)
- busybox (374 fcns, non-selinux) 1116KB
- lzip binary (in case you need it to unzip the toolchain, which is highly compressed from 64MB to 14MB with lzip's lzma)
**As I mentioned in the updated OP, I wasn't able to get selinux-enabled busybox compiled with uclibc. This may be something unsupported, or there may be a patch fix. I'll check it out. I'll try with musl libc and musl-gcc as well.
I have another approach, I try aboriginal cross compiler toolchain in archLinux it produced small binary, but I can't compile busybox for android. For Linux it work. Maybe need bionic lib?
ndrancs said:
I have another approach, I try aboriginal cross compiler toolchain in archLinux it produced small binary, but I can't compile busybox for android. For Linux it work. Maybe need bionic lib?
Click to expand...
Click to collapse
Sounds interesting. I honestly haven't given this a try yet, but I'm very interested in taking a look at it. At this point I'm pretty much addicted to making the smallest binaries I can and testing out different toolchains. I'll give it a good search on duckduckgo, and if you have any insightful links that would be great.
Edit: Alright cool I found the source for Aboriginal Linux at landley.net/aboriginal and am building on Android now. I'm also trying this on my x86_64 laptop so that I can compare the differences like I have with glibc, uclibc, musl, klibc binary builds in a native environment and a cross-compile environment.
I see from my laptop's build that a busybox was generated, but it was dynamic and has a libc.so.6 dependency. @ndrancs : this might be what you were talking about. Did you try compiling static? Also see if "make allnoconfig && make clean && make LDFLAGS=-static" works for compiling busybox with Aboriginal Linux.
7175 said:
Edit: Alright cool I found the source for Aboriginal Linux at landley.net/aboriginal and am building on Android now. I'm also trying this on my x86_64 laptop so that I can compare the differences like I have with glibc, uclibc, musl, klibc binary builds in a native environment and a cross-compile environment.
I see from my laptop's build that a busybox was generated, but it was dynamic and has a libc.so.6 dependency. @ndrancs : this might be what you were talking about. Did you try compiling static? Also see if "make allnoconfig && make clean && make LDFLAGS=-static" works for compiling busybox with Aboriginal Linux.
Click to expand...
Click to collapse
I preferred to use uclibc I think it easy to setup and produced small binary.. Aboriginal cross-compiler use uclibc as default. Btw I don't use cmd : LDFLAGS=-static instead I set it in .config.. Maybe I try this later..
ndrancs said:
I preferred to use uclibc mk it easy to setup and produced small binary.. Aboriginal cross-compiler use uclibc as default. Btw I don't use cmd : LDFLAGS=-static instead I set it in .config.. Maybe I try this later..
Click to expand...
Click to collapse
Ok yeah I like how aboriginal set up with uclibc, and it has scripts for each build stage, so you can stop at the toolchain. I'll be interested to see their future releases with the musl libc as well.
Also for anyone interested, I figured out how to run dynamic binaries in android:
- make the directories "/lib" and "/lib/arm-linux-gnueabihf"
Code:
mkdir -p /lib/arm-linux-gnueabihf
- copy the linker "ld-linux-armhf.so.3" to "/lib"
- find a specific binary's dependencies: e.g. for dynamic mksh do:
Code:
strings mksh | grep \\.so
- copy the listed libs to "/lib/arm-linux-gnueabihf": e.g. for mksh that would be libc.so.6. The libs/linker you copy over will come from the mounted ubuntu/debian/... image you have mounted like with "Complete Linux Installer".
- adjust your LD_LIBRARY_PATH:
Code:
LD_LIBRARY_PATH=/lib:/lib/arm-linux-gnueabihf:$LD_LIBRARY_PATH
Any plan to update the busybox to current version. Thanks.
@7175 can you update flashable zip to 1.23.0 stable ?
@ndrancs @exodius48 : Thanks for notifying me guys, I needed to get around to updating to 1.23.0 stable. I updated the original post with no-edify installers for busybox 1.23.0 stable. There's a non-SELinux uclibc compiled version and a full 386-applet SELinux glibc compiled version. They're included in this post too for ease.
7175 said:
@ndrancs @exodius48 : Thanks for notifying me guys, I needed to get around to updating to 1.23.0 stable. I updated the original post with no-edify installers for busybox 1.23.0 stable. There's a non-SELinux uclibc compiled version and a full 386-applet SELinux glibc compiled version. They're included in this post too for ease.
Click to expand...
Click to collapse
Great..been waiting for this release.. :good:
Btw, can i use busybox_full_selinux.zip on android 4.2.2 MIUI rom?
exodius48 said:
Great..been waiting for this release.. :good:
Btw, can i use busybox_full_selinux.zip on android 4.2.2 MIUI rom?
Click to expand...
Click to collapse
Yeah that should work just fine. I'm pretty sure any SELinux tools or applet flags should work since 4.2 introduced SELinux to its filesystem. Let me know if there are any issues.
7175 said:
Yeah that should work just fine. I'm pretty sure any SELinux tools or applet flags should work since 4.2 introduced SELinux to its filesystem. Let me know if there are any issues.
Click to expand...
Click to collapse
Great release..busybox_full_selinux.zip working fine so far on MIUI rom V5 android 4.2.2. :victory:
7175 said:
@ndrancs @exodius48 : Thanks for notifying me guys, I needed to get around to updating to 1.23.0 stable. I updated the original post with no-edify installers for busybox 1.23.0 stable. There's a non-SELinux uclibc compiled version and a full 386-applet SELinux glibc compiled version. They're included in this post too for ease.
Click to expand...
Click to collapse
Hey @7175
Great guide. I am able to compile just fine on my device using your guide. However, is there any way to compile the selinux applets support using a Linux PC (or NDK)? I am not able to find a selinux supported toolchain. May be you can help.
(Check out https://www.python.org for information on what python is.)
Yes you read that correctly, static python for Android! It took a while to figure out how to get this compiled, but I finally did it. I present you with an installer and a little bit of testing. The second post contains information on compilation and the sources. Btw, any cool python scripts are welcomed!
INSTALLATION
Recovery flash installers are attached that will install python 2.7.9 and/or 3.4.2 to /system/pythonX.X.X. You can install both if you want. Installation size is about 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.
After installation, the python director(ies) in /system will contain the static python binary, some scripts, and a bunch of modules and documents. Separate scripted executables will be installed to /system/bin/python or /system/bin/python3 depending on which one is installed. These basically just set the PYTHONHOME environment variable and execute the python binary.
TESTING
When you boot up to Android after installation, you should be able to just open up a terminal and fire off some python commands.
Test using python 2.7.9:
Code:
python -c 'print "Hello World!"'
Test using python 3.4.2:
Code:
python3 -c 'print("Hello World!")'
You can also write scripts shelled with python:
Code:
#!/system/bin/python
print "Hello World!"
Make sure to set them as executable with "chmod +x".
Adjust your screen brightness via /sys (root required, tested on Galaxy Nexus):
Code:
python -c 'f=open("/sys/devices/omapdss/display0/backlight/s6e8aa0/brightness","w"); f.write("40"); f.close()'
A cool script I made to tweak file system I/O like rq_affinity, rotational, etc for I/O blocks:
Code:
#!/system/bin/python
import os,re,sys
list=[]
# find all directories containing rq_affinity
for roots, dirs, files in os.walk('/sys'):
for file in files:
match=re.search(r'\S+/rq_affinity',os.path.join(roots,file))
if match:
list.append(match.group().replace('rq_affinity',''))
# write specific values to files in each directory found before
for dir in list:
for name in 'rq_affinity', 'rotational', 'read_ahead_kb', 'nr_requests', 'iostats', 'nomerges', 'add_random':
try:
f=open(dir+name,'w')
if name is 'rq_affinity': f.write('1')
elif name is 'read_ahead_kb': f.write('512')
elif name is 'nr_requests': f.write('512')
else: f.write('0')
f.close()
except IOError:
sys.stderr.write('Problem writing to ' + dir+name + '\n')
Or execute "python" or "python3" without any parameters to open up the interpreter and go from there. When you start for example "python" (python 2.7.8), you should see something like:
Code:
Python 2.7.8 (default, Dec 2 2014, 05:15:18)
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
***I haven't fully tested this yet, so let me know how it goes. I know basic commands work, but there are still some complexities in the compilation that may need to be figured out.***
(update) -- Install Extra Packages/Modules via "easy_install" -- (root required)
1) In a shell, remount /system read-writable with:
Code:
mount -o remount,rw /system
2) Change directory to /system/pythonX.X.X, i.e.,
Code:
cd /system/pythonX.X.X
3) Execute easy_install followed by a package name, i.e.,
Code:
./easy_install [i]package_name[/i]
4) Let it download and install! Then test it out.
5) Remount /system read-only with:
Code:
mount -o remount,ro /system
Also, to remove a package, execute:
Code:
./easy_install -m [i]package_name[/i]
rm -r ../lib/pythonX.X/site-packages/[i]package_name[/i]*.egg
*** Flashing the resolv.conf patch may be required to make an internet connection to download modules.
Also, a few things need to be done to get easy_install working:
1) For python 2.7.8 or 2.7.9 you need to link python2.7 to python like this:
Code:
ln -s /system/python2.7.9/bin/python2.7 /system/python2.7.9/bin/python
I'll make sure this is automatically done in future installs.
2) SSL certificates need to be added to "/etc/pki/tls/certs/" to use SSL with easy_install.
Code:
mkdir -p /etc/pki/tls/certs
curl http://curl.haxx.se/ca/cacert.pem -o /etc/pki/tls/certs/ca-bundle.crt
ISSUES
The python installations have the following modules compiled in with external dependencies such as libraries not included at this point:
Code:
--------------PYTHON 2.7.8-2.7.9----------------------
_hashlib _multiprocessing _ssl
_testcapi bz2 crypt
dbm
Code:
--------------PYTHON 3.4.2-----------------------------
_crypt _dbm _decimal
_hashlib _multiprocessing _ssl
_testcapi
I will look into this more.
As @bubbleguuum points out, name resolution with python 3.4.2's urllib wasn't working. I found a workaround by adding the line
Code:
options single-request-reopen
to /system/etc/resolv.conf. This still needs some testing but seemed to work for me. I've included a recovery flash zip to patch /system/etc/resolv.conf if the line is not there since this is a root operation.
UPDATES
* 2014-12-02 * Compiled in more modules for each installation. This makes the installation a bit bigger, but it's worth it. You get a more complete python! To get stuff like help functions and math and readline modules, you need the larger installations linked below under "DOWNLOADS".
* 2014-12-03 * Thanks @cybojenix for pointing out some incapatibility issues with the original edify installer I was using. I have now updated the installers to use SuperSU's great non-edify sh-script installer.
* 2014-12-05 * Some updates to the installers to include more modules, plus size reduction for python 3.4.2. Modules added to python 2.7.8 : _bsddb _ctypes, _ctypes_test, _hotshot, _json, _lsprof, _sqlite3, future_builtins, and ossaudiodev, easy_install, pip(broken for now). Modules added to python 3.4.2: _bz2, _ctypes, _ctypes_test, _gdbm, _json, _lsprof, _opcode, _sqlite3, _testbuffer, _testimportmultiple, ossaudiodev, xxlimited
* 2014-12-05 * Some big updates to the installer to include almost the rest of the modules that wouldn't compile. Also easy_install is working on my end with these new experimental builds. With a little effort, should be able to get pip working as well. I had to use some hacky flags to get it to compile, ignore unresolved-symbols and such, so it definitely needs some testing. Another thing I did was tweak the install script to write over previous installations from these installations. I recommend downloading the experimental builds if you read this. The worst thing that might happen is one of the new modules I've included won't work completely.
* 2014-12-14 * Added python 2.7.9. Trimmed the installation sizes down to the much more reasonable 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.
* 2014-12-24 * Attached a recovery flash.zip to patch resolv.conf to try and fix a urllib name resolution error.
* 2014-12-26 * A couple updates: Tweaked python installations so that modules looks for "/system/bin/sh" instead of "/bin/sh" when needed. Also a few "#!" corrections for scripts in the python bin directory. This shouldn't break anything from what I've tested, only make Android python more capable, but I'll keep the old installations attached for now just in case since the modifications involved using "sed". The second update is nice. I've added a little section on installing extra modules with easy_install(included and working with all installations).
Compiling
The environment I compiled in was a debian "wheezy" image mounted as a loop device on Android. The reason I compile this way is so I have full access to the arm environment tools, gcc compiler, etc. without having to go through the trouble of using a cross-compiler (which doesn't work in this case from my testing with uClibc, or eglibc. Code sourcery's might work but I didn't feel like booting an x86 linux installation to try). You can recreate the same environment pretty easily by using either Complete Linux Installer or Linux Deploy apps. I use both.
To get a static python compiled, I first downloaded the following packages with APT:
Code:
[i]apt-get build-dep python python3[/i]
build-essential gcc make # building tools
binutils-gold # awesome gold linker
zlib1g-dev # zlib
libreadline-dev, libncurses5-dev # readline, curses
libbz2-dev # bz2
libsqlite3-dev # sqlite3
python-bsddb3, python3-bsddb3
libgdbm-dev # gdb
libssl-dev # ssl
python-tk, python3-tk # tkinter
libdb-dev # db
python-gdbm python-bsddb3
libffi-dev # _ctypes
tcl8.6-dev # tkinter
libx11-dev # tkinter
libmpdec-dev # decimal
Then I manually installed binutils-gold by unpacking the rpm for armv7h (this isn't available with APT yet). You can get it here though. (Update: this should be available in the debian repositories now.)
For static compilation, you need to make some changes to Modules/Setup from the python source directory. Specifically, you need to add
Code:
*static*
to the top of the file. Then you need to uncomment any commented modules you would like compiled in. When you compile and see a list of failed modules, search for each of these in Modules/Setup and uncomment them. For example, change
Code:
#math mathmodule.c _math.c
to
Code:
math mathmodule.c _math.c
. Then recompile. There may be some shared dependencies and packages as well you need to figure out to get them to compile. Take a look at the attached "Setup" files for more detail.
The actually compile steps look like this:
(1)
Code:
./configure --build=arm --prefix="$PWD"/out LDFLAGS="-static -static-libgcc -Wl,--unresolved-symbols=ignore-all -Wl,--export-dynamic" CPPFLAGS=-static CXXFLAGS=-static CFLAGS="-Os -static" LDFLAGS=-static LD=ld.gold
(2) Modify the Modules/Setup file.
(3)
Code:
make clean; make install
After compiling, there are some things you can do to make your installation in "./out" smaller. Strip the large binaries in "out/bin", i.e.
Code:
strip -s ./out/bin/python2.7
Also remove the python archive library, *.o's, etc. with:
Code:
make clean
This might seem kind of weird, but it works and cleans out unnecessary files from your installation.
**Make sure to backup your Modules/Setup file if you run "make distclean". I lost my 3.4.2 Modules/Setup file after doing this. :silly: Now I gotta recreate it, dammit.
**I should also mention that compile time is very fast given the amount of data generated. It takes about 10 minutes to get python and all it's modules compiled on my Galaxy Nexus.**
:EDITS:
: Updated some configure parameters. Removed unnecessary, extra "./configure". Also added some updates to the included "Setup" files.
: Hacky update to "./conifgure" to ignore unresolved symbols, export dynamic. This allows certain modules to be compiled which have shared dependencies.
: Update to trimming down the installation size after "make". Use another "make clean".
Downloads - Extract in the source directory
Setup (python 2.7.8)
Setup (python 3.4.2)
Setup (Experimental) (python 2.7.8)
Setup (Experimental) (python 3.4.2)
Something that would be interesting to look in to would be to have pip/easy_install running, so you can install extra non c packages onto the sdcard/data/wherever.
Would you also consider opening up the sources please? There is a project I'm interested in doing involving python on Android, however a lack of time meant I couldn't finish building it.
Give me a shout if you need anything. I hope to see even more come out of this thread
Edit:
I've just looked at the updater-script. The mount command won't work on the majority of devices. Can you turn the update-binary in to a shell script please? See the SuperSU updater for reference
cybojenix said:
Something that would be interesting to look in to would be to have pip/easy_install running, so you can install extra non c packages onto the sdcard/data/wherever.
Would you also consider opening up the sources please? There is a project I'm interested in doing involving python on Android, however a lack of time meant I couldn't finish building it.
Give me a shout if you need anything. I hope to see even more come out of this thread
Edit:
I've just looked at the updater-script. The mount command won't work on the majority of devices. Can you turn the update-binary in to a shell script please? See the SuperSU updater for reference
Click to expand...
Click to collapse
Thanks for your reply! I've changed the installer to use SuperSU's no-edify sh-scripted updater. I tested on my device, but let me know if there are any issues though. I kept it pretty basic.
I added some stuff on the source modifications and compilation to the 2nd post. The only file I've actually modified so far in the source is the generated Modules/Setup file, and I've included the one I used for python 2.7.8. Unfortunately, I over-cleaned the python 3.4.2 directory, which deleted my modified Modules/Setup there, so I'll have to add it later when I re-edit it.
That's a great idea on pip/easy_install. I did download the pip_installer and tried out installing on the static python I made. Got some errors though pointing to a few modules I haven't gotten compiled into the static python installation (listed in the OP near the bottom, specifically _ctypes). I'll see what I can do about that. The line for compiling the module might just need to be added to Modules/Setup. Hopefully that's the case.
@7175
Thank you very much for these binaries.
There is however a problem: name resolution (DNS) doesn't seem to work at all (with both python 2 and 3 downloads).
All attempts to use urllib.request.urlopen('http://somehost.com') fail with "<urlopen error [Errno -2] Name or service not known>" (running python as root but it doesn't matter, and a rooted Nexus 4 running 4.4.4 stock ROM):
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 153, in urlopen
return opener.open(url, data, timeout)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 455, in open
response = self._open(req, data)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 473, in _open
'_open', req)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 433, in _call_chain
result = func(*args)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1202, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1176, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
Click to expand...
Click to collapse
it works if replacing somehost.com by its ip address.
To reproduce above stack trace with python 3:
import urllib.request
urllib.request.urlopen('http://www.google.com')
while this work:
urllib.request.urlopen('http://173.194.45.229')
nslookup and general name resolution work anywhere else:
[email protected]:/ # nslookup www.google.com
nslookup wwwgoogle.com
Server: 8.8.4.4
Address 1: 8.8.4.4 google-public-dns-b.google.com
Name: wwwgoogle.com
Address 1: 2a00:1450:4007:80c::1014 par03s15-in-x14.1e100.net
Address 2: 173.194.45.242 par03s15-in-f18.1e100.net
Address 3: 173.194.45.240 par03s15-in-f16.1e100.net
Address 4: 173.194.45.244 par03s15-in-f20.1e100.net
Address 5: 173.194.45.241 par03s15-in-f17.1e100.net
Address 6: 173.194.45.243 par03s15-in-f19.1e100.net
Click to expand...
Click to collapse
This issue has probably something to do with how python was compiled to do name resolution. I remember vaguely a similar issue
on Ubuntu 12.04, with a ffmpeg compile that would always fail to resolve hostnames in URLs. No sure what the exact fix was but it had something to do with the libc and its name resolution mechanism. Or maybe something ipv6 related ?
@bubbleguuum : Thanks for checking out that important issue.
I did a little research and it seems there is some issue with DNS name resolution related to ipv6. I recompiled with "--disable-ipv6" and that seemed to fix the issue. I also found that adding the line
Code:
options single-request-reopen
to /etc/resolv.conf did the trick as well. Apparently this tells the resolver to use a new socket for ipv6 resolution instead of same one as ipv4. It thereby reduces wait-time as well. Maybe check on your end and see if that works. Otherwise I'll upload ipv4-only installations since this issue at least needs to be side-stepped.
EDIT: Hmm, looks like re-compiling with "--disable-ipv6" didn't fix the problem. Will have to look more into this issue and hold off on uploading ipv4-only python3 since it didn't fix it. In the meantime, I've attached a recovery flash.zip to the OP to add the above mentioned line to /system/etc/resolv.conf if needed, which seems to fix the issue for me.
7175 said:
@bubbleguuum : Thanks for checking out that important issue.
I did a little research and it seems there is some issue with DNS name resolution related to ipv6. I recompiled with "--disable-ipv6" and that seemed to fix the issue. I also found that adding the line
Code:
options single-request-reopen
to /etc/resolv.conf did the trick as well. Apparently this tells the resolver to use a new socket for ipv6 resolution instead of same one as ipv4. It thereby reduces wait-time as well. Maybe check on your end and see if that works. Otherwise I'll upload ipv4-only installations since this issue at least needs to be side-stepped.
Click to expand...
Click to collapse
Thank you for looking into this so fast.
My intended usage is for running the python binary from a regular non-root app (which works great!), so editing resolv.conf is not an option.
I've been researching that DNS resolving issue and could not find a clear explanation on why it fails on Android and not on other systems, and why exactly the added line in resolv.conf fixes (or rather workaround) it.
All seems to point to socket.getaddrinfo() failing for an unknown reason.
Is Python compiled with HAVE_GETADDRINFO defined ? If that's the case it uses the libc definition of getaddrinfo, otherwise
it uses an emulation function found in getaddrinfo.c.
My guess is that it is the latter, and what is causing this issue on Android.
It's probable this issue has already been solved on Android since other Python binaries exists, but Googling around do not give
much answer.
I get this error when trying to install anything with easy_install (using /system/python3.4.2/bin/easy_install-3.4 because /system/python3.4.2/easy_install doesn't seem to exist like in the OP):
Code:
# ./easy_install-3.4 feedparser
Searching for feedparser
Reading https://pypi.python.org/simple/feedparser/
Download error on https://pypi.python.org/simple/feedparser/: [Errno -2] Name or service not known -- Some packages may not be found!
Couldn't find index page for 'feedparser' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [Errno -2] Name or service not known -- Some packages may not be found!
No local packages or download links found for feedparser
error: Could not find suitable distribution for Requirement.parse('feedparser')
i got curl and the certs installed in /etc/pki/tls/certs/ as per instrucions in the OP, any help?
Thoughts on PIL?
Hmm... new problems here. Android Lollipop have no /system/etc/resolv.conf by-default and urllib patch is no more works.
How it can be fixed?
http://qpython.net/index.php
Where I can find source code and related documents for project
Pratik Raj said:
Where I can find source code and related documents for project
Click to expand...
Click to collapse
I would like to know the same, as well as what license it is under... Also, I don't care about the network, but does it work on Oreo? Also, do you think I could customise the installer to install to a non-system location (/tmp) so that my flashable zip can use python?
hackintosh5 said:
I would like to know the same, as well as what license it is under... Also, I don't care about the network, but does it work on Oreo? Also, do you think I could customise the installer to install to a non-system location (/tmp) so that my flashable zip can use python?
Click to expand...
Click to collapse
Idk about OP, but I can surely post my source for python 2.7 on arm/arm64. I compiled it like a year ago, but it wasn't too bad, just required a bunch of patches and code for dlopen
I just want to say thank you so much.
To research a way to "Run a Linux distro natively" on the Galaxy Tab3 10.1.
XDA:DevDB Information
Run a Linux distro natively, Tool/Utility for the Samsung Galaxy Tab 3 10.1
Contributors
r2d23cpo, CalcProgrammer1, moonbutt74, Angel_666
Version Information
Status: Testing
Created 2015-01-21
Last Updated 2015-02-08
reserved
Wao I wrote a nice introduction and posted on first post and after I edited the project every is gone. So I will post down here something again.
For now I will research the possibility of adapting Kexecboot to our X86 device.
Kexecboot is a program that implements a second-stage bootloader using a stripped-down Linux and the 'kexec' feature to boot into the final running Linux image.
Click to expand...
Click to collapse
Please do search in youtube for kexecboot there are a number of short videos.
h__p://kexecboot.org link not working today
Did some search for you and here are the top ones on XDA.
Kexecboot on the TF700t(TF700t-AKBI v2.6.5) by @workdowg
http://forum.xda-developers.com/showthread.php?t=2387133
[BOOT] Kexecboot / CM-Recovery by @threader
http://forum.xda-developers.com/showthread.php?t=2520663
[DEV/WIP] Kexecboot Bootloader for Galaxy Note 3 N900T - Boot Multiple Kernels by @CalcProgrammer1
http://forum.xda-developers.com/not.../dev-wip-kexecboot-bootloader-galaxy-t2864109
[BOOTLOADER]KexecBoot - A graphical kernel bootloader[Ver: 14Nov2014] by @apapousek
http://forum.xda-developers.com/ico...ootloader-kexecboot-graphical-kernel-t2937006
Improve kexecboot
http://elinux.org/Improve_kexecboot
Now I had started to search and I think I found that Tasssadar/multirom is base in kexecboot. This could be even better, maybe we can convince @Tasssadar to help us create Multirom for the TAB3!
So here is a link to his thread
[MOD][JAN 15] MultiROM v30 [email protected]
http://forum.xda-developers.com/showthread.php?t=2011403
Hopefully the see their mention here and give us a hand by pointing us out where to start.
Step one is to get a kexec-capable kernel working. Kexecboot is just a frontend for kexec. There are two options - normal kexec and kexec-hardboot. I managed to make hardboot work with the Note 1 using some patches but haven't been successful with either on the Note 3. I don't know about this device but if it is x86 then hardboot is likely out as it has ARM-only patches. Then again, if it's x86 you may be able to use GRUB or something. Reading the GSMArena page for the Tab3 says Marvell ARM7 with PowerVR though. In that case you can forget GPU acceleration ever being an option as right now there is no X11/Mesa capable PowerVR driver nor is there a development effort for one.
@CalcProgrammer1 and others
I do really appreciate your comments. Please forgive my ignorance in this area. I am about to say some stupiditty. jijijiji. I see you have point some concerned about " PowerVR" and "GPU acceleration". I just want to boot into Linux Distro as best at it can. Acceleration of any kind is not a priority. But if any driver limitation prevent us from displaying in a std mode then that can be a show stop.
I understood you say
Code:
Step one is to get a kexec-capable kernel working. Kexecboot is just a frontend for kexec..
1-Our Samsung Tab3 10.1 "OS Open source" is incomplete, that is why CM11 has been to slow on release. Still guys at "[SIGNUP] [DEVELOPMENT] signup sheet for aosp build team" had work hard to produce a pre-alpha.
2-Good news is that Open Source for the Kernel works fine.
So can you pointing me in the right direction. How to build this " kexec-capable kernel". I had play with kernel build and even added some missing modules to have CIFS capability. SO compiling Kernel should not be hard for me.
Clearly adding "kexec" may not bee that easy. I am hoping to see an option in the building process.
Code:
kexec is a system call that enables you to load and boot into another kernel from the currently running kernel.
I had played some what with boot.img initramfs and hacked /init to do perform similar task.
Code:
!/sbin/busybox sh
# initramfs pre-boot init script
# Mount the /proc and /sys filesystems
/sbin/busybox mount -t proc none /proc
/sbin/busybox mount -t sysfs none /sys
/sbin/busybox mount -t tmpfs none /dev
# Something (what?) needs a few cycles here
/sbin/busybox sleep 1
# Populate /dev
/sbin/busybox mdev -s
# Mount the root filesystem, second partition on micro SDcard
/sbin/busybox mount -t ext4 -o noatime,nodiratime,errors=panic /dev/mmcblk1p2 /data/local/mnt/ubuntu
# Clean up
/sbin/busybox umount /proc
/sbin/busybox umount /sys
/sbin/busybox umount /dev
# Transfer root to SDcard
[COLOR="Red"]exec[/COLOR] /sbin/busybox [COLOR="Red"]switch_root[/COLOR] /data/local/mnt/ubuntu /etc/init.stage1]
This is a C& P from =>
As you see I take the second partition in External SDCARD "/dev/mmcblk1p2" and mounted in "/data/local/mnt/ubuntu" and transfer to a secondary init in Ubuntu with "exec switch root".
Code:
[COLOR="Red"]exec[/COLOR] /sbin/busybox [COLOR="Red"]switch_root [/COLOR]/data/local/mnt/ubuntu /etc/init.stage1
I guess this exec switch root is performing similar function as "kexec"
Can we work with this? -> Edited: I had already compile kexec in kernel.
Any help will be really really appreciated. Thanks.
@CalcProgrammer1
Ok
First step DONE!
I had compiled a new kernel with kexec. kexec stock from Kernel, just enabled. Do not know about hardboot but willing to see the patch!.
Listen Arm/mips/x86 should be the same kernel(clearly on same kernel number!). So, arm should not matter. Any way the code for kexec seems small enogth to compare with yours.
One thing I have to mention I am working with 4.2.2 since it is less molested by SELinux protection.
What is next step? Pleaseeeeeee.
@JoinTheRealms @cogano @jcfunk any kexec help or advice for these guys?
@workdowg @JoinTheRealms @cogano @jcfunk
Wao pretty nice string of mention jijiji. Do not feel obligated, but any help really appreciated.
Listen guys do not feel ignore but since @CalcProgrammer1 first approach me, I am trying to follow his work at [UTIL] Kexecboot Bootloader for Galaxy Note i717 - Boot Multiple Kernels. Still I you point me to another one I will gladly read on it.
So I build a kernel with kexec enabled! Will that maters or I need to compile one kexec from source?
Next I had downloaded your kernel_kexecboot_quincyatt_v2.zip and open the initramfs. Also compare it to your source. Looks similar. I see I need the following files:
Code:
ramdisk/bin/busybox need to veryfy my busybox compitibility but ok in general
ramdisk/bin/[COLOR="Blue"]kexecboot[/COLOR]
ramdisk/bin/[COLOR="Blue"]lvm[/COLOR]
ramdisk/bin/sh->busybox should be ok
ramdisk/bin/[COLOR="Blue"]tssrv[/COLOR]
ramdisk/sbin/kexec
ramdisk/sbin/[COLOR="Blue"]refresh[/COLOR]
ramdisk/init -> it is a sh script ok
So assuming my enabled kexec is ok for now, I also need to compiled kexecboot, lvm, tssrv, refresh but I have no source for lvm, tssrv, refresh.
Where can I get sources for: lvm, tssrv, refresh? I will check if lvm and tssrv may be part of any busybox, toolbox or recovery I have around.
Files from CalcProgrammer1 project "[UTIL] Kexecboot Bootloader for Galaxy Note i717 - Boot Multiple Kernels."
h__ps://github.com/CalcProgrammer1/kernel_quincyatt_kexec
h__ps://mega.co.nz/#F!0ct3EaTD!wHWnGo1M_2smyKdzGMIYmw
r2,
chroot to your distro and run in terminal
apt-get update
apt-get install gcc gcc-4.7
check /etc/apt/sources.list to see if you have source repositories entries, for example [in debian]
deb-src http://ftp.us.debian.org/debian/ wheezy main
your distro will vary, then
apt-cache search lvm <-----and so forth for every package you need from [src] and compile right on your tab natively
i've been doing this on arm, the end/native result should be similar. Be aware you may need to establish a repository locally for your
kernel source in case you need to generate headers for your toolchain
cd [kernel-source-$TOP]
make mrproper
make ARCH=x86 [correct_defconfig]
make ARCH=x86 headers_check
make ARCH=x86 [your toolchain's correct location for]/sysroot/usr headers_install
with your mount points correct for your ubuntu project you should be able to do all that via terminal.
m
r2d23cpo said:
So assuming my enabled kexec is ok for now, I also need to compiled kexecboot, lvm, tssrv, refresh but I have no source for lvm, tssrv, refresh.
Where can I get sources for: lvm, tssrv, refresh? I will check if lvm and tssrv may be part of any busybox, toolbox or recovery I have around
Click to expand...
Click to collapse
You don't need them. My sloppy work is showing lol. I took most of that from another kexecboot from the HP TouchPad. Chances are your Tab3 doesn't use LVM, though the TouchPad did so that's where that came from. You can remove it.
The tssrv binary is the userspace HP TouchPad touchscreen driver. Most tablets and phones have a kernelspace touchscreen driver, so you can just remove this (plus kexecboot isn't touch driven anyways).
Refresh was something I was testing for my Note 3, as it uses a software-refreshed panel. If your Tab3 uses a video mode panel you won't need a software refresher but if it does, well, you may run into an issue that I don't know about as refreshing seems SoC-specific and I'm only used to Qualcomm chips. Remove it for now as well.
Wao spent over 18 consecutive hours, I had to sleep. I had trouble always with this so call automatic makefile that never work!! It is simpler to create my own makefile and adjust, ad subtract commands as it show error.
So I am working. just having some programing issues that do not let me go at a better speed. See you latter friends.
IMPORTANT:
PLEASE if you downloaded the zimage I have previously gave link, DO NOT USED IT.
Yes it has KEXEC Enabled 100%. BUTTTTT, I did something wrong in the selection that prevent WIFICommunication!!! You been warned. If you had such an accident, Use Odin To reinstall a good Booot Image or a zip package using the External SDCARD. If You need help please ask. But As I said I am going to sleep now.
r2d23cpo said:
Wao I wrote a nice introduction and posted on first post and after I edited the project every is gone. So I will post down here something again.
For now I will research the possibility of adapting Kexecboot to our X86 device.
Please do search in youtube for kexecboot there are a number of short videos.
h__p://kexecboot.org link not working today
Did some search for you and here are the top ones on XDA.
Kexecboot on the TF700t(TF700t-AKBI v2.6.5) by @workdowg
http://forum.xda-developers.com/showthread.php?t=2387133
[BOOT] Kexecboot / CM-Recovery by @threader
http://forum.xda-developers.com/showthread.php?t=2520663
[DEV/WIP] Kexecboot Bootloader for Galaxy Note 3 N900T - Boot Multiple Kernels by @CalcProgrammer1
http://forum.xda-developers.com/not.../dev-wip-kexecboot-bootloader-galaxy-t2864109
[BOOTLOADER]KexecBoot - A graphical kernel bootloader[Ver: 14Nov2014] by @apapousek
http://forum.xda-developers.com/ico...ootloader-kexecboot-graphical-kernel-t2937006
Improve kexecboot
http://elinux.org/Improve_kexecboot
Now I had started to search and I think I found that Tasssadar/multirom is base in kexecboot. This could be even better, maybe we can convince @Tasssadar to help us create Multirom for the TAB3!
So here is a link to his thread
[MOD][JAN 15] MultiROM v30 [email protected]
http://forum.xda-developers.com/showthread.php?t=2011403
Hopefully the see their mention here and give us a hand by pointing us out where to start.
Click to expand...
Click to collapse
Congratulations, it´s a very useful idea and i hope it gets developed Good luck!
Good morning Folks! It is 12:07AM in the Atlantic Zone with a nice star sky.
So after 322 views and 11 replys @rjmm13 is our first non-developer that had show and post interest. Congratulations! [Applause ]
And we are ready do some more tests today.
To my friends with the right knowledge I need the full kexec instruction I will use to boot Ubuntu. From the Ubuntu manuals it seems some thing like:
Code:
kexec -l /boot/vmlinux --append=root=/dev/hda1 --initrd=/boot/initrd
For now I am using @CalcProgrammer1 work on Galaxy Note i717. init shows he mounted proc, sys & dev. I had place or should I say extracted Ubuntu in my 2nd ext4 partition on the external SdCard or partition "/dev/mmcblk1p2" So I guess my command should look like
Code:
kexec -l /boot/vmlinux --append=root=/dev/mmcblk1p2 --initrd=/boot/initrd
But I wonder, that partition is not mounted!!!! Or I am wrong.
My other Option will be to mount mmcblk1p2 in /data/local/mnt/ubuntu
Then partial code may looks like:
Code:
busybox mount -t ext4 /dev/block/mmcblk1p2 /data/local/mnt/ubuntu
kexec -l /boot/vmlinux --append=root=/data/local/mnt/ubuntu --initrd=/boot/initrd
So witch one is the good command sequence?
------------------------------------------------------------------------------
Edited.
To @moonbutt74 I have not ignored you in try to build on same TAB3 device. But If I do then I will have to worry about backing up every time I have to do test on TAB3!!
Listen I am having troubles compiling automated makefiles. So I recall your comments. And I am up to building kexec-tools with same Ubuntu deb packages. That should be pretty easy.
I have Ubuntu amd64 desktop but I will try to install i386 debs for maximum compatibility! Just as a test.
Code:
apt-get update
apt-get install kexec-tools_2.0.2-3ubuntu4_i386
Do you know if there is a way to then force my out kernel source to used Ubuntu Desktop files? I know may sound stupid.
Now regards
Code:
cd [kernel-source-$TOP]
make mrproper
make ARCH=x86 [correct_defconfig]
make ARCH=x86 headers_check
make ARCH=x86 [your toolchain's correct location for]/sysroot/usr headers_install
What "make mrproper" commads do?
What "correct_defconfig" do you used?
"make ARCH=x86 [your toolchain's correct location for]/sysroot/usr headers_install" I do not recalled doing this And I compiled it ok. What the kernel sources ask was to edit a config file and add my tool location?
Thanks body. Take your time I know you are busy in your own projects.
I am doing it again, making a full conversation my self! jijijiji
@moonbutt74 you are genius, if I run Ubuntu in a VNC server in TAB3, I can work in my PC. And walaaaaa! I am in fact running from external sdcard partition. I do not have to worry about backup!!! As always you are the man.
With the advantage that the code generated by Ubuntu should be 100% compatible with TAB310.1 x86.
In the future I do really have to look more deep in your comments.
I will test to see how it goes.
I waist 7 hours doing some needed backup, my desktop was getting low in memory. Almost lost my files due to corruption. But I am back.
So I tested leaving the tablet as Ubuntu VNC Server and works but I still think is slow. But I do not need VNC.!!! Nor the GUI.
I had recall installing "shh" previously in my Ubuntu, So I can just do Putty "shh" in Win7 and it works much better!!! In fact I can just use WinSCP also to even load or take files easily from the TAB3 Ubuntu system file. Thanks body for the Idea.
Now then I will start doing what you did a week ago, Installing inside the Tab310.1 the SDK toolchain, platform and the Kernel source just in case I need it. I know I am continue wasting more time without doing real-work on kexecboot. But if it works I will improve my developing speed in the future. I hope to report soon.
r2d23cpo said:
Code:
kexec -l /boot/vmlinux --append=root=/dev/mmcblk1p2 --initrd=/boot/initrd
But I wonder, that partition is not mounted!!!! Or I am wrong.
My other Option will be to mount mmcblk1p2 in /data/local/mnt/ubuntu
Then partial code may looks like:
What "make mrproper" commads do?
What "correct_defconfig" do you used?
"make ARCH=x86 [your toolchain's correct location for]/sysroot/usr headers_install" I do not recalled doing this And I compiled it ok. What the kernel sources ask was to edit a config file and add my tool location?
Thanks body. Take your time I know you are busy in your own projects.
Click to expand...
Click to collapse
1. The root= option will automatically mount that given device as the root partition (aka /). If you use root= you do not need an initramfs at all, because the kernel will mount the partition and run the /init script automatically. If you don't use a root= command or wish to run more complex boot sequences, then you will need an initramfs. In that case, the initramfs gets mounted as / and then must do its own work to mount the real root partition and do switch_root to replace the temporary initramfs root with the new proper root partition.
2. "make mrproper" is a better version of "make clean". It basically resets your entire kernel build to a clean state. Do this if you want to make sure your build is pure rather than reusing already-compiled temporary files.
3. You must create your own defconfig since you're using a new device. Use make menuconfig to edit your .config file to how you like it, then copy it to arch/<architecture>/configs/your_defconfig_name and then you will be able to use make your_defconfig_name to setup your .config file each time you want to rebuild it. I think x86 is "i386" in the tree, ARM is "arm".
4. If you're building for an x86 device on another x86 device you should just be able to make without the ARCH or CROSS_COMPILE flags. If you're using an x86_64 machine you'll want to set up a 32-bit chroot or something though if you want to build a 32-bit kernel. If you're building for ARM on an x86/amd64/anything-else-not-arm machine then you have to do ARCH=arm CROSS_COMPILER=name-of-your-arm-toolchain- so it builds it with the cross compiler.
@CalcProgrammer1
The following Is NOT a correction Nor disputing your comments. Please continue helping. If I write is to clarify what is my understanding.
I am lost in #1. I bet you are trying to explain what my options are by using "root=". Still it get me confuse. But listen what I need is the following. We has said that Kexecboot is a front end of kexec. So what I want is to test kexec first ALONE so see how it behaves. For that I need the correct kexec construction sentence to call Ububtu in the secon partiyion of the SDCARD. So I look in the kexec manual an I got what I wrote before. Now I am asking you if you know or you can look at your Kexec config file for that sentence. I hope you understand me now.
#2 is a nice explanation. I see. Thanks
#3 Open Source instruction said to use "android_santos10_open_r00_user_defconfig" But I agree with you, So what I did I use instead was "make ARCH=i386 android_santos10_open_r00_eng_defconfig" Then "make menuconfig" and went and try to enable what I needed. Basically verify the kexec was enabled, and that my CIFS was also enabled. Still I learn to copy ".config" and place it a my future config with a different name. Thanks.
#4 I tend to agree with you some what in that the x86 binary code of my Ubuntu Desktop may work in the x86 tablet. Well that is almost the case. As an example I downloaded the Open Office Ubuntu deb package for x86 desktop and then installed in the tablet. 100% working. But some of binary code compiled not always work. Yes my Ubuntu Desktop is x86 64. And I bet you that if the code gets generated for 64 bit I am not sure is it will work. Then the other factor I think is the possibility of missing libraries that Ubuntu desktop my have against Android minimal system.
Now You raise a good point, I could run a chroot Ubuntu x86 ( not 64). But then again It is simpler to have another partition and run Grub2 with Ubuntu x86 without any chroot. That is what I did normally. But I have 1.5 tetra of garbage. I am in the process of backing up, so that I can liberate HD space.
Regards "CROSS_COMPILER" jijiji yes it seems funny. But the real answer is that it is false that the process do "CROSS_COMPIling". That statement just select the toolchain to use. It does not matter if it is mips, arm or Intel. Just point to the place of the toolchain. The the output clearly is going to be the output for what the toolchain was build for.
So Can you look at your kexecboot config and tell me if the command is there.
By the way The Documentation for Kexecboot at "http://kexecboot.org" is no longer available. It seem that the site is dead. In fact many others related to kexecboot are dead too!!. So I ask you for any chance did you save that documentation? Can I have it?
Thanks body I really appreciate your
PD, I am Create a kernel in my desktop (i7 8core), but I have to mention that I did not include V8, so I guess it used 1 or 2 cores only. It took 25 minutes. I am compiling the kernel also in my Tab3. It has pass 1 hour and I guess is half way!!
But, I had 2 outages of electrical power! My Tab3 is unaffected to outages of power jijijiji. This is why I want to have Linux natively. This is weird It never happens. I guess there have to be a construction near by that require the cut of power.
hey guys,
just a quick note in reference to,
4. If you're building for an x86 device on another x86 device you should just be able to make without the ARCH or CROSS_COMPILE flags.
Click to expand...
Click to collapse
that is true if you are building/compiling natively on this tab. However if you are compiling on a pc [x86][x86_64} you do need to cross compile and pass flags.
The reason is you need to link [for some reason] with ld.gold for the binary to execute else you get the bad/incorrect/invalid magic error.
If building on pc for this tab through buildroot set arch for i386 and arch variant for i686. for building through gcc on your own makefile recipe
you will need to pass the following flags for CFLAGS , CXXFLAGS and LDFLAGS ; -m32 -static, and pass/export ARCH=x86,this goes for building on 32bit or 64bit pc.
m
moonbutt74 said:
...If building on pc for this tab through buildroot set arch for i386 and arch variant for i686. for building through gcc on your own makefile recipe you will need to pass the following flags for CFLAGS , CXXFLAGS and LDFLAGS ; -m32 -static, and pass/export ARCH=x86,this goes for building on 32bit or 64bit pc....
Click to expand...
Click to collapse
Nicessss, I was looking for that. So it does not requires the "toolchain" other that really compiling on a desire vesrion number. Did I understood correct or I need to setup toolchain some where? What about gcc on the Ubuntu?
What I do is create my own Makefile like
Code:
NDKROOT=/home/<username>/bin/androidndk
NDKSYSROOT=$(NDKROOT)/platforms/android-8/arch-x86
TOOLCHAIN := $(NDKROOT)/toolchains/x86-4.6/prebuilt/linux-x86
CC := $(TOOLCHAIN)/bin/i686-linux-android-gcc
#INCLD_DIR := $(TOOLCHAIN)/usr/include
INCLD_DIR := $(TOOLCHAIN)/include
LIB_DIR := $(TOOLCHAIN)/lib
#LIBS = -L$(LIB_DIR)
TARGET := kexecboot
#CFLAGS := -Wall
CFLAGS = -DNO_MNTENT -UHAVE_ICONV -DNO_MALLINFO=1 \
--sysroot=$(NDKSYSROOT) \
-isystem $(NDKSYSROOT) \
-I$(NDKSYSROOT)/usr/include \
-mandroid \
-D_GNU_SOURCE=1 \
-L$(NDKSYSROOT)/usr/lib
# -I$(NDKSYSROOT)/usr/include/sys \
# -I$(NDKSYSROOT)/usr/include/linux \
LDFLAGS := -L$(TOOLCHAIN)/lib -Wl,-rpath-link $(TOOLCHAIN)/lib
SRCS := $(TARGET).c util.c cfgparser.c devicescan.c evdevs.c fb.c gui.c \
menu.c xpm.c rgb.c tui.c fstype/fstype.c machine/zaurus.c
OBJS := $(TARGET).o util.o cfgparser.o devicescan.o evdevs.o fb.o gui.o \
menu.o xpm.o rgb.o tui.o fstype.o
#OBJS := util.o cfgparser.o devicescan.o evdevs.o fb.o gui.o \
# menu.o xpm.o rgb.o tui.o fstype.o zaurus.o
LIBS := -L$(LIB_DIR)
#-lutil -lpthread
$(TARGET): $(OBJS)
# $(CC) $(CFLAGS) $(LDFLAGS) $(OBJS) $(LIBS) $(TARGET).c -o $(TARGET)
$(CC) $(CFLAGS) $(LDFLAGS) $(OBJS) $(LIBS) -o $(TARGET)
$(TARGET).o: $(TARGET).c $(TARGET).h
$(CC) $(CFLAGS) -c $(TARGET).c
util.o: util.c util.h
$(CC) $(CFLAGS) -c util.c
cfgparser.o: cfgparser.c cfgparser.h
$(CC) $(CFLAGS) -c cfgparser.c
devicescan.o: devicescan.c devicescan.h
$(CC) $(CFLAGS) -c devicescan.c
evdevs.o: evdevs.c evdevs.h
$(CC) $(CFLAGS) -c evdevs.c
fb.o: fb.c fb.h
$(CC) $(CFLAGS) -c fb.c
gui.o: gui.c gui.h
$(CC) $(CFLAGS) -c gui.c
menu.o: menu.c menu.h
$(CC) $(CFLAGS) -c menu.c
xpm.o: xpm.c xpm.h
$(CC) $(CFLAGS) -c xpm.c
rgb.o: rgb.c rgb.h
$(CC) $(CFLAGS) -c rgb.c
tui.o: tui.c tui.h
$(CC) $(CFLAGS) -c tui.c
fstype.o: fstype/fstype.c fstype/fstype.h
$(CC) $(CFLAGS) -c fstype/fstype.c
#zaurus.o: machine/zaurus.c machine/zaurus.h
# $(CC) $(CFLAGS) -c machine/zaurus.c
clean:
rm -f *.o *~ $(TARGET)
By the way, my Kernel Build took 2 hours on the TAB310.1. Now I need to test it.
r2,
hi,
if compiling on the tab for the tab [natively through tab's ubuntu distro], compile straight and you should be good to go.
if compiling outside the tab [host/build] for the tab [target] [cross compile], then yes you need to pass flags and such.
For my experiments when i had the tab, i first built my toolchain through the 32bit version of android ndk and geared to
jellybean api.
In my makefile the flags i included were
CFLAGS := -m32
LDFLAGS := -static <------this is/can be optional, i like static binaries for portability reasons.
on -Wall , i usually leave that on.
on -werror, i usually turn that off.
m
Wao still tryint to leave the Starting Line but one after the other. jijiji
But Good news. To strip out all problems I decided to start from scrach. And after loking at the config as it is I found that kexec was set to enable from the beguining.
hsbadr said in his thread [KERNEL] [KEXEC] Kernel EXECution for locked devices [N900V] [WIP] to check kexec by doing cat /proc/kallsyms | grep kexec
and here is the output of the system calls:
Code:
[email protected]:/ # cat /proc/kallsyms | grep kexec
00000000 t machine_kexec_page_table_set_one
00000000 t machine_kexec_free_page_tables
00000000 T machine_kexec_prepare
00000000 T machine_kexec_cleanup
00000000 T machine_kexec
00000000 T log_buf_kexec_setup
00000000 W compat_sys_kexec_load
00000000 t kexec_crash_loaded_show
00000000 t kexec_loaded_show
00000000 t kexec_crash_size_store
00000000 t kexec_crash_size_show
00000000 T kexec_should_crash
00000000 T sys_kexec_load
00000000 T crash_kexec
00000000 T kernel_kexec
00000000 d kexec_loaded_attr
00000000 d kexec_crash_loaded_attr
00000000 d kexec_crash_size_attr
00000000 d event_exit__kexec_load
00000000 d event_enter__kexec_load
00000000 d __syscall_meta__kexec_load
00000000 d kexec_mutex
00000000 d types__kexec_load
00000000 d args__kexec_load
00000000 t __event_exit__kexec_load
00000000 t __event_enter__kexec_load
00000000 t __p_syscall_meta__kexec_load
00000000 B in_crash_kexec
00000000 B kexec_crash_image
00000000 B kexec_image
[email protected]:/ #