Related
i want to alias ls='ls --color=always'. is there something like a .bashrc file for busybox sh?
bilclintonsdaman said:
i want to alias ls='ls --color=always'. is there something like a .bashrc file for busybox sh?
Click to expand...
Click to collapse
Just had a look at the busybox source and it says it uses the contents of environment variable ENV as a startup script so you could do
Code:
ENV=/system/sd/.bashrc ash
and it'll read /system/sd/.bashrc for your startup commands. Altrernatively, if you invoke it as a login shell (ash -l), it'll source /etc/profile then .profile from the current directory so if you create /system/sd/.profile to contain alias ls='ls --color=always' then as long as you do this
Code:
cd /system/sd
ash -l
you'll have your alias (I just tried this with JF1.51 and it worked great).
I also saw that it will record command history to the file named in HISTFILE so you may want to make your startup script look like this
Code:
alias ls='ls --color=always'
HISTFILE=/system/sd/.histfile
Also, keep in mind that this isn't bash -- you need to use ash shell scripting semantics.
Sweet! Thanks so much
~/.PROFILE
if you edit (may need to create it first) the .profile file in your home directory you shouldn't have an issue
i have the following in my ~/.profile file:
Code:
alias dir='ls -Alp'
alias cls='clear'
The next time you log in, you'll have your alias's
pre-compiled gnu bash
I can't remember where I found it, however, if you know enough to set it up, I have a pre-compiled gnu bash 3.2 file that allows you to execute something like this:
su -c '/path/to/bash --rcfile /path/to/.bashrc'
in the terminal emulator and you'll execute a bash shell with the .bashrc
I attached the pre-compiled gnu bash 3.2 in a zip file on the post, good luck getting it running! google is your friend!
I also compiled bash 4.0 for myself to use, but its kind of buggy since it was actually one of the first binaries that i've ever compiled on linux but it works! its just really slow >.>
I followed this tutorial to compile it
http://www.kbrandt.com/category/bash
EDIT: Just updated post 2 you probably want to read both this post and that post before trying to build.
you can skip the part of getting the official source and the cyanogen source by just doing
Code:
repo init -u git://github.com/CyanogenMod/android.git -b froyo
First some prereqs You must be using linux and building for an evo 4G aka htc_supersonic
Install the android SDK as instructed here: http://developer.android.com/sdk/installing.html
You will need JDK most linux distros have this in the respositorys.
Then just follow step 2 and make sure that you can run adb from any directory in terminal.
Now to the good stuff.
You will now need to open a terminal window and run the following as root
Code:
cd /usr/bin/; curl http://android.git.kernel.org/repo > repo; chmod a+x /usr/bin/repo
now as user do the following
Code:
mkdir ~/android
cd ~/andoid
repo init -u git://android.git.kernel.org/platform/manifest.git
repo sync
This can take FOREVER like 4-5 hours so you might want to let it go when you go to sleep.
Depending on how fast your computer is the following step can take a long time too
Code:
cd ~/andoid
make
This will build the source code for a generic device it should build everything correctly if you have all of the compilers and what not. If it fails just install the app that it is complaining about.
Now for the EVO specific stuff
Code:
cd ~/android/device/htc
git clone http://github.com/CyanogenMod/android_device_htc_supersonic.git supersonic
That will get the build files however you are going to have to edit them to get it to work.
~/android/device/htc/supersonic/BoardConfig.mk needs to be edited
Code:
TARGET_RECOVERY_UI_LIB := librecovery_ui_supersonic librecovery_ui_htc
TARGET_RECOVERY_UPDATER_LIBS += librecovery_updater_htc
should be
Code:
#TARGET_RECOVERY_UI_LIB := librecovery_ui_supersonic librecovery_ui_htc
#TARGET_RECOVERY_UPDATER_LIBS += librecovery_updater_htc
Some other files may have to be edited to your liking. Personaly I changed ADWLauncher to launcher2 and removed all of the cyanogen stuff.
Now to setup the build enviroment.
Code:
cd ~/android
cp ./vendor/cyanogen/products/cyanogen_supersonic.mk ./buildspec.mk
. build/envsetup.sh
lunch cyanogen_supersonic-eng
Now to do the building.
Code:
make CYANOGEN_WITH_GOOGLE=true otapackage ## if you want google apps (market etc.)
make otapackage ## 100% Open Source only!
By the way for some reason you have to inclued the otapackage even if you just want the .img files because otherwise it crashes at least for me.
Let me know if you encounter any problems and I will try to help.
UPDATES
Copy your ~android folder somewhere and replace ~/android in this tutorial with the directory that you copied it to. This will allow you to update the overlay when needed.
To add google files you will need to do this
Code:
cd ~/android/vendor/cyanogen
./get-google-files
This gets the htc files and I think taht it might be required.
Code:
cd ~/android/device/htc/supersonic
./extract-files.sh
To make a clean rom like mine do the following:
Edit the file ~/android/vendor/cyanogen/products/common.mk to your liking. I just changed the build name and rom version.
I added the files sysrw and sysro ad you can see in the file just put them in a directory and tell it where they are and where you want them. Also I commented out rommanager.apk as I don't like or use it. Also I use RA recovery so it wont work for me anyway.
Also edit the file ~/android/vendor/cyanogen/products/cyanogen_supersonic.mk
I chaged teh build number to the latest spring and removed the boot animation as I like the stock android one better (and it is faster).
YOU NEED TO do this if you want a keyboard...
edit ~/android/vendor/cyanogen/Android.mk and remove LatinIME!
Now we delete some files.
go to ~/android/packages/apps
delete any application that you don't want. You can optionally add program source code that you do want.
I deleted CM* and Protips
I also edited some code in Settings to remove Cyanogen stuff. Actually I took the stock android stuff an edited that with one little fix but I forgot what it was. Just try to build and it will tell you in an error message.
another reserved post
Really cool
Awesome! Thanks so much for posting this.
thanks dude!!
Thanks man.
question...
im having an issue with this step
Code:
cd ~/adnroid
cp ./vendor/cyanogen/products/cyanogen_supersonic.mk ./buildspec.mk
. build/envsetup.sh
lunch cyanogen_supersonic-eng
specifically (lets ignore the type in the instructions
Code:
$ cp ./vendor/cyanogen/products/cyanogen_supersonic.mk ./buildspec.mk
cp: cannot stat `./vendor/cyanogen/products/cyanogen_supersonic.mk': No such file or directory
$ find ~/android -name "cyanogen_supersonic.mk"
$ echo $?
0
anyone else having this issue?
So what is the difference between this and running CM6?
Will this avoid the market issues with CM6?
Once it's built, how do you get it installed on the phone?
Here is the link to cyanogens wiki how too......
http://wiki.cyanogenmod.com/index.php?title=Compile_CyanogenMod_for_Supersonic
Anyway to do this on Windows 7? Another question, would this still work if I just ran Ubuntu from disk?
Thanks, will def.ly give this a try soon...
Sent from my PC36100 using Tapatalk
onicrom said:
question...
im having an issue with this step
Code:
cd ~/adnroid
cp ./vendor/cyanogen/products/cyanogen_supersonic.mk ./buildspec.mk
. build/envsetup.sh
lunch cyanogen_supersonic-eng
specifically (lets ignore the type in the instructions
Code:
$ cp ./vendor/cyanogen/products/cyanogen_supersonic.mk ./buildspec.mk
cp: cannot stat `./vendor/cyanogen/products/cyanogen_supersonic.mk': No such file or directory
$ find ~/android -name "cyanogen_supersonic.mk"
$ echo $?
0
anyone else having this issue?
Click to expand...
Click to collapse
there was a typo in my code I edited the post and it should work now.
cteneyck said:
Here is the link to cyanogens wiki how too......
http://wiki.cyanogenmod.com/index.php?title=Compile_CyanogenMod_for_Supersonic
Click to expand...
Click to collapse
So really, this is just CM6 built on your own Linux box?
Sent from my PC36100 using Tapatalk
Can this be done from ubuntu on a virtual machine?
And what's the difference between this and cm6 ?.....
how do you replace the adw launcher ?
how can you add additional applications?
onicrom said:
how do you replace the adw launcher ?
how can you add additional applications?
Click to expand...
Click to collapse
in vendor/cyanogen/products
replace this
Code:
#PRODUCT_PACKAGES += ADWLauncher
PRODUCT_PACKAGES += Launcher2
also all of the people asking what the differnece between this and cm is nothing unless you change the source files or add remove packages. they can be found in the packages directory.
Igotsanevo4g said:
Can this be done from ubuntu on a virtual machine?
And what's the difference between this and cm6 ?.....
Click to expand...
Click to collapse
yes you can do this in VM but i don;t know the exact steps to set that up just once you get into your linux VM follow the tutorial.
nice tut. thanks for this...sure it will help MANY
l33tlinuxh4x0r said:
in vendor/cyanogen/products
replace this
Code:
#PRODUCT_PACKAGES += ADWLauncher
PRODUCT_PACKAGES += Launcher2
also all of the people asking what the differnece between this and cm is nothing unless you change the source files or add remove packages. they can be found in the packages directory.
Click to expand...
Click to collapse
Thanks much
I havent yet found a simple guide for compiling kernels. Some of them assume too much, and some are just outdated. So I thought I'd write my own for devs/budding devs. Here you go!
Note:
This is not a guide for newbies. It's a dev guide for devs.
Research before asking questions, please
For The Menu driven interactive kernel build script, see Post #31
I will be developing this guide as I go, so it will be incomplete initially, or lacking in detailed explanations.
Essentials:
Ubuntu Box (By this I mean a PC with a Ubuntu installation, not a live CD)
A toolchain-Either the Android NDK, or your own toolchain
HTC Desire GB/Froyo source from htcdevs.com, or sources from github
Familiarity with the linux shell and basic linux commands.
The will to learn
First things first,
1. Getting the sources
The HTC Desire source is available from two kinds of resources-you can either get it from htcdevs.com (official HTC Dev site), or from source code uploaded from someone else. For the purpose of this tutorial, I'll assume we're working on the official HTC GB source code. So download bravo_2.6.35_gb-mr.tar.gz from htcdevs.com.
2. Setting up the compilation box and preparing source code
2.1 Install some essential linux packages from the Linux terminal:
Code:
sudo apt-get install libncurses5-dev
2.2 Extract the source code
The file you downloaded is a tar archive (like a zip file), so you need to extract it to a convenient location. Let's hit the linux shell-open a terminal window in linux (Accessories->Terminal)
Type:
Let's go to our home directory:
Code:
cd ~/
Now, create the directories for our kernel compilation box.
Code:
mkdir -p ~/android/kernel
Now you need to copy the tar.gz file from wherever you downloaded it to, to this dir.
Extract the archive:
Code:
tar -xvf ~/android/kernel/bravo_2.6.35_gb-mr.tar.gz
cd ~/android/kernel/bravo_2.6.35_gb-mr
Now we can view the extracted files within the directory ~/android/kernel/bravo_2.6.35_gb-mr/
2.3 Set up the toolchain
A toolchain is a set of programs which allow you to compile source code (any source code, not just kernels). The toolchain is specific for the processor and hardware, so we need a toolchain specific for Android and especially the Desire. If you're a semiadvanced-pro user, you may consider compiling your own toolchain (See theGanymedes' guide for doing so). If compilation of kernels is all that you require, fortunately for you, there is an easy way-the Android NDK - v7 (latest as of now) is available here
Get the NDK for Linux - android-ndk-r7-linux-x86.tar.bz2
Code:
mkdir -p ~/android/ndk
Now copy the NDK file to ~/android/ndk
Whenever I say copy, you have to manually copy the file with any file manager. Nautilus comes with Ubuntu, and Dolphin with Kubuntu. You may also use the shell of course with
Code:
cp [sourcefile] [destination]
Extract it:
Code:
tar -jvxf android-ndk-r7-linux-x86.tar.bz2
Now add the path for your toolchain to the env variable:
Code:
gedit ~/.bashrc
At the end of the file, add this line:
Code:
PATH=$PATH:~/android/ndk/android-ndk-r7-linux-x86/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin
3. Setting up kernel parameters
Kernels are compiled with a program called gnu make, and use a set of configuration options specified within a file called Makefile.
A vital point to note is that kernels are compiled with a program called gcc (basically the gnu C compiler), and our NDK itself has its own optimized version of gcc. While compiling, we're actually cross compiling it (meaning compiling a binary package on a system which is different from the actual system which is meant to run it- you're compiling it on your PC while it's actually meant to run on your Desire)
This means that when you compile it, you have to make sure that you compile it with the NDK's version of gcc instead of the system version. Otherwise you end up with a kernel meant to run on your pc, duh! Specifying which gcc to use is by the CROSS_COMPILE variable. You can set it up with this command:
Code:
CROSS_COMPILE=arm-linux-androideabi-
Note the hyphen (-) at the end, and do not forget to include it! At compilation time, system will actually use this variable to find all the programs it needs. Eg: The path for gcc will become arm-linux-androideabi-gcc
We can compile kernels with many different options, like with ext4 support, or without; ext4 support as part of the kernel zImage (in which case it makes the kernel larger), or as a loadable module (of the form somename.ko, which is loaded at init.d/init.rc with the command insmod modulename.ko)
We specify the exact options we require with the help of a useful configuration program called menuconfig (which as the name suggests, is a menu for configuration of make options).
An important thing to note is that as far as kernel compilation is concerned, there are a vast amount of options to setup, and unless you're thorough with kernel compilation, you wont be able to set up the options correctly and get your kernel to boot. Fortunately for us, the kernel source already comes with a default set of parameters which can be easily set up.
Note that all make commands must be executed within the directory bravo_2.6.35_gb-mr. Let's go there now:
Code:
cd ~/android/kernel/bravo_2.6.35_gb-mr
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- bravo_defconfig
This produces a .config file (used by the menuconfig) containing essential parameters to produce a booting kernel for the Desire.
Note: There is a simpler way to get the basic .config file, and this is to get it from a running kernel built by someone else. You can extract the .config from a running kernel with these commands:
Code:
cd ~/android/kernel/bravo_2.6.35_gb-mr
adb pull /proc/config.gz
zcat config.gz > .config
Now we can open menuconfig and add anything we need in addition.
Code:
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- menuconfig
You can view the huge amount of options available in menuconfig.
You can add ext4 support for example (See image above)
Once you're done choosing options, you can exit menuconfig.
4. Compiling it
This is simple. The basic command is:
make ARCH=arm CROSS_COMPILE=arm-linux-androideabi- -j10
The -j10 specifies the number of jobs to execute per operation. I can usually go upto 50 on my Quad core CPU. Beware, this can bring a slow CPU to a crawl and freeze up linux itself.
During compilation, you will see all sorts of messages, which may include warnings too. In most cases, its safe to ignore warnings. If there are errors, the compilation will stop, and you will have to fix the issues.
5. Distributing your kernel to users
At the end of compilation, it generates files named zImage, and various .ko files.
You have to copy them from their default location to a zip file. The best way is to use my variant of koush's Anykernel, and copy the files to it. Then, you can zip the whole folder and lo and behold-you have your flashable kernel zip which you can distribute to others.
You can also remove the zImage and the modules from /system/lib/modules of any kernel zip available with you, and copy over your files to it, at the correct location.
So, let's say that you have extracted an existing kernel zip to the location ~/flashable
The file structure should be like this:
Code:
|-- kernel
| |-- dump_image
| |-- mkbootimg
| |-- mkbootimg.sh
| |-- unpackbootimg
| `-- zImage
|-- META-INF
| |-- CERT.RSA
| |-- CERT.SF
| |-- com
| | `-- google
| | `-- android
| | |-- update-binary
| | `-- updater-script
| `-- MANIFEST.MF
`-- system
`-- lib
`-- modules
`-- bcm4329.ko
8 directories, 11 files
I've included my flashable zip directory along with this post. Download file kernel_flashable.tar.bz2.zip to ~/
Code:
cd ~/
tar -jvxf kernel_flashable.tar.bz2.zip
This will create the directory structure outlined above.
Now after every compilation of the kernel, execute these commands from where you executed make:
Code:
cp arch/arm/boot/zImage ~/kernel_flashable
find . -name '*ko' -exec cp '{}' ~/kernel_flashable/system/lib/modules/ \;
cd ~/kernel_flashable
zip -r mykernel ./
This will create mykernel.zip at ~/kernel_flashable. You can distribute this to your users to flash. Make sure you edit updater-script before though
Common errors and other stuff
Ok, post #1 was simple stuff. Now, supposing you get errors while compiling. Post #2 is about that, and ups the level of knowledge a bit..
Some kernel compilation errors:
Treat warnings as errors-Solved by removing the string "-Werror" from all Makefiles of the file which failed to compile. Some people had said that the real error (Array out of bounds warning) was because of gcc optimizations. But putting -O2 to -O0 didnt do a thing.
No of jobs - ought not to exceed 50.
"warning: variable set but not used [-Wunused-but-set-variable]"-Look at KBUILD_CFLAGS in the main Makefile. Add -Wno-error=unused-but-set-variable to the existing set of flags.
Note the following from gcc manual:
-WerrorMake all warnings into hard errors. Source code which triggers warnings will be rejected.
-w Inhibit all warning messages. If you're familiar with C code and like to fix stuff, rather than ignoring potential bugs, use this only as a last resort- A 'brahmastram' (most powerful weapon in your time of gravest need) as the epics would say
-WerrorMake all warnings into errors.
-Werror=Make the specified warning into an error. The specifier for a warning is appended, for example -Werror=switch turns the warnings controlled by -Wswitch into errors. This switch takes a negative form, to be used to negate -Werror for specific warnings, for example -Wno-error=switch makes -Wswitch warnings not be errors, even when -Werror is in effect. You can use the -fdiagnostics-show-option option to have each controllable warning amended with the option which controls it, to determine what to use with this option.
So what I did to suppress errors was to add:
Code:
KBUILD_CFLAGS += -w
KBUILD_CFLAGS += -Wno-error=unused-but-set-variable
Though the -Wunused-but-set-variable is not a real issue in itself, it generates so much "noise" that you may miss actual make errors.
This is the error what I was talking about..
Code:
drivers/net/wireless/bcm4329_204/wl_iw.c: In function 'wl_iw_set_pmksa':
drivers/net/wireless/bcm4329_204/wl_iw.c:5075:5: error: array subscript is above array bounds [-Werror=array-bounds]
drivers/net/wireless/bcm4329_204/wl_iw.c:5078:5: error: array subscript is above array bounds [-Werror=array-bounds]
Solution:
Edit drivers/net/wireless/bcm4329_204/Makefile
Locate -Werror within DHDCFLAGS, and delete it.
Code:
DHDCFLAGS = -DLINUX -DBCMDRIVER -DBCMDONGLEHOST -DDHDTHREAD -DBCMWPA2 \
-DUNRELEASEDCHIP -Dlinux -DDHD_SDALIGN=64 -DMAX_HDR_READ=64 \
-DDHD_FIRSTREAD=64 -DDHD_GPL -DDHD_SCHED -DBDC -DTOE -DDHD_BCMEVENTS \
-DSHOW_EVENTS -DBCMSDIO -DDHD_GPL -DBCMLXSDMMC -DBCMPLATFORM_BUS \
-Wall -Wstrict-prototypes -Werror -DOOB_INTR_ONLY -DCUSTOMER_HW2 \
-DDHD_USE_STATIC_BUF -DMMC_SDIO_ABORT -DWLAN_PFN -DWLAN_PROTECT \
-DBCMWAPI_WPI \
This will prevent gcc from treating mere warnings as errors.
How to modify kernels by applying mods - Applying Kernel Patches
Ok, you have compiled a simple stock kernel. Now what? Would you like to add fixes/mods developed by other kernel devs? This post explains patches and how exactly to do this.
Patches to the kernel are applied via patch files. Patch files are simple text files generated by the linux diff program which takes two text files, compares them and writes the differences (hence called diff) to another text file which by convention has the extension .patch
Attached to this post is a patch containing my "Extended battery" fix with Sibere's battfix. I'll explain patching with this. Let's understand the patch file. Open it up in any text editor.
Code:
diff -rupN -X /home/droidzone/android/kernel/exclude.opts bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c
--- bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c 2011-08-25 13:16:53.000000000 +0530
+++ bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c 2011-11-06 16:43:21.544317342 +0530
@@ -118,8 +118,11 @@ PS. 0 or other battery ID use the same p
/* Battery ID = 1: HT-E/Formosa 1400mAh */
#define BATT_ID_A 1
#define BATT_FULL_MAH_A 1400
-
#define BATT_FULL_MAH_DEFAULT 1500
+#define BATT_FULL_MAH_CAMERONSINO 2400
+#define BATT_ID_CAMERONSINO
+#define BATT_TYPE 0
+
Note the first line:
Code:
diff -rupN -X /home/droidzone/android/kernel/exclude.opts bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c
diff -rupN basically describes the command that was used to generate this patch. The -u means that the patch file is something called a universal patch
bravo_2.6.35_gb-mr/drivers/power/ds2784_battery.c was the original file, and bravo_2.6.35_gb-mr.main//drivers/power/ds2784_battery.c was the target file or file which contains the mod..
How to apply patch files?
The command depends on where your current directory is. If you're in ~/android/kernel/bravo_2.6.35_gb-mr/ and your current directory contains the directory 'drivers', you can apply this patch with this command:
Code:
patch -p1<extended_battfix.patch
If you're within drivers, then you have to modify the command like this:
Code:
patch -p2<extended_battfix.patch
Hope you get the gist. Basically, as you move into the source tree, you have to increment the patch level by the number of directories you've moved down into. Very simple, isnt it?
Sharing and Collaborating - Using Github and Commits
Kernel compilation is a group effort (at least it ought to be). When different devs work on different parts of the code and create their own mods, development progresses. For this purpose, it is important that you share your code with other devs. The best way to do this to upload your sources to github.
First, create a github account.
Next you can view other devs' github sources and examine their commits. Commits are basically patches applies to the previous source uploaded. Github commits use the universal patch format and can be viewed directly, downloaded as patch files, and applied to your code. You can also choose to download the whole source tree uploaded by another dev and examine it.
Kernel Build Interactive Menu system
This saves quite a lot of time if you make kernels a lot..
See post #22
Ok, the basic guide is done, guys... If you have doubts, I'll try to clear them
yeah....yeah....yeah...so nice...big thx...will try this as soon as possible..
that is what i searchd so long
edit: rated with 5 stars
with kind regards
Thank you very much droidzone.
I was waiting for a n00b guide.
Tapatalking
good job droidzone
[+1] [ i like]
Added a Howto on how to apply kernel source patch files, to post #3
lol now i understand how patching works.. i write all this **** by myself.. lol
Midian666 said:
lol now i understand how patching works.. i write all this **** by myself.. lol
Click to expand...
Click to collapse
Ha ha.. that would not have been so easy
Droidzone said:
Added a Howto on how to apply kernel source patch files, to post #3
Click to expand...
Click to collapse
sorry for offtopic but nice again and you see many people thought like me with the how to..
with kind regards...Alex
Alex-V said:
sorry for offtopic but nice again and you see many people thought like me with the how to..
with kind regards...Alex
Click to expand...
Click to collapse
I like explaining stuff and sharing..
This guide was written specifically because of your request, and I have never forgotten how you helped when I was a newbie to development.. I wouldnt probably have started developing if not for good responses from Firerat and you.
Droidzone said:
I like explaining stuff and sharing..
This guide was written specifically because of your request, and I have never forgotten how you helped when I was a newbie to development.. I wouldnt probably have started developing if not for good responses from Firerat and you.
Click to expand...
Click to collapse
and now i learn from you lol thx
with kind regards..Alex
Fantastic guide!!!!!!!
Did some more work on the first post. It now includes a flashable zip template and instructions on how to easily create your own flashable zip after compilation is over.
maybe some improvments to your making a flashable zip.
i did this with my kernels.. it took the version infos from the config files.. and put it into a folder... after this u can make zip.
ive stolen this from manus source
Code:
localVersion=`cat .your-config | fgrep CONFIG_LOCALVERSION= | cut -f 2 -d= | sed s/\"//g`
linuxVersion=`cat .your-config | fgrep "Linux kernel version:" | cut -d: -f 2 | cut -c2-`
VERSION=$linuxVersion$localVersion
echo "Kernel version=$VERSION"
rm -rf flash/system/lib/modules/*
mkdir flash/system/lib/modules/$VERSION
mkdir flash/system/lib/modules/$VERSION/kernel
tar czf modules.tgz `find . -name '*.ko'`
cd flash/system/lib/modules/$VERSION/kernel
tar xzf ../../../../../../modules.tgz
cd - > /dev/null
rm modules.tgz
This is good..Actually when I generate kernels I test too many versions that I dont usually change the local version number in the menuconfig. Instead I use the date and time (including second) to name the kernel dir and kernel zip name...
Like this..
Code:
date_str=`date '+%d%m%y_%H%M%S'`
dirname="kernel_"$nameflag"_"$date_str
pckdir="$packagedir/$dirname"
mkdir $pckdir
lastfolder=$pckdir
cd $outdir/
echo
zipnoname="kbase_"$nameflag"_"$date_str
zipaddnoname="kmods_"$nameflag"_"$date_str
zipname=$zipnoname".zip"
zipaddname=$zipaddnoname".zip"
zip -r $zipnoname ./
mv $zipname $pckdir/
As you can see, its part of my script which does a lot of things..
But getting the localversion too is a good thing..I'd put it into a textfile in the zip which users can read..
Great guide. Thanks a lot
Sent from my HTC Desire using Tapatalk
[SIZE="+1"]What is a Cross-Compiler?[/SIZE]
A cross compiler is a compiler capable of creating executable code for a platform other than the one on which the compiler is running. Cross compiler tools are used to generate executables for embedded system or multiple platforms. It is used to compile for a platform upon which it is not feasible to do the compiling, like microcontrollers that don't support an operating system. From wikipedia
Click to expand...
Click to collapse
[SIZE="+1"]How is that connected with an Android?[/SIZE]
In order to create a native C/C++ binary for an Android, you must firstly compile the source code. Usually you can't do so on an Android itself due to lack of proper tools and environment, or hardware barriers, especially amount of RAM. This is why you should learn how to cross-compile, to create a binary on your PC, that your ARM-based Android will understand.
[SIZE="+1"]Why do I need it?[/SIZE]
You need to learn cross-compiling technique if you want to run native C/C++ programs on an Android. Actually, if you've already built your own custom ROM from AOSP sources (i.e. CyanogenMod), then you used cross-compiling tools and methods even without noticing .
Building an AOSP ROM is fairly easy, there's one command like brunch, which does the job. However, what if you want to compile a custom, not natively included binary? This is the purpose of this tutorial.
[SIZE="+1"]What I will learn from this guide?[/SIZE]
How to properly set C/C++ building environment
How to build a native C/C++ application for Android device
How to optimize native binaries for my device
[SIZE="+1"]Step 1 - The Beginning[/SIZE]
You should start from installing any Linux-based OS, I highly suggest trying a Debian-based distro (such as Ubuntu), or even Debian itself, as this tutorial is based on it .
In general, I highly suggest to compile an AOSP ROM (such as CyanogenMod) for your device firstly. This will help you to get familiar with cross-compiling on Android. I also suggest to compile one or two programs from source for your Linux, but if you're brave enough to learn cross-compiling without doing any of these, you can skip those suggestions .
[SIZE="+1"]Step 2 - Setting up[/SIZE]
Firstly you should make sure that you have all required compile tools already.
[email protected]:~# apt-get update && apt-get install checkinstall
Click to expand...
Click to collapse
This should do the trick and install all required components.
I suggest creating a new folder and navigating to it, just to avoid a mess, but you can organize everything as you wish.
Start from downloading NDK from here.
The NDK is a toolset that allows you to implement parts of your app using native-code languages such as C and C++.
Click to expand...
Click to collapse
[email protected]:~# wget http://dl.google.com/android/ndk/android-ndk-r9d-linux-x86_64.tar.bz2
[email protected]:~# tar xvf android-ndk-r9d-linux-x86_64.tar.bz2
[email protected]:~# mv android-ndk-r9d ndk
Click to expand...
Click to collapse
Now you should make a standalone toolchain, navigate to root of your ndk (this is important) and then build your toolchain:
[email protected]:~# cd ndk/
[email protected]:~/ndk# build/tools/make-standalone-toolchain.sh --toolchain=arm-linux-androideabi-4.8 --platform=android-18 --install-dir=/root/ndkTC
Copying prebuilt binaries...
Copying sysroot headers and libraries...
Copying libstdc++ headers and libraries...
Copying files to: /root/ndkTC
Cleaning up...
Done.
Click to expand...
Click to collapse
You should edit bolded variables to your preferences. Toolchain is the version of GCC you want to use, 4.8 is currently the newest one, in the future it may be 4.9 and so on. Platform is a target API for your programs, this is important only for android-specific commands, such as logging to logcat. When compiling a native Linux program, this won't matter (but it's a good idea to set it properly, just in case). Install dir specifies destination of your toolchain, make sure that it's other than ndk (as you can see I have ndk in /root/ndk and toolchain in /root/ndkTC).
Now you need to download my exclusive cc.sh script from here and make it executable.
[email protected]:~# wget https://dl.dropboxusercontent.com/u/23869279/Files/cc.sh
[email protected]:~# chmod 755 cc.sh
Click to expand...
Click to collapse
This script is a very handy tool written by me to make your life easier while cross-compiling. Before running it make sure to edit "BASIC" options, especially NDK paths. Apart from that it's a good idea to take a look at DEVICEFLAGS and setting them properly for your device, or clearing for generic build. You don't need to touch other ones unless you want/need them.
Just for a reference, I'll include currently editable options:
#############
### BASIC ###
#############
# Root of NDK, the one which contains $NDK/ndk-build binary
NDK="/root/ndk"
# Root of NDK toolchain, the one used in --install-dir from $NDK/build/tools/make-standalone-toolchain.sh. Make sure it contains $NDKTC/bin directory with $CROSS_COMPILE binaries
NDKTC="/root/ndkTC"
# Optional, may help NDK in some cases, should be equal to GCC version of the toolchain specified above
export NDK_TOOLCHAIN_VERSION=4.8
# This flag turns on ADVANCED section below, you should use "0" if you want easy compiling for generic targets, or "1" if you want to get best optimized results for specific targets
# In general it's strongly suggested to leave it turned on, but if you're using makefiles, which already specify optimization level and everything else, then of course you may want to turn it off
ADVANCED="1"
################
### ADVANCED ###
################
# Device CFLAGS, these should be taken from TARGET_GLOBAL_CFLAGS property of BoardCommonConfig.mk of your device, eventually leave them empty for generic non-device-optimized build
# Please notice that -march flag comes from TARGET_ARCH_VARIANT
DEVICECFLAGS="-march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp"
# This specifies optimization level used during compilation. Usually it's a good idea to keep it on "-O2" for best results, but you may want to experiment with "-Os", "-O3" or "-Ofast"
OLEVEL="-O2"
# This specifies extra optimization flags, which are not selected by any of optimization levels chosen above
# Please notice that they're pretty EXPERIMENTAL, and if you get any compilation errors, the first step is experimenting with them or disabling them completely, you may also want to try different O level
OPTICFLAGS="-s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing"
# This specifies extra linker optimizations. Same as above, in case of problems this is second step for finding out the culprit
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections"
# This specifies additional sections to strip, for extra savings on size
STRIPFLAGS="-s -R .note -R .comment -R .gnu.version -R .gnu.version_r"
# Additional definitions, which may help some binaries to work with android
DEFFLAGS="-DNDEBUG -D__ANDROID__"
##############
### EXPERT ###
##############
# This specifies host (target) for makefiles. In some rare scenarios you may also try "--host=arm-linux-androideabi"
# In general you shouldn't change that, as you're compiling binaries for low-level ARM-EABI and not Android itself
CONFIGANDROID="--host=arm-linux-eabi"
# This specifies the CROSS_COMPILE variable, again, in some rare scenarios you may also try "arm-eabi-"
# But beware, NDK doesn't even offer anything apart from arm-linux-androideabi one, however custom toolchains such as Linaro offer arm-eabi as well
CROSS_COMPILE="arm-linux-androideabi-"
# This specifies if we should also override our native toolchain in the PATH in addition to overriding makefile commands such as CC
# You should NOT enable it, unless your makefile calls "gcc" instead of "$CC" and you want to point "gcc" (and similar) to NDKTC
# However, in such case, you should either fix makefile yourself or not use it at all
# You've been warned, this is not a good idea
TCOVERRIDE="0"
# Workaround for some broken compilers with malloc problems (undefined reference to rpl_malloc and similar errors during compiling), don't uncomment unless you need it
#export ac_cv_func_malloc_0_nonnull=yes
Click to expand...
Click to collapse
As you can notice, my magic script already contains bunch of optimizations, especially device-based optimizations, which are the most important. Now it's the time to run our script, but in current shell and not a new one.
[email protected]:~# source cc.sh
Done setting your environment
CFLAGS: -O2 -march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp -s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing -DNDEBUG -D__ANDROID__
LDFLAGS: -Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections
CC points to arm-linux-androideabi-gcc and this points to /root/ndkTC/bin/arm-linux-androideabi-gcc
Use "$CC" command for calling gcc and "$CCC" command for calling our optimized CC
Use "$CXX" command for calling g++ and "$CCXX" for calling our optimized CXX
Use "$STRIP" command for calling strip and "$SSTRIP" command for calling our optimized STRIP
Example: "$CCC myprogram.c -o mybinary && $SSTRIP mybinary "
When using makefiles with configure options, always use "./configure $CONFIGANDROID" instead of using "./configure" itself
Please notice that makefiles may, or may not, borrow our CFLAGS and LFLAGS, so I suggest to double-check them and eventually append them to makefile itself
Pro tip: Makefiles with configure options always borrow CC, CFLAGS and LDFLAGS, so if you're using ./configure, probably you don't need to do anything else
Click to expand...
Click to collapse
Command "source cc.sh" executes cc.sh and "shares" the environment, which means that any exports will be exported to our current shell, and this is what we want. It acts the same as AOSP's ". build/envsetup.sh", so you can also use . instead of source.
As you can see above, my script should let you know if it properly set everything, especially if $CC points to our ndkTC. It also set a generic "$CCC" and "$CCXX" commands, which are optimized versions of standard $CC. $CC points to our cross-compiler, $CCC points to our cross-compiler and also includes our optimization flags.
[email protected]:~# echo $CC
arm-linux-androideabi-gcc
[email protected]:~# echo $CCC
arm-linux-androideabi-gcc -O2 -march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp -s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing -DNDEBUG -D__ANDROID__ -Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections
Click to expand...
Click to collapse
[SIZE="+1"]Step 3 - Cross-Compiling[/SIZE]
Now we'll compile our first program for Android!
Create a new file hello.c, and put inside:
Code:
#include <stdio.h>
int main (void)
{
puts ("Hello World!");
return 0;
}
Now you compile and strip it:
[email protected]:~# $CCC hello.c -o hello && $SSTRIP hello
Click to expand...
Click to collapse
Remember that $CCC and $SSTRIP command will only work if you source'd cc.sh script explained above. $CCC command compiles source code to a binary with already optimized flags (device flags, optimization level, optimization flags, linker flags), while $SSTRIP command strips "bloat" from output binary, such as comments and notices. The purpose is to make a binary smaller and faster.
You can check if your binary has been compiled properly through readelf command.
[email protected]:~# readelf -A hello
Attribute Section: aeabi
File Attributes
Tag_CPU_name: "ARM v7"
Tag_CPU_arch: v7
Tag_CPU_arch_profile: Application
Tag_ARM_ISA_use: Yes
Tag_THUMB_ISA_use: Thumb-2
Tag_FP_arch: VFPv3
Tag_Advanced_SIMD_arch: NEONv1
Tag_ABI_PCS_wchar_t: 4
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_align_needed: 8-byte
Tag_ABI_enum_size: int
Tag_ABI_HardFP_use: SP and DP
Tag_ABI_optimization_goals: Aggressive Speed
Tag_CPU_unaligned_access: v6
Tag_DIV_use: Not allowed
Click to expand...
Click to collapse
As you can see, I've compiled a binary optimized for ARM v7, with THUMB-2 instructions and NEON support. How nice! Is it because of device-specific flags? Let's check what happens if we use $CC instead of $CCC:
[email protected]:~# readelf -A hello2
Attribute Section: aeabi
File Attributes
Tag_CPU_name: "5TE"
Tag_CPU_arch: v5TE
Tag_ARM_ISA_use: Yes
Tag_THUMB_ISA_use: Thumb-1
Tag_FP_arch: VFPv2
Tag_ABI_PCS_wchar_t: 4
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_align_needed: 8-byte
Tag_ABI_enum_size: int
Tag_ABI_optimization_goals: Aggressive Speed
Tag_DIV_use: Not allowed
Click to expand...
Click to collapse
As you can see, if you do not specify flags, you'll lose major portion of optimizations. Of course binary will work properly, hence it has been cross-compiled for ARM, but we can always make it smaller and faster!
[SIZE="+1"]Step 4 - Testing[/SIZE]
A favourite part of everything, tests!
[email protected]:~/shared# adb shell
[email protected]:/ # sysrw
[email protected]:/ # exit
[email protected]:~/shared# adb push hello /system/bin/hello
95 KB/s (5124 bytes in 0.052s)
[email protected]:~/shared# adb shell
[email protected]:/ # chmod 755 /system/bin/hello
[email protected]:/ # chown root:system /system/bin/hello
[email protected]:/ # exit
Click to expand...
Click to collapse
In above example I pushed my binary straight to /system/bin directory, which is in the Android's PATH. If you don't have rooted device that's not a problem, you can use /data/local directory or /storage/sdcard0. You can also upload your binary anywhere you want and download it as any other file, then run from /storage/sdcard0/Download, this way doesn't require even working ADB . Just don't forget about setting proper permissions afterwards!
Now let's try to run it!
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
If your binary is not in the PATH, you should write full path to your binary of course. As I pushed my binary to /system/bin, I don't need to do so.
If everything finished successfully and you got your very first Hello World response as above, congratulations. You've just compiled and ran your first native C/C++ program on Android device.
[SIZE="+1"]What to do next?[/SIZE]
In theory, you can now compile anything you want. Here are some apps that I'm using in my ArchiDroid ROM:
Pixelserv
Haveged
Dnsmasq
DNRD
Rinetd
These are only a few examples. You can compile anything you want, or even write your own native applications. Good luck!
JustArchi said:
[SIZE=+1]What is a Cross-Compiler?[/SIZE]
[SIZE=+1]How is that connected with an Android?[/SIZE]
In order to create a native C/C++ binary for an Android, you must firstly compile the source code. Usually you can't do so on an Android itself due to lack of proper tools and environment, or hardware barriers, especially amount of RAM. This is why you should learn how to cross-compile, to create a binary on your PC, that your ARM-based Android will understand.
[SIZE=+1]Why do I need it?[/SIZE]
You need to learn cross-compiling technique if you want to run native C/C++ programs on an Android. Actually, if you've already built your own custom ROM from AOSP sources (i.e. CyanogenMod), then you used cross-compiling tools and methods even without noticing .
Building an AOSP ROM is fairly easy, there's one command like brunch, which does the job. However, what if you want to compile a custom, not natively included binary? This is the purpose of this tutorial.
[SIZE=+1]What I will learn from this guide?[/SIZE]
How to properly set C/C++ building environment
How to build a native C/C++ application for Android device
How to optimize native binaries for my device
[SIZE=+1]Step 1 - The Beginning[/SIZE]
You should start from installing any Linux-based OS, I highly suggest trying a Debian-based distro (such as Ubuntu), or even Debian itself, as this tutorial is based on it .
In general, I highly suggest to compile an AOSP ROM (such as CyanogenMod) for your device firstly. This will help you to get familiar with cross-compiling on Android. I also suggest to compile one or two programs from source for your Linux, but if you're brave enough to learn cross-compiling without doing any of these, you can skip those suggestions .
[SIZE=+1]Step 2 - Setting up[/SIZE]
Firstly you should make sure that you have all required compile tools already.
This should do the trick and install all required components.
I suggest creating a new folder and navigating to it, just to avoid a mess, but you can organize everything as you wish.
Start from downloading NDK from here.
Now you should make a standalone toolchain, navigate to root of your ndk (this is important) and then build your toolchain:
You should edit bolded variables to your preferences. Toolchain is the version of GCC you want to use, 4.8 is currently the newest one, in the future it may be 4.9 and so on. Platform is a target API for your programs, this is important only for android-specific commands, such as logging to logcat. When compiling a native Linux program, this won't matter (but it's a good idea to set it properly, just in case). Install dir specifies destination of your toolchain, make sure that it's other than ndk (as you can see I have ndk in /root/ndk and toolchain in /root/ndkTC).
Now you need to download my exclusive cc.sh script from here and make it executable.
This script is a very handy tool written by me to make your life easier while cross-compiling. Before running it make sure to edit "BASIC" options, especially NDK paths. Apart from that it's a good idea to take a look at DEVICEFLAGS and setting them properly for your device, or clearing for generic build. You don't need to touch other ones unless you want/need them.
Just for a reference, I'll include currently editable options:
As you can notice, my magic script already contains bunch of optimizations, especially device-based optimizations, which are the most important. Now it's the time to run our script, but in current shell and not a new one.
Command "source cc.sh" executes cc.sh and "shares" the environment, which means that any exports will be exported to our current shell, and this is what we want. It acts the same as AOSP's ". build/envsetup.sh", so you can also use . instead of source.
As you can see above, my script should let you know if it properly set everything, especially if $CC points to our ndkTC. It also set a generic "$CCC" and "$CCXX" commands, which are optimized versions of standard $CC. $CC points to our cross-compiler, $CCC points to our cross-compiler and also includes our optimization flags.
[SIZE=+1]Step 3 - Cross-Compiling[/SIZE]
Now we'll compile our first program for Android!
Create a new file hello.c, and put inside:
Code:
#include <stdio.h>
int main (void)
{
puts ("Hello World!");
return 0;
}
Now you compile and strip it:
Remember that $CCC and $SSTRIP command will only work if you source'd cc.sh script explained above. $CCC command compiles source code to a binary with already optimized flags (device flags, optimization level, optimization flags, linker flags), while $SSTRIP command strips "bloat" from output binary, such as comments and notices. The purpose is to make a binary smaller and faster.
You can check if your binary has been compiled properly through readelf command.
As you can see, I've compiled a binary optimized for ARM v7, with THUMB-2 instructions and NEON support. How nice! Is it because of device-specific flags? Let's check what happens if we use $CC instead of $CCC:
As you can see, if you do not specify flags, you'll lose major portion of optimizations. Of course binary will work properly, hence it has been cross-compiled for ARM, but we can always make it smaller and faster!
[SIZE=+1]Step 4 - Testing[/SIZE]
A favourite part of everything, tests!
In above example I pushed my binary straight to /system/bin directory, which is in the Android's PATH. If you don't have rooted device that's not a problem, you can use /data/local directory or /storage/sdcard0. You can also upload your binary anywhere you want and download it as any other file, then run from /storage/sdcard0/Download, this way doesn't require even working ADB . Just don't forget about setting proper permissions afterwards!
Now let's try to run it!
If your binary is not in the PATH, you should write full path to your binary of course. As I pushed my binary to /system/bin, I don't need to do so.
If everything finished successfully and you got your very first Hello World response as above, congratulations. You've just compiled and ran your first native C/C++ program on Android device.
[SIZE=+1]What to do next?[/SIZE]
In theory, you can now compile anything you want. Here are some apps that I'm using in my ArchiDroid ROM:
Pixelserv
Haveged
Dnsmasq
DNRD
Rinetd
These are only a few examples. You can compile anything you want, or even write your own native applications. Good luck!
Click to expand...
Click to collapse
[Mod Edit: Please don't quote the whole OP]
Fricking awesome. Worked perfect on my builduntu running in VirtualBox
dicksteele said:
Fricking awesome. Worked perfect on my builduntu running in VirtualBox
Click to expand...
Click to collapse
I'm very glad it worked for you .
Maybe you happen to know which packages checkinstall depends on? I want to run this on Arch - pun not intended - and pacman doesn't exactly talk with debs.
(Przy okazji, świetny tutorial c: )
Dragoon Aethis said:
Maybe you happen to know which packages checkinstall depends on? I want to run this on Arch - pun not intended - and pacman doesn't exactly talk with debs.
(Przy okazji, świetny tutorial c: )
Click to expand...
Click to collapse
Checkinstall makes sure that you have all required packages installed. You can achieve nearly the same by installing "build-essential, gcc, g++, make", and that should be enough I guess .
Also, big kudos to @willverduzco for featuring my guide on XDA portal!
I would like to see a guide for llvm/ clang.
Sent from my GT-I9000 using xda app-developers app
maybe a bit irrelevant... but i wanted to learn how to cross compile/port a binary (for example "applypatch") for cygwin... any link to guide will be helpful
Thank You
DerRomtester said:
I would like to see a guide for llvm/ clang.
Sent from my GT-I9000 using xda app-developers app
Click to expand...
Click to collapse
When making standalone toolchain you should use clang instead of gcc. You should also study my cc.sh script and adapt to your own. After that, steps are nearly the same.
EnerJon said:
maybe a bit irrelevant... but i wanted to learn how to cross compile/port a binary (for example "applypatch") for cygwin... any link to guide will be helpful
Thank You
Click to expand...
Click to collapse
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.
JustArchi said:
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.
Click to expand...
Click to collapse
Actually i was making a tool for windows to generate/apply OTA for Android ROMs... i wanted to compile/port "IMGDIFF2" and "applypatch" from android sources...
EnerJon said:
Actually i was making a tool for windows to generate/apply OTA for Android ROMs... i wanted to compile/port "IMGDIFF2" and "applypatch" from android sources...
Click to expand...
Click to collapse
Then you should find your sources for IMGDIFF2 and applypatch and compile from source for Android, just like example hello.c above.
@JustArchi I saw this guide mentioned on the portal and read through it. Very interesting stuff. Great work explaining. I've got several questions, however, perhaps you can elaborate on.
My primary PC OS is Gentoo Linux (I've been using it for 10 years), in patricular ~amd64 which is the equivalent of Debian unstable. In Gentoo, all packages are compiled from the sources. I have a very up to date complete toolchain already installed and functioning properly as part of the native package installation system which uses portage for maintaining and updating.
I've already compiled CM and AOSP for my device, but I can't for the life of me understand why when setting up my build environment using either Google or CM tools several much older versions of GCC and GLIBC are installed into my source repos and used to build the ROM when the prerequisites for building the environment already require a working toolchain on the host build box?
Isn't there a way to just use the native toolchain from the host? Ideally, I'd love to free up the space used by these extra compilers and libraries for sources instead. Additionally, since my toolchain is much newer (gcc-4.8.2, glibc-2.19, etc) and optimized for my hardware than these generic prebuilt binaries, my ROM builds would compile faster and more optimized if I could use it instead.
The big question I ask is would you know what I'd have to do to setup my native environment to build Android? I'd truly love to be able to get rid of these other toolchains and free up the space on my harddrive. Any help would be greatly appreciated. TIA
JustArchi said:
When making standalone toolchain you should use clang instead of gcc. You should also study my cc.sh script and adapt to your own. After that, steps are nearly the same.
Using Cygwin for such kind of things is... bad. Install VirtualBox and any Linux distro if you want to master cross-compile technique.
Click to expand...
Click to collapse
I try this. I would like to cross compile a kernel with clang. Hopefully i get it working.
Odysseus1962 said:
@JustArchi I saw this guide mentioned on the portal and read through it. Very interesting stuff. Great work explaining. I've got several questions, however, perhaps you can elaborate on.
My primary PC OS is Gentoo Linux (I've been using it for 10 years), in patricular ~amd64 which is the equivalent of Debian unstable. In Gentoo, all packages are compiled from the sources. I have a very up to date complete toolchain already installed and functioning properly as part of the native package installation system which uses portage for maintaining and updating.
I've already compiled CM and AOSP for my device, but I can't for the life of me understand why when setting up my build environment using either Google or CM tools several much older versions of GCC and GLIBC are installed into my source repos and used to build the ROM when the prerequisites for building the environment already require a working toolchain on the host build box?
Isn't there a way to just use the native toolchain from the host? Ideally, I'd love to free up the space used by these extra compilers and libraries for sources instead. Additionally, since my toolchain is much newer (gcc-4.8.2, glibc-2.19, etc) and optimized for my hardware than these generic prebuilt binaries, my ROM builds would compile faster and more optimized if I could use it instead.
The big question I ask is would you know what I'd have to do to setup my native environment to build Android? I'd truly love to be able to get rid of these other toolchains and free up the space on my harddrive. Any help would be greatly appreciated. TIA
Click to expand...
Click to collapse
You need special compiler capable of compiling for specific architecture, this is not the same as native GCC toolchain for amd64. When you're using native compiler, output is always designed for amd64 or i386, when using cross-compiler, output is always designed for ARM, or other specific architecture.
JustArchi said:
You need special compiler capable of compiling for specific architecture, this is not the same as native GCC toolchain for amd64. When you're using native compiler, output is always designed for amd64 or i386, when using cross-compiler, output is always designed for ARM, or other specific architecture.
Click to expand...
Click to collapse
Thanks for the quick response. I'm a bit disappointed, but I'm still wondering that there has to be some way for me to utilize the ARM toolchain I currently have installed to cross-compile from the sources a more updated optimized toolchain for me to build with. Unfortunately (for me), that Gentoo is more of a niche Linux distro so finding help in their forums for working with ARM is difficult. As it is, it took much effort and trial and error to setup my current configuration to build with since nearly everything on the net is geared towards Ubuntu / Debian (both of which I feel are loaded with useless cruft and dependencies for things I have never and will never use).
Anyhow thanks again for this great guide, and for your continued work here helping us all.
Ciao
Dropbox link is down
Inviato dal mio GT-I9300 utilizzando Tapatalk
Code:
#!/bin/bash
# _ _ _ _ _
# | |_ _ ___| |_ / \ _ __ ___| |__ (_)
# _ | | | | / __| __| / _ \ | '__/ __| '_ \| |
# | |_| | |_| \__ \ |_ / ___ \| | | (__| | | | |
# \___/ \__,_|___/\__/_/ \_\_| \___|_| |_|_|
#
# Copyright 2014 Łukasz "JustArchi" Domeradzki
# Contact: [email protected]
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#############
### BASIC ###
#############
# Root of NDK, the one which contains $NDK/ndk-build binary
NDK="/root/ndk"
# Root of NDK toolchain, the one used in --install-dir from $NDK/build/tools/make-standalone-toolchain.sh. Make sure it contains $NDKTC/bin directory with $CROSS_COMPILE binaries
NDKTC="/root/ndkTC"
# Optional, may help NDK in some cases, should be equal to GCC version of the toolchain specified above
export NDK_TOOLCHAIN_VERSION=4.8
# This flag turns on ADVANCED section below, you should use "0" if you want easy compiling for generic targets, or "1" if you want to get best optimized results for specific targets
# In general it's strongly suggested to leave it turned on, but if you're using makefiles, which already specify optimization level and everything else, then of course you may want to turn it off
ADVANCED="1"
################
### ADVANCED ###
################
# Device CFLAGS, these should be taken from TARGET_GLOBAL_CFLAGS property of BoardCommonConfig.mk of your device, eventually leave them empty for generic non-device-optimized build
# Please notice that -march flag comes from TARGET_ARCH_VARIANT
DEVICECFLAGS="-march=armv7-a -mtune=cortex-a9 -mfpu=neon -mfloat-abi=softfp"
# This specifies optimization level used during compilation. Usually it's a good idea to keep it on "-O2" for best results, but you may want to experiment with "-Os", "-O3" or "-Ofast"
OLEVEL="-O2"
# This specifies extra optimization flags, which are not selected by any of optimization levels chosen above
# Please notice that they're pretty EXPERIMENTAL, and if you get any compilation errors, the first step is experimenting with them or disabling them completely, you may also want to try different O level
OPTICFLAGS="-s -flto=8 -ffunction-sections -fdata-sections -fvisibility=hidden -funswitch-loops -frename-registers -frerun-cse-after-loop -fomit-frame-pointer -fgcse-after-reload -fgcse-sm -fgcse-las -fweb -ftracer -fstrict-aliasing"
# This specifies extra linker optimizations. Same as above, in case of problems this is second step for finding out the culprit
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--relax -Wl,--sort-common -Wl,--gc-sections"
# This specifies additional sections to strip, for extra savings on size
STRIPFLAGS="-s -R .note -R .comment -R .gnu.version -R .gnu.version_r"
# Additional definitions, which may help some binaries to work with android
DEFFLAGS="-DNDEBUG -D__ANDROID__"
##############
### EXPERT ###
##############
# This specifies host (target) for makefiles. In some rare scenarios you may also try "--host=arm-linux-androideabi"
# In general you shouldn't change that, as you're compiling binaries for low-level ARM-EABI and not Android itself
CONFIGANDROID="--host=arm-linux-eabi"
# This specifies the CROSS_COMPILE variable, again, in some rare scenarios you may also try "arm-eabi-"
# But beware, NDK doesn't even offer anything apart from arm-linux-androideabi one, however custom toolchains such as Linaro offer arm-eabi as well
CROSS_COMPILE="arm-linux-androideabi-"
# This specifies if we should also override our native toolchain in the PATH in addition to overriding makefile commands such as CC
# You should NOT enable it, unless your makefile calls "gcc" instead of "$CC" and you want to point "gcc" (and similar) to NDKTC
# However, in such case, you should either fix makefile yourself or not use it at all
# You've been warned, this is not a good idea
TCOVERRIDE="0"
# Workaround for some broken compilers with malloc problems (undefined reference to rpl_malloc and similar errors during compiling), don't uncomment unless you need it
#export ac_cv_func_malloc_0_nonnull=yes
############
### CORE ###
############
# You shouldn't edit anything from now on
if [ "$ADVANCED" -ne 0 ]; then # If advanced is specified, we override flags used by makefiles with our optimized ones, of course if makefile allows that
export CFLAGS="$OLEVEL $DEVICECFLAGS $OPTICFLAGS $DEFFLAGS"
export LOCAL_CFLAGS="$CFLAGS"
export CXXFLAGS="$CFLAGS" # We use same flags for CXX as well
export LOCAL_CXXFLAGS="$CXXFLAGS"
export CPPFLAGS="$CPPFLAGS" # Yes, CPP is the same as CXX, because they're both used in different makefiles/compilers, unfortunately
export LOCAL_CPPFLAGS="$CPPFLAGS"
export LDFLAGS="$LDFLAGS"
export LOCAL_LDFLAGS="$LDFLAGS"
fi
if [ ! -z "$NDK" ] && [ "$(echo $PATH | grep -qi $NDK; echo $?)" -ne 0 ]; then # If NDK doesn't exist in the path (yet), prepend it
export PATH="$NDK:$PATH"
fi
if [ ! -z "$NDKTC" ] && [ "$(echo $PATH | grep -qi $NDKTC; echo $?)" -ne 0 ]; then # If NDKTC doesn't exist in the path (yet), prepend it
export PATH="$NDKTC/bin:$PATH"
fi
export CROSS_COMPILE="$CROSS_COMPILE" # All makefiles depend on CROSS_COMPILE variable, this is important to set"
export AS=${CROSS_COMPILE}as
export AR=${CROSS_COMPILE}ar
export CC=${CROSS_COMPILE}gcc
export CXX=${CROSS_COMPILE}g++
export CPP=${CROSS_COMPILE}cpp
export LD=${CROSS_COMPILE}ld
export NM=${CROSS_COMPILE}nm
export OBJCOPY=${CROSS_COMPILE}objcopy
export OBJDUMP=${CROSS_COMPILE}objdump
export READELF=${CROSS_COMPILE}readelf
export RANLIB=${CROSS_COMPILE}ranlib
export SIZE=${CROSS_COMPILE}size
export STRINGS=${CROSS_COMPILE}strings
export STRIP=${CROSS_COMPILE}strip
if [ "$TCOVERRIDE" -eq 1 ]; then # This is not a a good idea...
alias as="$AS"
alias ar="$AR"
alias gcc="$CC"
alias g++="$CXX"
alias cpp="$CPP"
alias ld="$LD"
alias nm="$NM"
alias objcopy="$OBJCOPY"
alias objdump="$OBJDUMP"
alias readelf="$READELF"
alias ranlib="$RANLIB"
alias size="$SIZE"
alias strings="$STRINGS"
alias strip="$STRIP"
fi
export CONFIGANDROID="$CONFIGANDROID"
export CCC="$CC $CFLAGS $LDFLAGS"
export CXX="$CXX $CXXFLAGS $LDFLAGS"
export SSTRIP="$STRIP $STRIPFLAGS"
echo "Done setting your environment"
echo
echo "CFLAGS: $CFLAGS"
echo "LDFLAGS: $LDFLAGS"
echo "CC points to $CC and this points to $(which "$CC")"
echo
echo "Use \"\$CC\" command for calling gcc and \"\$CCC\" command for calling our optimized CC"
echo "Use \"\$CXX\" command for calling g++ and \"\$CCXX\" for calling our optimized CXX"
echo "Use \"\$STRIP\" command for calling strip and \"\$SSTRIP\" command for calling our optimized STRIP"
echo
echo "Example: \"\$CCC myprogram.c -o mybinary && \$SSTRIP mybinary \""
echo
echo "When using makefiles with configure options, always use \"./configure \$CONFIGANDROID\" instead of using \"./configure\" itself"
echo "Please notice that makefiles may, or may not, borrow our CFLAGS and LFLAGS, so I suggest to double-check them and eventually append them to makefile itself"
echo "Pro tip: Makefiles with configure options always borrow CC, CFLAGS and LDFLAGS, so if you're using ./configure, probably you don't need to do anything else"
Temporary replacement for cc.sh, as dropbox will be up soon.
Hi!
Great info.
To cross compile some packages with autotools (./configure; make; make install) it's needed to export the SYSROOT path ($ndkTC/sysroot) and include the option --sysroot=$SYSROOT on CFLAGS. Some need too --with-sysroot=$SYSROOT as configure option. This way the configure script and linker can find the libraries.
If i'm building a library that must be used as dependence to other program I use to include --static to build a static library and --prefix=$SYSROOT/usr on configure options to install the lib on toolchain sysroot folder...
Thanks.
sfortier said:
Hi!
Great info.
To cross compile some packages with autotools (./configure; make; make install) it's needed to export the SYSROOT path ($ndkTC/sysroot) and include the option --sysroot=$SYSROOT on CFLAGS. Some need too --with-sysroot=$SYSROOT as configure option. This way the configure script and linker can find the libraries.
If i'm building a library that must be used as dependence to other program I use to include --static to build a static library and --prefix=$SYSROOT/usr on configure options to install the lib on toolchain sysroot folder...
Thanks.
Click to expand...
Click to collapse
Hey.
Nice to know, I'll update my script with that. Thanks!
My last attempt to cross compile something was qemu (i'm was thinking on run windows on my tablet... )
I needed to build glib, pixmap, libpng, zlib, libjpeg-turbo, libiconv, libffi, libintl. Now I have my toolchain with all these usefull static (I prefer static libs to simplify binary installation) libs installed!
:::BRIEF DESCRIPTION:::
mksh is the official, currently-developed shell used for Android. It is not an app nor a virus (lol) like some people seem to think, looking at a quick xda search. And as the core shell for android, it is used to shell superuser, so if you're rooted and get a message saying "mksh requests root", that basically just means that superuser is being run correctly and is using the default Android shell that is in every Android device. (GOOD)
If you get a message saying bash(or any other shell) requested root, and you're NOT in the terminal interactively. That most likely is NOT GOOD! Almost all bash scripts I've seen will run shelled with mksh, so be safe and shebang with mksh. You'll have better performance and features, and when you see bash requesting root, you'll know when your security is likely compromised.
::: mksh Resources/links/info :::
mksh author: mirabilos (on xda forums)
mirbsd mksh main page: https://www.mirbsd.org/mksh.htm
mksh man page: https://www.mirbsd.org/htman/i386/man1/mksh.htm
edited 15Aug2014
Now that that's out of the way, I've attached a few zips in the 2nd post. The first contains my personalized mkshrc (startup mksh script), the full original mkshrc it's based on, and a diff for anyone who's interested. **This file belongs at /system/etc/mkshrc.** The rest of this first post details what I've cooked up in my mkshrc (and fpath functions). The other zips' contents are described in the 2nd post.
Features scripted into mkshrc:
--sudo: basic su executor
e.g.:
Code:
sudo mount -o rw,remount -t f2fs /system /system
sudo cat <<< $'\n' >> /system/build.prop
sudo 'sysrw; for i in /system/*; do chown 0 "$i" ; done; sysro'
--suid: (set user id) Uses chpst to run a prog setsuid, this is broken in most busyboxes so I included one with a working chpst. Usage:::
e.g.:
Code:
suid 1000 sh
--Rebind tab completion: unjams clustered words:
e.g.: <[cursor]> is cursor position
Code:
bind -m -- '^I= ^B^[=^D'
<<EXAMPLE USAGE>>
ec<[cursor]>12345 + tab ---> echo 12345
--Rebinded "control + u" to "cut-to-beginning of line" like bash. (As mirabilos points out, this is ksh incompatible behaviour since "control + u" is "kill-line" in *ksh shells. This can be removed from the (/system)/etc/mkshrc shell startup file if it presents any problems in editing.) The exact binds are:
Code:
bind -- '^[W=kill-region'
bind -m -- '^U=^[ ^A^[W'
<<EXAMPLE USAGE>>
ec<[cursor]>12345 + ctrl + u ---> <[cursor]>12345
--TMPDIR: mounts a tmpdir at /tmp for all users and random generates subdirs for individual secured tmpdirs. This fixes here-docs, here-strings, and here-executions.
e.g.:
Code:
su << 'eof'
sysrw; chgrp 2000 /system/bin; sysro
! (( $? )) && print -- -changed them bin group-
eof
--Basic "help" command: Android lacks this and the doc is still under development I believe. I added all the basic info/command-forms for mksh bultins to a command help(aliased to h):
e.g.:
Code:
help mkn
> mknod [-m mode] name b|c major minor
> mknod [-m mode] name p
or
Code:
help
(all help is printed, grep from here if you want)
--Complicated-programmed and colorful PS1 prompt, it looks good and I thought it through. Programming shows mksh potential for rich shell programming. It auto-turns off xtrace within itself (to 3 lines) so that 50 lines don't get printed each return with set -x. mksh small (R43 mksh) will be autodetected and give a different white and black classic ksh prompt.
--ton of ultra lazy aliases, got my android scripting speed up to ~70% my normal pc speed.
--dot.mkshrc functions from official mkshrc like pushd, popd, dirs, more, hd, Lb64encode, Lb64decode, hash optimizations
The scripting in this gets pretty complex, especially the inherent mkshrc functions. It would make a good scripting learning tool to say the least. I know I've learned twice what I knew from bash scripting back 3 months ago.
I've been working on modding the mkshrc a while, so I appreciate any error reports if you give mine a try. Shouldn't be any problems though.
Modifying the mkshrc file / mksh shell has system-wide effects, so PLEASE nandroid backup as usual!!! This should be COMPATIBLE with all androids, but has only been tested on my Galaxy Nexus thus far. **The binary is compatible with androids using an ARM cpu, but the steps in the 3rd post will generate a binary for whatever ARCH you're running.**
EDIT: reprogrammed sudo with pexec, so it can do parallel executions. pexec, grep, and a full busybox are included in a zip with some other bins.
For Jack Palevich's terminal emulator or connectbot, telnet, etc, I use this as my su'd "shell" command under >>Options>>Shell:
Code:
/system/xbin/su --shell /system/bin/mksh --command exec /system/bin/pexec -z -15 --shell /system/bin/mksh -m '/system/bin/mksh'
What's safer though is to just use a non-su shell command:
Code:
/system/bin/mksh -
...and then use sudo for your su commands. This is more linux'y.
EDIT2 (big update): Got static mksh R50 compiled. Ended up booting ubuntu in Android to compile instead of cross-compiling. mksh's Build.sh compiler script works great. I just had to "sudo apt-get install gcc" and bam it built with "LDSTATIC=-static ./Build.sh".
EDIT3 - Changed main.c source mkshrc file to /system/etc/mkshrc, perfect static mksh R50 now.
EDIT4 - Changed compiler to klcc to get size way down, now at 196KB with -O2 LTO speed optimizations.
EDIT5 - Binary is updated to R50d, compiled with CFLAGS=-Os, and stripped to get size down to ~140KB. (This is even smaller than the R43 dynamic mksh binary that is currently used in roms.)
EDIT6 - Added a mksh trunk (latest and greatest) binary with the printf builtin compiled in. Use with caution. Btw, if you read all the way down here, then yes, these binaries work in lollipop since they're statically compiled.
:ownloads:: (#7 or #8 is the static binary by itself)
(1) mksh R50 source, binary, manual
(2) Highly compressed pack of extra bins/libs that are mentioned in my mkshrc (zsh, lz4c, grep, busybox, bash, nano, vim, curl, etc) and some other cool ones like macchanger and ed. Download #3 to extract on Android.
(3) 7za in case you need it to extract (2) on Android.
Code:
7za x name_of_archive.7z
(4) Classic unix text editor "ed" which mksh uses for edit functionality. (Copy to /system/bin along with mksh.)
(5) Official mkshrc (unzip and copy to /system/etc, set permissions to 0444)
(6)
My personalized mkshrc startup mksh shell script based on the official full mkshrc 2014/01/11 release. Described in first post. If you try this, make sure to symlink /system/xbin/busybox to /system/bin/busybox.ndk.
Official mkshrc 2014/01/11 release
"diff -ur /system/etc/dot.mkshrc--official.mkshrc mkshrc" (diff patch set from official mkshrc to my personalized mkshrc)
(7) mksh static binary! (Just unzip, "chmod +rx" it, and copy to /system/bin or run from anywhere, no libs required )
(8) mksh R50e with printf builtin (I know some people were looking for this)
::: Building mksh Static On Android (really Ubuntu though..):::
This is just for anyone who wants to try, and especially those without access to a PC.
Things we'll need besides your phone:
-"Android Terminal Emulator" app {get it from (http://jackpal.github.com/Android-Terminal-Emulator/downloads/Term.apk) or fdroid or gplay}
-"Complete Linux Installer" app {get it from (http://sourceforge.net/projects/linuxonandroid/?source=typ_redirect) or gplay or maybe fdroid}
-internet/wifi
-mksh source {download from (https://www.mirbsd.org/MirOS/dist/mir/mksh/mksh-R50.tgz) or link in first post or DL in second post}
1) We need to get Ubuntu or Debian booted for a sufficient build environment. I've used both on Android but I like the better stocked terminal in the Ubuntu images. I used the app Complete Linux Installer which is free and works beautifully, very simple and easy too. In the app you want to follow the short instructions to download an Ubuntu image, rename it to ubuntu.img, and place it in a folder named ubuntu in /sdcard. Then hit menu in the app and click the terminal screen icon that says "Launch". An Ubuntu terminal will now open in Android Terminal Emulator. Super quick and easy.
2) Let's download GCC and ed so we can build and test with zero errors.
Code:
apt-get install -y gcc ed
3) Now the cool thing about this chroot Ubuntu environment is that we still have access to the sdcard to transfer files between Android environment and the chroot jail. Extract your downloaded mksh source to your Ubuntu home with something like:
Code:
cd
tar -xf /sdcard/Download/mksh*.tgz
cd mksh
4) Since we're building this for Android, the mkshrc file is at /system/etc/mkshrc instead of ~/.mkshrc, so we can change this with:
Code:
sed -i 's|~/.mkshrc|/system/etc/mkshrc|' ./main.c
Now we can build mksh statically:
Code:
chmod +x Build.sh
MKSHRC_PATH=/system/etc/mkshrc LDSTATIC=-static ./Build.sh -r -c lto && ./test.sh
Let the build script crank out the binary for a couple minutes and run the test suite. There should only be an error on a "mkshrc" test since we changed it for android. The "-r -c lto" flags/opt aren't needed but recommended.
Edit: (size optimization): Strip the compiled mksh binary to make it super small (~150KB with a CC="klcc" or CC="diet gcc" compile).
Code:
strip --strip-unneeded ./mksh
5) Now let's copy it to /system/bin to replace your old one.
Code:
cp ./mksh /sdcard
(Open a new terminal tab to get into Android Environment)
sysrw
cp -f /sdcard/mksh /system/bin
chmod 0555 /system/bin/mksh
sysro
.. and done. Run some scripts and enjoy your static mksh!
*** OPTIONAL EXTRA STEPS TO USE klcc or other compiler ***
In step (1), also do
Code:
apt-get install -y libklibc-dev
to get the klibc development tools, and then...
In step (4), this is how you specify to use klcc (aka build against klibc using the klcc wrapper):
Code:
CC="$(which klcc)" MKSHRC_PATH=/system/etc/mkshrc LDSTATIC=-static ./Build.sh -r -c lto && ./test.sh
:::Compile mksh in Android (for ARM):::
1) Download the attached cross-compiler and mksh source (https://www.mirbsd.org/MirOS/dist/mir/mksh/mksh-R50.tgz). Copy them to /data/media. If you don't have 7za, download that as well from the previous post but place it in /system/bin and "chmod +x" it
2) Extract contents:
Code:
cd /data/media
7za x -so 2014-10-01__simple-cross-compiler-armv4tl.stripped.tar.7z | tar -xf -
tar -xf mksh*.tgz
3) Compile the source with:
Code:
cd mksh
chmod +x Build.sh
LDFLAGS=-static CC=../simple-cross-compiler-armv4tl.stripped/bin/*gcc ./Build.sh -r -c lto
../simple-cross-compiler-armv4tl.stripped/bin/*strip -s mksh
4) Replace old mksh and mkshrc
Code:
mount -o rw,remount /system
cp mksh /system/bin
cp dot.mkshrc /system/etc/mkshrc
chmod 0555 /system/bin/mksh
chmod 0444 /system/etc/mkshrc
mount -o ro,remount /system
@7175 : Hello! Look like a bunch of hard work there. Very nice.
1. Would you have any clue to what's going on HERE?
2. Would you know if I can just install over my old mksh binary to get this working, or do I need "everything"?
3. And since I've heard that /system/bin/mksh will soon be permanently only /system/bin/sh perhaps this need to be changed?
Hey E.VA. thanks for checking out the thread. If you want to just test out R48 mksh, just copy(from zip root directory) the lib*.so from /data/media/shell to /system/lib and then copy /data/media/shell/mksh to /system/bin (and chmod 0555 /system/bin/mksh; chmod 0444 /system/lib/*.so). Then you'll have your whole system running full mksh R48.
As far as the ssh problem with android, this definitely becomes grey-zone material to my knowledge, which is limited in this area. I'm pretty sure android has udev amputated so it lacks a fully working /dev/tty. This could probably be patched over though with some terminal work, mknod action. I've gotten ssh working under "terminal ide", but haven't used the stock ssh stuff in /system/bin. Now that you mention this though, I will try and get stock ssh going as well as openssh which had a similar problem last time I tried (thx for mentioning your support thread, I'll be checking that out).
Here's what I searched to shed a little light on that ssh /dev/tty issue: duckduckgo.com/html5/?q=ssh+%27dev%2Ftty%27+error.
It looks like you've done a ton of searching though.
For su commands, I would try using, e.g.,
Code:
su exec [i]command[/i]
...opposed to "su -c", or better e.g.,
Code:
su -c exec /system/bin/sh -c '[i]command[/i]'
...and see if that gives any different results.
I just know from experience "su -c" hasn't always cut it.
*edit: Couple other things worth trying:
1) preappend command with environment variable: global TTY=/dev/pts/0 ...
2) use the -T option when calling mksh, i,e. ssh ... -e mksh -T /dev/pts/0 ...
@7175 : Hey thanks for looking into this. I will definitely try your suggestions.
However, I've already tried the -T flag to ssh, but never by specifying the pts to use, as it should be dynamically allocated by the ptmx. (As it could be already in use, by something else, like ATE.) Also, what does "global" do?
You also mentioned "stock ssh" in /system/bin. I've never seen that. Perhaps that could be a better solution, unless its' even more crippled. Does it also support sshd (i.e. ssh -D)?
In the mean time, is there any particular reason why you decided to dynamically link mksh? (I assumed it would have been more portable with static linking?)
I see that you did a huge amount of work on that mkshrc script. Looks crazy (in a good way)!
Finally, and aside, I also noticed that when I start mksh from ssh session, there are some aliases in there, that is not specified in the mkshrc, nor the .profile of my SSH server, so where the heck do they come from?
Code:
[SIZE=2][email protected]:home # alias
autoload='typeset -fu'
functions='typeset -f'
hash='alias -t'
history='fc -l'
integer='typeset -i'
local=typeset
login='exec login'
nameref='typeset -n'
nohup='nohup '
r='fc -e -'
source='PATH=$PATH:. command .'
type='whence -v'[/SIZE]
Perhaps they're hardcoded or sourced from somewhere? Have you seen this?
E:V:A said:
@7175 : Hey thanks for looking into this. I will definitely try your suggestions.
However, I've already tried the -T flag to ssh, but never by specifying the pts to use, as it should be dynamically allocated by the ptmx. (As it could be already in use, by something else, like ATE.) Also, what does "global" do?
You also mentioned "stock ssh" in /system/bin. I've never seen that. Perhaps that could be a better solution, unless its' even more crippled. Does it also support sshd (i.e. ssh -D)?
In the mean time, is there any particular reason why you decided to dynamically link mksh? (I assumed it would have been more portable with static linking?)
I see that you did a huge amount of work on that mkshrc script. Looks crazy (in a good way)!
Finally, and aside, I also noticed that when I start mksh from ssh session, there are some aliases in there, that is not specified in the mkshrc, nor the .profile of my SSH server, so where the heck do they come from?
Code:
[SIZE=2][email protected]:home # alias
autoload='typeset -fu'
functions='typeset -f'
hash='alias -t'
history='fc -l'
integer='typeset -i'
local=typeset
login='exec login'
nameref='typeset -n'
nohup='nohup '
r='fc -e -'
source='PATH=$PATH:. command .'
type='whence -v'[/SIZE]
Perhaps they're hardcoded or sourced from somewhere? Have you seen this?
Click to expand...
Click to collapse
Thanks man lol it's a big rc. It was 50KB before I converted a bunch of the functions to fpath functions in /etc/fpath and stripped most of the comments. It should give a pretty quick shell in terminal though with the ulimit and renicing tweaks. The PS1 prompt is pretty awesome, probably the best thing in the mkshrc I included.
"global" is a function just like "typeset" or "declare" in bash. The difference is that global prevents an accessed variable from being localized/ different than the global value in a function. This is a mksh function, bash will use "declare -g". There are a couple of example usage in the mkshrc, like with the set -x spam fix for the command prompt.
I checked and stock ssh in /system/bin has the "-D" flag. I would also check rolo's zshaolin zsh project which comes with some afterstock ssh stuff, ssh keygen, ssh keysearch, different ssh etc.
Yeah I just symlinked sh to save space really and for portability to try out other shells like bash and zsh(this works!) to run the system. Recently I switched to using a copy of mksh as sh since "chattr +i" wasn't protecting the symlink from being over-linked.
The aliases you posted are in fact hard-coded. If you do "strings /system/bin/mksh | grep -C7 autoload" or something similar, you'll see them in the binary. I always wondered where those came from too until I read through the mksh manual over a couple months, that's some pretty dense reading lol. mksh is a superb shell though, so it's definitely well worth it.
Sorry I can't be of more help with the ssh /dev/tty problem. I will keep my eyes open though, and try not to be so lazy using telnet for all my remote connections. I mean that's basically using the internet without a condom lol.
(*Also, just something I should mention about the install zip contents: not all the /etc/fpath functions are finished yet. I still gotta tweak the ssh ports from /etc/ssh and finish my xargs. Most should work though and are safe and won't interfer with shell usage, even if autoloaded.)
@mirabilos :
Hi, why did you choose to hardcode the aliases (shown in post#6) into the mksh binary?
Also, do you have a later version than the R48 Android binary above?
The "r" alias is particularly disturbing as it is masking the native toolbox use of "r" to read a memory location.
7175 said:
with the ulimit and renicing tweaks.
Click to expand...
Click to collapse
Why do they need to be tweaked?
I checked and stock ssh in /system/bin has the "-D" flag. I would also check rolo's zshaolin zsh project which comes with some afterstock ssh stuff, ssh keygen, ssh keysearch, different ssh etc.
Click to expand...
Click to collapse
Sorry my bad, I was sleepy and mistakenly understood "-D" as running SSH in daemon mode, when in fact it does the opposite and prevents it. Seem like his links are dead or I'm just looking in the wrong place.
Recently I switched to using a copy of mksh as sh since "chattr +i" wasn't protecting the symlink from being over-linked.
Click to expand...
Click to collapse
Not sure what you're saying here...
Sorry I can't be of more help with the ssh /dev/tty problem. I will keep my eyes open though, and try not to be so lazy using telnet for all my remote connections. .
Click to expand...
Click to collapse
Thanks, but if telnet works with tty/pts jobcontrol and vi etc, then I'm fine with that for now. I tried your command variations above and got this:
Code:
[SIZE=2]#ssh -2 [email protected] -p 2222 -T su -c exec /system/bin/sh -i # This worked!
#ssh -2 [email protected] -p 2222 -T su -c exec /tmp-mksh/tmp-mksh -T /dev/pts/0
/tmp-mksh/tmp-mksh: chvt: can't find tty /dev/pts/0
#ssh -2 [email protected] -p 2222 -T su -c exec /tmp-mksh/tmp-mksh -T /dev/ptmx
/tmp-mksh/tmp-mksh: chvt: new shell is potentially insecure, can't revoke /dev/ptmx
[/SIZE]
The fist one is good to get direct root prompt, the second fails, since pts/0 has not been created yet, and if it has, it fails with permission issue, since another process is trying to steal the pts. The reason why the third one fails, is very cryptic and I don't understand the error message and neither the code generating it.
Do you think one could use the ATE app to get a working root shell by the use of a more complex ssh command? Any suggestions how that would look?
Finally, I'm a bit worried about replacing the libc.so dependency, since the whole system depend upon it, and would probably not boot if an incompatible libc.so is used. This is why I asked about a static binary. I'd much rather have a bloated mksh than having to add library files. (We have plenty of memory and space these days.)
Regarding compiling, I would recommend using Linaro a build, rather than the crappy and often broken/outdated Code Sourcery junk.
Check this thread:
[LINARO GCC] Cross Compiler Toolchains [Linaro GCC 4.9 2014.07, 4.8 and 4.7 2014.04][19/07/2014]
Cheers!
7175 said:
--Better tab completion: unjams clustered words:
--Added "control + u" to "cut-to-beginning of line" like bash.
Click to expand...
Click to collapse
Hi! As the author of mksh, I am hereby explicitly requesting you to add a suffix to the shell version, similar to what the PLD Linux guys do here, because this changes the behaviour of the shell in an incompatible way. (Just add your own string there after a space, the @distro@ is just a placeholder that gets replaced with the PLD Linux package version.)
Do note that ^U in Korn Shell (and others) is “delete entire line”.
7175 said:
--Very complicated-programmed and colorful PS1 prompt, it looks good and I thought it through. Programming shows mksh potential for rich shell programming. It auto-turns off xtrace within itself so that 50 lines don't get printed each return with set -x. mksh small will be autodetected and give a different white and black classic ksh prompt.
Click to expand...
Click to collapse
Interesting idea, I think I will pick up the “set +x” thing.
Do you mind sharing a patchset of all your changes, so I can incorporate those I feel would be good to have in general?
7175 said:
EDIT: It appears the libc required for full mksh isnt always friendly
Click to expand...
Click to collapse
Just link the shell statically, then.
E:V:A said:
1. Would you have any clue to what's going on HERE?
Click to expand...
Click to collapse
You are annoying me. I've told you times and times again to disable SEAndroid, or to fix its SELinux policies.
E:V:A said:
2. Would you know if I can just install over my old mksh binary to get this working, or do I need "everything"?
Click to expand...
Click to collapse
Define “everything”. mksh in AOSP is /system/bin/mksh plus mkshrc.
E:V:A said:
3. And since I've heard that /system/bin/mksh will soon be permanently only /system/bin/sh perhaps this need to be changed?
Click to expand...
Click to collapse
I'm working with enh of Google on that. I would like for mksh to be installed as /system/bin/mksh and /system/bin/sh to be a link/copy of it, always. He said he'd agree I could change that back.
E:V:A said:
Hi, why did you choose to hardcode the aliases (shown in post#6) into the mksh binary?
Click to expand...
Click to collapse
These are part of the shell, and have always been, even in pdksh times.
E:V:A said:
Also, do you have a later version than the R48 Android binary above?
Click to expand...
Click to collapse
I do not deal in binaries normally. I'm somewhat working on updating AOSP to mksh CVS HEAD, in tandem with enh from Google, but when I had the time for that, it appears they changed the build system requirements. But it will be there.
It's not as if a newer mksh version would fix your SELinux problem, though…
E:V:A said:
The "r" alias is particularly disturbing as it is masking the native toolbox use of "r" to read a memory location.
Click to expand...
Click to collapse
The “r” alias is especially required for the Emacs command line editing mode. Interesting that toolbox uses it too. Thanks for bringing this to my attention, I will ask enh to change “r” in toolbox.
@E:V:A : Glad the first execution you mentioned worked. I would try messing with quotes/double quotes around the shell command, i.e. ssh ... -T su -c 'exec ...' or try ssh ... -T su -c exec "...", as something to test out further. This can make a lot of difference.
The ulimit shell function can control all sorts of performance related stuff, priority, max niceness, etc. You can see all values with "ulimit -a".
I understand your hesitance about the libc.so replacement. I would keep a copy of the old one and twrp backup before trying out a new one. I get what you're saying about statically linked binaries. I just fixed my i7 desktop that had a blown power supply, so I'll be looking into this. Thanks for the linaro link!
@mirabilos : Thanks for taking a look at this and providing us with mksh! I updated my OP with the specifics about the rebinds for "control I" and "control U" that you mentioned, and I added a suffix to the beginning comments of the file. Something like:
Code:
# $MirOS: src/bin/mksh/dot.mkshrc,v 1.88 2014/01/11 18:09:39 tg Exp $Android_mod by [email protected] 2014/08/04$
Not sure if that's what you wanted me to add in. I included an extra zip to download in the 2nd post that contains the official mkshrc 2014/01/11, my personal mkshrc, and a "diff -ur" patch (also attached to this post).
I will look into getting a statically linked mksh compiled, that sounds like something really handy to keep around.
edit: Here's basically what I added to the beginning and end of PS1 to trim down its "set -x" verbosity to 3 lines. I'd like to get it down more but haven't figured it out.
Code:
PS1=$'\001\r'; PS1+='${|
local e=$?
[[ -o -x ]] && set +x && global -x XTR_DBG=1 || global -x XTR_DBG=0
...
...
...
(( XTR_DBG )) && set -x
return ${e}
} '
7175 said:
@mirabilos : Thanks for taking a look at this and providing us with mksh!
Click to expand...
Click to collapse
You’re welcome!
7175 said:
Not sure if that's what you wanted me to add in.
Click to expand...
Click to collapse
Ah, so you did not patch the mksh binary, only the mkshrc file?
But then, that’s okay as you wrote, sure.
7175 said:
I included an extra zip to download in the 2nd post that contains the official mkshrc 2014/01/11, my personal mkshrc, and a "diff -ur" patch (also attached to this post).
Click to expand...
Click to collapse
OK, thanks. Will have a look at it within the next weeks (August is going to be a busy month for me).
I’m assuming you are okay with me adding some of your changes back to main mksh?
7175 said:
I will look into getting a statically linked mksh compiled, that sounds like something really handy to keep around.
Click to expand...
Click to collapse
Indeed, it is.
The first mksh run on Android actually was statically compiled against µClibc, though that’s cheating. (Also, dietlibc and klibc are better than µClibc). I think it should work against Bionic, too.
7175 said:
edit: Here's basically what I added to the beginning and end of PS1 to trim down its "set -x" verbosity to 3 lines. I'd like to get it down more but haven't figured it out.
Click to expand...
Click to collapse
I don’t think it works in less, considering you have to save the previous $?. There are a few handy tricks around. I tried these four:
#1 is:
Code:
local e=$?
[[ -o -x ]]; local x=$?; set +x
(( e )) && REPLY+="$e|"
[…]
(( x )) || set -x
return $e
This is about what you did. #2 is:
Code:
local e=$? x=$([[ -o -x ]]; echo $?); set +x
(( e )) && REPLY+="$e|"
[…]
(( x )) || set -x
return $e
This only looks different (I’d prefer #1 over this.) #3 is:
Code:
set +x $? $([[ -o -x ]]; echo $?)
(( $1 )) && REPLY+="$1|"
[…]
(( $2 )) || set -x
return $1
This reduces visual clutter. #4 is:
Code:
set +x $? $([[ -o -x ]]; echo $?)
(( $1 )) && REPLY+="$1|"
[…]
(( $2 )) && return $1
(( $1 )) && set -x && return $1
set -x
This reduces visual clutter even more, in that, when $? is 0, one line less is shown. But it makes the number of lines shown be not always the same. YMMV on whether to use this, #3 or #1.
This could be made even nicer *iff* mksh had the EXIT trap when leaving a function. The manpage says it’s TODO. This part of the manpage is from 1999 or older. (I started mksh in late 2002, early 2003.) Go figure… But now I have a reason to add it ☺
Another thing you could do is “set +o inherit-xtrace”, which makes functions not run with “set -x” when the main body runs with it. But wait, I see now that it does not affect ${|…;} blocks. This is a bug, which I’ll fix ☻
@mirabilos:
You Wrote:
1. Would you have any clue to what's going on HERE?
Click to expand...
Click to collapse
You are annoying me. I've told you times and times again to disable SEAndroid, or to fix its SELinux policies.
Click to expand...
Click to collapse
Well, you can be as annoyed as you like. First of all, that question was not aimed at you, and second, as I have also said repeatedly, I have already tried using SELinux in Permissive mode and it doesn't work either. In addition, your suggestion to "fix its SELinux policies" is rather ridiculous as there are no tools available to do so. At least nothing out of the box and no available binaries anywhere, and surely no instructions how to do so. So if you know how to do it, why don't you just explain for us how to do so. (My policies are updated to latest according Androids policy update tools.)
In addition there is no such thing as "disable SEAndroid". It's either in Permissive or Enforced mode. That is, unless you flash a new custom FW, which is out of the question. In addition, ATE works just fine, in Enforcing mode.
So believe me, I am far more annoyed by Googles lack of SELinux/SEAndroid developer support, than you will ever be from me asking these questions.
Thanks for explaining and helping with everything else above.
@7175:
The ulimit shell function can control all sorts of performance related stuff, priority, max niceness, etc. You can see all values with "ulimit -a".
Click to expand...
Click to collapse
Yes, I know about all those things, but I was asking why you think they need to be tweaked? The defaults, seem to work just fine.
@mirabilos : Thank you for your detailed replies, and I would be more than happy to contribute if you see anything from the content I posted.
It is interesting to see the other possibilities for turning off xtrace within the PS1. I didn't even realize you could set positional parameters after the set -x... I think I like that one the most. It just seems more array-like and streamlined, although I guess the same number of variables is still used. I look forward to seeing what updates you'll have on the exit trap and localization. This is something I'd like to toy around with.
I took your advice and set out to get a mksh static built. I ended up compiling with GCC, running "LDSTATIC=-static ./Build.sh" in the mksh source directory. I'm pretty new to cross compiling, so I didn't really get that far or try that hard before chrooting to a fresh Ubuntu terminal on my Nexus, installing GCC, and compiling there. The only thing I changed in the source code was ${ENV:-~/.mkshrc} to ${ENV:-/system/etc/mkshrc} in the "main.c" source file. I can include this in the OP though as you mentioned before. (edit: just saw this as a compiler option with Build.sh)
Edit: I see the option now in the Build.sh for changing the mkshrc path. Also, got R50 rebuilt static with klibc. Damn that got the size way down from using glibc, 684K to 204K.
R50e is released
R50e is released -- see https://www.mirbsd.org/mksh.htm#clog
Hi @7175, good stuffs here man :good:
I've been using your modded mkshrc, and I can see a hell of a lot of work you put into it. Looks really cool. Plus it's really a good place to learn for someone like me, I definitely had learnt a lot for the past few days. Tons of aliases too, and I'm getting used to using it now. I really appreciate your stuffs here, hope to see more of it
alanthehat said:
R50e is released -- see https://www.mirbsd.org/mksh.htm#clog
Click to expand...
Click to collapse
Thanks for the heads up. Updated the binaries posted in the 2nd OP to mksh R50e.
Reinkaos said:
Hi @7175, good stuffs here man :good:
I've been using your modded mkshrc, and I can see a hell of a lot of work you put into it. Looks really cool. Plus it's really a good place to learn for someone like me, I definitely had learnt a lot for the past few days. Tons of aliases too, and I'm getting used to using it now. I really appreciate your stuffs here, hope to see more of it
Click to expand...
Click to collapse
Thanks for the feedback man! Glad it's been a good learning experience like it has been for me.
7175 said:
Thanks for the feedback man! Glad it's been a good learning experience like it has been for me.
Click to expand...
Click to collapse
Well, thanks for the updated binary! Anyway, anything you can recommend me to read other than the man page in OP? And I have one more question, what's the difference between the busybox (I got one from your other thread) and yours busybox.ndk?
While I’m the upstream developer of mksh, hacking it has also been a good learning experience for me.
So it is for a friend of mine, who has taken some fundamental script frameworks I wrote, and extended it to an impressive management system, learning to code good shell while doing so.
I’m always happy when my stuff helps other people, and they share their joy! ☺
By the way: new mksh release announcements are sent over the miros-mksh mailing list (send an eMail to postmaster at mirbsd dot either org or de to subscribe).
---------- Post added at 08:40 PM ---------- Previous post was at 08:34 PM ----------
Well, we have https://www.mirbsd.org/ksh-chan.htm which collects lots of Korn Shell-related resources, although most of it is for AT&T ksh93 (the “original” Korn Shell), much also applies to mksh, though the Android OS has some special handling (especially caused by the different user-space tooling) obviously.
The shell itself behaves exactly the same across *all* supported platforms though, so (once you have set $TMPDIR to something writable for your current user) you can write Pure mksh code exactly like on Unix or Cygwin or the Macintosh or even Haiku.
mirabilos said:
While I’m the upstream developer of mksh, hacking it has also been a good learning experience for me.
So it is for a friend of mine, who has taken some fundamental script frameworks I wrote, and extended it to an impressive management system, learning to code good shell while doing so.
I’m always happy when my stuff helps other people, and they share their joy!
By the way: new mksh release announcements are sent over the miros-mksh mailing list (send an eMail to postmaster at mirbsd dot either org or de to subscribe).
---------- Post added at 08:40 PM ---------- Previous post was at 08:34 PM ----------
Well, we have https://www.mirbsd.org/ksh-chan.htm which collects lots of Korn Shell-related resources, although most of it is for AT&T ksh93 (the “original” Korn Shell), much also applies to mksh, though the Android OS has some special handling (especially caused by the different user-space tooling) obviously.
The shell itself behaves exactly the same across *all* supported platforms though, so (once you have set $TMPDIR to something writable for your current user) you can write Pure mksh code exactly like on Unix or Cygwin or the Macintosh or even Haiku.
Click to expand...
Click to collapse
Thanks for the link, and for your works on mksh too! I'm really new to shell, I'm not even that familiar with unix/linux either. But I always have been interested in shell scripting, so I'm glad that I stumbled upon this thread
Whew, that's a lot of stuff in that link you gave, definitely gonna make use of it, thanks again!