Related
Hi all, here is a package of command line tools I've put together, here's the readme with a few of the tools listed.
Android Command Line Tools
This Is Working Prefectly On Samsung Galaxy S3 'GT-I9305'
Your Phone Should Be Rooted To Perform These Steps!!!
Installation:
1. Copy The android-tools.zip To Your SD Card (Internal Or External) And Extract It
You Should Now Have A Directory Called 'android-tools'.
2. Open A Terminal On Your Phone And Type The Following
su
mkdir /data/tmp
cat /sdcard/android-tools/busybox > /data/tmp/busybox
cat /sdcard/android-tools/busybox-installer > /data/tmp/busybox-installer
cat /sdcard/android-tools/pkgs.tar > /data/tmp/pkgs.tar
cat /sdcard/android-tools/install.sh > /data/tmp/install.sh
cd /data/tmp
chmod 755 *
./install.sh
3. In Android Terminal Emulator Preferences, Set The Shell Path to '/system/bin/bash'
And Leave The Initial Command Blank.
4. Quit Terminal Emulator And Restart.
OpenSSH: ssh-keygen Save To '/data/local/home/root/.ssh/id_rsa'.
'ssh' And 'scp' Binaries Will Look For Keys In '/data/local/home/root/.ssh/'
So You Dont Need The '-i' Option, 'ssh-copy-id' Gets It From There Too.
Arduino:
There Is A Build Environment To Build And Upload Arduino Sketches With A Script Called 'arduino' To Wrap It Up
Type 'arduino --help' In Terminal To See Its Options, It Uses The Arduino-1.0 Core Library And I have Tested
Every Board And Can Confirm It Builds For All Boards Including 'micro' And 'leonardo'. It Uses 'avrdude'
'avr-libc-1.6.7' 'gcc-avr-4.5.1', I Wrote A 'arduino.nanorc' File For Writing Sketches Which Has All The
Functions And Constants Of The Arduino Core Library With The Same Colours As In The IDE.
Other Tools:
Some Other Binaries I Have Added Or Modified Are 'tar' With All Options Enabled, 'iwconfig', 'grep egrep fgrep'
With Colour And PCRE Enabled, '7zip', 'ipctool', 'shc' For Compiling Shell Scripts, 'ssh-copy-id', 'sed' With All Options
Enabled, 'macchanger', 'bootimg_baseaddr' In bash/sh, Compiled GNU 'core-utils', 'ncurses-hexedit', 'nmap', 'ngrep', 'nano'
'strace', 'gcc', 'g++', 'unpack-bootimg' In bash, 'unrar' And 'vim'.
Backtrack 5 ARM Is Configured And Ready To Go, Just 'mkdir /sdcard/Chroot'
And Copy Your bt5.img File Into It And Type 'bootbt', Then Edit The Scripts
For VNC Server 'startvnc' And 'stopvnc' To Reflect Your Screen Size, These
Are In The pkgs.tar
Replacing '/system/etc/mkshrc' With The One Included In This Package Enables Bash As Default In ADB Shell
If The Bash Binary Is Found In '/system/bin/bash' Else The Default Shell Will Drop Back To mksh.
Nano Does Not Save Through ADB Shell Using ^O Or '^X Then y' Method, Works Fine In Terminal Emulator App On Device.
There are more tools than listed above, here's the link> https://www.dropbox.com/s/yjumfljy7e1yoyc/android-tools.zip
If you're on an custom ROM and can't use the terminal emulator when you restart terminal emulator, get /system/lib/libncurses.so from stock ROM and use adb to copy it into /system/lib on your current ROM, I will fix this tomorrow.
Sent from my GT-I9305 using xda app-developers app
Please post the right link... this one is malformed.
sciurius said:
Please post the right link... this one is malformed.
Click to expand...
Click to collapse
op just updated the link, package is now ready for download again!
thank you, andreotti09!!!
greetz,
sUsH
Yep, you read that correctly. I have optware, ssh, samba, transmission, and flexget working on my Minix X5 Mini. This should work for any rooted device which has an adb connection enabled. This will work on the original ROM. In fact, I use the stock ROM. For those not using a Minix device this should work on any ARM device. Sorry but all the binaries are built on ARM.
JUST AS EVERY OTHER DEVELOPER: I AM NOT RESPONSIBLE IF YOU BRICK YOUR DEVICE! MAKE A BACKUP!
Requirements:
Linux box with adb (don't ask me about windows, I don't support bad habits)
clockworkmod (for a backup)
root
internet connection
Process:
Make a backup of your ROM!
Download files (gitHub)
You have two options here:
Download the zip via https://github.com/erichlf/AndroidSeedBox/archive/master.zip and unzip it.
Clone the repo using git via 'git clone [email protected]:erichlf/AndroidSeedBox.git'
Make script executable
chmod +x optware-etc.sh
Obtain adb connection to device (covered in another thread)
Gain root access on local machine (adb seemed to require this for things to work)
sudo su
Run script and follow directions
./optware-etc.sh
Use SManager to run /opt/home/root/sysinit at every restart.
Notes:
The script can be modified to change the various programs that I install. You could exchange transmission for deluge for example.
Transmission can be accessed from the minix through localhost:9091 or from some other machine using your ip-address and the port 9091. If that doesn't work you should edit the config file located at /opt/home/root/.config/transmission-daemon/settings.json
username: root
password: you provided this in the install script
Without SManager nothing will start automatically. However, if you have a ROM which has init.d support you can move the scripts in /opt/etc/init.d to /etc/init.d I would suggest maybe linking the two instead of just moving the scripts or possibly adding a script to /etc/init.d which runs the items in /opt/etc/init.d The reason is because when installing things using ipkg the startup scripts will be placed in /opt/etc/init.d and not /etc/init.d However, it is extremely important that optware is started, and this is partly what sysinit accomplishes.
To list available packages
ipkg list
To install a new package use the command
ipkg install <new package>
To remove a package use the command
ipkg remove <package to remove>
cron is weird and I couldn't get it to work like it should, but I got it to work
While on the Android device (ssh or terminal emulator)
Create a .crond file in the home directory of your device (/opt/home/root/) with some schedule in it. Remember to leave a blank line at the end of the file.
Tell cron about the .crond file
crontab -u root /opt/home/root/.crond
Make sure cron sees the cron file
crontab -l
If you want to edit your cronfile use a text editor and edit the file directly and then tell cron about the file again.
Many things are installed in what seem like strange places, so use
which <binary you are looking for>
Feel free to help develop the code. I think what would be best is an update.zip or a CWM flashable zip. Right now I don't know how to do this, but once I get more time I will look into it. So, any help on this front is welcomed.
Enjoy!
I really wish you would have kept the repo up. It seems kind of pointless to go through all that trouble just to delete the repo and leave people wondering what you did.
I have been busy and didn't update this particular post, since there had been no activity on it.
git clone [email protected]:erichlf/androidseedbox.git
https://bitbucket.org/erichlf/androidseedbox/get/master.zip
Sorry, I didn't need to be rude. I was just excited to find this and then sad when it was gone. Thanks for pointing me in the right direction!
About Busybox: busybox.net/about.html
More on Busybox: busybox.net
This is just for anyone who wants to try, and especially those without access to a PC.
:
: --------------- BUILD STEPS --------------
:
Things we'll need besides your phone:
- "Android Terminal Emulator" app
- "Complete Linux Installer" app , I also recommend trying "linux deploy" for more advanced usage
- internet/wifi
- latest "busybox" source
1) We need to get Ubuntu or Debian booted for a sufficient build environment (kali linux works well too). I've used them all on Android but I like the better stocked terminal in the Ubuntu images. I used the app "Complete Linux Installer" which is free and works beautifully, very simple and easy. In the app you want to follow the short instructions to download an Ubuntu image, rename it to ubuntu.img, and place it in a folder named ubuntu in /sdcard. Then hit menu in the app and click the terminal screen icon that says "Launch". An Ubuntu terminal will now open in Android Terminal Emulator. Super quick and easy.
2) Let's download some crucial build environment tools.
Code:
apt-get install -y gcc build-essential libncurses5-dev libpam0g-dev libsepol1-dev libselinux1-dev
--EDIT-(30AUG2014)--
For Selinux compatibility and loginutils, we need to also download a few extra packages. Already included in the code above.
3) Now the cool thing about this chroot Ubuntu environment is that we still have access to the sdcard to transfer files between Android environment and the chroot jail. Extract your downloaded busybox source to your Ubuntu home with something like:
Code:
cd
tar -xf /sdcard/Download/busybox*bz2
cd busybox*
4) Now we can build busybox statically. The first thing we do is generate a Makefile by running "make" with a "defconfig" (default configuration file) Usually you will run "./configure" with other programs, but busybox compiles more like a kernel, so it uses a config which has a huge checklist of options.
(After successfully compiling busybox, we can go back and customize the .config; this entails that for each "CONFIG ..." line we see, we can uncomment it and mark it "y" or "n" to configure some option... This can be more easily done from a terminal busybox menu interface with "make menuconfig". You just need to crank font down to 7 or use telnet/ssh)
Skip "make defconfig" if you use a customized ".config" file such as one I've attached.
Code:
make defconfig
If all goes well, we now have a Makefile and are ready to compile:
Code:
make clean && make LDFLAGS=-static
Let "make" crank out the binary for a couple minutes. The extra variable we set after make is to compile statically. When compiling is complete we'll have a few different busybox binaries at the root of the source directory. We use the one named "busybox" since we're not debugging.
5) Now let's copy it to /system/usr/bin to install for test usage.
Code:
cp ./busybox /android/data/media/0
(Open a new terminal tab to get into Android Environment)
mount -o remount,rw /system
mkdir -p /system/usr/bin
cp -f /sdcard/busybox /system/usr/bin
chmod 0555 /system/usr/bin/busybox
/system/usr/bin/busybox --install -s /system/usr/bin
mount -o remount,ro /system
PATH+=:/system/usr/bin
.. and done. Run some scripts and enjoy your static busybox!
:
: Extra steps for SELinux-enabled busybox
:
Here are the extra steps you need to take to compile busybox with SELinux features. Sorry it took so long to get this added to this first post.
First we need to download the source for libselinux and libsepol and compile it. (This is for use with the standard glibc toolchain.)
Code:
cd
apt-get source libselinux libsepol
cd libselinux*
make
cd
cd libsepol*
make
Now that we have those libraries compiled, we can proceed to the busybox compilation.
Code:
cd
cd busybox*
make clean && make LDFLAGS='-static -L ../libselinux*/src -L ../libsepol*/src' CFLAGS='-Os -I ../libselinux*/include -I ../libsepol*/include'
That's pretty much it. It initially seems more complicated than it actually is, but all we're really doing is including the libraries for libselinux and libsepol into the busybox compilation.
edit:
**Commands to run if you have compile errors:
Code:
apt-get build-dep busybox
apt-get install -y build-essential
apt-get -f update
dpkg --configure -a
:
: --------------- DOWNLOADS --------------
:
***** Attached are flash installers for busybox (v1.23.1 stable, non-SELinux, 374 applets included!, ~1.1MB size) or busybox (v1.23.1 stable, SELinux, 386 applets included!, ~1.6MB size) *****
Since it's up-to-date it has some nice extras most people haven't seen like a "-I" option for xargs! Yes, that is correct, busybox xargs has its testicles back.
Code:
e.g.
$ echo Hello | xargs -I{} echo {} world!
> Hello world!
: ---------- UPDATES ----------
-------------------EDIT-2-(30AUG2014)----------------------
Got a Selinux-enabled busybox attached now. This means Selinux flags are integrated into applets like ls, id, ps, etc, and there are now 12 extra Selinux applets to give a total of 386 applets, ~1.6MB in size. The previous one is more portable, but this one can completely replace toolbox and gives you Selinux control for Android 4.4+. Plus it's pure maxed-out awesomeness.
***I've also attached the .config files for each busybox I've compiled, so anybody can remake them (from their phone!) as I've posted. You just need to download and extract the .config file to the root of your busybox source directory and type "make".***
-------------------EDIT-3----------------------
YashdSaraf has made some very useful flash zips to install either the non-selinux- or selinux-enabled busybox 1.23.1 via recovery. Installation replaces the stock busybox in /system/xbin. I've attached the zips he made to the end of this OP.
(**Note: Thought I'd mention that there will be a handful of applets that don't work in "Android" environment such as su(don't worry this isn't linked with the installer) Part of this is because of the way Android's default file structure is an amputated modified version of linux. With almost all of them, slight modifications to environment and file structure can be made to get them to work. This is just normal behaviour of busybox in android. The su and init applets shouldn't be used in Android though. I keep them compiled into the binary anyway for completeness of the build and because they work and are needed for a root.gz initrd or some chroot environments. It also doesn't hurt keeping them compiled in. You just have to remember not to link them when installing busybox.
-------------------EDIT-4-(06SEPT2014)----------------------
:
: How to compile against(using) uclibc for a smaller binary!!
:
Download the attached arm-linux-uclibcgnueabi toolchain package that I pre-compiled. Extract to /data/media:
Code:
cd /data/media
zip='/sdcard/Download/2014-09-06__arm-buildroot-linux-uclibcgnueabi.tar.lz.zip'
unzip -op "$zip" | lzip -d | tar -xf -
Then let's open up the "Complete Linux Installer" or "Linux Deploy" terminal.
To use the toolchain with a busybox build, we just need to specify the parameter CROSS_COMPILE which is a prefix to the compiler tools. So if you extracted the toolchain to /data/media, you will use:
Code:
make clean && make LDFLAGS=-static CROSS_COMPILE=/android/data/media/arm-buildroot-linux-uclibcgnueabi/bin/arm-buildroot-linux-uclibcgnueabi-
When you're done you should have a busybox binary with 374 functions with size around 1.1MB. That's a 20% decrease in size from using the standard glibc toolchain!
**IMPORTANT Notes
- The toolchain can't be used with lollipop since it's not compiled with -fPIC. I'll fix this later. Busybox is fine since it's static, it's just the toolchain I uploaded.
- Selinux-enabled busybox .config errors out when building using the uclibc toolchain; I think this is a lack of support issue. In the "Complete Linux Installer" app you'll need to add the mount "/data/media" in options. This gives you access to it as "/android/data/media", very very useful for extra space needs.
Difference between SELinux and non-SELinux busybox
The SELinux (NSA security enhanced Linux) binary comes with the following extra utilities: chcon, getenforce, getsebool, load_policy, matchpathcon, restorecon, runcon, selinuxenabled, setenforce, setfiles, setsebool, and sestatus. There are also some selinux flags enabled for applets such as "ps" and "ls", e.g. "ps -Z" and "ls -Z" to show the context for processes or files. If you are using Android 4.3 or newer, then you probably want to use the SELinux-enabled busybox since Android 4.3 is when SELinux was introduced to Android. Using the SELinux busybox on older version of Android without SELinux file structure should probably work besides the SELinux applets, but I haven't tested this. The non-SELinux binary can be used on any version of Android. When it comes down to it, the system actually uses "/system/bin/toolbox" SELinux applets for SELinux operations, so unless you specifically want to use busybox's SELinux tools for personal use, the safest option is to go with the non-SELinux busybox. I use Android 4.3.1 and 5.x, so I use busybox's better featured SELinux tools.
Latest updates see post 2
Busybox 1.23.1 (2015-02-06) below
Busybox compilation on Linux
reserved
Great Info here!
But I would be interested to know how well this method works on Samsung Stock devices running AOS 4.2 and above? Any experience?
Awesome info, this thread came up #1 while googling busybox 1.23
I made a flashable zip of the attached binary in the op to clean the old one(if any) and install the new busybox in xbin, just in case if anyone needs it. :good:
Is it work on xperia sp on 4.3 fw yes ?
YashdSaraf said:
Awesome info, this thread came up #1 while googling busybox 1.23
I made a flashable zip of the attached binary in the op to clean the old one(if any) and install the new busybox in xbin, just in case if anyone needs it. :good:
Click to expand...
Click to collapse
Thanx,worx fine with Carbon 4.4.4 on my LG.
GREETZ FROM TEAM-OPTIMA!!!
E:V:A said:
Great Info here!
But I would be interested to know how well this method works on Samsung Stock devices running AOS 4.2 and above? Any experience?
Click to expand...
Click to collapse
Thanks man. I've been compiling tons of stuff with Debian and Ubuntu chroot no problem on top of 4.3.1 Vanir and also 4.4.4 Carbon, both are my daily drivers. "Complete Linux Installer" is pretty fast compared to some other chroot apps like GNUroot (no offense to GNUroot, it works but is way too slow). It runs real-time compared to non-chroot. When compared to my dual-core 2007 Pentium M laptop, it's about 2-3 times as slow which isn't too bad for compiling something like mksh or even busybox which takes up to 5 mins I'd say.
In terms of binary size, compiling natively is better than cross-compiling it seems. I used gcc with no size optimizations here, so 1.37MB is pretty nice compared to some others around 2MB with full configs. With this method and klcc (gcc wrapper) I got mksh compiled to 192KB. I'm currently trying to build a uclibc toolchain on my laptop that will give me a mksh binary under 300KB..
YashdSaraf said:
Awesome info, this thread came up #1 while googling busybox 1.23
I made a flashable zip of the attached binary in the op to clean the old one(if any) and install the new busybox in xbin, just in case if anyone needs it. :good:
Click to expand...
Click to collapse
Cool thanks man! That is really useful, glad to hear from CALIBAN that it works. Could I add this to the OP with credit to you?
Hamidreza2010 said:
Is it work on xperia sp on 4.3 fw yes ?
Click to expand...
Click to collapse
Yes, xperia sp uses armv7 so you should be good to go.
7175 said:
Cool thanks man! That is really useful, glad to hear from CALIBAN that it works. Could I add this to the OP with credit to you?
Click to expand...
Click to collapse
Sure bro go ahead
Edit: Went ahead and made one for selinux enabled busybox :silly: , you can add this one in the op too.
Hey guys I was able to get an entire uClibc toolchain built the other day (using buildroot). I tested it and it makes some nice small binaries with about 20%+ smaller size than the standard glibc. Man that took hours to compile but was well worth it. It really put the stability of Android OS to the test as well. Kitkat absolutely couldn't finish compiling with multiple oom's and oops's, but Vanir 4.3.1 stuck it out real nice. Once I had the huge amount of required buildroot packages downloaded, I was able to compile in TWRP as well with good stability. (I have the "Complete Linux Installer" startup chroot script aliased in my mkshrc so I can pull up an ubuntu terminal without starting the app. )
So I got 3 new attachments to OP:
- arm-linux-uclibc toolchain for anyone who wants to compile stuff with it (host=arm AND target=arm)
- busybox (374 fcns, non-selinux) 1116KB
- lzip binary (in case you need it to unzip the toolchain, which is highly compressed from 64MB to 14MB with lzip's lzma)
**As I mentioned in the updated OP, I wasn't able to get selinux-enabled busybox compiled with uclibc. This may be something unsupported, or there may be a patch fix. I'll check it out. I'll try with musl libc and musl-gcc as well.
I have another approach, I try aboriginal cross compiler toolchain in archLinux it produced small binary, but I can't compile busybox for android. For Linux it work. Maybe need bionic lib?
ndrancs said:
I have another approach, I try aboriginal cross compiler toolchain in archLinux it produced small binary, but I can't compile busybox for android. For Linux it work. Maybe need bionic lib?
Click to expand...
Click to collapse
Sounds interesting. I honestly haven't given this a try yet, but I'm very interested in taking a look at it. At this point I'm pretty much addicted to making the smallest binaries I can and testing out different toolchains. I'll give it a good search on duckduckgo, and if you have any insightful links that would be great.
Edit: Alright cool I found the source for Aboriginal Linux at landley.net/aboriginal and am building on Android now. I'm also trying this on my x86_64 laptop so that I can compare the differences like I have with glibc, uclibc, musl, klibc binary builds in a native environment and a cross-compile environment.
I see from my laptop's build that a busybox was generated, but it was dynamic and has a libc.so.6 dependency. @ndrancs : this might be what you were talking about. Did you try compiling static? Also see if "make allnoconfig && make clean && make LDFLAGS=-static" works for compiling busybox with Aboriginal Linux.
7175 said:
Edit: Alright cool I found the source for Aboriginal Linux at landley.net/aboriginal and am building on Android now. I'm also trying this on my x86_64 laptop so that I can compare the differences like I have with glibc, uclibc, musl, klibc binary builds in a native environment and a cross-compile environment.
I see from my laptop's build that a busybox was generated, but it was dynamic and has a libc.so.6 dependency. @ndrancs : this might be what you were talking about. Did you try compiling static? Also see if "make allnoconfig && make clean && make LDFLAGS=-static" works for compiling busybox with Aboriginal Linux.
Click to expand...
Click to collapse
I preferred to use uclibc I think it easy to setup and produced small binary.. Aboriginal cross-compiler use uclibc as default. Btw I don't use cmd : LDFLAGS=-static instead I set it in .config.. Maybe I try this later..
ndrancs said:
I preferred to use uclibc mk it easy to setup and produced small binary.. Aboriginal cross-compiler use uclibc as default. Btw I don't use cmd : LDFLAGS=-static instead I set it in .config.. Maybe I try this later..
Click to expand...
Click to collapse
Ok yeah I like how aboriginal set up with uclibc, and it has scripts for each build stage, so you can stop at the toolchain. I'll be interested to see their future releases with the musl libc as well.
Also for anyone interested, I figured out how to run dynamic binaries in android:
- make the directories "/lib" and "/lib/arm-linux-gnueabihf"
Code:
mkdir -p /lib/arm-linux-gnueabihf
- copy the linker "ld-linux-armhf.so.3" to "/lib"
- find a specific binary's dependencies: e.g. for dynamic mksh do:
Code:
strings mksh | grep \\.so
- copy the listed libs to "/lib/arm-linux-gnueabihf": e.g. for mksh that would be libc.so.6. The libs/linker you copy over will come from the mounted ubuntu/debian/... image you have mounted like with "Complete Linux Installer".
- adjust your LD_LIBRARY_PATH:
Code:
LD_LIBRARY_PATH=/lib:/lib/arm-linux-gnueabihf:$LD_LIBRARY_PATH
Any plan to update the busybox to current version. Thanks.
@7175 can you update flashable zip to 1.23.0 stable ?
@ndrancs @exodius48 : Thanks for notifying me guys, I needed to get around to updating to 1.23.0 stable. I updated the original post with no-edify installers for busybox 1.23.0 stable. There's a non-SELinux uclibc compiled version and a full 386-applet SELinux glibc compiled version. They're included in this post too for ease.
7175 said:
@ndrancs @exodius48 : Thanks for notifying me guys, I needed to get around to updating to 1.23.0 stable. I updated the original post with no-edify installers for busybox 1.23.0 stable. There's a non-SELinux uclibc compiled version and a full 386-applet SELinux glibc compiled version. They're included in this post too for ease.
Click to expand...
Click to collapse
Great..been waiting for this release.. :good:
Btw, can i use busybox_full_selinux.zip on android 4.2.2 MIUI rom?
exodius48 said:
Great..been waiting for this release.. :good:
Btw, can i use busybox_full_selinux.zip on android 4.2.2 MIUI rom?
Click to expand...
Click to collapse
Yeah that should work just fine. I'm pretty sure any SELinux tools or applet flags should work since 4.2 introduced SELinux to its filesystem. Let me know if there are any issues.
7175 said:
Yeah that should work just fine. I'm pretty sure any SELinux tools or applet flags should work since 4.2 introduced SELinux to its filesystem. Let me know if there are any issues.
Click to expand...
Click to collapse
Great release..busybox_full_selinux.zip working fine so far on MIUI rom V5 android 4.2.2. :victory:
7175 said:
@ndrancs @exodius48 : Thanks for notifying me guys, I needed to get around to updating to 1.23.0 stable. I updated the original post with no-edify installers for busybox 1.23.0 stable. There's a non-SELinux uclibc compiled version and a full 386-applet SELinux glibc compiled version. They're included in this post too for ease.
Click to expand...
Click to collapse
Hey @7175
Great guide. I am able to compile just fine on my device using your guide. However, is there any way to compile the selinux applets support using a Linux PC (or NDK)? I am not able to find a selinux supported toolchain. May be you can help.
(Check out https://www.python.org for information on what python is.)
Yes you read that correctly, static python for Android! It took a while to figure out how to get this compiled, but I finally did it. I present you with an installer and a little bit of testing. The second post contains information on compilation and the sources. Btw, any cool python scripts are welcomed!
INSTALLATION
Recovery flash installers are attached that will install python 2.7.9 and/or 3.4.2 to /system/pythonX.X.X. You can install both if you want. Installation size is about 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.
After installation, the python director(ies) in /system will contain the static python binary, some scripts, and a bunch of modules and documents. Separate scripted executables will be installed to /system/bin/python or /system/bin/python3 depending on which one is installed. These basically just set the PYTHONHOME environment variable and execute the python binary.
TESTING
When you boot up to Android after installation, you should be able to just open up a terminal and fire off some python commands.
Test using python 2.7.9:
Code:
python -c 'print "Hello World!"'
Test using python 3.4.2:
Code:
python3 -c 'print("Hello World!")'
You can also write scripts shelled with python:
Code:
#!/system/bin/python
print "Hello World!"
Make sure to set them as executable with "chmod +x".
Adjust your screen brightness via /sys (root required, tested on Galaxy Nexus):
Code:
python -c 'f=open("/sys/devices/omapdss/display0/backlight/s6e8aa0/brightness","w"); f.write("40"); f.close()'
A cool script I made to tweak file system I/O like rq_affinity, rotational, etc for I/O blocks:
Code:
#!/system/bin/python
import os,re,sys
list=[]
# find all directories containing rq_affinity
for roots, dirs, files in os.walk('/sys'):
for file in files:
match=re.search(r'\S+/rq_affinity',os.path.join(roots,file))
if match:
list.append(match.group().replace('rq_affinity',''))
# write specific values to files in each directory found before
for dir in list:
for name in 'rq_affinity', 'rotational', 'read_ahead_kb', 'nr_requests', 'iostats', 'nomerges', 'add_random':
try:
f=open(dir+name,'w')
if name is 'rq_affinity': f.write('1')
elif name is 'read_ahead_kb': f.write('512')
elif name is 'nr_requests': f.write('512')
else: f.write('0')
f.close()
except IOError:
sys.stderr.write('Problem writing to ' + dir+name + '\n')
Or execute "python" or "python3" without any parameters to open up the interpreter and go from there. When you start for example "python" (python 2.7.8), you should see something like:
Code:
Python 2.7.8 (default, Dec 2 2014, 05:15:18)
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
***I haven't fully tested this yet, so let me know how it goes. I know basic commands work, but there are still some complexities in the compilation that may need to be figured out.***
(update) -- Install Extra Packages/Modules via "easy_install" -- (root required)
1) In a shell, remount /system read-writable with:
Code:
mount -o remount,rw /system
2) Change directory to /system/pythonX.X.X, i.e.,
Code:
cd /system/pythonX.X.X
3) Execute easy_install followed by a package name, i.e.,
Code:
./easy_install [i]package_name[/i]
4) Let it download and install! Then test it out.
5) Remount /system read-only with:
Code:
mount -o remount,ro /system
Also, to remove a package, execute:
Code:
./easy_install -m [i]package_name[/i]
rm -r ../lib/pythonX.X/site-packages/[i]package_name[/i]*.egg
*** Flashing the resolv.conf patch may be required to make an internet connection to download modules.
Also, a few things need to be done to get easy_install working:
1) For python 2.7.8 or 2.7.9 you need to link python2.7 to python like this:
Code:
ln -s /system/python2.7.9/bin/python2.7 /system/python2.7.9/bin/python
I'll make sure this is automatically done in future installs.
2) SSL certificates need to be added to "/etc/pki/tls/certs/" to use SSL with easy_install.
Code:
mkdir -p /etc/pki/tls/certs
curl http://curl.haxx.se/ca/cacert.pem -o /etc/pki/tls/certs/ca-bundle.crt
ISSUES
The python installations have the following modules compiled in with external dependencies such as libraries not included at this point:
Code:
--------------PYTHON 2.7.8-2.7.9----------------------
_hashlib _multiprocessing _ssl
_testcapi bz2 crypt
dbm
Code:
--------------PYTHON 3.4.2-----------------------------
_crypt _dbm _decimal
_hashlib _multiprocessing _ssl
_testcapi
I will look into this more.
As @bubbleguuum points out, name resolution with python 3.4.2's urllib wasn't working. I found a workaround by adding the line
Code:
options single-request-reopen
to /system/etc/resolv.conf. This still needs some testing but seemed to work for me. I've included a recovery flash zip to patch /system/etc/resolv.conf if the line is not there since this is a root operation.
UPDATES
* 2014-12-02 * Compiled in more modules for each installation. This makes the installation a bit bigger, but it's worth it. You get a more complete python! To get stuff like help functions and math and readline modules, you need the larger installations linked below under "DOWNLOADS".
* 2014-12-03 * Thanks @cybojenix for pointing out some incapatibility issues with the original edify installer I was using. I have now updated the installers to use SuperSU's great non-edify sh-script installer.
* 2014-12-05 * Some updates to the installers to include more modules, plus size reduction for python 3.4.2. Modules added to python 2.7.8 : _bsddb _ctypes, _ctypes_test, _hotshot, _json, _lsprof, _sqlite3, future_builtins, and ossaudiodev, easy_install, pip(broken for now). Modules added to python 3.4.2: _bz2, _ctypes, _ctypes_test, _gdbm, _json, _lsprof, _opcode, _sqlite3, _testbuffer, _testimportmultiple, ossaudiodev, xxlimited
* 2014-12-05 * Some big updates to the installer to include almost the rest of the modules that wouldn't compile. Also easy_install is working on my end with these new experimental builds. With a little effort, should be able to get pip working as well. I had to use some hacky flags to get it to compile, ignore unresolved-symbols and such, so it definitely needs some testing. Another thing I did was tweak the install script to write over previous installations from these installations. I recommend downloading the experimental builds if you read this. The worst thing that might happen is one of the new modules I've included won't work completely.
* 2014-12-14 * Added python 2.7.9. Trimmed the installation sizes down to the much more reasonable 41MB, 43MB, and 49MB respectively for python 2.7.8, python 2.7.9, and python 3.4.2.
* 2014-12-24 * Attached a recovery flash.zip to patch resolv.conf to try and fix a urllib name resolution error.
* 2014-12-26 * A couple updates: Tweaked python installations so that modules looks for "/system/bin/sh" instead of "/bin/sh" when needed. Also a few "#!" corrections for scripts in the python bin directory. This shouldn't break anything from what I've tested, only make Android python more capable, but I'll keep the old installations attached for now just in case since the modifications involved using "sed". The second update is nice. I've added a little section on installing extra modules with easy_install(included and working with all installations).
Compiling
The environment I compiled in was a debian "wheezy" image mounted as a loop device on Android. The reason I compile this way is so I have full access to the arm environment tools, gcc compiler, etc. without having to go through the trouble of using a cross-compiler (which doesn't work in this case from my testing with uClibc, or eglibc. Code sourcery's might work but I didn't feel like booting an x86 linux installation to try). You can recreate the same environment pretty easily by using either Complete Linux Installer or Linux Deploy apps. I use both.
To get a static python compiled, I first downloaded the following packages with APT:
Code:
[i]apt-get build-dep python python3[/i]
build-essential gcc make # building tools
binutils-gold # awesome gold linker
zlib1g-dev # zlib
libreadline-dev, libncurses5-dev # readline, curses
libbz2-dev # bz2
libsqlite3-dev # sqlite3
python-bsddb3, python3-bsddb3
libgdbm-dev # gdb
libssl-dev # ssl
python-tk, python3-tk # tkinter
libdb-dev # db
python-gdbm python-bsddb3
libffi-dev # _ctypes
tcl8.6-dev # tkinter
libx11-dev # tkinter
libmpdec-dev # decimal
Then I manually installed binutils-gold by unpacking the rpm for armv7h (this isn't available with APT yet). You can get it here though. (Update: this should be available in the debian repositories now.)
For static compilation, you need to make some changes to Modules/Setup from the python source directory. Specifically, you need to add
Code:
*static*
to the top of the file. Then you need to uncomment any commented modules you would like compiled in. When you compile and see a list of failed modules, search for each of these in Modules/Setup and uncomment them. For example, change
Code:
#math mathmodule.c _math.c
to
Code:
math mathmodule.c _math.c
. Then recompile. There may be some shared dependencies and packages as well you need to figure out to get them to compile. Take a look at the attached "Setup" files for more detail.
The actually compile steps look like this:
(1)
Code:
./configure --build=arm --prefix="$PWD"/out LDFLAGS="-static -static-libgcc -Wl,--unresolved-symbols=ignore-all -Wl,--export-dynamic" CPPFLAGS=-static CXXFLAGS=-static CFLAGS="-Os -static" LDFLAGS=-static LD=ld.gold
(2) Modify the Modules/Setup file.
(3)
Code:
make clean; make install
After compiling, there are some things you can do to make your installation in "./out" smaller. Strip the large binaries in "out/bin", i.e.
Code:
strip -s ./out/bin/python2.7
Also remove the python archive library, *.o's, etc. with:
Code:
make clean
This might seem kind of weird, but it works and cleans out unnecessary files from your installation.
**Make sure to backup your Modules/Setup file if you run "make distclean". I lost my 3.4.2 Modules/Setup file after doing this. :silly: Now I gotta recreate it, dammit.
**I should also mention that compile time is very fast given the amount of data generated. It takes about 10 minutes to get python and all it's modules compiled on my Galaxy Nexus.**
:EDITS:
: Updated some configure parameters. Removed unnecessary, extra "./configure". Also added some updates to the included "Setup" files.
: Hacky update to "./conifgure" to ignore unresolved symbols, export dynamic. This allows certain modules to be compiled which have shared dependencies.
: Update to trimming down the installation size after "make". Use another "make clean".
Downloads - Extract in the source directory
Setup (python 2.7.8)
Setup (python 3.4.2)
Setup (Experimental) (python 2.7.8)
Setup (Experimental) (python 3.4.2)
Something that would be interesting to look in to would be to have pip/easy_install running, so you can install extra non c packages onto the sdcard/data/wherever.
Would you also consider opening up the sources please? There is a project I'm interested in doing involving python on Android, however a lack of time meant I couldn't finish building it.
Give me a shout if you need anything. I hope to see even more come out of this thread
Edit:
I've just looked at the updater-script. The mount command won't work on the majority of devices. Can you turn the update-binary in to a shell script please? See the SuperSU updater for reference
cybojenix said:
Something that would be interesting to look in to would be to have pip/easy_install running, so you can install extra non c packages onto the sdcard/data/wherever.
Would you also consider opening up the sources please? There is a project I'm interested in doing involving python on Android, however a lack of time meant I couldn't finish building it.
Give me a shout if you need anything. I hope to see even more come out of this thread
Edit:
I've just looked at the updater-script. The mount command won't work on the majority of devices. Can you turn the update-binary in to a shell script please? See the SuperSU updater for reference
Click to expand...
Click to collapse
Thanks for your reply! I've changed the installer to use SuperSU's no-edify sh-scripted updater. I tested on my device, but let me know if there are any issues though. I kept it pretty basic.
I added some stuff on the source modifications and compilation to the 2nd post. The only file I've actually modified so far in the source is the generated Modules/Setup file, and I've included the one I used for python 2.7.8. Unfortunately, I over-cleaned the python 3.4.2 directory, which deleted my modified Modules/Setup there, so I'll have to add it later when I re-edit it.
That's a great idea on pip/easy_install. I did download the pip_installer and tried out installing on the static python I made. Got some errors though pointing to a few modules I haven't gotten compiled into the static python installation (listed in the OP near the bottom, specifically _ctypes). I'll see what I can do about that. The line for compiling the module might just need to be added to Modules/Setup. Hopefully that's the case.
@7175
Thank you very much for these binaries.
There is however a problem: name resolution (DNS) doesn't seem to work at all (with both python 2 and 3 downloads).
All attempts to use urllib.request.urlopen('http://somehost.com') fail with "<urlopen error [Errno -2] Name or service not known>" (running python as root but it doesn't matter, and a rooted Nexus 4 running 4.4.4 stock ROM):
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 153, in urlopen
return opener.open(url, data, timeout)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 455, in open
response = self._open(req, data)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 473, in _open
'_open', req)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 433, in _call_chain
result = func(*args)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1202, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/system/python3.4.2/lib/python3.4/urllib/request.py", line 1176, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
Click to expand...
Click to collapse
it works if replacing somehost.com by its ip address.
To reproduce above stack trace with python 3:
import urllib.request
urllib.request.urlopen('http://www.google.com')
while this work:
urllib.request.urlopen('http://173.194.45.229')
nslookup and general name resolution work anywhere else:
[email protected]:/ # nslookup www.google.com
nslookup wwwgoogle.com
Server: 8.8.4.4
Address 1: 8.8.4.4 google-public-dns-b.google.com
Name: wwwgoogle.com
Address 1: 2a00:1450:4007:80c::1014 par03s15-in-x14.1e100.net
Address 2: 173.194.45.242 par03s15-in-f18.1e100.net
Address 3: 173.194.45.240 par03s15-in-f16.1e100.net
Address 4: 173.194.45.244 par03s15-in-f20.1e100.net
Address 5: 173.194.45.241 par03s15-in-f17.1e100.net
Address 6: 173.194.45.243 par03s15-in-f19.1e100.net
Click to expand...
Click to collapse
This issue has probably something to do with how python was compiled to do name resolution. I remember vaguely a similar issue
on Ubuntu 12.04, with a ffmpeg compile that would always fail to resolve hostnames in URLs. No sure what the exact fix was but it had something to do with the libc and its name resolution mechanism. Or maybe something ipv6 related ?
@bubbleguuum : Thanks for checking out that important issue.
I did a little research and it seems there is some issue with DNS name resolution related to ipv6. I recompiled with "--disable-ipv6" and that seemed to fix the issue. I also found that adding the line
Code:
options single-request-reopen
to /etc/resolv.conf did the trick as well. Apparently this tells the resolver to use a new socket for ipv6 resolution instead of same one as ipv4. It thereby reduces wait-time as well. Maybe check on your end and see if that works. Otherwise I'll upload ipv4-only installations since this issue at least needs to be side-stepped.
EDIT: Hmm, looks like re-compiling with "--disable-ipv6" didn't fix the problem. Will have to look more into this issue and hold off on uploading ipv4-only python3 since it didn't fix it. In the meantime, I've attached a recovery flash.zip to the OP to add the above mentioned line to /system/etc/resolv.conf if needed, which seems to fix the issue for me.
7175 said:
@bubbleguuum : Thanks for checking out that important issue.
I did a little research and it seems there is some issue with DNS name resolution related to ipv6. I recompiled with "--disable-ipv6" and that seemed to fix the issue. I also found that adding the line
Code:
options single-request-reopen
to /etc/resolv.conf did the trick as well. Apparently this tells the resolver to use a new socket for ipv6 resolution instead of same one as ipv4. It thereby reduces wait-time as well. Maybe check on your end and see if that works. Otherwise I'll upload ipv4-only installations since this issue at least needs to be side-stepped.
Click to expand...
Click to collapse
Thank you for looking into this so fast.
My intended usage is for running the python binary from a regular non-root app (which works great!), so editing resolv.conf is not an option.
I've been researching that DNS resolving issue and could not find a clear explanation on why it fails on Android and not on other systems, and why exactly the added line in resolv.conf fixes (or rather workaround) it.
All seems to point to socket.getaddrinfo() failing for an unknown reason.
Is Python compiled with HAVE_GETADDRINFO defined ? If that's the case it uses the libc definition of getaddrinfo, otherwise
it uses an emulation function found in getaddrinfo.c.
My guess is that it is the latter, and what is causing this issue on Android.
It's probable this issue has already been solved on Android since other Python binaries exists, but Googling around do not give
much answer.
I get this error when trying to install anything with easy_install (using /system/python3.4.2/bin/easy_install-3.4 because /system/python3.4.2/easy_install doesn't seem to exist like in the OP):
Code:
# ./easy_install-3.4 feedparser
Searching for feedparser
Reading https://pypi.python.org/simple/feedparser/
Download error on https://pypi.python.org/simple/feedparser/: [Errno -2] Name or service not known -- Some packages may not be found!
Couldn't find index page for 'feedparser' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [Errno -2] Name or service not known -- Some packages may not be found!
No local packages or download links found for feedparser
error: Could not find suitable distribution for Requirement.parse('feedparser')
i got curl and the certs installed in /etc/pki/tls/certs/ as per instrucions in the OP, any help?
Thoughts on PIL?
Hmm... new problems here. Android Lollipop have no /system/etc/resolv.conf by-default and urllib patch is no more works.
How it can be fixed?
http://qpython.net/index.php
Where I can find source code and related documents for project
Pratik Raj said:
Where I can find source code and related documents for project
Click to expand...
Click to collapse
I would like to know the same, as well as what license it is under... Also, I don't care about the network, but does it work on Oreo? Also, do you think I could customise the installer to install to a non-system location (/tmp) so that my flashable zip can use python?
hackintosh5 said:
I would like to know the same, as well as what license it is under... Also, I don't care about the network, but does it work on Oreo? Also, do you think I could customise the installer to install to a non-system location (/tmp) so that my flashable zip can use python?
Click to expand...
Click to collapse
Idk about OP, but I can surely post my source for python 2.7 on arm/arm64. I compiled it like a year ago, but it wasn't too bad, just required a bunch of patches and code for dlopen
I just want to say thank you so much.
Now tested up to downloading AOSP and make toolbox you should be all set
Please give thanks to this thread: https://forum.xda-developers.com/newreply.php?do=newreply&p=43622764
Warning: I hacked my way through this stuff a few weeks ago I am not an expert!
How to compile Android Open Source Code modules
I don't compile C code on Windows machines I have no idea about that.
Notice
This guide is a quick and dirty how to make a module. It will not cover finalizing setting up the source codes for your device. It is only my goal to enable you to compile binaries such as grep, toolbox, dumpstate, dalvikvm, jack and etc.
===>] Setup Ubuntu 64bit [<===Unplug that Windows drive, plug in a fresh hard drive and install Ubuntu latest/greatest. Ignore the recommendation to downgrade gnu make!, for now.
Open a terminal and issue these commands (warning ppa repository for OpenJDK 7 is said to have a security issue?, isn't being updated?.. whatevs it works)
Code:
sudo apt-get update
sudo apt-get upgrade
sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt-get install openjdk-7-jdk
sudo apt-get install openjdk-8-jdk
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386 lib32z1 libbz2-1.0:i386
sudo apt-get install git ccache automake lzop bison gperf build-essential zip curl zlib1g-dev zlib1g-dev:i386 g++-multilib python-networkx libxml2-utils bzip2 libbz2-dev libbz2-1.0 libghc-bzlib-dev squashfs-tools pngcrush schedtool dpkg-dev liblz4-tool make optipng
(choose Java 1.7 in the following way)
Code:
sudo update-alternatives --config java
(let me know if I missed anything please)
"Tried the Android SDK only it is missing too many things we need as a developer"
===>] Setup Android Studio SDK & NDK [<===Installation Paths:
*** I install to /home/username/Android and /home/username/Android/Sdk and /home/username/Android/Sdk/ndk-bundle ***
NOTE: from here forward username will == droidvoider
Note: Android Studio IDE isn't necessary only the SDK & NDK are needed to compile AOSP.
Install Android Studio Proper: (don't worry about setting up paths we will cover that, just install it)
https://developer.android.com/studio/install.html
or
SDK Only:
Typically we install these things manually by creating the directory then just unzipping the files there.
https://developer.android.com/studio/index.html#linux-bundle (scroll down for sdk only)
Code:
mkdir /home/droidvoider/Android
mkdir /home/droidvoider/Android/Sdk
(then unzip the sdk zip to that directory. I recommend the file explorer copy/paste right click uncompress and done.)
https://dl.google.com/android/repository/tools_r25.2.3-linux.zip
Install NDK through the SDK Manger:
(from terminal 'studio.sh' and then configure, and then sdk manger --- if this is hard to figure out tell me I will elaborate)
or
Manually Install Native Development Kit -- 'c programming support'
Download the Native Development Kit from Google: https://developer.android.com/ndk/downloads/index.html
Code:
mkdir /home/droidvoider/Android/Sdk/ndk-bundle
Then just unzip the ndk files into the directory we created above.
===>] Setup your toolchain [<===** This example is arm64-v8a aarch64 **
1. Navigate to /home/droidvoider/Android/Sdk/ndk-bundle/build/tools and then open a terminal "right click open area"
2. mkdir /home/droidvoider/toolchains
3. ./make_standalone_toolchain.py --arch arm64 --api 23 --stl=libc++ --install-dir /home/mm/toolchains/aarch64-linux-android-4.9
4. cd /home/droidvoider
5. gedit .bashrc and morph this in at the bottom.. (AND edit or replace the existing PATH variable)
DON'T just PASTE IN *my* $PATH export!! I included my entire path statements to show you.
Code:
export PATH=$PATH:/usr/local/android-studio/bin:/home/droidvoider/Android/Sdk/platform-tools:/home/droidvoider/Android/Sdk/ndk-bundle:/home/droidvoider/Android/Sdk/tools
I feel this is human readable, for example change Android_Build_Out to be on your desktop instead if you want.
Code:
export PATH=$PATH:/home/droidvoider/toolchains/aarch64-linux-android-4.9
export NDK=/home/droidvoider/Android/Sdk/ndk-bundle
export SYSROOT=$NDK/platforms/android-23/arch-arm64
export TARGET=aarch64-linux-android
export HOST=$TARGET
export BUILD=x86_64-linux
export ANDROID_NDK_BIN=/home/droidvoider/toolchains/aarch64-linux-android-4.9/bin
export CC=$ANDROID_NDK_BIN/aarch64-linux-android-gcc-4.9
export CPP=$ANDROID_NDK_BIN/aarch64-linux-android-g++
export AR=$ANDROID_NDK_BIN/aarch64-linux-android-ar
export OUT_DIR_COMMON_BASE=/home/droidvoider/Android_Build_Out
Note: You might want to setup an alternate toolchain also but this is all of the puzzle pieces.
===>] Google's version of this How To -- Just for reference [<===https://source.android.com/source/requirements.html
https://source.android.com/source/initializing.html
===>] Install the repo tool [<===https://source.android.com/source/downloading.html
(don't type repo init or repo sync --- I will be taking back over from there on the next page)
Added Repair Notes -- Not part of the install!
Have you accidentally installed or removed something you shouldn't have? (welcome to development, here try this before reinstall)
sudo apt-get clean
sudo apt-get update
sudo apt-get install -f
sudo dpkg -a --configure
sudo apt-get dist-upgrade
sudo apt-get install -f
sudo dpkg -a --configure
Selecting the correct AOSP branch and downloading it.
Tested up to downloading AOSP and make toolbox -- you should be all set
===>] Match your build number to it's AOSP sources [<===preface: You can get this from your device if you're on the same build id as your the available source code from your vendor for your device. Otherwise you need to open the AP file from the firmware that matches those available sources to extract the system.img, to extract build.prop. I explain how to open a system.img file below under retrieving your hardware drivers. build.prop is in the main directory of system.img
(Many times the build number is the same. For me I believe all of MM builds are using this number.)
Assumes sources match current device, worked out true in my case
1. Plug in your device and get it connected. (DEVELOPER OPTIONS|USB DEBUGGING) and select allow on device
2. Retrieve the build number that matches the available sources for your device.
From your ubuntu terminal retrieve the build id using this command:
Code:
adb shell getprop | grep 'ro.build.id'
Yields something similar to this: [ro.build.id]: [MMB29K]
3. Match it up to the Nexus build numbers (This info is for AT&T Note 5 Marshmallow MMB29K, get your specific build number!)
https://source.android.com/source/build-numbers.html#source-code-tags-and-builds
MMB29K android-6.0.1_r1 Marshmallow Nexus 5, Nexus 5X, Nexus 6, Nexus 7 (flo/deb), Nexus 9 (volantis/volantisg)
===>] Bring down a specific AOSP source branch [<===
4. Make a directory for the source code.
Code:
mkdir /home/droidvoider/Desktop/AOSP_Android_6.01_r1
5.
Code:
cd /home/droidvoider/Desktop/AOSP_Android_6.01_r1
6. Bring down the sources, this one is approximately 13 gigabytes
Code:
repo init --depth=1 -u https://android.googlesource.com/platform/manifest -b android-6.0.1_r1
repo sync
===>] I'm not sure the rest of this is needed [<===For compiling toolbox the remainder wasn't needed.. But I have a large list of things to do so I can't test each item. If you can't compile a specific module continue reading.
===>] Merge Vendor sources & AOSP sources [<===
7. Download the available sources for your device. In this example I downloaded PE6 Marshmallow sources for AT&T Note 5:
http://opensource.samsung.com/reception/receptionSub.do?method=sub&sub=F&searchValue=SM-N920A
8. Read the readme file from the sources platform zip to understand how to merge them into the AOSP sources. For the 2 Samsungs I've worked with the idea is to replace any directory that already exists. But if there is just one file such as core.mk only replace the one file. Then edit the .mk files as described in your readme. (and/or take info from cyanogen/lineagos) -- <I can help more, ask>
note: you probably don't need to take the configs from LineageOS and put them into your .mk files. However, if you do need to get more configs then you should get a big fat message when you type make 'modulename'. At first only edit .mk files as described by vendor device source readme file.
===>] Merge in Hardware drivers and etc [<===possibly unnecessary depends what you're doing
9. Obtain a copy of the firmware for your device that matches the version of the source code you are able to download from your vendor.
for me that was Build Number: MMB29K.N920AUCU2BPE6 but your mileage will almost certainly vary!
10. Download https://github.com/anestisb/android-simg2img
11. Unzip it right in your download folder, open the folder and then 'open in terminal'
12. Make it and then move it a directory in your path. Warning: My command puts in in the Ubuntu default /bin folder.
Code:
make
sudo mv append2simg img2simg simg2img simg2simg simg_dump.py /bin
13. Uncompress the AP file from the matching firmware and extract the system.img into it's own directory
then select that folder, right click, open in terminal
Code:
simg2img system.img sys.raw
mkdir sys
sudo mount -t ext4 -o loop sys.raw sys/
14. A drive mounted, look on your task bar it should've wiggled too. Copy the etc and vendor folders into the main folder of the sources we are merging
===>] Listing and building modules [<===Navigate to the folder where you download the sources "/home/droidvoider/Desktop/AOSP_Android_6.01_r1" and open in terminal.
Code:
make modules -- list the available modules
make <module name> -- builds a specific module
example: make dumpstate
description: Will build everything needed for dumpstate and place it in the folder we specified in our export (above step). The final build line will read install and detail the final output folder
Example successful output:
[CODE]
Install: /home/droidvoider/Android_Build_Out/Android_6.01_r1/target/product/generic/system/bin/dumpstate
===>] Android Build System, basic intro [<===Notice: I built this how to to answer the same question from 3 people regarding working with toolbox and the dirtycow exploit. So I decided to give a direct example of using toolbox.c from farm-root
#ifdef
Our makefile is Android.mk and that's where we link things together. If you look at the Android.mk file for farm-root you will notice bridge.c is used 3 different times called different 'module' names. bridge_pull, bridge_push, bridge_pull_boot. Each of these will be binaries of those names.
Inside bridge.c you will see #ifdef FARM_PULL and then you will see #else and further you will see #endif which you may have noticed matches inside the Android.mk file for the bridge modules -DFARM_PULL -DFARM_BOOT <== Notice the double define on bridge_pull_boot
toolbox.c
toolbox.c is going to be the same way. You will need to copy shared.h and shared.c into the directory where toolbox.c resides. Then edit the Android.mk, in our example:
1. Navigate to this directory and open: system/core/toolbox/Android.mk
2. CTRL + F and search for "LOCAL_MODULE := toolbox"
3. Add: LOCAL_CFLAGS += -DFARM_PULL -DFARM_BOOT (in this example add one, both or even new ones you created)
4. Navigate to the main directory of the sources, you should see a Makefile and a build_64bit.sh
5. from terminal: make toolbox
Note: I think from here you can Google it out in a few minutes if that is not the case please let me know.
Working with C cross platformUbuntu is Linux based just like Android and this makes testing blocks of code extremely easy. You of course can't use Android headers and in some rare cases you can't test the code on Ubuntu at all but in most cases you can. When I want to design something for Android I open gedit and save it as a .c file. Then I compile it using gcc -o mycode mycode.c There's plenty of examples on using gcc with linux but just understand you can do it all. Then before too much work test it on Android. (helpful commands at end of post)
My advice really is to build out your small blocks of code on your linux box but then paste them into your Android program folder, edit your Android.mk, add it to your Makefile including your 'push' section so that you can simply type make push to test it.
I am in fact trying to encourage you to learn C and not so much trying to encourage you to hack things. But I know that interest/passion is what teaches, not my words and not someone else's curriculum. So in that spirit I will do my best to give examples to help you with 'whatever' it is you are passionate about. Let me know what's missing.
Don't forget to compile for Android first
Before you can test your code you will have compiled it using the cross compiler for Android. ndk-build, or the correct gcc cross compiler. (Personally I put the .c file into a directory with Android.mk and a Makefile then just type make to build it to Android)
see examples section I will add a couple examples.
Android Developer Bridge -- a developers tool
adb is included with the Android SDK along with some other tools. Some of those tools are fastboot for unlocking bootloaders and another way of flashing. There is monitor which is a cool tool for remotely viewing processes, logcat, memory dumps and etc.
But pointedly what we will use the most is simply adb.
Using adb to test your code on locked down Android systems
Shell has fairly high privileges, you may not be aware but you can execute binaries and bash scripts. We use /data/local/tmp/ for these things. You can create a directory, add or remove files, execute your binaries and even execute shell scripts using sh script.sh
ndk-build places the binary in libs/(arch type) .. For a quick test you can just open a terminal in that directory then:
Code:
adb push mybinary /data/local/tmp/
adb shell
cd data/local/tmp
chmod 777 mybinary
./mybinary
Added:
Examples of basic make files for Android.
happy coding
If you get an errorPlease reissue the command but pipe the output to a file.
make toolbox > /home/droidvoider/Desktop/build_toolbox-output.txt
zip that up with your source code, including your customized header files and attach it to this thread.
puzzles are fun but I like all the pieces
droidvoider said:
Tested up to downloading AOSP and make toolbox -- you should be all set
===>] Match your build number to it's AOSP sources [<===preface: You can get this from your device if you're on the same build id as your the available source code from your vendor for your device. Otherwise you need to open the AP file from the firmware that matches those available sources to extract the system.img, to extract build.prop. I explain how to open a system.img file below under retrieving your hardware drivers. build.prop is in the main directory of system.img
(Many times the build number is the same. For me I believe all of MM builds are using this number.)
Assumes sources match current device, worked out true in my case
1. Plug in your device and get it connected. (DEVELOPER OPTIONS|USB DEBUGGING) and select allow on device
2. Retrieve the build number that matches the available sources for your device.
From your ubuntu terminal retrieve the build id using this command:
Code:
adb shell getprop | grep 'ro.build.id'
Yields something similar to this: [ro.build.id]: [MMB29K]
3. Match it up to the Nexus build numbers (This info is for AT&T Note 5 Marshmallow MMB29K, get your specific build number!)
https://source.android.com/source/build-numbers.html#source-code-tags-and-builds
MMB29K android-6.0.1_r1 Marshmallow Nexus 5, Nexus 5X, Nexus 6, Nexus 7 (flo/deb), Nexus 9 (volantis/volantisg)
===>] Bring down a specific AOSP source branch [<===
4. Make a directory for the source code.
Code:
mkdir /home/droidvoider/Desktop/AOSP_Android_6.01_r1
5.
Code:
cd /home/droidvoider/Desktop/AOSP_Android_6.01_r1
6. Bring down the sources, this one is approximately 13 gigabytes
Code:
repo init --depth=1 -u https://android.googlesource.com/platform/manifest -b android-6.0.1_r1
repo sync
===>] I'm not sure the rest of this is needed [<===For compiling toolbox the remainder wasn't needed.. But I have a large list of things to do so I can't test each item. If you can't compile a specific module continue reading.
===>] Merge Vendor sources & AOSP sources [<===
7. Download the available sources for your device. In this example I downloaded PE6 Marshmallow sources for AT&T Note 5:
http://opensource.samsung.com/reception/receptionSub.do?method=sub&sub=F&searchValue=SM-N920A
8. Read the readme file from the sources platform zip to understand how to merge them into the AOSP sources. For the 2 Samsungs I've worked with the idea is to replace any directory that already exists. But if there is just one file such as core.mk only replace the one file. Then edit the .mk files as described in your readme. (and/or take info from cyanogen/lineagos) -- <I can help more, ask>
note: you probably don't need to take the configs from LineageOS and put them into your .mk files. However, if you do need to get more configs then you should get a big fat message when you type make 'modulename'. At first only edit .mk files as described by vendor device source readme file.
===>] Merge in Hardware drivers and etc [<===possibly unnecessary depends what you're doing
9. Obtain a copy of the firmware for your device that matches the version of the source code you are able to download from your vendor.
for me that was Build Number: MMB29K.N920AUCU2BPE6 but your mileage will almost certainly vary!
10. Download https://github.com/anestisb/android-simg2img
11. Unzip it right in your download folder, open the folder and then 'open in terminal'
12. Make it and then move it a directory in your path. Warning: My command puts in in the Ubuntu default /bin folder.
Code:
make
sudo mv append2simg img2simg simg2img simg2simg simg_dump.py /bin
13. Uncompress the AP file from the matching firmware and extract the system.img into it's own directory
then select that folder, right click, open in terminal
Code:
simg2img system.img sys.raw
mkdir sys
sudo mount -t ext4 -o loop sys.raw sys/
14. A drive mounted, look on your task bar it should've wiggled too. Copy the etc and vendor folders into the main folder of the sources we are merging
Click to expand...
Click to collapse
And where is exactly the main folder? Sorry, Im just confused
DigitalDoraemon said:
And where is exactly the main folder? Sorry, Im just confused
Click to expand...
Click to collapse
it's no problem this stuff isn't easy to just figure out on your own. remember to substitute droidvoider for your ubuntu user name
In this example my sources are on my desktop in a folder named Android_6.01_r1
Sources for toolbox for example:
/home/droidvoider/Desktop/Android_6.01_r1/system/core/toolbox/<sources will be here including Android.mk>
Script for modules, including toolbox
/home/droidvoider/Desktop/Android_6.01_r1/Makefile <--- this is our modules script, if you will
<open a terminal in the above folder then use that Makefile like so>
make toolbox <---- this will compile only what is needed to compile the module 'toolbox' (this takes a minute)
Out export folder we decided in ./home/droidvoider/bashrc
/home/droidvoider/Android_Build_Out/Android_6.01_r1/target/product/generic/system/bin
Anybody, please compile grep utility for arm and x86... Minimum Platform Version Android 4.0.3, API Level - 15
Thanks
Great & useful .