Database Info

Kernel Optimization

The lovely thing about having a Linux Kernel is that we can do a lot to optimize it.
I figured it would be a great idea to set up a thread where we can bring all these optimizations into one place.
Mind you, I'm no expert by any means...
For my (possibly useless) contribution:
When we compile the kernel to the G1, do we compile it with the general Arm eabi? If so, is that made for ALL ARM cpu's? We have an ARM6 in our G1's. If we set it up to optimize to ARM6, can we set it up to use the CPSID/CPSIE instruction instead of the SWP instruction? Are the drivers open source? Can we recompile them to do the same also? To learn more about why, read this:
http://forum.xda-developers.com/showthread.php?t=516223
That's where I got the idea from.
SWP is a deprecated opcode in armv6 that decreases performance due to the way it locks the bus.
Click to expand...
Click to collapse
Anyhow, whatever you guys post here, I will add to this post. Also, to the people to compile the kernels, can you answer my questions?

answers to some of your questions.
hi i usually don't post here since i am not a developer but in this case i think i can answer some of your questions (since i sometime make custom kernels for my laptop). kernel optimazation is used to removed excessive drivers and services from the kernel that you don't need that slows down your computer or in this case phone. i believe htc already does this when they configured android for the g1. most cell phone manufactures already do this to ensure that there products processor use energry efficiently and and run as smoothly as possible.
When we compile the kernel to the G1, do we compile it with the general Arm eabi? If so, is that made for ALL ARM cpu's? We have an ARM6 in our G1's
Click to expand...
Click to collapse
yes, arm eabi is made for all general arm cpu's i believe you can complie it for arm 6 specifically though. how do you know that htc hasn't already optimize the kernel for that specific processor? also if they didn't you wouldn't really see any noticible speed improvements by compliling the kernel for this specific processor.
Are the drivers open source?
Click to expand...
Click to collapse
no, futhermore they don't offer the drivers for the g1 except execpt for the usb driver so your computer can recognise the g1 as mass storage device.
If we set it up to optimize to ARM6, can we set it up to use the CPSID/CPSIE instruction instead of the SWP instruction?
Click to expand...
Click to collapse
i don't know sorry. thats all i know i'm really not a kernel, coding expert i hope this helps in your task.

bananajama said:
no, futhermore they don't offer the drivers for the g1 except execpt for the usb driver so your computer can recognise the g1 as mass storage device.
Click to expand...
Click to collapse
I think you meant to say "ADB interface" instead of "mass storage device."

t1n0m3n said:
I think you meant to say "ADB interface" instead of "mass storage device."
Click to expand...
Click to collapse
The system type is set to "Quaalcom MSM", so I would imagine that it's optimizing for arm6l.

cyanogen said:
The system type is set to "Quaalcom MSM", so I would imagine that it's optimizing for arm6l.
Click to expand...
Click to collapse
Sounds like it. Well, other than that, are there any other obtimizations anyone else can think of?

there are few things in linux that you can play with to see if you get any performance increase. It's mainly in memory management. run ls on /proc/sys/vm. you can play with the page cache but I doubt that you'll get much perf increase from it since, since most of the things we can play with are for write back cache. If we have access to the dalvikvm we may be able to do more optimization. The one thing that I've always wanted to be able to do is to pin Home in memory, and not allow garbage collector to kill it, so when we exit out of memory intensive apps, like browser, gallery, etc, we do not have to wait for Home to reload. If we have more control over garbage collection we may be able to do more, but I don't know much about the dalvikvm at all.

taking a quick look at the vm. The only nob that may have any effect on android may be vfs_cache_pressure. theoretically setting to 200 would push fs cache out quicker and give more memory to the vm. default is set to 100. I set it to 200 and 300 but didn't notice any difference.

knaries2000 said:
there are few things in linux that you can play with to see if you get any performance increase. It's mainly in memory management. run ls on /proc/sys/vm. you can play with the page cache but I doubt that you'll get much perf increase from it since, since most of the things we can play with are for write back cache. If we have access to the dalvikvm we may be able to do more optimization. The one thing that I've always wanted to be able to do is to pin Home in memory, and not allow garbage collector to kill it, so when we exit out of memory intensive apps, like browser, gallery, etc, we do not have to wait for Home to reload. If we have more control over garbage collection we may be able to do more, but I don't know much about the dalvikvm at all.
Click to expand...
Click to collapse
dxtop, which is a home replacement, does exactly that. It keeps itself in memory so going back to it is much faster than the default home application. And in low memory situations, it tries to free up as much resources as possible while still staying in memory.

Discussion on User.conf/tweaking AOSP ROMs

I shared my user.conf a while back with the guys on the xROM thread. I got really positive feedback from them so I decided to share it in a few other threads. JAC is included a lot of my ideas in xROM 1.4 and is going to include more in 1.5 It has grown and become more popular so I decided to start a thread at the request of a few people who use my config.
I didn't think it deserved the space until now to make a thread so I never started one. I'm not trying to make this thread all about me. I've said more than once, I would love for someone to come up with a config that is better than mine because I will be the first one to use it. I just don't want to keep derailing other conversations in other threads anymore. And while I appreciate all of the emails that I get, I think a separate thread will be an easier place to discuss this and help people out.
So I will start by posting giving a general outline of how I came up with my settings. If you want to give me feedback, I appreciate it. Even if you have emailed me in the past or posted something on one of the threads I would appreciate a post in this thread. If only the people who have problems post here I won't know how many people it is helping or not.
That is only part of the thread though. If you have a configuration that you have seriously tested and you want other people to try it, post your method and how to set it up and let's see what we can come up with. Please don't just post random configs hoping someone else will test them for you. I don't want the thread to turn into a confusing mess that doesn't help anyone. My goal is to have this thread spawn some new ideas that will make everyone's phone faster. If you've done your homework and you think you have something good, let's share it. I never thought that my config would be used by so many people. I was just trying to make my phone faster. Maybe you can do better.
Testing Method:
I generally use two apps to benchmark the performance of my phone. They are Benchmark Pi and Benchmark. Both are available on the market. I use Benchmark Pi for a rough comparison because it is faster and focuses mostly on how fast the CPU is running. When I get down to fine tuning I use Benchmark. It takes longer to run but it measure graphics, CPU, memory, and filesystem performance. For Benchmark Pi I usually will run it 10 times and use the average response. With Benchmark I set it to run each bench 5 times each and it automatically averages the values. Then I export that to a csv file and compare them that way.
I think I am a pretty heavy user of my phone so I took what I normally use and added a little bit to it. Here is what I used:
- All testing was done on xROM
- Over 120 apps installed
- 5 home screens
- 18 widgest and 8 shortcuts to apps (I can post specifics if anyone cares)
- I run the phone until it is definitely swapping. I play music, open large web pages, run CPU intesive apps, etc.
- ask me if you want more details
I use Advanced Task Manager, which I highly recommend, to keep an eye on what is running and kill any apps that are not being used by me or my desktop. I try to simulate heavy usage of my phone with only the things running that should be. I also use cat /proc/swaps, free, and sh /system/bin/swap -s (sh /system/sd/userinit.sh -s for othe ROMs).
I started out comparing Compcache and Linux Swap since those are the biggest choices. Compcache came out slightly better with the recommended configs that I could find at the time. Then I benchmarked and tested each of the Compcache settings. I found huge improvements made from tuning Compcache, especially adding backing swap. I know it got a bad rap in earlier builds because, well, it probably wasn't very good. But 0.6 is good and it has a bright future. Read this article http://lwn.net/Articles/334649/
Then I tuned swapiness, CPU, and tried on VM. It was mostly just painstaking, tedious testing of one setting to the next. I can honestly say that I have tuned every setting in the User.conf. If I change any of them the benchmarks go down. So when people tell me they tweaked this or that I don't want to say anything because I have probably tested that exact scenario and I got worse performance. But maybe it is working for them, I haven't seen any benchmark numbers. I tried on the VM but nothing seemed to help very much and I didn't want to cause problems so I stuck with the default settings there. Maybe someone can come up with better settings for the VM if that is their area of expertise.
Results:
I'm not going to post some grand report like I would at work. I'm not getting paid for this. Plus not very many people would look at it and I don't know how much it would mean in the long run.
The easiest comparision I can make is using Benchmark Pi. The average of all of the users who have ever tested using that app is around 15 seconds to calculate pi. Stock G1's will calcluate it in about 16 or 17 seconds depending on how loaded they are. xROM stock was getting a result of about 13.5 seconds with just scaling the CPU up to 528 MHz. After tweaking all of the other settings I got down to sub 12 seconds. My best time is about 11.6 seconds. I saw someone with 11.5 who used xROM 1.4. When I loaded CM 4.0.4 last night I was getting just under 13 seconds with my best times. Still a very good time considering everything that I have running.
I did a lot of other testing with Benchmark to monitor the filesystem and memory. I monitored /proc/swaps to see how much memory Compcache was using and how much was being swapped.
I learned a lot about the phone doing all of this. For instance, when you set the CPU max threshold really low, it is going to keep bumping up the CPU really fast until you get to the max. So you have a pretty good delay until you get to the max CPU frequency because the fequency keeps changing and pauses when that happnes. But then it is good until it drops again. If you make it too high, it will overwork the processor and you will see delays again. The key is to find the best setting that allows the processor to be exercised but not overworked. I don't have the time to write down everything I learned from it and you probably don't want to read it. But if you ask a specific question I will be happy to answer it.
I'd like to thank Huanyu for his thread (http://forum.xda-developers.com/showthread.php?t=542899). I wouldn't have been able to tune any of this without it and for JAC and Manup for helping me so much with xROM.
I hope this thread is helpful. Let me know what you think of my work and post your ideas too.
I have links to installing my User.conf in my signature. This post is long enough without putting that in here too. And this is more about discussion than pushing my settins on people. I don't know if mhy config will work at all with Hero. I haven't done much testing on it. I plan to do the same project for Hero when I have time. JAC has volunteered to help me out with that. If anyone wants to try it, go for it. But I don't want to confuse this thread too much by discussing Hero and the AOSP ROMs. Maybe I will start another thread for just Hero.
I look forward to seeing where this thread goes.

Ok, I decided to add more now that I have some extra time. I'm working to get backing swap working with a swap file instead of using a swap partition. A lot of people don't have or want a third partition on their phone and backing swap helps out a lot. Compcache 0.6 supports this but it wouldn't work on the G1 (or any ARM processor). I opened an issue with the compcache development team. They gave me a fix which allowed me to enable it but it ended up trashing my ext3 partition. I had to wipe my phone because the data became so corrupted. The issue is still open and the compcache developer is working on resolving it. When it is resovled, I will post an updated user.conf file that uses swap file instead of swap partition.

Many thanks for your test.
Just one question, why you don't turn the linux swap on in Cyanogen's ROM?
I found you turn on the compcache and backingswap in user.conf.
Is it better than compcache with linux-swap?

min scaling frequency 192000?
I applied the settings from your user.conf in the Cyan thread you linked to except the min scaling frequency because it seems to be set to a non-existent step (192000). Is that a typo or...? Is that user.conf the latest one you tested with CM 4.0.4?
Thanks for doing all this testing and sharing your settings!

fengwuyu said:
Many thanks for your test.
Just one question, why you don't turn the linux swap on in Cyanogen's ROM?
I found you turn on the compcache and backingswap in user.conf.
Is it better than compcache with linux-swap?
Click to expand...
Click to collapse
Compcache with backing swap uses the linux swap partition but it lets compcache manage both the compressed and non compressed storage. When a file can't or shouldn't be compressed it doesn't have to pass it back to the OS to deal with. It just moves it directly to swap. That is why it shows 100% good compression with backing swap turned on. All of my testing has shown that compcache + linux swap is much slower and more cpu intensive thand compcache + backing swap.

my experiences are non-scientific and completely anecdotal, but I like the way my phone behaves w/ a 92 meg linux swap partition enabled w/o compcache. I'm giving up a bit of responsiveness across the board (once it starts swapping) for more virtual memory. Since I have a class 6 card, the lag is bearable (i can't imagine it would be on a slower one).
I like to have my browser still be in memory after loading terminal, gmail, or a couple of other apps. but that's just me & how i use my phone. it's not a perfect config by any means, but it works best for me

ei8htohms said:
I applied the settings from your user.conf in the Cyan thread you linked to except the min scaling frequency because it seems to be set to a non-existent step (192000). Is that a typo or...? Is that user.conf the latest one you tested with CM 4.0.4?
Thanks for doing all this testing and sharing your settings!
Click to expand...
Click to collapse
It's not a typo. Some of the overclocking apps use it as an option. I've used it for a long time and so have other peole. If you want to change it, that is fine. I used 245760 for a while because of some apps I was running. The CPU frequency is one area that you can play with to meet your needs without changing much. I wouldn't suggest going below 192. You can try it but it makes the phone slow to wake up for some people.

alapapa said:
my experiences are non-scientific and completely anecdotal, but I like the way my phone behaves w/ a 92 meg linux swap partition enabled w/o compcache. I'm giving up a bit of responsiveness across the board (once it starts swapping) for more virtual memory. Since I have a class 6 card, the lag is bearable (i can't imagine it would be on a slower one).
I like to have my browser still be in memory after loading terminal, gmail, or a couple of other apps. but that's just me & how i use my phone. it's not a perfect config by any means, but it works best for me
Click to expand...
Click to collapse
That is a pretty good configuration. You will probably see better performance at a slightly higher swapiness with 92MB of swap. I think around 40 is a good setting.
You are right with a Class 6 card the performance is not that bad. You won't see a lot of lag until the file grows to the point that the OS needs to clean it up to make room for more memory. Then it will slow down. If you reboot every day it's not as much of an issue.
I've tried to come up with a configuration that works well for everyone. I helped a guy with a Class 2, 1GB card last night. He thought that my config was slowing him down on CM 4.0.4. We reparititoned and gave him a smaller ext (256 MB) to free up more memory on his FAT32 partition. I have a lot on my phone and my dalvik-cache is on my ext partition and I am using about 215MB so he should be fine with that setting. He got all of his apps installed and setup everything. Then he tested his it out with the stock settings. After he had an idea of how it was running he switched to mine and saw a noticable improvement. Using Benchmark Pi he had the same score I had with a Class 6 8GB card running on CM 4.0.4 the other day. Now my card will beat him out in other tests. Anything that requires swapping to the linux swap partition or a lot of I/O on the SD card will be better on my phone. But because he was using Compcache his performance was still pretty good.
Also, if you enable backing swap your browser should still be in memory after loading other apps. One of the biggest things people notice is that after they have been using the browser or something else that takes a lot of memory and they hit the home button they don't have to wait for it to reload. It's just there waiting for them.

Yes, he speaks the truth. After updating to a recent Cyanogen ROM, my phone just got super clunky and annoying to use, so I set out to do a completely fresh install: resizing, reformatting, wiping, fresh ROM flash, and fresh app dls. I thought since I was stuck using the stock 1GB SD card (for now at least) that is inherently Class 2, it was causing me a major bottleneck since CC and BackingSwap both employ fair usage of your SD card. Well after updating to Cyanogen's 4.1.2.1 and installing all my apps, Benchmark Pi clocked in at about 15.5ms - 16.5ms using Cyano's stock CC settings built into his ROM. Next I pushed Taylor00's user.conf with the 0.1.4.1 userinit.sh. After rebooting, my phone clocked in at 12.6ms - 13.2ms on Benchmark Pi, a definite improvement.
-Maleko48

miketaylor00 said:
That is a pretty good configuration. You will probably see better performance at a slightly higher swapiness with 92MB of swap. I think around 40 is a good setting.
You are right with a Class 6 card the performance is not that bad. You won't see a lot of lag until the file grows to the point that the OS needs to clean it up to make room for more memory. Then it will slow down. If you reboot every day it's not as much of an issue.
I've tried to come up with a configuration that works well for everyone. I helped a guy with a Class 2, 1GB card last night. He thought that my config was slowing him down on CM 4.0.4. We reparititoned and gave him a smaller ext (256 MB) to free up more memory on his FAT32 partition. I have a lot on my phone and my dalvik-cache is on my ext partition and I am using about 215MB so he should be fine with that setting. He got all of his apps installed and setup everything. Then he tested his it out with the stock settings. After he had an idea of how it was running he switched to mine and saw a noticable improvement. Using Benchmark Pi he had the same score I had with a Class 6 8GB card running on CM 4.0.4 the other day. Now my card will beat him out in other tests. Anything that requires swapping to the linux swap partition or a lot of I/O on the SD card will be better on my phone. But because he was using Compcache his performance was still pretty good.
Also, if you enable backing swap your browser should still be in memory after loading other apps. One of the biggest things people notice is that after they have been using the browser or something else that takes a lot of memory and they hit the home button they don't have to wait for it to reload. It's just there waiting for them.
Click to expand...
Click to collapse
if i notice it being laggy, i just swapoff & swapon and it's like i'm fresh off a reboot.
what would be epic is if someone ported anacron / atd to android -- i could set it to do this at 4am.

alapapa said:
if i notice it being laggy, i just swapoff & swapon and it's like i'm fresh off a reboot.
what would be epic is if someone ported anacron / atd to android -- i could set it to do this at 4am.
Click to expand...
Click to collapse
I believe that crontab is a part of busybox. You can use that.

I'm still not following
miketaylor00 said:
It's not a typo. Some of the overclocking apps use it as an option. I've used it for a long time and so have other peole. If you want to change it, that is fine. I used 245760 for a while because of some apps I was running. The CPU frequency is one area that you can play with to meet your needs without changing much. I wouldn't suggest going below 192. You can try it but it makes the phone slow to wake up for some people.
Click to expand...
Click to collapse
What I'm confused about is the significance of choosing 192000 as a scaling frequency at all. My understanding is that the CPU can only run at certain frequencies and 192000 is not one of them. If you set 192000 as your minimum scaling frequency, doesn't that have exactly the same effect as setting it for 245760? Since there are no frequencies available between 122880 and 245760, any setting (for minimum scaling frequency) between those would default to the higher frequency, in this case 245760. Am I missing something here?
The reason I thought it might be a typo is because there is a 19200 frrequency available, but from all reports anything near that low will just lock up the phone.

ei8htohms said:
I applied the settings from your user.conf in the Cyan thread you linked to except the min scaling frequency because it seems to be set to a non-existent step (192000). Is that a typo or...? Is that user.conf the latest one you tested with CM 4.0.4?
Thanks for doing all this testing and sharing your settings!
Click to expand...
Click to collapse
Here is a copy/paste of that section of the latest user.conf set with MT00's settings:
Code:
#cpu clock
proc_cpu{
proc_cpu_en=1 # enable(1) or disable(0) user cpu configurations
# freqency options
# 19200
# 122880
# 128000
# 245760
# 384000
# 528000
scaling_min_freq=192000 # default 245760
scaling_max_freq=528000 # default 528000
sampling_rate=2000000 # default 2000000 depending on kernel version
powersave_bias=0 # default 0, (200 since CM3.9.6+ )
up_threshold=45 # default 40, percent cpu usage before going up a speed step
While 192000 is not an option listed, here is the output of my userinit.sh:
Code:
C:\Documents and Settings\user>adb remount
remount succeeded
C:\Documents and Settings\user>adb shell
sh-3.2# sh /system/sd/userinit.sh -s
sh /system/sd/userinit.sh -s
=== user.conf ===
*** general ***
apps2sd=0
media2sd=0
*** CompCache ***
compcache_en=1
cc_memlimit=18
cc_disksize=32
cc_backingswap_en=1
cc_backingswap=/dev/block/mmcblk0p3
swappiness=28
*** Swap File ***
swap_file_en=0
linux_swap_file_size=32
linux_swap_file=/system/sd/swap.file
*** Linux Swap ***
linux_swap_en=0
linux_swap_partition=/dev/block/mmcblk0p3
*** VM ***
sys_vm_en=1
page_cluster=3
laptop_mode=0
dirty_expire_centisecs=3000
dirty_writeback_centisecs=500
dirty_background_ratio=5
dirty_ratio=10
*** CPU ***
proc_cpu_en=1
scaling_min_freq=192000
scaling_max_freq=528000
sampling_rate=2000000
powersave_bias=0
up_threshold=45
=== CompCache status ===
CompCache version 0.6+
Compcache enabled
CompCache: MemLimit 18432(system) 18432(user)
CompCache: BackingSwap /dev/block/mmcblk0p3(system) /dev/block/mmcblk0p3(user)
CompCache: cc_swappiness - 28(system) 28(user)
=== CompCache status output ===
BackingSwap: /dev/block/mmcblk0p3
DiskSize: 32130 kB
MemLimit: 18432 kB
NumReads: 8048
NumWrites: 10222
FailedReads: 0
FailedWrites: 0
InvalidIO: 0
NotifyFree: 3870
PagesDiscard: 0
ZeroPages: 177
GoodCompress: 100 %
NoCompress: 0 %
PagesStored: 4340
PagesUsed: 1060
OrigDataSize: 17360 kB
ComprDataSize: 4078 kB
MemUsedTotal: 4240 kB
BDevNumReads: 1233
BDevNumWrites: 1731
=== VM status ===
Set VM: page-cluster - 3(system) 3(user)
Set VM: laptop_mode - 0(system) 0(user)
Set VM: dirty_expire_centisecs - 3000(system) 3000(user)
Set VM: dirty_writeback_centisecs - 500(system) 500(user)
Set VM: dirty_background_ratio - 5(system) 5(user)
Set VM: dirty_ratio - 10(system) 10(user)
=== CPU status ===
Set CPU: scaling_min_freq - 192000(system) 192000(user)
Set CPU: scaling_max_freq - 528000(system) 528000(user)
Set CPU: sampling_rate - 2000000(system) 2000000(user)
Set CPU: powersave_bias - 0(system) 0(user)
Set CPU: up_threshold - 45(system) 45(user)
That setting seems to take.

You can set the frequncy to 1234 if you want to. I wouldn't recommend it but you can. I don't know why it isn't listed as an option. It should be. It is a valid setting.

miketaylor00 said:
You can set the frequncy to 1234 if you want to. I wouldn't recommend it but you can. I don't know why it isn't listed as an option. It should be. It is a valid setting.
Click to expand...
Click to collapse
You actually cant set the frequencies to any arbitrary number. There are a set of defined frequencies in the arch/arm/mach-msm/clock.c file. So you can't try to clock your CPU at let's say 523.12 or 99.8 mHz atm. Unless that frequency table is expanded to include every single frequency. Where's coolbho3000 he might be able to explain it better. lol

After the phone has rebooted you can use the following command to see if everything is running properly:
Code:
su
sh /system/sd/userinit.sh -s
Everything seems to work right for me on the install but after I reboot and enter this code. I get "sh: Can't open /system/sd/userinit.sh".
Does this mean it did not work?

bigragu said:
After the phone has rebooted you can use the following command to see if everything is running properly:
Code:
su
sh /system/sd/userinit.sh -s
Everything seems to work right for me on the install but after I reboot and enter this code. I get "sh: Can't open /system/sd/userinit.sh".
Does this mean it did not work?
Click to expand...
Click to collapse
Which ROM are you on? Different builds require the userinit in different places.
Have you done this?
Code:
su
cd /system/sd
chmod 755 userinit.sh
chmod 755 user.conf
reboot

overground said:
Which ROM are you on? Different builds require the userinit in different places.
Have you done this?
Code:
su
cd /system/sd
chmod 755 userinit.sh
chmod 755 user.conf
reboot
Click to expand...
Click to collapse
I'm on CyanogenMod v4.0.4. I clicked the link from this post which lead me to the code he had posted in another thread. I followed the instructions there. I don't believe the code was exactly what you have here. Will try. Thanks.

andonnguyen said:
You actually cant set the frequencies to any arbitrary number. There are a set of defined frequencies in the arch/arm/mach-msm/clock.c file. So you can't try to clock your CPU at let's say 523.12 or 99.8 mHz atm. Unless that frequency table is expanded to include every single frequency. Where's coolbho3000 he might be able to explain it better. lol
Click to expand...
Click to collapse
That is not true at all I just set mine to 192123. Are you telling me that is in the clock.c file?
Code:
# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq;
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq;
192123

bigragu said:
After the phone has rebooted you can use the following command to see if everything is running properly:
Code:
su
sh /system/sd/userinit.sh -s
Everything seems to work right for me on the install but after I reboot and enter this code. I get "sh: Can't open /system/sd/userinit.sh".
Does this mean it did not work?
Click to expand...
Click to collapse
Most likely you don't have a userninit.sh. If you are on CM 4.x it doesn't come by default. Follow the steps in this thread:
http://forum.xda-developers.com/showthread.php?t=542899

ram development "lets show google how its done"

ok its simple we all know the benefits of more ram. we all know that android slices up the ram into static addresses within the kernel. i would like to change that as we are in 2009 almost 2010 and we still rely on static ram allocation for the camera, gpu, sensors, radio. the g1 has 192mb of ram. yet we only get 97 for programs to use. i would like to place the camera, gpu, etc into dynamic addresses that we can change within the os without flashing a new kernel. a service, kernel code, module, or any other method could do this. we just have to make it happen. we could do it three ways i see.
1. a system managed ram allocation using a service to detect the percentage used and increase ram on the go.
2. a kernel script or service that only activates and allocates ram when needed. such as once a 3d program is loaded the 8mb magically comes back.
3. a virtual device ram allocation service that makes the device think it has the ram all along, but only actually gets it when it needs it. kinda like the way apps2sd works.
4.(by IConrad01) run perl scripts through ase and use ksplice to patch the gpu1 memory in kernel. one script on and one for off.
any other methods to implement this proper are very welcome and encouraged.
the idea behind this is a proper way to get back more ram for our programs. the idea is simple when 3d is not in use we can used the 8mb it would have used for something else. this applys to everything that has its own ram. camera, etc. say we did this and only got 120 free. thats almost 25% more ram. plus we could go the extra step and let the user choose how much each thing gets(gpu settings 4mb,8mb,12mb. camera on, off. etc) so when it needs it thats what it gets. great if you like/dislike 3d games.
on another note this is the kind of work that gets you noticed. the kind that gets you hired or gets you respect. google would be proud and might implement it and you and i know it. i hope we are successful and google uses it mainstream.
so lets debate a way/ways to do this
provide alpha tests of different ways to do this
and finally lets post a beta so everyone can enjoy.
atm ksplice with perl scripts is where we are going will update as this progresses.

Doesnt Jacs update to his hero rom have this, im probably mistaken

jad011 said:
Doesnt Jacs update to his hero rom have this, im probably mistaken
Click to expand...
Click to collapse
No. The current implementation of this basically cuts the graphics memory in half.
This would dynamically allocate the memory as needed. Much more efficient way of doing it.

he has a static change that takes the ram away from the gpu by a kernel edit. this is to have both more ram but still give the gpu what it needs to perform properly but only when it needs it or you chose to give it.

Are you referring to RAM or storage space? You say RAM in the title but then it sounds like you're talking about storage space before making me think it's RAM again later on. I'm confused :/ I'd really like additional free RAM, anything that helps reduce swapping to SD card is good but storage space I have plenty of.

all of this would apply to ram only. swap and compcache could use the same idea to dynamically allocate swap but that is still a cache for programs. sorry if its confusing. simple way to explain. i want all the ram i can get but still use everything the phone has properly, like games and the camera.

Gotcha and understood. I see so many people say RAM and memory when they're referring to storage space sometimes that I want to scream but extra RAM on the G1 would be more than welcome. Hopefully others with the brains to make it happen can do it

actually anyone with a basic knowledge can help. mostly there are a couple things to developing a good idea.
first the concept.
then ideas on how to do it.
next a map or guide to what does what.
coding different parts
putting them together
debug and test
even if all you do is photoshop the ui or provide ideas it helps. once some different people get aboard this should be working sooner than you think.

Of the three options listed, the kernel script (via ksplice) seems like it would be the shortest route to functionality, but frankly it also seems a bit... well, hackish -- when compared to the virtual device allocation scheme. It would take a hefty re-write of the kernel, I suspect, to accomplish; but being able to resize various devices on the fly would be a good thing. Shrinking CompCache based on real RAM usage for example would probably speed up the phone under light loads.
Just thinking in text here.

personal i think its going to start simple(script or root app) and go straight to gpu shared on or off. choose and reboot. however i do not want to stop there like winmo did. someone could create a short kernel patch that checks for a service or script and gets value or defaults to standard method. then we would have to come up with how to manage the switch(realtime vs. reboot) ideally we would remap the memory from scratch and rebuild the memory management services to dynamically allocate the ram for all devices(128+ ram) as needed. we have a start, we know where and how to turn the gpu1 shared on or off. now lets make it more than a hack and work on how to make our phones better. btw this is not a waste of time. this extends to other phones and the future way to set up our phones from default.

jokersax11 said:
personal i think its going to start simple(script or root app) and go straight to gpu shared on or off. choose and reboot. however i do not want to stop there like winmo did. someone could create a short kernel patch that checks for a service or script and gets value or defaults to standard method. then we would have to come up with how to manage the switch(realtime vs. reboot) ideally we would remap the memory from scratch and rebuild the memory management services to dynamically allocate the ram for all devices(128+ ram) as needed. we have a start, we know where and how to turn the gpu1 shared on or off. now lets make it more than a hack and work on how to make our phones better. btw this is not a waste of time. this extends to other phones and the future way to set up our phones from default.
Click to expand...
Click to collapse
The only problem with this is when the ram is divided, everything in it is cleared, thats why it is set at boot in the kernel. Think of it like repartitioning your SD. Now it would probably be possible have everything mapped to ONE big RAM table & then build something similar to a scheduler like BFS for RAM that would allocate it as necessary. This is messy though.
Just my $.02

well yes but it is possible to create shared memory through c++ http://www.go4expert.com/forums/showthread.php?t=8461
however i think the vm heaps has either a 6 or 16mb limit. anyways the service would basically do a memory patch over the kernel in realtime(like overclock) telling the gpu the memory is there all the time. meanwhile it allocates dynamic shared memory from the application memory for the gpu to read and write. meanwhile an app tied to a script would set the parameters and settings.

Okay -- I know ksplice works with perl; and I know that we can run perl scripts using the ASE .apk.
So... it seems to me that it /should/ be possible to use a patch that reverts the patched kernel back to the previous state, and a patch that switches the original state to the GPU1=0 MB state. From there, we could set up two scripts; one for GPU1-off and one for GPU0-on. Each would simply be an invocation of ksplice to input the specific patch (which would have to be stored on the sd card. [it would be safer if in the ext partition]).
We'd have to set up a secondary RAM device, rather than just adding it to the standard RAM pool... something like establishing it as a /dev/block/shm.
The point of all the above is that when you click on the GPU1-off script, it "reclaims" that RAM and gives you a device that's standard RAM. When you click on the GPU1-on script, it disables that device (forcing the reload of all programs using that RAM) and re-enables the GPU1 device.
Even better yet, the Android Scripting Environment program, much like GScript, supports desktop icons.
So -- very initially speaking, assuming we can get ksplice working on Android (which just requires a static binary, IIRC), we can get a simple pair of icons; one of which frees up the GPU1's RAM; and the other of which re-enables the GPU1.
Since the only times it's going to be "called up" are when a game is actively being played, or something similar, this should be a mostly-satisfactory route to begin with, before we can get into the whole scheduled/scaled script running as a daemon in the background.
Once we get /there/, then we can start looking at true virtual device allocations and shared memory and the like.
Although, I do have to say that I'm not entirely sure how comfortable I am constantly repatching the kernel. Definitely, we're going to want to recommend that people back up their original, unpatched, boot.img files!

IConrad01 i added your method to first post as it was a great start to this project. if you want me to rephrase just post what you want and ill change it. the only issue i might see is what geniusdog254 said it might unallocate the wrong thing and create a reboot. but some clever coding should prevent this.

Nah, that's all good.
There's an easy way to prevent it from becoming a reboot-able issue. Enable the GPU-space device as "swap". This could be done one of two ways; tell the phone that the new 8 MB RAM "device" is to be considered the /first/ swap device to be used, and just leave it "as is". The r/w times would be for true RAM. The other option assumes we have the latest CompCache (0.6+) -- as it allows for multiple instances of CompCache. The theme here should be obvious.
Either way, the result is that only swappable pagefiles would be lost when the device is turned back into GPU memory. No system-vital memory would get tossed. Seems simple enough to accomplish, perhaps. But then again, I Am Not A Programmer.

ok so what are the alternatives to ksplice for the swap device change. since we could make it swap/compcache anyways is it possible to change on the fly like the way overclocking widget does?

Ksplice allows you to make kernel updates and have them implement themselves without having to reboot the device. The swap device stuff would have to be implemented on a script-level. Two different things: I.e.; the script invokes ksplice to install a specific patch, which causes the kernel to recognize the newly freed-up RAM as a specific device (I.e.; such as /dev/ram1 as opposed to /dev/ram0). I am not sure I'm saying/thinking of this properly here: I'm operating on the edge of what I know about Linux.
Anyhow -- the trick is, when we get the kernel to disable the GPU's memory, we enable a new RAM-disc swap partition with its swappiness set to the maximum rating so it picks up everything first. Since it is /actually RAM/, it'll perform like real RAM. We're only 'declaring' it as swap in order to prevent forced reboots when turning it off.
The above wouldn't just be a ksplice thing; ksplice is just to implement daproy's kernel patch and thus free up the devoted memory to the GPU1. This memory would then be allocated to a new device (thus preventing the RAM-dump forced reboot) and /that/ would be set to swap status to ensure no "mission-critical" files go on it.
Then, once we get that built up, we just build it in "reverse" -- turn off swap (dumping its info back to the system, where CompCache / linux-swap would pick up the load) and re-enabling the GPU1.
Once we get /those/ built, we can then create a daemon script, include an invocation of it into the latest userinit.sh, and have that script basically monitor GPU0's load levels. If they go above a certain threshold (scaling, just like with the CPU), it would invoke the second script above, patching the kernel and re-enabling GPU1. This would then monitor the GPU, and once //its// memory load falls back down to where GPU0 can handle it all, turns GPU1 off and frees that RAM back up for individual use.
I can already foresee a complication where turning GPU1 off "on the fly" would cause a reload of all graphics, but... considering how rare it is to even pull a load above what GPU0 can handle in ordinary use, this isn't much of an issue. It'd take some tweaking and beta-testing (which I'm pretty much set to be a tester of at this point, apparently ) but that would be about it.
Now, //ideally// we could create an alternative solution which wouldn't rely on constantly patching the kernel. As I said; that can get... well, "messy".
I just don't know enough to know what that alternative path would be. Virtual shared memory space seems like it would be a good idea. An alternative solution could be something along the lines of enabling GPU1 at all times and simply having it call to a swapfile which is located in /tmp (I.e.; a non-compressing swap-file which is actually located in RAM). Said swapfile could be resized freely according to need and would thus be no issue at all.
Frankly, I can already see one reason why that would be a superior solution to all the ksplice shenanigans above: said swapfile could be made to be /larger/ than 8 MB, and thus actually provide for an increased performance due to higher available memory for the graphics side of things. We'd want some way to make sure that the swapfile itself wouldn't get pages dumped onto an actual swap partition. This seems like a much more elegant solution than the ksplice routine. I simply have no idea if it's even feasible. The swapfile would have to be mounted into /tmp (or it's Android equivalent) //at the time the GPU1 was enabled//. Is that even /possible/ at boot? Could we simply enable the device very late in the boot stage?
Quickly! Someone with /actual programming expertise/, rescue me from my aimless mental tinkering!

Since it seems there's little activity in this thread, I thought I'd share:
There is a debian armel package for ksplice.
( http://packages.debian.org/squeeze/ksplice )
This is ARM-compiled. We would simply need to cobble together a script which invokes ksplice properly. I myself know nothing of how this would be done.
Usage example of ksplice here:
http://www.ksplice.com/example-update

Looks pretty simple. You just need a working kernel build environment. Anyone got one of those laying around? I have wanted to do that for awhile but am too lazy to set up a linux VM, lol.

Hell yeah!!
Im not much good for this project other then willingness to try betas and give feedback, but I'm excited to see where this goes.. It would be great to reclaim the ram being used from the system when you aren't using anything that needs it and use it for what you are using. Has anyone contacted one of the devs that have been modifying the kernels and ask them if they could try building it with the ksplice functionality?? I would think one of them would be happy to build it so you can start playing with it. Wish I could help more on it, but this is a little above my head, but I will be following this thread and learning what I can as i go.. Good Luck!!

Confused about I/O slow down (looking for technical explanation) ?

So everyone (including me) has noticed that the transformer slows down when doing i/o. I originally thought this was a hardware issue (slow memory? slow bus?) but from various threads it sounds like third party os fix the issue. So I have two questions:
Can someone explain what the asus kernel does wrong (or how third party kernels) fix the issue ?
Why asus cannot copy the fixes from third party kernels into their kernel (I presume this is a kernel issue and not support software around the kernel but maybe that presumption is incorrect; maybe it is a driver issue or maybe there really is a hardware issue?)

jake21 said:
So everyone (including me) has noticed that the transformer slows down when doing i/o. I originally thought this was a hardware issue (slow memory? slow bus?)
Click to expand...
Click to collapse
Random write of small blocks to the internal eMMC is slow. Flash memory has huge erase blocks (typically a couple of megabytes) and large write blocks. Writing 4KB is a relatively slow process.
jake21 said:
but from various threads it sounds like third party os fix the issue.
Click to expand...
Click to collapse
They can't fix it, they can only work around the issue by tweaking the kernel's caching parameters. Or, in the extreme case, disabling the fsync system call. Usually, an application calls fsync to ensure data has been written to the disk, so that even in case of a following crash or unexpected power loss the data on the medium is consistent. And normally fsync waits until the write command has completed. If you disable fsync, the app no longer has to wait, therefore no more lag. The data still resides in the RAM and is eventually written to the card by the background cache flush thread.
Downside of disabling fsync: If the tablet crashes in the wrong moment, you may in the worst case lose all your data, run into a bootloop, etc.

If this is true, how come with original release ICS 4.0.3 everything runs smooth and fast. I installed latest Jelly Bean 4.1 and slow real bad, i downgraded back to ICS 4.03 and it is fast again. Is it a driver issue. It can't be hardware.
Thanks
_that said:
Random write of small blocks to the internal eMMC is slow. Flash memory has huge erase blocks (typically a couple of megabytes) and large write blocks. Writing 4KB is a relatively slow process.
They can't fix it, they can only work around the issue by tweaking the kernel's caching parameters. Or, in the extreme case, disabling the fsync system call. Usually, an application calls fsync to ensure data has been written to the disk, so that even in case of a following crash or unexpected power loss the data on the medium is consistent. And normally fsync waits until the write command has completed. If you disable fsync, the app no longer has to wait, therefore no more lag. The data still resides in the RAM and is eventually written to the card by the background cache flush thread.
Downside of disabling fsync: If the tablet crashes in the wrong moment, you may in the worst case lose all your data, run into a bootloop, etc.
Click to expand...
Click to collapse

Ok I can understand the issue with fsync and small writes but then all tablets would have this issue (unless the infinity used particularly poor chioce of hardware). Also does this indicate if writes were disabled in (for example) browsers then they would be silky smooth ?
-
It would be nice if the tablet could mark certain directories as critical and flush those faster than other directories (perhaps abusing the meaning of the sticky bit on the directory). Certain non critical data can avoid having immediate flush (though if andriod apps are calling fsync explicitly there might be some stickyness in changing the behavior of the api). Anyways is my understanding correct that you are indicating that Asus used a particularly poor choice of eMMC or tweaked the kernel to flush more frequently ?
_that said:
Random write of small blocks to the internal eMMC is slow. Flash memory has huge erase blocks (typically a couple of megabytes) and large write blocks. Writing 4KB is a relatively slow process.
They can't fix it, they can only work around the issue by tweaking the kernel's caching parameters. Or, in the extreme case, disabling the fsync system call. Usually, an application calls fsync to ensure data has been written to the disk, so that even in case of a following crash or unexpected power loss the data on the medium is consistent. And normally fsync waits until the write command has completed. If you disable fsync, the app no longer has to wait, therefore no more lag. The data still resides in the RAM and is eventually written to the card by the background cache flush thread.
Downside of disabling fsync: If the tablet crashes in the wrong moment, you may in the worst case lose all your data, run into a bootloop, etc.
Click to expand...
Click to collapse

gordo2000 said:
If this is true, how come with original release ICS 4.0.3 everything runs smooth and fast. I installed latest Jelly Bean 4.1 and slow real bad, i downgraded back to ICS 4.03 and it is fast again. Is it a driver issue. It can't be hardware.
Click to expand...
Click to collapse
I didn't see a big performance difference between ICS and JB, even though JB should be even faster after all the "Project Butter" work. What is slow for you on JB?
---------- Post added at 07:35 PM ---------- Previous post was at 07:30 PM ----------
jake21 said:
Ok I can understand the issue with fsync and small writes but then all tablets would have this issue (unless the infinity used particularly poor chioce of hardware). Also does this indicate if writes were disabled in (for example) browsers then they would be silky smooth ?
Click to expand...
Click to collapse
Many people said the TF700's eMMC is slower than good microSD cards, and that can be reproduced with benchmarks.
To check how the browser performs if it doesn't have to write to the eMMC, install Browser2RAM, which moves the browser cache to a ramdisk. In my experience, there is still lag on some pages - so not all slowdowns seem to be I/O-related. It would be interesting to find out the real cause of this.
There may be another I/O situation except random writes: large writes which block small reads from another process. HPI should help here, but I think the 3.1 kernel doesn't support it yet.

_that said:
I didn't see a big performance difference between ICS and JB, even though JB should be even faster after all the "Project Butter" work. What is slow for you on JB?
---------- Post added at 07:35 PM ---------- Previous post was at 07:30 PM ----------
Many people said the TF700's eMMC is slower than good microSD cards, and that can be reproduced with benchmarks.
To check how the browser performs if it doesn't have to write to the eMMC, install Browser2RAM, which moves the browser cache to a ramdisk. In my experience, there is still lag on some pages - so not all slowdowns seem to be I/O-related. It would be interesting to find out the real cause of this.
There may be another I/O situation except random writes: large writes which block small reads from another process. HPI should help here, but I think the 3.1 kernel doesn't support it yet.
Click to expand...
Click to collapse
Agreed. However, even with Browser2Ram, I'm betting that there is still some i/o with the emmc that cannot be hijacked by broser2ram, and therein lies the problem. If nothing else, using RAM like that may force the tablet (b/c of screwy coding) to start paging data to...yup, you guessed it, emmc ... a lot sooner than it actually needs to.
Has anyone tried B2R along with dev settings to kill apps ASAP that are not in use? Perhaps this could lengthen the time before paging starts to occur?
Sent from my ASUS Transformer Infinity TF700 running Android JB (rooted) via Tapatalk

Overall, everything runs smooth on ICS 4.0.3. Browser, opening app, games, there is no hick ups when watching movies, that happen alot on JB 4.1.1. The whole OS is smoth redrawing. On JB, there is always a wait few seconds to open application folders or closing it. I did reformat to default but no help.
_that said:
I didn't see a big performance difference between ICS and JB, even though JB should be even faster after all the "Project Butter" work. What is slow for you on JB?
---------- Post added at 07:35 PM ---------- Previous post was at 07:30 PM ----------
Many people said the TF700's eMMC is slower than good microSD cards, and that can be reproduced with benchmarks.
To check how the browser performs if it doesn't have to write to the eMMC, install Browser2RAM, which moves the browser cache to a ramdisk. In my experience, there is still lag on some pages - so not all slowdowns seem to be I/O-related. It would be interesting to find out the real cause of this.
There may be another I/O situation except random writes: large writes which block small reads from another process. HPI should help here, but I think the 3.1 kernel doesn't support it yet.
Click to expand...
Click to collapse

johnlgalt said:
Agreed. However, even with Browser2Ram, I'm betting that there is still some i/o with the emmc that cannot be hijacked by broser2ram, and therein lies the problem.
Click to expand...
Click to collapse
Not all I/O automatically leads to problems. The latest version of Browser2RAM only redirects the browser cache, it does not affect browser settings, bookmarks, etc. - which is usually a good thing.
johnlgalt said:
If nothing else, using RAM like that may force the tablet (b/c of screwy coding) to start paging data to...yup, you guessed it, emmc ... a lot sooner than it actually needs to.
Click to expand...
Click to collapse
No. Paging does not occur *to* the eMMC (no swap space is configured on the TF700), but only *from* the eMMC, to fetch pages of executable files. While it is true that the ramdisk for the cache uses some memory, it would only make a difference if you have lots of background apps competing for RAM.
A first step to see how much I/O happens is to watch the output of "iostat".

Thanks for the heads up. So, why does it still cause pauses and the like then?
Sent from my ASUS Transformer Infinity TF700 running Android JB (rooted) via Tapatalk

johnlgalt said:
So, why does it still cause pauses and the like then?
Click to expand...
Click to collapse
Good question. It's time to find out.
Arm yourself with multiple adb shells and watch the output of iostat, top, free, or whatever else you can think of that displays interesting metrics. Then do something that causes lag and see if you notice a specific pattern.
(I am currently away from my main PC, and the SSD in my laptop decided yesterday it no longer wants to read ntkrnlpa.exe - so no adb for me right now)

I'll need a bit more specifics - I know adb well enough and can shell, but these other ... executables you're mentioning are new to me.
I'm on vacation in Hawaii, and have a Windows 7 based laptop that I can use, so I can do this no problem - but not today. About to go see some sights before going on a Lava Boat tour at 4 PM local, which means I'll be bushed when I get back - plus I'm fighting a nasty ear infection that aches something awful.
AFAIK, though, I have no real plans for tomorrow or Friday, so I can take some time and investigate.
Also, FWIW: I'm rooted but have not (yet) unlocked my bootloader - mainly b/c I purchased the 64 GB version of the tablet and it is a C50, so I'm hoping something 'breaks' enough for me to get a replacement (c70? C90 even? )- and hoping even more that it is running something under ICS .30 so I can nvflash a backup and not have to worry about goofing things up when I *do* unlock the bootloader.
Sent from my ASUS Transformer Infinity TF700 running Android JB (rooted) via Tapatalk

iostat and top are standard unix utilities. They would only be useful if run on the phone so I must presume andriod has versions. A bit of 'googling' and htere appears to be a developer's kit that includes stuff like iostat. If hte switches are the norm then something like "iostat -x 2" will produce nice output of performance of each 'disk'. top is a tool that shows cpu usage.
-
I've not done any developing for andriod (maybe I should bite hte bullet?) so have never tried to use adb or similar but i've done a bit of system development on linux (though I very rarely muck with the kernel; i prefer to work one layer above the kernel).

johnlgalt said:
I'll need a bit more specifics - I know adb well enough and can shell, but these other ... executables you're mentioning are new to me.
Click to expand...
Click to collapse
If you haven't done it yet, install BusyBox on your TF700. Then just open one or more command windows on your PC, run adb shell in each, and run "iostat 1" in one window, "top" in another, and maybe also adb logcat in yet another window.
That gives you up-to-date statistics about I/O and processes which currently use CPU time. Then try to use your tablet normally, and when it lags, watch the output on your PC if you see a big number of writes or a process eating CPU.
But don't forget to enjoy your vacation.

_that said:
If you haven't done it yet, install BusyBox on your TF700. Then just open one or more command windows on your PC, run adb shell in each, and run "iostat 1" in one window, "top" in another, and maybe also adb logcat in yet another window.
That gives you up-to-date statistics about I/O and processes which currently use CPU time. Then try to use your tablet normally, and when it lags, watch the output on your PC if you see a big number of writes or a process eating CPU.
But don't forget to enjoy your vacation.
Click to expand...
Click to collapse
Awesome. Busybox already installed here, so this should be easy enough.
And I never forget to enjoy ... anything. lol
Sent from my ASUS Transformer Infinity TF700 running Android JB (rooted) via Tapatalk

_that said:
Good question. It's time to find out.
Arm yourself with multiple adb shells and watch the output of iostat, top, free, or whatever else you can think of that displays interesting metrics. Then do something that causes lag and see if you notice a specific pattern.
(I am currently away from my main PC, and the SSD in my laptop decided yesterday it no longer wants to read ntkrnlpa.exe - so no adb for me right now)
Click to expand...
Click to collapse
hey fellas, have you seen chainfire's app : PerfMon
http://www.xda-developers.com/android/perfmon-floats-your-devices-performance-on-screen/
real time stats including io reads/writes to both mmcblk0/mmcblk1... :good:
ps: i have always loved "android status" aswell ... oldy but a goody
https://play.google.com/store/apps/details?id=com.AndroidStatus&hl=en

[APP][4.1+] Tuxoid v0.1.1 - A full Linux desktop environment on your Android

Code:
*** Disclaimer
As usual, I am not responsible for thermonuclear war or other apocalyptic futures.
TL;DR: If you break your device with this project, it's not my fault (although I will of course be willing to help you in your troubles!!).
Introduction
Have you ever wanted to use your Android device as a fully-fledged desktop Linux box? Now you can, with Tuxoid!
A year or two ago, I discovered the Linux-on-Android project. (http://forum.xda-developers.com/showthread.php?t=1585009) I was fascinated by the possibility of running Linux on top of Android and using a desktop environment. However, I was slightly disappointed by the lack of some features, as well as a lack of development over the time of the project's existence. Thus I embarked upon an epic quest (well maybe not quite that dramatic) to build my own version from the ground up, basing it upon the ideas explored by Linux-on-Android.
After some on and off hacking on my OnePlus One, I now have some very early working code.
Below is a video demonstration of the current state of Tuxoid, demonstrating a few of its features.
Here is a list of some of these features:
- The big one: fast and high quality display system
- Native keyboard and mouse support (automatically disables Android's built-in system)
- Audio support. Audio from the desktop environment is piped into Android's audio system, so you can use your device's speakers, headphones or bluetooth etc
UPDATE: First builds!
I've built the first publicly available versions of the project. Before you get excited, I should warn you that they are in an unstable state. You can grab the APKs in the Downloads section of this project, as well as on the Releases page of the project on GitHub. These builds are not for n00bs! If you're still determined to give them a try, here are the instructions:
Requirements
An Android device running 4.1 or newer (it will be availabe for older versions in the future, but for know only Lollipop is supported) Older versions now supported!
Root with SuperSU for managing permissions. I will not explain this here, as there are already hundreds of guides available. If you don't know how to root your device, this project (at least in these early stages) is not for you.
A reasonable amount of internal storage (at least 1-2 GB free)
A stable internet connection. Make sure you're using WiFi for best results.
A mouse and keyboard to control the desktop environment. You should be able to use wired ones (via an OTG adapter) or wireless via bluetooth. As long as Android recognizes it, you should be good to go.
This is not a requirement, but you won't be able to do much without it (for now): knowledge of the Linux command line and preferably how to use the pacman package manager for Arch Linux, as that is the distro upon which Tuxoid is based (more distros will be available down the line!)
Setup and installation
Once your device is rooted, open up the SuperSU app and go to the Settings tab. Find an option labelled "Mount namespace separation" and ensure it is UNchecked. (Make sure to reboot before continuing if you had to do this)
Download and install the APK for Tuxoid. Again, if you don't know how to do this, you shouldn't be attempting to install an early build Tuxoid...
Open the Tuxoid app and tap the gear in the actionbar to go to the settings menu. There are a few options you can tweak here, but the main one you should be interested in right now is 'Disk image size'. Select a size that suits the amount of space you have free on your internal storage. I would recommend 4GB (the default) to allow room for installation of your own packages later. If you have less space free on your internal storage than you select here, the setup process will fail.
Go back to the main screen of the app and ensure you have a stable internet connection. Now connect your keyboard and mouse (you must connect them before booting Tuxoid up every time). Then, press 'Start' to begin the setup process. This will probably take a while, so just leave your device down for a while and check up on it every once and a while.
Eventually, if all went well, you should see a screen with a grey background and an xterm window. Feel free to run some commands. To start, I'd recommend installing the Chromium browser (open source version Google Chrome). To do this, run 'sudo pacman -S chromium' (without quotes). The password is the same as your username. Once installed, run 'chromium' to start the browser. For those who are interested, the window manager installed by default is Openbox. If you want to install some more packages, the list of available ones in Arch Linux for ARM devices is available here: http://archlinuxarm.org/packages If you weren't able to get up and running (i.e. no screen with a cursor and no xterm), you can try some of the steps described in Troubleshooting & Tips.
When you're finished playing around, follow the steps below to ensure clean shutdown of Tuxoid. If you want to boot up again, you can just open up the app at any time and tap on the start button. Tuxoid will use your existing system image.
Shutdown procedure
When you're done, simply press the back key on your device to close the GUI and press the stop button to shutdown Tuxoid. After a few seconds, the log will show "SHUTDOWN!", meaning the shutdown procedure was completed successfully.
Troubleshooting & Tips
If Tuxoid failed to boot during the initial setup, you can retry by first removing files in the 'droidtop' folder on your internal storage (usually 'system.img' and 'arch.tar.gz') and then pressing 'Start' again in the app.
If your mouse and keyboard aren't working in Tuxoid, ensure that they were connected to your device BEFORE you pressed the start button and make sure they were detected by Android. If they weren't detected by Android itself the they definitely won't work with Tuxoid.
If you something goes wrong at any stage while using Tuxoid, please press the 'Send log' button in the app to send me a copy of the log shown on the main screen.
If the app crashes, a dialog will pop up asking you to send a report. Please do this as it makes it much easier for me to fix bugs!
If you have any feedback and suggestions, feel free to post them here in the forums! You can also submit issues on the GitHub project (although this is really more orientated towards developers)
If you are a developer and want to get involved in the project, you can post here in the forums and/or submit pull requests on GitHub.
Source code
I've published all of the source code for the app to GitHub (http://github.com/jackos2500/tuxoid) Feel free to fork and make changes, and, even better, submit pull requests to move the project forward!
Finally, I would like to say thanks to the guys over at Linux-on-Android for their great work, without which the idea for this project would never have existed!
XDA:DevDB Information
Tuxoid, App for all devices (see above for details)
Contributors
jackos2500
Version Information
Status: Testing
Created 2015-06-13
Last Updated 2015-06-14

Reserved
Changelog
v0.1.1
Adds compatibility for Android versions as far back as 4.1 (API level 16).
Bug fixes:
screen resolution could be detected incorrectly
'setenforce' could fail to execute due to its lack of existence on older platform versions
v0.1
Initial public release.

Reserved

Its really interesting to hear about linux on android

shahzu3 said:
Its really interesting to hear about linux on android
Click to expand...
Click to collapse
Yup! I think I'll put out a build later with some instructions so that people can try it for themselves.

Mm

shahzu3 said:
Mm
Click to expand...
Click to collapse
Well I've pushed out the first public build. Give it a try and tell me what you think!

Hi. Your project sounds great ??
Is really necessary to have so internal space or could it be possible one day to have the whole stuff on the external sd.??

That is just amazing, thanks for your work, a tablet running this would be really nice... I'm going to try it out.

Hello! This looks very promising but I can't find any download link. Is it missing?
Edit: My phone just didn't load the page fully, the download section is on the top.

DirkStorck said:
Hi. Your project sounds great
Is really necessary to have so internal space or could it be possible one day to have the whole stuff on the external sd.?
Click to expand...
Click to collapse
Good point! I'll see if I can do something about that sometime soon. Unfortunately if the chroot environment is not stored in a disk image the filesystem it is stored on must support Unix permissions, so no FAT32 formatted sdcards without disk images. FAT32 also limits file size to 4GB, so that would be the max size for a disk image there. I'll probably add a way of optionally adding an extra partition to your SD to get around this at some stage.
WideBRs said:
That is just amazing, thanks for your work, a tablet running this would be really nice... I'm going to try it out.
Click to expand...
Click to collapse
Cool, tell me what you think when you give it a go!

jackos2500 said:
Good point! I'll see if I can do something about that sometime soon. Unfortunately if the chroot environment is not stored in a disk image the filesystem it is stored on must support Unix permissions, so no FAT32 formatted sdcards without disk images. FAT32 also limits file size to 4GB, so that would be the max size for a disk image there. I'll probably add a way of optionally adding an extra partition to your SD to get around this at some stage.
Cool, tell me what you think when you give it a go!
Click to expand...
Click to collapse
I have been using Linux Deploy on my tablet and it worked quite well. They have the environment as an image on the internal or external sd card. I think 4GB is a enough for a Linux system. For those who want more would have to format the external as ext4.

DirkStorck said:
I have been using Linux Deploy on my tablet and it worked quite well. They have the environment as an image on the internal or external sd card. I think 4GB is a enough for a Linux system. For those who want more would have to format the external as ext4.
Click to expand...
Click to collapse
I've pushed a new release with support for devices running Android 4.1 and above. I think I'm going to start working on improving the user-friendliness of the project tomorrow. This will probably include improving the UI and setup process, as well as better error handling. Once that code is in place I won't have to refactor any new features I add to the new UI, so it makes sense to do it first. It will take a while, and whenever it's done, I'll probably start working on some of those new features.

Hi,
Could you tell me if:
a) my phones built-in keyboard will work?
b) if there's an option to use the touch screen to control the mouse?
Thanks

moodroid said:
Hi,
Could you tell me if:
a) my phones built-in keyboard will work?
b) if there's an option to use the touch screen to control the mouse?
Thanks
Click to expand...
Click to collapse
Right now, no. Both of these are definitely a high priority though.

jackos2500 said:
Right now, no. Both of these are definitely a high priority though.
Click to expand...
Click to collapse
Excellent, thanks. I'll keep an eye on this...

Can i decompile and compile apk with this Linux for Android? I havent pc

this is really amazing, as it is very useful for tablets, at least for my tab s 10.5 inch, keep it up!
Currently using Xserver Xsdl, with the pain of slow refresh rate and input interrupt.
Thank you for this effort!

Hello,
So far, I had these issues before it worked, otherwise its awesome, the only way to render x server on android smoothly and no input latency without using direct (kernel) framebuffer:
1)Latest arch armv7hf use xorg-server and xf86-input-evdev compiled for/by version 1.18.x , had to recompile 1.17 from source since your module (the source code isnt released in your github :/, guess you forgot) is compiled for ABI module 20, currently 23.
2) High cpu usage, probably implement refresh rate limitation on the app side for displaying the shared memory framebuffer, though having the source code for the module would really help, guess its a modified dummy.so (?) to copy framebuffer to shm? Anyways ill try to workaround the issue using a compositor.
Thats all! working perfectly... If anyone need the compiled 1.17 versions of xorg-server and evdev, just use ABS or pm me.
Thanks!
EDIT: I think it would be better to turn this into X server framebuffer viewer app, and combine it with Linux deploy (+ 'custom scripts' option is already implemented in Linux Deploy, useful for evdev and xorg config) for more linux flavours

apparently, it seems that you have copied droidtop project without mentioning it (license: MIT) : https://code.google.com/p/droidtop/
which has been archived now with no public access