linux chroot and hardware or kernel problems - Asus Transformer TF700

I have only been an Android user for a couple of months, but I've been a Linux user and software developer for almost 20 years so this might be a strange question; Is it normal to put up with so many random application crashes on this platform? I am not used to such instability with Linux except in the rare case when I have introduced a bug in some kernel driver I was developing.
In addition to force close messages and mainstream applications just disappearing with no message, this family of tablets or perhaps it is the Tegra, seem to have more than the normal problems running Linux distros via chroot.
I started with a TF300 and returned it after 30 days in part because I suspected flaky hardware was to blame for the application crashes. Now I see about the same behavior with the TF700.
It has already been reported that Unity and Firefox don't work in Ubuntu 12.04 and no one seems to have any idea why. In the process of investigating this, while building a debug version of Firefox inside a Ubuntu 12.04 chroot, I encountered several non-reproducible bus errors with gcc which it reported as possible hardware or kernel errors. Restarting the make succeeded in building that object, only to fail with another random signal 7 later in another part of the build. So I switched to a Debian image and found even stranger results with processes hanging randomly (ps shows they are sleeping in one of two places in the kernel).
I have also noticed that Ubuntu process such as the vnc server just dissapear sometimes. I would suspect Android of killing it thinking it was "managing" resource use, but given the other issues, I suspect it just encountered a random bus error or segfault. If this wasn't my second sample and if others hadn't reported problems with linux installs on the Tegra, I would assume I had a bad processor.
These are not the normal symptoms of application errors. So my question is: Do we have a nasty kernel bug which is causing these random crashes and hangs? Is it a hardware problem? Or are the Android application crashes typical and the chroot Linux environment behaviors caused by some incompatability between the generic ARM builds and our Tegra architecture or perhaps some processor errata?
I have searched the forums for clues but even though the Tegra and the Transformer families have been around for a little while, I can't find any answers.
Thanks,
--Brad

You definitely get much more of this than on a stable Linux distro (although I find X-Window generally unstable as compared to other parts of the system). I have used many Android devices and it seems both build/version- and device-specific, some of them had a lot of FCs, some of them almost none. Then there come many apps with a really bad code. It should be better from the JB onwards and current code control under Android.
I think so far the Infinity is on the worse side of this, but I'm not sure what to blame, some say Tegra, some say firmware. I really hate GMail app crashing (newest app / newest firmware), I feel it's just ridiculous.
I'm not sure how deep debugging is going to get you under Android (wouldn't even want to guess ;>). If you really want to know, I guess you'll have to find out by yourself. For now, I don't think the Infinity is particularly stable, but, having sad that, I believe it's getting better, as I don't experience more random reboots under .26 firmware now, my uptime is only a few days now, but just because my testing of some custom tweaks in /system/

I havent really experienced any of those issues running a chrooted debian. Perhaps your setup is a little too heavy ram wise? Try a lighter window manager like LXDE or fluxbox. Alternatively, perhaps you're running into the IO issues that seem to plague the Tegra 3 transformers.

Jotokun said:
I havent really experienced any of those issues running a chrooted debian. Perhaps your setup is a little too heavy ram wise? Try a lighter window manager like LXDE or fluxbox. Alternatively, perhaps you're running into the IO issues that seem to plague the Tegra 3 transformers.
Click to expand...
Click to collapse
I am already running LXDE which I installed on the large Ubuntu image from the complete linux installer project and I intentionally avoided doing anything else while I was building. The build slowed things down to the point that the tablet was not very fun to use anyway. Have you tried building any large programs? Other than an occasional mysterious exit of the VNC server, that's where I started seeing most of the nasty random bus errors. Unless someone knows of a reason low memory could cause bus errors, I don't think the slow I/O would be related. Though I suppose slow I/O could cause low memory as ram gets filled up with dirty buffers. But the kernel will start to put processes to sleep when it can't find a free page to satisfy an allocation and the writes should catch up so I don't expect allocations would actually fail due to slow I/O. I do wish I could enable swap without replacing the boot loader. And I wish I could replace the boot loader without voiding my warranty.
BTW, another irritation on this tablet is the huge number of messages the kernel is spewing out constantly about starting and stopping processors and other power management and routine events. There could be an interesting message in the kernel logs related to the bus errors but I would never see it in dmesg because of the noise which quickly flushes anything else out of the circular buffer. That fact that ASUS has not taken these debug messages back out of the kernel makes it feel like it is still in development.
Has anyone else tried to build anything large under Ubuntu chroot? BTW, Firefox is so large, I allocated another 4G file system just to put the debian source in so I could run dpkg-buildpackage without filling up the file system. It looks like I could have grown my image beyond 4G. I didn't realize until just now that sdcard wasn't a fat32 file system. Or I suppose I could reformat my microSD as ext3 and move my chroot work there.

I had whole servers running under FreeBSD's jails years ago, I think it is much better than chroot.

d14b0ll0s said:
I had whole servers running under FreeBSD's jails years ago, I think it is much better than chroot.
Click to expand...
Click to collapse
That may be, at least in the case of multi-user systems where resource contention is a real problem. But in general there is no technical reason I can think of that chroot sandboxes should be unstable. The API which the kernel presents to the user level is very stable, even across major differences in kernel versions. I use linux chroot sandboxes all the time for my job and they are very stable and useful. I find that a library version mismatch (which chroot avoids) is far more likely to cause trouble than an older or newer kernel. There is something else going on here. Other ICS based tablets are able to run these chroot images with no trouble.
--Brad

An update in case anyone is interested:
I haven't learned anything about the tegra specific issues with chroot, but I do think I understand where at least half of the random android application crashes I experience are coming from.
I have been running dmesg after any android application crashes and I noticed that in over half of the cases, the kernel oops at the end of this post was reported. The oops report indicates that the nvidia driver was attempting to allocate a 64K block of physical memory. Examining nvhost_job.c in the kernel source makes it clear that the nvidia driver is poorly written with respect to dynamic allocation of large blocks of physical memory in order to satisfy dynamic application gl operations. A bit of research turned up that this bug has been reported previously way back in March on transformerforums.com. I'm not sure why it seems to have never been discussed here and I'm not sure why it has taken ASUS and Nvidia so long to fix it.
I believe this rather nasty and stupid bug explains many of the android application crashes I and others have seen, including google play, google maps (this one tends to persist until you reboot), various vnc viewers, etc. Note that this bug is more likely to be triggered the longer we go without a reboot. And stressing the system with respect to memory use, for example, by performing lots of output to the transformer's slow storage, or running lots of programs inside a chroot triggers the bug sooner.
The good news is I checked the recently available TF300 Jelly Bean kernel source on the ASUS site and was happy to see that the driver has been reworked to use virtual memory for this purpose which should solve this class of application crashes. I assume the same fix will be in JB for the TF700 when it is released.
I wish the root cause of the other Tegra specific problems I see inside the Ubuntu chroot including the compiler bus errors were as easy to debug.
--Brad
<4>[162943.445514] ogle.android.gm: page allocation failure. order:4, mode:0xd0
<4>[162943.445656] [<c004f4d0>] (unwind_backtrace+0x0/0x12c) from [<c00e86e4>] (__alloc_pages_nodemask+0x5f4/0x658)
<4>[162943.445778] [<c00e86e4>] (__alloc_pages_nodemask+0x5f4/0x658) from [<c010cddc>] (cache_alloc_refill+0x2c0/0x52c)
<4>[162943.445915] [<c010cddc>] (cache_alloc_refill+0x2c0/0x52c) from [<c010d0e0>] (__kmalloc+0x98/0xe0)
<4>[162943.446031] [<c010d0e0>] (__kmalloc+0x98/0xe0) from [<c024a874>] (nvhost_job_realloc+0x4c/0x134)
<4>[162943.446221] [<c024a874>] (nvhost_job_realloc+0x4c/0x134) from [<c024b494>] (set_submit+0x58/0x90)
<4>[162943.446336] [<c024b494>] (set_submit+0x58/0x90) from [<c024b714>] (nvhost_channelctl+0x248/0x44c)
<4>[162943.446406] [<c024b714>] (nvhost_channelctl+0x248/0x44c) from [<c011efb8>] (do_vfs_ioctl+0x500/0x57c)
<4>[162943.446544] [<c011efb8>] (do_vfs_ioctl+0x500/0x57c) from [<c011f068>] (sys_ioctl+0x34/0x54)
<4>[162943.446665] [<c011f068>] (sys_ioctl+0x34/0x54) from [<c00495c0>] (ret_fast_syscall+0x0/0x30)
<4>[162943.446801] Mem-info:
<4>[162943.446858] Normal per-cpu:
<4>[162943.446915] CPU 0: hi: 186, btch: 31 usd: 1
<4>[162943.447036] HighMem per-cpu:
<4>[162943.447094] CPU 0: hi: 186, btch: 31 usd: 0
<4>[162943.447180] active_anon:91412 inactive_anon:172 isolated_anon:0
<4>[162943.447184] active_file:31694 inactive_file:44876 isolated_file:32
<4>[162943.447187] unevictable:227 dirty:8 writeback:0 unstable:0
<4>[162943.447190] free:16204 slab_reclaimable:2875 slab_unreclaimable:3047
<4>[162943.447193] mapped:18781 shmem:208 pagetables:2670 bounce:0
<4>[162943.447635] Normal free:62760kB min:3224kB low:4028kB high:4836kB active_anon:172696kB inactive_anon:112kB active_file:82056kB inactive_file:109704kB unevictable:0kB isolated(anon):0kB isolated(file):128kB present:650240kB mlocked:0kB dirty:32kB writeback:0kB mapped:37376kB shmem:180kB slab_reclaimable:11500kB slab_unreclaimable:12188kB kernel_stack:7312kB pagetables:10680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
<4>[162943.448281] lowmem_reserve[]: 0 2880 2880
<4>[162943.448608] HighMem free:2056kB min:360kB low:816kB high:1272kB active_anon:192952kB inactive_anon:576kB active_file:44720kB inactive_file:69800kB unevictable:908kB isolated(anon):0kB isolated(file):0kB present:368656kB mlocked:0kB dirty:0kB writeback:0kB mapped:37748kB shmem:652kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
<4>[162943.450980] lowmem_reserve[]: 0 0 0
<4>[162943.451323] Normal: 3378*4kB 4302*8kB 901*16kB 40*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 63752kB
<4>[162943.452059] HighMem: 256*4kB 35*8kB 27*16kB 6*32kB 0*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2056kB
<4>[162943.452864] 76613 total pagecache pages
<4>[162943.452924] 0 pages in swap cache
<4>[162943.452983] Swap cache stats: add 0, delete 0, find 0/0
<4>[162943.453091] Free swap = 0kB
<4>[162943.453149] Total swap = 0kB
<4>[162943.498866] 261632 pages of RAM
<4>[162943.498982] 17680 free pages
<4>[162943.499041] 11846 reserved pages
<4>[162943.499118] 5922 slab pages
<4>[162943.500755] 258372 pages shared
<4>[162943.500869] 0 pages swap cached

Related

Kernel Optimization

The lovely thing about having a Linux Kernel is that we can do a lot to optimize it.
I figured it would be a great idea to set up a thread where we can bring all these optimizations into one place.
Mind you, I'm no expert by any means...
For my (possibly useless) contribution:
When we compile the kernel to the G1, do we compile it with the general Arm eabi? If so, is that made for ALL ARM cpu's? We have an ARM6 in our G1's. If we set it up to optimize to ARM6, can we set it up to use the CPSID/CPSIE instruction instead of the SWP instruction? Are the drivers open source? Can we recompile them to do the same also? To learn more about why, read this:
http://forum.xda-developers.com/showthread.php?t=516223
That's where I got the idea from.
SWP is a deprecated opcode in armv6 that decreases performance due to the way it locks the bus.
Click to expand...
Click to collapse
Anyhow, whatever you guys post here, I will add to this post. Also, to the people to compile the kernels, can you answer my questions?
answers to some of your questions.
hi i usually don't post here since i am not a developer but in this case i think i can answer some of your questions (since i sometime make custom kernels for my laptop). kernel optimazation is used to removed excessive drivers and services from the kernel that you don't need that slows down your computer or in this case phone. i believe htc already does this when they configured android for the g1. most cell phone manufactures already do this to ensure that there products processor use energry efficiently and and run as smoothly as possible.
When we compile the kernel to the G1, do we compile it with the general Arm eabi? If so, is that made for ALL ARM cpu's? We have an ARM6 in our G1's
Click to expand...
Click to collapse
yes, arm eabi is made for all general arm cpu's i believe you can complie it for arm 6 specifically though. how do you know that htc hasn't already optimize the kernel for that specific processor? also if they didn't you wouldn't really see any noticible speed improvements by compliling the kernel for this specific processor.
Are the drivers open source?
Click to expand...
Click to collapse
no, futhermore they don't offer the drivers for the g1 except execpt for the usb driver so your computer can recognise the g1 as mass storage device.
If we set it up to optimize to ARM6, can we set it up to use the CPSID/CPSIE instruction instead of the SWP instruction?
Click to expand...
Click to collapse
i don't know sorry. thats all i know i'm really not a kernel, coding expert i hope this helps in your task.
bananajama said:
no, futhermore they don't offer the drivers for the g1 except execpt for the usb driver so your computer can recognise the g1 as mass storage device.
Click to expand...
Click to collapse
I think you meant to say "ADB interface" instead of "mass storage device."
t1n0m3n said:
I think you meant to say "ADB interface" instead of "mass storage device."
Click to expand...
Click to collapse
The system type is set to "Quaalcom MSM", so I would imagine that it's optimizing for arm6l.
cyanogen said:
The system type is set to "Quaalcom MSM", so I would imagine that it's optimizing for arm6l.
Click to expand...
Click to collapse
Sounds like it. Well, other than that, are there any other obtimizations anyone else can think of?
there are few things in linux that you can play with to see if you get any performance increase. It's mainly in memory management. run ls on /proc/sys/vm. you can play with the page cache but I doubt that you'll get much perf increase from it since, since most of the things we can play with are for write back cache. If we have access to the dalvikvm we may be able to do more optimization. The one thing that I've always wanted to be able to do is to pin Home in memory, and not allow garbage collector to kill it, so when we exit out of memory intensive apps, like browser, gallery, etc, we do not have to wait for Home to reload. If we have more control over garbage collection we may be able to do more, but I don't know much about the dalvikvm at all.
taking a quick look at the vm. The only nob that may have any effect on android may be vfs_cache_pressure. theoretically setting to 200 would push fs cache out quicker and give more memory to the vm. default is set to 100. I set it to 200 and 300 but didn't notice any difference.
knaries2000 said:
there are few things in linux that you can play with to see if you get any performance increase. It's mainly in memory management. run ls on /proc/sys/vm. you can play with the page cache but I doubt that you'll get much perf increase from it since, since most of the things we can play with are for write back cache. If we have access to the dalvikvm we may be able to do more optimization. The one thing that I've always wanted to be able to do is to pin Home in memory, and not allow garbage collector to kill it, so when we exit out of memory intensive apps, like browser, gallery, etc, we do not have to wait for Home to reload. If we have more control over garbage collection we may be able to do more, but I don't know much about the dalvikvm at all.
Click to expand...
Click to collapse
dxtop, which is a home replacement, does exactly that. It keeps itself in memory so going back to it is much faster than the default home application. And in low memory situations, it tries to free up as much resources as possible while still staying in memory.

ram development "lets show google how its done"

ok its simple we all know the benefits of more ram. we all know that android slices up the ram into static addresses within the kernel. i would like to change that as we are in 2009 almost 2010 and we still rely on static ram allocation for the camera, gpu, sensors, radio. the g1 has 192mb of ram. yet we only get 97 for programs to use. i would like to place the camera, gpu, etc into dynamic addresses that we can change within the os without flashing a new kernel. a service, kernel code, module, or any other method could do this. we just have to make it happen. we could do it three ways i see.
1. a system managed ram allocation using a service to detect the percentage used and increase ram on the go.
2. a kernel script or service that only activates and allocates ram when needed. such as once a 3d program is loaded the 8mb magically comes back.
3. a virtual device ram allocation service that makes the device think it has the ram all along, but only actually gets it when it needs it. kinda like the way apps2sd works.
4.(by IConrad01) run perl scripts through ase and use ksplice to patch the gpu1 memory in kernel. one script on and one for off.
any other methods to implement this proper are very welcome and encouraged.
the idea behind this is a proper way to get back more ram for our programs. the idea is simple when 3d is not in use we can used the 8mb it would have used for something else. this applys to everything that has its own ram. camera, etc. say we did this and only got 120 free. thats almost 25% more ram. plus we could go the extra step and let the user choose how much each thing gets(gpu settings 4mb,8mb,12mb. camera on, off. etc) so when it needs it thats what it gets. great if you like/dislike 3d games.
on another note this is the kind of work that gets you noticed. the kind that gets you hired or gets you respect. google would be proud and might implement it and you and i know it. i hope we are successful and google uses it mainstream.
so lets debate a way/ways to do this
provide alpha tests of different ways to do this
and finally lets post a beta so everyone can enjoy.
atm ksplice with perl scripts is where we are going will update as this progresses.
Doesnt Jacs update to his hero rom have this, im probably mistaken
jad011 said:
Doesnt Jacs update to his hero rom have this, im probably mistaken
Click to expand...
Click to collapse
No. The current implementation of this basically cuts the graphics memory in half.
This would dynamically allocate the memory as needed. Much more efficient way of doing it.
he has a static change that takes the ram away from the gpu by a kernel edit. this is to have both more ram but still give the gpu what it needs to perform properly but only when it needs it or you chose to give it.
Are you referring to RAM or storage space? You say RAM in the title but then it sounds like you're talking about storage space before making me think it's RAM again later on. I'm confused :/ I'd really like additional free RAM, anything that helps reduce swapping to SD card is good but storage space I have plenty of.
all of this would apply to ram only. swap and compcache could use the same idea to dynamically allocate swap but that is still a cache for programs. sorry if its confusing. simple way to explain. i want all the ram i can get but still use everything the phone has properly, like games and the camera.
Gotcha and understood. I see so many people say RAM and memory when they're referring to storage space sometimes that I want to scream but extra RAM on the G1 would be more than welcome. Hopefully others with the brains to make it happen can do it
actually anyone with a basic knowledge can help. mostly there are a couple things to developing a good idea.
first the concept.
then ideas on how to do it.
next a map or guide to what does what.
coding different parts
putting them together
debug and test
even if all you do is photoshop the ui or provide ideas it helps. once some different people get aboard this should be working sooner than you think.
Of the three options listed, the kernel script (via ksplice) seems like it would be the shortest route to functionality, but frankly it also seems a bit... well, hackish -- when compared to the virtual device allocation scheme. It would take a hefty re-write of the kernel, I suspect, to accomplish; but being able to resize various devices on the fly would be a good thing. Shrinking CompCache based on real RAM usage for example would probably speed up the phone under light loads.
Just thinking in text here.
personal i think its going to start simple(script or root app) and go straight to gpu shared on or off. choose and reboot. however i do not want to stop there like winmo did. someone could create a short kernel patch that checks for a service or script and gets value or defaults to standard method. then we would have to come up with how to manage the switch(realtime vs. reboot) ideally we would remap the memory from scratch and rebuild the memory management services to dynamically allocate the ram for all devices(128+ ram) as needed. we have a start, we know where and how to turn the gpu1 shared on or off. now lets make it more than a hack and work on how to make our phones better. btw this is not a waste of time. this extends to other phones and the future way to set up our phones from default.
jokersax11 said:
personal i think its going to start simple(script or root app) and go straight to gpu shared on or off. choose and reboot. however i do not want to stop there like winmo did. someone could create a short kernel patch that checks for a service or script and gets value or defaults to standard method. then we would have to come up with how to manage the switch(realtime vs. reboot) ideally we would remap the memory from scratch and rebuild the memory management services to dynamically allocate the ram for all devices(128+ ram) as needed. we have a start, we know where and how to turn the gpu1 shared on or off. now lets make it more than a hack and work on how to make our phones better. btw this is not a waste of time. this extends to other phones and the future way to set up our phones from default.
Click to expand...
Click to collapse
The only problem with this is when the ram is divided, everything in it is cleared, thats why it is set at boot in the kernel. Think of it like repartitioning your SD. Now it would probably be possible have everything mapped to ONE big RAM table & then build something similar to a scheduler like BFS for RAM that would allocate it as necessary. This is messy though.
Just my $.02
well yes but it is possible to create shared memory through c++ http://www.go4expert.com/forums/showthread.php?t=8461
however i think the vm heaps has either a 6 or 16mb limit. anyways the service would basically do a memory patch over the kernel in realtime(like overclock) telling the gpu the memory is there all the time. meanwhile it allocates dynamic shared memory from the application memory for the gpu to read and write. meanwhile an app tied to a script would set the parameters and settings.
Okay -- I know ksplice works with perl; and I know that we can run perl scripts using the ASE .apk.
So... it seems to me that it /should/ be possible to use a patch that reverts the patched kernel back to the previous state, and a patch that switches the original state to the GPU1=0 MB state. From there, we could set up two scripts; one for GPU1-off and one for GPU0-on. Each would simply be an invocation of ksplice to input the specific patch (which would have to be stored on the sd card. [it would be safer if in the ext partition]).
We'd have to set up a secondary RAM device, rather than just adding it to the standard RAM pool... something like establishing it as a /dev/block/shm.
The point of all the above is that when you click on the GPU1-off script, it "reclaims" that RAM and gives you a device that's standard RAM. When you click on the GPU1-on script, it disables that device (forcing the reload of all programs using that RAM) and re-enables the GPU1 device.
Even better yet, the Android Scripting Environment program, much like GScript, supports desktop icons.
So -- very initially speaking, assuming we can get ksplice working on Android (which just requires a static binary, IIRC), we can get a simple pair of icons; one of which frees up the GPU1's RAM; and the other of which re-enables the GPU1.
Since the only times it's going to be "called up" are when a game is actively being played, or something similar, this should be a mostly-satisfactory route to begin with, before we can get into the whole scheduled/scaled script running as a daemon in the background.
Once we get /there/, then we can start looking at true virtual device allocations and shared memory and the like.
Although, I do have to say that I'm not entirely sure how comfortable I am constantly repatching the kernel. Definitely, we're going to want to recommend that people back up their original, unpatched, boot.img files!
IConrad01 i added your method to first post as it was a great start to this project. if you want me to rephrase just post what you want and ill change it. the only issue i might see is what geniusdog254 said it might unallocate the wrong thing and create a reboot. but some clever coding should prevent this.
Nah, that's all good.
There's an easy way to prevent it from becoming a reboot-able issue. Enable the GPU-space device as "swap". This could be done one of two ways; tell the phone that the new 8 MB RAM "device" is to be considered the /first/ swap device to be used, and just leave it "as is". The r/w times would be for true RAM. The other option assumes we have the latest CompCache (0.6+) -- as it allows for multiple instances of CompCache. The theme here should be obvious.
Either way, the result is that only swappable pagefiles would be lost when the device is turned back into GPU memory. No system-vital memory would get tossed. Seems simple enough to accomplish, perhaps. But then again, I Am Not A Programmer.
ok so what are the alternatives to ksplice for the swap device change. since we could make it swap/compcache anyways is it possible to change on the fly like the way overclocking widget does?
Ksplice allows you to make kernel updates and have them implement themselves without having to reboot the device. The swap device stuff would have to be implemented on a script-level. Two different things: I.e.; the script invokes ksplice to install a specific patch, which causes the kernel to recognize the newly freed-up RAM as a specific device (I.e.; such as /dev/ram1 as opposed to /dev/ram0). I am not sure I'm saying/thinking of this properly here: I'm operating on the edge of what I know about Linux.
Anyhow -- the trick is, when we get the kernel to disable the GPU's memory, we enable a new RAM-disc swap partition with its swappiness set to the maximum rating so it picks up everything first. Since it is /actually RAM/, it'll perform like real RAM. We're only 'declaring' it as swap in order to prevent forced reboots when turning it off.
The above wouldn't just be a ksplice thing; ksplice is just to implement daproy's kernel patch and thus free up the devoted memory to the GPU1. This memory would then be allocated to a new device (thus preventing the RAM-dump forced reboot) and /that/ would be set to swap status to ensure no "mission-critical" files go on it.
Then, once we get that built up, we just build it in "reverse" -- turn off swap (dumping its info back to the system, where CompCache / linux-swap would pick up the load) and re-enabling the GPU1.
Once we get /those/ built, we can then create a daemon script, include an invocation of it into the latest userinit.sh, and have that script basically monitor GPU0's load levels. If they go above a certain threshold (scaling, just like with the CPU), it would invoke the second script above, patching the kernel and re-enabling GPU1. This would then monitor the GPU, and once //its// memory load falls back down to where GPU0 can handle it all, turns GPU1 off and frees that RAM back up for individual use.
I can already foresee a complication where turning GPU1 off "on the fly" would cause a reload of all graphics, but... considering how rare it is to even pull a load above what GPU0 can handle in ordinary use, this isn't much of an issue. It'd take some tweaking and beta-testing (which I'm pretty much set to be a tester of at this point, apparently ) but that would be about it.
Now, //ideally// we could create an alternative solution which wouldn't rely on constantly patching the kernel. As I said; that can get... well, "messy".
I just don't know enough to know what that alternative path would be. Virtual shared memory space seems like it would be a good idea. An alternative solution could be something along the lines of enabling GPU1 at all times and simply having it call to a swapfile which is located in /tmp (I.e.; a non-compressing swap-file which is actually located in RAM). Said swapfile could be resized freely according to need and would thus be no issue at all.
Frankly, I can already see one reason why that would be a superior solution to all the ksplice shenanigans above: said swapfile could be made to be /larger/ than 8 MB, and thus actually provide for an increased performance due to higher available memory for the graphics side of things. We'd want some way to make sure that the swapfile itself wouldn't get pages dumped onto an actual swap partition. This seems like a much more elegant solution than the ksplice routine. I simply have no idea if it's even feasible. The swapfile would have to be mounted into /tmp (or it's Android equivalent) //at the time the GPU1 was enabled//. Is that even /possible/ at boot? Could we simply enable the device very late in the boot stage?
Quickly! Someone with /actual programming expertise/, rescue me from my aimless mental tinkering!
Since it seems there's little activity in this thread, I thought I'd share:
There is a debian armel package for ksplice.
( http://packages.debian.org/squeeze/ksplice )
This is ARM-compiled. We would simply need to cobble together a script which invokes ksplice properly. I myself know nothing of how this would be done.
Usage example of ksplice here:
http://www.ksplice.com/example-update
Looks pretty simple. You just need a working kernel build environment. Anyone got one of those laying around? I have wanted to do that for awhile but am too lazy to set up a linux VM, lol.
Hell yeah!!
Im not much good for this project other then willingness to try betas and give feedback, but I'm excited to see where this goes.. It would be great to reclaim the ram being used from the system when you aren't using anything that needs it and use it for what you are using. Has anyone contacted one of the devs that have been modifying the kernels and ask them if they could try building it with the ksplice functionality?? I would think one of them would be happy to build it so you can start playing with it. Wish I could help more on it, but this is a little above my head, but I will be following this thread and learning what I can as i go.. Good Luck!!

[Q] Can someone please explain what the transformer I/O problem means?

I've seen it in threads that the "I/O" is a problem on the tf700.
What does that mean? I don't know what it is :silly:
ap3604 said:
I've seen it in threads that the "I/O" is a problem on the tf700.
What does that mean? I don't know what it is :silly:
Click to expand...
Click to collapse
http://en.wikipedia.org/wiki/Input/output
To add just a bit of context to the rather unfriendly and a bit too concise a reply above, the TF700 -- as all other Transformer edition tablets -- suffer from lags and hangups when a disk-intensive task is run concurrently with some other task (for example when installing or updating an app in the background and browsing the web or watching a (high-resolution) video at the same time) or when transferring large amounts of data to or from both internal and external memory (like that video you wanted watch while letting your apps update in the background ). Note that the term 'disk-intensive' is a relative one, since it also happens with operations so limited that it is hard to believe the hardware would be overwhelmed to any significant degree.
I was able to see this easily last night installing large apps and try to use the play st ore while downloading. Never had this on my og transformer.
Otherwise this is a beautiful tablet
MartyHulskemper nails the description, so I am listing some specific examples.
1. Installing applications stalls other action/task
Presumably, installing application actively access your flash drive and this basically freezes your system temporary even it is non-intensive application such as simply non-flash based site browsing.
2. Downloading large file stalls other action/task
Just as #1, but again this is writing/accessing your flash drive.
3. Copy/Write from PC to Tablet by USB
Small files flies. Big files inconsistently but relatively frequently stops/freezes. Presumably some sort of issue from writing to flash disk.
4. Slower PDF opening (I am speculating this)
Compared to my former system, which was dual core, opening large PDF is much slower and almost consistently gives me "No response" Error. This I am blaming on the reading from flash drive is somehow the issue.
So You will encounter the problems here and there as I/O is pretty much used everywhere.

Confused about I/O slow down (looking for technical explanation) ?

So everyone (including me) has noticed that the transformer slows down when doing i/o. I originally thought this was a hardware issue (slow memory? slow bus?) but from various threads it sounds like third party os fix the issue. So I have two questions:
Can someone explain what the asus kernel does wrong (or how third party kernels) fix the issue ?
Why asus cannot copy the fixes from third party kernels into their kernel (I presume this is a kernel issue and not support software around the kernel but maybe that presumption is incorrect; maybe it is a driver issue or maybe there really is a hardware issue?)
jake21 said:
So everyone (including me) has noticed that the transformer slows down when doing i/o. I originally thought this was a hardware issue (slow memory? slow bus?)
Click to expand...
Click to collapse
Random write of small blocks to the internal eMMC is slow. Flash memory has huge erase blocks (typically a couple of megabytes) and large write blocks. Writing 4KB is a relatively slow process.
jake21 said:
but from various threads it sounds like third party os fix the issue.
Click to expand...
Click to collapse
They can't fix it, they can only work around the issue by tweaking the kernel's caching parameters. Or, in the extreme case, disabling the fsync system call. Usually, an application calls fsync to ensure data has been written to the disk, so that even in case of a following crash or unexpected power loss the data on the medium is consistent. And normally fsync waits until the write command has completed. If you disable fsync, the app no longer has to wait, therefore no more lag. The data still resides in the RAM and is eventually written to the card by the background cache flush thread.
Downside of disabling fsync: If the tablet crashes in the wrong moment, you may in the worst case lose all your data, run into a bootloop, etc.
If this is true, how come with original release ICS 4.0.3 everything runs smooth and fast. I installed latest Jelly Bean 4.1 and slow real bad, i downgraded back to ICS 4.03 and it is fast again. Is it a driver issue. It can't be hardware.
Thanks
_that said:
Random write of small blocks to the internal eMMC is slow. Flash memory has huge erase blocks (typically a couple of megabytes) and large write blocks. Writing 4KB is a relatively slow process.
They can't fix it, they can only work around the issue by tweaking the kernel's caching parameters. Or, in the extreme case, disabling the fsync system call. Usually, an application calls fsync to ensure data has been written to the disk, so that even in case of a following crash or unexpected power loss the data on the medium is consistent. And normally fsync waits until the write command has completed. If you disable fsync, the app no longer has to wait, therefore no more lag. The data still resides in the RAM and is eventually written to the card by the background cache flush thread.
Downside of disabling fsync: If the tablet crashes in the wrong moment, you may in the worst case lose all your data, run into a bootloop, etc.
Click to expand...
Click to collapse
Ok I can understand the issue with fsync and small writes but then all tablets would have this issue (unless the infinity used particularly poor chioce of hardware). Also does this indicate if writes were disabled in (for example) browsers then they would be silky smooth ?
-
It would be nice if the tablet could mark certain directories as critical and flush those faster than other directories (perhaps abusing the meaning of the sticky bit on the directory). Certain non critical data can avoid having immediate flush (though if andriod apps are calling fsync explicitly there might be some stickyness in changing the behavior of the api). Anyways is my understanding correct that you are indicating that Asus used a particularly poor choice of eMMC or tweaked the kernel to flush more frequently ?
_that said:
Random write of small blocks to the internal eMMC is slow. Flash memory has huge erase blocks (typically a couple of megabytes) and large write blocks. Writing 4KB is a relatively slow process.
They can't fix it, they can only work around the issue by tweaking the kernel's caching parameters. Or, in the extreme case, disabling the fsync system call. Usually, an application calls fsync to ensure data has been written to the disk, so that even in case of a following crash or unexpected power loss the data on the medium is consistent. And normally fsync waits until the write command has completed. If you disable fsync, the app no longer has to wait, therefore no more lag. The data still resides in the RAM and is eventually written to the card by the background cache flush thread.
Downside of disabling fsync: If the tablet crashes in the wrong moment, you may in the worst case lose all your data, run into a bootloop, etc.
Click to expand...
Click to collapse
gordo2000 said:
If this is true, how come with original release ICS 4.0.3 everything runs smooth and fast. I installed latest Jelly Bean 4.1 and slow real bad, i downgraded back to ICS 4.03 and it is fast again. Is it a driver issue. It can't be hardware.
Click to expand...
Click to collapse
I didn't see a big performance difference between ICS and JB, even though JB should be even faster after all the "Project Butter" work. What is slow for you on JB?
---------- Post added at 07:35 PM ---------- Previous post was at 07:30 PM ----------
jake21 said:
Ok I can understand the issue with fsync and small writes but then all tablets would have this issue (unless the infinity used particularly poor chioce of hardware). Also does this indicate if writes were disabled in (for example) browsers then they would be silky smooth ?
Click to expand...
Click to collapse
Many people said the TF700's eMMC is slower than good microSD cards, and that can be reproduced with benchmarks.
To check how the browser performs if it doesn't have to write to the eMMC, install Browser2RAM, which moves the browser cache to a ramdisk. In my experience, there is still lag on some pages - so not all slowdowns seem to be I/O-related. It would be interesting to find out the real cause of this.
There may be another I/O situation except random writes: large writes which block small reads from another process. HPI should help here, but I think the 3.1 kernel doesn't support it yet.
_that said:
I didn't see a big performance difference between ICS and JB, even though JB should be even faster after all the "Project Butter" work. What is slow for you on JB?
---------- Post added at 07:35 PM ---------- Previous post was at 07:30 PM ----------
Many people said the TF700's eMMC is slower than good microSD cards, and that can be reproduced with benchmarks.
To check how the browser performs if it doesn't have to write to the eMMC, install Browser2RAM, which moves the browser cache to a ramdisk. In my experience, there is still lag on some pages - so not all slowdowns seem to be I/O-related. It would be interesting to find out the real cause of this.
There may be another I/O situation except random writes: large writes which block small reads from another process. HPI should help here, but I think the 3.1 kernel doesn't support it yet.
Click to expand...
Click to collapse
Agreed. However, even with Browser2Ram, I'm betting that there is still some i/o with the emmc that cannot be hijacked by broser2ram, and therein lies the problem. If nothing else, using RAM like that may force the tablet (b/c of screwy coding) to start paging data to...yup, you guessed it, emmc ... a lot sooner than it actually needs to.
Has anyone tried B2R along with dev settings to kill apps ASAP that are not in use? Perhaps this could lengthen the time before paging starts to occur?
Sent from my ASUS Transformer Infinity TF700 running Android JB (rooted) via Tapatalk
Overall, everything runs smooth on ICS 4.0.3. Browser, opening app, games, there is no hick ups when watching movies, that happen alot on JB 4.1.1. The whole OS is smoth redrawing. On JB, there is always a wait few seconds to open application folders or closing it. I did reformat to default but no help.
_that said:
I didn't see a big performance difference between ICS and JB, even though JB should be even faster after all the "Project Butter" work. What is slow for you on JB?
---------- Post added at 07:35 PM ---------- Previous post was at 07:30 PM ----------
Many people said the TF700's eMMC is slower than good microSD cards, and that can be reproduced with benchmarks.
To check how the browser performs if it doesn't have to write to the eMMC, install Browser2RAM, which moves the browser cache to a ramdisk. In my experience, there is still lag on some pages - so not all slowdowns seem to be I/O-related. It would be interesting to find out the real cause of this.
There may be another I/O situation except random writes: large writes which block small reads from another process. HPI should help here, but I think the 3.1 kernel doesn't support it yet.
Click to expand...
Click to collapse
johnlgalt said:
Agreed. However, even with Browser2Ram, I'm betting that there is still some i/o with the emmc that cannot be hijacked by broser2ram, and therein lies the problem.
Click to expand...
Click to collapse
Not all I/O automatically leads to problems. The latest version of Browser2RAM only redirects the browser cache, it does not affect browser settings, bookmarks, etc. - which is usually a good thing.
johnlgalt said:
If nothing else, using RAM like that may force the tablet (b/c of screwy coding) to start paging data to...yup, you guessed it, emmc ... a lot sooner than it actually needs to.
Click to expand...
Click to collapse
No. Paging does not occur *to* the eMMC (no swap space is configured on the TF700), but only *from* the eMMC, to fetch pages of executable files. While it is true that the ramdisk for the cache uses some memory, it would only make a difference if you have lots of background apps competing for RAM.
A first step to see how much I/O happens is to watch the output of "iostat".
Thanks for the heads up. So, why does it still cause pauses and the like then?
Sent from my ASUS Transformer Infinity TF700 running Android JB (rooted) via Tapatalk
johnlgalt said:
So, why does it still cause pauses and the like then?
Click to expand...
Click to collapse
Good question. It's time to find out.
Arm yourself with multiple adb shells and watch the output of iostat, top, free, or whatever else you can think of that displays interesting metrics. Then do something that causes lag and see if you notice a specific pattern.
(I am currently away from my main PC, and the SSD in my laptop decided yesterday it no longer wants to read ntkrnlpa.exe - so no adb for me right now)
I'll need a bit more specifics - I know adb well enough and can shell, but these other ... executables you're mentioning are new to me.
I'm on vacation in Hawaii, and have a Windows 7 based laptop that I can use, so I can do this no problem - but not today. About to go see some sights before going on a Lava Boat tour at 4 PM local, which means I'll be bushed when I get back - plus I'm fighting a nasty ear infection that aches something awful.
AFAIK, though, I have no real plans for tomorrow or Friday, so I can take some time and investigate.
Also, FWIW: I'm rooted but have not (yet) unlocked my bootloader - mainly b/c I purchased the 64 GB version of the tablet and it is a C50, so I'm hoping something 'breaks' enough for me to get a replacement (c70? C90 even? )- and hoping even more that it is running something under ICS .30 so I can nvflash a backup and not have to worry about goofing things up when I *do* unlock the bootloader.
Sent from my ASUS Transformer Infinity TF700 running Android JB (rooted) via Tapatalk
iostat and top are standard unix utilities. They would only be useful if run on the phone so I must presume andriod has versions. A bit of 'googling' and htere appears to be a developer's kit that includes stuff like iostat. If hte switches are the norm then something like "iostat -x 2" will produce nice output of performance of each 'disk'. top is a tool that shows cpu usage.
-
I've not done any developing for andriod (maybe I should bite hte bullet?) so have never tried to use adb or similar but i've done a bit of system development on linux (though I very rarely muck with the kernel; i prefer to work one layer above the kernel).
johnlgalt said:
I'll need a bit more specifics - I know adb well enough and can shell, but these other ... executables you're mentioning are new to me.
Click to expand...
Click to collapse
If you haven't done it yet, install BusyBox on your TF700. Then just open one or more command windows on your PC, run adb shell in each, and run "iostat 1" in one window, "top" in another, and maybe also adb logcat in yet another window.
That gives you up-to-date statistics about I/O and processes which currently use CPU time. Then try to use your tablet normally, and when it lags, watch the output on your PC if you see a big number of writes or a process eating CPU.
But don't forget to enjoy your vacation.
_that said:
If you haven't done it yet, install BusyBox on your TF700. Then just open one or more command windows on your PC, run adb shell in each, and run "iostat 1" in one window, "top" in another, and maybe also adb logcat in yet another window.
That gives you up-to-date statistics about I/O and processes which currently use CPU time. Then try to use your tablet normally, and when it lags, watch the output on your PC if you see a big number of writes or a process eating CPU.
But don't forget to enjoy your vacation.
Click to expand...
Click to collapse
Awesome. Busybox already installed here, so this should be easy enough.
And I never forget to enjoy ... anything. lol
Sent from my ASUS Transformer Infinity TF700 running Android JB (rooted) via Tapatalk
_that said:
Good question. It's time to find out.
Arm yourself with multiple adb shells and watch the output of iostat, top, free, or whatever else you can think of that displays interesting metrics. Then do something that causes lag and see if you notice a specific pattern.
(I am currently away from my main PC, and the SSD in my laptop decided yesterday it no longer wants to read ntkrnlpa.exe - so no adb for me right now)
Click to expand...
Click to collapse
hey fellas, have you seen chainfire's app : PerfMon
http://www.xda-developers.com/android/perfmon-floats-your-devices-performance-on-screen/
real time stats including io reads/writes to both mmcblk0/mmcblk1... :good:
ps: i have always loved "android status" aswell ... oldy but a goody
https://play.google.com/store/apps/details?id=com.AndroidStatus&hl=en

[Q] Speed of Emulator on Eclipse

I have Android practicals at college and we use Eclipse IDE on 32bit Linux machines.
Since it's a college, we all know that all the machines are equally configured, and I checked, they indeed are.
Now here's the confusion: we run the same and ditto configured AVDs and the speed is different across the machines. Here I am having the emulator running fast and the very next PC is having it very slow.
The PC's config:
3.8 GiB
Intel® Core™ i3-2120 CPU @ 3.30GHz × 4
Graphics: unknown
And the thing that amazed me was, I tried running the emulator on machines having just 2 GB RAM and it still worked fine. So what could it be? It is varying with machines, doesn't matter if the processor is fast or slow or same or different. :|
Please help me find this problem. Thanks in advance.
This is a very theoretical answer and I am myself not pretty conviced by it but if you emulate something it is emulated on the condition the computer has at that moment.
So maybe one of them was in another condution
(i.e. overheat, backgroundprocess, maybe even an old driver that doesn't support so well,...)
Was it only one computer that emulated it very slow and the others not?
Next question: Was it slow all the time? Or only few in a moments (like at the start of it?)
GuiltyGod said:
This is a very theoretical answer and I am myself not pretty conviced by it but if you emulate something it is emulated on the condition the computer has at that moment.
So maybe one of them was in another condution
(i.e. overheat, backgroundprocess, maybe even an old driver that doesn't support so well,...)
Was it only one computer that emulated it very slow and the others not?
Next question: Was it slow all the time? Or only few in a moments (like at the start of it?)
Click to expand...
Click to collapse
There are a few specific machines who will always run it faster. I also had a thought that some kind of external process would be affecting this. Though the processes list didn't tell me anything.
Thanks.

Categories

Resources