ok its simple we all know the benefits of more ram. we all know that android slices up the ram into static addresses within the kernel. i would like to change that as we are in 2009 almost 2010 and we still rely on static ram allocation for the camera, gpu, sensors, radio. the g1 has 192mb of ram. yet we only get 97 for programs to use. i would like to place the camera, gpu, etc into dynamic addresses that we can change within the os without flashing a new kernel. a service, kernel code, module, or any other method could do this. we just have to make it happen. we could do it three ways i see.
1. a system managed ram allocation using a service to detect the percentage used and increase ram on the go.
2. a kernel script or service that only activates and allocates ram when needed. such as once a 3d program is loaded the 8mb magically comes back.
3. a virtual device ram allocation service that makes the device think it has the ram all along, but only actually gets it when it needs it. kinda like the way apps2sd works.
4.(by IConrad01) run perl scripts through ase and use ksplice to patch the gpu1 memory in kernel. one script on and one for off.
any other methods to implement this proper are very welcome and encouraged.
the idea behind this is a proper way to get back more ram for our programs. the idea is simple when 3d is not in use we can used the 8mb it would have used for something else. this applys to everything that has its own ram. camera, etc. say we did this and only got 120 free. thats almost 25% more ram. plus we could go the extra step and let the user choose how much each thing gets(gpu settings 4mb,8mb,12mb. camera on, off. etc) so when it needs it thats what it gets. great if you like/dislike 3d games.
on another note this is the kind of work that gets you noticed. the kind that gets you hired or gets you respect. google would be proud and might implement it and you and i know it. i hope we are successful and google uses it mainstream.
so lets debate a way/ways to do this
provide alpha tests of different ways to do this
and finally lets post a beta so everyone can enjoy.
atm ksplice with perl scripts is where we are going will update as this progresses.
Doesnt Jacs update to his hero rom have this, im probably mistaken
jad011 said:
Doesnt Jacs update to his hero rom have this, im probably mistaken
Click to expand...
Click to collapse
No. The current implementation of this basically cuts the graphics memory in half.
This would dynamically allocate the memory as needed. Much more efficient way of doing it.
he has a static change that takes the ram away from the gpu by a kernel edit. this is to have both more ram but still give the gpu what it needs to perform properly but only when it needs it or you chose to give it.
Are you referring to RAM or storage space? You say RAM in the title but then it sounds like you're talking about storage space before making me think it's RAM again later on. I'm confused :/ I'd really like additional free RAM, anything that helps reduce swapping to SD card is good but storage space I have plenty of.
all of this would apply to ram only. swap and compcache could use the same idea to dynamically allocate swap but that is still a cache for programs. sorry if its confusing. simple way to explain. i want all the ram i can get but still use everything the phone has properly, like games and the camera.
Gotcha and understood. I see so many people say RAM and memory when they're referring to storage space sometimes that I want to scream but extra RAM on the G1 would be more than welcome. Hopefully others with the brains to make it happen can do it
actually anyone with a basic knowledge can help. mostly there are a couple things to developing a good idea.
first the concept.
then ideas on how to do it.
next a map or guide to what does what.
coding different parts
putting them together
debug and test
even if all you do is photoshop the ui or provide ideas it helps. once some different people get aboard this should be working sooner than you think.
Of the three options listed, the kernel script (via ksplice) seems like it would be the shortest route to functionality, but frankly it also seems a bit... well, hackish -- when compared to the virtual device allocation scheme. It would take a hefty re-write of the kernel, I suspect, to accomplish; but being able to resize various devices on the fly would be a good thing. Shrinking CompCache based on real RAM usage for example would probably speed up the phone under light loads.
Just thinking in text here.
personal i think its going to start simple(script or root app) and go straight to gpu shared on or off. choose and reboot. however i do not want to stop there like winmo did. someone could create a short kernel patch that checks for a service or script and gets value or defaults to standard method. then we would have to come up with how to manage the switch(realtime vs. reboot) ideally we would remap the memory from scratch and rebuild the memory management services to dynamically allocate the ram for all devices(128+ ram) as needed. we have a start, we know where and how to turn the gpu1 shared on or off. now lets make it more than a hack and work on how to make our phones better. btw this is not a waste of time. this extends to other phones and the future way to set up our phones from default.
jokersax11 said:
personal i think its going to start simple(script or root app) and go straight to gpu shared on or off. choose and reboot. however i do not want to stop there like winmo did. someone could create a short kernel patch that checks for a service or script and gets value or defaults to standard method. then we would have to come up with how to manage the switch(realtime vs. reboot) ideally we would remap the memory from scratch and rebuild the memory management services to dynamically allocate the ram for all devices(128+ ram) as needed. we have a start, we know where and how to turn the gpu1 shared on or off. now lets make it more than a hack and work on how to make our phones better. btw this is not a waste of time. this extends to other phones and the future way to set up our phones from default.
Click to expand...
Click to collapse
The only problem with this is when the ram is divided, everything in it is cleared, thats why it is set at boot in the kernel. Think of it like repartitioning your SD. Now it would probably be possible have everything mapped to ONE big RAM table & then build something similar to a scheduler like BFS for RAM that would allocate it as necessary. This is messy though.
Just my $.02
well yes but it is possible to create shared memory through c++ http://www.go4expert.com/forums/showthread.php?t=8461
however i think the vm heaps has either a 6 or 16mb limit. anyways the service would basically do a memory patch over the kernel in realtime(like overclock) telling the gpu the memory is there all the time. meanwhile it allocates dynamic shared memory from the application memory for the gpu to read and write. meanwhile an app tied to a script would set the parameters and settings.
Okay -- I know ksplice works with perl; and I know that we can run perl scripts using the ASE .apk.
So... it seems to me that it /should/ be possible to use a patch that reverts the patched kernel back to the previous state, and a patch that switches the original state to the GPU1=0 MB state. From there, we could set up two scripts; one for GPU1-off and one for GPU0-on. Each would simply be an invocation of ksplice to input the specific patch (which would have to be stored on the sd card. [it would be safer if in the ext partition]).
We'd have to set up a secondary RAM device, rather than just adding it to the standard RAM pool... something like establishing it as a /dev/block/shm.
The point of all the above is that when you click on the GPU1-off script, it "reclaims" that RAM and gives you a device that's standard RAM. When you click on the GPU1-on script, it disables that device (forcing the reload of all programs using that RAM) and re-enables the GPU1 device.
Even better yet, the Android Scripting Environment program, much like GScript, supports desktop icons.
So -- very initially speaking, assuming we can get ksplice working on Android (which just requires a static binary, IIRC), we can get a simple pair of icons; one of which frees up the GPU1's RAM; and the other of which re-enables the GPU1.
Since the only times it's going to be "called up" are when a game is actively being played, or something similar, this should be a mostly-satisfactory route to begin with, before we can get into the whole scheduled/scaled script running as a daemon in the background.
Once we get /there/, then we can start looking at true virtual device allocations and shared memory and the like.
Although, I do have to say that I'm not entirely sure how comfortable I am constantly repatching the kernel. Definitely, we're going to want to recommend that people back up their original, unpatched, boot.img files!
IConrad01 i added your method to first post as it was a great start to this project. if you want me to rephrase just post what you want and ill change it. the only issue i might see is what geniusdog254 said it might unallocate the wrong thing and create a reboot. but some clever coding should prevent this.
Nah, that's all good.
There's an easy way to prevent it from becoming a reboot-able issue. Enable the GPU-space device as "swap". This could be done one of two ways; tell the phone that the new 8 MB RAM "device" is to be considered the /first/ swap device to be used, and just leave it "as is". The r/w times would be for true RAM. The other option assumes we have the latest CompCache (0.6+) -- as it allows for multiple instances of CompCache. The theme here should be obvious.
Either way, the result is that only swappable pagefiles would be lost when the device is turned back into GPU memory. No system-vital memory would get tossed. Seems simple enough to accomplish, perhaps. But then again, I Am Not A Programmer.
ok so what are the alternatives to ksplice for the swap device change. since we could make it swap/compcache anyways is it possible to change on the fly like the way overclocking widget does?
Ksplice allows you to make kernel updates and have them implement themselves without having to reboot the device. The swap device stuff would have to be implemented on a script-level. Two different things: I.e.; the script invokes ksplice to install a specific patch, which causes the kernel to recognize the newly freed-up RAM as a specific device (I.e.; such as /dev/ram1 as opposed to /dev/ram0). I am not sure I'm saying/thinking of this properly here: I'm operating on the edge of what I know about Linux.
Anyhow -- the trick is, when we get the kernel to disable the GPU's memory, we enable a new RAM-disc swap partition with its swappiness set to the maximum rating so it picks up everything first. Since it is /actually RAM/, it'll perform like real RAM. We're only 'declaring' it as swap in order to prevent forced reboots when turning it off.
The above wouldn't just be a ksplice thing; ksplice is just to implement daproy's kernel patch and thus free up the devoted memory to the GPU1. This memory would then be allocated to a new device (thus preventing the RAM-dump forced reboot) and /that/ would be set to swap status to ensure no "mission-critical" files go on it.
Then, once we get that built up, we just build it in "reverse" -- turn off swap (dumping its info back to the system, where CompCache / linux-swap would pick up the load) and re-enabling the GPU1.
Once we get /those/ built, we can then create a daemon script, include an invocation of it into the latest userinit.sh, and have that script basically monitor GPU0's load levels. If they go above a certain threshold (scaling, just like with the CPU), it would invoke the second script above, patching the kernel and re-enabling GPU1. This would then monitor the GPU, and once //its// memory load falls back down to where GPU0 can handle it all, turns GPU1 off and frees that RAM back up for individual use.
I can already foresee a complication where turning GPU1 off "on the fly" would cause a reload of all graphics, but... considering how rare it is to even pull a load above what GPU0 can handle in ordinary use, this isn't much of an issue. It'd take some tweaking and beta-testing (which I'm pretty much set to be a tester of at this point, apparently ) but that would be about it.
Now, //ideally// we could create an alternative solution which wouldn't rely on constantly patching the kernel. As I said; that can get... well, "messy".
I just don't know enough to know what that alternative path would be. Virtual shared memory space seems like it would be a good idea. An alternative solution could be something along the lines of enabling GPU1 at all times and simply having it call to a swapfile which is located in /tmp (I.e.; a non-compressing swap-file which is actually located in RAM). Said swapfile could be resized freely according to need and would thus be no issue at all.
Frankly, I can already see one reason why that would be a superior solution to all the ksplice shenanigans above: said swapfile could be made to be /larger/ than 8 MB, and thus actually provide for an increased performance due to higher available memory for the graphics side of things. We'd want some way to make sure that the swapfile itself wouldn't get pages dumped onto an actual swap partition. This seems like a much more elegant solution than the ksplice routine. I simply have no idea if it's even feasible. The swapfile would have to be mounted into /tmp (or it's Android equivalent) //at the time the GPU1 was enabled//. Is that even /possible/ at boot? Could we simply enable the device very late in the boot stage?
Quickly! Someone with /actual programming expertise/, rescue me from my aimless mental tinkering!
Since it seems there's little activity in this thread, I thought I'd share:
There is a debian armel package for ksplice.
( http://packages.debian.org/squeeze/ksplice )
This is ARM-compiled. We would simply need to cobble together a script which invokes ksplice properly. I myself know nothing of how this would be done.
Usage example of ksplice here:
http://www.ksplice.com/example-update
Looks pretty simple. You just need a working kernel build environment. Anyone got one of those laying around? I have wanted to do that for awhile but am too lazy to set up a linux VM, lol.
Hell yeah!!
Im not much good for this project other then willingness to try betas and give feedback, but I'm excited to see where this goes.. It would be great to reclaim the ram being used from the system when you aren't using anything that needs it and use it for what you are using. Has anyone contacted one of the devs that have been modifying the kernels and ask them if they could try building it with the ksplice functionality?? I would think one of them would be happy to build it so you can start playing with it. Wish I could help more on it, but this is a little above my head, but I will be following this thread and learning what I can as i go.. Good Luck!!
Related
The lovely thing about having a Linux Kernel is that we can do a lot to optimize it.
I figured it would be a great idea to set up a thread where we can bring all these optimizations into one place.
Mind you, I'm no expert by any means...
For my (possibly useless) contribution:
When we compile the kernel to the G1, do we compile it with the general Arm eabi? If so, is that made for ALL ARM cpu's? We have an ARM6 in our G1's. If we set it up to optimize to ARM6, can we set it up to use the CPSID/CPSIE instruction instead of the SWP instruction? Are the drivers open source? Can we recompile them to do the same also? To learn more about why, read this:
http://forum.xda-developers.com/showthread.php?t=516223
That's where I got the idea from.
SWP is a deprecated opcode in armv6 that decreases performance due to the way it locks the bus.
Click to expand...
Click to collapse
Anyhow, whatever you guys post here, I will add to this post. Also, to the people to compile the kernels, can you answer my questions?
answers to some of your questions.
hi i usually don't post here since i am not a developer but in this case i think i can answer some of your questions (since i sometime make custom kernels for my laptop). kernel optimazation is used to removed excessive drivers and services from the kernel that you don't need that slows down your computer or in this case phone. i believe htc already does this when they configured android for the g1. most cell phone manufactures already do this to ensure that there products processor use energry efficiently and and run as smoothly as possible.
When we compile the kernel to the G1, do we compile it with the general Arm eabi? If so, is that made for ALL ARM cpu's? We have an ARM6 in our G1's
Click to expand...
Click to collapse
yes, arm eabi is made for all general arm cpu's i believe you can complie it for arm 6 specifically though. how do you know that htc hasn't already optimize the kernel for that specific processor? also if they didn't you wouldn't really see any noticible speed improvements by compliling the kernel for this specific processor.
Are the drivers open source?
Click to expand...
Click to collapse
no, futhermore they don't offer the drivers for the g1 except execpt for the usb driver so your computer can recognise the g1 as mass storage device.
If we set it up to optimize to ARM6, can we set it up to use the CPSID/CPSIE instruction instead of the SWP instruction?
Click to expand...
Click to collapse
i don't know sorry. thats all i know i'm really not a kernel, coding expert i hope this helps in your task.
bananajama said:
no, futhermore they don't offer the drivers for the g1 except execpt for the usb driver so your computer can recognise the g1 as mass storage device.
Click to expand...
Click to collapse
I think you meant to say "ADB interface" instead of "mass storage device."
t1n0m3n said:
I think you meant to say "ADB interface" instead of "mass storage device."
Click to expand...
Click to collapse
The system type is set to "Quaalcom MSM", so I would imagine that it's optimizing for arm6l.
cyanogen said:
The system type is set to "Quaalcom MSM", so I would imagine that it's optimizing for arm6l.
Click to expand...
Click to collapse
Sounds like it. Well, other than that, are there any other obtimizations anyone else can think of?
there are few things in linux that you can play with to see if you get any performance increase. It's mainly in memory management. run ls on /proc/sys/vm. you can play with the page cache but I doubt that you'll get much perf increase from it since, since most of the things we can play with are for write back cache. If we have access to the dalvikvm we may be able to do more optimization. The one thing that I've always wanted to be able to do is to pin Home in memory, and not allow garbage collector to kill it, so when we exit out of memory intensive apps, like browser, gallery, etc, we do not have to wait for Home to reload. If we have more control over garbage collection we may be able to do more, but I don't know much about the dalvikvm at all.
taking a quick look at the vm. The only nob that may have any effect on android may be vfs_cache_pressure. theoretically setting to 200 would push fs cache out quicker and give more memory to the vm. default is set to 100. I set it to 200 and 300 but didn't notice any difference.
knaries2000 said:
there are few things in linux that you can play with to see if you get any performance increase. It's mainly in memory management. run ls on /proc/sys/vm. you can play with the page cache but I doubt that you'll get much perf increase from it since, since most of the things we can play with are for write back cache. If we have access to the dalvikvm we may be able to do more optimization. The one thing that I've always wanted to be able to do is to pin Home in memory, and not allow garbage collector to kill it, so when we exit out of memory intensive apps, like browser, gallery, etc, we do not have to wait for Home to reload. If we have more control over garbage collection we may be able to do more, but I don't know much about the dalvikvm at all.
Click to expand...
Click to collapse
dxtop, which is a home replacement, does exactly that. It keeps itself in memory so going back to it is much faster than the default home application. And in low memory situations, it tries to free up as much resources as possible while still staying in memory.
At the android-platform group, we've been hashing out a scheme for adding in official apps-to-sd support to AOSP. We have a couple of google engineers following along/helping out and are now at a state where the initial testing implementation (we're using an incremental development approach) steps are defined in a fairly simple manner and we are ready to start at it from an actual implementation details/start coding perspective.
The actual thread is located here: http://groups.google.com/group/android-platform/browse_thread/thread/bf0709c157451cd9
Basically, if implemented, it will do the following;
1) totally obsolete current hacker apps2sd approaches by allowing actual sdcard removal from device.
2) ultimately ship with devices stock (when in a state where it is easy to use, stable, and at least as secure for non-root users as internal storage currently is).
3) keep application data on the same device as the actual application with no side-effects (like internal apps being broken while waiting for second partition to mount).
4) allow MULTIPLE sdcards containing apps to be swapped on the same device.
5) allow sdcard containing apps to be swapped between DIFFERENT devices.
Note: 4 and 5 are not in the initial implementation, first proof of concept and working system, then enhancement with additional features. 4 and 5 are not requirements for inclusion in AOSP, but they are cool features that ultimately should be implemented.
What we need:
Several good developers, web storage w/source/patch management, etc.
Anyone interested, please read the thread to get an idea of the current state of thought, and please don't pollute that thread with nonsense. There is a current state of organization, and though not set in stone, it should be considered as NOT open for major architectural changes (i.e., the google engineers don't have any major problems with the proposal that we can't work through). Minor glitches and implementation details will be handled along the way. If you must pollute a thread with nonsense, use this one.
Really? Nobody AT ALL is interested?
This is the *ONE* major feature missing from AOSP!
Id PM twistedumbrella and cyanogen and shafty
JAC would prolly be interested but hes been busy with personal stuff i guess?
Just keep bumping this thread to keep it at the top. this needs to be done, and is long overdue on android...
It's a must
I'm sorry that I'm not a developer. Good speed!
I really don't think a2sd is a good solution at all (I've been following the discussion at android groups), rather, I believe the lack of an a2sd solution will eventually lead to device manufacturers to increase the amount of internal storage available on the device for applications (this is what this project is all about, isn't it, not enough storage for apps?) like Samsung did with it's Galaxy.
We shouldn't assume that a device is going to be used a particular way because then we'll run into problems. We shouldn't assume that an user will want to have their device used that particular way, be it partitioned or with a custom, secure filesystem stored in the SD. How do we explain that they'll lose some of their sdcard to app storage? If we make it automatic, how do we allow the user to disable it if they do not want it? How do we make it if an user wants to have one SD card with apps on it and another one without them?
Again I believe we should let the demand for more storage drive the evolution for the next android devices instead of just making it work and have manufacturers ignore the real need for increased internal storage.
I disagree with it not being a good solution. Technology is always advancing, but people can't always follow suit with what is the latest. Be it financials or whatever, Having this as an option will allow older hardware to run more efficiently, Bring costs down for manufacturers and give everyone more options.
@Jubeh, All the questions you raised I believe could be addressed, Have a new settings menu and let them choose. If they select it, it will give it pop up saying "x amount of space will be reserved on your SD card for app storage".
And with AOSP, Android is not just a mobile phone os anymore, It is a mobile platform. Imagine if you buy and download apps on your phone, You save them to your SD card because of this suggested add-in. Now you also own a media tablet that runs android, For example something with a bigger screen usually used for movies and gaming, Now if we had this you could put your sd card in that device and have all your apps like that. I think that would be an amazing feature for android.
I can think of big issues being encryption, piracy seems like it would be easy to accomplish with something like this, but again this should still be addressed and at least attempted to make available. It would be a huge step for the android platform. My 2 and a 1/2 cents worth
I dont think its a bad idea at all...
Jubeh while i agree with your ideas, we definitely need to get more on board memory. But things like apps, and even most cache's shouldnt hinder or take up precious phone storage. I mean seriously, are we hoping for gigs in the near future? Probably not, lol. But the apps2sd is something we can and should change now, to help bring on future ideas.
And what about those already locked into their devices, or where purchasing a newer improved version isnt feasable? Its hard to rationalize a new smart phone every year, even though we all do it, lol. But some bought the g1 hoping to not have to purchase a new device for multiple years, dont they deserve some kind of back compatablity as well?
Whether it should or should not be implemented is not open to debate. The objective is to actually IMPLEMENT it -- in a manner that meets the stability and security requirements of AOSP. One way or another, community needs WILL implement this system, problem is that the current implementations are just crazy HACKS --- unstable, unreliable, etc. As someone who WILL be installing applications to sdcard, *I* want the system to actually WORK PROPERLY, and I'm sure that not only most everyone else (with VERY VERY few exceptions...) does.
Also, the fact that anyone (jubeh) would bring up those completely retarded points about "assumptions regarding use cases" proves in no uncertain terms that they didn't read the thread linked to (even if they did make themselves look completely retarded by replying in it).
In other words jubeh: If you don't read before you reply, you will make yourself look like an a$$. Now run along.
Oh, and what did I say about keeping the NONSENSE out of that thread? Really... you need to learn to READ.
lbcoder, I have to hand it to you. You killed your project quicker than anybody else possibly could have. While many users wouldn't necessarily agree with what jubeh said, he was raising what he considered were valid points in a fairly reasonable manner. Instead of pointing out that you had already worked on those points and that you didn't want to rehash them, you trashed him (three times) and made it pretty clear that you would be an a$$ to work with. I wish you luck in finding devs who want to put up with that.
I think either member have the right to say what they please.
While lbcoder was a bit harsh, I can understand his frustration.
They're both senior members however, and have both have contributed MASS amounts to the comunity. If they want to hash out a problem so be it.
All its doing is keeping this thread at the top
sykokenndogg said:
I think either member have the right to say what they please.
While lbcoder was a bit harsh, I can understand his frustration.
They're both senior members however, and have both have contributed MASS amounts to the comunity. If they want to hash out a problem so be it.
All its doing is keeping this thread at the top
Click to expand...
Click to collapse
i agree. this should DEFINITELY stay at the top non-rooted g1 users at the very least should have these a2sd AOSP updates... and everyone else can just get the regular updates because they have enough internal memory
lbcoder said:
the current implementations are just crazy HACKS --- unstable, unreliable, etc.
Click to expand...
Click to collapse
Not to fill this thread with more nonsense but I have to disagree with you on saying the current apps2sds are just crazy hacks. Hacks? yes. Crazy, unstable, and unreliable? No. The new roms that are out currently automatically move your apps to your ext partition on startup if the ext partition is there. If not then the apps will not move there. The fact that you can dual mount your sd now also illiminates any FCs while you have the phone mounted to a pc. I am not saying that the method can not improve but anyone that is currently running an Enom or Cyan rom can tell you if you didnt personally create the partition then you would have no idea that the apps were on the sd.
Agreed, A2SD is stable
If you follow the directions, Apps2SD is more stable than most of the apps on it, imho.
I think the problem that people are having with stability involve the several ways to get there, the fact that each is a multi-step process, and Android users seem to run the gamut from someone who could hack into Sun Microsystem's payroll to someone who just got their first ,uh, smartphone. Most of us tend toward the latter. If you wrest the control from the user and automate it, then I think we'd see the last of A2SD instability.
Internal memory isn't just for apps, and I think it'll grow regardless. People like high numbers on boxes. WM (WP?) has had this since pre-turn of the century, and the demand for more phone memory hasn't decreased. As a matter of fact, the ROMs just grew, and grew, and grew.
Hey, it's cheaper, it's pretty much just as fast, and if it's easy, people will be able to figure out what the different partitions are once they get them and have to manage them, so it'll teach the masses. I'm all for it. Can't code for diddly, but I like the idea.
Yeah. Bump.
Edit: Yes, you will catch more flies with honey. In the friendliest way I can say it, lose the 'tude, or you'll lose out on a lot, lot, lot of other stuff, and you likely won't be able to figure out why things aren't working out for you. You can't really look back and say what might've been, either. You can, and please do, still say what you need to say, maybe even more, but *how* you say it really matters.
a2sd is FAR from being a stable, reliable, sane solution to the device's storage problem, I've said it time and time again.
Being "Senior Member" is in no way a measure of reliability, experience, or knowledge. I could fill 10,000 posts with 4/5ths of them being "Reported 10 chars" and be a senior member. Also, although I've tried to help where I can, I don't think I've yet contributed anything significant, mainly to avoid the barrage of posts afterwards asking how to make it work... and that brings me back to topic; the storage of apps on SD-card would be hell for carrier's support lines. The implementation is mostly non-existent in MASS MARKET headsets, and although you're right to point out that Android is making strides beyond the phone market, I believe the implementation would be shunned by google for several reasons; the formerly mentioned carrier support hell, both carriers and manufacturer's desire for handsets to become obsolete, google's desire to keep android appealing to both carriers and manufacturers, and possible competition in the thin-portable client and netbook spaces against it's own upcoming Chrome OS.
At this point already, the hope that the feature will "2) ultimately ship with devices stock" is pretty, pretty slim.
As opposed to what most members here might think, we're in the minority (rooted Dream users), and although a2sd does cater to some rooted users, we're still talking about the minority of Dream devices out there (since really, it seems the only reason behind implementing a2sd is the Dream's stock 70 MB app storage space, most other devices at least double that amount). Normal people (read: not us geeks) change devices often almost as a fashion statement, so any solution, if it did make it as an update, would be to support the desire of a small fraction of an almost obsolete device.
Besides, even starting with the way apps are currently handled by the device, it would require a major re-working of the platform to get this monstrosity working. Currently, apps are handled in two spaces, system apps, which can't be un-installed, and user apps, which can be un-installed, updated, etc, but not by the user, but by the package-manager. A better solution would be a third app space for sd-card installed apps. The system/package manager would not install these apks downloaded directly to the sd-card's fat32, rather, they would just show up on the app launcher (we could have scans for new apps every time an sd-card was inserted/removed). With donut's on-demand dexopting, we could create another directory in /data, say, /sd-dalvik-cache, or even leave the .dex in the sdcard while the app was in use and remove it when the app stops (and clear any .dex on sd-card mount), and create a third category of apps that could be installed to sd (in lieu of it, apps would get thrown into /data/app and moved back to sd as soon as one was available, of course, after prompting the user). This way, developers would be able to choose for their apps to be installed to SD and they could take the appropriate security measures to ensure the safety of their code, if that's what they want.
A2SD should have been an option for android in first place. Windows mobile has it, why not android? Is it stable and usable the way it is - sure. But what happens if I want to take out my sdcard and put it in a card reader?
It's one of the major failures of android along with it not supporting adhoc
networks, bluetooth obex as default and some other significant issues.
Don't get me wrong here - there are many things I love abut the platform but
flaws are there too. I've had winmo standart, winmo pro and now an android phone and in terms of "getting the job done" all three have their + and -.
The *current* mechanism to install applications on SD is an EXTREMELY hacky piece of junk.
Though it will work, it will only do so under the following conditions;
1) the user is fully aware of the limitations of the system and doesn't do anything that will stress it out,
2) the sdcard is *always* in the device, never removed.
3) it is impossible to use multiple sdcards in the same device.
Let me pose this question to everyone;
WHAT HAPPENS if you are using hack-apps2sd and you remove the sdcard? You know, just PULL IT OUT... This is something that "regular" users do *all the time*.
This is only one of many conditions that need to be managed by an apps2sd system before it can be considered for inclusion in a consumer device.
Needs to be done;
1) The user needs to be able to chose whether or not to enable apps-to-sd and must set itself up on the phone itself by just the click of a button.
2) The user must be able to SWAP SDCARDS at will. This includes the case where they just rip the card out without unmounting it.
3) When an sdcard is inserted containing apps, the system must automatically set it up and add those applications to the package manager.
4) UID collisions must NEVER happen.
5) External apps must be able to be sanely removed from the package manager upon unmount (planned or unplanned).
6) Processes with open file handles must be politely shut down upon a planned unmount.
7) Processes with open file handled must be CLEANLY killed off upon an UNPLANNED unmount.
8) PROTECTED-APPS must be copy protected when stored on the sdcard to at least equal security to that used internally, i.e. they should be encrypted using a randomly generated key stored in a root-only location within /data.
9) The user must be able to chose where to install a new application.
10) Application home directory and dalvik-cache must be stored on the same media as the application is installed to, i.e. internally installed apps should have their home directory and dalvik-cache stored internally, externally installed apps should have their home directory and dalvik-cache stored externally.
11) Optional: Ability to grow/shrink the amount of storage on the sdcard devoted to applications.
In other words, the user experience should be like this;
1) With a regular sdcard inserted (or no sdcard inserted), the user experience must not be any different than it is currently.
2) User can go to Settings-->SD card & phone storage-->(SD card) Enable application install to SD card. This prompts the user for how much space to devote to applications (default, say equal to internal), and then sets it up.
2B) optional -- user can go to Settings-->SD card & phone storage-->(SD card) "Change SD card space reserved for applications". Prompts for new size (min size = current space used, max size = current available + total sdcard available).
3) User goes to install a new app, if the card has application storage enabled, the installer asks where to install the application to (internal or sdcard).
4) User safely unmounts sdcard -- if applications are running, prompt "There are applications running on the sdcard (list them), these will be terminated. Continue?", terminates applications, removes them from package manager, unmounts.
5) User unsafely pulls sdcard -- if applications were running, message "These applications were running on the sdcard. They have been terminated and any unsaved data has been lost."
6) User inserts or mounts sdcard, system scans if application install is enabled on the card, if it is, the applications are added to package manager.
discussion management
lbcoder,
The thread at groups.google is definitely the technical thread, so I am using this one to comment on your reply dated Oct 30 2:39 pm.
Hands down I believe that for the sake of keeping the discussion open (one of the pillars of the scientific method) is to allow comments that may or may not agree with your or anyone else's point of view.
I agree on that Armando's idea is wrong, just like you do. Although he does have some valid points, which anyone who reads carefully can see. He is probably out of line writing what he did on the technical thread instead of here; and should be scolded for that. But not for sharing his thoughts. I won't elaborate on my own ideas on the matter this because it is not my purpose with this post.
My purpose is to ask everyone working on both this and the technical thread to tone it down, please. XDA sometimes becomes a battleground, sometimes funny and sometimes wasteful and even annoying and both this and the technical thread at groups.google could be very valuable for the platform.
BTW: I'm a well seasoned developer, with well over 15 yrs of experience and who leads reasonably big projects.
Thanks for the thread. It is well worth it, whatever the outcome is.
fosormic said:
lbcoder,
The thread at groups.google is definitely the technical thread, so I am using this one to comment on your reply dated Oct 30 2:39 pm.
Hands down I believe that for the sake of keeping the discussion open (one of the pillars of the scientific method) is to allow comments that may or may not agree with your or anyone else's point of view.
I agree on that Armando's idea is wrong, just like you do. Although he does have some valid points, which anyone who reads carefully can see. He is probably out of line writing what he did on the technical thread instead of here; and should be scolded for that. But not for sharing his thoughts. I won't elaborate on my own ideas on the matter this because it is not my purpose with this post.
My purpose is to ask everyone working on both this and the technical thread to tone it down, please. XDA sometimes becomes a battleground, sometimes funny and sometimes wasteful and even annoying and both this and the technical thread at groups.google could be very valuable for the platform.
BTW: I'm a well seasoned developer, with well over 15 yrs of experience and who leads reasonably big projects.
Thanks for the thread. It is well worth it, whatever the outcome is.
Click to expand...
Click to collapse
There is no place in this discussion for opinions. Its not about battling, its not about opinions, its not about any of that BS. What I am asking is for anyone INTERESTED in CONTRIBUTING (either in code, or in rational discussion regarding implementation details) to come forward and do so. Everything else is irrelevant and out of place.
As for his having valid points... not relevant since ALL of his valid points have been addressed. His purpose (if he has any at all) is therefore simply to disrupt progress.
And since he has effectively destroyed this thread with his nonsense, I may cease monitoring this thread. Anyone interested in contributing, please contact me by PM. Anyone interested in being disruptive, don't waste your time -- really, just go away.
lbcoder said:
The *current* mechanism to install applications on SD is an EXTREMELY hacky piece of junk.
Though it will work, it will only do so under the following conditions;
1) the user is fully aware of the limitations of the system and doesn't do anything that will stress it out,
2) the sdcard is *always* in the device, never removed.
3) it is impossible to use multiple sdcards in the same device.
Let me pose this question to everyone;
WHAT HAPPENS if you are using hack-apps2sd and you remove the sdcard? You know, just PULL IT OUT... This is something that "regular" users do *all the time*.
This is only one of many conditions that need to be managed by an apps2sd system before it can be considered for inclusion in a consumer device.
Needs to be done;
1) The user needs to be able to chose whether or not to enable apps-to-sd and must set itself up on the phone itself by just the click of a button.
2) The user must be able to SWAP SDCARDS at will. This includes the case where they just rip the card out without unmounting it.
3) When an sdcard is inserted containing apps, the system must automatically set it up and add those applications to the package manager.
4) UID collisions must NEVER happen.
5) External apps must be able to be sanely removed from the package manager upon unmount (planned or unplanned).
6) Processes with open file handles must be politely shut down upon a planned unmount.
7) Processes with open file handled must be CLEANLY killed off upon an UNPLANNED unmount.
8) PROTECTED-APPS must be copy protected when stored on the sdcard to at least equal security to that used internally, i.e. they should be encrypted using a randomly generated key stored in a root-only location within /data.
9) The user must be able to chose where to install a new application.
10) Application home directory and dalvik-cache must be stored on the same media as the application is installed to, i.e. internally installed apps should have their home directory and dalvik-cache stored internally, externally installed apps should have their home directory and dalvik-cache stored externally.
11) Optional: Ability to grow/shrink the amount of storage on the sdcard devoted to applications.
In other words, the user experience should be like this;
1) With a regular sdcard inserted (or no sdcard inserted), the user experience must not be any different than it is currently.
2) User can go to Settings-->SD card & phone storage-->(SD card) Enable application install to SD card. This prompts the user for how much space to devote to applications (default, say equal to internal), and then sets it up.
2B) optional -- user can go to Settings-->SD card & phone storage-->(SD card) "Change SD card space reserved for applications". Prompts for new size (min size = current space used, max size = current available + total sdcard available).
3) User goes to install a new app, if the card has application storage enabled, the installer asks where to install the application to (internal or sdcard).
4) User safely unmounts sdcard -- if applications are running, prompt "There are applications running on the sdcard (list them), these will be terminated. Continue?", terminates applications, removes them from package manager, unmounts.
5) User unsafely pulls sdcard -- if applications were running, message "These applications were running on the sdcard. They have been terminated and any unsaved data has been lost."
6) User inserts or mounts sdcard, system scans if application install is enabled on the card, if it is, the applications are added to package manager.
Click to expand...
Click to collapse
well the extremely hacky piece of junk took a lot of hard work from the developers here......show some respect
I'm no developer, scripter, or any type of person with a thorough understanding of software but I will try to offer a suggestion to those who wish to give me the time of day.
After flashing countless ROMs looking for the fastest and most simple platform to use everyday I have discovered a few things that might help if people agree with me or not. After loading every ROM and first booting... I realize that Advanced Task Manager shows the available system memory at less than half and i haven't even started running apps or installing programs yet. Processes start when the phone boots up that are running in background (asleep but still using system resources) and i believe that a fair percentage of them are not required by the everyday user.
Why is Alarm Clock running in the background when i haven't started it? Why is Email client running and i haven't setup exchange or an email account? Messaging, Maps, Settings, Gallery, these all start up without being physically executed and are using resources. If there is a reason for them to be there like not being able to receive SMS if Messaging is not running, then that is fine and im an idiot... But if there is no need to startup the application from boot then why do it?
My second point is when packages are installed as the ROM is flashed, they cant be uninstalled right? well what if we didnt want tethering apps and facebook apps and proprietary software (Moto apps)? Most of these are available in the market and if not, a 5 min Google search will find an .apk on the net anyway.
So how about a bare bones install with the essentials and nothing else? Youtube, Phone, Messaging, Gmail, Contacts, MT Browser , Maps, Calculator... then anything else can be added at the users will.
If anyone else has something to say... or prove me wrong, then please elaborate.
If you delete the alarm clock task, you're likely to sleep into the morning as the alarm won't go off in the morning. Mail is probably needed for pop3 checks (I rely on my google account).
Anyway, I usualy remove the apps widgets from the ZIP which I don't need prior to flashing, resulting in a liter ROM and less processes which I don't need running in background.
I also used to add the apps I use (ASTRO...) directly into /system/app prior to flashing, putting them to the system partition and ... at this point I am not sure.... this leaves one with more space (internal memory) accessible for apps from market (if you do not use APP2SD). I stopped however doing so, as this makes ROM upgrading painful, as you do have to prepare each .zip before flash.
With regard to operating memory you preferably use either Taskkiller or have your systems autokill file optimized as many ROMS do (CursorSense). Refer to http://forum.xda-developers.com/showthread.php?t=622666
I'm not Linux-guy but I know that Android does this to put those programs in memory ASAP because those are usually the first few apps many people use once they boot up their phone -- meaning those apps, when opened, will quickly open and run faster.
Remember, Linux is made to handle all those opened apps and resources; "Unused RAM is useless RAM".
hol17 said:
I'm not Linux-guy but I know that Android does this to put those programs in memory ASAP because those are usually the first few apps many people use once they boot up their phone -- meaning those apps, when opened, will quickly open and run faster.
Remember, Linux is made to handle all those opened apps and resources; "Unused RAM is useless RAM".
Click to expand...
Click to collapse
Correct!!
People have to stop comparing Android (linux) with WinMo devices, they handle RAM completely different.
dione1200 said:
Correct!!
People have to stop comparing Android (linux) with WinMo devices, they handle RAM completely different.
Click to expand...
Click to collapse
Yes, true.... and well known. However Linux was developped for servers and not mobile phones.
Launching an application into free RAM is slower than pulling it from a suspended state. So much is true.
But first having to free up up RAM (by killing tasks) before launching the application into the freed up RAM is certainly slower than launching into free RAM. Probably that's why guys are tweaking around, as noted in the link of my post above. Also in my opinion the default settings are not optimal, that's why I use tweaked settings. Probably that depends on how you use your phone.
But I agree. Coming from Windows the whole LINUX concept is a f**** nightmare. There are thousands of different methods (Cache/Compcache/ Ext Cache)...
So it's good to have an expert around. Could you please enlight me, whether the system partition is of fixed size or size is been alocated during flashing. In other words, is the Userdata size fixed or depending on the system size? .... and is there a benefit of putting apps to the /system/app as they will go to system parition and I will have more internal storage available?
Thanks in advance.
feicher said:
So it's good to have an expert around. Could you please enlight me, whether the system partition is of fixed size or size is been alocated during flashing. In other words, is the Userdata size fixed or depending on the system size? .... and is there a benefit of putting apps to the /system/app as they will go to system parition and I will have more internal storage available?
Thanks in advance.
Click to expand...
Click to collapse
Your Android Device is divided Into partitions -> (Device memory)
1. Hboot (spl)
2. Radio
3. Data (Where u store ur data apps / passwords / other thing)
4. Cache (temp)
5. System (system dump)
6. Boot (to start the device)
so on
Now partition of memory card is important becoz
1. EXT2/3/4 More Amount of Data Section -> if device memory is less
2. Swap -> To fast up the system
*I m Not best Explaination giving person but i think this is understandable language in which i hav written
I think I side with the OP on this one, I would rather have a rom with less in it and then choose to install Amazon MP3 or whatever by myself. Generally speaking you could argue less apps = better performance, as there will be less happening in the background. That said, I have included Facebook in my rom.
However Linux was developped for servers and not mobile phones.
Click to expand...
Click to collapse
Not really true, Linux is just the kernel and can be ported to anything, it's been in embedded small devices for years and not just installed on servers.
ahmgsk said:
Your Android Device is divided Into partitions -> (Device memory)
1. Hboot (spl)
2. Radio
3. Data (Where u store ur data apps / passwords / other thing)
4. Cache (temp)
5. System (system dump)
6. Boot (to start the device)
so on
Now partition of memory card is important becoz
1. EXT2/3/4 More Amount of Data Section -> if device memory is less
2. Swap -> To fast up the system
*I m Not best Explaination giving person but i think this is understandable language in which i hav written
Click to expand...
Click to collapse
Thank you very much for the feedback! Most appreciated and helps me to improve my general understanding . I find it really hard to come by information, that is not absolutely basic or high level expert.
Thanks again.
Ive got a clock radio that wakes me up... if i have lagging issues with my phone what i do is use advanced task manager to stop all processes im not using (alarm clock, maps, messaging, email, etc) and everything runs fast again.
No use in reboot cause all the apps start again from boot.
ive tried removing zips from rom before flashing... when i goto upgrade the update stops cause of the missing files > incomplete manifest. is this cause you have to re-zip and sign it?
You can either do that and re-sign the zip file.
OR you could just close this thread and use adb to remove things that you dont want from the ROMs.
PS. root explorer also works wonders.
feicher said:
If you delete the alarm clock task, you're likely to sleep into the morning as the alarm won't go off in the morning. Mail is probably needed for pop3 checks (I rely on my google account).
Anyway, I usualy remove the apps widgets from the ZIP which I don't need prior to flashing, resulting in a liter ROM and less processes which I don't need running in background.
I also used to add the apps I use (ASTRO...) directly into /system/app prior to flashing, putting them to the system partition and ... at this point I am not sure.... this leaves one with more space (internal memory) accessible for apps from market (if you do not use APP2SD). I stopped however doing so, as this makes ROM upgrading painful, as you do have to prepare each .zip before flash.
With regard to operating memory you preferably use either Taskkiller or have your systems autokill file optimized as many ROMS do (CursorSense). Refer to http://forum.xda-developers.com/showthread.php?t=622666
Click to expand...
Click to collapse
when i try to remove an app from a ZIP it wont let me flash the ROM. even when i replace an app with a newer version renaming it to match the one im replacing, it still doesnt work. what do i have to edit in order to allow the update to go through?
You need to sign the zip file again. Google signing ROMs, its the first thing that pops up.
Apps starting on their own. Anyone have these issues? All the att stuff is gone but the video and music player are open randomly as well as quik office. I can see gmail opening and others tho check for updates
Sent from my SAMSUNG-SGH-I897 using XDA App
Yes, this annoys the hell out of me. I wish I knew what to do to stop it. I've even downloaded an app that's supposed to stop things from auto-starting, and it doesn't even work.
Search for StartUp Auditor on the market, it may work for you.
Android Market description
Startup Auditor is one of the original task killer & task manager apps. It is a boot monitor and startup monitor tool which displays a list of applications you may disable.
Terminate processes automatically.
Enable/Disable apps& keep disabled. No root.
Full version eliminates ads and can disable unlimited!
Package: com.vesperaNovus.app.StartupAuditorFree
Click to expand...
Click to collapse
On the advice of this forum I uninstalled my app killer. From time to time I use Android System to see what's running and I notice a number of apps that start themselves. These include the stock music and video players along with Slacker Radio, AT$T Maps, eBay, GPS Status and a dozen others at various times. I don't mean they remain running after exiting, I mean they start without my having used them. What could be causing these apps to start when not called upon? The ones that use GPS put a significant drain on the battery without me knowing it. Any ideas what triggers these apps? I think I'm going back to using an app killer.
Miami_Son said:
On the advice of this forum I uninstalled my app killer. From time to time I use Android System to see what's running and I notice a number of apps that start themselves. These include the stock music and video players along with Slacker Radio, AT$T Maps, eBay, GPS Status and a dozen others at various times. I don't mean they remain running after exiting, I mean they start without my having used them. What could be causing these apps to start when not called upon? The ones that use GPS put a significant drain on the battery without me knowing it. Any ideas what triggers these apps? I think I'm going back to using an app killer.
Click to expand...
Click to collapse
Mine does the same thing. I have Advanced task killer on mine. I dont use it to kill apps, just to look and see how much ram is free and whats running. But all kinds of junk opens by itself.
so does anyone have a fix for this?
derek4484 said:
Mine does the same thing. I have Advanced task killer on mine. I dont use it to kill apps, just to look and see how much ram is free and whats running. But all kinds of junk opens by itself.
Click to expand...
Click to collapse
What use is free RAM when it is kept free all the time?
AJerman said:
Yes, this annoys the hell out of me. I wish I knew what to do to stop it. I've even downloaded an app that's supposed to stop things from auto-starting, and it doesn't even work.
Click to expand...
Click to collapse
Use "start up auditor". With personal experience, this only slowed down my system.
dieselstation said:
so does anyone have a fix for this?
Click to expand...
Click to collapse
This is now android works. The way I see this, this is very much similar to "pre-fetch" concept in windows 7.
I have a 6 GB RAM laptop. Base OS uses less than 1.5 GB of RAM. But like an hour or so when I see my RAM usage, its to the tune of 3-4 GB. What I have noticed is that my most frequently/recently used apps are loaded to RAM and kept there idle. Some amount of RAM is always kept free instead of using up all RAM. This way apps start faster. When I load a different memory heavy program, it pushes the existing one out and loads this.
More or less the same in android too. When u go to any task manager app n see the running apps, u'll notice that many of the apps loaded are the ones u use frequently.
These apps do NOT use any CPU. They are just loaded to memory and kept there for quick access.
When I boot up my phone I have like 190+ MB free RAM. Though left in standy mode, within an hour, I see my free RAM fall to 80-120 MB range. I never saw it go less than 80 MB. And the apps in memory are the ones I used the last time, and the ones I use all the time.
Even if u use a task killer to kill these "inactive" apps at intervals, they would be loaded again sooner or later. That's the principle of android. So by using task killers, though u feel u r freeing up memory, in fact, u r only draining ur battery. What's the use of memory if u r not using it effectively.
Don't worry abt free RAM amount. Let android manage it. Systems are intelligent enough these days.
Hope this helps.
All well and good, but when I look at what apps are using the most resources, the GPS-enabled apps are almost always near the top of the list even when I haven't been doing anything with them. That tells me that they are stealing processor time even when not being used and that can't be good for battery usage.
Miami_Son said:
All well and good, but when I look at what apps are using the most resources, the GPS-enabled apps are almost always near the top of the list even when I haven't been doing anything with them. That tells me that they are stealing processor time even when not being used and that can't be good for battery usage.
Click to expand...
Click to collapse
Install Spare Parts and see battery usage. Do u see any GPS time under CPU usage. Though my system has all these apps loaded up, they don't show up as using CPU time or any sensors unless I use the app.
Android Memory Management
Android Memory Management
Android is a Linux based OS with 2.6.x kernel, stripped down to handle most tasks pretty well. It uses native open source C libraries that have powered Linux machines for years. All the basic OS operations like I/O, memory management, and so on, are handled by the native stripped-down Linux kernel.
How to use memory for each application
Android’s process and memory management is a little unusual. Like Java and .NET, Android uses its own run time and virtual machine to manage application memory. Unlike either of these frameworks, the Android run time also manages the process lifetimes. Android ensures application responsiveness by stopping and killing processes as necessary to free resources for higher-priority applications.
Each Android application runs in a separate process within its own Dalvik instance, relinquishing all responsibility for memory and process management to the Android run time, which stops and kills processes as necessary to manage resources.
Dalvik and the Android run time sit on top of a Linux kernel that handles low-level hardware interaction including drivers and memory management, while a set of APIs provides access to all of the under- lying services, features, and hardware.
Dalvik Virtual Machine Dalvik is a register-based virtual machine that’s been optimized to ensure that a device can run multiple instances efficiently. It relies on the Linux kernel for threading and low-level memory management.
The Dalvik Virtual Machine
One of the key elements of Android is the Dalvik virtual machine. Rather than use a traditional Java virtual machine (VM) such as Java ME (Java Mobile Edition), Android uses its own custom VM designed to ensure that multiple instances run efficiently on a single device.
The Dalvik VM uses the device’s underlying Linux kernel to handle low-level functionality including security, threading, and process and memory management.
All Android hardware and system service access is managed using Dalvik as a middle tier. By using a VM to host application execution, developers have an abstraction layer that ensures they never have to worry about a particular hardware implementation.
The Dalvik VM executes Dalvik executable files, a format optimized to ensure minimal memory foot- print. The .dex executables are created by transforming Java language compiled classes using the tools supplied within the SDK.
Understanding Application Priority and Process States
The order in which processes are killed to reclaim resources is determined by the priority of the hosted applications. An application’s priority is equal to its highest-priority component.
Where two applications have the same priority, the process that has been at a lower priority longest will be killed first. Process priority is also affected by interprocess dependencies; if an application has a dependency on a Service or Content Provider supplied by a second application, the secondary application will have at least as high a priority as the application it supports.
All Android applications will remain running and in memory until the system needs its resources for other applications.
It’s important to structure your application correctly to ensure that its priority is appropriate for the work it’s doing. If you don’t, your application could be killed while it’s in the middle of something important.
The following list details each of the application states shown in Figure (see the attached image) explaining how the state is determined by the application components comprising it:
Active Processes Active (foreground) processes are those hosting applications with components currently interacting with the user. These are the processes Android is trying to keep responsive by reclaiming resources. There are generally very few of these processes, and they will be killed only as a last resort.
Active processes include:
* Activities in an “active” state; that is, they are in the foreground and responding to user events. You will explore Activity states in greater detail later in this chapter.
* Activities, Services, or Broadcast Receivers that are currently executing an onReceive event handler.
* Services that are executing an onStart, onCreate, or onDestroy event handler.
Visible Processes Visible, but inactive processes are those hosting “visible” Activities. As the name suggests, visible Activities are visible, but they aren’t in the foreground or responding to user events. This happens when an Activity is only partially obscured (by a non-full-screen or transparent Activity). There are generally very few visible processes, and they’ll only be killed in extreme circumstances to allow active processes to continue.
Started Service Processes Processes hosting Services that have been started. Services support ongoing processing that should continue without a visible interface. Because Services don’t interact directly with the user, they receive a slightly lower priority than visible Activities. They are still considered to be foreground processes and won’t be killed unless resources are needed for active or visible processes.
Background Processes Processes hosting Activities that aren’t visible and that don’t have any Services that have been started are considered background processes. There will generally be a large number of background processes that Android will kill using a last-seen-first-killed pat- tern to obtain resources for foreground processes.
Empty Processes To improve overall system performance, Android often retains applications in memory after they have reached the end of their lifetimes. Android maintains this cache to improve the start-up time of applications when they’re re-launched. These processes are rou- tinely killed as required.
How to use memory efficiently
Android manages opened applications which are running in the background, so officially you shouldn’t care about that. This means that it closes the applications when the system needs more memory. However, most android users are not very satisfied with how it does its things because sometimes it leaves too many processes running which causes sluggishness’ in everyday performance. We can use advanced task killer/task manager and it does its job very well.
Source:
Code:
http://mobworld.wordpress.com/2010/07/05/memory-management-in-android/
Still doesn't answer the question of why the OS is starting apps randomly and without need. For instance, I have Titanium Backup and SGS Tools installed, but can't use them because I am not yet rooted. So why do I keep finding both programs started in the process list? Is the app making the call to start? If so, what can I change to stop it? Whatever Android does to manage memory, it just seems stupid for it to load programs that are hardly used without my asking it to.
Miami_Son said:
Still doesn't answer the question of why the OS is starting apps randomly and without need. For instance, I have Titanium Backup and SGS Tools installed, but can't use them because I am not yet rooted. So why do I keep finding both programs started in the process list? Is the app making the call to start? If so, what can I change to stop it? Whatever Android does to manage memory, it just seems stupid for it to load programs that are hardly used without my asking it to.
Click to expand...
Click to collapse
This is how the systems work in most systems, not just in android.
Take the new cars for instance. Power is generally sent to brakes when in low gears, though u r not actively engaging them. Why? Since u r at low speeds, higher probability of applying brakes. And when u do apply, since there is already some power already being sent, its faster to brake.
On similar lines is this principle of loading apps into memory in android too. It won't load apps to fill up ur memory. Plus the apps loaded into memory in this way don't use any CPU time (this means no battery usage). This speeds up system.
On a side note, if u feel u really do not need an app, and dont' want android loading that app, get "startup auditor" from market. Select the apps u don't want to persist in memory and don't want to auto start. Beware of one thing here: if u make an app as 'do not persist' in memory, and that app is started manually or by any other app, it would be killed giving FC in the app that's using it.
Note: I coined the term 'do not persist'. I do not have the app anymore to check for the exact term.
Edit: In windows, this pre-fetching is done through a service. And one can disable that service, though not recommended. I am not sure if this is done through a service that is "stoppable" in android.
Attention!
If you're looking for scripts, here it is:
[CWM][SCRIPTS][TWEAKS] Thunderbolt!
Script Reviews are here:
Script Reviews
Introduction
I've been meaning to share this with the whole Android community. It seems unfair that only the i9000 folks have access to the vast research that I've done so far. Hence I'm sharing this in the general Android thread in hopes that it'll benefit everyone in the long run
A lot of people often asked about how Android really works and the optimizations can be made to their Android to make it perform better in terms of:
- battery life
- raw performance
- GUI smoothness
With Android based on Linux, and with the experience I have with Linux/Unix (I deal with Unix/Linux on a daily basis for my work) I managed to find some tweaks that we can do to optimize our phones.
I also hope that people will experiment with the tweaks posted here and feedback to me if there are certain ways that I'm doing incorrectly or there are better ways than what I've posted here.
Script tweaks
There are a lot of scripts in XDA that specifically uses the optimizations (Linux or Android based) that I will explain below to actually increase performance/battery life. However, you need to know what those scripts do before using it.
I've done some reviews of the script here:
Script Reviews by Pikachu01
There are good scripts that are properly tested and made sure that it works. It's a script by Zacharias maladroit, the kernel dev who makes the platypus kernel for CM7 and snail kernel for i9000. I modded it to be compatible with i9000 and it'll probably do more than some of the scripts I reviewed
From there, I've also taken the liberty of modifying it some more based on my knowledge and research I've done. It's called ThunderBolt!:
LINK
Now onwards to what actually goes on beneath the hood.
Android Governors
Some custom kernels provide customized governors for you to choose. These are currently known:
- Ondemand (CPU scales to the highest frequency immediately after a certain CPU threshold)
- Conservative (CPU scales to the highest or 1 scale lower than that after a certain threshold gradually and scales down if CPU is below a certain threshold
- OndemandX (Has sleep codes that scales down the CPU when device is asleep. Threshold is set to scale slower)
OndemandX
OndemandX has a suspend frequency that it maintains when the phone is sleeping. It does have a known issue of waking up too slowly when a call comes in.
Conservative
Conservative is used when you would like the CPU to scale to the maximum through each frequency slowly before reaching the maximum. It saves more battery at the expense of smoothness. There are a few tunables that can be configured in Conservative governor, namely the freq_step and both the up and down thresholds.
freq_step is a tunable that determines how much frequency (a percentage of the maximum frequency of your device) to increase at each sampling rate. e.g. you have 5 as freq_step and your maximum frequency is 1GHz. At each sampling rate, it'll increase by 50MHz. If your current frequency is 100MHz, it'll increase to 150MHz at the next sampling. What if your device doesn't support 150MHz? This is done by splitting time between 100MHz and 200MHz (assuming your next frequency is 200MHz) so that in average, your phone is performing at 150MHz.
The up_threshold is how much threshold to wait for before increasing the base frequency to the next freq_step. I.e. if it is 80, the CPU will wait until it is 80% loaded before increasing the frequency. The down threshold is the opposite of up_threshold, which is the threshold to wait until the CPU downscales the frequency to the next frequency step.
Ondemand
Ondemand is a governor that basically a simpler version of conservative. One difference is that it scales to the maximum frequency after load has exceeded a threshold. This is usually 80. After that, it looks at the load again when it's at the max frequency. If it's below 80 (example), it'll scale down one frequency step. The frequency step mentioned here is the available frequency step. I.e. if your device has frequencies of 100,200,400,800 to 1GHz then it'll scale down from 1GHZ to 800, 400 then 200 and 100.
One optimization I have for Ondemand for i9000 TalonDev is that I tuned it to be 80 rather than 65. I have not felt any slowdowns from using this setting and it saves a tiny bit more battery. You can experiment on this value however you want, but bear in mind that I'm not responsible for any slowdowns or damage that is caused by it. Read up a bit more on thresholds to know more (Google it).
I/O scheduler tweaks
Another tweak that can be done is the I/O scheduler tweak. Some kernels come with a few standard I/O schedulers while others have extra schedulers that you can choose from. I'll explain the few that I know and if you have more information on the schedulers, you can post it here for everyone to discuss/share
CFQ - Completely fair. Most will come with this, but this is not optimized for android. Some kernels do optimize it and use it.
Noop - Simple and least overhead of all I/O schedulers. Produces the best outcome for some cases and some do swear by it. It's downside is that it is easily bogged down by high I/O transactions.
Deadline - Optimized for mobile-like devices like Android. Also, some do swear by this. If its writes_starve sysfs is tweaked to be fair, it's the same as SIO.
SIO - It's a fair deadline scheduler. It's the best scheduler. Nuff said.
VR - A newer I/O scheduler that places penalty on reverse seeks. Not for flash drives, though the R penalty can be tweaked to be a multiplier of 1. Even if this is done, it has a high overhead.. Read more here: http://www.gelato.unsw.edu.au/IA64wiki/IOScheduling/VRscheduler
BFQ - Too high of an overhead, it is optimized for spindle disks.
You can pick and choose from Voltage Control or other UV/OC apps and apply it immediately to feel the difference.
Here's my assessment of all the schedulers that I know:
SIO> NOOP> Deadline > VR > BFQ > CFQ
NOOP is a simple I/O scheduler without overhead that tries to do each I/O transaction as it comes (FIFO). When a group of transactions is detected, it will try to merge it together to make batches of transactions (makes the whole transaction faster). NOOP doesn't have starvation detection, hence if an I/O transaction takes a painfully long amount of time, it will still continue to do it rather than switch the CPU into doing something else e.g. GUI interrupts (i.e. scrolling lists, flicking screens). All other schedulers also have the "merge" feature. NOOP is the only one that makes the "merge" feature its only feature.
CFQ is a complex I/O scheduler that tries to determine the address space of the transaction and applies a cost algorithm in that if the address is close together, it will group them up and perform them. It also tries to make the transaction incremental (i.e. reading/writing through address incrementally so that the disk spindle needs to wind down the least in conventional platter hard disks) The problem is, our flash devices have very little delta between reading a far reaching address space (than the one currently being written/read) or a closer one as it doesn't rely on spindle/rotations. Hence, having this costing algorithm adds overhead and slows down the overall transaction. CFQ has a lot of algorithms to make sure each process gets a fair slice of time on I/O transcations. Too much overhead.
Deadline has a starvation detector and is simple enough that it doesn't have all the overhead of doing rotational/costing disks algorithm. However, reads are done 2x more likely than writes as it has a algorithm based on weights in that reads must be done first if both a read and a write is detected. It has a 2:1 ratio of read to write weights coded into the scheduler (that can be tweaked - writes_starved, will include it in the next version of system_tweak). Hence it's not a fair scheduler.
SIO is Simple I/O that tries to implement a NOOP type scheduler that has starvation detection. Hence, long I/O transactions will be preempted and given CPU time only after other faster transactions are completed, guaranteeing smoothness. Also, it doesn't have overhead of calculating costs. Also, it has a fair number of writes to read, guaranteeing that all transactions are equal.
BFQ has a lot of algorithms to make sure each process gets a fair slice of bandwidth (Budget Fair Queueing). Too much overhead.
V(R) tries to make sure that each transaction has a weight associated with it, being R. And if the seek is reversed, the R will be multiplied by a penalty making it less likely to be processed. Not for flash drives as reverse seek in a flash drive is just as fast as a forward seek.
In benchmarking tests, the tests normally consists of testing the time it takes for a contiguous I/O transfer from 1 point to another. NOOP excels at that because it won't let itself get interrupted to perform another I/O task. This would mean that in real life testing, NOOP will let a long arduous task to finish while at the expense of UI functionality (you will get UI lags)
SIO on the other hand will perform badly at benchmarking as it gets pre-empted when the contiguous tasks takes more than 0.5secs (for synchronous tasks as benchmarking tools perform a synchronous I/O task from one point to another) to another more important task like UI interrupt (when you're scrolling) or Kernel interrupt (when your kernel needs to perform garbage collecting or swap memory etc)
The 0.5 secs can be tuned in the SIO tuneables though, but I would think 0.5 secs for synchronous tasks and 5 secs for async tasks is pretty good to maintain a balance between long/short tasks enabling a smoother experience in Android.
LowMemoryKiller
The LowMemoryKiller is a constant debate between more free RAM and more multitasking capabilities as free RAM (more than 60MB free) is actually wasted RAM. The fact is SGS only has a puny amount of RAM left after a few big chunks are allocated to the Graphics, Modem, Sound and Video (and some others that I do not know a lot about). With that knowledge, Samsung decided to give the SGS SL a bigger amount of RAM when they released it ( about ~650MB of RAM read that from somewhere, can't remember).
The LowMemoryKiller is actually a feature in the Android OS for better memory management.
You can read about it more here:
http://forum.xda-developers.com/showpost.php?p=5442369&postcount=1
This is an important feature due to the perennial problem of having low free memory causing lagginess and slowness in launching apps. When you have free memory lingering around the number of 40MB or less, the Android OS just lags like hell.
What this would mean is, you would want to tweak the LMK to not have the situation of it having less than 40MB (or even close to that).
The modern Linux machine in the Android ecosystem relies on a mechanism called Low Memory Killer (LMK) to consistently free up RAM. This is due to Android's internal mechanism of caching apps (and never fully exiting them) when you press the back button. This is to enable faster app switching and provide a seamless experience for apps usage model. Android also, by itself will also constantly look for often used apps to cache them for faster app opening. This will happen even before your system fully boots.
Now, when you mention LMK, the most obvious thoughts that come up are minfrees and Out Of Memory (OOM) groupings. Yes, those two are integral parts when it comes to LMK. The issue here is that no one actually mentioned that there are two LMK systems in Android, that being:
- Linux LMK
- Android Dalvik VM LMK
Both are separate entities that kills/removes app/dalvik cache from RAM when RAM reaches a certain level.
What confuses most people (including me) is that the OOM groupings of both mechanisms have the same names being (Android 2.3 based):
LMK App Categories
FOREGROUND_APP: Your apps in the foreground, being used currently, interfacing with the user.
VISIBLE_APP: Visible app that is still viewable by the user, but not interacting with the user, example could be apps that reside in the status bar, and giving live information i.e. Os monitor graphs.
PERCEPTIBLE_APP: App that the user can still perceive i.e. Music app that is playing music.
HEAVY_WEIGHT_APP: RAM heavy apps that are not being interacted with, but will be a pain to load if its information is cleared in the memory.
SECONDARY_SERVER: App that acts as a secondary server. Not sure what this really means, but client-server thingy but secondary? Could be something that syncs with an app, but is not syncing currently?
BACKUP_APP: App that is currently making a backup (like Titanium backup?)
HOME_APP: Your launcher.
HIDDEN_APP: An exited app that still significant residual memory in the RAM. Exited only some time ago, or is constantly used by the user.
EMPTY_APP: An exited app that only has small remnants of residual memory in RAM. App that is exited quite some time ago.
Take note that this is my understanding of the OOM groupings. If there are mistakes, please correct me
Tuning LMK
Both the LMK and Dalvik VM has this groupings and they can be tunable by using prop and lowmemorykiller/parameters/adj respectively.
One errata about this is that the Dalvik LMK has one extra tunable which is CONTENT_PROVIDER, which affects widgets that are not currently refreshing or displaying information. The ADJ (which I will explain later) is not available to be tuned though. And it is clearly missing from the Linux LMK.
The ADJ is the so-called priority of the app categories, with 0 being the highest (should not be killed) and 15 being the lowest (should be the first to be killed). I haven't seen values other than 0-15 but I have a feeling that the numbers are arbitrary, and can be extended up to 65535, but who would want to do that since they're only 9 app categories
LMK are specified by pages, with memory = 4 * pages.
e.g.
4096 pages represents 16MB RAM (16 *1.024)
From the Linux LMK source code, the parameters can be adjusted as such:
Code:
* For example, write "0,8" to /sys/module/lowmemorykiller/parameters/adj and
* "1024,4096" to /sys/module/lowmemorykiller/parameters/minfree to kill processes
* with a oom_adj value of 8 or higher when the free memory drops below 4096 pages
* and kill processes with a oom_adj value of 0 or higher when the free memory
* drops below 1024 pages.
Take note that the oom_adj so far takes only 6 values (although I am confident that it can take more than that). Haven't experimented enough to validate that, but I shouldn't need to as assigning different oom_adj should be a given for all the app categories. This is to make the LMK intelligent enough to determine which app to kill first rather than grouping different app categories into one priority, in which a lot of popular LMK settings that devs provide do that.
And for the Dalvik LMK, my understanding is that it will remove all dalvik caches of <INSERT_APP_CATEGORY_ADJ> or higher if free RAM/pages reaches the level specified.
Hence, two things are learned here:
- Linux LMK will kill an app category of that particular RAM level or higher if free RAM drops below a certain level e.g.
Hidden_APP has a minfree of 16384 pages. If pages fall below this level, Empty_App will be killed first, free pages is recalculated, then if RAM is still below that, Hidden_Apps will be killed next.
- Dalvik LMK will clear all dalvik caches in RAM of all the App category if free RAM reaches the level e.g.
SECONDARY_SERVER_MEM is 8192 pages. When RAM reaches 8192 pages, all SECONDARY_SERVER caches are cleared.
Best Practices
Now, how do we tweak both so that it is efficient in doing its job, i.e. LMK.
From my experiments I can safely say that:
- Below 60MB free RAM you will feel lag 5-10% of the time.
- Below 52MB free RAM you will feel lag 5-15% of the time.
- Below 46MB free RAM you will feel lag 15-20% of the time.
- Below 42MB free RAM you will feel lag 20-50% of the time.
- Below 36MB free RAM you will feel lag 100% of the time.
The experience of lag increases exponentially from 60 down to 36MB. Hence, the LMK should be tweaked against that.
Also, it is considered best practices to group oom_adj settings of Linux LMK with the Dalvik LMK as when free RAM drops to that level, both the Linux LMK and Dalvik LMK will work together to free up at both sides.
The 6 oom_adj that can be tuned for Linux minfree also need to be tuned correctly, failing this would make the LMK too aggressive/under aggressive in mid levels (even if your EMPTY_APP is aggressive, if your oom_adj is selected incorrectly, you'll get to a point where memory leak occurs).
Talon LMK - This is obsolete, TalonDev is using ThunderBolt! LMK from 0.5.x
Take for example, the Talon default LMK setting where:
LMK_ADJ is 0,1,2,3,7,15.
It's EMPTY_APP is 15
It's HIDDEN_APP is 5
It's HEAVYWEIGHT, SECONDARY and BACKUP is 3
Last three values of MINFREE is 4096, 9216, 12288.
Where's the issue here? If you noticed it congrats.
If you didn't, here's the issue:
The minfree 12288 is tied to EMPTY_APP which has a OOM_ADJ of 15.
The minfree 9216 is tied to nothing but has a OOM_ADJ of 7
The minfree 4096 is tied to HEAVYWEIGHT,SECONDARY and BACKUP with the OOM_ADJ of 3.
The LMK_ADJ is important as it represents the checkpoints where the Linux kernel will check for minfrees and kill them. Here's a situation:
Memory is below 12288 pages, OOM is triggered and Linux looks for EMPTY_APP to kill and kills all of them. Job done. Memory is back to >12288 pages.
After a few days, memory is below 12288 pages again, OOM kills all EMPTY_APP, but free memory is still lower than that. It sees if the next checkpoint is fulfilled, which is 9216 pages, and yes memory is still below 9216 pages. It looks for an OOM_ADJ of 7 and above to kill, but fails to find any as 7 and above OOM_ADJ only consists of EMPTY_APP.
Here comes the quandary. The next checkpoint is 4096. We're not ever going to reach that as you'll need memory to fall below 16MB, and at that time it'll be super sluggish to even support GUI. Hence BAM, memory leak or lag or something.
Supercharger LMK 512HP - Multitasking
Let me just paste this to avoid a lot of retyping
Code:
FOREGROUND_APP_ADJ=0
VISIBLE_APP_ADJ=4
PERCEPTIBLE_APP_ADJ=2
HEAVY_WEIGHT_APP_ADJ=5
SECONDARY_SERVER_ADJ=6
BACKUP_APP_ADJ=7
HOME_APP_ADJ=1
HIDDEN_APP_MIN_ADJ=8
EMPTY_APP_ADJ=15
FOREGROUND_APP_MEM=1536
VISIBLE_APP_MEM=3072
PERCEPTIBLE_APP_MEM=1024
HEAVY_WEIGHT_APP_MEM=10240
SECONDARY_SERVER_MEM=10240
BACKUP_APP_MEM=15360
HOME_APP_MEM=1024
HIDDEN_APP_MEM=15360
EMPTY_APP_MEM=25600
LMK_ADJ="0,4,6,8,14,15"
LMK_MINFREE="1536,3072,10240,15360,20480,25600"
A few things here:
It's EMPTY_APP and HIDDEN_APP is too large, 80 and 100MB respectively, but the 80MB is a false flag hence it doesn't really matter if you tweak that and you can get a very minor degree of multitasking because of this false flag. I'll get to that in a moment.
It's PERCEPTIBLE_APP has higher OOM_ADJ than VISIBLE_APP (odd selection, but RAM rarely goes below 40MB before it starts lagging like hell) People would reboot their phones when it gets to that, but it rarely gets to that for this configuration as LMK is pretty aggressive here.
Now, it's LMK_ADJ is "0,4,6,8,14,15"
15 is EMPTY_APP,
14 is nothing
8 is HIDDEN_APP - it looks like it'll only kill HIDDEN_APPs when it reaches 60MB, with HIDDEN_APPs being the heavy hitter of apps as most apps will end up in this category. Hence why multitasking is still doable (to an extent) but an aggressive LMK means taking a call or switching to a browser will kill your game (losing saves). Inconvenient, but it works for some people who haven't played a long game without saving and never took a call in between. Imagine the horror LOL!
Juwe's RAM Script
Code:
if [ -e /sys/module/lowmemorykiller/parameters/adj ]; then
echo "0,1,2,4,6,15" > /sys/module/lowmemorykiller/parameters/adj
fi
if [ -e /sys/module/lowmemorykiller/parameters/minfree ]; then
echo "2560,4096,5632,10240,11776,14848" > /sys/module/lowmemorykiller/parameters/minfree
fi
No OOM_ADJ are given, hence the stock ones are in effect here.
From init.rc, this is the stock 2.3.5 OOM_ADJ.
FOREGROUND 0
VISIBLE, PERCEPTIBLE 1
HEAVY_WEIGHT, SECONDARY_SERVER, BACKUP 2
HOME 4
HIDDEN_APP 7
EMPTY_APP 15
This one also suffers from the false flag of the 2nd last minfree not being used. It will start killing HIDDEN_APP at 40MB, which is usable (you'll only get lags 15-20% of the time, and most probably after long usage). Other than that, it's pretty stable.
Misc
There are a few known scripts that tweaks the LMK as well as the app priority (yes, the priority of the process categories can be changed too by using setprop.)
Supercharger is one of them:
http://forum.xda-developers.com/showthread.php?t=991276
Take note that Talon doesn't permit Supercharger to be used (it will override it when booting through boot scripts) as Talon itself uses its own LMK settings that are optimized for ZRAM/Swap.
Kickasskernel script is also included in Supercharger. The script will clash with Zach's script in ThunderBolt! too as both of them tries to set different values for the same settings.
Note that Zach's script doesn't tweak LMK settings as i9000 Talon doesn't permit it. With that, if you're using Zach's scripts, you would need to find other ways of tweaking the LMK. Using Auto Memory Manager is good. You can download Supercharger, look at the presets and apply it using Auto Memory Manager. The preset is the only thing important here
LMK App Category Priorities
Using setprop, there is a way to set the priorities of the category. Since, app categories are different, I can only offer one recommendation: The HOME_APP_ADJ.
This prop setting sets how high of a priority your launcher is. A 1 or 2 is sufficient to make sure your launcher doesn't get killed. However, do you really want it to be on 1 or 2. Based on the previous article (LMK/OOM), you would know that OOM priorities are comparatively based on one another. Setting it 1 or 2 in stock 2.3.5 values would basically override PERCEPTIBLE/VISIBLE app priorities. Do you want your launcher to be killed only after your keyboard/music player to be killed? You decide
Journaling/Barriers
This has been a touchy subject here in XDA for most people who debate about it. Most recently, the SAS/Acid tweaks included a way to disable journaling on these partitions:
/system (System is read only, it's safe to remove journaling. However, you will not see speed increase by removing it as you're not writing onto /system 99.99% of the time unless you're using Titanium backup to remove system apps or copying init.d scripts to it)
/cache (Cache can be rebuilt on the fly. Data corruption on it is not game breaking)
/data (All of your data on your phone is here. Removing journaling can risk data corruption. Read more below)
On whether we need journaling or not, I will pose this situation:
Journaling is required to maintain data consistency in events that could lead to data corruption. Data could get corrupted in a number of situations:
- Misbehaving app that constantly writes without syncing/committing data to the disk
- Power loss due to forced reboots or bootloops when data is partially written/committed into disk
Android will normally buffer its writes before committing to the disk. This way, data stays in the RAM before data would only get written into the disk after a period of time. Take note that we as Android enthusiasts seek ways to better our phone to get the most out of it through experimental and sometimes wacky ways. Take OC/UV for example. When we OCed or UVed too much, sometimes our phone will get deadsleep or constant reboots. This leads to corruption if journaling is not turned on. However, we can get corruption too even with journaling but the risk is smaller.
Mount options too, play a role in whether journaling can be safe/unsafe. Most lagfixes will place a "barrier=1" in the /data to make sure that journals are actually committed before an out-of-order write is done. With that, the risk is greatly reduced at the cost of performance. You could experiment by removing barriers and see if it works for you. No guarantees on data if you remove barriers! More info on ext4/barriers:
http://lwn.net/Articles/283161/
http://kernelnewbies.org/Ext4#head-25c0a1275a571f7332fa196d4437c38e79f39f63
If you do disable journaling, make sure to manually run an fsck on your disk. Make sure the fsck binaries are in your binary paths (echo $PATH to show your binary paths). Don't ask how to do this if you're unable to do fsck. It is your own risk that you are disabling journaling/barriers.
One experiencing I would like to relate is this:
I've personally disabled ext4 journaling on my device. After experiencing a sudden power loss (forced reboot), my apps started FCing left and right and device became unstable that it sometimes rebooted when the device is sleeping. In other words, it's a lost case. Reflashing was the only way to restore my phone. This happened 1 day after disabling my journals.
Also note that ext2 is basically an ext4 without journaling. However, ext4 is updated all the time while ext2 source is not updated for quite some time (2 or 3 years?). Most optimizations are already in ext4 while ext2 even without journaling is slower than ext4 without journaling. A comparison of performance:
No journal:
ext4 > ext2
With Journal:
ext2 (can't do journaling in ext2) > ext4 > ext3
Update: Attached is a JournalingOn.zip for i9000 only that you can flash to rejournal your partitions (Only for i9000 devices!). Also attached is Acid Tweaksv6 - Removed Useless Stuff that will disable journaling only for /system and /cache, if anyone is interested. You will need to rejournal first before unjournaling. To be sure that /system and /cache has no journaling and your mounts are applied correctly, type "mount" without the quotes.
You'll be able to see that /data will have:
Code:
barrier=1, data=ordered
/system and /cache will have:
Code:
barrier=0, data=writeback
To be really sure, you can check by using tune2fs included in both the JournalingOn.zip and Acid Tweaksv6 - Removed Useless Stuff.zip.
Copy the tune2fs to /tmp and set rwxr-xr-x as its permissions using Root Explorer/File Expert
Code:
/tmp/tune2fs -l /dev/block/mmcblk0p2 | grep features
You can substitute the /dev/block/mmcblk0p2 (which is /data) to /dev/block/stl10 (/dbdata) /dev/block/stl9 (/system) /dev/block/stl11 (/cache) to check each partition.
Continued in Post #2 - Lack of space
Still posting ...
Undervolting/Overclocking
This is also another touchy subject, but more towards people complaining about phones that get forced reboots or bootloops. OCing is done to get better performance on your android. WARNING: Ocing will cause more wear and tear on your CPU and will reduce your CPU lifetime. How much it will reduce it? No one knows. It depends on how sturdy your CPU is, and that cannot be measured by any means. It could be reducing months out of year e.g. 6months out of 3 years? No one actually knows.
UVing on the other hand reduces battery drain especially when you UV your 100MHz as 90% of the time in normal usage, your CPU will idle at 100MHz. How much can it save? No one actually did a benchmark on it. Save to say, UVing does not harm your CPU, it will actually extend the life of it by reducing wear and tear Oh, bootloops do corrupt data if you're without journaling
Always remember to clear your UV/OC settings before flashing another kernel!
Take note that the ability to OC/UV depend on your phone. Some SGS can UV better or OC better and some might not.
On how to UV:
1. You can UV more on the lower frequencies than the higher frequencies.
2. Do a UV for
100MHz with (950-150=800) - Take note that 950 is the base value for the voltage. Some kernels might have changed this value to 925 or 900. This is why clearing your OC/UV settings is important as the OC/UV settings will only load the voltage difference, which in this case -150. If the base voltage is 900 and you do a -150 for it, you'll end up with 750 which might cause bootloops.
3. Limit the max frequency to 100MHz. WARNING: Doing this will significantly slow down your system to a crawl, but it is still running. This is done only to test the stability of your system. You can skip this if you want, but you would need to find a way to stress test your CPU at the 100MHz value (and other values as well by limiting the max frequency). Failing in testing this will result in deadsleeping (as your phone is in 100MHz when you sleep and if you undervolt too much, you deadsleep)
3. Test with RockPlayer (or MX Video Player) and play an RMVB file using software decoding for 10mins. If your settings is unstable, Android will freeze up and reboot. (this is just an example, but I find that this way stresses the CPU to the max. I fail to find another way of stressing it as even playing NFS is stable on some voltages but fails when I play rockplayer. Hence rockplayer is the best way so far)
4. After doing this, remove the limit and move on with the next few frequencies.
The rest of the voltages, just rinse and repeat.
200MHz with (950-100 = 850)
400MHz with (1050-75 = 975)
800MHz with (1200-25= 1175)
1000MHz with (no change)
You can experiment with lesser values than these if you get freezes/bootloops as this is by no means a one size fits all value. Some might think this UV is overly aggressive or some might think that this UV value is too underwhelming.
Mine if you're interested:
Code:
100MHz = -200 (950 base)
200MHz = -150 (950 base)
400MHz = -125 (1050 base)
800MHz = -75 (1200 base)
1000MHz = -50 (1275 base)
As for OCing, you would do the same to test the stability. Just that you will be enabling a higher clock for your L0 frequency (L0 is the highest frequency). Some kernels might add more frequency states i.e. from the 5 states above to 6. You can try using Tegrak OC app too. It all depends on your own experimentation.
Memory Leaks
If you found out that your Android is laggy after sometime and a reboot will make it faster, then you're experiencing memory leaks. "free" is a command to show your currently free memory. It will not necessarily be the same value as your phone's free memory (as it takes into account swaps and buffers)
1. Go to terminal emulator (or download the script at Post #3)
Code:
su <ENTER>
free <ENTER>
This will show your buffers and free memory.
My example:
Code:
total used free shared buffers
Mem: 348548 340280 8268 0 1320
Swap: 163836 47292 116544
Total: 512384 387572 124812
Then do this:
Code:
sync <ENTER>
echo 3 > /proc/sys/vm/drop_caches <ENTER>
Type:
Code:
free<ENTER>
And you'll realize that the buffers will reduce and free memory will increase, like this example:
Code:
total used free shared buffers
Mem: 348548 305212 43336 0 88
Swap: 163836 46804 117032
Total: 512384 352016 160368
That means it's done.
Click to expand...
Click to collapse
If this solves the laggginess problem, it's due to vfs_cache_pressure being too low and dirty_ratio being too high. However, in most cases, you don't have to do this as Linux will manage to clear the RAM in a timely manner as long as there are no apps that constantly hold on to the memory (not releasing it). Hence, only do this when necessary (lagginess and such).
With that, I've updated the system_tweak yet again, with this knowledge by making the VM settings more aggressive, which will trigger the memory leak faster. I made this so, as there's a workaround to clear the caches when it gets full.
Having a low vfs_cache_pressure and a high dirty_ratio will save battery and make your device perform faster at the cost of higher RAM usage.
You can actually automate it by using Script Manager by making a script, pasting the instructions and making a widget out of it. To learn more, read the android market description or the in-help guide in the app.
Some commonly asked questions about drop_caches that were answered by me from flolep:
http://forum.xda-developers.com/showpost.php?p=17727859&postcount=3428
http://forum.xda-developers.com/showpost.php?p=17731938&postcount=3446
http://forum.xda-developers.com/showpost.php?p=17736421&postcount=3450
Checking for Battery Drain/ Saving battery
Starting from Gingerbread, you can't really look for Partial wakelocks anymore to determine what is draining your battery at night. BetterBatteryStats is a way to check for that. It also can provide the process that drains your battery most of the time.
A high partial wakelock usually says that something is waking up your phone at night. With that, checking partial wakelock, you can see if these are the culprit:
-Network Location
-Location Manager Service
If so, disable Lattitude or any location check-in app that you're using. It's checking you in every few minutes that drains the battery.
Disabling Wifi/3G/HSDPA saves battery too. I find that Wifi/Edge drains the least amount of battery when idling. To switch to EDGE as opposed to HSDPA/3G.
Go to:
Code:
Settings > Wireless > Mobile Networks > Network Mode > GSM Only
Disabling "Use Wireless Networks" in location and security would save a little bit of battery too I guess. It's in:
Code:
Settings > Locations and Security > Use wireless networks
CSC
CSC is a folder that defines your APN settings and country/region specific configurations aside from Language/Time that is configured in build.prop(also for Samsung Apps).
Having to manually set your APN is annoying right?
Your product code being KOR by default is another annoying thing to deal with. How do I resolve this?
1. Get your CSC from your own country here: http://forum.xda-developers.com/showthread.php?t=948790
2. Extract from the zip file /system/csc/<folder> (where <folder> is your Country's CSC)
3. Use Root Explorer/File Expert to copy it to the phone's /system/csc/.
4. Use nitrality to change your product code. WARNING: This will wipe all your data.
Odexed vs Deodexed
I've been using Deodexed ROMs since I first flash my custom ROMs. One gripe I have is that I seem to have lost the stock smoothness/speed that I experience when I'm using a stock ROM. I found out that deodexing actually extracts the customization files back to the APK (using APK manager) and Odexed files are actually done to optimize the speed of those files itself.
The only reason ROM chefs are using deodexed files is that it is easier to theme. There is no need to decompile the odex files and make the changes as all the theme files are in the APK themselves.
Only other reason that I know of is the overscroll glow. Odexed files can't support it. However, that is a small price to pay for speed/smoothness that I am willing to sacrifice.
Busybox
Busybox is required to perform all of your superuser activities in your android phone. There are some problems associated with this when ROM developers decide to use a certain version of Busybox that are incompatible with the binaries that we use in our phones. When you found out that after changing your kernel/ROM:
- You still have superuser and Titanium Backup still works but,
- Root Explorer/File Expert can't copy files correctly to the /system after mounting rw
- Voltage Control/ Pimp My CPU/ Control Freak/ SetCPU cannot create the Svolt_scheduler file in /etc/init.d
You have a clashing busybox version issue. This can be remedied by:
- Installing the busybox installer from the market
- Installing busybox 1.17.x into /xbin
- Removing all other busybox binaries from your binary path. (This is optional if you found out that after installing busybox, everything works. Try it first before removing) To know your binary path, do an
Code:
echo $PATH
Nandroid Backup
Nandroid Backup is done in recovery (VolumeUp + Home + Power). In recovery, choose Backup And Restore. What it does is it backs up your:
- Boot
- System
- Data
- Cache
When you restore, you should preferably use Advanced restore and restore the /system and /data. All others can be built from scratch (except /boot). If you're converting from CM7 to Samsung Roms, it's best to start from scratch either ways. Restoring nandroid from any is considered dangerous and not supported by most cases. Best is to use Titanium Backup to backup as it can be restored both ways (only restore app though, restore data sometimes can lead to FCs)
Voodoo Color
Voodoo Color is also a point of contention among users. Some like it and some hate it with a passion. If your kernel does support it, here's a way to tweak it:
1. Tweak the color profiles. Can't see a difference, hence I would just pick Voodoo most of the time.
2. Tweak the gamma first to a comfortable value. I.e. drag the 3 sliders together to a comfortable gamma level. After that, try looking in the gradient (the gray slab below the slider) and try to find for neutral gray values. Adjust each slider until you find a neutral gray (i.e. left and right until you see that slab is truly gray)
3. Tweak the RGB multiplier until you have a comfortable level (3 sliders together). Then tweak it the same by referring to the gradient until you're comfortable.
4. Switch off your display for a while and look at your surroundings to reset your eyes. Then turn on your screen again and see if its too green/yellow/blue etc. Adjust the gamma/RGB multiplier until you're satisfied.
This might take a few rounds of testing, but in the end you'll be truly satisfied with it
i9000 Specific
i9000 Kernels - In Depth
You will need to have a stock odexed (why odexed? read a few passages below)ROM with root. You can easily do this by flashing stock XXJVS 2.3.5 and then flashing a custom kernel through Odin like:
- Dark Core - Uses Voodoo initramfs
- CF-root - Uses CF-root initramfs
- Semaphore - Uses CF-root initramfs
- Midnight - Uses Speedmod initramfs
- Galaxian - Uses CF-root initramfs
- Voodoo - Uses Voodoo initramfs
You can flash the custom kernel through Odin by placing the tar file into the PDA section and pressing Start. At this point, do not convert your partitions to ext4 (lagfix) unless you're using a Voodoo based initramfs as Voodoo based initramfs will convert your partitions to ext4 automatically at first boot.
After flashing a custom kernel through Odin, you will gain root. From there, you can choose to use any other custom kernels that use CWM to install or stick with the custom kernel that you've flashed through Odin like:
- TalonDev - Uses Voodoo initramfs
- TalonSH - Uses Voodoo initramfs
There are other kernels as well, but since I did not stress test them, I will not include them here. This thread lists the full repertoire of custom kernels that you can choose from:
http://forum.xda-developers.com/showthread.php?t=1196704
On which custom kernel to pick, you're advised to look at the individual threads original post as well as the last 5 pages of the thread to know about its stability and features. I pick TalonDev myself as it has compache as well as the latest upstream (kernel updates from the Linux OS as well as the official Android Open Source Project/AOSP and optimizations) patches.
At this point, you should decide if you would want to convert to ext4 (lagfix) or not. Your phone by default will reside in an RFS system. The RFS at this point in time is pretty stable and its performance can easily match ext4. Too bad Quadrant scores are bad for RFS, but who cares about Quadrant anyways. I myself choose to use ext4 as the TalonDev kernel has a lot of ext4 upstream patches that optimizes the usage of ext4 on Android.
If you're using Voodoo initramfs, you'll be converted to ext4 automatically.
If you're using CF-root initramfs, you will require the CF-root ext4 app to convert that can be found here:
http://forum.xda-developers.com/showpost.php?p=12651371&postcount=7
Note that on some kernels that use CF-root initramfs, the ext4 app will warn you that you're not on CF-root type kernels. Ignore this warning. You'll be able to convert to ext4 anyways unless you're on Galaxian. Certain Galaxian versions (since the kernels are not versioned, I can't tell which versions are not working) will bar you from converting to ext4 using the ext4 app. If that is the case, just flash CF-root, convert to ext4 and then flash back to Galaxian.
Also not that CF-root based initramfs kernels will remove your bootsounds on first boot. Don't be alarmed by this.
In terms of which lagfix is better (Voodoo vs CF-root lagfix), I can't tell at this point in time. Both are equally good. Since TalonDev uses the Voodoo initramfs, I am inclined to use the same when I am using the same kernel. That should be a good yardstick to follow.
BIGMEM/ Non-BIGMEM:
Some kernel developers released BIGMEM versions of their kernel:
-TalonDev BIGMEM
-TalonSH BIGMEM
-Semaphore bm version
The only difference a bigmem kernel can bring is more memory at the expense of 720p video recording. Playing a 720p video is still possible. Recording it is not on a bigmem kernel. Bigmem kernels normally have 13MB more than their non-bigmem counterparts. How this 13MB could affect you? It depends on the apps you use and how aggressive your LowMemoryKiller is.
XXJVT System apps
System apps can be frozen/removed to make your system faster/less battery draining.
In order to remove/freeze system apps, you would need Titanium Backup (Pro) or System Tuner.
The choice whether we want to remove or freeze the system apps should now be decided. Titanium Backup Pro and System Tuner uses a built in Android mechanism to freeze your system apps - PM disable. PM disable will actually disable the app from the system itself. It will not drain the battery or even launch itself. It is a safer way so to speak. Removing will physically delete the APK app from the system freeing up disk space. If you decide to remove, make sure to have a backup before proceeding. Even if you have a backup, restoring from Titanium Backup may sometime fail. Hence, do this only if you're sure you want to remove that certain app. Freezing an app will make it easier to defrost without any consequences.
Here are the apps that I disabled:
Code:
AngryGPS > Used to manually configure GPS. Not needed if GPS works OK.
BluetoothTest
Buddies Now
lcdtest
screencapture > Screen capture app
Daily Briefing
Days
EncryptApp
Factory Test
Gallery -> I use QuickPic as a replacement. It's faster than Gallery.
Google Partner Setup
Google Search > Google search widget
Home screen tips
HTML Viewer
Market Feedback Agent
Market Updater > I froze this to avoid being updated to Market 3.0. It currently sucks now
Mobile tracker > Mobile tracker, only if you use it, then keep it
Mobile tracker settings > Mobile tracker, only if you use it, then keep it
Perso
PhoneSetupWizard > Required if you're first booting/installing the ROM, not needed afterwards
PopupuiReceiver
Press Reader
Print via Bluetooth
RoseEuKor
Samsung Account
Samsung Keypad -> I use Swype beta instead
Self Test Mode
Service mode
SimDetachNotifier > Will notify you if you detached your sim card. Seems pointless
SNS > Only necessary to sync your contacts if you're using Social hub to do so or syncing calendar from Facebook
SNSAccount > Same as above
Social Hub
Software Update
Synchronise
TwLauncher -> Using Go Launcher Ex instead
wipeoutreceiver > Wipe if phone is stolen. Not needed if you don't use the Samsung service
WlanTest
wssyncmlnps
There are others that you can remove as well. I'm hoping to gather feedback on the other APKs that can be removed and what will be affected by its removal. Please post if you have more information
Disabling Lagfix
Most ROMs would advice you to disable lagfix before flashing a new ROM. Although this is not necessary in most cases (as all Gingerbread kernels support ext4 from the get-go) moving from one kernel to another might need it (to optimize the initramfs it is used on). Hence if you're swapping from one kernel (that has a different initramfs) to another. It is adviseable to undo the lagfix.
CF's lagfix can be undone by using the ext4 app.
Voodoo lagfix can be undone by using the Voodoo Control App.
Speedmod lagfix can be undone by using it's recovery.
Odexed ROM
You might say, hey if I'm using stock, I'll be missing extended power menu, battery percentage and all other theme mods that are not in the stock ROM. Fear not, you can get all these back including Gtalk2, CRT screen off etc here:
$omator's stock+
Gtalk2
There are odexed themes too, in which the developers would decompile the odex files into APKs, make the changes then compile the APKs into odex files again. It is more tedious than deodexed themes but odexing generally makes the phone smoother
You can find the themes here:
http://forum.xda-developers.com/forumdisplay.php?f=666
WIFI Issue on JVR/JVS/JVT
You might be facing some wifi connectivity issues on JVR/JVS/JVT ROMs i.e. it disconnects every few seconds.
This is due to some changes in the ROM that doesn't permit a "Forever" lease time from your modem. Change your lease time to an hour (or two) or if you don't have that option, remove your address reservation option from your modem (Yes, it's a modem only tweak, nothing you can do on your phone)
Memory Freak
Memory Freak is an app in TalonDev that tweaks the ZRAM/Swap and LMK settings. There's a golden rule to the ZRAM/Swap that is:
Swapping to and from the ZRAM will be slower if the amount of ZRAM is more than 50% of the total usable memory, . Hence, I set mine to 160MB (from the 340 total usable RAM). Swappiness in most Linux machines is 60. This means, the system will be inclined to swap in/out more than just using the RAM. I set mine to 50 as a personal preference so that there would be 50% chance that the system will use swap or RAM.
HiddenApp is the amount of free memory to be when LMK starts looking for hiddenapps to kill. I set mine to 52 for my multitasking preference. Experiment with it to see how much multitasking vs smoothness you're looking for. My preference will change in time. Check my signature for the most updated ones.
WIP:
This is obviously a work in progress article that will undergo changes as the Android scene improves. I will spend time to make edits and additions to the post whenever I see fit. Sometimes I will post changelogs at the end of the thread, sometimes I won't if I only make minor edits. Just be sure to check back often to see any new tips offered
Credits:
I claim no credit on these findings except for the ones that I've researched on my own. However, all of it is not of original work since I form my own opinions after reading a lot. I also don't claim that the settings I posted are the most optimal there is. It might not even work well for most of you guys. The key here is tweaking it to your own perfection as there is no setting that will cater to everyone's needs. Hence the real people who deserve credits are:
* All the devs' kernel/apps that I posted here
* XDA forum
* Google
Attention!
If you're looking for scripts, here it is:
[CWM][SCRIPTS][TWEAKS] Thunderbolt!
Script Reviews are here:
Script Reviews
GUIDE UPDATES
UPDATE 1/10/12
* Edited some sections with some new knowledge that I have. A few places, can't name them all Check them out and see if you can spot them.
UPDATE 10/19/11
* Moved around some guides and grouped them together
* Updated some guides to make them more Android centric (as opposed to i9000 centric)
UPDATE 10/18/2011 - Added an indepth article about LMK/OOM. CTRL-F for LMK.
UPDATE 10/12/2011
* Added some comments on the apps removed/frozen so that you can make an informed choice of why the apps can be removed/frozen.
* Added my analysis of most schedulers that I know of and why they are good/bad.
* Moved the scripts to ThunderBolt!: LINK
* Added a guide on the wifi issue for JVR/JVS/JVT
UPDATE 10/1/2011 - Moved the script reviews to this post LINK
UPDATE 9/23/2011:
* Removed the JournalingOn.zip and Acid Tweaksv6 - Removed Useless Stuff.zip because people might use it for devices other than the i9000. This is because in other devices such as SGS4G, the /dev/block/stl10 is /data while in i9000 it's /dbdata. The JournalingOn.zip will not enable journaling on /dbdata without some editing. Hence, I removed it as I don't have a time to put up a warning or notice, and didn't have time to edit the script inside the zip file.
* Readded an edited JournalingOn.zip and readded the Acid Tweaksv6 - Removed Useless Stuff.zip (did not change) to the first post. Only for i9000 devices!!
* Added another way to check if you're journaled or unjournaled. Check the guide below.
* Added some more explanation on common scripts I found.
* Moved the guides around due to lack of space. I added some minor changes to most of the guides. See if you can detect them
Reserved - 10 char
Reserved - 10 char again ...
Changed the name to get more attention. Surprised that no one gave a comment about it
Appreciate any comments or criticisms or correction of what I've written here.
Thanks!
Excellent Read!
Thank you very much!
carnagecjb said:
Excellent Read!
Thank you very much!
Click to expand...
Click to collapse
Glad you enjoyed it
this is actually really helpful. half the time I really didnt know what I was messing around with thanks.
Dataslycer said:
this is actually really helpful. half the time I really didnt know what I was messing around with thanks.
Click to expand...
Click to collapse
Cool Glad that you liked it.
Hope this gets more attention as I would want it to be beneficial to people who uses tweaks and to know how to use them
Nice read Mate. I used to wonder about I/O schedulers and which one to use etc. I like to add build.prop scripts and int.d scripts. This thread just helped me to understand more about it.
Thanks!
read every word, thank you very much. I'll be talking to my kernel dev about BIGRAM... on miui our 720p recording doesn't work right anyways. Thanks again for the great info.
Sent from my MIUI SCH-i500
ShyamSasi said:
Nice read Mate. I used to wonder about I/O schedulers and which one to use etc. I like to add build.prop scripts and int.d scripts. This thread just helped me to understand more about it.
Thanks!
Click to expand...
Click to collapse
Thank you for reading. Hope the knowledge comes in handy when you tweak your phone
sageDieu said:
read every word, thank you very much. I'll be talking to my kernel dev about BIGRAM... on miui our 720p recording doesn't work right anyways. Thanks again for the great info.
Sent from my MIUI SCH-i500
Click to expand...
Click to collapse
Wow, great! Hope you enjoy reading it as much as I enjoyed writing it
having trouble with your thunderbolt tweak. I downloaded the file for I9000 placed it on my phone (SD card) mounted my system and data in recovery mode then proceeded to install the thunderbolt zip file. Using script manager i found the bolt_scripts file and ticked the run at boot button. Then i hit the run button and got 14 permission denied messages asking are you root. Do i have to run this as root. Might be a dumb question but im very new to trying this. Thanks
kevnac said:
having trouble with your thunderbolt tweak. I downloaded the file for I9000 placed it on my phone (SD card) mounted my system and data in recovery mode then proceeded to install the thunderbolt zip file. Using script manager i found the bolt_scripts file and ticked the run at boot button. Then i hit the run button and got 14 permission denied messages asking are you root. Do i have to run this as root. Might be a dumb question but im very new to trying this. Thanks
Click to expand...
Click to collapse
Yes, you need to run everything as root . Scripts in init.d would already be running as root. You just need to make sure Script Manager is running as root for the remount script
Excellent thread, very well written, had a quick skim read, but I'm going to take some time tonight to absorb it all ;-).
Thank you!
zeekiz said:
Excellent thread, very well written, had a quick skim read, but I'm going to take some time tonight to absorb it all ;-).
Thank you!
Click to expand...
Click to collapse
Cool Thank you for reading such a long article
Let me know if you have any questions or if you think I made a mistake
superb thread !
testomat said:
superb thread !
Click to expand...
Click to collapse
Thank you for reading
testomat said:
superb thread !
Click to expand...
Click to collapse
+1
10 chars
rom-g said:
+1
10 chars
Click to expand...
Click to collapse
Thank you for the support