Related
Hello,
I apologize if this has been covered... It seems that our forum is unable to search for 3 character terms, specifically "192" and "ram".
The G1 is listed as having 192 megs of memory. Yet, running "free" from my rooted RC33(JF1.41) shows:
Code:
$ su
# free
total used free shared buffers
Mem: 99040 96292 2748 0 316
Swap: 0 0 0
Total: 99040 96292 2748
#
So, where's my other 96 megs?! If -all- this extra ram is being used for 3d (god forbid) I'd gladly tweak a setting to reclaim more ram for RUNNING APPS, versus bling bling.
Anyone have any ideas here?
mystica555 said:
Hello,
I apologize if this has been covered... It seems that our forum is unable to search for 3 character terms, specifically "192" and "ram".
The G1 is listed as having 192 megs of memory. Yet, running "free" from my rooted RC33(JF1.41) shows:
Code:
$ su
# free
total used free shared buffers
Mem: 99040 96292 2748 0 316
Swap: 0 0 0
Total: 99040 96292 2748
#
So, where's my other 96 megs?! If -all- this extra ram is being used for 3d (god forbid) I'd gladly tweak a setting to reclaim more ram for RUNNING APPS, versus bling bling.
Anyone have any ideas here?
Click to expand...
Click to collapse
http://forum.xda-developers.com/showthread.php?p=3262422
mystica555 said:
Code:
$ su
# free
total used free shared buffers
Mem: 99040 96292 2748 0 316
Swap: 0 0 0
Total: 99040 96292 2748
#
Click to expand...
Click to collapse
And ofcourse I end up reading the reply to my post -after- i write this one. Edited as I'm an idiot.
It still seems to me that 192 megs is A; misleading and B; clearly NOT enough to run this OS, along with radio "firmware" (why cant the radio have its own bloody discrete chip!?)
I presume this huge gaping usage of ram is why the Sprint Touch Pro shows 288 megs of ram; its really 384 with the rest being used by the phone's radio and framebuffer. Gah.
I wonder, will new Android hardware overcome this paltry 96 meg limit?
mystica555 said:
And ofcourse I end up reading the reply to my post -after- i write this one. Edited as I'm an idiot.
It still seems to me that 192 megs is A; misleading and B; clearly NOT enough to run this OS, along with radio "firmware" (why cant the radio have its own bloody discrete chip!?)
I presume this huge gaping usage of ram is why the Sprint Touch Pro shows 288 megs of ram; its really 384 with the rest being used by the phone's radio and framebuffer. Gah.
I wonder, will new Android hardware overcome this paltry 96 meg limit?
Click to expand...
Click to collapse
indeed .. but that would be why rooting the G1 is so nice .. move files over to a linux EXT2 partition on the SD card and viola!! ultimate space
i have 1.5G dedicated to the G1 .. keeps the phone free and clear
Flash is not SDRAM
LucidREM said:
indeed .. but that would be why rooting the G1 is so nice .. move files over to a linux EXT2 partition on the SD card and viola!! ultimate space
i have 1.5G dedicated to the G1 .. keeps the phone free and clear
Click to expand...
Click to collapse
I could care less about the total amount of installed apps; I've got plenty of free yaffs2 flash for what I end up running...I simply don't want my apps to get oom-killed.
I simply wish to run, ALL CONCURRENTLY, (cpu speed be damned!)
Connectbot
HelloAIM!
PicPush
StreamFurious or Last.fm
Gmail
AND the browser
without one or more (or the home screen) forcibly being killed due to out of memory errors.
Load Browser, 2 or 3 of those always end up dying.
A normal commute for me on the bus: I'm chatting on AIM. I've got a irssi session open on my VPS in Connectbot. Last.fm playing some futurepop. I tap the notification bar to read the new email I have, which contains an HTML link. Click the link...Boom, music stops playing. Close browser, homescreen reloads slowly. Go back to connectbot, well, im disconnected.
RAM = Godly.
Let's try this...
Code:
# cd /sdcard
# dd if=/dev/zero of=swap.fil bs=1M count=128
128+0 records in
128+0 records out
# mkswap ./swap.fil
Setting up swapspace version 1, size = 134213632 bytes
# swapon ./swap.fil
# free
total used free shared buffers
Mem: 99040 97112 1928 0 556
Swap: 131064 0 131064
Total: 230104 97112 132992
#
mystica555 said:
Code:
# cd /sdcard
# dd if=/dev/zero of=swap.fil bs=1M count=128
128+0 records in
128+0 records out
# mkswap ./swap.fil
Setting up swapspace version 1, size = 134213632 bytes
# swapon ./swap.fil
# free
total used free shared buffers
Mem: 99040 97112 1928 0 556
Swap: 131064 0 131064
Total: 230104 97112 132992
#
Click to expand...
Click to collapse
What does that do? Does it create a swap file on the sd card?
andonnguyen said:
What does that do? Does it create a swap file on the sd card?
Click to expand...
Click to collapse
Yessir
mystica555 said:
Code:
# cd /sdcard
# dd if=/dev/zero of=swap.fil bs=1M count=128
128+0 records in
128+0 records out
# mkswap ./swap.fil
Setting up swapspace version 1, size = 134213632 bytes
# swapon ./swap.fil
# free
total used free shared buffers
Mem: 99040 97112 1928 0 556
Swap: 131064 0 131064
Total: 230104 97112 132992
#
Click to expand...
Click to collapse
i'm slightly hesitant to try .. anyone? good / bad / indifferent???
What exactly will this do and will a reboot cancel this?
widto08 said:
What exactly will this do and will a reboot cancel this?
Click to expand...
Click to collapse
even if a reboot didn't cancel it .. the next update.zip would .. unless something was built into it
yeah a reboot will cancel... how can I make this permanent? What file can I put the swapon command into? I just tried init.rc, but that didn't work.
PROOF OF CONCEPT ONLY! Serious risks involved, still working on this!
As a test, it works. however:
This has a few(lot of?) problems!
Biggest: your sd card cant be mounted as a disk unless you stop swapping. or reboot.
Next: your sd card will die sooner rather than later, as you are doing what it never was intended to do (writing moving data back and forth when its really good just for storing files that may change. sometime.)
Finally, things can start to run slow...VERY slow...
Biggest issue, after my busride with last.fm never stopping, browsing, all the stuff that kept making me angry before. Then i got home and tried to make a phonecall... The dialer worked *reasonably* responsive.. All of about 10s to bring it up. However, MAKING the call took...an eternity.
Hit the green button to dial...phone starts ringing, friend answers, we start talking.. I look at the phone.. black screen...ITS WAITING FOR THE ANDROID PIC AND CALL TIMER THING TO COME UP...40s after i started the call, got a message saying that one process was not responding, force or wait. at this point, when *that* dialog came out of swap, lo and behold the process actually not responding was working fine behind said dialog...
So yeah. its got its benefits, but very large downfalls.
As you *WILL* destroy your flash after an unspecified amount of time, *do not* try to put this on your internal flash. Not in /cache for example. Flash cards are cheap; replacing the g1 because you nuked 64 megs of flash ram is not...
I will test a few methods of separating the flash card from the vfat filesystem (mountable via usb_mass_storage) and report back. Probably partitioning the card, one as swap and such.
ON the other hand, I may also try partitioning 128m as fat16, to make certain the small card's wear levelling scheme is able to do whatever it can to help mitigate the destruction-by-swapping.
Fat32 may not have the same wear levelling mechanisms at the card level on anything less than an SDHC, and guaranteed ext2 wont. Raw swap won't either... As far as I remember, flash wear levelling on sdcards is based entirely on how FAT allocates blocks. (someone knowledgeable about the SD/SDHC and wear levelling feel free to chime in!)
I'll post more later when I try a few more things!
I really hope this turns into a sound solution...and then I can die happy.
Or live with my G1 happily...yea thats better.
mystica555 said:
As a test, it works. however:
This has a few(lot of?) problems!
Biggest: your sd card cant be mounted as a disk unless you stop swapping. or reboot.
Click to expand...
Click to collapse
Which is why I put it in /system/sd/swap/swap.fil
mystica555 said:
Next: your sd card will die sooner rather than later, as you are doing what it never was intended to do (writing moving data back and forth when its really good just for storing files that may change. sometime.)
Click to expand...
Click to collapse
Not an issue for me. By the time it wears out, I will have upgraded to a newer faster card anyway.
mystica555 said:
Finally, things can start to run slow...VERY slow...
Biggest issue, after my busride with last.fm never stopping, browsing, all the stuff that kept making me angry before. Then i got home and tried to make a phonecall... The dialer worked *reasonably* responsive.. All of about 10s to bring it up. However, MAKING the call took...an eternity.
Hit the green button to dial...phone starts ringing, friend answers, we start talking.. I look at the phone.. black screen...ITS WAITING FOR THE ANDROID PIC AND CALL TIMER THING TO COME UP...40s after i started the call, got a message saying that one process was not responding, force or wait. at this point, when *that* dialog came out of swap, lo and behold the process actually not responding was working fine behind said dialog...
So yeah. its got its benefits, but very large downfalls.
Click to expand...
Click to collapse
Could this be due to the speed of your SD card? I am already noticing a marked performance increase.
I tell you what, if you can give me an idea of where to make this permanent, I am willing to run this card to destruction. I have another 8G class 6 card ready to go, and can swap out easily at any time. All I have to do is make regular backups on my laptop and I should be good to go. (And even then data loss is not really an issue for me anyway.)
Wear levelling..Different than I thought.
mystica555 said:
ON the other hand, I may also try partitioning 128m as fat16, to make certain the small card's wear levelling scheme is able to do whatever it can to help mitigate the destruction-by-swapping.
Fat32 may not have the same wear levelling mechanisms at the card level on anything less than an SDHC, and guaranteed ext2 wont. Raw swap won't either... As far as I remember, flash wear levelling on sdcards is based entirely on how FAT allocates blocks. (someone knowledgeable about the SD/SDHC and wear levelling feel free to chime in!)
Click to expand...
Click to collapse
Ok, I read a bit into this myself and it seems even back in 2003 wear leveling is done at a very low level: CHS/LBA blocks, versus FAT Blocks.
Sandisk Whitepaper from 2003
I would expect that this sort of wear-leveling is in effect on any of their current cards, including the microsdhc the G1 came with.
As such, it seems swap/ext2 won't have much if any difference compared to FAT filesystems. Its recommended to mount the ext2 with option "noatime" however, so that metadata won't be written for -every single read-.
t1n0m3n said:
Which is why I put it in /system/sd/swap/swap.fil
Click to expand...
Click to collapse
It seems I haven't read enough about where you can put files.. Is this /system/sd normal for a G1, or is this after you went and moved apps/caches to the SD card?
t1n0m3n said:
Not an issue for me. By the time it wears out, I will have upgraded to a newer faster card anyway.
Click to expand...
Click to collapse
Indeed! This is mainly a warning to not try and use the phone's internal flash.
t1n0m3n said:
Could this be due to the speed of your SD card? I am already noticing a marked performance increase.
Click to expand...
Click to collapse
I am 90% certain that it is the speed of the card. Note: I am still using the 1gb card that was installed initially, sadly this phone broke me. Payday friday though means a trip to microcenter
That, or you haven't run 64 megs of apps out of main ram into swap. I had a -lot- of junk running in the background, in addition to the laundry list of apps enumerated earlier in the thread. I think it was: connectbot, helloaim, browser with 4 windows, terminal, alarmclock, calendar (twice! wtf) tmo myfaves switcher, last.fm, maps, voice dialer (twice as well?) google talk messaging klaxon and about 2 or 3 others I cant remember. At peak usage, I had 68 megs swap used and all the ram used as well.
There should be a rather quick and dirty way of getting this into an init script. I'll mess with it at lunch tomorrow.
Right now, sleep is calling and this is one call I'm not forwarding to voicemail!
mystica555 said:
It seems I haven't read enough about where you can put files.. Is this /system/sd normal for a G1, or is this after you went and moved apps/caches to the SD card?
Click to expand...
Click to collapse
/system/sd is my ext2 partition on my SD card that I moved my apps/caches to.
Has anybody tried compcache yet? Its compressed swap and it is ideal for this device, if it can be made to run.
http://code.google.com/p/compcache/
Hello again all. I have been using Fatal1ty's Hero builds for a while now which are fantastic Thank you again Fatal1ty. But no matter what i seem to set swapper to it makes the phone run really slow. When I actually disable swapper is when this build runs the smoothest for me. Was wondering what settings everyone here uses for swapper to get the performance increase from it.
currently is set to the following
/sdcard/swapfile.swp
65mb
20 swapiness
i have tried 20 60 and 100 swapiness with upto 256 mb and really didn't seem to help. just wondering what settings that everyone else uses to have it run more speedy. Also I was wondering if it is possible to put to system memory like in the space for applications although I think that may be a bad idea depending on type of memory as will more then likely make that memory fail with all the read / write cycles
I've always run compcache on my ROMs when possible and I recently had the idea that setting the full amount of RAM to compcache could be an interesting test. The theory being that it may be slightly slower overall to have ramzswap compressing pages, but minfree would have heaps of memory to play with and apps would never quit, so multitasking would be faster as no apps are writing out to /data and then having to relaunch when I switch to them again.
I've set compcache with the init script like so
Code:
insmod /system/lib/compcache/ramzswap.ko;
rzscontrol /dev/block/ramzswap0 -i -d 98304;
busybox swapon /dev/block/ramzswap0;
And Linux reports the swap is working
Code:
# free
total used free shared buffers
Mem: 107332 105972 1360 0 8
Swap: 98296 48940 49356
Total: 205628 154912 50716
But no matter what I do, even if I launch a heap of apps, I can never get swap usage above the ~48Mb RAM seen here.
Furthermore, apps like Auto Killer and Free Memory only report I have 102Mb RAM total.
What's going on here? Why does Android think it only has 100Mb when free is reporting near on 200Mb total? Why does compcache only fill up halfway and no more? Does this mean that ~48Mb RAM is the "sweet spot" for compcache and any more is pointless?
The phone does feel faster when switching between apps but I could just be imagining it. Out of interest this is a 32B Magic running Dwang 1.17.1, which is basically just Donut AOSP with a faster framework and kernel.
Any ideas/help/suggestions would be appreciated?
Swap is never, ever, regarded by the system as real memory, hence why the system won't report it, since you really only have ~100 mb physical ram.
As to why you're only using 48, I believe that 48 is 1/2 96, so it probably means pages are being compressed 2:1, so the full memory is being compressed and dumped into RAM, and it only occupies 48 MB, leaving you 48 free for other processes.
Again, remember comcapche is swap, not real memory.
jubeh said:
Swap is never, ever, regarded by the system as real memory, hence why the system won't report it, since you really only have ~100 mb physical ram.
As to why you're only using 48, I believe that 48 is 1/2 96, so it probably means pages are being compressed 2:1, so the full memory is being compressed and dumped into RAM, and it only occupies 48 MB, leaving you 48 free for other processes.
Again, remember comcapche is swap, not real memory.
Click to expand...
Click to collapse
I don't think that that analysis is quite right....
First off, you MUST maintain SOME amount of real memory available... otherwise it'll crash in a spectacular way. I believe that the linux kernel itself may have a safety feature that maintains a certain minimum amount of physical ram available. There ARE certain things that the linux kernel will not be willing to swap, such as ITSELF.
Just imagine what would happen if the kernel swapped itself..... any attempt to do this wouldn't end well. Especially if it tried to swap its ENTIRE self since the kernel MUST be in memory in order for it to run.
There is also the swappiness setting... it controls the system's tendency to swap.
And finally, there is the possibility that you may simply not be starting enough processes to consume the full memory!
So imagine this; you have your compcache set for a certain size. It grows to that size and then finally, the kernel says "screw you, you can't have any more memory!" blows an error back to compcache, which complains back to the kernel "sorry, swap is screwed." Yep.... the kernel tells compcache which tells the kernel rather than the kernel just knowing.
You definitely don't want this happening.
Note: I can forsee some serious stability problems that this could result in related to the low memory process killer. Specifically, your compcache grows to its maximum allowed size, you start an application, the low memory process killer figures that you've got plenty of memory available, doesn't kill anything off, tries to start some application, crashes spectacularly when the kernel complains back that it doesn't have any memory. I don't know if this would happen with a stock low memory process killer, but definitely would with the swap hacks added....
lbcoder said:
Note: I can forsee some serious stability problems that this could result in related to the low memory process killer. Specifically, your compcache grows to its maximum allowed size, you start an application, the low memory process killer figures that you've got plenty of memory available, doesn't kill anything off, tries to start some application, crashes spectacularly when the kernel complains back that it doesn't have any memory. I don't know if this would happen with a stock low memory process killer, but definitely would with the swap hacks added....
Click to expand...
Click to collapse
92 MB of compcache doesn't really need 92MB of compcache... that's the point of being compcache.
Compcache file in RAM grows when cache gets stuffed inside compcache.
Setting a low swappiness will cause compcache to just swap what's needed.
And even with full compcache, in the end you end up having around 140 mb (or so) free ram. 92mb of compcache that takes like 50mb and 42 extra mb of normal ram.
I think this idea is great (I was just too lazy to try yet...). Instead of dalvik vm having to free up memory it can stuff some more mb in compcache. should be faster.
I didn't really think my post through... but I hope some of you understood some of the things I wanted to communicate xD
domenukk said:
92 MB of compcache doesn't really need 92MB of compcache... that's the point of being compcache.
Compcache file in RAM grows when cache gets stuffed inside compcache.
Setting a low swappiness will cause compcache to just swap what's needed.
And even with full compcache, in the end you end up having around 140 mb (or so) free ram. 92mb of compcache that takes like 50mb and 42 extra mb of normal ram.
I think this idea is great (I was just too lazy to try yet...). Instead of dalvik vm having to free up memory it can stuff some more mb in compcache. should be faster.
I didn't really think my post through... but I hope some of you understood some of the things I wanted to communicate xD
Click to expand...
Click to collapse
Interesting in theory, but if you actually read what I said, you would note that this is entirely IMPOSSIBLE and would crash spectacularly if not for and in some cases in SPITE of certain safety features built into the kernel.
Note: If you have 70 MB worth of data that CAN'T be swapped, that leaves 20 MB ***PEAK*** available to compcache.
It is neither fair nor sensible to think of all memory as being equal. Running processes ***MUST*** have REAL MEMORY.
A little off-topic, but this discussion (the possibility of REAL "compressed" memory) sparked a thought/question:
Would/could KSM* bring any benefit to Android? (Not sure if the KSM module can even compile/work on ARM)
I know KSM is normally used for detecting and sharing duplicate pages among KVM guests, but I wonder how many pages in a typical running Android installation are duplicated, and thus candidates for sharing/de-duplication.
*I can't posts links yet, so those that don't know what KSM is, will just have to google for it.
lbcoder said:
There ARE certain things that the linux kernel will not be willing to swap, such as ITSELF.
Click to expand...
Click to collapse
I was under the impression the kernel keeps itself in RAM and then reports free memory to the rest of the OS. This is why the phone physically has 192Mb RAM, but only reports 96Mb free (or 107Mb with RAM hack). Perhaps my understanding of Linux/Android memory reporting is not correct?
lbcoder said:
It is neither fair nor sensible to think of all memory as being equal. Running processes ***MUST*** have REAL MEMORY.
Click to expand...
Click to collapse
I think this is likely what is happening. Home, Phone, System and other processes with a low oom are refusing to swap out as they are still running. If the compcache allocation in RAM is dynamic as domenukk says, then those processes are occupying enough RAM that the ramzswap allocation can only grow to ~48Mb as I am seeing. I didn't think of this.
Nor have I tweaked swappiness. It's currently set to 60 (default) so I'd assume it's not too fussy with paging out. I will try playing with this at 10 and 100 to see if I can force anything more into swap or if it's less willing to swap.
brainbone said:
*I can't posts links yet, so those that don't know what KSM is, will just have to google for it.
Click to expand...
Click to collapse
I do not know either, but here are some links people may wish to look at
http://fedoraproject.org/wiki/Features/KSM
http://lwn.net/Articles/306704/
http://lwn.net/Articles/330589/
http://www.linux-kvm.com/content/using-ksm-kernel-samepage-merging-kvm
Ok I am not at all experienced in this area but this is just a suggestion. You say that you can only get 48mb of swap to be compressed at a time. If those 48mb were uncompressed, then that would occupy the 98mb you alloted to it. If you set the cc at say 128mb, then in (my) theory up to 64mb of it the actual ram would be used. I don't know how much sense I'm making but not sure exactly how to explain it. If you don't get it I'll try explaining my logic a little more in depth.
mejorguille said:
If you set the cc at say 128mb, then in (my) theory up to 64mb of it the actual ram would be used.
Click to expand...
Click to collapse
It appears you are right. Even with swappiness set to 100 and opening every app on my phone I'm not able to fill more than half of 128Mb compcache before minfree stats gracefully closing processes:
Code:
/opt/android-sdk-linux_86/tools$ ./adb shell free
total used free shared buffers
Mem: 107332 105956 1376 0 32
Swap: 131064 65520 65544
Total: 238396 171476 66920
Super Jamie said:
It appears you are right. Even with swappiness set to 100 and opening every app on my phone I'm not able to fill more than half of 128Mb compcache before minfree stats gracefully closing processes:
Code:
/opt/android-sdk-linux_86/tools$ ./adb shell free
total used free shared buffers
Mem: 107332 105956 1376 0 32
Swap: 131064 65520 65544
Total: 238396 171476 66920
Click to expand...
Click to collapse
I rock=p
So what's your performance like, compared to say 32mb cc or no cc at all?
It's different but I like it.
There is sometimes a slight (<2 second) pause when launching a new app (I assume this is compcache compressing old pages to swap to make way for the new app) however once the app is up and running, it almost never "exits" so switching between previously-launched apps is noticeably faster than without compcache. I run HelixLauncher Donut and it's never closed and re-launched while I've been trying this, however it did sometimes with 32Mb cc and quite often without cc at all.
I had 5 day uptime last week with 96Mb compcache (rebooted whilst testing another app for a friend) so I don't think stability is an issue. The CM wiki indicates performance with cc is better upon boot then gradually declines, even if that is the case, rebooting my phone once a week is no big issue.
Unless I run into any major issues, I'll be keeping my phone with large compcache
Super Jamie said:
I was under the impression the kernel keeps itself in RAM and then reports free memory to the rest of the OS. This is why the phone physically has 192Mb RAM, but only reports 96Mb free (or 107Mb with RAM hack). Perhaps my understanding of Linux/Android memory reporting is not correct?
I think this is likely what is happening. Home, Phone, System and other processes with a low oom are refusing to swap out as they are still running. If the compcache allocation in RAM is dynamic as domenukk says, then those processes are occupying enough RAM that the ramzswap allocation can only grow to ~48Mb as I am seeing. I didn't think of this.
Nor have I tweaked swappiness. It's currently set to 60 (default) so I'd assume it's not too fussy with paging out. I will try playing with this at 10 and 100 to see if I can force anything more into swap or if it's less willing to swap.
I do not know either, but here are some links people may wish to look at
http://fedoraproject.org/wiki/Features/KSM
http://lwn.net/Articles/306704/
http://lwn.net/Articles/330589/
http://www.linux-kvm.com/content/using-ksm-kernel-samepage-merging-kvm
Click to expand...
Click to collapse
ksm sound cool. As every app runs inside dalvik vm. Not sure though... somebody should ask cyanogen
I am happy thatlarge compcache works out so well for you.
BTW doesn't cyanogenmod 5 count the whole swap as real memory since test4 or so?
Oh and overclocking ondemand to as high as possible will speed up app opening and switching a lot while on compcache
domenukk said:
doesn't cyanogenmod 5 count the whole swap as real memory since test4 or so?
Click to expand...
Click to collapse
It's counted as available "swap" memory, but not "real" ram. Pages stored in "real" ram (memory that the cpu can directly execute code in) still need to be freed up (moved to swap) before previously swapped pages can be moved back in to "real" memory to be executed.
domenukk said:
ksm sound cool.
Click to expand...
Click to collapse
The beauty of KSM is that it does no "swapping". It simply combines 4KB pages that are identical -- so instead of two identical 4KB pages using 8KB of ram, they only take 4KB. The code is executed in place.
KSM would, however, still require swap. If at any time a virtual shared 4KB page is written to, it needs to be copied to a free page to avoid corrupting the virtual page it was a duplicate of before the write. Swap is needed to ensure that there will always be enough available free pages when this happens.
domenukk said:
As every app runs inside dalvik vm.
Click to expand...
Click to collapse
KSM is not dependent on a VM, but the existence of VMs (java or otherwise) increases the likelihood of duplicate pages.
domenukk said:
Not sure though... somebody should ask cyanogen
Click to expand...
Click to collapse
I'd certainly be interested in cyanogen's thoughts on this, but I'm sure there are others that would be able to chime in as well.
Relevant excerpt from kernel.org/doc/ols/2009/ols2009-pages-19-28.pdf
KSM and embedded
KSM is suitable to be run on embedded systems too; the important thing is not to register in KSM regions that won’t likely have equal pages. For each virtual page scanned, KSM has to allocate some rmap_item and tree_item, so while these allocations are fairly small, they can be noticeable if lots of virtual areas are scanned for no good.
Furthermore, these KSM internal rmap/tree data structures are not allocated in high memory. To avoid early out of memory conditions, it is especially important to limit the amount of lowmem allocated on highmem 32bit systems that might have more than 4GB of memory, but these shouldn’t fit in the embedded category in the first place.
Click to expand...
Click to collapse
Super Jamie said:
I was under the impression the kernel keeps itself in RAM and then reports free memory to the rest of the OS. This is why the phone physically has 192Mb RAM, but only reports 96Mb free (or 107Mb with RAM hack). Perhaps my understanding of Linux/Android memory reporting is not correct?
Click to expand...
Click to collapse
Memory reporting is a tricky thing.
But in general, when RAM is allocated to HARDWARE, it is NOT REPORTED.
The chunk of the 192 that is not reported is assigned PRIMARILY to the RADIO. Another chunk is assigned to the GPU. Still more is assigned to the AUDIO HARDWARE.
The part used by the kernel itself IS reported. The most trivial empirical evidence to prove this to you is that when you change KERNELS, it doesn't affect the total system memory, despite your KNOWING that different kernels use different amounts of RAM.
Another bit of empirical proof is that activating COMPCACHE does not reduce the total physical ram reported. And yes, COMPCACHE is part of the kernel...
Now here's another concept of crash and burn: IF everything in memory could be cached in compcache, then what would stop compcache from caching compcache in a horrible infinite memory sucking loop? That would be very very bad, LOL.
I think this is likely what is happening. Home, Phone, System and other processes with a low oom are refusing to swap out as they are still running. If the compcache allocation in RAM is dynamic as domenukk says, then those processes are occupying enough RAM that the ramzswap allocation can only grow to ~48Mb as I am seeing. I didn't think of this.
Click to expand...
Click to collapse
NOW you're getting the idea! Running processes, kernel, etc., all need physical RAM (though actually those processes you mention most definitely CAN be swapped...), and therefore you CAN'T make the ENTIRE RAM into compcache!
Nor have I tweaked swappiness. It's currently set to 60 (default) so I'd assume it's not too fussy with paging out. I will try playing with this at 10 and 100 to see if I can force anything more into swap or if it's less willing to swap.
Click to expand...
Click to collapse
Just beware of possible crash-and-burn
Super Jamie said:
It appears you are right. Even with swappiness set to 100 and opening every app on my phone I'm not able to fill more than half of 128Mb compcache before minfree stats gracefully closing processes:
Code:
/opt/android-sdk-linux_86/tools$ ./adb shell free
total used free shared buffers
Mem: 107332 105956 1376 0 32
Swap: 131064 65520 65544
Total: 238396 171476 66920
Click to expand...
Click to collapse
That doesn't actually follow from what you've posted here.
What follows is that 105956-(65520/2)=73196 of what occupies your memory can't be swapped (kernel, running processes, etc.).
lbcoder said:
What follows is that 105956-(65520/2)=73196 of what occupies your memory can't be swapped (kernel, running processes, etc.).
Click to expand...
Click to collapse
Do you know why swap constantly "settles" at almost exactly half usage regardless of what size compcache I set?
For example, I set 64Mb compcache yesterday and rebooted. Just using my phone normally (browser, genie widget, music) I have this:
Code:
total used free shared buffers
Swap: 63992 32096 31896
Does this mean I am effectively reducing the amount of RAM the phone has for the kernel, "foreground app", "visible app" and "secondary servers" (to use the minfree terms), whilst at the same time allowing more "hidden app" and lower processes to swap out instead of terminating gracefully?
This disturbs me
-------------------------------------
Sent via the XDA Tapatalk App
brainbone said:
I'd certainly be interested in cyanogen's thoughts on this, but I'm sure there are others that would be able to chime in as well.
Click to expand...
Click to collapse
He doesn't know much about it, yet. But he seems inerested.
Attached a short conversation over #twitter
# Dominik domenuk
@cyanogen Is ksm any good 2 save RAM? probably not - you would have already done it http://lwn.net/Articles/329123/
# Steve Kondik cyanogen
@domenuk I don't know too much about it, I think its meant for sharing between distinct virtual machines
@domenuk it could have a lot of potential though
# Dominik domenuk
@cyanogen basically yes. But he states its also for normal apps. I have no idea to what extend android apps have similar memory, though...
# Steve Kondik cyanogen
@domenuk a lot, Android is all about IPC and shared memory. I wouldn't be surprised if the Dalvik people are thinking about it.
Click to expand...
Click to collapse
Here is a way to make Android use more than 50 percent of a swap partition. I am not sure how it will act with compcache... Should be the same... Run the following commands from a terminal or add them following to your userinit.sh file:
Code:
su
echo 80 > /proc/sys/vm/swappiness
echo 150 > /proc/sys/vm/vfs_cache_pressure
!!WARNING!! - Messing with VM settings can cause data loss and system instability... Not liable for damages...
Using the above the "free" output is:
Code:
total used free shared buffers
Mem: 97708 95168 2540 0 356
Swap: 125296 88756 36540
Total: 223004 183924 39080
While we are at it... If anyone is willing... they can try this as well:
echo 1 > /proc/sys/vm/oom_kill_allocating_task
Reference : Article to Linux Insight
Been having pretty good results with it...
Here is a link to the rest of the vm settings...
Linux Insight article listing vm settings
L8r
Hi! I try xdandroid rom from his official site.
Apart little things maybe fixed on other rom that i've found here, there's a method to increase the available RAM? Because I've a Raph100 and it use only 50/60 Mb (I saw that in the "running applications" menu).
Thanks!
TheXeno said:
Hi! I try xdandroid rom from his official site.
Apart little things maybe fixed on other rom that i've found here, there's a method to increase the available RAM? Because I've a Raph100 and it use only 50/60 Mb (I saw that in the "running applications" menu).
Thanks!
Click to expand...
Click to collapse
Well obviously to physically get more RAM you'd have to get the soldering iron out and be pretty brave.
To manage memory better, read my Speed Improvements thread. There's a topic that covers memory management - you can get as detailed as you want with it, or just use the SuperCharger script. Either way, depends on how much you want to learn it .
I don't understand very well, but i don't think this will change the physical ram used. I mean, if it runs under/with win mobile, maybe we can change one parameter to release more memory. Like the page pool setting: maybe applying this directly to haret.exe
But the problem is, winmo is shutdown completely; if I'm not mistaken. You know, the whole reason why you have to reboot your phone in order to get winmo back. Usually apps that run under winmo, you just shut down and either run cleanRAM to regain the memory or you could reboot at that time but you do it while winmo is still running in either case.
TheXeno said:
I don't understand very well, but i don't think this will change the physical ram used. I mean, if it runs under/with win mobile, maybe we can change one parameter to release more memory. Like the page pool setting: maybe applying this directly to haret.exe
Click to expand...
Click to collapse
You don't understand, WinMo is completely kicked out of memory - no trace of it is left. Many have done extensive tests, WinMo ROM speed has no effect on Android speed - why would it? It's not running "under" WinMo. HaRET is not a 'virtual machine' or an emulator. We're actually running Linux on the hardware - about all WinMo does for us is initialize said hardware.
Understand. And ok... but the question remains: my raph has 288Mb of RAM. Why Android use only 60? (in application menu: (plus or minus) "xx Mb used of 60 total")
TheXeno said:
Understand. And ok... but the question remains: my raph has 288Mb of RAM. Why Android use only 60? (in application menu: (plus or minus) "xx Mb used of 60 total")
Click to expand...
Click to collapse
Well 288mb is the total amount. In WinMo, you don't have that much total because of various things like video etc. Lots are beyond me.
Perhaps RAPH doesn't have all the banks available? I doubt it. Have you checked with free? In the terminal.
My RHOD says 171mb total, when I also have 288mb physically. I would think it would show more than 60 tho... Perhaps you've done something wrong? Using the wrong startup perhaps...?
You say you have a RAPH100, that isn't one of the crippled models.
So it's not only my diam100 who can not use all the RAM! I'm relieved!
I knew that even with 192MB, 64MB were blocked for VRAM (I think)
Winmo 6.5 shows a total of 111.66Mb
In settings, xdandroid 2.3.7 (compiled by me) shows only 30Mb, with free 0Mb
using the command FREE in the ADB, however, shows a total of 103Mb, with 1 or 2 MB free
In these days I'm doing some tests with compcache off, 24Mb (1 / 4 of RAM) or more
using Compcache the maximum memory is increased, but free is always 1 or 2Mb!
In the next days I will try with FRX07.1, we will see
Fabiett83 said:
So it's not only my diam100 who can not use all the RAM! I'm relieved!
I knew that even with 192MB, 64MB were blocked for VRAM (I think)
Winmo 6.5 shows a total of 111.66Mb
In settings, xdandroid 2.3.7 (compiled by me) shows only 30Mb, with free 0Mb
using the command FREE in the ADB, however, shows a total of 103Mb, with 1 or 2 MB free
In these days I'm doing some tests with compcache off, 24Mb (1 / 4 of RAM) or more
using Compcache the maximum memory is increased, but free is always 1 or 2Mb!
In the next days I will try with FRX07.1, we will see
Click to expand...
Click to collapse
Based on free, that sound accurate. Settings -> About Phone might be using some different metrics (I honestly don't know).
And what about the 1 or 2 Mb free? they're too less to be ALWAYS 2 Mb, or not?
TheXeno said:
And what about the 1 or 2 Mb free? they're too less to be ALWAYS 2 Mb, or not?
Click to expand...
Click to collapse
What is this, I don't even...
I mean that it isn't normal which there are always 2Mb free, in according to Fabiett83 (he says this)
I'm sorry, before i meant 1-2 mb of uncompressed ram, there was still free memory in compcache
I think android needs more than 100Mb of ram, so it keep closing open apps and processes when i/it need ram for anything else
this could explain the ever full ram and the long time needed for opening apps
even answering to calls needs more than a minute if the phone app is not open!
Edit:
This is the result of ADB Free command in a fresh FRX7.1 install
_____ ---- total --- used --- free - shared - buffers
Mem: -- 103160 - 101116 -- 2044 ------ 0 ----- 72
Swap: - 102392 -- 64004 - 38388
Total: - 205552 - 165120 - 40432
Android uses a total of 165Mb of ram
Now with compcache set to 80Mb in froyo.user.conf
_____ ---- total --- used --- free - shared - buffers
Mem: -- 103160 - 101332 -- 1828 ------ 0 ---- 256
Swap: -- 81912 -- 54032 - 27880
Total: - 185072 - 155364 - 29708
Here uses 155Mb, but leaves less compcache free
Last with compcache set to 64Mb
_____ ---- total --- used --- free - shared - buffers
Mem: -- 103160 - 101812 -- 1348 ------ 0 ---- 352
Swap: -- 65528 -- 34356 - 31172
Total: - 168688 - 136168 - 32520
I Don't know... seem to always use the max possible uncompressed ram and thel load the rest in compcache
what i can clearly see is that the less compcache i use the more responsive the system is!
I will stick to 64mb for a while, i hope 168Mb is enough for normal use
Hi all,
With this Xiaomi device SoC source code was never released so only stock kernel can be used. Is it possible to turn on ZRAM and change ZRAM values with it still if I was to gain root?
Many thanks
LaurenceGough said:
Hi all,
With this Xiaomi device SoC source code was never released so only stock kernel can be used. Is it possible to turn on ZRAM and change ZRAM values with it still if I was to gain root?
Many thanks
Click to expand...
Click to collapse
You already have zRAM enabled by default. AFAIK modern Android devices don't use a swap partition (except RAM+ features).
see your zRAM:
Code:
su
cat /proc/swaps
adjust your zRAM:
Code:
su
swapon --help
the size of your zRAM is defined in
/vendor/etc/*fstab*
WoKoschekk said:
You already have zRAM enabled by default. AFAIK modern Android devices don't use a swap partition (except RAM+ features).
see your zRAM:
Code:
su
cat /proc/swaps
adjust your zRAM:
Code:
su
swapon --help
the size of your zRAM is defined in
/vendor/etc/*fstab*
Click to expand...
Click to collapse
Thanks very much, that command is super useful and no amount of Googling I did could find it!
/vendor/etc/*fstab* reports:
"/dev/block/zram0 none swap defaults zramsize=55%"
This sounds like ZRAM should be enabled at 2.2GB.
The problem with this device is the swap partition is reported as always 2252MB, yet even with the "RAM" boost turned on or off the "swap" partition is reported as the exact same size. I suspected based on the really, really poor memory related performance it is disk based SWAP... Anything more than 2-3 simple apps and they reload constantly like they were fully closed when switching apps. All power saving features are turned off.
P.S I am not rooted just yet, I was just trying to see if I could improve my performance as the CPU is quite strong but memory is the issue.
Thanks again
Edit:
I just ran /vendor/etc/*fstab* again after enabling "RAM boost" and restarting, it says it's on in the app switch view (+2GB) but the output is the exact same of this file, I am not sure if this is to be expected or not. Performance is exactly the same so I guess it's just another MIUI bug?
This "SWAP" partition is also not always full usually only 1GB out of the 2GB in use. Checking against a Pixel 7, it's 3GB of ZRAM (reported as SWAP, but of course no RAM boost or SWAP exists on Pixels) it's fully utilized almost always despite having double the physical RAM.
LaurenceGough said:
at 2.2GB
Click to expand...
Click to collapse
LaurenceGough said:
always 2252MB
Click to expand...
Click to collapse
we calculate with base 2!
=> 1024^2 = MiB = 1.048.576 Byte
=> 1.048.576 x 2258 = 2.367.684.608 Byte = 2,2GiB!!
=> 2258MiB = 2,2GiB
There is no partition or used storage for zRAM/swap on your device. Consider that zRAM is compressed with lz4 (=50% compression). So, 1GiB RAM storage needs only 512MiB zRAM storage. The (de)compression is done by the kernel.
zRAM (and also swap) can't boost your system. These partitions are only used for cached data. Before your kernel deletes this data to clear RAM storage it gets compressd and stored in the zRAM area. It's the most useless data.
zRAM is a dynamic partition on your RAM. If RAM only needs 128MiB zRAM then zRAM won't be greater than 128MiB. It never increases to the full available size until it's really needed. If fsrab states "55%" than 55% is the maximum size of zRAM in your RAM and not a persistent size that is always occupied.
Don't change anything regarding to the RAM management as Android is nearly perfect handling thas by itself. You can only make it worse.
Thanks again. The issue I'm facing is that just a few basic apps struggle to run when multitasking, I have disabled all power saving features and MIUI optimisations / powerkeeper daemon but when switching apps or webpages they very often reload fully. I am certain this is an issue with low memory, and I know 4GB is not a lot of memory these days, but I thought I could run more than two tabs in a web browser, and more than a few simple apps without them reloading.
My understanding of zRAM is that it compresses RAM as you say, whilst this taxes the CPU more it should be able to compress more data into the RAM, effectively allowing more apps / webpages (pages) to stay available to use, rather than being killed by the low memory killer dameon lmkd.
Is my understanding correct that this 55% zRAM would provide an extra ~3.3GB or so of compressed RAM, making the total addressable RAM approx ~5.3GB?
If the 2GB of "RAM boost" or SWAP as it should be called if I am correct was working I'd expect it to be able to keep a few more apps or tabs running in memory and prevent a full reload? Is there any way I could check if this is running correctly without root? I know SWAP partitions are not ideal and they have a few downsides but I think this phone has relatively fast storage for its price range.
I am on the lookout for a device with more RAM but it'd be nice to understand a bit more about this situation and learn.
LaurenceGough said:
making the total addressable RAM approx ~5.3GB?
Click to expand...
Click to collapse
Your RAM storage remains 4096MiB. The swapped data in zRAM is compressed and can't processed until it gets decompressed. All data in zRAM actually should have been deleted and this storage isn't acting as additional RAM storage increasing your performance per se.
Apart from that Android's RAM management always uses 60-70% under normal conditions. For peaks, due to some heavy memory using apps, it can increase to 80-90% while at the same time zRAM increases, too. It's correlating. You can't force RAM to swap everthing into zRAM for having 2-3GB available. That's not the meaning of zRAM.