So with all the excitement going on in the page pool thread I was doing some external reading. One topic that caught my interest is NAND and NOR ROM.
I got a nice summary from here:
http://blogs.msdn.com/windowsmobile/archive/2005/08/19/453784.aspx
Basically, the synopsis is:
NOR: Faster to read, slower to write.
NAND: Slower to read, faster to write.
But, more importantly, NOR ROM let's you perform XIP operations. Now i remember seeing this XIP acronym before...in a directory when I extracted a ROM. This leads me to believe that at least part of the ROM in the Kaiser is NOR ROM.
XIP means eXecute In Place. It basically allows code to be executed directly from the ROM without first being copied into the RAM. This means less RAM utilization. As the article states i works for programs only, not user data files.
If we look in the XIP directory of an extracted ROM we see subdirectories like:
busenum.dll
diskcache.dll
imgfs.dll
These are things like low level bus, disk, and file system drivers. These things make perfect sense to XIP.
My question then is... if indeed we do have NOR ROM that can do XIP operations, how much is free on a typical ROM? AND can we cook in other applications into this XIP NOR location instead of into the NAND ROM and thus have those eXecute in Place and free up additional RAM?
It would be nice to get the 3% of my RAM back from Voice Command, or the 1% from PocketCM, etc, etc...
Could it be as easy as moving .dll files from \SYS to \ROM\XIP before cooking the ROM? I doubt it, but is it possible?
I'm just throwing a concept out there and asking about it's feasability. I'm not really a developer so I don't know how much further I can take this.
Thoughts?
bengalih said:
My question then is... if indeed we do have NOR ROM that can do XIP operations, how much is free on a typical ROM? AND can we cook in other applications into this XIP NOR location instead of into the NAND ROM and thus have those eXecute in Place and free up additional RAM?
It would be nice to get the 3% of my RAM back from Voice Command, or the 1% from PocketCM, etc, etc...
Could it be as easy as moving .dll files from \SYS to \ROM\XIP before cooking the ROM? I doubt it, but is it possible?
I'm just throwing a concept out there and asking about it's feasability. I'm not really a developer so I don't know how much further I can take this.
Click to expand...
Click to collapse
I don't think that blog applies to the Kaiser-like devices anymore. We already use all the ROM space we get. The partitioning between XIP and storage is out of the same space and controlled by how the ROM image is built and whatever they set the pagepool to.
Did you run out of space of the 100MB+ internal storage we all get?
I don't know why it wouldn't apply...it's not a WM5 vs WM6 issue.
Based on the only specs I have seen it lists the Kaiser with 256 ROM and 128 RAM.
The question is what type of ROM is it? Devices can mix NOR and NAND ROM. And based on what I see on the extracted ROM, at least some of it must be NOR (because of the existence of the XIP directories).
And this isn't a question of storage space. I am not trying to get more usable storage...I have a 8GB SDHC card for that. I am trying to maximize my available application space.
The point of my post is... can an application, let's say like Voice Command be moved from what might be the NAND portion of the ROM into the NOR portion of the RAM (from \SYS to \ROM\XIP). It would take up the same space of the TOTAL ROM, but when it executed it DOES NOT NEED BE LOADED INTO RAM and thus your available application space is not decreased.
Again, I am not a developer, so I may be way off in asking if this can be done. However your response is one that doesn't speak to the theory I am proposing.
if I remember right this was talked about before but on a different device I think the hermes. I am not sure of the reason it couldnt be done but I just remember it couldnt lol. Something about allocated memory maps
Some interesting things
I found this wile reading the link you posted. Some great information in there.
By the way, every SD and CF card is made out of NAND flash. So, no, you can't XIP programs stored on a storage card.
Click to expand...
Click to collapse
So am I reading this right. That any program we have stored on our Storage Card won't utilize the XIP
I've been working on & subsequently screwing up, page pool alteraitions for awhile. Because I'm messing with a Kaiser(CE5) & a UMPC (CE6) based device, I can tell you that while the pagepool will save you some seconds, mostly with the loading of contacts, your inbox, & the boot to os speed of the device, for the most part you're right, you won't see much difference.
However, with CE6, that will change. With CE 6 based devices you will be able to completely comtrol paging, be it XIP or Data (Read Only) paging. Maybe wm7 will introduce CE6 to PDA devices. With CE6, formerly & frequently confused with WM6, you will have 2 page pools & several controls over them including compaction.
The effectiveness of page pool sizes can vary widely depending on the types of processes & programs you use, but suffice to say, the average user will take little to no benefit from a larger paging file.
For all of you truly interested, there is a PB process file called DevHealth.exe, that can be used via SD card to report the actual status of the paging pool. Google it, you will find it. Kind of interesting to see what your device is actually doing before & after the changes.
AllTheWay said:
I found this wile reading the link you posted. Some great information in there.
So am I reading this right. That any program we have stored on our Storage Card won't utilize the XIP
Click to expand...
Click to collapse
No but what the OP is asking is if we can take apps and put them in the XIP section and run them from there. I think someone should try it out just cook up a rom with a complete app cooked in the XIP and see what happens. the worst is bad blocks i would guess but maybe POF or OLI should chime in on this one.
I do believe you can XIP from an SD card, I believe MS has done this with a few test devices that utilize under battery sd cards. I think it's not something they've done mostly because of problems in system stability when the SD is removed.
austinsnyc said:
No but what the OP is asking is if we can take apps and put them in the XIP section and run them from there. I think someone should try it out just cook up a rom with a complete app cooked in the XIP and see what happens. the worst is bad blocks i would guess but maybe POF or OLI should chime in on this one.
Click to expand...
Click to collapse
THANK YOU AUSTIN!
Yes, this is what I am saying. Forget about SD cards (which, according to what I have read are all NAND and thus can't XIP).
I am just asking if some of the applications that we are cooking into the flash (the things that extract to \SYS) like MS Voice Command, CM Contacts, QuickGPS, etc... if instead those things can be places in XIP.
Again, this is under the assumption (which is a big assumption) that what is in \XIP gets placed in the NOR ROM and what is in \SYS gets placed in the NAND.
What this would mean is that when you execute any of the programs I mention, like say Quick GPS you won't see the RAM utilization on your device go up, meaning you will have the same amount of available free memory. This is because of the XIP (based on the description I have read) it can be executed from the ROM without being copied into RAM.
Now, my guess is that even though the NOR has faster reads than NAND, it still might be slower than RAM. So, it might take another second to open Quick GPS. However for some apps I think I would prefer the slight delay in order for my available memory to be increased.
GSLEON3 said:
I've been working on & subsequently screwing up, page pool alteraitions for awhile. Because I'm messing with a Kaiser(CE5) & a UMPC (CE6) based device, I can tell you that while the pagepool will save you some seconds, mostly with the loading of contacts, your inbox, & the boot to os speed of the device, for the most part you're right, you won't see much difference....
For all of you truly interested, there is a PB process file called DevHealth.exe, that can be used via SD card to report the actual status of the paging pool. Google it, you will find it. Kind of interesting to see what your device is actually doing before & after the changes.
Click to expand...
Click to collapse
Good info Leon...but better to put this in the pagepool thread so we can discuss it there (and please do). What I'm trying to get at here is not directly related to pagepool sizes and speeds.
austinsnyc said:
if I remember right this was talked about before but on a different device I think the hermes. I am not sure of the reason it couldnt be done but I just remember it couldnt lol. Something about allocated memory maps
Click to expand...
Click to collapse
That's entirely possible. Again, I'm being the "idea guy" as I'm trying to synthesize some info I have absorbed. The most programming I do is high level scripting and VB so I don't know details about how this stuff would actually work down at the memory map level. There may be some other issues as well. I was hoping there was someone in these forums who actually had the knowledge (and wasn't just following cooking tutorials like most of us) of how this stuff truly interacts.
Also please reference my post in the pagepool thread, has some good thoughts (I think!):
http://forum.xda-developers.com/showpost.php?p=2101324&postcount=99
bengalih said:
http://forum.xda-developers.com/showpost.php?p=2101324&postcount=99
Click to expand...
Click to collapse
In addition to the above, I have found some more info which may partially defeat my reasoning:
XIPKernel
There are portions of the deepest parts of the OS that have to XIP. If you're on NOR, that code just XIPs like everything else. Not so on NAND. For a NAND system to boot, it needs to load this code into RAM first and then run it from there. When the system is running, it can't really tell if it's running from RAM or ROM, so it assumes it's running from ROM and doesn't count this space.
The XIPKernel region tends to be between 1.5 and 2M.
Click to expand...
Click to collapse
So it seems that just because we have a \XIP directory doesn't mean we have NOR ROM. It could just be the area the ROM places data that in NEEDS to XIP, and therefore means that it will get copied to RAM on boot (instead of just XIPing from where it is in ROM). This could also account for the additional "missing" RAM up to 128MB.
Therefore moving an application to this \XIP pre-cook doesn't mean that it will save us any RAM (again, assuming we DON'T have NOR ROM). It could however speed up that application since we are basically "pre-loading" it into RAM instead of waiting for the load to be user initiated.
I guess a question now is does all XIPed code run at the same time and is that all at boot? It does us no good to try and load up voice command (or QuickGPS, etc.) before the supporting code from the OS has loaded.
So, this may all be a wash, but it does help explain some of the interaction better and even if it leads no where, at least we will better understand how the devices memory system works.
I have done a search, looked at the Wiki and other articles describing exactly what page pool is and what it is used for. I have been flashing ROMs on my Touch Pro and trying different page pool sizes, but haven't really seen a noticeable effect, other than how much RAM I have I have available after a soft reset. I tried page pools ranging from the stock 6 all the way to 32 MB and really can't say I saw any noticeable performance increase or decrease.
Maybe I wasn't looking in the right place? Where do you tend to see better performance from a larger page pool? Why would more page pool benefit more than having more free RAM?
Yeah..that question is bothering me too....As I noticed .... I used 5mb pool page and in netfront sometimes I had ,,cannot load page..low memory " ..smthing like that. Now with 6mb as I surfed ...I didn`t had that problem with the memory yet. I dunno maybe is because of the bigger poolpage that I have now on this rom.
wow... really? No one here feels like answering this question huh?
there is no answer depend on the apps one use
and how many apps one wish to run at the time
lower page pool mean lower io speed == slower programs which depend on
loading stuff
lower ram mean few apps running at once without running out of mem
judgment call for each user I'd say
Okay, for the sake of this question:
ROM = Read Only Memory
RAM = Random-Access Memory
When embedding programs, such as Mobile Notes, into an image the ROM size of the program is decreased due to digital compression. However, does it also decrease the RAM usage for that program?
Off the cuff, I would think so. Kinda like how an Assembler-language program can run on older machines faster than an optimized C-language program when compiled on the same machine. The tighter the binary conversion, the faster the program can run.
Not sure this is true though. I am trying to decide if it would be better to cook for myself and add the 3rd-party apps that I use, or to use something cooked by someone else and then adding via cab files the apps I want.
Bump.
Still wondering...
hey fellow XDA WM devs and users and forum people,
Greetings,
so I am asking whether not WM devices are capped on using 25+mb of RAM for each app...
If so, can I assume that this is the reason why we cannot have stunning graphic games just like the other compatitor platforms (iOS, Android) ??
also, a reply tweet from one of my favourite game dev:
http://twitter.com/Isotope244/status/16152606483
any comments or answers are greatly apprerciated
Kind Regards
Processes in winmo can only access 32 mb of virtual memory (with a max of 32 processes). There's normally at least 7 mb or so of memory allocated to dll's for all processes (sometimes more). 32-7=25 mb. I think that's what the developer is talking about. This is changing with WP7, where all process will have full access to 1 GB of virtual memory (limited by the amount of availabe RAM).
Farmer Ted said:
Processes in winmo can only access 32 mb of virtual memory (with a max of 32 processes). There's normally at least 7 mb or so of memory allocated to dll's for all processes (sometimes more). 32-7=25 mb. I think that's what the developer is talking about. This is changing with WP7, where all process will have full access to 1 GB of virtual memory (limited by the amount of availabe RAM).
Click to expand...
Click to collapse
thanks farmer tad for your reply....
now its all clear to me
thanks.
(oh lol... then what's thhe point of phones having all those extra RAM while the OS cannot use them, at all?)
Well sadly i don't have an answer for that question yet...
I'm trying to think of a way to put KSM to the test on my android device.
As far as i understand it is possible that the kernel actually causes high CPU usage trying to map and unmap memory pages over and over again.
This issue is known for linux and other virtual machines so it is possible that the Same effect will be on the android vm
Testings that i found are not relevant to android.
For example:
The result is a dramatic decrease in memory usage in virtualization environments. In a virtualization server, Red Hat found that thanks to KSM, KVM can run as many as 52 Windows XP VMs with 1 GB of RAM each on a server with just 16 GB of RAM. Because KSM works transparently to userspace apps, it can be adopted very easily, and provides huge memory savings for free to current production systems. It was originally developed for use with KVM, but it can be also used with any other virtualization system - or even in non virtualization workloads, for example applications that for some reason have several processes using lots of memory that could be shared.
Click to expand...
Click to collapse
http://kernelnewbies.org/Linux_2_6_32
What i would really want to know is what would happen if each of these VMs Would run a different application/game/audio/graphics software at the same time ? or what if the same vm will run many different apps ? and also to compare cpu usage with and without KSM
Guess i'll need a tool for that. something like 'iostat' but for memory diagnostic and another tool to see a per process CPU usage but 'top' is not good enough for that.
Any way, the best test should present clear results with precised data.
I'll keep looking for legit way to put it to the test.
If you can think of a way to test KSM with android, please let me know.
This is a technique that relates mostly to processes like virtualisation. For example, when you load 5 windows XP VMs, you'll have a good 10 - 20 services that are practically the same in memory in each VM. Instead of each service using 10mb (ie, 10mb x 5 = 50mb), you only need say 15 or 20mb using KSM. If you use different applications, it is very unlikely that anything would be saved FOR THAT APPLICATION. However, the main elements of a Windows XP System would still be there (drivers, explorer, firewall, logon, search and so on). Means little in one setup, but when you have several VMs it is shown to be a huge advantage. As we know a simple XP install can use 500mb of RAM actively, and this is fairly uniform across instals.
With android, i don't know if there are specific RAM savings to be had. Don't know enough about the inner workings and the sandbox android puts its apps in or how apps interact with system services. Sadly, i can't think of a good way to test it out either, but i'll be keeping an eye on this topic for someone (much) more knowledgeable to come along.
Harbb said:
Sadly, i can't think of a good way to test it out either, but i'll be keeping an eye on this topic for someone (much) more knowledgeable to come along.
Click to expand...
Click to collapse
Enter bedalus, stands there with a vacant expression on his face. Harbb looks disappointed.
kernels ; battery ; ROM ; gov/sched
That entire paragraph was dedicated to you bedalus, we both know that.
Lol
I hope someone can answer this though.
kernels ; battery ; ROM ; gov/sched
Wait for someone............
Sent from my Nexus S using xda premium
KSM does not improve performance on Android just like that - all enabling KSM does, is enable SUPPORT for the Feature but Applications would have to make use of the feature, which they don't.
You can easily verify this like that :
echo 1 > /sys/kernel/mm/ksm/run
<wait and/or run the Applications of your choice>
cat /sys/kernel/mm/ksm/pages_sharing
IF the above shows a value > 0 then you are making use of KSM else it's just available, without anyone using the feature.
Here's an interesting Article that gives a little more insight :
http://www.linux-kvm.com/content/using-ksm-kernel-samepage-merging-kvm
By the way, the same is true for ZCACHE. If you really want to make better use of your Memory (RAM) then using ZRAM as a Swapdevice does work (and may often make sense, too).
That all said : There appear to be efforts to make use of KSM http://forum.xda-developers.com/showthread.php?t=1464758 - so things may well change ...
any update on this...?