I suspect that compcache is not interacting properly with android but I need some stats to really analyze it. This should be standardized so please answer as I request and avoid any "me too" type answers.
Here's what you need:
Compcache running or a linux swap partition or file (swapper counts).
The default browser bound to a shortcut (search-b) and set to load www.cnn.com as your homepage. Close extra windows before starting and go there at least once. Make sure you're blocking popups.
A homescreen where you have icons and widgets that you have noticed in the past have taken time to load.
Music player to run in the background if you need to increase the system load.
Optionally: Gmail bound to a keyboard shortcut. (search-g for me)
Adb shell and knowledge to run these tests.
Here's what you're doing:
adb shell into your phone
Grab following stats (you can copy and paste in on one line):
Code:
uname -r;uptime;cat /proc/sys/vm/swappiness;cat /proc/swaps;cat /proc/ramzswap;free
Hit home button and go to homescreen. Take note of whether the homescreen needed to reload everything or if it was snappy. After completely done loading, run 'free' and record.
Keyboard shortcut to browser. Take note of whether the browser had to download and reload everything. After completely done loading, run 'free' and record.
Repeat 3-4 until either:
switching is always snappy,
the value of 'free' for swap ceases to change significantly while apps are reloading (like +-100 over 2-3 slow reloads),
or the value of 'free' for swap maxes out (note if you start getting lots of force closes).
Code:
total used free shared buffers
Mem: 97876 95492 2384 0 644
Swap: 43880 [B] 404[/B] 43476
Total: 141756 95896 45860
If its always snappy, put more load on the system by setting the music player in the background. If that still doesn't stress the system, throw in Gmail in the mix randomly. Note if snappy, and record 'free' stats.
If you're using both compcache and a linux swap, try to get information at the point where you've crossed over from compcache to linux swap. So if you're compcache is only 8 megs, seeing the difference as your swap fills over the 8 meg boundary is useful.
Finally, here's what you should report, either by putting it into a text file and attaching, or by placing it into a
Code:
block.
[list=1][*]Mod used. e.g. Cyanogen 3.9
[*]Info from step 2) above. [code]2.6.29-cm
12:47:45 up 3:43, load average: 2.13, 2.05, 2.09
15
Filename Type Size Used Priority
/dev/ramzswap0 partition 12284 408 -1
/system/sd/swapfile.swp file 31596 0 -2
DiskSize: 12288 kB
NumReads: 33
NumWrites: 102
FailedReads: 0
FailedWrites: 0
InvalidIO: 0
PagesDiscard: 0
ZeroPages: 3
GoodCompress: 77 %
NoCompress: 9 %
PagesStored: 99
PagesUsed: 31
OrigDataSize: 396 kB
ComprDataSize: 118 kB
MemUsedTotal: 124 kB
total used free shared buffers
Mem: 97876 95720 2156 0 632
Swap: 43880 408 43472
Total: 141756 96128 45628
[*]Results from repeating 3-4. If it ended up being too responsive and you couldn't stress it just include one cycle. Mark what you swapped to and whether it reloaded (Y for yes). e.g.
Code:
home
total used free shared buffers
Mem: 97876 95824 2052 0 292
Swap: 24464 2268 22196
Total: 122340 98092 24248
browser Y
total used free shared buffers
Mem: 97876 96032 1844 0 252
Swap: 24464 2540 21924
Total: 122340 98572 23768
home
total used free shared buffers
Mem: 97876 94436 3440 0 264
Swap: 24464 2940 21524
Total: 122340 97376 24964
browser Y
total used free shared buffers
Mem: 97876 95960 1916 0 280
Swap: 24464 5292 19172
Total: 122340 101252 21088
home Y
total used free shared buffers
Mem: 97876 92868 5008 0 272
Swap: 24464 6484 17980
Total: 122340 99352 22988
[/list]
Thanks!
Reserved for my test results.
Another good test is iMeem. This app is a memory whore and really puts pressure on the system.
Code:
Cyanogen 3.6.8.1
2.6.29-cm
15:19:09 up 5:28, load average: 7.58, 5.31, 4.33
20
Filename Type Size Used Priority
/dev/ramzswap0 partition 24464 15968 -1
DiskSize: 24468 kB
NumReads: 7817
NumWrites: 8908
FailedReads: 0
FailedWrites: 0
InvalidIO: 0
PagesDiscard: 0
ZeroPages: 241
GoodCompress: 65 %
NoCompress: 12 %
PagesStored: 5871
PagesUsed: 2370
OrigDataSize: 23484 kB
ComprDataSize: 9323 kB
MemUsedTotal: 9480 kB
total used free shared buffers
Mem: 97876 95852 2024 0 560
Swap: 24464 15968 8496
Total: 122340 111820 10520
home Y
total used free shared buffers
Mem: 97876 96348 1528 0 776
Swap: 24464 14600 9864
Total: 122340 110948 11392
browser Y
total used free shared buffers
Mem: 97876 96528 1348 0 420
Swap: 24464 14844 9620
Total: 122340 111372 10968
home Y
total used free shared buffers
Mem: 97876 96432 1444 0 752
Swap: 24464 14792 9672
Total: 122340 111224 11116
browser Y
total used free shared buffers
Mem: 97876 96484 1392 0 404
Swap: 24464 14844 9620
Total: 122340 111328 11012
home Y
total used free shared buffers
Mem: 97876 96464 1412 0 668
Swap: 24464 15048 9416
Total: 122340 111512 10828
No other posts? C'mon everyone, these statistics will help YOU get a faster, better configured phone.
**********
sha.goyjo said:
I'm going to be the first to say it. Doing tests on compcache .5 is somewhat pointless atm. If you follow Cyanogens, you'll see that he's currently working with the compcache people on an arm revision of .6, which uses a VERY different code base. I'd contribute to your thread if it wasn't trying to fiddle with a version that was just an arm patch on the existing compcache branch. You'd be better off waiting till the .6 goes stable on the trunk. All that being said, if you want to try and rewrite kernel modules that are in the process of being re-re-written, be my guest.
Not trying to be an ass, just trying to save us all some time by working towards things efficiently.
That being said, .6 could REALLY use some people working on it. Once it gets working we'll be able to use backing swap files instead of a partition, which would be great. No more Linux-swap on an sdcard.
Click to expand...
Click to collapse
Hmm. So what you are saying is that we should find no significance in the fact the cyanogen already posted in this thread with supportive info for the tests.
Also, in regards to your work on .6, what benchmarking plan did you have to prove:
1) that .6 performs better than .5 on our phones
2) that .6 performs better than a linux swap on our phones
3) that .6 performs better than .5 with a backing swap on our phones
Please note that any answers to this thread could help build these benchmarks for you.
Running CM 3.8
Browser only reloaded once...even with music playing, and running wisepilot! Even the one reload was a quick one.
Code:
2.6.29-cm
18:05:17 up 1:13, load average: 2.75, 3.69, 4.14
60
Filename Type Size Used Priorit
/dev/ramzswap0 partition 31996 21264 -1
DiskSize: 32000 kB
NumReads: 15024
NumWrites: 16558
FailedReads: 0
FailedWrites: 0
InvalidIO: 0
PagesDiscard: 246
ZeroPages: 196
GoodCompress: 75 %
NoCompress: 4 %
PagesStored: 7804
PagesUsed: 2977
OrigDataSize: 31216 kB
ComprDataSize: 11516 kB
MemUsedTotal: 11908 kB
total used free shared buffers
Mem: 97876 96376 1500 0 576
Swap: 31996 21264 10732
Total: 129872 117640 12232
#
From CNN to Home
**snappy
total used free shared buffers
Mem: 97876 95752 2124 0 588
Swap: 31996 22040 9956
Total: 129872 117792 12080
From Home to CNN
**snappy no reload
total used free shared buffers
Mem: 97876 96032 1844 0 520
Swap: 31996 22020 9976
Total: 129872 118052 11820
#
ADDED MUSIC PLAYER BACKGROUND And checked mem
total used free shared buffers
Mem: 97876 95964 1912 0 624
Swap: 31996 17516 14480
Total: 129872 113480 16392
FROM HOME TO CNN
**Reload
total used free shared buffers
Mem: 97876 96120 1756 0 448
Swap: 31996 24780 7216
Total: 129872 120900 8972
CNN to Home
**Snappy...no redraw
total used free shared buffers
Mem: 97876 96400 1476 0 492
Swap: 31996 29248 2748
Total: 129872 125648 4224
Home to Browser
***Snappy NO RELOAD WITH MUSIC RUNNING!!!
total used free shared buffers
Mem: 97876 96288 1588 0 464
Swap: 31996 26660 5336
Total: 129872 122948 6924
ADDED WISEPILOT NAVIGATION mUSIC STILL RUNNING
total used free shared buffers
Mem: 97876 96532 1344 0 396
Swap: 31996 27592 4404
Total: 129872 124124 5748
WISEPILOT RUNNING, MUSIC RUNNING, SWITCHED TO CNN from within wisepilot (search b)
***NO RELOAD, NO LAG....
total used free shared buffers
Mem: 97876 95988 1888 0 492
Swap: 31996 27208 4788
Total: 129872 123196 6676
#
double posted
by the way, in case everyone missed it:
I have two G1's. Both have brand new installs of CM 3.9. One has compcache (with swapiness set to 20 and 24mb of space allocated) and one does not. After a reboot on both devices and waiting a five minutes for everything to settle, I filmed the opening of the camera app. The one without compcache is noticeably quicker. Is compcache slowing our machines down, at least in certain instances?
By the way, cat /proc/ramzswap confirms compcache is enabled.
http://www.youtube.com/watch?v=gNYv-5WLDVA
edit:swappiness for the above tests was at 60....
Running Cyanogen 3.8.1 with the Palm Pre theme ( dunno if it made a diff.) and HTC Music app running.
Code:
2.6.29-cm
03:53:53 up 3 min, load average: 2.22, 1.47, 0.60
60
Filename Type Size Used Priority
/dev/ramzswap0 partition 31996 11612 -1
DiskSize: 32000 kB
NumReads: 1902
NumWrites: 3415
FailedReads: 0
FailedWrites: 0
InvalidIO: 0
PagesDiscard: 0
ZeroPages: 137
GoodCompress: 72 %
NoCompress: 4 %
PagesStored: 3278
PagesUsed: 1128
OrigDataSize: 13112 kB
ComprDataSize: 4461 kB
MemUsedTotal: 4512 kB
total used free shared buffers
Mem: 97876 96488 1388 0 652
Swap: 31996 11612 20384
Total: 129872 108100 21772
Home Y
total used free shared buffers
Mem: 97876 95704 2172 0 668
Swap: 31996 11612 20384
Total: 129872 107316 22556
Browser Y
total used free shared buffers
Mem: 97876 96308 1568 0 432
Swap: 31996 15528 16468
Total: 129872 111836 18036
Home N
total used free shared buffers
Mem: 97876 86336 11540 0 468
Swap: 31996 14936 17060
Total: 129872 101272 28600
Browser Y
total used free shared buffers
Mem: 97876 95196 2680 0 520
Swap: 31996 14932 17064
Total: 129872 110128 19744
Home N
total used free shared buffers
Mem: 97876 96176 1700 0 460
Swap: 31996 15056 16940
Total: 129872 111232 18640
Browser (Opened up really quick) N
total used free shared buffers
Mem: 97876 96432 1444 0 540
Swap: 31996 15200 16796
Total: 129872 111632 18240
Home N
total used free shared buffers
Mem: 97876 96096 1780 0 540
Swap: 31996 15208 16788
Total: 129872 111304 18568
pinetreehater said:
by the way, in case everyone missed it:
I have two G1's. Both have brand new installs of CM 3.9. One has compcache (with swapiness set to 20 and 24mb of space allocated) and one does not. After a reboot on both devices and waiting a five minutes for everything to settle, I filmed the opening of the camera app. The one without compcache is noticeably quicker. Is compcache slowing our machines down, at least in certain instances?
By the way, cat /proc/ramzswap confirms compcache is enabled.
http://www.youtube.com/watch?v=gNYv-5WLDVA
edit:swappiness for the above tests was at 60....
Click to expand...
Click to collapse
I'm not surprised it was.
Without compcache : Start camera app, realize your out of memory, kill another application (meaning it will start slower the next time, going through full initialization).
With compcache : Start camera, push inactive application into compcache.
I'd like to see some similar tests with more tests run. I think I might actually write a little app for this.
derfolo said:
Hmm. So what you are saying is that we should find no significance in the fact the cyanogen already posted in this thread with supportive info for the tests.
Also, in regards to your work on .6, what benchmarking plan did you have to prove:
1) that .6 performs better than .5 on our phones
2) that .6 performs better than a linux swap on our phones
3) that .6 performs better than .5 with a backing swap on our phones
Please note that any answers to this thread could help build these benchmarks for you.
Click to expand...
Click to collapse
No, I apologize for being offensive. Part of my reasoning has more to do with the fact that, as far as I have seen, regardless of what works best people tend to move to the latest version. It wasn't a utility prediction. In that respect, you are going in the right direction. I do think that the plans to implement dynamic swapping between backing and ramz in the future really make a big difference, however, in any kind of work we are trying to do here. Also, from what I've looked at the difference in the modules, they look significantly different now.
I didn't mean to press your buttons, but what I'm saying does have merit.
I say this because it's not about benchmarking the current version vs. the new version. That kind of a benchmark doesn't work, because the new version has an improved feature set. Software with a larger feature set rarely performs better than software with a smaller feature set. However, the features inherent in compcache .6 are VERY compelling for android phones, specifically the ability to use backing_swap from a FILE instead of a partition. The fewer partitions on your sdcard, the better. Less wasted space, and less time spend confusing the kernel as to which /dev/ it should be writing to at the time.
As to cyanogen having "supported" your tests, there are a lot of good reasons for it. You could discover a kernel interfacing problem. You could discover a problem in the arm patch that would be relevant to the new version or upgraded versions of CM.
All I think is that the compcache project could use all the help it can get rolling out .6 for android, and that if more people helped them with that, maybe we could try and find ways to fix the version of compcache we'll be using, instead of the one we'll be moving away from.
One thing people need to realize is that there is no free lunch. Swap (whether the partition or compcache) is not for speeding up the system but for allowing you to run many big apps together at the same time.
swap will slow you down a bit, whether its compcache (cpu overhead for compression) or partition (I/O overhead). So, stop fooling yourself. If you want a fast phone, just use basic things and leave swap out of picture. if you want a fast phone AND want to run tonnes of stuff like touchflo, get a phone with more RAM.
test results
devsk said:
One thing people need to realize is that there is no free lunch. Swap (whether the partition or compcache) is not for speeding up the system but for allowing you to run many big apps together at the same time.
swap will slow you down a bit, whether its compcache (cpu overhead for compression) or partition (I/O overhead). So, stop fooling yourself. If you want a fast phone, just use basic things and leave swap out of picture. if you want a fast phone AND want to run tonnes of stuff like touchflo, get a phone with more RAM.
Click to expand...
Click to collapse
Correct me if I'm wrong, but it sounds rather like you're saying, "Stop trying to make things better." Rather an odd sentiment for a Development forum, no?
I ran this test with just Calendar and Weather widgets running on the home screen and, although I have occasionally found the entire home screen slow to reload during regular use, during this test the home screen never seemed to reload at all, even with Music playing and an occasional jump to Gmail.
I'm running CM 3.9 with a simple Compcache script (no backing swap) initiated from Gscript for default disk size and 20 swappiness:
Code:
2.6.29-cm
12:23:08 up 36 min, load average: 3.37, 3.65, 2.79
20
Filename Type Size Used Priority
/dev/ramzswap0 partition 24464 6992 -1
DiskSize: 24468 kB
NumReads: 587
NumWrites: 1761
FailedReads: 0
FailedWrites: 0
InvalidIO: 0
PagesDiscard: 0
ZeroPages: 126
GoodCompress: 66 %
NoCompress: 5 %
PagesStored: 1635
PagesUsed: 602
OrigDataSize: 6540 kB
ComprDataSize: 2338 kB
MemUsedTotal: 2408 kB
total used free shared buffers
Mem: 97876 92880 4996 0 552
Swap: 24464 6992 17472
Total: 122340 99872 22468
Browser: N
total used free shared buffers
Mem: 97876 92856 5020 0 552
Swap: 24464 6992 17472
Total: 122340 99848 22492
Home: N
total used free shared buffers
Mem: 97876 92856 5020 0 552
Swap: 24464 6992 17472
Total: 122340 99848 22492
(after stressing with Music playing AND switching to Gmail)
Browser: Y
total used free shared buffers
Mem: 97876 95752 2124 0 452
Swap: 24464 7892 16572
Total: 122340 103644 18696
Home: N
total used free shared buffers
Mem: 97876 96496 1380 0 444
Swap: 24464 8276 16188
Total: 122340 104772 17568
ei8htohms said:
I ran this test with just Calendar and Weather widgets running on the home screen and, although I have occasionally found the entire home screen slow to reload during regular use, during this test the home screen never seemed to reload at all, even with Music playing and an occasional jump to Gmail.
Click to expand...
Click to collapse
And it probably doesn't, with compcache you have enough memory to keep it in, while without it needs to go through the whole onCreate and xml inflation.
I am not saying don't make it better. What I am saying is your expectations should be correct.
I use compcache myself. I use it with memlimit_kb=32000 disksize_kb=48000 backing_swap=/dev/block/mmcblk0p3 and a swappiness of 20. But I don't expect it to make my system fly. I just expect that more of my apps will stick around rather than killed and reloaded later. So, I am paying a little swapping cost upfront for payback later.
any lag once compcache fills up?
devsk said:
I am not saying don't make it better. What I am saying is your expectations should be correct.
I use compcache myself. I use it with memlimit_kb=32000 disksize_kb=48000 backing_swap=/dev/block/mmcblk0p3 and a swappiness of 20. But I don't expect it to make my system fly. I just expect that more of my apps will stick around rather than killed and reloaded later. So, I am paying a little swapping cost upfront for payback later.
Click to expand...
Click to collapse
Understood about managing expectations. I'm still considering the potential negative impact on battery life as well, since this is one notorious weak spot on the G1 (when it came out you didn't hear huge numbers of people complaining about how SLOW it was, but everyone complained about battery life...).
Do you mind explaining a little about your memlimit and disksize settings? I read a little about those options in the Google Code page, but I can't say I fully grokked the significance or difference.
I also can't seem to find a good explanation of how compcache and backing-swap work together. Most reports on XDA point to massive lag after compcache maxes out, but is it only then that backing-swap kicks in (and maybe a lot of those folks were using the 60 or 100 swappiness settings)? Can Linux-swap and compcache be run in parallel or separately from each other in some way (I'm guessing no)?
ei8htohms said:
Understood about managing expectations. I'm still considering the potential negative impact on battery life as well, since this is one notorious weak spot on the G1 (when it came out you didn't hear huge numbers of people complaining about how SLOW it was, but everyone complained about battery life...).
Do you mind explaining a little about your memlimit and disksize settings? I read a little about those options in the Google Code page, but I can't say I fully grokked the significance or difference.
I also can't seem to find a good explanation of how compcache and backing-swap work together. Most reports on XDA point to massive lag after compcache maxes out, but is it only then that backing-swap kicks in (and maybe a lot of those folks were using the 60 or 100 swappiness settings)? Can Linux-swap and compcache be run in parallel or separately from each other in some way (I'm guessing no)?
Click to expand...
Click to collapse
According to the Dev:
**********************************************************
Usage:
- modprobe ramzswap [memlimit_kb=<val>|disksize_kb=<val>] [backing_swap=<dev>]
memlimit_kb: This param is applicable only when backing_swap is given.
It is limit on amount compressed data stored in memory. Any
additional data is forwarded to backing_swap. It cannot be greater
than backing device size. If missing or 0, default value is used:
15% of RAM or backing device size, whichever is smaller.
disksize_kb: This param is applicable only when backing_swap is not given.
It is limit on amount of *uncompressed* worth of data stored in
memory. For e.g. disksize_kb=1024 means it can hold 1024kb worth of
uncompressed data even if this data compresses to just, say, 100kb.
If missing or 0, default value is used: 25% of RAM.
backing_swap: This is block device to be used as backing store for ramzswap.
It must be a valid swap partition. We move data to this device when we
encounter incompressible page or memlimit is reached. TODO: we may also
move some pages from ramzswap to this device in case system is really
low on memory.
This device is not directly visible to kernel as a swap device
(/proc/swaps will only show /dev/ramzswap0 and not this device).
Managing this backing device is the job of ramzswap module.
Examples:
1) modprobe ramzswap memlimit_kb=10240 backing_swap=/dev/sda2
sets ramzswap limit as 10MB and /dev/sda2 as backing swap device.
NOTE: here /dev/sda2 is a valid swap partition.
2) modprobe ramzswap backing_swap=/dev/sda2
same as (1) but memlimit is set to default: 15% of RAM or size of
backing swap device, whichever is smaller.
3) modprobe ramzswap disksize_kb=10240
sets ramzswap disk size as 10MB.
4) modprobe ramzswap.ko
same as (3) but ramzswap disk size will be set to default:
25% of RAM size.
Once module is loaded, activate this swap with highest priority:
swapon /dev/ramzswap0 -p 100
(-p param set swap priority)
Notes:
- ramzswap stats are exported via /proc/ramzswap
- If you give non-swap partition as backing_swap, nothing bad will happen -
swapon will simply fail to recognize /dev/ramzswap0 as swap partition.
So, in this case, unload the module and reload with correct backing_swap.
**********************************************************
thanks! very interesting
uwonsum said:
According to the Dev:
memlimit_kb: applicable only when backing_swap is given...limit on amount compressed data stored...additional data is forwarded to backing_swap.
disksize_kb: applicable only when backing_swap is not given...limit on amount of *uncompressed* worth of data stored.
Click to expand...
Click to collapse
So if I'm reading this right, memlimit and disksize would never be used together, right? And when backing_swap is NOT used, compcache is using the disksize allocation for uncompressed data, like virtual swap within RAM? Huh?
Once module is loaded, activate this swap with highest priority:
swapon /dev/ramzswap0 -p 100
(-p param set swap priority)
Click to expand...
Click to collapse
This swap priority info seems to directly conflict the info on the CompilingAndUsing page of the Google code project page:
Add ramzswap as swap device
swapon /dev/ramzswap0 -p 1 (give this swap device the highest priority).
Click to expand...
Click to collapse
Is 1 the highest priority or is 100 the highest priority? Is this the same as setting the swappiness? There must be a typo somewhere. Any help?
[EDIT]NVM, "-p 1" seems to be an invalid parameter.[/EDIT]
BTW, I started playing with your (rather more exotic) script from the CM Wheel of Death thread:
http://forum.xda-developers.com/showpost.php?p=4205132&postcount=518
Have you had any further findings about the various settings employed?
Thanks again!
First of all, this is not another lame post about why vodafone's magic has less ram than htc one. This is just... another thing about partly the same issues.
Second, this is also not about rosie. If you're going to say if this will make rosie work better, don't post. I really don't give a **** about rosie. I'm just trying to get more of this hardware.
Now for the topic. As you will already know, there are two variants of the Sapphire. One, supposedly has 192 Mb of RAM (Vodafone) and the other one has 288 (HTC). So far OK.
BUT, Vodafone Magic kernel reports 98Mb of available RAM. Wait, what? Even if the radio firmware eats part of the memory (about 20mb on msm7200 SoC), the kernel should report more than half of the memory the ram ic supposedly has. Furthermore, looking at the kernel source I found this on the memory fixup code (wich I assume is where it reserves the baseband ram area, like on other phones):
Code:
mi->nr_banks = 1;
mi->bank[0].start = PHYS_OFFSET;
mi->bank[0].node = PHYS_TO_NID(PHYS_OFFSET);
if (smi_sz == 32) {
mi->bank[0].size = (84*1024*1024);
} else if (smi_sz == 64) {
mi->bank[0].size = (101*1024*1024);
} else {
printk(KERN_ERR "can not get smi size\n");
/*Give a default value when not get smi size*/
smi_sz = 64;
mi->bank[0].size = (101*1024*1024);
printk(KERN_ERR "use default : smisize=%d\n", smi_sz);
}
As I stated before, qualcomm phones seem to use part of the ram for the baseband firmware, and, at least on Kaiser and Nikes (wich are the ones I know better, at least on the code side), have splitted the ram banks so the kernel doesn't try to write to the baseband memory corrupting it all, for example, kaiser:
Code:
mi->nr_banks=2;
mi->bank[0].start = 0x10000000;
mi->bank[0].node = 0;
mi->bank[0].size = 0x03800000;
mi->bank[1].start = 0x14000000;
mi->bank[1].node = 0;
mi->bank[1].size = 0x02d00000;
But on the Magic, there's no splitting. If it detects smi_size=32 (htc, though it's not really working) it throws 101 mb of RAM. if it detects smi_size=64( vodafone), it throws 88 mb
I might be completely wrong, and maybe there's another routine that I haven't seen or something like that, but it feels to me like they ran out of time while coding it and they left the fixup so it did boot, but didn't bother debugging to get the maximum ram amount, so they coded the minimal and left the rest for the modem "just in case". I'm going to do a few tests, but if someone knows something I don't please feel free to enlighten me
I was partially right
First test, first success
Before tricking:
/proc # cat meminfo
MemTotal: 97876 kB
MemFree: 3324 kB
Buffers: 384 kB
Cached: 22788 kB
SwapCached: 0 kB
Active: 37820 kB
Inactive: 40320 kB
Active(anon): 26548 kB
Inactive(anon): 29000 kB
Active(file): 11272 kB
Inactive(file): 11320 kB
Unevictable: 256 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 55248 kB
Mapped: 13360 kB
Slab: 6592 kB
SReclaimable: 1136 kB
SUnreclaim: 5456 kB
PageTables: 5108 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 48936 kB
Committed_AS: 1487028 kB
VmallocTotal: 155648 kB
VmallocUsed: 70820 kB
VmallocChunk: 33788 kB
After first attempt:
MemTotal: 102004 kB
MemFree: 4420 kB
Buffers: 2384 kB
Cached: 51580 kB
SwapCached: 0 kB
Active: 53508 kB
Inactive: 33744 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 33312 kB
EDIT: Okay, definitely, we can get more free ram than what it's actually available. After a few test, I'm getting 106mb of total RAM. This is only tricking the first memory bank, without telling the kernel there's a hole (wich by the way there is as I have already trashed it twice ) in the middle of the memory space. More proof:
Fourth attempt:
# cat proc/meminfo
MemTotal: 106068 kB
MemFree: 25916 kB
Buffers: 20 kB
Cached: 45920 kB
SwapCached: 0 kB
Active: 41660 kB
Inactive: 30048 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 25812 kB
Mapped: 21176 kB
Slab: 3596 kB
SReclaimable: 964 kB
SUnreclaim: 2632 kB
PageTables: 2528 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 53032 kB
Committed_AS: 693744 kB
VmallocTotal: 147456 kB
VmallocUsed: 61148 kB
VmallocChunk: 65532 kB
Regards!
Impressive!
32B owners : prepare yourself for a whole new experience
Could this be related to the fact that 32A kernels are not working on a 32B radio?
I think so, yes. As memory sizes are different, memory addresses change, that could be one of the reasons why 32a and 32b kernels aren't compatible between devices. Though I don't know if there'll be something else I don't know (wich probably there is).
So far, I found that on the beginning of the memory space, I can get up to 108 Mb of RAM without corruption (or reboots, or strange things) on my 32B magic. I have splitted the memory in two banks on the code, and trying to get some more free space at the end of the memory, but with no luck yet (working on this for an hour or two, I don't expect to be THAT fast )
But, it's a quite nice start. Even if we can't get the full 192 mb (wich we can't because of the radio), maybe we can get up to 150 or 160Mb, wich would be quite better than what we have. What I have to find is where the radio memory ends so I don't screw it, and then get to the last writable address. Trial and error
Amon_RA said:
32B owners : prepare yourself for a whole new experience
Could this be related to the fact that 32A kernels are not working on a 32B radio?
Click to expand...
Click to collapse
Do you mean that this may help track down why when 32a owners flash the 32b radio we lose all of our RAM?
Good luck man! I wish I can help with this technical stuff
biktor_gj said:
Trial and error
Click to expand...
Click to collapse
Hmm, perhaps not quite. Your approach seems a bit "unscientific". You might get the impression that you're not overwriting anything, but what if some seldom used part of the memory is overwritten. You might not notice until the baseband gets in a special situation.
(Sorry, "unscientific" sounds rude, but it's not meant that way. I think you're making great progress!)
Is it possible to map part of the memory as read-only?
If that is the case, couldn't we build a test kernel that maps 64mb of the first memory as accessible, and the rest as readonly. Then do a custom recovery initrd that just dumps all memory to the PC and analyse the layout there?
@wire103: The way I see it, and the way it behaves so far, If I take a single byte reserved for the baseband, the kernel doesn't finish booting. It's easy to understand why, the kernel writes to a reserved address, then the baseband or a) gets corrupt and reboots or b) writes to the reserved address corrupting the data the kernel has written, so the kernel gets corrupt and reboots. In the end, if the phone ends up booting it means it all worked quite well. At the end of the reserved memory space, on other msm's at least, resides the audio tables (audioparam*.csv) the baseband uses to change between profiles (headset, earcouple and speaker). If I overwrite that, boom! the phone reboots, and start over again, or, in the best case scenario, it will break audio and reboot just the baseband losing phone connectivity, wich is obvious to see and to debug
On the kernel side, I can't mark anything as read only, I'm telling it that there IS more RAM so it can handle it, and that it's work in the end. And as I cannot get access to /dev/mem (since it's not there) I haven't found an easy fix to dump the entire memory.
@bcrook: I think it's not the radio what makes you loose ram, but the kernel itself. But without the HTC patches I cannot be sure until I see it. Although I can try to prepare some custom kernel with a little declared Ram to see if it helps booting a 32b kernel with a 32A radio.
In any case, 32A's original radio is version 3.22.20.17 and 32B radio is 2.22.19.26I so there might be some other things we don't know about.
And, so you all know what I know so far, 32A's kernel reports not 288 Mb of RAM as HTC Specification's page shows, but 198 Mb:
<6>[ 0.020000] Memory: 198MB = 198MB total
<5>[ 0.020000] Memory: 196608KB available (2824K code, 941K data, 104K init)
(taken from a dmesg some collaborative user posted on another thread)
So it seems in both cases, there's only one mapped bank, and it's eating about 80 mb of ram in both cases. Maybe these msm basebands need more ram to operate than previous models (I'm talking here about msm7201 vs msm7200). That's something I still don't know and I still have to test, but what I do know is that radios designed for both msm7200 and 7201 take more or less the same amount of space on flash (about 20mb), and on 7200 the needed ram for the baseband is 30mb, wich is way less than 80 mb wich is what this thing is eating.
EDIT: hmmm... I think I found the hole... shared memory (for gpu, camera...)
After looking at the code for the Diamond, Raphael and Blackstone, and after checking again the header files for the sapphire, I've seen that, apart from the baseband memory, wich will probably be 30 Mb more or less, other things are eating the ram. The most usable memory without corruption I can get is 109 Mb (11 mb more than stock kernels) without noticeable changes or problems (not that I've done a lot of debugging, but using the phone functions and opengl didn't give any problems of any kind)
So, we can take a little more RAM but not nearly as much as I thought.
For the code, in case anyone is interested, here you have:
Code:
if (smi_sz == 32) {
mi->bank[0].size = (84*1024*1024);
} else if (smi_sz == 64) {
mi->bank[0].size = (108*1024*1024);
} else {
printk(KERN_ERR "can not get smi size\n");
Or, if you want to tune it to the most you can get:
Code:
if (smi_sz == 32) {
mi->bank[0].size = (84*1024*1024);
} else if (smi_sz == 64) {
mi->bank[0].size = 0x06d00000;
} else {
printk(KERN_ERR "can not get smi size\n");
Almost anything above 0x06d00000 will make the phone reboot endlessly, and this will give you 109Mb of RAM. You can tune it further by altering the reserved memory addresses for the SMI, but I don't think that's a good idea (if they have those values it's for a reason, and you might end up seeing green lines on the screen, lockups, or other kind of errors by reducing the space for the framebuffer, adsp etc)
biktor_gj said:
@bcrook: I think it's not the radio what makes you loose ram, but the kernel itself. But without the HTC patches I cannot be sure until I see it. Although I can try to prepare some custom kernel with a little declared Ram to see if it helps booting a 32b kernel with a 32A radio.
In any case, 32A's original radio is version 3.22.20.17 and 32B radio is 2.22.19.26I so there might be some other things we don't know about.
Click to expand...
Click to collapse
I would be willing to test this on my 32A if you want to put something together. It would be superb if we could get 32B ROM's working on the 32A using the full available RAM, or even more available RAM than a default 32A ROM.
PM me if you want to use me as a mule.
This looks very interesting the outcome could be of huge benefit to all.
Just wish I could add some tech knowledge but I cant.
Will continue to read with interest well done great work so far
EDIT: nm, it is in the kernel source.
I can try something for 32A users, if you don't mind risking your phone.,.
Let's see if I get it right.
Given a 32A board, can you pick a 32b boot.img, radio and rom, flash it, and boot all the way to the home screen, and the only thing you'd loose is the upper memory? (making the phone think it has 100mb of ram)?
If that's the case, and 32b kernels CAN boot on 32A boards given the right Radio version, it's a matter of patching the memory bank sizes so the kernel knows it has more ram than what it's declared...
biktor_gj said:
I can try something for 32A users, if you don't mind risking your phone.,.
Let's see if I get it right.
Given a 32A board, can you pick a 32b boot.img, radio and rom, flash it, and boot all the way to the home screen, and the only thing you'd loose is the upper memory? (making the phone think it has 100mb of ram)?
If that's the case, and 32b kernels CAN boot on 32A boards given the right Radio version, it's a matter of patching the memory bank sizes so the kernel knows it has more ram than what it's declared...
Click to expand...
Click to collapse
That's exactly right. Using a 32B radio I can boot any 32B ROM but am down 100mb. If you can compile a 32B kernel for me that uses more RAM I will try it out.
For reference, here is what my meminfo looks like currently:
MemTotal: 197144 kB
MemFree: 15380 kB
Buffers: 264 kB
Cached: 36244 kB
SwapCached: 0 kB
Active: 104564 kB
Inactive: 18568 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 86648 kB
Mapped: 17496 kB
Slab: 48268 kB
SReclaimable: 41132 kB
SUnreclaim: 7136 kB
PageTables: 6244 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 98572 kB
Committed_AS: 1669088 kB
VmallocTotal: 319488 kB
VmallocUsed: 65884 kB
VmallocChunk: 201724 kB
Aweome if it works...
1 Lab Rat signing up for 32A!
There's a value that seems to change between boards. That's the SMI size. On 32B boards its value is 64, and on 32A boards is 32. Can you flash it all with 32b rom and radio, and post some info elsewhere?
What I need is the kernel messages log. You can get it by getting a root shell (adb shell), and running the command:
dmesg > /sdcard/dmesg.txt
Then uploading that file somewhere to pick up (the easiest, pastebin.com)
I need to see if even if it's running a 32b kernel, it detects the correct smi size, or if it picks the generic one. If it detects the correct size, we may have a chance of autodetecting the board revision on boot.
biktor_gj said:
There's a value that seems to change between boards. That's the SMI size. On 32B boards its value is 64, and on 32A boards is 32. Can you flash it all with 32b rom and radio, and post some info elsewhere?
What I need is the kernel messages log. You can get it by getting a root shell (adb shell), and running the command:
dmesg > /sdcard/dmesg.txt
Then uploading that file somewhere to pick up (the easiest, pastebin.com)
I need to see if even if it's running a 32b kernel, it detects the correct smi size, or if it picks the generic one. If it detects the correct size, we may have a chance of autodetecting the board revision on boot.
Click to expand...
Click to collapse
Doing it now, standby
biktor_gj said:
There's a value that seems to change between boards. That's the SMI size. On 32B boards its value is 64, and on 32A boards is 32. Can you flash it all with 32b rom and radio, and post some info elsewhere?
What I need is the kernel messages log. You can get it by getting a root shell (adb shell), and running the command:
dmesg > /sdcard/dmesg.txt
Then uploading that file somewhere to pick up (the easiest, pastebin.com)
I need to see if even if it's running a 32b kernel, it detects the correct smi size, or if it picks the generic one. If it detects the correct size, we may have a chance of autodetecting the board revision on boot.
Click to expand...
Click to collapse
so you want some one with a 32A to flash a 32B rom and then get the dmesg? or the origanal 32A?
Killadude said:
so you want some one with a 32A to flash a 32B rom and then get the dmesg? or the origanal 32A?
Click to expand...
Click to collapse
I need the 32B kernel dmesg when running on a 32A board
Sorry for my english...
Ok, attached is dmesg-32A and dmesg-32B.
dmesg-32A is using a 32A ROM (from Amon_RA) and dmesg-32B is using cyanogens 3.9.7 (32B) ROM, both running on my Rogers 32A.
http://www.mediafire.com/?sharekey=87a195e0707967d00f83d91f6dff7c385048c6f0b06623c55621d66e282a0ee8
Resultant meminfo:
MemTotal: 80724 kB
MemFree: 1356 kB
Buffers: 316 kB
Cached: 24352 kB
SwapCached: 0 kB
Active: 30156 kB
Inactive: 38332 kB
Active(anon): 21284 kB
Inactive(anon): 23068 kB
Active(file): 8872 kB
Inactive(file): 15264 kB
Unevictable: 248 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 44080 kB
Mapped: 14460 kB
Slab: 5244 kB
SReclaimable: 812 kB
SUnreclaim: 4432 kB
PageTables: 2808 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 40360 kB
Committed_AS: 727200 kB
VmallocTotal: 172032 kB
VmallocUsed: 44404 kB
VmallocChunk: 82940 kB
hmmm... cool! it detects the SMI Size!
If this works, you will have about 150 or 160 mb of ram and no wifi (but the wifi thing is because of the wlan.ko)
Let's see what it does!
http://rapidshare.com/files/267026365/bootnew.img
do a fastboot boot bootnew.img, don't even bother flashing for now
If it doesn't work, it will either get stucked, or show the bootlogo, and after a while, it will reboot, then start booting the firmware stored on flash, but it won't break anything.
If it boots all the way, get a shell and do a
cat /proc/meminfo to see how much ram you get, or take a look at the dmesg