Here are my results for all three benchmarks right after I killed my Tasks
Im running KC1 and using tegrak Overclock to 1.3
Post what yah got.
btw, I noticed alot of other folks posting up multi-colored quandrant results. Is there another app that I should be using?
fknfocused said:
Here are my results for all three benchmarks right after I killed my Tasks
Im running KC1 and using tegrak Overclock to 1.3
Post what yah got.
btw, I noticed alot of other folks posting up multi-colored quandrant results. Is there another app that I should be using?
Click to expand...
Click to collapse
Quadrant Advanced provides the breakdown for each individual area (multi-colored).
http://slideme.org/application/quadrant-advanced
The thing about quadrant however, is it's not optimized or suited for the processor in our phones (Hummingbird). It was designed for snapdragon processors.
Additionally, Quadrant is sub par in my opinion on benchmark test apps (even though it's well known) because it does operations that aren't typical. If you get advanced you'll see that it does like 60% all IO operations, however thats a really strange test because it would be like you running a full on database on your phone.
So the scenarios and tests it runs are not representative of the scenarios you are going to use your phone to perform.
I like SmartBench better because it gives you an index for Gaming and Productivity.
Samsung Galaxy S 4G's will slightly outperform the snapdragon/adreno combo that you see in the MT4G etc. Very slight advantage. In regard to productivity it does well too.
Thanks for the info Joe, Im new to all of this and ran accross benchmarks so I tried it. It was something else to do while I await the almighty CWM. haha
fknfocused said:
Thanks for the info Joe, Im new to all of this and ran accross benchmarks so I tried it. It was something else to do while I await the almighty CWM. haha
Click to expand...
Click to collapse
No worries it was one of those things where I was bummed that my wife's MT4G was triple the score on my SGS4G so I investigated haha.
Yep same here . It's real close from what I can tell.
Found and interesting article and i thought i'd throw it out there. Basically when it comes to quadrants you will have two very polarizing point of views. You will have those that LOVE the program and they must test everything. And those that swear never to use it. But i found an interesting command line from the CMD prompt.
Code:
mount -t tmpfs tmpfs /data/data/com.aurorasoftworks.quadrant.ui.standard
Long story short it mounts the Quadrant Standard application on the RAM of the phone so you can get a true reading. Since the I/O is what bottle necks everything
I ran three test with the QS after three test i got a high of 2528, and a low of 1918. 610 point difference. Not to shabby for running CM7N75 O/C'd to 1516 with a performance governor
Then i mounted QS on the RAM of the phone and ran three test. The highest being 2997 and the lowest being 2785. 212 point difference.
just throwing out some food for thought.
Why do people always cry about Quadrant? those lil silly numbers means nothing I love how all the idiots who review the roms and make videos always do it and gives general public this idea that its real.
5th March 2011 said:
As I said before as syntactic benchmark means nothing as it does not translate in to real world performance. If someone wanted to really inflate it lol they wouldn't have to do much other than allocate it on tfs or ram as they will get higher score on I/O and R/W which will inflate the score which is very easy to spot as its never consistent with regular.
Click to expand...
Click to collapse
UsrBkp said:
Why do people always cry about Quadrant? those lil silly numbers means nothing I love how all the idiots who review the roms and make videos always do it and gives general public this idea that its real.
Click to expand...
Click to collapse
first and foremost - personal attacks aren't needed, wanted, or warranted.
I am sorry if you thought I personally attacked you but I assure you that was not the case. As I was speaking more in general, but what I said still stands.
UsrBkp said:
I am sorry if you thought I personally attacked you but I assure you that was not the case. As I was speaking more in general, but what I said still stands.
Click to expand...
Click to collapse
I know you weren't coming after me. But still in general. We are here to help one another and expand horizons. Not belittle each other.
UsrBkp said:
Why do people always cry about Quadrant? those lil silly numbers means nothing I love how all the idiots who review the roms and make videos always do it and gives general public this idea that its real.
Click to expand...
Click to collapse
I personally don't use quadrant, but any time someone talks about performance they can either post quadrant/linpack results, or they can say "it's really fast." You can't call someone who posts quadrant results an idiot unless you call anyone who has ever commented on a phone's performance an idiot. We talk about this stuff all the time, no need to demean anyone unless you have the one-stop end all of performance gauges.
I love it when people talk and talk and talk but they never listen. As once again the message was lost, which still exist on the original message. I don't know where he got the idea I was attacking him as that maybe some type of paranoia or delusion.
darinmc that will never happen due to the different hardware archstructure. Even when device is using ARMv7 the instruction set is interpreted differently from manufacture to manufacture. Great example is Snapdragon vs Hummingbird where NEON is utilized to improve the IOPs. If its allocated on top of davlik it can be cheated its simple as that.
UsrBkp said:
I love it when people talk and talk and talk but they never listen. As once again the message was lost, which still exist on the original message. I don't know where he got the idea I was attacking him as that maybe some type of paranoia or delusion.
Click to expand...
Click to collapse
Sorry for the typo
i guess that " n't " means alot. Sorry again.
Quadrant has become very popular with people to see where their phones stand performance wise, but at times I find it not to be the most accurate...Unlike Neocore or Linpack which I think are better ways to test GPU/CPU instead. The scoring system may need some work to make it balanced. Personally I think it needs work, earlier I tried Faux's Ginger rom and it scored about the same as the stock MT4G rom...This is clearly odd since the ginger is lighter and actually has a much much smoother experience and higher response times. Quadrant doesnt deal with real world usage. Feel free to disagree if you feel differently.
how about smartbench, is that any better than quadrant?
clarknick27 said:
how about smartbench, is that any better than quadrant?
Click to expand...
Click to collapse
From what I seen Quadrant favors Snapdragon while Smartbench favors Hummingbird. But try GLBenchmark thats what we mostly use as its more comprehensive.
"As I said before as syntactic benchmark means nothing as it does not translate in to real world performance." - HKM
UsrBkp said:
From what I seen Quadrant favors Snapdragon while Smartbench favors Hummingbird. But try GLBenchmark thats what we mostly use as its more comprehensive.
"As I said before as syntactic benchmark means nothing as it does not translate in to real world performance." - HKM
Click to expand...
Click to collapse
thanks Ill give it a shot
UsrBkp said:
From what I seen Quadrant favors Snapdragon while Smartbench favors Hummingbird. But try GLBenchmark thats what we mostly use as its more comprehensive.
"As I said before as syntactic benchmark means nothing as it does not translate in to real world performance." - HKM
Click to expand...
Click to collapse
Hahahaha, where have you been? Thought you retire drop by Sensation section help me out with the SamSux troll over there
Sent from my HTC Vision using XDA Premium App
epsix said:
Hahahaha, where have you been? Thought you retire drop by Sensation section help me out with the SamSux troll over there
Sent from my HTC Vision using XDA Premium App
Click to expand...
Click to collapse
LOL I see people still recognize me. I mean hopefully "they" don't know you know who, as ill try to keep low profile for now. Only reason I came back was due to the recent interest in "fail-pu" which I was trying to shed some lights in here few months back and got myself you know what in CM7NB thread. Hopefully "they" don't go crazy and start issuing you know what as originally ordered by the 2 heads. I am sure MT4G community will suffer if they do it, I mean here I hacked the mmcblk and posted the info and risked my device to help others yet Mr New.Sheriff wanted to show himself as the big man. Oooh well ill help ya much as I can and look in to the Sensation section.
What makes you think that mounting the application in RAM (thus minimizing I/O interactions) gives you a "true reading"? It's not like all your applications are stored in RAM. They access the file system also. So the speed of your phone's filesystem obviously affects the overall speed of the phone and should be included in a good benchmark. Unless you're only interested in comparing CPU/GPU speeds.
sundayhustler said:
What makes you think that mounting the application in RAM (thus minimizing I/O interactions) gives you a "true reading"? It's not like all your applications are stored in RAM. They access the file system also. So the speed of your phone's filesystem obviously affects the overall speed of the phone and should be included in a good benchmark. Unless you're only interested in comparing CPU/GPU speeds.
Click to expand...
Click to collapse
I don't think anyone in this thread said allocating it to tfs gives true reading. All the OP was doing was posting how one can easily hack the score. Its which we known ages ago but the point was when you make an standard everyone must follow it. So obviously if you allocate the whole ROM in RAM instead of NAND or SDCard the IOPs will always be higher.
If its software it can be altered simple as that and someone will always do so and try to pretend they have the legit score but for people like me we can easily tell what is real and what is fake. The legit max score verified by me was 3618 I think without any type of hack all I did was strip the rom and made it cleaner. Which you folks can get around 3200-3400 using AOSP with no problem. Now if you scoring 3800-4000 then well you know whats going on. I am not going to name folks but come on they aint fooling anyone.
neidlinger said:
Found and interesting article and i thought i'd throw it out there. Basically when it comes to quadrants you will have two very polarizing point of views. You will have those that LOVE the program and they must test everything. And those that swear never to use it. But i found an interesting command line from the CMD prompt.
Code:
mount -t tmpfs tmpfs /data/data/com.aurorasoftworks.quadrant.ui.standard
Long story short it mounts the Quadrant Standard application on the RAM of the phone so you can get a true reading. Since the I/O is what bottle necks everything
I ran three test with the QS after three test i got a high of 2528, and a low of 1918. 610 point difference. Not to shabby for running CM7N75 O/C'd to 1516 with a performance governor
Then i mounted QS on the RAM of the phone and ran three test. The highest being 2997 and the lowest being 2785. 212 point difference.
just throwing out some food for thought.
Click to expand...
Click to collapse
Off topic, but what font are you using in those screenies? And a link maybe?
UsrBkp said:
darinmc that will never happen due to the different hardware archstructure. Even when device is using ARMv7 the instruction set is interpreted differently from manufacture to manufacture. Great example is Snapdragon vs Hummingbird where NEON is utilized to improve the IOPs. If its allocated on top of davlik it can be cheated its simple as that.
Click to expand...
Click to collapse
That was pretty much my point, that an end-all doesn't exist. I'm no android pro, just think that if nothing can define performance then everything is game. It's all subjective anyway. My phone, for instance, is really really really fast. Yours is probably just really fast. Mine is 2 really's faster. It's how I roll.
darinmc said:
That was pretty much my point, that an end-all doesn't exist. I'm no android pro, just think that if nothing can define performance then everything is game. It's all subjective anyway. My phone, for instance, is really really really fast. Yours is probably just really fast. Mine is 2 really's faster. It's how I roll.
Click to expand...
Click to collapse
lol and that kind of the reason is why we have Quadrant in first place. Its all about false sense of security as the owner of the device is just simply fooling themselves. Now I am not sure if you know as all chips are different which actually depends on each wafer. But in perfect world with same defect rate as 2 let say you and me both have same phone. You running ROM X1.0 and I am also running ROM X1.0 and with the same settings. Now if you score 100 and I score 105 thats discrepancy. For it to be truly applicable it has to yield the same result over and over. Now thinking it would yield different result is known as insanity unless your fan of quantum mechanics that is yet ironically do to that we have chips today lol.
Think of it as 2 Fords they both running in the same road same model and one of them goes ahead. They are still bound by the same hardware but do to other variables it yields different results. Now those variables are not always predefined and it can be altered without any hardware modifications. But for it to be standard it has to be same. Now if you also have a Ford but have 300mph v8 under the hood you just cheated and inflated the score.
I am not sure if what I am saying is making any sense to anyone but to think your MT4G is better than someone else's is just crazy lol good luck putting that on ebay/cl saying you scored +300points extra on quadrant than other owners see how that goes for ya IRL.
my 3800 quadrant phone sold for eleventy million dollars thank you very much (it came with a case).
Hi would like to know if smartbench is accurate. Cause my games index point seems to be higher than other device by alot. 3095 pts
Sent from my Nexus S using XDA Premium App
dplate07 said:
Hi would like to know if smartbench is accurate. Cause my games index point seems to be higher than other device by alot. 3095 pts
Sent from my Nexus S using XDA Premium App
Click to expand...
Click to collapse
i dont trust smartbench. i get games indexes from 6000-1000 while overclocked at 1.544ghz. generally, benchmarks do not mean anything, but are nice to use just to compare devices.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Oh that's interesting. simms22, do you get that result every time you run the app or just occasionally? That is a bug that needs to be fixed!
Thanks.
Acei said:
Oh that's interesting. simms22, do you get that result every time you run the app or just occasionally? That is a bug that needs to be fixed!
Thanks.
Click to expand...
Click to collapse
i get a constant 5000-10000. but it ranges between 5000-10000 on every run. its not consistent around a certain score.
Acei said:
Oh that's interesting. simms22, do you get that result every time you run the app or just occasionally? That is a bug that needs to be fixed!
Thanks.
Click to expand...
Click to collapse
i figured theres something wrong, so i havent been publicizing these.
lol, heres a few more
Acei said:
Oh that's interesting. simms22, do you get that result every time you run the app or just occasionally? That is a bug that needs to be fixed!
Thanks.
Click to expand...
Click to collapse
Are you going to throw out all results and go reestablish a new baseline for every device?
Acei said:
Oh that's interesting. simms22, do you get that result every time you run the app or just occasionally? That is a bug that needs to be fixed!
Thanks.
Click to expand...
Click to collapse
morfic said:
Are you going to throw out all results and go reestablish a new baseline for every device?
Click to expand...
Click to collapse
i have to wonder the same the same thing. are ALL results invalid now? if there is a bug, then it cant be trusted with any device until the bug is squashed. who knows which results are valid or not.
Results are relatively easy to hash out, at least so far. These high index scores are caused by one or more sub-tests that somehow fails and ends up with unrealistic scores (well into millions). The server submit script rejects these results so they don't really make it to the DB (although I will check it out manually to make sure this is the case). I do have a script that runs nightly and looks for other anomalies as well.
As you can see in this link, you won't find your scores in the DB:
http://smartphonebenchmarks.com/ind...bench2011:Games&filter_cpu=all&filter_gpu=all
The real question though is why it is reporting these high values - I need to know which test fails and reports false numbers when it is performing the final calculation.
Going back to the same question - is it happening to you on every single test run, or does it only happen occasionally? You are the first person to report this so far.
Thanks again.
morfic said:
Are you going to throw out all results and go reestablish a new baseline for every device?
Click to expand...
Click to collapse
I would do this gladly if I find any evidences that the results contain garbage. So far though, the results either look good or they are close to infinite (very small % of the runs cause this though) so it's pretty easy to figure out which ones are wrong.
Acei said:
I would do this gladly if I find any evidences that the results contain garbage. So far though, the results either look good or they are close to infinite (very small % of the runs cause this though) so it's pretty easy to figure out which ones are wrong.
Click to expand...
Click to collapse
Do you have a way to proof that a 3600 when usual avg is 3400 is not caused by a minor glitch?
Why not change the app behavior to not submit anything when something failed?
How much can the current results be trusted if we rely on "i think nothing else but the 'obvious' ones went through"?
Acei said:
Results are relatively easy to hash out, at least so far. These high index scores are caused by one or more sub-tests that somehow fails and ends up with unrealistic scores (well into millions). The server submit script rejects these results so they don't really make it to the DB (although I will check it out manually to make sure this is the case). I do have a script that runs nightly and looks for other anomalies as well.
As you can see in this link, you won't find your scores in the DB:
http://smartphonebenchmarks.com/ind...bench2011:Games&filter_cpu=all&filter_gpu=all
The real question though is why it is reporting these high values - I need to know which test fails and reports false numbers when it is performing the final calculation.
Going back to the same question - is it happening to you on every single test run, or does it only happen occasionally? You are the first person to report this so far.
Thanks again.
Click to expand...
Click to collapse
i get them every time. but the lower the ghz kernel i use, the lower the score.
as far as i remember(i can be wrong) i had to submit my scores to the site to get added to the DB, but that was months ago. btw, i did get these scores a few weeks ago, but never posted them anywhere because they are just too ridiculous.
morfic said:
Do you have a way to proof that a 3600 when usual avg is 3400 is not caused by a minor glitch?
Click to expand...
Click to collapse
I can't guarantee that every single test run results stored in DB are accurate of course. That's one of the reason why I am continuously capturing more results to determine more accurate average results. But you are right, 3600 vs 3800, there is no way to prove that.
Why not change the app behavior to not submit anything when something failed?
How much can the current results be trusted if we rely on "i think nothing else but the 'obvious' ones went through"?
Click to expand...
Click to collapse
Yes, I have been doing this for a while now. Smartbench is an app written by a human after all. Just as other apps I have written since the past 15 years, this one has bugs too. I have been fixing bugs and I am determined to fix new ones that get discovered as well, that's why I am here right now.
The server-side scripts are there just in case more bugs are discovered on the client-side code. The goal is, as you mentioned here already, fix it at the root which is in the app itself.
simms22 said:
i get them every time. but the lower the ghz kernel i use, the lower the score.
Click to expand...
Click to collapse
Hmm, that is indeed interesting.
Did you get this from day 1 or after some mods?
as far as i remember(i can be wrong) i had to submit my scores to the site to get added to the DB, but that was months ago. btw, i did get these scores a few weeks ago, but never posted them anywhere because they are just too ridiculous.
Click to expand...
Click to collapse
Yes those scores are too ridiculous and I do intend to find out why it is reporting them, on your particular phone and not on others.
Do you let them run without any interruptions?
A while back, some have found a way to "cheat" the app by performing certain actions so I had to add some code to check for those.
Going back to your run, I just had a look in the DB. I can see one of your runs - you got high scores because your JellyFish score was really high. Did your Jelly Fish test run properly from the beginning to the end? All I'm doing here is capturing a time before the test and another one at the end, down to a nano-second accuracy. I could easily reject these results on the client side and report invalid runs but I would prefer to find out why it is causing this in the first place...
Hi guys,
I have just released a v1.2.1 which fixed the problem described here.
If you are interested, you can check out the following article (I wrote it) here:
http://bit.ly/lfxS1a
Thanks!
EDIT: For some strange reason, I don't see the latest version in the Android Market yet. In the past, the update has always been instantaneous. I hope this one gets updated soon.
EDIT2: I see the update now!
Acei said:
Hi guys,
I have just released a v1.2.1 which fixed the problem described here.
If you are interested, you can check out the following article (I wrote it) here:
http://bit.ly/lfxS1a
Thanks!
EDIT: For some strange reason, I don't see the latest version in the Android Market yet. In the past, the update has always been instantaneous. I hope this one gets updated soon.
EDIT2: I see the update now!
Click to expand...
Click to collapse
Coming to think of it, we saw similar in the past, where high scores caused a rejection of the scores even if everything finished through.
Why not add a failed score as a zero, total number of scores / number of scores == avg
Not just succeeded scores, this way devices where tests fail will score low, as would be expected.
BUT, if tests fail, can you be verbose what failed, how far it got before it failed?
I have not seen 2011 complete on my phones in a long time (i mean not gotten a "valid score") and i would like to know where the problem is. And none of those with gfx show signs of early exits, numerical tests i can't get visual clues from.
Thanks,
Daniel
Acei said:
Hi guys,
I have just released a v1.2.1 which fixed the problem described here.
If you are interested, you can check out the following article (I wrote it) here:
http://bit.ly/lfxS1a
Thanks!
EDIT: For some strange reason, I don't see the latest version in the Android Market yet. In the past, the update has always been instantaneous. I hope this one gets updated soon.
EDIT2: I see the update now!
Click to expand...
Click to collapse
i just updated and tested. im still getting high scores.for example, overclocked to 1.1ghz i just scored a 12000+ on games. lol, thats just funny. but i do see the jellyfish test as faster than id expect. but not that much. interesting to note, i had a few failed benchmarks this time. but more high scores after.
morfic said:
Coming to think of it, we saw similar in the past, where high scores caused a rejection of the scores even if everything finished through.
Why not add a failed score as a zero, total number of scores / number of scores == avg
Not just succeeded scores, this way devices where tests fail will score low, as would be expected.
Click to expand...
Click to collapse
I'm pretty sure I'll get arguments from both sides on this. Some units will overclock better than others (even if they are the same devices/models), so for those lucky owners with more headroom will want to see results that don't include failed results. Or at least that is my opinion. At the end of the day, you guys will need to tell me which makes more sense.
In the worst case, I can implement both, and allow you to select which "mode" you want to see.
BUT, if tests fail, can you be verbose what failed, how far it got before it failed?
I have not seen 2011 complete on my phones in a long time (i mean not gotten a "valid score") and i would like to know where the problem is. And none of those with gfx show signs of early exits, numerical tests i can't get visual clues from.
Thanks,
Daniel
Click to expand...
Click to collapse
This is definitely possible. There are currently several checks I do to see if test runs are valid:
1. If an app has lost its focus, I assume the test is invalid. This was done to prevent people from switching out of the app during the test run and end up with some funky scores. Somehow users were getting interesting scores!
2. If any of the test suit throws exceptions (i.e. crashes), then I assume the test is invalid. As you mentioned here already, it is possible to find out at least which test suit failed.
It appears that it is still possible to end the test early without crashing as I have witnessed in this thread. So I will look further into other possibilities as well.
simms22 said:
i just updated and tested. im still getting high scores.for example, overclocked to 1.1ghz i just scored a 12000+ on games. lol, thats just funny. but i do see the jellyfish test as faster than id expect. but not that much. interesting to note, i had a few failed benchmarks this time. but more high scores after.
Click to expand...
Click to collapse
Very interesting! So that means your result is caused by not one but two issues. I may have to go as far as checking to see if it has completed rendering the right # of frames during the test.
simms/morfic, would you guys mind testing for few more scenarios for me? I can produce a special build to do some further experiment.
Thanks much!
Acei said:
Very interesting! So that means your result is caused by not one but two issues. I may have to go as far as checking to see if it has completed rendering the right # of frames during the test.
simms/morfic, would you guys mind testing for few more scenarios for me? I can produce a special build to do some further experiment.
Thanks much!
Click to expand...
Click to collapse
I've always been a fan of your work actually being on the forums and working bugs out. There aren't many devs dedicated like you so anything you want to test, hit me up as well, fortunately/unfortunately mine hasn't shown one of these erroneous scores so I'm not sure if my phone will help, my game index score is always in the 3000 range.
Sent from my Nexus S 4G using Tapatalk
dreamsforgotten said:
I've always been a fan of your work actually being on the forums and working bugs out. There aren't many devs dedicated like you so anything you want to test, hit me up as well, fortunately/unfortunately mine hasn't shown one of these erroneous scores so I'm not sure if my phone will help, my game index score is always in the 3000 range.
Sent from my Nexus S 4G using Tapatalk
Click to expand...
Click to collapse
Thanks for the offer.
Even a positive confirmation that the latest build works is helpful.
I just did the update and notice quadrant scores in the 1300-1600 range.....my old phone was pulling 2000+ no problem, anyone else having the same results?
makkunbakkun said:
I just did the update and notice quadrant scores in the 1300-1600 range.....my old phone was pulling 2000+ no problem, anyone else having the same results?
Click to expand...
Click to collapse
There's something that HTC did that's causing this, other devices on ginger are also getting low values...But don't worry b/c the score doesnt mean anything...Quadrant isn't very consistent and is not needed. It may not work with 2.3.4, it was updated months ago for ginger, but that was for 2.3, not the newer .x versions. Which may explain the numbers you get. If you use linpack, you'll see the cpu score is about the same, ~40's, just like on froyo, but the gpu portions of the test are probably causing the quad score to be low....Use something like nenmark or vellamo and you'll see the phone is fine...