So, I've been at this for about 48 hours now (not continuously, but closer than you might think) and I figured I should take a break from modifying project files and puzzling over alignment issues to discuss the project, share some of the problems I've been having and ask if anybody can help, and so on.
The general idea is "Chromium build for Windows (on x86/x64) and build on ARM (for Linux), so there must be a way to build it for Windows on ARM". For the most part, that even looks like it's true. Probably at least 80% of 654 Visual Studio projects (no, that's not a joke) either build just fine with only minor amounts of work, or are things that we don't actually need (I'll try building the test suites... once everything else builds!!)
Areas that have given me problems (caution: some chance of brief rants ahead):
v8. Less than you might think, though. Setting the flags for Arm seems to have been enough.
Sandbox. There's a fair bit of thunking coded in assembly going on in the sandbox for x86. Not sure what's up with it (I don't know exactly how the Chromium sandbox works) but it'll have to come out or be replaced. The Linux (including ARM) sandbox seems to be SELinux-based, which doesn't help at all.
Native Client (NaCl). I think all the assembly is in test code, though, so I may just boldly #ifdef if all away.
libjpg-turbo (libjpg). Piles of carefully optimized assembly... for x86 and x64. There is a set of ARM assembly (for Linux) that Visual Studio won't compile, but something else might... or I may tweak until it works. Of course, I could also just accept the speed hit and use the version of libjpg implemented in nice, portable C.
Anything where the developers tried to use some SSE to speed things up. I may be able to replace it with NEON code, or I may just remove it and hope **** doesn't break. We'll see.
Inline assembly in general. Even when it's ARM assembly, Visual Studio / CL.exe don't want anything to do with it (__asm is apparently now an invalid keyword). I suspect I'll have to just pull the assembly out into stand-alone functions in their own files, then compile them to object files and link them back in later. If I can figure out the best way to do this (for example, I'll want to inline the asm functions) then it shouldn't impact performance. Seriously though, I kind of hate inline assembly. I can read assembly just fine, but I'm usually staring at it in a debugger or disassembly tool, not in the middle of source code I'm trying to build...
Everywhere that the current state of the CPU is cared about (exception and crash handlers, in particular) because the CONTEXT structure is, of course, CPU-specific. They're pretty easy to get past, though.
Low-level functions, like MemoryBarrier. Fortunately, it's implemented in ntdll.h... but as a macro, which breaks at least half the places it's referenced. Solution: where it breaks things, undefine the macro and just have it be an inline function that does what the macro did.
Running out of memory. Not even joking... well, OK, a little bit. I've got 32GB; I won't actually run out. Both Visual Studio and cl.exe do at times, though!. Task Manager says VS is currently using 1,928 MB, and before I restarted it, it broke 2.5GB private working set. Pretty good for a program that for some reason is still 32-bit...
Goddamn compiler flags. Seriously, every single project (I mentioned there are over 600, right?) has its LIBPATHs hardcoded to point at x86. Several projects have /D:_X86_ or similar (that's supposed to be set by the build tools, not the user, you idiots...) which plays merry hell with the #ifdef guards. Everything has /SAFESEH specified, not in the actual property table where the IDE could have removed it (unneeded and invlaid on ARM) but in the "extra stuff we'll pass on the build command line" field, which means every single .EXE/.DLL project must be modified or the linker will fail.
My current biggest goal is the JPG library; nobody wants to use a browser without it. After that, I'll tackle the sandbox, leaving NaCl for last... well, last before whatever else crops up.
Anyhow, thoughts/comments/advice are welcome... in the mean time, I'm going to go eat something (for the first time in ~22 hours) and then get some sleep.
Kudos for having the patience to look though this monster.
It's my understanding that NaCl is still a pretty niche thing at the moment. Is it possible to easily either disable it or completely hack it out, or do other more critical parts of Chromium now depend on it?
I don't think anything truly depends on it. I'll look in the VS dependency hierarchy and see how many things list it, and how awful it would be to remove them.. after I get the other stuff working. I may pass on the sandbox as well, if possible; it makes the security guy in me cringe something awful, but as they say, shipping is a feature..
great
Please make that happen !
Working on it! I've gotten over half of the projects to build and link, but some other stuff is adamantly refusing to work. I'm beginning to suspect I'll need to work from the other direction - rather than starting at the bottom and building all the dependencies, then combining them into browser components, and then eventually combining all the components into a complete piece of software, I may have to work from the top, removing components until the whole thing builds (at which point it will likely be useless, or all-but) and then seeing what I can add back in. I thought it would be faster to just assume everything can be made to work and only exclude something if it proved intractable, but at this point I've got a ton of very small components and almost no ability to combine them.
It would also help if VS was better at managing such truly immense tasks. For example, I have no simple graph of what all is and is not building, so I'm being forced to manually map that onto the VS dependency tree and see what is blocking a given component from building successfully, and how much is dependent upon it, one erroring project at a time (and there are a *lot* of erroring projects - my last attempt to build any substantial part of the system saw 50 of 400 projects fail).
GoodDayToDie said:
Working on it! I've gotten over half of the projects to build and link, but some other stuff is adamantly refusing to work. I'm beginning to suspect I'll need to work from the other direction - rather than starting at the bottom and building all the dependencies, then combining them into browser components, and then eventually combining all the components into a complete piece of software, I may have to work from the top, removing components until the whole thing builds (at which point it will likely be useless, or all-but) and then seeing what I can add back in. I thought it would be faster to just assume everything can be made to work and only exclude something if it proved intractable, but at this point I've got a ton of very small components and almost no ability to combine them.
It would also help if VS was better at managing such truly immense tasks. For example, I have no simple graph of what all is and is not building, so I'm being forced to manually map that onto the VS dependency tree and see what is blocking a given component from building successfully, and how much is dependent upon it, one erroring project at a time (and there are a *lot* of erroring projects - my last attempt to build any substantial part of the system saw 50 of 400 projects fail).
Click to expand...
Click to collapse
I thinkt tht is a mutch better taktic and mutch less frustrading.
I would love to see just a minimal version of it. After that all the small componens can follow.
50 of 400 is pretty good i think. Better then i expected
Bear in mind that the entire thing is 650 projects. If 50 fail at that level, many of the higher-level ones (dependent upon the lower-level) will fail too. I'll see what I can do. I may or may not be able to get v8 actually working (without it, the JS speed will be very bad, think IE8 at best) and I may have to fall back to the legacy libjpeg (which will cut JPEG render speeds by at least a factor of 2). Skia (2D drawing library used by Chrome) has a bunch of assembly optimizations that I need to get it to use the Arm version of instead. There's a couple of total hacks with the library files I've had to pull, which may or may not result in a working final build. We'll see.
GoodDayToDie said:
Bear in mind that the entire thing is 650 projects. If 50 fail at that level, many of the higher-level ones (dependent upon the lower-level) will fail too. I'll see what I can do. I may or may not be able to get v8 actually working (without it, the JS speed will be very bad, think IE8 at best) and I may have to fall back to the legacy libjpeg (which will cut JPEG render speeds by at least a factor of 2). Skia (2D drawing library used by Chrome) has a bunch of assembly optimizations that I need to get it to use the Arm version of instead. There's a couple of total hacks with the library files I've had to pull, which may or may not result in a working final build. We'll see.
Click to expand...
Click to collapse
the v8 engine ( used in nodejs ) has been ported to ARM :
I still can't link : htt p://ww w.it-wars.com/article305/compiler-node-js-pour-arm-v5
perhaps it will help you
Edit : oups, I just see that another great user of this forum made the port of nodejs to RT
Yep... but they did it without v8. That's not an encouraging result, but I feel like I'm so close...
Is there a GitHub repo so we can help or track the progress of the project ?
Sorry, not at present. There probably should be. The sheer size of the codebase is incredible (about 2.4GB) and having some way to share it practically would be good.
Also, I suspect this would go a lot faster if I don't have to repeat the work of others. I know that there's a working Webkit DLL out there, for example (though with several features, including the V8 JS engine, missing) and if I could get my hands on that it would drastically reduce the number of additional components I need to build. Currently I'm working on the sandbux, but expect that I will need to rip the whole thing out and basically have the browser run as though it was always passed the --no-sandbox parameter, at least for the first build. Too damn much assembly.
http://www.engadget.com/2013/01/22/google-chrome-native-client-arm-support/
This wouldn't have any impact on this project, would it?
Sent from my SCH-I535 using xda-developers app, complete with annoying signatures.
It probably means that NaCl on Windows RT will be possible in the future. At present, I'm cutting it out of the build - too much x86-specific stuff there to port it over myself, and it owuldn't be able to run x86-compiled NaCl code anyhow.
You might have bit off more than you could chew. It'd better if you put your current progress under version control on some public site so that other people may be able to help you.
It's a big and complex project. You are taking a lot of time, and understandably so. But just open up to other people and you could get this done faster.
Yeah, this is probably true. My life also got unexpectedly *busy* in the last week; a couple weeks ago I had many times as much free time as I do now, and so porting has slowed down.
My upload speed would take ages (literally probably at least a day of solid activity; it's embarassingly slow) to push the full source anywhere, but I may make the effort anyhow. I'll have to post it somewhere for GPL compliance in any case...
You may upload only the diff files, they'll probably be smaller then the whole distribution.
Not to pour cold water on you however, IE10 is already faster than the latest Chrome build in Windows Phone, Windows 8.
I don't see the point of this.
I have personally jumped from IE8 > FF > Chrome and finally back to IE10 over the years depends on its usability, smoothness, speed, etc
Speed isn't the only reason to use a browser. I actually prefer IE myself, but there are some things that other browsers do better than it (in the case of Chrome, parts of HTML5, the syncing across Google services, etc.) Also, Chrome gets updated far more often than IE; IE9 was equal with Chrome on speed at its release, and was far behind by the time IE10 came out.
The reason for this project, though, is a mixture of interest in what it takes, and a desire to benefit the community. Microsoft has deeped that only software which they have blessed may run on the Windows RT desktop. I disagree, and have chosen (among several other things) to port a web browser because I feel that it's important for users to have choice.
LastBattle said:
Not to pour cold water on you however, IE10 is already faster than the latest Chrome build in Windows Phone, Windows 8.
I don't see the point of this.
Click to expand...
Click to collapse
Some websites do not get along with the trident rendering engine. Some webdevs are so "Oh f*** IE I don't care" and block access to features just because it is IE. I have experienced this first hand on IE10 on my surface where it tells me to come back when I have a decent browser, only to not have the choice to do that.
This really isn't the webdevs fault either, for years IE was the scum of the internet, only recently has IE caught up to the rest of the browsers (and in my opinion exceeded some) but the years of IE being bad have left a lot of disjointed webdevs who won't even consider giving the latest IE a chance.
Related
This all works on stock/nonroot phones
I got Mono running on Android.
http://www.koushikdutta.com/2008/12/mono-on-android-success-at-last.html
Started working on Java/C# interop and found out that DLR works on Mono:
http://www.koushikdutta.com/2009/01/microsoft-dlr-and-mono-bring-python-and.html
As a result, you can write applications in Python and Ruby on Android too now.
Anyhow, if anyone else is interested in working on this project with me, please let me know! I've already gotten all the relevent source hosted at Google Code: http://code.google.com/p/androidmono. Basically the next bit of work involves implemeting a Java interop using the DLR.
Nifty.
As for Dalvik & JIT, I think dexopt already replaces some heavy usage calls with inline native code. Hopefully dalvik vm will get full JIT in the future?
This makes me very happy!!!
Thank you for your work!!
jashsu said:
Nifty.
As for Dalvik & JIT, I think dexopt already replaces some heavy usage calls with inline native code. Hopefully dalvik vm will get full JIT in the future?
Click to expand...
Click to collapse
It doesn't do any inline native replacements: dexopt optimizes the dex file. Which includes inline dex byte code replacements. There is no JIT at all, but Google said it would be something definitely on the horizon. Personally I think DEX is a pretty stupid move on Google's part; they could have just gone with CIL-- and write a Java compiler for that instead. The Mono JIT compiler works on many platforms; so V1 of Android could have been running JIT compiled native code with that route... which is an order of magnitude better in performance.
I'm going to be doing some performance comparisons of Mono vs Dalvik; Mono will obviously win, but it will be interesting to see the margin. I'll also experiment with binding Mono to the Android runtime to create Android applications in C#.
Optimization
Virtual machine interpreters typically perform certain optimizations the first time a piece of code is used. Constant pool references are replaced with pointers to internal data structures, operations that always succeed or always work a certain way are replaced with simpler forms. Some of these require information only available at runtime, others can be inferred statically when certain assumptions are made.
The Dalvik optimizer does the following:
For virtual method calls, replace the method index with a vtable index.
For instance field get/put, replace the field index with a byte offset. Also, merge the boolean / byte / char / short variants into a single 32-bit form (less code in the interpreter means more room in the CPU I-cache).
Replace a handful of high-volume calls, like String.length(), with "inline" replacements. This skips the usual method call overhead, directly switching from the interpreter to a native implementation.
Prune empty methods. The simplest example is Object.<init>, which does nothing, but must be called whenever any object is allocated. The instruction is replaced with a new version that acts as a no-op unless a debugger is attached.
Append pre-computed data. For example, the VM wants to have a hash table for lookups on class name. Instead of computing this when the DEX file is loaded, we can compute it now, saving heap space and computation time in every VM where the DEX is loaded.
All of the instruction modifications involve replacing the opcode with one not defined by the Dalvik specification. This allows us to freely mix optimized and unoptimized instructions. The set of optimized instructions, and their exact representation, is tied closely to the VM version.
Most of the optimizations are obvious "wins". The use of raw indices and offsets not only allows us to execute more quickly, we can also skip the initial symbolic resolution. Pre-computation eats up disk space, and so must be done in moderation.
There are a couple of potential sources of trouble with these optimizations. First, vtable indices and byte offsets are subject to change if the VM is updated. Second, if a superclass is in a different DEX, and that other DEX is updated, we need to ensure that our optimized indices and offsets are updated as well. A similar but more subtle problem emerges when user-defined class loaders are employed: the class we actually call may not be the one we expected to call.
These problems are addressed with dependency lists and some limitations on what can be optimized.
Click to expand...
Click to collapse
Koush said:
Personally I think DEX is a pretty stupid move on Google's part; they could have just gone with CIL-- and write a Java compiler for that instead. The Mono JIT compiler works on many platforms; so V1 of Android could have been running JIT compiled native code with that route... which is an order of magnitude better in performance.
I'm going to be doing some performance comparisons of Mono vs Dalvik; Mono will obviously win, but it will be interesting to see the margin. I'll also experiment with binding Mono to the Android runtime to create Android applications in C#.
Click to expand...
Click to collapse
Google had different objectives, they didn't go after maximum performance. Remember, that handsets have different constrains than desktops and laptops. So they went after minimizing RAM usage (byte code interpreter => maximum possible sharing of read-only memory pages among processes) and battery life. Performance had to be acceptable, not priority.
You would not be able to fit everything into RAM if you used Mono and you would get the patent problems with Net/Mono/etc as a bonus.
lu_tze said:
Google had different objectives, they didn't go after maximum performance. Remember, that handsets have different constrains than desktops and laptops. So they went after minimizing RAM usage (byte code interpreter => maximum possible sharing of read-only memory pages among processes) and battery life. Performance had to be acceptable, not priority.
You would not be able to fit everything into RAM if you used Mono and you would get the patent problems with Net/Mono/etc as a bonus.
Click to expand...
Click to collapse
.NET, C#, IL, et al are all ECMA standards. Mono is LGPL/GPL. There are no patent or licensing issues with it that is unfamiliar to the OHA. They reuse plenty of open source projects.
An interpreter is not power efficient OR performant, simply due to the fact is it doing the 10 times as much work to do the same thing as native code. In addition, Mono features an Ahead Of Time compiler (AOT) that would let you compile everything to native code before it even hits the phone (or just once, and cache it). Most of Android's power and memory optimizations currently comes from Google's application life cycle (activities can be killed and resumed at the system's whim) -- that has nothing to do with Dalvik. I'm not criticizing the API or the implementation, just the runtime.
They could have spent their time making the Mono runtime play nicely with the shared memory subsystem.
I'm rebuilding mono with a minimal configuration to check out the disk and memory footprint.
Koush said:
Not that most of you will care, but I got Mono running on Android.
Click to expand...
Click to collapse
I'm a noob.... how can I install this? Very good Job Kush!
pic.micro23 said:
I'm a noob.... how can I install this? Very good Job Kush!
Click to expand...
Click to collapse
I haven't released anything yet. I'm trying to figure out how to statically link all it's dependencies, minimize the size, bind to the Android runtime, convert DEX to CIL and then CIL to ARM, and all sorts of other goodness. Basically a lot of experimenting to do before anything is "released". It's just in proof of concept phase right now.
Dalvik sucks
http://www.koushikdutta.com/2009/01/dalvik-vs-mono.html
Koush said:
Dalvik sucks
http://www.koushikdutta.com/2009/01/dalvik-vs-mono.html
Click to expand...
Click to collapse
Lol. nice article Koush. It's surprising mono is that much faster
Koush said:
I haven't released anything yet. I'm trying to figure out how to statically link all it's dependencies, minimize the size, bind to the Android runtime, convert DEX to CIL and then CIL to ARM, and all sorts of other goodness. Basically a lot of experimenting to do before anything is "released". It's just in proof of concept phase right now.
Click to expand...
Click to collapse
Just read the dalvik vs mono article. It's certainly interesting work. I agree that by ignoring JIT they're certainly not going after the most beneficial optimizations. That said, I don't think it's something they've completely excluded from future implementation.
I think you should consider trying to get Mono added as an external project. If nothing, having unofficial support for a vm which supports C#/CIL could bring in a significant amount of developer interest from the WinMo dev community. The coretech team would be the folks to set up a new project.
I've now gotten mono working on all G1s. You don't need Debian OR root. Still a couple kinks to work out, but I have it on the market for anyone interested in playing with it. More information at the link below.
http://www.koushikdutta.com/2009/01/mono-for-android-now-available-on.html
I thought this was only a platform for development but its made my g1 much faster and reduced memory from steel and stock browsers as well. My market is still 12 mb and mono is about 11 mb. Is this normal?
Also everytime, I run mono, does it do the same thing it does the first time it was installed and opened?
Sorry, i know nothing about mono but i can tell its definitely optimizing the performance on the g1 though.
great work koushe,
hbguy
This awesome, I have been waiting to see how this all turned out after reading your first post about it a few days ago.
hbguy said:
I thought this was only a platform for development but its made my g1 much faster and reduced memory from steel and stock browsers as well. My market is still 12 mb and mono is about 11 mb. Is this normal?
Also everytime, I run mono, does it do the same thing it does the first time it was installed and opened?
Sorry, i know nothing about mono but i can tell its definitely optimizing the performance on the g1 though.
great work koushe,
hbguy
Click to expand...
Click to collapse
Koush would know more than I would, but installing mono shouldn't affect everything else on Android. It's not like everything is suddenly using mono instead of dalvik. I suspect you have a strong case of the placebo effect
hbguy said:
I thought this was only a platform for development but its made my g1 much faster and reduced memory from steel and stock browsers as well. My market is still 12 mb and mono is about 11 mb. Is this normal?
Also everytime, I run mono, does it do the same thing it does the first time it was installed and opened?
Sorry, i know nothing about mono but i can tell its definitely optimizing the performance on the g1 though.
great work koushe,
hbguy
Click to expand...
Click to collapse
Yeah, you're just imagining things I haven't even attempted like DEX->CIL yet.
The first APK of Mono is quite large though. I've updated it with a number of bug fixes and also am making it use eglib now. This trimmed the size by a few MB. Getting Mono to work with Bionic might not be possible... (that would trim off another 2MB).
Once again, the APK is just a developers release... something to play with and test.
I have been messing with mono on my G1.
Is it safe to say it will only work with command line apps?
I got my own hello world and a few other things running, but if I try and run any sort of gui I get errors.
Yes, only command line stuff will work. WinForms will not work on Android.
However, you should be able to get OpenGL ES working via PInvoke. I haven't tried it, but it should work just fine.
Koush said:
Yes, only command line stuff will work. WinForms will not work on Android.
Click to expand...
Click to collapse
That is what I thought, I think PInvoke is just a bit out of my skill set.
Thanks for the work none the less.
I got mono building in the Android build environment, and the Mono team accepted a patch to make it work on Android. There's also some changes external to Mono which can be found at the androidmono google code repository:
http://code.google.com/p/androidmono/
I have a simple question about Android in which I have not found the simple answer to... (Although I think I know, I just want some clarification). I recently switched over to Windows Phone 7 because of various reasons, I will not name them here as that is an entirely different subject, however one of the reasons i switched was because of overall responsiveness of the OS. Why does Android's touch response feel sooo clunky? Yeah transitions and app launches are nice and quick, but I mean like pinch-to-zoom, and scrolling... I have played with the latest and greatest both rooted (with and without custom rom) and non rooted (with or without OEM UI), Motorola Xoom, Atrix 4G whatever is being claimed latest and greatest. But no matter what they all have the same touch response lag no matter what. This, believe it or not, is a major deal breaker for me, and before the majority of you speak, I'll speak for you; "why is something so simple and small, barely even considered a nuisance, be such a nuisance?" for me, i love fluidity, so, it just is. At this question however i do retort; if its such a "simple" or "small" nuisance, why can't it "simply" be coded to feel as fluid as Windows Phone 7, or iOS?
Luisraul924 said:
I have a simple question about Android in which I have not found the simple answer to... (Although I think I know, I just want some clarification). I recently switched over to Windows Phone 7 because of various reasons, I will not name them here as that is an entirely different subject, however one of the reasons i switched was because of overall responsiveness of the OS. Why does Android's touch response feel sooo clunky? Yeah transitions and app launches are nice and quick, but I mean like pinch-to-zoom, and scrolling... I have played with the latest and greatest both rooted (with and without custom rom) and non rooted (with or without OEM UI), Motorola Xoom, Atrix 4G whatever is being claimed latest and greatest. But no matter what they all have the same touch response lag no matter what. This, believe it or not, is a major deal breaker for me, and before the majority of you speak, I'll speak for you; "why is something so simple and small, barely even considered a nuisance, be such a nuisance?" for me, i love fluidity, so, it just is. At this question however i do retort; if its such a "simple" or "small" nuisance, why can't it "simply" be coded to feel as fluid as Windows Phone 7, or iOS?
Click to expand...
Click to collapse
I will try to answer since I've been using android before. android now as I believe is still in development stage. especially because it started from open source, where many developers get involved to participate in android development. unlike windows or the IOS platform. development is only done by the company itself through microsoft and apple. Except for third-party application development
android is a system (for run smoothly) with very powerful hardware. so that the source code would require a very complicated of encoding. Its a very difficult job to sync between the needs of software with hardware which is available. and vice versa. in an application such as pinching and scrolling there is more than one command which contains a lot of code. and should be remembered that this is a system. which are all related to each other for the overall operations to run smoothly based on the minimum demand of the hardware required. if there is one character in which is wrong of encoding or difference may cause the application not running properly.
for high-end android device such as Xoom, atrik 4G I'm sure the hardware is not an issue. I'm sure it was more caused by the complexity of encoding in one of the applications listed that is running inside in the whole operating system, making it not running smoothly. because so many commands which must be running at the same time is what make pinching and scrolling activity to be "clunky" like you said. you can differentiate by turning off the internet connection or turn off unnecessary applications running in the background. But I'm sure very soon android operating system will have a system which is more stable and efficient in encoding such as those held by the windows or apple.
My answer may be added by other members of the more expert in these matters. as a newbie, i am just trying to help based on the knowledge I had acquired over the years. CMIIW
Yeah I figured it would be something like that, I owned a Droid (1st gen.) and I had multiple setups from completely stock to my favorite, Cyanogenmod (always on the latest stable build, although I've already flashed CM7 RC2 and its probably the fastest its ever been at 800 MHz) everything was perfect except; scrolling and pinch-to-zoom. The scrolling is almost there, it actually lags for a bit but if I leave my finger on the page, it locks on to that position and stays there, but once I lift it to continue scrolling down or up it'll lag a bit again. The pinch zooming is just horrible no matter what. Unfortunately, given the nature of open source, and coding software in general, there is no such thing as "finished" software, so since this is open source, and the software is basically written to run on "nearly" whatever device you choose to flash it on, i don't think that problem will ever be solved. However, if Android does eventually reach that richness of responsiveness, then i will more than gladly switch back.
issues of a system running smoothly is different from one device to another device.
due to the wide variety of different android devices that causes the emergence of issues on the system stability. it was time to google as the main developer sets the standards for the development of next android os. while there is no standardization of hardware is set by google. it will be very difficult for other developers to write code/adjust performance in the operating system command. all this time writing code is must be adapted to the device from vendor itself. This will bring up the differences result of writing the code on other devices from another vendors (competitors). so if we running bencmark test or head to head test on both devices from different vendor the result will not be the same.
and if there will be a standarization set by google i believed it will not againts a spirit of an open source
I think the hardware that the WinCE (well...the shoe still fits) and Android phones are made on is essentially the same, in terms of the CPU power, the actual CPUs, the memory and the various other systems (graphics, etc.). Maybe not identical but overlapping classes and performance.
I haven't played with the new WinPhones but have noticed that every Android phone, no matter how fast and how "bare" factory, sometimes goes out to lunch. Apparently that's just the way the OS is written, it sometimes goes off to do other things internally (loading code? checking hardware states?) and you can't do anything except wait for it to come back.
But then again, almost every OS does that at times, including the main Windows OSes. That's just how they are done these days. If you had a cell phone fifteen year ago, you could turn it on and dial NOW. With any of the new cell phones? Can you do a cold power up and have a functioning phone in less than 30 seconds? Uh, no. But they call that progress, because you rarely have to power them off these days.
Every OS has tradeoffs, if the WinPhone makes you happier, by all means do it.
Rred said:
I think the hardware that the WinCE (well...the shoe still fits) and Android phones are made on is essentially the same, in terms of the CPU power, the actual CPUs, the memory and the various other systems (graphics, etc.). Maybe not identical but overlapping classes and performance.
I haven't played with the new WinPhones but have noticed that every Android phone, no matter how fast and how "bare" factory, sometimes goes out to lunch. Apparently that's just the way the OS is written, it sometimes goes off to do other things internally (loading code? checking hardware states?) and you can't do anything except wait for it to come back.
But then again, almost every OS does that at times, including the main Windows OSes. That's just how they are done these days. If you had a cell phone fifteen year ago, you could turn it on and dial NOW. With any of the new cell phones? Can you do a cold power up and have a functioning phone in less than 30 seconds? Uh, no. But they call that progress, because you rarely have to power them off these days.
Every OS has tradeoffs, if the WinPhone makes you happier, by all means do it.
Click to expand...
Click to collapse
I agree with you, I do like both OS's for their own benefits, currently I do like WP7 better than Android and keeps me "happy". However if you notice; that's not my prime motive in starting this thread, I didn't come here to say one is better than the other. I just want to know why those two simple things (scrolling and pinch zooming) are not fluid on Android. You can't use the excuse that it's different hardware because Microsoft is playing that trick too. You can't use the "its busy doing other things" excuse either, while WP7 doesn't have multi-tasking, iOS does (somewhat) so it can go "do" something else but will still feel fluid. In a multi-OEM environment it is up to the OEM to optimize it for the device it runs on, which is why it baffles me that even Sense and MotoBlur and others make performance decline a bit and still has the lag. Shouldn't it be the opposite?
Nothing? So no one can tell me why Android's responsiveness (scrolling, pinch-zooming) sucks?
Luisraul924 said:
Nothing? So no one can tell me why Android's responsiveness (scrolling, pinch-zooming) sucks?
Click to expand...
Click to collapse
The answer is quite simple (and the above replies are miles off the mark). Hardware acceleration.
WP7 has it, Android doesn't.
FloatingFatMan said:
The answer is quite simple (and the above replies are miles off the mark). Hardware acceleration.
WP7 has it, Android doesn't.
Click to expand...
Click to collapse
So the hardware acceleration runs throughout the entire OS? I thought it was mainly just the XNA and Silverlight stuff that was accelerated (I do believe those are different than native OS code, as Microsoft isnt allowing developers to write apps with native code. Future compatibility issues I guess)
Of course it's the entire OS. Why do you think MS's minimum spec stipulations are so high? This is what Windows Mobile was so plagued with, and how MS fixed that problem.
Luis-
"So the hardware acceleration runs throughout the entire OS?"
It isn't so much that the hardware acceleration runs in the OS, but that the hardware itself has certain routines built into it, on the firmware level, so the OS can just call those routines instead of trying to calculate them.
To oversimplify a bit, for instance, a hardware accelerator for "zoom in" might be programmed into the video chip system to automatically tell it "take the 50 pixels around this spot and blow up up to 250 pixels, refresh screen" where the OS would be saying "OK, let's take this spot, draw a square with a 50 pixel radius around it, now let's take each of those pixels and transpose it over twice the radius and go fill..." sending a long slow string of commands, each computed by the CPU.
When the CPU can offload all of that into a simple "zoom" command to the video chip, the CPU is now free to do other things. Like, respond to your next input, or push the next menu onto the display.
When you have ironclad control over the hardware--it can be a great way to make systems faster. And more stable.
Rred said:
Luis-
"So the hardware acceleration runs throughout the entire OS?"
It isn't so much that the hardware acceleration runs in the OS, but that the hardware itself has certain routines built into it, on the firmware level, so the OS can just call those routines instead of trying to calculate them.
To oversimplify a bit, for instance, a hardware accelerator for "zoom in" might be programmed into the video chip system to automatically tell it "take the 50 pixels around this spot and blow up up to 250 pixels, refresh screen" where the OS would be saying "OK, let's take this spot, draw a square with a 50 pixel radius around it, now let's take each of those pixels and transpose it over twice the radius and go fill..." sending a long slow string of commands, each computed by the CPU.
When the CPU can offload all of that into a simple "zoom" command to the video chip, the CPU is now free to do other things. Like, respond to your next input, or push the next menu onto the display.
When you have ironclad control over the hardware--it can be a great way to make systems faster. And more stable.
Click to expand...
Click to collapse
Great answer. Makes sense, thanks. Now given that this is an Android section lets talk more on that, will it ever be possible to have hardware acceleration on Android, Whether it be through custom ROMs or OEM devices?
"will it ever be possible to have hardware acceleration on Android,"
Possible? Sure, I've seen pigs on the wing.<G> Don't hold your breath for it though. Android is an unruly place where even ordinary hardware is often not supported by the OS and software breaks on every new phone. In order for hardware acceleration to work, the OS needs to have routines and drivers for standard hardware, which means locking down a hardware spec. Which is so very Undroid.
Can't see that happening, unless ten year from now someone invents a "standard universal Android cell phone chipset" and all the manufacturers get paid to exclusively use it. That's the ticket--use our chipset, we'll pay you to use it, and oh, yes, it will play one of "our" ads every time your screen turns on. Or you launch a new app. Or place a call.
(See? Things could get worse!<G>)
Here's an interesting discussion...
http://code.google.com/p/android/issues/detail?id=6914
burtcom said:
Here's an interesting discussion...
http://code.google.com/p/android/issues/detail?id=6914
Click to expand...
Click to collapse
Well as far as I read, it was just a bunch of "me too" and "I agree" lol I got bored reading that I still dont think Google has an official statement on the matter do they?
AerialX said:
Alright, thanks for the results everyone. I asked because I wanted to decide which framebuffer method would be selected as the default... I decided on the "slow" version - 8bdefb7e - as it's the most compatible option. The "fast" version can be chosen as a preference for those who prefer the speed. It may not work on all devices, and seems to produce artifacts and some tearing, though it probably is preferable for most simply due to the speed.
So, here you go, latest version: Download
Built for ARM and ARMv7, does not include the NEON extensions. You may want to uninstall the old one before installing this.
EDIT: Fine, have a NEON build too: Download
Hint: the main change is the inclusion of settings. I consider this build to be very close to the initial release for the application on the Android Market. Just a few things remain...
What's Left
In order of priority...
Filtering of non-white-screen presets. The app will include the projectM preset library by default, excluding any that just result in a white screen. I don't believe I will be able to distribute any of the Winamp/MilkDrop library itself, as awesome as it is.
Stability enhancements. It's probably still somewhat crashy. I'm not sure how much, though. I'd love to get some detailed reports on how stable/crashy it is for you guys.
snoop() support, otherwise the wallpaper requires 2.3... I don't have anything older though. Would anyone be able to test on an older OS for me if I supplied a build?
x86 support. Means writing a few missing STL pieces.
NEON detection, and the loading of an appropriate optimized library. I get the feeling that this is just a placebo, but will try to get it in.
Optimizations. Would be nice if it didn't essentially restart every time you visit your home screen. I'm probably leaving this for a 1.1 release though.
Localizations. If anyone wants to help translate the settings page into their native tongue, I'll happily include them.
Anything else I forgot...
What I Need
It would be great if anyone who tries it can comment on its stability. Whether it often dies, reverts to some other wallpaper, completely kills your system, or whatever. Just give me your overall experience with the wallpaper. I'm not necessarily interested in hearing about its performance at this point - if it runs slowly on your phone, I suggest switching to the "fast" render method and only using presets that run speedily. Of course, miscellaneous suggestions / comments / criticism are welcome as well!
Another thing I was hoping someone could do is go through all the presets and filter out the bad "white" ones. The new settings page makes it easy to test a single preset at will. So if anyone is willing to do this, I could really use two zips: one containing all working presets from the projectM set, one containing all from the MilkDrop set.
As a base for the projectM set, please use this: Download
For the base MilkDrop set, find the attachment from a few pages ago.
You may also want to note any that run particularly slow, but please don't exclude them from these packages.
EDIT: As a test, feel free to try the exact same preset on both the non-NEON and NEON builds and see if there's any noticeable improvement there.
Click to expand...
Click to collapse
Hands down best app ever
Sent from my LG-P999 using XDA App
Any screenshots?
Um seems to be a few post missing in this thread... Usually when something starts like this:
Alright, thanks for the results everyone. I asked because I wanted to decide which framebuffer method would be selected as the default...
Click to expand...
Click to collapse
What results are we talking about here? What is this exactly? Kinda like writing a book that starts:
But when the baby came out looking like death on a hot tin roof, Jim finally found the nerve to call her out and say, "Devil wench, I will not accept that thing as my my spawn! I know you have been sleeping with your brother!"
I implemented a browser based encryption solution which runs on Windows RT (and many other Windows computers). All I wrote was the HTML page, I am leveraging Crypto.JS javascript library for encryption algorithm. I am using the HTML 5 File API implementation which Microsoft provides for reading and writing files.
I make no claim on this but seems to work good for me. Feel free to feedback if you have any suggestions. The crypto.js library supports many different algorithms and configuration so feel free to modify it to your own purposes.
You can download the zip file to your surface, extract it and load the TridentEncode.htm file into Internet Explorer.
If you want to save to custom directory you probably need to load it from the Desktop IE instead of metro IE (to get the file save dialog). I usually drag and drop the file onto desktop IE and from there I can make favorite. This should work in all IE 11 and probably IE 10 browsers... if you use other browsers you may need to copy paste into the fields since the File API implementation seems rather browser specific. Running the html page from the local filesystem means that there is no man-in-the-middle which helps eliminate some of the vulnerabilities of using a javascript crypto implementation. You could also copy the attached zip file to your skydrive to decrypt your files from other computers.
Skydrive files in theory are secure (unless they are shared to public) so this might be useful for adding another layer of protection to certain info.
Again, use at your own risk, but feel free to play around and test it, and offer any suggestions or critiques of its soundness, or just use it as a template for your own apps.
Ok... this is really cool! Nice idea, and a good first implementation.
With that said, I have a few comments (from a security perspective). As an aside, minimized JS is the devil and should be annihilated with extreme prejudice (where not actually being used in a bandwidth-sensitive context). Reviewing this thing took way too long...
1) Your random number generation is extremely weak. Math.random() in JS (or any other language I'm aware of, for that matter) is not suitable for use in cryptographic operations. I recommend reading http://stackoverflow.com/questions/4083204/secure-random-numbers-in-javascript for suggestions. The answer by user ZeroG (bottom one, with three votes, as of this writing) gets my recommendation. Unfortunately, the only really good options require IE11 (or a recent, non-IE browser) so RT8.0 users are SOL.
NOTE: For the particular case in question here (where the only place I can see that random numbers are needed is the salt for the key derivation), a weak PRNG is not a critical failing so long as the attacker does not know, before the attack, what time the function is called at. If they do know, they can pre-compute the likely keys and possibly succeed in a dictionary attack faster than if they were able to generate every key only after accessing the encrypted file.
2) Similarly, I really recommend not using a third-party crypto lib, if possible; window.crypto (or window.msCrypto, for IE11) will provide operations that are both faster and *much* better reviewed. In theory, using a JS library means anybody who wants to can review the code; in practice, the vast majority of people are unqualified to either write or review crypto implementations, and it's very easy for weaknesses to creep in through subtle errors.
3) The default key derivation function (as used for CryptoJS.AES.encrypt({string}, {string})) is a single iteration of MD5 with a 64-bit salt. This is very fast, but that is actually a downside here; an attacker can extremely quickly derive different keys to attempt a dictionary attack (a type of brute-force attack where commonly used passwords are attempted; in practice, people choose fairly predictable passwords so such attacks often succeed quickly). Dictionary attacks can be made vastly more difficult if the key derivation process is made more computationally expensive. While this may not matter so much for large files (where the time to perform the decryption will dominate the total time required for the attack), it could matter very much for small ones. The typical approach here is to use a function such as PBKDF2 (Password-Based Key Derivation Function) with a large number of iterations (in native code, values of 20000-50000 are not uncommon; tune this value to avoid an undesirably long delay) although other "slow" KDFs exist.
4) There's no mechanism in place to determine whether or not the file was tampered with. It is often possible to modify encrypted data, without knowing the exact contents, in such a way that the data decrypts "successfully" but to the wrong output. In some cases, an attacker can even control enough of the output to achieve some goal, such as compromising a program that parses the file. While the use of PKCS7 padding usually makes naïve tampering detectable (because the padding bytes will be incorrect), it is not a safe guarantee. For example, a message of 7 bytes (or 15 or 23 or 31 or any other multiple of 8 + 7) will have only 1 byte of padding; thus there is about a 0.4% (1 / 256) chance that even a random change to the ciphertext will produce a valid padding. To combat this, use an HMAC (Hash-based Message Authentication Code) and verify it before attempting decryption. Without knowing the key, the attacker will be unable to correct the HMAC after modifying the ciphertext. See http://en.wikipedia.org/wiki/HMAC
5) The same problem as 4, but from a different angle: there's no way to be sure that the correct key was entered. In the case of an incorrect key, the plaintext will almost certainly be wrong... but it is possible that the padding byte(s) will be correct anyhow. With a binary file, it may not be possible to distinguish a correct decryption from an incorrect one. The solution (an HMAC) is the same, as the odds of an HMAC collision (especially if a good hash function is used) are infinitesimal.
6) Passwords are relatively weak and often easily guessed. Keyfiles (binary keys generated from cryptographically strong random number generators and stored in a file - possibly on a flashdrive - rather than in your head) are more secure, assuming you can generate them. It is even possible to encrypt the keyfile itself with a password, which is a form of two-factor authentication: to decrypt the data that an attacker wants to get at, they need the keyfile (a thing you have) and its password (a thing you know). Adding support for loading and using keyfiles, and possibly generating them too, would be a good feature.
The solutions to 3-5 will break backward compatibility, and will also break compatibility with the default parameters for openssl's "enc" operation. This is not a bad thing; backward compatibility can be maintained by either keeping the old version around or adding a decrypt-version selector, and openssl's defaults for many things are bad (it is possible, and wise, to override the defaults with more secure options). For forward compatibility, some version metadata could be prepended to the ciphertext (or appended to the file name, perhaps as an additional extension) to allow you to make changes in the future, and allow the encryption software to select the correct algorithms and parameters for a given file automatically.
Wow thanks GDTD that's great feedback
Not sure about his minified sources, the unminified aes.js in components is smaller than the minified version (which I am using) in rollups. I'll have to look into what his process for 'rollup' is to see if I can derive a functional set of non-minified script includes. If I can do that it would be easier to replace (what I would guess is) his reliance on Math.random.
His source here mirrors the unminified files in components folder : https://code.google.com/p/crypto-js/source/browse/tags/3.1.2/src
msCrypto that would be great, I had no idea that was in there. I found a few (Microsoft) samples so I will have to test them out and see if I can completely substitute that for crypto.js. Would be more keeping in line with the name I came up with.
Currently this version only works for text files, I am using the FileAPI method reader.readAsText(). I have been trying to devise a solution for binary files utilizing reader.readAsArrayBuffer but as yet I haven't been able to convert or pass this to crypto.js. I will need to experiment more with base64 or other interim buffer formats (which Crypto.js or msCrypto can work with) until I can get a better understanding of it.
Metadata is a great idea, maybe i can accommodate that with a hex encoded interim format.
You seem extremely knowledgeable in the area of encryption, hopefully i can refine the approach to address some of the issues you raised by setting up proper key, salt, and IV configuration... I'm sure I will understand more of your post as i progress (and after reading it about 20 times more as a reference).
Too bad we don't a web server for RT, that would at least open up localStorage for json serialization (mostly for other apps I had in mind). I guess they might not allow that in app store though. Could probably run one of a developers license though (renewed every 1-2 months)?
nazoraios said:
Too bad we don't a web server for RT, that would at least open up localStorage for json serialization (mostly for other apps I had in mind). I guess they might not allow that in app store though. Could probably run one of a developers license though (renewed every 1-2 months)?
Click to expand...
Click to collapse
I cant comment too much on the encryption, GoodDayToDie has covered anything I could contribute and more. But there is a functioning web server on RT. Apache 2.0 was ported: http://forum.xda-developers.com/showthread.php?t=2408106 I dont know if everything is working on it, I dont own an RT device and last time I tried I couldnt get apache to run on 64 bit windows 8 anyway (needed it at uni, spent hours going through troubleshooting guides and it never worked on my laptop, gave up and ran it under linux in virtualbox where it took 2 minutes to have functioning the way I needed it to).
Curious about the performance. Speaking of encryption, 7-Zip has it built-in, and from the discuss in StackExchange, it seems pretty good.
One of the neat things about this thing (local web app? Pseudo-HTA (HTml Application)? Not sure if there's a proper name for such things) is that it runs just fine even on non-jailbroken devices. That's a significant advantage, at least for now.
Running a web server should be easy enough. I wrote one for WP8 (which has a subset of the allowed APIs for WinRT) and while the app *I* use it in won't be allowed in the store, other developers have taken the HTTP server component (I open-sourced it) and packaged it in other apps which have been allowed just fine. With that said, there are of course already file crypto utilities in the store anyhow... but they're "Modern" apps so you might want to develop such a server anyhow so you can use it from a desktop web browser instead.
Web cryptography (window.crypto / window.msCrypto) is brand new; it's not even close to standardization yet. I'm actually kind of shocked MS implemented it already, even if they put it in a different name. It's pretty great, though; for a long time, things like secure random numbers have required plugins (Flash/Java/Silverlight/whatever). Still, bear in mind that (as it's still far from standardized), the API might change over time.
Yep, I think of them as Trident apps since trident is what Microsoft calls their IE rendering engine, but I guess they are sort of offline web apps (which come from null domain). Being from null domain you are not allowed to use localstorage which is domain specific. You also are not allowed to make ajax requests. You just have file api and json object serialization to make do with I/O.
Another app I am working on is a kind of Fiddler app similar to http://jsfiddle.net/ where you can sandbox some simple script programs.
Kind of turning an RT device into a modern/retro version of a commodore 64 or other on-device development environments. Instead of basic interpreter you've got your html markup and script.
I have an attached demo version which makes available jquery, jquery-ui, alertify javascript libraries in a sandbox environment that you can save as .prg files.
I put a few sample programs in the samples subfolder. Some of the animation samples (like solar system) set up timers which may persist even after cleared so you might need to reload the page to clear those.
It takes a while to extract (lots of little files for all the libraries) but once it extracts you can run the html page and I included a sample program 'Demo Fiddle.prg' you can load and run to get an idea.
I added syntax highlighting editors (EditArea) which seems to work ok and let's you zoom each editor full screen.
The idea would be to take the best third party javascript libraries and make them available and even make shortcuts or minimal API for making it easier to use them. Common global variable, global helper methods, ide manipulation. I'd like to include jqplot for charting graphs, maybe for mathematical programs and provide api for user to do their own I/O within the environment.
These are just rough initial demos, and obviously open source so if anyone wants to take the ideas and run with them i'd be interested in seeing what others do. Otherwise I will slowly evolve the demos and release when there are significant changes.
Fair warning: I know only mostly what I want, I have no clue how difficult it will be to achieve, and therefor have no idea how to price the work. There are several people interested in this project though, so I'm sure a good budget can be reached.
I have several copies of Lenovo+Google's ill-fated virtual reality headset, which I would like to turn into actually functional devices. The kernel is Android 8 with some additional hooks (I have a copy with the additional required files). I also have one that has developer access of some kind, which may make the process easier.
The primary problems the device needs fixed:
Needs USB File transfer (disabled completely currently)
Needs access to cameras (Especially necessary for AR passthrough)
Needs Task Manager
Needs better widget and notification area systems (Currently, to skip a song on, say, spotify, you have to exit the app you're in, navigate to spotify, switch the song, restart the app you were in. The notifications are also not context aware, so you can't click on one to go to the referenced app)
Support for alternate controllers (basic, such as keyboard or bluetooth PSX)
According to the last person I attempted to hire for this, it would be easier to start from scratch than to implement an altered rom, but I don't think he understood how the 360 virtual environment and camera tracking systems were integrated (not that I do, but I know enough to know that building them out from scratch would be a massive undertaking).
Here is the current road map document, which is still a work in progress.
Any assistance, direction, or paid work would be appreciated, as I've been stumped on where to even begin for over a year.
MOD ACTION:
Thread closed since bounty threads are not allowed on XDA.
@bridgebrain