Database Info

[Dec 2012] Android SMS Importer from WM PimBackup files

This utility allows to convert PimBackup files to XML one which is
compatible with Android application "SMS Backup&Restore" created by
Ritesh Sahu (* check the author to match the right application on Market *).
After the conversion the XML file contains all your SMS and you can Import
them to you Android smartphone.
This utility doesn't require any installation process or additional DLLs,
in order to start using it, just run the executable file.
WARNING: I've tried to restore about 11,000 SMS with success on "Motorola Atrix 4G" and it seems to freeze many times while running stock "Text Messaging" application.....so I think 11,000 SMS is too much to handle for Android OS
Versions History
=============
* Version 2.0.0.4 (2012-12-13)
- New Draft Folder support
- Messages in Custom Folders are now moved to Inbox or Outbox
- Minor bug fixes
* Version 2.0.0.2 (2012-11-28)
- New option (only for CMS files): Discard self-test messages
- New option to convert not supported folders in In/Outbox
* Version 2.0.0.1_beta1 (2012-11-14)
- New advanced graphic design
- PimBackup binary files (*.pbm) supported!!!!!!
* Version 1.0.1.0 (2011-09-28)
- If it enconters an irreversible error it will asks if you want to continue without stopping conversion
* Version 1.0.0.9 (2011-09-26)
- Managed splitted Conversation/SMS text in multi-rows (messages with Carriage Return)
* Version 1.0.0.8 (2011-07-27)
- Fixed negative values for TimeZone offsets
* Version 1.0.0.7 (2011-07-27)
- Added TimeZone and Date/Time management to increase/decrease according to your local TimeZone
* Version 1.0.0.6 (2011-07-04)
- Fixed bug while using special characters like "&" or quote
* Version 1.0.0.5 (2011-04-21)
- Fixed bug for Inbox messages if SenderName and/or SenderAddress is empty
* Version 1.0.0.4 (2011-04-21)
- Managed empty "SenderName" and "SenderAddress" for Outgoing messages. Now those messages will be considered well.
- Draft, Trash and Outgoing folders (from CSM file) are not converted in any case because Android hasn't got those folders.
* Version 1.0.0.3 (2011-04-19)
- Added "Text Encoding" option to decide output file text encoder ("UTF-8" is the most common)
- Managed empty "SenderName" and "SenderAddress" data from input files. Now those messages will be discarted
* Version 1.0.0.2 (2011-04-19)
- Added an option to stop decodification after have decoded a specified amount of messages
- Managed "line-brakes" in messages. Those line-brakes will be removed from the message
- Managed "self-test" messages. Those self-test messages will not be added to output file
- Fixed a bug that truncates TextMessage at first "semicolon" character
- Added an "Error" section to show occurred errors
* Version 1.0.0.1 (2011-04-18)
- Uncompressed Executable file for better compatibility on multiple Windows OS
* Version 1.0.0.0 (2011-04-06)
- First release
Create a good backup file with PimBackup
=================================
The software is dummy and need a good/perfect file to work fine; in other words it needs
a good PimBackup file.
Let's create that file:
* open PimBackup in your Windows Mobile phone/smartphone
* choose "Backup" from dropdown list
* select sections as you wish but not forget "Messages"!!
* press "Next"
* (if you have a lot of SMS do not select "Create temp files in memory")
* press "Next" and go-on following the Wizard....
Find CSM or PBM file
===================
At the end of procedure above you will have a single PIB file.
This file is compressed and you can open it with WinZip/WinRAR/7Zip.
* So, please open it and look for "*.cms" or "*.pbm" file.
* Extract it from the compressed file.
That's it!!
Start the conversion
==================
Ones located CSM/PBM file you will want to convert it, right?
* Open "PimbackupDroid.exe" (this software of course!)
* Click "Menu --> Open" and select a CSM/PBM file
* [only for CMS] Enter "Your phone number for comparison". This field is necessary because
this is the only way to identify you as the sender of outgoing messages.
You can use simple numbers without international prefix (Ex: 0039 or +39).
* (optional) Check "Only between dates" to force convert just SMS between
two specified dates. It works for both incoming and outgoing messages.
* (optional) Check "Blacklist" OR "Whitelist" radio button if you want to
EXCLUDE (blacklist) some contacts or you just WANT (whitelist) convert
just some specified contacts.
!!WARNING!! - Blacklist and Whitelist using "Partial" input data, in
other words if you will add "Edward" in one of those all
contacts CONTAINS "Edward" word will be blacklisted/whitelisted,
for example also "Edward2" will be affected.
Both Blacklist and Whitelist are case insensitive (no difference between
"Smith" and "SMITH").
* Press "Start" button. It will generate a NEW file with the same name of original
one but with ".xml" extension. (!!WARNING!! - it doesn't ask for overwriting)
Import XML file on Android OS
========================
Now you have an XML file generated with this utility and you need to download
a small (and FREE) application from Android Market:
*SMS Backup & Restore" created by "Ritesh Sahu" (!!!check the author name!!!)
Install it
Before run this new installed utility you need to create a new "SMSBackupRestore"
folder in you Internal Phone Storage's root directory.
Now you can copy your XML file to this new folder and run the new Android application.
This Android utility is very very easy to understand: just press "Restore" button
and select your XML file.
Now wait until errors (ops!!) or successfully popup window!
System Requirements
==================
This utility works properly on all 32-bit/64-bit versions of Windows,
including Windows 98/ME, Windows NT, Windows 2000, Windows XP, Windows
Server 2003, Windows Vista, Windows Seven and Windows 8.
Personally tested on: Windows 7 Ultimate 64bit, Windows XP 32bit and Windows 8 Pro 64bit
Linux users: I think this software will works with "WINE" but not tested.
Disclaimer
==========
The software is provided "AS IS" without any warranty, either expressed
or implied, including, but not limited to, the implied warranties of
merchantability and fitness for a particular purpose. The author will not
be liable for any special, incidental, consequential or indirect damages
due to loss of data or any other reason.
Issues/Limitations
===============
- MMS/e-mails not supported
- Attachments (like images or files) not supported
Screenshot
===========
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Download at Sourceforge.net
======================
https://sourceforge.net/projects/pimbackupdroid/
Any idea on what to add more to this software??

Post reserved to author

This sounds great. I have a lot of old SMS I'm missing after switching to android. I'm going to test it as soon as I'm with my PC tommorow morning. Will report back then.
Sent from my HTC HD2 using XDA App

emandt, I tried to download your SW from the page you have listed, but it looks like folder is empty...?

As I've wrote in first post: project just opened at SF.net, so be patient
I'm creating a small README as Guide and prepare source code to place it on SF.net very soon.

OK, I was under impression at least test version is available. I'll wait

Released the first version few hours ago.
Check the Limitations (first post) before post a bug

I've realized my PIM backup files are .pib, not .csm. Is there any way how to move it to .csm? Except of restoring and archiving again through WM...I have only one phone
Thanks.

Please read the description at first post or the README file following the EXE.....
The ".CSM" file IS INTO the PIB file.
PIB file is just a compressed file to handle multiple files in single one.
Use WinZip / WinRaR / 7-Zip to open the PIB file and extract CSM one.
If your backup was created with "Binary backup(faster)" options it's impossibile to convert with my software (no any description/protocol about "How To Read" a Binary file created by BimBackup)....so, in this case, I think you need to Restore all messages and start the backup (without THAT option!!) again....sorry.
However I will study (raw mode with thousands of tries) "Binary backups" to allow them as non-binary ones.

I'm sorry I have overlooked this step. The problem is that even when I open .pib file, there is no .csm. Actualy, all extensions begin with .p**. Could be the reason that I have somehow checked "binary backup" in WM PIM?
I'm hot to test your SW but obviously I'm not a good tester
BTW, why don't you move this topic between general android applications, to have it more visible in the forum?

I'm trying to read binary backup files but without any description from author is very very hard to resolve this step. I've not found a schema for the data into binary files yet.

I've studied BINARY backup and already found half-solution, at least for NAME, TELEPHONE_NUMBER and TEXT_MESSAGE.....not I'm stuck at "Date and Time field"....
I've wrote to PimBackup author and I hope he will respond me with the solution.
Stay tuned

Nice initiative
Hi,
first of all, congrats for the iniative. Sorrowfully, it doesn't work for me (Access Violation at address 004ccea0 [...] @ address 00000004
If this helps: W7 x64 Pro @i5
For people who have their backups in binary format, from this same forum (don't remember the threads tho, sorry) came these tips:
1) Download Microsoft Windows mobile 6.5 Professional Developer Tools (goole it or "Windows Mobile 6.5 emulator");
2) run the appropriate version of the emulator;
3) "install PimBackup and restore your backup (set up a "shared Folder" for that, it will appear as a storage card);
4) redo your backup without binary format unchecked;
5) have fun! (if you're reading this thread, you know the rest already )
Also, see this link: Transferring contact list from Windows Mobile to Android SANS OUTLOOK
Have fun! I'll have to wait the next stable version.

It's strange....I'm using Win 7 Ultimate 64bit in italian language.
What OS language?
What is your OS alphabet in use?
I've developed this application quiclky and I don't have considered alphabets and non-latin characters. Maybe it could be this.
I'll spend more time to make it works on other OS settings

@tuliogs: if your Error is generated during startup of application you can try new version (v.1.0.0.1 folder on download section) which has an uncompressed Executable file to increase compatibility on Windows OS.
Otherwise: I need more explanations about that Error.

not yet
I get the error when trying to convert the file. Win7 here is in brazilian portuguese, I guess it's UTF-8 (or else, probably Windows-1251, which in turn is an ISO 8859-1 variant, I guess).
Just tried the new version... same issue, but it now shows where it stops: on a self-test SMS (explained below). It was enough to change the "phone number for comparison" field and _that_ issue was solved. However, please keep reading.
I found some "anomalies" in some messages in my file, though:
some are self-tests I sent to myself or just empty drafts. this seems to be the issue described before;
some contain the whole phone numbers including country codes and the plus sign (no trouble with those ones);
some archived messages contain line breaks: these are plain skipped;
duplicate messages, some within a short interval (3 secs or less) disappear also;
different cellphones use different text encodings so you guess the mess... however, overall text is more important than specific characters, so it's relevant to say I had no problem at all with these, apart from getting alien symbols (which don't bother me).
I had to abandon Android on my Kaiser to revamp it from scratch (therefore, WM again aawwww), so I can't test the final result now, but there's a Samsung Galaxy S (Froyo, I guess) coming in a few days. So if you'd wish to attack those issues in the meantime, I'll gladly test the final result when time comes.
Thank you again and keep up the good work.

Nice to have so detailed error description!!
New version was compiled just without "Compress executable" option because I thought your error were caused on startup and not during conversion.
Can you send me just a complete row (you can manually edit a row to create fake text) of CSM file with many brazilian characters??
Now I'll try with:
- self test messages
- many special characters (also line-breaks)

outgoing SMS is sucessfully converted,
but I cannot found any incoming message in the output files...
Here are the first few line of the output:
Code:
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<smses count="196">
<sms protocol="0" address="" date="1284300650000" type="1" body="回到家了啦" read="1" status="-1" contact_name="" />
<sms protocol="0" address="" date="1286260959000" type="1" body="整緊" read="1" status="-1" contact_name="" />

@azothterry: TYPE="1" means "Inbox" but perhaps this wrong decode depends of strange Input data.
I've tested with my latinian-alphabet without special chars (comma, points and few single-quote chars) and it works.
To allow other alphabets/characters I need few lines from your CSM file, expecially with special chars.
Don't forget to set "Number for comparison" field with your partial (without international prefix) phone number.

Hi, emandt
Here are some line of my csm file, many thx.
Code:
0x00,0x00;"";"";;"";"回到家了啦";"";0;"\\%MDF1";"SMS";"IPM.SMStext";;;33;262144;2011,04,13,15,07,28;2010,09,12,22,10,50;1;"524296\;Kat\;99999999\;1\;0\;SMS";0;""
0x00,0x00;"Kain Kwok";"\"Kain Kwok\" <88888888>";;"";"幫我開電腦pls";"";0;"\\%MDF1";"SMS";"IPM.SMStext";;16;1;;2008,01,01,16,38,55;2010,09,22,09,29,45;0;"";0;""
0x00,0x00;"";"";;"";"整緊";"";0;"\\%MDF1";"SMS";"IPM.SMStext";;;1;262144;2008,01,01,16,38,56;2010,10,05,14,42,39;1;"524296\;Kat\;99999999\;1\;0\;SMS";0;""
0x00,0x00;"Kat";"\"Kat\" <99999999>";;"";"飲檸檬茶中..";"";0;"\\%MDF1";"SMS";"IPM.SMStext";;14;1;;2011,04,13,15,07,28;2010,10,05,14,46,55;0;"";0;""
0x00,0x00;"";"";;"";"飲緊燈汁!明天俾一支你!";"";0;"\\%MDF1";"SMS";"IPM.SMStext";;;33;262144;2011,04,13,15,07,28;2010,10,05,14,51,24;1;"524296\;Kat\;99999999\;1\;0\;SMS";0;""
0x00,0x00;"Kat";"\"Kat\" <99999999>";;"";"何時買?";"";0;"\\%MDF1";"SMS";"IPM.SMStext";;8;1;;2011,04,13,15,07,28;2010,10,05,14,52,55;0;"";0;""
0x00,0x00;"";"";;"";"今早";"";0;"\\%MDF1";"SMS";"IPM.SMStext";;;33;262144;2011,04,13,15,07,28;2010,10,05,14,53,23;1;"524296\;Kat\;99999999\;1\;0\;SMS";0;""

Link contact adress to Nokia Drive

I use my7rom on my Omnia 7.
Is there anyway to link a contacts adress to Nokia Drive instead of Maps (stock wp7 app). It would be much more practical if Nokia Drive opened a navigation session instead.
Anyone up for the challenge? A reg-tweak perhaps?
// Manneman
Skickat från Windows Phone 7.8

There's two parts here. The first is identifying the correct "filetype" or URI scheme that is used for navigation. That shouldn't be too hard; a little digging in HKCU should reveal it. We already know about ones like callto: and http: and I'm actually (slowly) working on an app to allow people to easily change them. The second part is finding the correct command to load that address or route in the Nokia Maps app. If the app supports pinning routes or destinations to Start, this is probably possible. If not, it may not be possible in the app. Most apps aren't designed to accept command-line parameters, so even if you make them the default handler for a given filetype or URI scheme, they ignore the value you send them and just start as though launched from Start.

GoodDayToDie said:
If the app supports pinning routes or destinations to Start, this is probably possible.
Click to expand...
Click to collapse
Nokia Drive supports pinning to start so it should be possible. Unfortunately I can't tell you exact command line parameters 'cause my Lumia 900 still "in jail"

Let me see if I have a copy of the Nokia Drive XAP handy. I'll need to decompile it to figure out the correct parameters for launching it with the intent of navigating to a specific location. Note also that this might not be possible directly - for example, the app might store a list of locations internally, and the tiles only provide an index into that list rather than providing the location directly - but that just requires another layer of indirection.
In that case, you create an app that gets registered as the navigation handler, and in response to a navigation request, it writes the requested location into the Nokia Drive app and then chain-launches Nokia Drive with the index of the newly written location. That's just an example of one way that this might go wrong, but overall, the odds are actually pretty good. Obviously, all of this will require, at a minimum, write access to the HKCR hive in the registry.

Ah, guys! You are so kind helping me out. I´m really certain alot of members in the WP7 section would love for this to work!
// Manneman

GoodDayToDie said:
I'll need to decompile it to figure out the correct parameters for launching it with the intent of navigating to a specific location
Click to expand...
Click to collapse
GoodDayToDie, you may try much simpler solution. Just create assembly (dll) to show startup parameters in message box, and replace main Nokia Drive dll (but pin some location first).

That's actually harder than it sounds; even if the app is sideloaded (which would mean I already have the DLL) my fake would have to mimic the internal structure of the real app to a degree (namespaces, class names, default actions, etc.). That's not hard, but decompiling .NET is pretty trivial too.

AFAIR, Nokia Drive is obfuscated (but I'm not 100% sure). Also, you don't need to duplicate all names and structures; just a stuff mentioned in WMAppManifest (I hope so). BTW, I forgot: I still have unlocked handset; if I'll found time, will try today later.
Update: tried but without of luck What I did:
- installed Nokia Drive first;
- downloaded map and pinned current location;
- created fake app with same app guid and namespace name ("Drive"), and performed app update (that operation completely override whole solution but NokiaDrive tile still pinned to the start screen);
- tried a few different page names (_default.xaml, QuickStartPage.xaml, DestinationPickerPage.xaml, FavoritesPage.xaml) with code
Code:
protected override void OnNavigatedTo(System.Windows.Navigation.NavigationEventArgs e)
{
MessageBox.Show("Hello from fake dll");
if (e.NavigationMode == System.Windows.Navigation.NavigationMode.New)
{
string[] keys = NavigationContext.QueryString.Keys.ToArray();
string[] values = NavigationContext.QueryString.Values.ToArray();
string param = "";
for (int i = 0; i < keys.Length; i++)
{
param += keys[i] + " -> " + values[i] + "\n";
}
MessageBox.Show("parameters: " + param);
}
}
But result always the same: app doesn't start from the pinned tile

Update 2: Finally, I did it
The trick is:
- do the same as I've described above (you should have pinned tile from ND);
- add following code to the start page:
Code:
public MainPage()
{
InitializeComponent();
var appTile = ShellTile.ActiveTiles.Last();
if (appTile != null)
{
//MessageBox.Show(appTile.NavigationUri.OriginalString);
EmailComposeTask emailTask = new EmailComposeTask();
emailTask.Subject = "NokiaDrive pinned parameters";
emailTask.Body = appTile.NavigationUri.OriginalString;
try
{
emailTask.Show();
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "Error", MessageBoxButton.OK);
}
}
}
- run app as usual (not from tile);
We allset - all params are sent to our email (I'm too lazy to manually copy all stuff )
Here we are (start parameters; bold values are changed for privacy reason ):
/_default?destination.name=200 SomeName Street&destination.latitude=49.5255378801376&destination.longitude=-72.4296837244183&destination.address.street=SomeName Street&destination.address.houseno=200&destination.address.district=&destination.address.county=&destination.address.city=SomeCity&destination.address.state=&destination.address.country=USA&destination.address.postcode=05720&destination.hashCode=371767793destination.address.statecode=MA&pinnedFrom=Favorites
P.S. Just found: Navigon also has ability to pin address to the start tile So, if you find the way to modify map protocol (or how it calls), it will be a really nice hack! BTW, could you remind me: do we have ability to launch assembly by GUID (on the full-/policy-unlocked phones)? If "yes", it's possible to write a real nice "proxy" app to handle map requests

I don't know about launching assemblies directly, but it's certainly possible to launch apps by GUID. It doesn't even require anything more than dev-unlock in fact (although of course you can only launch apps that you could launch anyhow). So yes, a proxy app is totally possible. That's actually what I'm working on (started as a project to make a Kindle ebook file loader, that would pur .mobi/.prc file in the Kindle app's folder and then launch the app).

GoodDayToDie, could you please, take a look to the registry, for default map protocol handler and figure out how to change that stuff? I'm pretty busy these days (and probably will be extremely busy couple of next months) but we can cooperate and create this app...

I'll investigate, but you're not the only one busy. If you've noticed a lack of software from me recently, it's due to the nex job I got some months back; I love it, but it leaves me with a lot less time for phone hacking if I want to still have a life outside of that.
With that said, this actually ties into the work I'm already doing with things like filetype handling and default browser switching. I can send you my HKCRlib, at a minimum; it's a library that simplifies interacting with HKCR, including creating backups of important values when they change, and reverting the backups.

GoodDayToDie, truly, I'm not much interested (personally) in that hack 'cause I can't use it for my Lumia 900. So it's only for the community needs but because of lack of time, I believe, we may put it on hold.

[Beta] Win86emu: Running x86 apps on WinRT devices

The project is abandoned.
As I no longer own a Windows RT device and I'm not willing to use Windows RT anymore, unless Microsoft would make it more open (at least to run your own desktop apps) - I've decided to stop working on this project.
As usual I'm publishing complete sources of this tool. Feel free to use them in your own projects or to continue developing this one - only leave my copyrights somewhere.
Don't ask me how to build the sources or to explain anything in them. Figure that out yourself.
The project is abandoned. Sorry.
I'm presenting a tool that allows running a set of x86 Windows applications on Windows RT (ARM) tablets. Its goal is to support all apps except for those that:
- require much CPU power,
- use complex features that were cut out from WinRT like D3D9 extensions or OpenGL,
- require drivers or specific services,
- make heavy use of COM interfaces,
- use undocumented windows internals,
- apps that use .NET framework,
- x86 Metro apps,
- 16 or 64 bit Windows programs,
- buggy apps that require special workarounds.
The tool is currently on a beta stage, so don't expect much from it. It is far from being complete, but at least it runs something.
Current version: 0.061
Just a minor update. The project is not dead, I just had no time to continue the development.
Attached the fixed ntdll.nt.dll that works under Windows RT 8.1 (Microsoft removed some NTDLL exports, so I had to add more stubs). This fix is not needed on RT 8.0.
To install it: extract the attached 0.061-ntdll.nt.dll.zip to c:\x86node\windows\SystemNT\ overwriting the existing file.
Autostarting x86 programs does not work on RT 8.1 ("can't install CreateProcessInternal hook"). I'll look on this later.
Don't ask on jailbreaking the 8.1 beta in this thread - there is a good progress on it, more info would be on release (in october or when WZOR would leak the RTM).
Current version: 0.06
Seems that archive is too big to be attached, so I've uploaded it to google drive and here
Installation: extract the archive on your unlocked Windows RT device, run the MSI file and follow the instructions.
Note: Uninstall the previous version before installing a new one.
List of compatible apps is in this post: http://forum.xda-developers.com/showthread.php?p=40924456
Trademarks
Windows is a registered trademark of Microsoft Corporation. ReactOS is a registered trademark or a trademark of ReactOS Foundation. All other trademarks are the property of their respective owners.
Disclaimer
This software is provided "as is". Use it on your own risk. I make no warranties as to performance, merchantability, fitness for a particular purpose, or any other warranties whether expressed or implied. No oral or electronic communication with me shall create a warranty of any kind. Under no circumstances should I be liable for direct, indirect, special, incidental, or consequential damages resulting from the use, misuse, or inability to use this software, even if I has been advised of the possibility of such damages.
I'm trying my best to make the software working, but I can't guarantee that it is free from defects.
All beta versions of this tool would be freeware. You may freely use it for your own, embed it into your tool, but you can't use it commercially without my confirmation. You can disassemble, analyze or modify this tool for yourself - later I'll provide SDK and document its internals. The only thing that is prohibited is changing embedded copyright notices. I reserve the right of making the project commercial, but this does not mean that this would ever happen.
This software contains unmodified binaries from the ReactOS project: a registry editor, cmd.exe, ole32.dll to name the few. Those binaries are left unmodified and are covered by LGPL license. Future versions may contain redistributable binaries provided by Microsoft and/or other companies.
Some more information may be found in my blog. If you want to support development - use the link or press the button on the left side of the post.

Changes:
15 may 2013: DInput and DInput8 changes for Fallout2 keyboard compatibility.
12 may 2013: A minor update. Fallout 2 now works, tested on Russian version from 1C.
01 may 2013: Added the ability to automatically launch x86 applications. Added the shell32 interfaces - so that installers now work (at least NSIS and InstallShield installers are known to be working).
05 apr 2013: Fixed a few bugs.
04 apr 2013: Uploaded a new build after a long delay. Now emulator supports 256-color modes. But due to a limitation on an updated Nvidia driver - 640x480 and 800x600 display modes are no longer supported on Windows RT. You'll see the black lines to the right and bottom of the screen if the program tries to set such mode.
25 feb 2013: The tool now outputs its version to log. Now one x86 program may launch another - so some of the installers and, for example, 7Z GUI frontend can now run under emulation. Added ~80 DLLs. Some of them are stubs (like D3D9.DLL), others are mostly untested. I have not done all that I've planned for this build, publishing it just as an update to show that the work is going on. Do not expect it to run much more than the previous build.
13 feb 2013: more informative errors from launcher. Emulator now supports program paths with spaces. EXE files with relocations are now processed correctly. Some bugfixes in kernel32 and advapi32.
11 feb 2013: fixed a typo in winmm.dll emulation, now pinball has sound. Also updated the launcher.
10 feb 2013: now the program reached the beta stage.
Known problems
No D3D and most of COM interfaces. Lots of programs would crash, don't run or have different issues.
Notes:
The program keeps its settings in the HKCU\Software\x86node\Settings registry key. Supported REG_SZ (string) values are:
DosboxCore: "dynamic", "simple" or "normal". Dynamic is the default as it is the fastest, but the most buggy core.
LogFile: path to the log file. If not present - log file is %temp%\win86emu.log
Supported REG_DWORD values:
LogLevel: 0=no log (default), 4=max logging
added 13 feb 2013: now default log level is 2: warnings+errors, so you don't need to edit registry
There are several compatibility hacks that may be useful. Compatibility settings are stored in HKCU\Software\x86node\Compatibility\[filename.exe] key. "filename.exe" - a name of the emulated EXE file without path. All values are DWORD:
SetProcessAffinityMask = bitmask. Specify which CPUs to use for running a program, read SetProcessAffinityMask description in MSDN. 0 or unset == run on all cores.
NoRaiseException = 1. RaiseException would just return. Now exceptions are emulated correctly, so this hack is no longer needed.
UseDirectRegistry = 1. Do not redirect emulated registry keys to HKCU\Software\x86node. Be careful when using it.
MaxProcessorFeaturePresent = max processor feature number that is "supported". See the IsProcessorFeaturePresent function in MSDN. All requests for the value above specified would return 0. Default: 0 (IsProcessorFeaturePresent always returns 0).
SimulateAdminRights = 1. Lie to installers that call OpenSCManager function to determine that it is running as administrator, allowing these programs to run without elevation. Redirect the "common start menu" and similar folders to the per-user folders.
You can fake the OS version to a running program. Default XP SP3:
OSVersionLo=dword:00000001
OSVersionHi=dword:00000005
OSVersionBuild=dword:00000a28
OSServicepackLo=dword:00000000
OSServicepackHi=dword:00000003
Some information on the project internals may appear in my blog: http://mamaich-eng.blogspot.ru, but this thread on XDA would be the main discussion place.

I may be missing something...but this won't run for me. The exe just tells me it can't be run like normal exe does. I am jailbroken and can run arm compiled exe.

That is very impressive, mamaich.

lucas.scott said:
I may be missing something...but this won't run for me. The exe just tells me it can't be run like normal exe does. I am jailbroken and can run arm compiled exe.
Click to expand...
Click to collapse
Reread the directions ?

I don't have an ARM tablet to test this on, but this type of development is what will get me onto an ARM tablet for the next go-round. I love my S7S, but I really hated paying the price.

dan-htc-touch said:
Reread the directions ?
Click to expand...
Click to collapse
ugh...embarassed.
Great work mamaich. Thank you!

Notepad from Windows 95 seems to run too.

wow,this is an awesome project!

Amazing! Thank you!!!!!!!

Would you mind giving a technical explanation?

mamaich said:
I'm presenting a prototype of a tool that allows running Windows programs compiled for a desktop PC (x86) on an unlocked Windows RT (arm) tablet. The tool emulates x86 instructions and passes Windows API calls to WinRT kernel with necessary modifications.
This build is an early alpha version. It can run only very simple apps that use rather small subset of Win32 API that I've already implemented. Archive contains clock.exe from NT4 distribution as an example of such app. As I'll continue work on the project - the list of supported applications would grow up.
This tool would support only 32-bit windows native applications. It would not allow running drivers or .NET apps that were written for old .NET versions nor Win16 or DOS apps. And current version supports emulation only of EXE files that contain relocations section (this would be fixed later).
Instructions:
1. Unlock your device with this tool: http://forum.xda-developers.com/showthread.php?t=2092158
2. Unzip the archive to any directory
3. Run _start_clock.cmd
This would execute the clock.exe from NT4 in the emulation mode.
This post would be updated as I'll make more progress.
Note: This is an early alpha version, and do not expect that it would run anything except the provided file. Do not ask me what programs would be supported and when the next builds would appear - I don't know, as I work on this project only on spare time. I would not publish the complete sources of the tool, but it would be extensible by users, and some plugins (at least bochs/dosbox emulation engines and some of the API wrappers) would be opensource.
"Yact" in the file names stands for "yet another code translator", it was the original name of the project.
Edited 13.01.2012:
- added a few ARM WinAPI workarounds, added calc.exe as a second example (run _start_calc.cmd).
Click to expand...
Click to collapse
heh, is this a port of the app you showed off back in the CE days? -awesome getting that ported over

This is great, really opens up a lot more things and means that we don't need to recompile everything either.

heh, is this a port of the app you showed off back in the CE days? -awesome getting that ported over
Click to expand...
Click to collapse
Not exactly a port, it is a clean remake based on the old ideas.
clrokr said:
Would you mind giving a technical explanation?
Click to expand...
Click to collapse
The idea is very simple:
- a PE file loader (load files, process relocs, run TLS callbacks in an emulation mode). Support import loops (DLL A imports B while B imports A), ordinals, etc.
- a set of wrapper x86 DLLs (kernel32_stub.dll and so on) that "look like" the corresponding Win API functions for an emulated program:
Code:
#define DEFINE_FUNC1(name) \
static const ModuleDef str_##name={DLL_NAME,#name}; \
EXTERN_C DW STUB_EXPORT stub_##name(DW p1) \
{ \
DW *p=&p1; \
__asm { mov eax,p } \
__asm { jmp f1 } \
__asm { mov eax,offset str_##name } \
f1: __asm { in eax,0xe5 } \
__asm { mov p,eax } \
return (DW)p; \
}
.....
#define DEFINE_FUNC3(name) \
static const ModuleDef str_##name={DLL_NAME,#name}; \
EXTERN_C DW STUB_EXPORT stub_##name(DW p1,DW p2,DW p3) \
{ \
DW *p=&p1; \
__asm { mov eax,p } \
__asm { jmp f1 } \
__asm { mov eax,offset str_##name } \
f1: __asm { in eax,0xe5 } \
__asm { mov p,eax } \
return (DW)p; \
}
....
DEFINE_FUNC1(AddAtomA)
DEFINE_FUNC1(AddAtomW)
DEFINE_FUNC7(CreateFileA) -- number in macro == number of parameters to a __stdcall WinAPI function.
Compiler automatically generates "ret N*4" at the end of such function.
I've decided to use such c+asm approach instead of making a tiny assebler stub,
as I can easily implement some of such functions in C directly in a stub DLL plus it
simplifies debugging. And the functions have a usual C prologue/epilogue, so that
the emulated program may even patch them in runtime, for example for hooks.
...
- a 32-bit x86 emulation engine (currently 2 engines: from bochs and from dosbox, planning on adding my own) that intercepts the command "in eax,0xe5", determines which API is needed by a program and passes it to a handler.
- native (arm) API handler DLLs (kernel32_yact.dll and so on). They are mostly autogenerated too:
Code:
#define DEFINE_FUNC1(name) \
EXTERN_C DW STUB_IMPORT name(DW); \ -- this behaves like a function prototype to compiler
EXTERN_C DW STUB_EXPORT yact_##name(DW *R) \ -- R - pointer to the x86 stack
{ \
DW r=name(p1); \ // call the func passing it paramers from the emulated stack, p1==R[0], p2==R[1] and so on
LEAVE(1); \ // empty macro, as the stack is unwinded in x86 stub DLL now
return r; \
}
...
#define DEFINE_FUNC3(name) \
EXTERN_C DW STUB_IMPORT name(DW,DW,DW); \
EXTERN_C DW STUB_EXPORT yact_##name(DW *R) \
{ \
DW r=name(p1,p2,p3); \
LEAVE(3); \
return r; \
}
...
DEFINE_FUNC1(AddAtomA)
DEFINE_FUNC1(AddAtomW)
DEFINE_FUNC7(CreateFileA) // as you see - implementation part is identical to an x86 stub, so I can use the same stub-generator tool
Some of the functions require complex emulation due to their absence in ARM or due to the callbacks to x86 code:
Code:
static DWORD WINAPI ThreadProc(
LPVOID lpParameter // [0] == orig func, [1] == orig param
)
{
__EXCEPTION_REGISTRATION_RECORD R;
DWORD *Parm=(DWORD*)lpParameter;
DWORD *TEB=(DWORD*)PeLdrGetCurrentTeb();
R.Next=(__EXCEPTION_REGISTRATION_RECORD*)-1;
R.Handler=(void*)CbReturnToHost();
TEB[0]=(DWORD)&R; // in case of unhandled exception - just return
PeLdrNotifyNewThread(NULL,DLL_THREAD_ATTACH);
DWORD Ret=EmuExecute(Parm[0],1,Parm[1]); // 1 == number of parameters to the emulated function
delete Parm;
return Ret;
}
EXTERN_C DW STUB_EXPORT yact_CreateThread(DW *R)
{
DWORD* Parm=new DWORD[2];
Parm[0]=p3; // TODO: no out-of-memory checking for now
Parm[1]=p4;
DWORD StackSize=p2;
if(StackSize)
StackSize+=1024*1024; // I reserve some space for my own needs (debugging)
else
StackSize=2*1024*1024; // TODO: I don't support autogrow stacks, so reserve 2 Mb
DWORD t=(DWORD)CreateThread((LPSECURITY_ATTRIBUTES)p1,StackSize,ThreadProc,Parm,p5,(LPDWORD)p6);
LEAVE(6);
return t;
}
Some of the COM interfaces are already implemented, for example DirectDraw and DirectSound, though not heavily debugged. On a desktop emulator build I can already run "Heroes of might and magic 3" and old WinRAR, but there are several RT-specific OS limitations I need to bypass before making them run on ARM. Current work in progress is: overcoming the RT limitations, manually implementing the API functions that callback to a program code (like CreateThread, RegisterClassA and so on), adding stubs for other system DLLs/COM objects.
Manually thrown SEH exceptions are fully supported, but access violation, int3 and similar OS-generated exceptions would cause program to crash. Some of the TEB fields (TLS and the fields required by the Borland compilers) are implemented too.
I don't make pointer translation in an emulated code nor make parameter checks passed to API. As a side-effect - the emulated program may trash the emulator in memory, but this greatly increases speed.
Most of the x86 EXE files don't contain relocations section and need to be loaded on the specific addresses (typically 0x400000). This is not a problem on a desktop, as I can rebase my emulator's EXE to any address I need, and free the corresponding RAM addrs for emulated program, but on ARM - this is a main problem. So currently only EXEs with relocs are supported for emulation, but there are ways to overcome this problem. And some EXEs produced by old Borland compilers contain "broken" relocs, this is a small problem too.

HI mamaich, sorry for disturbing you, may i know how do you compile visual studio project for arm ?, i already change windowsarmdesktop to true. But i can't find arm options in build settings. Any suggestion ?

rheza02 said:
HI mamaich, sorry for disturbing you, may i know how do you compile visual studio project for arm ?, i already change windowsarmdesktop to true. But i can't find arm options in build settings. Any suggestion ?
Click to expand...
Click to collapse
This is completely unrelated to the topic and has been covered in at least 3 threads, multiple times in each, in the past couple days. Use the search function.

netham45 said:
This is completely unrelated to the topic and has been covered in at least 3 threads, multiple times in each, in the past couple days. Use the search function.
Click to expand...
Click to collapse
Yes, and I've answered it here: http://forum.xda-developers.com/showpost.php?p=36644799&postcount=131

Wow, this is awesome work! While recompiling (for native speed, lower memory footprint, launch time, lower battery usage, script transparency, etc.) is still obviously preferred, this finally offers a way to run closed-source or otherwise un-recompilable legacy apps. Well done; I'll be watching this closely.
A thought for making it easier to run the apps (including the aforementioned script transparency): Windows (at least on x86 and x64, and I'm pretty sure on ARM too) supports specifying executable names that, when they would be executed, are instead passed as a parameter to another executable. This is usually used for testing or debugging purposes (for example, always load a given app under a debugger or have Application Verifier hook into it at launch) but it can be used for other purposes too. One, which would be fantastic here, is to always run a program through a compatibility layer... I've never before seen it used for a full instruction set translation compatibility layer, but why not?
Create the key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\<IMAGE FILE NAME>
i.e. HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\starcraft.exe
Then create a REG_SZ (String) value under that key called "Debugger" and set the value to the full path of the program you want to host the executable.
i.e. C:\Program Files\x86_peldr\peldr.exe
Source: http://support.microsoft.com/kb/824344
I can't promise that this will work for executable images that wouldn't be loadable normally, but it's probably worth a shot. Another, slightly less seamless option: change the extension of the executables (for example, starcraft.ex86) and register your app as the handler for that file type.

GoodDayToDie said:
Wow, this is awesome work! While recompiling (for native speed, lower memory footprint, launch time, lower battery usage, script transparency, etc.) is still obviously preferred, this finally offers a way to run closed-source or otherwise un-recompilable legacy apps. Well done; I'll be watching this closely.
A thought for making it easier to run the apps (including the aforementioned script transparency): Windows (at least on x86 and x64, and I'm pretty sure on ARM too) supports specifying executable names that, when they would be executed, are instead passed as a parameter to another executable. This is usually used for testing or debugging purposes (for example, always load a given app under a debugger or have Application Verifier hook into it at launch) but it can be used for other purposes too. One, which would be fantastic here, is to always run a program through a compatibility layer... I've never before seen it used for a full instruction set translation compatibility layer, but why not?
Create the key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\<IMAGE FILE NAME>
i.e. HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\starcraft.exe
Then create a REG_SZ (String) value under that key called "Debugger" and set the value to the full path of the program you want to host the executable.
i.e. C:\Program Files\x86_peldr\peldr.exe
Source: http://support.microsoft.com/kb/824344
I can't promise that this will work for executable images that wouldn't be loadable normally, but it's probably worth a shot. Another, slightly less seamless option: change the extension of the executables (for example, starcraft.ex86) and register your app as the handler for that file type.
Click to expand...
Click to collapse
What I was thinking of for doing that was hooking the 'This doesn't run on this PC' error that explorer gives and making it call this instead of the error.

Nice... but that doesn't cover things like launching a program from the command line (script or manually), or one program launching another, or launching Windows services (although I'm not sure you'd want to emulate those anyhow... the battery hit would suck), or so on. Still an interesting goal.
One other thought for the "it'd-be-awesome" list (not even really a wish-list): support doing this for DLLs loaded into other processes (i.e. somehow get between LoadLibrary and the x86 binary, and interpose your translator). Why, you ask? Plugins. Browser plugins, media codecs (which are DLLs, whatever their extension), Control Panel files, COM objects in general, etc.

## PRO-TIPS for your (non-pro) Windows RT ##

Well I guess this is more of a 'list of utilities, commands, and features' that might be buried so deeply into windows rt that you didn't know it was there. I don't know about other surface/winrt users but I have gotten more advanced configuring my surface rt than I ever did on an x86. Now I usually end up needing to configure my x86 devices to provide the same functionality I expect on my tablets.
So I don't think we have a list like this and I'm sure a lot of you have your own 'pro-tips', so maybe we can compile a list of hidden gems we can use with windows rt into this thread so that we collectively can reference or individually don't forget
So here is a short list to start off :
Utilities -- Many more utilities can be found by browsing c:\ProgramData\Microsoft\Windows\Start Menu\Programs
WF.msc (type at command prompt or Win-R key run window) : (Windows Firewall with Advanced Security) although they removed the control panel applet it seems they left in this mmc plugin.
gpedit.msc : group policy editor... somewhat hidden but they left this in
resmon (Resource Monitor)
msconfig (System Configuration)
msinfo32 (System Information)
steps (Steps Recorder) : this is a utility that will capture your activity as screenshots to play back or upload for others to see.
Modify whether links or html files open in Desktop or Metro versions of Internet Explorer :
Open desktop Internet Explorer, click on gear icon and choose Internet options. Now click on 'Programs' tab and select your preference from the 'Opening Internet Explorer' section dropdown.
Keyboard Shortcuts :
Metro snap via keyboard : Hold Windows and tap . (period) Keep holding down windows key and tap left, right, up, or down
Win-Tab : While alt-tab lets you cycle 'desktop' programs, Windows key + Tab cycles through 'metro' app list
Win-X : Show advanced start menu (same as right clicking start menu)
Win-D : Show/Hide desktop
Win-P : Pick from various multi-monitor configurations (duplicate/extend/pc only/external monitor only)
Console Commands
'HELP' : Show a list of all commands you can get more help on.
'WHERE' : Type where wf.msc (for example) and it will tell you where that program/file exists if it is in any of your PATH'ed folders.
'WMIC' : a very deep well of information, type WMIC /? to see list of all modules
wmic volume list brief (shows all mounted drives/volumes)
wmic qfe list brief (shows all installed windows updates)
'SYSTEMINFO' : another system info command, sort of a command line version of msinfo32
'TREE' : show an 'ascii-art' tree of sub-folders for the folder you are in.
'ASSOC' : command line ability to set file associations
'CERTUTIL' : can be used (among other things) for computing file hashes
certutil -hashfile [filename] [MD5|SHA1|SHA256] so for example :
certutil -hashfile myfile.zip sha1
Associations / Context menu / 'Send To' programs :
Create a shortcut to control.exe /name Microsoft.DefaultPrograms to quickly go to this control panel applet.
Use command line ASSOC to set file associations
Manage 'Send To' folder by opening 'windows' explorer, typing shell:sendto into the location bar to be taken to a folder showing all your 'send to' programs'. You can drop shortcuts to other programs (provided those programs accept command line arguments). Now you can right click a file in windows explorer and 'send to' programs like 7-Zip, Paint.NET, notepad++, etc.
Add an "Open with Notepad++" or other top level context menu for all files :
Open registry editor and browse to : HKEY_CLASSES_ROOT\*\shell.
Right-click on “shell” and choose to create a new key, calling it “Open with Notepad++”
Create a new key below that one called “command”
Double-click on the (Default) value in the right-hand pane and enter "c:\path_to_npp\notepad++.exe" $1
To have the menu item show the notepad++ icon do these extra steps :
Right click the "Open with Notepad++" key (which you created) and add a new string value called "icon"
Double click new "icon" value and set its string value to "c:\path_to_npp\notepad++.exe"
In all steps above make sure you enter the path to notepad++ on your machine instead of the path entered.
http://www.howtogeek.com/howto/wind...th-notepad-to-the-context-menu-for-all-files/
Make 'command prompt here' context menu when right clicking folder in windows explorer : http://www.howtogeek.com/howto/wind...-always-display-for-folders-in-windows-vista/
Enable 7-Zip Context Menu :
Run regedit (Registry Editor), and drill into / expand HKEY_CLASSES_ROOT\*\shellex\ContextMenuHandlers.
Now right click on 'ContextMenuHandlers' and choose 'permissions'.
Click 'Advanced' button.
On advanced Security Settings screen (at the top) choose Change (owner) link.
When selecting a user or group screen comes up click 'Advanced', then 'Find Now', and in search results choose 'Administrators' (may need to widen column to distinguish from singular 'Administrator').
After highlighting Administrators, click OK button to go back to Advanced Security Settings screen.
Now you are able to click on 'Administrators' and click 'Edit' and select 'Full Control' and click ok to apply permission.
Keep clicking ok on dialogs till you are back at registry editor.
Now run 7-Zip as administrator, go into tools menu and select options.
Then on 7-Zip tab choose 'Integrate 7-Zip to shell context menu' and click ok to close dialog.
Context menu should now appear when right clicking on files in windows explorer.
Disabling Connected Standby :
This is only a good idea if your tablet will always be left on a charger or 'shut down' when not in use. If you turn off connected standby and set your display to turn off after... say 5 minutes, then after 5 minutes your display will really power off (backlight as well) but any desktop programs will continue to run. You might want to do this if you are running a web server or some other process in background. When you disable connected standby, many other options appear in your power options 'advanced power settings' page. If you do this I recommend you 'Create a power plan' on the 'Choose or customize a power plan screen first'.
To toggle connected standby feature on or off, modify the :
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Power\CsEnabled key, setting to 0 for 'off' or 1 for 'on' and reboot to take effect.
Node Chakracore :
This is really an undocumented bonus 'add-on', made by Microsoft which you can run even on windows rt devices which are not in test-signing mode. For those who are not familiar, NodeJS is a console-based javascript scripting environment. Standard Node.js uses v8 javascript engine (same as google chrome), however this 'node-chakracore' version uses the chakra javascript engine (which is the latest version used in edge browsers). Microsoft is providing this for raspberry pi users running Windows 10 on ARM but they also work on Windows RT.
I made a utility here if you prefer to try an automated means of installing
Install Node.JS (node-chakracore) :
Download the latest ARM release from : https://github.com/nodejs/node-chakracore/releases
Extract that archive to (for example) c:\nodejs
Add that directory (which you extracted to) to your path environment variable.
(Optional) Install/Setup NPM (lets you install packages from npm repository) :
Download https://github.com/npm/npm/archive/master.zip, save somewhere.
Extract it to (for example) c:\npm.
Open windows explorer and type %AppData% into the address bar and hit enter.
Within your AppData directory, create an "npm" directory.
Double click on npm folder you just created from windows explorer
Click into the address bar to see the full path and remember (or copy) this path for next step and later.
Save the following to file named "npm.bat" and save into the directory you just created/copied
Code:
@node c:\npm\npm-master\cli.js %*
(If you extracted npm elsewhere, update the batch file to reflect actual location)
Create an etc directory under this appdata npm folder
Create an separate etc directory under your node folder (c:\nodejs if you used suggested default)
Copy .npmrc file from c:\npm\npm-master to c:\node
Rename and get rid of the '.' from the filename so the filename is just npmrc
edit that npmrc file In notepad and add the following lines :
Code:
globalconfig=c:\users\username\appdata\roaming\npm\etc\npmrc
prefix=c:\users\username\appdata\roaming\npm
(make sure to substitute your username in the above this base folder is the folder I told you to remember/copy in previous steps)
Having saved those changes to npmrc under c:\node\etc, copy that file to your appdata\npm\etc directory as well.
Add the path of your appdata\npm directory your path environment variable. If you install global packages they will install commands/batch files into this folder. If it is pathed you can exec them as native commands. This path is also where we put the npm.bat so it can be run from anywhere.
A useful demonstration if you have configured the both npm and node correctly is to install cash (an implementation of unix shell commands) by typing from a command prompt :
Code:
npm install -g cash
That should take a several minutes to complete (hopefully successfully). When done you can run 'cash' or just '$' at a command prompt to bring up simulated Linux prompt from which you can type 'help' to see supported commands. These commands interop with windows commands as well so you can mix (see cash link above).​
(Thanks to @black_blob for discovering this node/npm)
I will add more info or cleanup as needed, feel free to post your own, if needed I can compile it into first post to work towards a guide if anyone wanted to make one.

I modded the Windows 10 Chakra(non-core) to run on 8.1 as a part of my earlier NodeJS port. By the way, I also had Microsoft Edge running with it at a time.(EdgeHTML)
I wonder if I should continue my backports from Win10

You mean other programs/frameworks? You have brought over some great stuff, so yes absolutely! Anything you have in mind?

nazoraios said:
You mean other programs/frameworks? You have brought over some great stuff, so yes absolutely! Anything you have in mind?
Click to expand...
Click to collapse
I have in mind to backport the UWP framework, or maybe even a FrankenKernel(10 kernel, 8.1 userspace). Microsoft Edge works without needing UWP, as you can use edgehtml like mshtml for embedding

You would definitely be my hero if you could bring over Edge, preferably with somewhat recent (EdgeHTML? rendering) and Chakra javascript engine.
Not sure how FrankenKernel and UWP support could work out but both could potentially be awesome... guess its the details which I'm sure you have a better idea than me. So do you think we would be able to run (open source?) Windows 10 apps and would the 'developer mode' be brought over with the kernel or is that 8.1 userspace which would require 8.1 developer license signing (and renewing every 6 months)? Or would the UWP part just be for console or node-chakracore like it is on IOT?

nazoraios said:
You would definitely be my hero if you could bring over Edge, preferably with somewhat recent (EdgeHTML? rendering) and Chakra javascript engine.
Not sure how FrankenKernel and UWP support could work out but both could potentially be awesome... guess its the details which I'm sure you have a better idea than me. So do you think we would be able to run (open source?) Windows 10 apps and would the 'developer mode' be brought over with the kernel or is that 8.1 userspace which would require 8.1 developer license signing (and renewing every 6 months)? Or would the UWP part just be for console or node-chakracore like it is on IOT?
Click to expand...
Click to collapse
Windows 10 kernel & system, Windows 8.1 explorer.exe, dwm and other tiny GUI bits
I have a Windows Server ARM64 build, but that's not 32-bit

Yea that sounds daring... Don't brick your device please Too bad no 32bit gui bits to borrow. I doubt they would bring the Ubuntu for Windows 10 subsystem to arm, but technically i'm sure they could since Ubuntu has arm and pi2 image.... still would be neat. Also I think they added ssh server to pi2 but not sure if it would be easy to 'extract' and borrow.
We will be watching you commander...

nazoraios said:
Yea that sounds daring... Don't brick your device please Too bad no 32bit gui bits to borrow. I doubt they would bring the Ubuntu for Windows 10 subsystem to arm, but technically i'm sure they could since Ubuntu has arm and pi2 image.... still would be neat. Also I think they added ssh server to pi2 but not sure if it would be easy to 'extract' and borrow.
We will be watching you commander...
Click to expand...
Click to collapse
The Ubuntu support uses the Linux binary compatibility SubSystem, also known as Astoria

black_blob said:
I have in mind to backport the UWP framework, or maybe even a FrankenKernel(10 kernel, 8.1 userspace). Microsoft Edge works without needing UWP, as you can use edgehtml like mshtml for embedding
Click to expand...
Click to collapse
That would be even more amazing than a running build of Windows 10 Mobile. So you'd have a "full" OS with the befinits of running UWP. That would def. insure the future of Windows RT.
It would be even better if astoria would work on it too. So you'd even able to run android apps.

I need help for the nodejs and the npm instructions. Can you explain it out in details for morons?

Sorry late reply... which steps are you getting confused at (for node/npm install) ? So for starters :
- if you open a command prompt and type "node -v" does it tell you the version of node your running? If so, what version does it say?
- if you type "npm -v" does it tell you the version of npm you are running or give an error? Tell version or error.
So i'll start with that to make sure your paths are set up.

I created an installer utility to install node-chakracore along with NPM. Hopefully this will make it easier for other or myself to install or upgrade as new versions of node-chakracore and npm come out.
The final step in the process (updating the system PATH environment variable) requires admin permissions so right click executable and run 'As Administrator'. If you prefer to set the environment variables yourself, this step will tell you the paths it would add.
9/13/16 : Updated default node chakracore download link to new 7.0.0-pre9 release, added a 'Complete' tab with links to further resources and made UI fix

Share exercise data from the Huawei Health app

Huawei TCX Converter
A makeshift python tool that enables the extraction of TCX files from the Huawei Health app. Your phone must be a Huawei Phone or Rooted to access Huawei Health app data!
Introduction
Users of Huawei Watches/Bands sync their fitness data with the Huawei Health App. It is notoriously difficult to get the data out of this app, but through some cunning you can find HiTrack files which seem to contain some run data. This program allows you to take these files and generate .TCX files for use in your tracking app of choice (e.g. Strava). The outputted .TCX files will contain timestamped GPS, altitude, heart-rate, and cadence data where available.
Accessing the App Data
The Huawei Health app generates a 'HiTrack' file when it displays the trajectory of an particular exercise event. We can access these as follows:
Rooted phones
Navigate to data/data/com.huawei.health/files/
Copy the HiTrack files to your computer
Unrooted Huawei phones
Download the Huawei Backup App onto your phone.
Start a new unencrypted backup of the Huawei Health app data to your external storage (SD Card)
Navigate to Huawei/Backup/***/backupFiles/***/ and copy com.huawei.health.tar to your computer
Unzip the file and navigate to com.huawei.health/files/ and you should should see a number of HiTrack files
Using the Huawei TCX Converter
You need python 3 to use this tool.
Download the Huawei TCX Converter and save it as a Python script in the same folder as your HiTrack file.
The tool is run on the command line by passing it the name of your file as a command line argument. Other command line arguments:
-v - validate the final TCX file in order to check that the conversion has worked (requires xmlschema and an internet connection to download the TCX schema to check against)
-f - attempt to filter out any records in which GPS signal was lost, or cadence/heart-rate/altitude are invalid
-b - change sport to Biking
You can rename your HiTrack files if you wish, but for clarity in the examples below I leave mine exactly as I found it.
Illustration
I have copied the Huawei-TCX-Converter.py file to the directory containing my HiTrack file (HiTrack_1551732120000155173259000030001). Now I can run the tool as follows:
Code:
python Huawei-TCX-Converter.py HiTrack_1551732120000155173259000030001 -f -v
This gives me the output:
Code:
---- Input File ----
reading: OKAY
filtering: OKAY
processing gps: OKAY
processing heart-rate/cadence: OKAY
---- Details ----
sport: Running
start: 2019-03-04 20:42:00
duration: 00:07:49
distance: 1700m
---- XML file ----
generating: OKAY
saving: OKAY
validating: OKAY
Here are both the HiTrack file and the resultant TCX file for you to have a go with.
Next steps
The files are now ready (e.g. for upload to Strava). Some users have recommended the TCX Converter tool to add altitude data to your TCX files once they've been converted. This may overwrite altitude data extracted from your device, if it collects this.
Comparison
This is an image of the GPS trace from the .tcx file. The command line output above also lists the start time as 2019-03-04 20:42:00, the distance as 1.70km, and the duration as 00:07:49.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
For comparison, below is the data visable on the Huawei Health App. You can see that the distance is off by about 80m, and the duration off by 1 second, but the GPS trace is spot on.
Contributing
Go to the Github Hompage of this tool to contribute.
This is a very early alpha version of this tool, so please help me by making it better! There are some scripts in the Development Tools folder that I find useful for debugging. I'll accept any improvements, but if you're looking for inspiration you could start with this to-do list:
Remove reliance on using the original filename
Enable changing sport type from running (default) to biking
Read timestamped heart-rate, cadence, and altitude data where available
See if we really need to add the unused data elements (e.g. Calories) to the TCX (edit: we do as there is no minOccurs in the schema)
Check that this works for files other than those generated using the Huawei Band 2 Pro:
Confirmed working on a file from a Huawei Watch GT
Improve the distance measurement method (currently using Viscenty's Formulae)
Try and work out what tp=b-p-m is
Add interpolated heart-rate/pace/average speed data to each location element
Work on splitting data into Laps/Tracks rather than shoving it all into one
Try to call on an open API to get altitude data for location points that don't have it
Inspect other files in com.huawei.health to see if we can get any more relevant data out of them
XDA:DevDB Information
Huawei TCX Converter, Tool/Utility for the Smart Watches
Contributors
aricooperdavis
Source Code: https://github.com/aricooperdavis/Huawei-TCX-Converter
Version Information
Status: Testing
Created 2019-03-11
Last Updated 2019-03-11

Thanks for sharing man! I can't work out if I have a use for this, because I'm already using google fit and strava. But I would like to be able to extract sleep history from Huawei Health. Is that something you would plan to look at? I guess step data and continuous heart rate would be good too. I don't like being locked into an app with no cloud backup/access.

mrlb said:
I would like to be able to extract sleep history from Huawei Health.
Click to expand...
Click to collapse
So far I haven't found anything resembling sleep data in those HITRACK files - I think sleep data must be held elsewhere. I will keep poking around, but at the moment it's probably not at the forefront of my efforts as the other databases are encrypted so I cannot access them to read info. I'll still add it to the todo list though!

Dears, Thx for the type but i'm locked in step with the tar files. I've backuped it with Hisuite on my computer (not SD 'cause i didn't have it - bad version of P20).
But i can't extract the TAR files, it's seems it's corrupt. I've already tried to save the output TAR to different folders on my computer, on my local network etc.. but same issue. TAR is corrupte.
Did you have any idea ?

The best!!!
it works perfectly, Thanks!!!!!

works great... everything is great... only off by several number but overall satisfied...
also i have the schema file "TrainingCenterDatabasev2.xsd"... where should i put it? or this should be done automatically? thanks...

It worked very well !! How to implement more things in this great code? for example the sport swimming I tried most does not convert to .tcx so it is not possible to import to strava. how to import racing data about time and cadence in every 1km race? Thanks for what we already have works very well

negbd said:
Dears, Thx for the type but i'm locked in step with the tar files. I've backuped it with Hisuite on my computer (not SD 'cause i didn't have it - bad version of P20).
But i can't extract the TAR files, it's seems it's corrupt. I've already tried to save the output TAR to different folders on my computer, on my local network etc.. but same issue. TAR is corrupte.
Did you have any idea ?
Click to expand...
Click to collapse
Hi
maybe somehow crypted TAR ?

Thanks for making this it's really useful, shame the GT watch GPS is not very accurate and/or low sampling rate. The same run/cycle etc recorded on other devices is far more accurate and detailed. A 30 mile bike ride was 1.5 miles less on the Huawei watch! (Nothing you can do about that as it's the same in health app and after exporting to strava sadly)

It is working

Hi all,
I have created a petition asking Huawei to add export and sync to Strava. Please sign if you agree.
https://ipetitions.com/petition/huawei-please-add-gpxtcx-export-and-sync-to
Thanks

Signed

I was trying to create a backup using the HiSuite and the Backup and Restore app but the Heath App does not show up in the app list, anymore.
Can someone else confirm this?

Hello,
I'm planning to buy huawei smartband, but i want to export heart rate to google fit / garmin app...etc
Heart rate about the day, not about an activity.
With this is possible?

Hi sir/mam, im having issue running the latest version 2.3... im still using the old version with -v -b arguments... it is working with me but not the newest version 2.3... here are my batch file and error result... and how to use XML validation... i already downloaded the XSD file... but still nothing... then how to use PIP INSTALL XMLSCHEMAS... thanks...
----------------------------
batch file
---------------------------- @Echo off
echo.
"C:\Users\*******\AppData\Local\Programs\Python\Python37\python.exe" Huawei-TCX-Converter.py --file %1 --sport Cycle
echo.
pause
----------------------------
error output
----------------------------
Info - External library xmlschema could not be imported.
It is required when using the --validate_xml argument.
It can be installed using: pip install xmlschema
Traceback (most recent call last):
File "Huawei-TCX-Converter.py", line 926, in <module>
class TcxActivity:
File "Huawei-TCX-Converter.py", line 944, in TcxActivity
filename_prefix: str = None):
NameError: name 'xmlschema' is not defined

python Huawei-TCX-Converter.py --file HiTrack_1569078600000156908002700030001
File "Huawei-TCX-Converter.py", line 54
def __init__(self, activity_id: str, activity_type: str = TYPE_UNKNOWN):
^
SyntaxError: invalid syntax
I get this error if anyone can help me

python Huawei-TCX-Converter.py -f HiTrack_1551732120000155173259000030001 - this syntax works for me in newer version of python + dont forget to install schema = pip install xmlschema = search for youtube video called PIP INSTALL COMMAND IN PYTHON 3.6 ( i cannot post links as new user so sorry for that)
BUT
i also have honor band 4 running edition with tracking angles, forces etc. When i create files from those records - data are not here becouse they are not defined to translate in python program. can somebody help me with providing sources from garmin etc so i will try to add it by myself if nobody is interested to do this ?
the file from HB 4 running edition consists of those records : tp=rp;k=5;gct=532;gia=12;sa=65;ee=15;fsp=0;wsp=1;hsp=1;
can somebody help me to translate this?
from my own research so far -
1. ground contact time = GCT
2. Landing impact = gia
3. eversion excursion = ee
4. swing angle = sa
5. forefoot strike pattern = fsp
6. W? midfoot strike pattern = wsp
7. Heel strike pattern = hsp
How those things should be written in tcx files?

not work
Hello
I don'know python, I just launch the IDLE program from windows menu, I copied the script in the same hitrack folder and I type in the python shell the command
python Huawei-TCX-Converter --file HiTrack_myfilename
It shows syntax error and higlight in red the command "huawei"
Can you explain easily how to proceed? Is there a rule where put the folder?
Thanks

avvocato.rossi said:
Hello
I don'know python, I just launch the IDLE program from windows menu, I copied the script in the same hitrack folder and I type in the python shell the command
python Huawei-TCX-Converter --file HiTrack_myfilename
It shows syntax error and higlight in red the command "huawei"
Can you explain easily how to proceed? Is there a rule where put the folder?
Thanks
Click to expand...
Click to collapse
just read my previous post
syntax python Huawei-TCX-Converter.py -f HiTrack_1551732120000155173259000030001 works

regarding Distance Calculation (haversine instead of viscenty)y)
@aricooperdavis
Checked another calculation for distance between coordinates (haversine) but the results were the same (in your example: 1700.43m for Viscenty, 1699.3m for Haversine). I really do not know which is the algorithm used in HUAWEI Health (80 meters difference)
FYI, code used:
def _haversine(self, point1: tuple, point2: tuple) -> float:
R = 6378137
Phi1 = math.radians(point1[0])
Phi2 = math.radians(point2[0])
dPhi = math.radians(point2[0]-point1[0])
dLamda = math.radians(point2[1]-point1[1])
a = math.sin( dPhi/2 ) * math.sin(dPhi/2) + math.cos(Phi1) * math.cos (Phi2) * math.sin (dLamda/2)* math.sin(dLamda/2)
c = 2 * math.atan2(math.sqrt(a), math.sqrt(1-a))
d = R * c
return round (d,6)
https://www.movable-type.co.uk/scripts/latlong.html