Hi,
I've been trying to get my own kernel with few modifications running on my ASUS Transformer. I've followed few guides around with no luck. What I did:
Tried two source trees:
1) Official from ASUS
2) Roach2010s tree from github (https://github.com/Roach2010/android_kernel_TF101.git)
Used .config from my current kernel which is running fine (Prime kernel) without any changes.
Compiled kernel.
So far looks good, with few modifications to config I got new modules that works so crosscompiler is not misscompiling. Now the part where I'm doing something wrong and can't figure out what.
I started with Prime Kernel from http://forum.xda-developers.com/showthread.php?t=1251044
* Unziped the archive
* blobunpack blob
* created blob.LNX in several ways described bellow
* blobpack blob.HEADER blob LNX blob.LNX
* dd if=blob of=/dev/block/mmcblk0p4
* reboot
How I created blob.LNX
A) Use extracted blob.LNX and use abootimg to replace kernel
* abootimg -u blob.LNX -k zImage
B) Extracted all parts and recreated image using abootimg
* abootimg -x blob.LNX
* abootimg --create blob.LNX -f bootimg.cfg -k zImage -r initrd.img
C) Extracted all parts and recreated image using bootunpack and mkbootimg
* bootunpack blob.LNX
* mkbootimg --kernel zImage --ramdisk ramdisk.gz -o blob.LNX
In addition I tried few modifications:
* enlarging bootsize in bootimg.cfg to make sure everything fits
* passing cmdline my current kernel booted up with as default in .config, as cmdline in bootimg.cfg and both
All my efforts ended up on ASUS boot up screen, no matter what I try. So my question is, am I missing something? Did I skipped some important part? Have I done something wrong? Any ideas appreciated.
If nobody has any idea, can somebody please create kernel with enabled kexec for my ASUS Transformer? That was the ultimate goal of trying to get my own kernel, but if I can't get working just recompiled one...
-miska- said:
Hi,
I've been trying to get my own kernel with few modifications running on my ASUS Transformer. I've followed few guides around with no luck. What I did:
Tried two source trees:
1) Official from ASUS
2) Roach2010s tree from github (https://github.com/Roach2010/android_kernel_TF101.git)
Used .config from my current kernel which is running fine (Prime kernel) without any changes.
Compiled kernel.
So far looks good, with few modifications to config I got new modules that works so crosscompiler is not misscompiling. Now the part where I'm doing something wrong and can't figure out what.
I started with Prime Kernel from http://forum.xda-developers.com/showthread.php?t=1251044
* Unziped the archive
* blobunpack blob
* created blob.LNX in several ways described bellow
* blobpack blob.HEADER blob LNX blob.LNX
* dd if=blob of=/dev/block/mmcblk0p4
* reboot
How I created blob.LNX
A) Use extracted blob.LNX and use abootimg to replace kernel
* abootimg -u blob.LNX -k zImage
B) Extracted all parts and recreated image using abootimg
* abootimg -x blob.LNX
* abootimg --create blob.LNX -f bootimg.cfg -k zImage -r initrd.img
C) Extracted all parts and recreated image using bootunpack and mkbootimg
* bootunpack blob.LNX
* mkbootimg --kernel zImage --ramdisk ramdisk.gz -o blob.LNX
In addition I tried few modifications:
* enlarging bootsize in bootimg.cfg to make sure everything fits
* passing cmdline my current kernel booted up with as default in .config, as cmdline in bootimg.cfg and both
All my efforts ended up on ASUS boot up screen, no matter what I try. So my question is, am I missing something? Did I skipped some important part? Have I done something wrong? Any ideas appreciated.
If nobody has any idea, can somebody please create kernel with enabled kexec for my ASUS Transformer? That was the ultimate goal of trying to get my own kernel, but if I can't get working just recompiled one...
Click to expand...
Click to collapse
Here is what I've done. If you have successfully built a kernel with the resulting zImage, then you are part way there, I believe there is a kernel config option to enable kexec support but I haven't tried that. Next, you can take some other kernel's .zip file (CWM flashable) and unzip it. You may need to download a zip utility. You'll have 2 folders and a kernel blob. If you bootunpack this kernel blob, you'll end up with the kernel blob and a some *.LNX file. This *.LNX file is the same as a boot.img file. You can use dsixda's Android kitchen to split this into the initrd and the kernel (zImage) parts. Replace the zImage with your own and move any modules you may have also built if necessary into the initrd part, join them back together into a boot.img in the kitchen.
copy this boot.img back to where you unzipped the kernel.zip, delete the original *.LNX file, rename the boot.img to the same name as the previous *.LNX file and then bootpack it together and flash it through CWM. Zip the 2 folders and the kernel blob you just made back together with whatever name you want. You can edit the text in the updater script before you zip it all up, but whether you do or not it should boot.
Yes, there is kexec config option, but I haven't suceeded even without changing anything so enabling it doesn't make kernel boot :-D I tried Android Kitche to split boot image and I ended up with the same files (compared the content to check) as with abootimg -x. Tried recreating update.zip and sign it using Android Kitchen, just to be sure, that something in android is not in the way to the actualization from running system. Still no luck :-(
-miska- said:
Yes, there is kexec config option, but I haven't suceeded even without changing anything so enabling it doesn't make kernel boot d:-D I tried Android Kitche to split boot image and I ended up with the same files (compared the content to check) as with abootimg -x. Tried recreating update.zip and sign it using Android Kitchen, just to be sure, that something in android is not in the way to the actualization from running system. Still no luck :-(
Click to expand...
Click to collapse
I didn't even sign mine as I have signature verification turned off in CWM recovery. Didn't change the text either as I was mostly experimenting and learning. I know kexec works under linux, but I think it requires separate package(s) and configuration to do so. I got a bit confused with blobpack, blobunpack info, but figured out that with just the kernel you don't seem to need the mentioned header file, just the .LNX which is the same as boot.img which is the combined kernel zImage and initramfs. If the kernel blob is still there and you use the same name for the output file then it doesn't matter because it will get overwritten anyway. Worked for me at least using source of kernel I've booted before and my running .config.
sidneyk said:
I didn't even sign mine as I have signature verification turned off in CWM recovery. Didn't change the text either as I was mostly experimenting and learning. I know kexec works under linux, but I think it requires separate package(s) and configuration to do so. I got a bit confused with blobpack, blobunpack info, but figured out that with just the kernel you don't seem to need the mentioned header file, just the .LNX which is the same as boot.img which is the combined kernel zImage and initramfs. If the kernel blob is still there and you use the same name for the output file then it doesn't matter because it will get overwritten anyway. Worked for me at least using source of kernel I've booted before and my running .config.
Click to expand...
Click to collapse
hmmm, zip file I had as an example was using blobed boot image going through mmcblk0p4. Do you have some link to .zip file that does it differently?
kexec is a way how to boot something else from linux directly without need to fiddle with bootloader. To use it, two parts are needed - kernel that supports it (that's what I can't get) and tool to actually use it/call it. Tool is not a problem, got that one hopefully ready, but without the kernel...
-miska- said:
hmmm, zip file I had as an example was using blobed boot image going through mmcblk0p4. Do you have some link to .zip file that does it differently?
kexec is a way how to boot something else from linux directly without need to fiddle with bootloader. To use it, two parts are needed - kernel that supports it (that's what I can't get) and tool to actually use it/call it. Tool is not a problem, got that one hopefully ready, but without the kernel...
Click to expand...
Click to collapse
Have you tried Koush's "anykernel.zip" code (probably requires a few mods)? It appears to be trying to replace the blob based updater-scripts that are all over the place.
I've used it successfully, but it has mostly been on other devices, and it is very easy to use. I think some of the templates are too generic and maybe confusing, and without figuring out how edify scripting actually works, it is mysterious, but I'd look at this code, git it and try to use it:
I'll try to provide a working example since I just added a few modules to one of the kernels 2.6.36-4 that're out there for the tf101, but I need to be sure it's working first. I think there's perhaps one difference at least between what Koush shows for the xoom and the tf101 so am working on it.
https://github.com/koush/AnyKernel
Good luck -
-miska- said:
hmmm, zip file I had as an example was using blobed boot image going through mmcblk0p4. Do you have some link to .zip file that does it differently?
kexec is a way how to boot something else from linux directly without need to fiddle with bootloader. To use it, two parts are needed - kernel that supports it (that's what I can't get) and tool to actually use it/call it. Tool is not a problem, got that one hopefully ready, but without the kernel...
Click to expand...
Click to collapse
I was using clemsyn-blades_kernel_ver22a zip file. I don't know if it was doing it different or not, haven't checked that far into it.
sidneyk said:
I was using clemsyn-blades_kernel_ver22a zip file. I don't know if it was doing it different or not, haven't checked that far into it.
Click to expand...
Click to collapse
hmmm, checked that one, uses blobed image and 'dd if=/tmp/blob of=/dev/block/mmcblk0p4' as well. Maybe I'll try different crosscompiler anyway, that's the one thing I haven't altered yet :-/
hachamacha said:
Have you tried Koush's "anykernel.zip" code (probably requires a few mods)? It appears to be trying to replace the blob based updater-scripts that are all over the place.
I've used it successfully, but it has mostly been on other devices, and it is very easy to use. I think some of the templates are too generic and maybe confusing, and without figuring out how edify scripting actually works, it is mysterious, but I'd look at this code, git it and try to use it:
I'll try to provide a working example since I just added a few modules to one of the kernels 2.6.36-4 that're out there for the tf101, but I need to be sure it's working first. I think there's perhaps one difference at least between what Koush shows for the xoom and the tf101 so am working on it.
Click to expand...
Click to collapse
Haven't tried that one, looks interesting... This one doesn't use blobed update and wites image directly somewhere. Just would require to check that that somewhere is the right place :-D Thanks, will take a look at that and what other edify commands are availble in updater, sounds like interesting alternative approach...
-miska- said:
Haven't tried that one, looks interesting... This one doesn't use blobed update and wites image directly somewhere. Just would require to check that that somewhere is the right place :-D Thanks, will take a look at that and what other edify commands are availble in updater, sounds like interesting alternative approach...
Click to expand...
Click to collapse
I'm modifying the script I've seen passed around (not quite Koush's git repo version) passed around to see if I can get it to work on the tf101. The 'write it somewhere' edify command is the question mark, but I think it is going on it's (the device's) internal partition table and vectored to 'boot', which is either a terrific generic idea, or terrible depending upon what edify does. I can't really find a heck of a lot explaining anything about the individual edify commands. I'm just getting rid of the 'showstoppers' where partition names like mmc0p* are used that are clearly wrong for the tf101. I made the mistake of trying one that I only later realized thought that partition 1 was data, when it is actually partition 7. Good thing I can make nvflash backups on my 'old' transformer.
I'll post back later today with any results I get. I'm not concerned about whether my kernel worked since it is completely experimental , just that it got written there, so I might use a working version with a different kernel name (in Makefile) just so I can get 'proof of concept' .
On a slightly different note but having to do with what you're doing, I tried the blob route this week, and for some reason, blobunpack/pack right from Rayman's git repo do not unpack the blobs correctly for say 'clemsyms' or 'Prime's' blobs, which has me wondering about some change that maybe took place. In any case, it forces me down this other path anyway.
If they are working OK for you, could you tell me a couple things?
1) Your linux distro and architecture (x86/x86_64)
2) did you build them from Rayman's repo? Did you get binaries from somewhere, if so where?
3) parameters? I don't think mine take any but the blob name.
4) Output suffixes. I only get .LNX from any of the above blobs which is useless.
EDIT: I was recalling that 'edify' in CWM came into being somewhere (maybe) past the only version that works with the tf101 (we're on ~v3.x and edify ~v4/5+). If that's the case, then we're all stuck with blobs because that one write command is edifi(ed) most likely. I'll stare at the git CWM source today too to figure out if it used the edify stuff in this version. I think Solarnz had it in his git hub.
hachamacha said:
I'm modifying the script I've seen passed around (not quite Koush's git repo version) passed around to see if I can get it to work on the tf101. The 'write it somewhere' edify command is the question mark, but I think it is going on it's (the device's) internal partition table and vectored to 'boot', which is either a terrific generic idea, or terrible depending upon what edify does. I can't really find a heck of a lot explaining anything about the individual edify commands. I'm just getting rid of the 'showstoppers' where partition names like mmc0p* are used that are clearly wrong for the tf101. I made the mistake of trying one that I only later realized thought that partition 1 was data, when it is actually partition 7. Good thing I can make nvflash backups on my 'old' transformer.
I'll post back later today with any results I get. I'm not concerned about whether my kernel worked since it is completely experimental , just that it got written there, so I might use a working version with a different kernel name (in Makefile) just so I can get 'proof of concept' .
On a slightly different note but having to do with what you're doing, I tried the blob route this week, and for some reason, blobunpack/pack right from Rayman's git repo do not unpack the blobs correctly for say 'clemsyms' or 'Prime's' blobs, which has me wondering about some change that maybe took place. In any case, it forces me down this other path anyway.
If they are working OK for you, could you tell me a couple things?
1) Your linux distro and architecture (x86/x86_64)
2) did you build them from Rayman's repo? Did you get binaries from somewhere, if so where?
3) parameters? I don't think mine take any but the blob name.
4) Output suffixes. I only get .LNX from any of the above blobs which is useless.
EDIT: I was recalling that 'edify' in CWM came into being somewhere (maybe) past the only version that works with the tf101 (we're on ~v3.x and edify ~v4/5+). If that's the case, then we're all stuck with blobs because that one write command is edifi(ed) most likely. I'll stare at the git CWM source today too to figure out if it used the edify stuff in this version. I think Solarnz had it in his git hub.
Click to expand...
Click to collapse
Blobs are used on the tf101 because they are the ONLY way of flashing boot/recovery, there is no block device mapping of them on our device
lilstevie said:
Blobs are used on the tf101 because they are the ONLY way of flashing boot/recovery, there is no block device mapping of them on our device
Click to expand...
Click to collapse
OK: Thanks lilstevie,
That takes care of that. Time for me to make peace with blobs.
After steve's reply, I just went to using blobs. I've got my own kernel running fine on the tf101 using that method.
For the best reference I've seen on using blobs and boottools , try this post:
http://forum.xda-developers.com/showthread.php?t=1193737
---
Just got back from work, will ply with it some more, but I'll start with answering the questions...
hachamacha said:
1) Your linux distro and architecture (x86/x86_64)
Click to expand...
Click to collapse
Gentoo x86-64
hachamacha said:
2) did you build them from Rayman's repo? Did you get binaries from somewhere, if so where?
Click to expand...
Click to collapse
Compiled from git repo. I always tried to find the most upstream repo for each tool and then compiled it by myself.
hachamacha said:
3) parameters? I don't think mine take any but the blob name.
4) Output suffixes. I only get .LNX from any of the above blobs which is useless.
Click to expand...
Click to collapse
These two comes together:
'blobunpack blob' - takes a blob as input and ouptuts blob.HEADER and blob.LNX
'bootunpack blob.LNX' - takes blob.LNX as input and outputs blob.LNX-kernel.gz, blob.LNX-ramdisk.cpio.gz and blob.LNX-config
'abootimg -x blob.LNX' - takes blob.LNX as input and outputs zImage, initrd.img and bootimg.cfg
Resulting files from bootunpack and abootimg are almost same, only difference is the configuration file
To repack:
'abootimg --create newblob/blob.LNX -f bootimg.cfg -k zImage -r initrd.img'
or
'mkbootimg --kernel zImage --ramdisk blob.LNX-ramdisk.cpio.gz -o newblob/blob.LNX'
and then
'blobpack blob.HEADER newblob/blob LNX newblob/blob.LNX'
Unless I change kernel, everything works just fine :-D
-miska- said:
Just got back from work, will ply with it some more, but I'll start with answering the questions...
Gentoo x86-64
Compiled from git repo. I always tried to find the most upstream repo for each tool and then compiled it by myself.
These two comes together:
'blobunpack blob' - takes a blob as input and ouptuts blob.HEADER and blob.LNX
'bootunpack blob.LNX' - takes blob.LNX as input and outputs blob.LNX-kernel.gz, blob.LNX-ramdisk.cpio.gz and blob.LNX-config
'abootimg -x blob.LNX' - takes blob.LNX as input and outputs zImage, initrd.img and bootimg.cfg
Resulting files from bootunpack and abootimg are almost same, only difference is the configuration file
To repack:
'abootimg --create newblob/blob.LNX -f bootimg.cfg -k zImage -r initrd.img'
or
'mkbootimg --kernel zImage --ramdisk blob.LNX-ramdisk.cpio.gz -o newblob/blob.LNX'
and then
'blobpack blob.HEADER newblob/blob LNX newblob/blob.LNX'
Unless I change kernel, everything works just fine :-D
Click to expand...
Click to collapse
Pretty similar, although the kernel zImage itself is always a mystery unless you've not changed anything, but even then, getting it built with the right toolchain, etc isn't guaranteed. So lets assume that just works for now since it'll become obvious as it goes along.
I guess I have not heard of 'abootimg' as a tool for this, so I've been using the more manual way of dissecting the initrd as follows:
Code:
gunzip -dc ../blob.LNX-ramdisk.cpio.gz | cpio -i
If you need to change something , for example, in default.prop like ro.secure=0, then you'd do it there.
Then repack into a new ramdisk:
Code:
find . | cpio -o -H newc | gzip > ../newramdisk.cpio.gz
Finally I just had a somewhat heavily modified zImage from my build, so did this to make the blob (I'd copied zImage to blob.LNK-zImage.gz below):
Code:
./mkbootimg --kernel blob.LNX-zImage.gz --ramdisk newramdisk.cpio.gz -o boot.img
./blobpack blob.HEADER newblob LNX boot.img
zip -r imagename.zip blob MET* system // whatever the syntax was.
NOTE: I did this on a native 64 bit ubuntu LTS 10.04 box.
Unless I typo'd up there, that 'should' work. If it does boot, then first thing, take a look at settings, and kernel info so you can verify that you're running the kernel you desired (hopefully you renamed it in Makefile the first 4-5 lines).
Solved
Ok, got it working!!! Problem was bad crosscompiler :-( Modules I crosscompiled worked fine, so I ruled crosscompiler out :-/ Looks like I was too quick in judgement :-( Now I have kernel recompiled with original settings and evne the modified one and it still works and boot. Now I'm going to play with new features I got! Thanks a lot for all help!!!
Just for the record, crosscompiler I was originally using was codesourcery 2011.03 and to make it work I switched to official crosscompiler from NDK. Rest of the commands was Ok, I was just suspecting wrong step as I was quite familiar with kernel building and quite unfamiliar with the blob stuff :-(
Congrats!
For some reason I avoid the codesourcery stuff and stick with either the prebuilt toolchains or else just build my own from gnu source.
Anyway, glad you figured it out.
I have been following a few different instructions for the tools and was concentrated on just learning to rebuild a kernel on my own setup - Ubuntu 11.10. I only installed Ubuntu since it was the distro mostly referenced in the tutorials. I've also tried a couple different tool chains, some work, some don't.
I then find an existing *.zip CWM flashable kernel to work with, usually trying to use one I've successfully ran before, and unzip it. This gives 2 folders and a blob file. Whenever I run bootunpack on the blob I only get a resultant blob.LNX file and, so far never any blob.HEADER file. I understood that the blob.LNX was the same as boot.img from reading through and use dsixda's kitchen to split up the .LNX file I've renamed to boot.img. I then replace the zImage with the one I've just built and repack to boot.img in the kitchen. Then I move that boot.img back to unzipped kernel directory and rename to blob.LNX and run bootpack with blob as output and just ignore the .HEADER part. I then rezip the 2 folders (after replacing any modules in there) and blob into a new zip file and reflash in CWM. If it was based on a kernel I've booted before then it usually works without any problems. I can replace text in the updater-script, if I want and am just reusing the initramfs from the original zip. I have signature verification turned off in CWM, so that doesn't choke it. I need to read more about building initramfs before I do it. So far, this works for me, but I haven't really done any modification to the source, other than rebuilding it with my running config.
sidneyk said:
I have been following a few different instructions for the tools and was concentrated on just learning to rebuild a kernel on my own setup - Ubuntu 11.10. I only installed Ubuntu since it was the distro mostly referenced in the tutorials. I've also tried a couple different tool chains, some work, some don't.
I then find an existing *.zip CWM flashable kernel to work with, usually trying to use one I've successfully ran before, and unzip it. This gives 2 folders and a blob file. Whenever I run bootunpack on the blob I only get a resultant blob.LNX file and, so far never any blob.HEADER file. I understood that the blob.LNX was the same as boot.img from reading through and use dsixda's kitchen to split up the .LNX file I've renamed to boot.img. I then replace the zImage with the one I've just built and repack to boot.img in the kitchen. Then I move that boot.img back to unzipped kernel directory and rename to blob.LNX and run bootpack with blob as output and just ignore the .HEADER part. I then rezip the 2 folders (after replacing any modules in there) and blob into a new zip file and reflash in CWM. If it was based on a kernel I've booted before then it usually works without any problems. I can replace text in the updater-script, if I want and am just reusing the initramfs from the original zip. I have signature verification turned off in CWM, so that doesn't choke it. I need to read more about building initramfs before I do it. So far, this works for me, but I haven't really done any modification to the source, other than rebuilding it with my running config.
Click to expand...
Click to collapse
The architecture really seems to make a big difference in some configurations.
I have one native linux box with 64 bit 10.04 LTS on it, and it always behaves as well as possible, so I did this blob/boot/tools work on it, and it went as it should (creating HEADER and LNX) files, etc.
Then in addition I use several linux distros in VMs, one of them being more like yours, an 11.10 distro with just the androidSDK and all the build tools, prebuilt chains, etc. That will do exactly as you said. I actually built those blobtools/boottools from Koush's git, and they don't work correctly in that one environment. What is different to make that happen? I'm just guessing that something important like the native x86_64 gcc world is different enough to foul things up. It really doesn't matter. Once I got the tools working on the native box, I just transferred them to the other boxes including 11.10 and they work fine.
If you're using 64 bit and would like them I can probably stick them into a .tar.bz2 or whatever and stick up a link to them, or maybe if you can find working binaries to download, you might get those working. Once the blobunpack is returning only the .LNX file, you've pretty well had it as far as progress.
Good luck
hachamacha said:
The architecture really seems to make a big difference in some configurations.
I have one native linux box with 64 bit 10.04 LTS on it, and it always behaves as well as possible, so I did this blob/boot/tools work on it, and it went as it should (creating HEADER and LNX) files, etc.
Then in addition I use several linux distros in VMs, one of them being more like yours, an 11.10 distro with just the androidSDK and all the build tools, prebuilt chains, etc. That will do exactly as you said. I actually built those blobtools/boottools from Koush's git, and they don't work correctly in that one environment. What is different to make that happen? I'm just guessing that something important like the native x86_64 gcc world is different enough to foul things up. It really doesn't matter. Once I got the tools working on the native box, I just transferred them to the other boxes including 11.10 and they work fine.
If you're using 64 bit and would like them I can probably stick them into a .tar.bz2 or whatever and stick up a link to them, or maybe if you can find working binaries to download, you might get those working. Once the blobunpack is returning only the .LNX file, you've pretty well had it as far as progress.
Good luck
Click to expand...
Click to collapse
If by 'native' you mean a hard disk install as opposed to a VM install, then that's where I'm at. I have Ubuntu 11.10 x86_64 installed to a separate partition. I have the recommended stuff installed including the ia32 libs, but I never see a blob.HEADER file with either kernel.zips or ROM zips. I can unpack and repack kernels without the HEADER though and they boot just fine.
But, yes, if you don't mind posting a link with your files I'll give them a try sometime. Thanks.