Linksys WRT54GL / LEDE / Kernel 4.9.52
Foreword
I'm not an expert in CFE stuff and the booting process. This page just summarises
my current knowledge of these things.
My assumptions:
At reset, the CPU executes the CFE located at 0x0 in flash. The CFE eventually loads
the loader, which is part of the (LEDE) firmware and extracts the lzma compressed kernel.
The loader then hands over to the decompressed kernel and the system boots.
Observations
While other brcm47xx boards booted fine with newer kernels, the WRT54 often got stuck
after "Starting program at 0x80001000". This is the case for 4.9 but also 4.4 in current
trunk. Sometimes one could work around the issue by disabling the KALLSYMS configuration.
However, 4.9 refused to boot in every configuration.
The "loader" takes about 2.5 to 3 seconds for decompressing the kernel and then
hands over. For me, it was not clear, wheter the board was hanging in the loader stage or in the kernel.
By using JTAG, I further observed (by putting the target in HALT and checking the program counter (PC)):
- During decompression, the PC was at addresses 0x80400___.
- A kernel during the first steps of bootup was at 0x8000____ and 0x8010____.
- However, the kernels which refused to boot were executing at adresses 0xbfc00380, 0xb0008a3c
and 0xbfc029e4. (Which partly is in the mapped flash addresses?!?)
Other CFEs
During "debugging" I have tried CFEs from WRH-G54S and WL500gPv2 (with which I could successfully
boot kernel 4.9 on a WRT54GL!).
I noticed a small difference between these CFEs:
in the first few lines of an original WRT54GL CFE, there is
Total memory used by CFE: 0x80300000 - 0x803A39B0 (670128)
while for the other (working) CFEs, that line read:
Total memory used by CFE: 0x80400000 - 0x80499A30 (629296)
"Solution" or still Workaround?
So I cloned the repo at http://bues.ch/cgit/cfe.git and built my own CFE :-)
The code needs some minor changes to make it work with the current buildroot: cfe.patch.
With the original configuration in /cfe/build/broadcom/bcm947xx/Makefile:
CFG_TEXT_START := 0x80300000
I experienced the same boot problems as stated above.
Once I changed that section's address to
CFG_TEXT_START := 0x80400000
the kernel finally booted fine (see bootlog).
If you quickly want to try, grab a copy of that CFE here:
CFE.BIN, Last Modified: 2017-10-04 15:29:53, MD5: 1bf57914e36edcff56791b5bdeffab76, SHA256: b11045fb0e43183c562f77f9f4af05ec79359d82ff8c17ea9389eabd0bad38bb
However, you still have to integrate your own router's default nvram configuration
into that CFE image:
# get your default configuration (on the router):
dd if=/dev/mtd0ro of=nvram_defaults.bin bs=4096 skip=1 count=1
# split my CFE:
dd if=CFE.BIN of=cfe-pre.bin bs=4096 count=1
dd if=CFE.BIN of=cfe-post.bin bs=4096 skip=2
# then merge all parts:
cat cfe-pre.bin nvram_defaults.bin cfe-post.bin > cfe_new.bin
# and finally flash the cfe_new.bin
Full Bootlog
CFE version 1.0.37 for BCM947XX (32bit,SP,LE)
Build Date: Wed, 04 Oct 2017 12:44:08 +0200 (lede@debian-compiler)
Copyright (C) 2000,2001,2002,2003 Broadcom Corporation.
Initializing Arena
Initializing Devices.
et0: Broadcom BCM47xx 10/100 Mbps Ethernet Controller 4.150.10.29
CPU type 0x29008: 250MHz
Total memory: 32768 KBytes
Total memory used by CFE: 0x80400000 - 0x80499A30 (629296)
Initialized Data: 0x80430DC0 - 0x80433160 (9120)
BSS Area: 0x80433160 - 0x80433A30 (2256)
Local Heap: 0x80433A30 - 0x80497A30 (409600)
Stack Area: 0x80497A30 - 0x80499A30 (8192)
Text (code) segment: 0x80400000 - 0x8042D9E0 (186848)
Boot area (physical): 0x0049A000 - 0x004DA000
Relocation Factor: I:00000000 - D:00000000
Device eth0: hwaddr 58-6D-8F-__-__-__, ipaddr 192.168.1.1, mask 255.255.255.0
gateway not set, nameserver not set
Loader:raw Filesys:tftp Dev:eth0 File:: Options:(null)
Loading: Failed.
Could not load :: Timeout occured
Loader:raw Filesys:raw Dev:flash0.os File: Options:(null)
Loading: .. 3848 bytes read
Entry at 0x80001000
Closing network.
Starting program at 0x80001000
[ 0.000000] Linux version 4.9.52 (lede@debian-compiler) (gcc version 5.4.0 (LEDE GCC 5.4.0 r4683+286-370aacf532) ) #0 Mon Oct 2 16:26:53 2017
[ 0.000000] CPU0 revision is: 00029008 (Broadcom BMIPS3300)
[ 0.000000] bcm47xx: Using ssb bus
[ 0.000000] ssb: Found chip with id 0x5352, rev 0x00 and package 0x02
[ 0.000000] ssb: Sonics Silicon Backplane found at address 0x18000000
[ 0.000000] Determined physical RAM map:
[ 0.000000] memory: 02000000 @ 00000000 (usable)
[ 0.000000] Initrd not found or empty - disabling initrd
[ 0.000000] Primary instruction cache 16kB, VIPT, 2-way, linesize 16 bytes.
[ 0.000000] Primary data cache 8kB, 2-way, VIPT, no aliases, linesize 16 bytes
[ 0.000000] Zone ranges:
[ 0.000000] Normal [mem 0x0000000000000000-0x0000000001ffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000000000-0x0000000001ffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x0000000001ffffff]
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 8128
[ 0.000000] Kernel command line: noinitrd console=ttyS0,115200
[ 0.000000] PID hash table entries: 128 (order: -3, 512 bytes)
[ 0.000000] Dentry cache hash table entries: 4096 (order: 2, 16384 bytes)
[ 0.000000] Inode-cache hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.000000] Memory: 28364K/32768K available (3090K kernel code, 135K rwdata, 372K rodata, 168K init, 282K bss, 4404K reserved, 0K cma-reserved)
[ 0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] NR_IRQS:128
[ 0.000000] MIPS: machine is Linksys WRT54G/GS/GL
[ 0.000000] clocksource: MIPS: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 15290083572 ns
[ 0.000034] sched_clock: 32 bits at 125MHz, resolution 8ns, wraps every 17179869180ns
[ 0.000155] Calibrating delay loop... 249.44 BogoMIPS (lpj=1247232)
[ 0.070131] pid_max: default: 32768 minimum: 301
[ 0.070605] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.070688] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.081132] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[ 0.081238] futex hash table entries: 256 (order: -1, 3072 bytes)
[ 0.083589] NET: Registered protocol family 16
[ 0.085471] Can't analyze schedule() prologue at 80301a4c
[ 0.144606] clocksource: Switched to clocksource MIPS
[ 0.153847] NET: Registered protocol family 2
[ 0.156936] TCP established hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.157135] TCP bind hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.157290] TCP: Hash tables configured (established 1024 bind 1024)
[ 0.157992] UDP hash table entries: 256 (order: 0, 4096 bytes)
[ 0.158155] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
[ 0.159092] NET: Registered protocol family 1
[ 0.180617] Crashlog allocated RAM at address 0x1f00000
[ 0.187378] workingset: timestamp_bits=30 max_order=13 bucket_order=0
[ 0.216140] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 0.216212] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[ 0.258890] io scheduler noop registered
[ 0.258962] io scheduler deadline registered (default)
[ 0.259594] Serial: 8250/16550 driver, 2 ports, IRQ sharing enabled
[ 0.262087] console [ttyS0] disabled
[ 0.282455] serial8250.0: ttyS0 at MMIO 0xb8000300 (irq = 3, base_baud = 1250000) is a U6_16550A
[ 0.613694] console [ttyS0] enabled
[ 0.638799] serial8250.0: ttyS1 at MMIO 0xb8000400 (irq = 3, base_baud = 1250000) is a U6_16550A
[ 0.650434] physmap platform flash device: 02000001 at 1c000000
[ 0.656841] physmap-flash.0: Found 1 x16 devices at 0x0 in 16-bit bank. Manufacturer ID 0x0000c2 Chip ID 0x0022a8
[ 0.667740] Amd/Fujitsu Extended Query Table at 0x0040
[ 0.673012] Amd/Fujitsu Extended Query version 1.1.
[ 0.678312] number of CFI chips: 1
[ 0.697220] 3 bcm47xxpart partitions found on MTD device physmap-flash.0
[ 0.704081] Creating 3 MTD partitions on "physmap-flash.0":
[ 0.709943] 0x000000000000-0x000000040000 : "boot"
[ 0.725567] 0x000000040000-0x0000003f0000 : "firmware"
[ 0.737960] 3 trx partitions found on MTD device firmware
[ 0.743498] Creating 3 MTD partitions on "firmware":
[ 0.748771] 0x00000000001c-0x000000000920 : "loader"
[ 0.760825] 0x000000000920-0x00000012a800 : "linux"
[ 0.772636] 0x00000012a800-0x0000003b0000 : "rootfs"
[ 0.784872] mtd: device 4 (rootfs) set to be root filesystem
[ 0.790731] 1 squashfs-split partitions found on MTD device rootfs
[ 0.797211] 0x000000340000-0x0000003b0000 : "rootfs_data"
[ 0.809624] 0x0000003f0000-0x000000400000 : "nvram"
[ 0.829136] libphy: Fixed MDIO Bus: probed
[ 0.833797] b44: Broadcom 44xx/47xx 10/100 PCI ethernet driver version 2.0
[ 0.854100] libphy: b44_eth_mii: probed
[ 0.978484] b53_common: found switch: BCM5325, rev 0
[ 0.983654] Broadcom B53 (1) 1:1e: attached PHY driver [Broadcom B53 (1)] (mii_bus:phy_addr=1:1e, irq=-1)
[ 0.993562] b44 ssb0:0 eth0: Broadcom 44xx/47xx 10/100 PCI ethernet driver 58:6d:8f:__:__:__
[ 1.003771] bcm47xx-wdt bcm47xx-wdt.0: BCM47xx Watchdog Timer enabled (30 seconds, Software Timer)
[ 1.013378] GPIO_WDT: failed to register misc device
[ 1.021764] NET: Registered protocol family 10
[ 1.036877] NET: Registered protocol family 17
[ 1.041654] 8021q: 802.1Q VLAN Support v1.8
[ 1.059407] VFS: Mounted root (squashfs filesystem) readonly on device 31:4.
[ 1.069561] Freeing unused kernel memory: 168K
[ 1.074124] This architecture does not have kernel memory protection.
[ 2.080528] init: Console is alive
[ 2.084992] init: - watchdog -
[ 2.497395] random: fast init done
[ 3.489996] kmodloader: loading kernel modules from /etc/modules-boot.d/*
[ 3.594243] usbcore: registered new interface driver usbfs
[ 3.600517] usbcore: registered new interface driver hub
[ 3.606551] usbcore: registered new device driver usb
[ 3.630182] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 3.641480] ehci-platform: EHCI generic platform driver
[ 3.656433] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 3.666280] ohci-platform: OHCI generic platform driver
[ 3.671926] ohci-platform ohci-platform.0: Generic Platform OHCI controller
[ 3.679365] ohci-platform ohci-platform.0: new USB bus registered, assigned bus number 1
[ 3.688080] ohci-platform ohci-platform.0: irq 6, io mem 0x18003000
[ 3.762352] hub 1-0:1.0: USB hub found
[ 3.767713] hub 1-0:1.0: 2 ports detected
[ 3.786192] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[ 3.805018] init: - preinit -
[ 6.859950] b44 ssb0:0 eth0: Link is up at 100 Mbps, half duplex
[ 6.866267] b44 ssb0:0 eth0: Flow control is off for TX and off for RX
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level
[ 7.924761] b44 ssb0:0 eth0: Link is Up - 100Mbps/Full - flow control off
[ 10.506737] jffs2: notice: (339) jffs2_build_xattr_subsystem: complete building xattr subsystem, 0 of xdatum (0 unchecked, 0 orphan) and 0 of xref (0 dead, 0 orphan) found.
[ 10.526573] mount_root: switching to jffs2 overlay
[ 10.600619] urandom-seed: Seeding with /etc/urandom.seed
[ 10.757797] b44 ssb0:0 eth0: powering down PHY
[ 10.816416] procd: - early -
[ 10.819765] procd: - watchdog -
[ 11.044895] b44 ssb0:0 eth0: Link is Down
[ 11.604316] procd: - watchdog -
[ 11.608977] procd: - ubus -
[ 11.742976] procd: - init -
Please press Enter to activate this console.
[ 12.875790] kmodloader: loading kernel modules from /etc/modules.d/*
[ 12.945566] ip6_tables: (C) 2000-2006 Netfilter Core Team
[ 12.984491] Loading modules backported from Linux version v4.14-rc2-0-ge19b205be43d
[ 12.992479] Backport generated by backports.git v4.14-rc2-1-0-g8bfe0960
[ 13.008020] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 13.038577] nf_conntrack version 0.5.0 (1024 buckets, 4096 max)
[ 13.192193] xt_time: kernel timezone is -0000
[ 13.682209] PPP generic driver version 2.4.2
[ 13.694784] NET: Registered protocol family 24
[ 13.842022] b43-phy0: Broadcom 5352 WLAN found (core revision 9)
[ 13.878831] b43-phy0: Found PHY: Analog 2, Type 2 (G), Revision 7
[ 13.885229] b43-phy0: Found Radio: Manuf 0x17F, ID 0x2050, Revision 2, Version 0
[ 13.919348] Broadcom 43xx driver loaded [ Features: PNL ]
[ 13.988972] kmodloader: done loading kernel modules from /etc/modules.d/*
[ 29.909876] b44 ssb0:0 eth0: Link is up at 100 Mbps, full duplex
[ 29.916204] b44 ssb0:0 eth0: Flow control is off for TX and off for RX
[ 29.982461] br-lan: port 1(eth0.1) entered blocking state
[ 29.988195] br-lan: port 1(eth0.1) entered disabled state
[ 29.997395] device eth0.1 entered promiscuous mode
[ 30.002324] device eth0 entered promiscuous mode
[ 30.080769] br-lan: port 1(eth0.1) entered blocking state
[ 30.086470] br-lan: port 1(eth0.1) entered forwarding state
[ 30.965323] b44 ssb0:0 eth0: Link is Up - 100Mbps/Full - flow control off
BusyBox v1.27.2 () built-in shell (ash)
_________
/ /\ _ ___ ___ ___
/ LE / \ | | | __| \| __|
/ DE / \ | |__| _|| |) | _|
/________/ LE \ |____|___|___/|___| lede-project.org
\ \ DE /
\ LE \ / -----------------------------------------------------------
\ DE \ / Reboot (SNAPSHOT, r4683+286-370aacf532)
\________\/ -----------------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@LEDE:/# uname -a
Linux LEDE 4.9.52 #0 Mon Oct 2 16:26:53 2017 mips GNU/Linux
root@LEDE:/# cat /proc/cpuinfo
system type : Broadcom BCM5352
machine : Linksys WRT54G/GS/GL
processor : 0
cpu model : Broadcom BMIPS3300 V0.8
BogoMIPS : 249.44
wait instruction : yes
microsecond timers : yes
tlb_entries : 32
extra interrupt vector : yes
hardware watchpoint : no
isa : mips1 mips2 mips32r1
ASEs implemented :
shadow register sets : 1
kscratch registers : 0
package : 0
core : 0
VCED exceptions : not available
VCEI exceptions : not available
If there are any new findings, problems or questions contact me at: p [dot] wassi [at] gmx [dot] at
Last modified: 2017-10-04 15:57:37