T4 - Flash mem failed again

Moderators: Gully, peteru

Post Reply
stevebow
Master
Posts: 482
Joined: Thu Sep 03, 2015 11:21
Location: Sydney

T4 - Flash mem failed again

Post by stevebow » Sat May 18, 2019 09:48

My T4 wouldn't boot from deep standby yesterday. It would get as far as the round blue Beyonwiz logo and hang there until it would reboot itself a few minutes later, in a loop. Looking at the serial log, it appeared to be a problem with flash mem (bitflips), with ubi0 not mounted and a rather prominent "UBIFS: recovery needed" message in the log.

I reflashed the firmware via USB and all is good again. I think this is the third or fourth time this has happened since I've had the T4 (Oct. 2015), so it looks like a once-a-year thing for me anyway. :roll:

Below are the serial logs for anyone interested, the first is a regular boot, the second with the flash mem corrupted.

Has anyone else been seeing flash mem issues like this other than me?

Code: Select all

UBI: scanning is finished
UBI: attached mtd0 (name "rootfs", size 2041 MiB) to ubi0
UBI: PEB size: 1048576 bytes (1024 KiB), LEB size: 1040384 bytes
UBI: min./max. I/O unit sizes: 4096/4096, sub-page size 4096
UBI: VID header offset: 4096 (aligned 4096), data offset: 8192
UBI: good PEBs: 2039, bad PEBs: 2, corrupted PEBs: 0
UBI: user volume: 1, internal volumes: 1, max. volumes count: 128
UBI: max/mean erase counter: 2/0, WL threshold: 4096, image sequence number: 1134490118
UBI: available PEBs: 0, total reserved PEBs: 2039, PEBs reserved for bad PEB handling: 78
ALSA device list:
  No soundcards found.
UBI: background thread "ubi_bgt0d" started, PID 68
UBIFS: background thread "ubifs_bgt0_0" started, PID 69

*** FAILED BOOT LOG WILL DEVIATE FROM HERE ***

UBIFS: mounted UBI device 0, volume 0, name "rootfs"
UBIFS: LEB size: 1040384 bytes (1016 KiB), min./max. I/O unit sizes: 4096 bytes/4096 bytes
UBIFS: FS size: 2025627648 bytes (1931 MiB, 1947 LEBs), journal size 12550144 bytes (11 MiB, 13 LEBs)
UBIFS: reserved for root: 0 bytes (0 KiB)
UBIFS: media format: w4/r0 (latest is w4/r0), UUID 05573954-D033-4F2D-A3CF-C5FBAFC1672F, small LPT model
VFS: Mounted root (ubifs filesystem) on device 0:11.
devtmpfs: mounted
Freeing unused kernel memory: 260K (8092f000 - 80970000)

INIT: version 2.88 booting

[etc...]

Code: Select all

UBI: scanning is finished
UBI: attached mtd0 (name "rootfs", size 2041 MiB) to ubi0
UBI: PEB size: 1048576 bytes (1024 KiB), LEB size: 1040384 bytes
UBI: min./max. I/O unit sizes: 4096/4096, sub-page size 4096
UBI: VID header offset: 4096 (aligned 4096), data offset: 8192
UBI: good PEBs: 2039, bad PEBs: 2, corrupted PEBs: 0
UBI: user volume: 1, internal volumes: 1, max. volumes count: 128
UBI: max/mean erase counter: 3/1, WL threshold: 4096, image sequence number: 1134490118
UBI: available PEBs: 0, total reserved PEBs: 2039, PEBs reserved for bad PEB handling: 78
UBI: background thread "ubi_bgt0d" started, PID 69
ALSA device list:
  No soundcards found.
UBIFS: background thread "ubifs_bgt0_0" started, PID 70

*** LOG DEVIATES HERE ***

UBIFS: recovery needed
brcmnand brcmnand.0: bitflips oob(0) data(1) at 0x2336fc00
brcmnand brcmnand.0: bitflips in apparent erased page at 0x2336f000
UBIFS error (pid 1): check_lpt_type: invalid type (15) in LPT node type 1
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.14.2 #2
Stack : 00000000 00000000 00000000 00000000 00000000 00000000 80976a52 00000033
  00000000 00000000 00000001 00010000 cfc22d08 808e0ce7 8082613c 00000000
  00000001 809738d8 00000000 00000008 0001808d 80765abc 10008703 8002dae0
  219c4000 00000000 8082c2f8 cfc39adc cfc39adc cfc22d08 00000000 00000000
  00000000 00000000 00000000 00000000 00000000 00000000 00000000 cfc39a58
  ...
Call Trace:
[<8000c94c>] show_stack+0x64/0x7c
[<80766748>] dump_stack+0xc0/0xf0
[<802c4e9c>] check_lpt_type.constprop.9+0x5c/0x64
[<802c66dc>] ubifs_unpack_nnode+0x40/0xe8
[<802c6b40>] ubifs_read_nnode+0x1e4/0x24c
[<802c6f8c>] ubifs_lpt_lookup_dirty+0x204/0x3c8
[<802b9984>] ubifs_replay_journal+0x1048/0x1544
[<802acdd4>] ubifs_mount+0x1084/0x17a0
[<800fd76c>] mount_fs+0x20/0xe4
[<8011b46c>] vfs_kern_mount+0x68/0x13c
[<8011e090>] do_mount+0x1e4/0xac4
[<8011edbc>] SyS_mount+0xb0/0x12c
[<8093001c>] mount_block_root+0x14c/0x290
[<809303a0>] prepare_namespace+0xe0/0x1fc
[<8092fc58>] kernel_init_freeable+0x270/0x2a0
[<807637cc>] kernel_init+0x10/0x100
[<80006ce8>] ret_from_kernel_thread+0x14/0x1c

UBIFS error (pid 1): ubifs_read_nnode: error -22 reading nnode at 8:98445
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.14.2 #2
Stack : 00000000 00000000 00000000 00000000 00000000 00000000 80976a52 00000033
  00000000 00000000 00000001 00010000 cfc22d08 808e0ce7 8082613c 00000000
  00000001 809738d8 ffffffea 00000008 0001808d 80765abc 10008703 8002dae0
  219c4000 00000000 8082c2f8 cfc39b2c cfc39b2c cfc22d08 00000000 00000000
  00000000 00000000 00000000 00000000 00000000 00000000 00000000 cfc39aa8
  ...
Call Trace:
[<8000c94c>] show_stack+0x64/0x7c
[<80766748>] dump_stack+0xc0/0xf0
[<802c6b78>] ubifs_read_nnode+0x21c/0x24c
[<802c6f8c>] ubifs_lpt_lookup_dirty+0x204/0x3c8
[<802b9984>] ubifs_replay_journal+0x1048/0x1544
[<802acdd4>] ubifs_mount+0x1084/0x17a0
[<800fd76c>] mount_fs+0x20/0xe4
[<8011b46c>] vfs_kern_mount+0x68/0x13c
[<8011e090>] do_mount+0x1e4/0xac4
[<8011edbc>] SyS_mount+0xb0/0x12c
[<8093001c>] mount_block_root+0x14c/0x290
[<809303a0>] prepare_namespace+0xe0/0x1fc
[<8092fc58>] kernel_init_freeable+0x270/0x2a0
[<807637cc>] kernel_init+0x10/0x100
[<80006ce8>] ret_from_kernel_thread+0x14/0x1c

UBIFS: background thread "ubifs_bgt0_0" stops
List of all partitions:
1f00         2089984 mtdblock0  (driver?)
1f02            7168 mtdblock2  (driver?)
1f03             640 mtdblock3  (driver?)
1f04            1920 mtdblock4  (driver?)
1f05             512 mtdblock5  (driver?)
1f06             512 mtdblock6  (driver?)
1f07             256 mtdblock7  (driver?)
1f08             256 mtdblock8  (driver?)
1f09               8 mtdblock9  (driver?)
0800       976762584 sda  driver: sd
  0801       976760832 sda1 1b7805a8-7dbb-46c9-96e3-453b619bb54d
No filesystem could mount root, tried:  ubifs
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,1)
Rebooting in 180 seconds..

prl
Wizard God
Posts: 32697
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Re: T4 - Flash mem failed again

Post by prl » Sat May 18, 2019 11:41

My first suspects would be the power supply and the on-board regulators that generate the other supply voltages.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

stevebow
Master
Posts: 482
Joined: Thu Sep 03, 2015 11:21
Location: Sydney

Re: T4 - Flash mem failed again

Post by stevebow » Fri Jun 14, 2019 15:01

Sorry for the late reply. I wanted to scope the flashmem's supply pins for noise and my scope has been out on loan.

My original thoughts on this problem were random bit-flipping of the flashmem, but you have a valid point, it's worth checking the supply.

The flashmem is U15 on the board, a Micron MT29F32G08CBACA, datasheet here:

https://d4u-datasheet.netdna-ssl.com/pd ... Micron.pdf

The pins themselves are quite microscopic (to my not-so-young eyes anyway!), but fortunately there are a couple of bypass caps that were much easier to pick off measurements from, right next to both Vcc pins: C147 and C170.

Voltage was 3.34VDC, right where you'd expect it, and hash/noise was 17mV P-P. At 975 Hz, interestingly enough. This is very minimal and I would not expect hash at this level to impact operation of the flashmem, let alone bit-flipping.

You could argue the relevant regulator perhaps "chucks a wobbly" every now and then, but I think my money would be on random bit-flipping of the device. If it happens again I'll just reflash it.

Post Reply

Return to “Hardware Discussion”