[bug]: Channel Transaction has over 800 confirmations but still pending and LND node failure #7803
Description
Background
I opened a channel a few days ago, it got confirmed the same day, but the channel hasn't showed up. It's still pending.
Upon reviewing my logs in an effort to diagnose this problem, I've noticed that my lnd node is consistently failing and restarting on its own via systemd. Each failure incident is invariably preceded by the same log entry, implying a potential connection between this recurring log pattern and the node's instability.
This continuous failure and auto-restart sequence is quite troubling and seems to be indicative of a deeper issue within the lnd.
I've attached the relevant log excerpt below for reference. I would appreciate any assistance in resolving these persistent issues.
Your environment
*version of lnd : v0.16.3
*operating system: ubuntu-20.04-x86_64 on a server
*backend: Bitcoin Core version v24.0.1 (pruned)
Steps to reproduce
Initiate a channel opening transaction as usual through Thunderhub.
Expected behaviour
The transaction is confirmed, the channel appears, and the LND continues to run as expected
Actual behaviour
The transaction gets confirmed, but the channel remains in a "pending" state.
The LND node intermittently stops and restarts.
Relevant Log
Jul 03 10:38:39 lnd[589808]: 2023-07-03 10:38:39.748 [INF] DISC: Creating new GossipSyncer for peer=027e05064da4abbe0b6655819867aeca327e2865a22594383ab3a204a3475a843f
Jul 03 10:38:39 lnd[589808]: 2023-07-03 10:38:39.749 [INF] HSWC: ChannelLink(8966cf22ad3abc4a7eb362f6d32040200ef0ed9a25198b5817cf3554c2094257:1): received re-establishment message from remote side
Jul 03 10:38:48 lnd[589808]: 2023-07-03 10:38:48.626 [INF] WTCL: (anchor) Client stats: tasks(received=0 accepted=0 ineligible=0) sessions(acquired=0 exhausted=0)
Jul 03 10:38:48 lnd[589808]: 2023-07-03 10:38:48.627 [INF] WTCL: (legacy) Client stats: tasks(received=0 accepted=0 ineligible=0) sessions(acquired=0 exhausted=0)
Jul 03 10:38:56 lnd[589808]: 2023-07-03 10:38:56.633 [INF] NTFN: Historical spend dispatch finished for request outpoint=6310df9b5ea48533c67965db43f957357e9eba230beb917cce1bcb5761b24eb2:0, script=0 0ffc7719879896d3d20a81f3fe25b21b7669b275442dd209fe379f74665d7707 (start=796029 end=796988) with details: 43d7f9ec080970d9ae3d4470b80866c523550c95ebb75bf81c24f5c4dba6d5f1[0] spending 6310df9b5ea48533c67965db43f957357e9eba230beb917cce1bcb5761b24eb2:0 at height=796029
Jul 03 10:38:56 lnd[589808]: 2023-07-03 10:38:56.652 [INF] NTFN: Dispatching confirmed spend notification for outpoint=6310df9b5ea48533c67965db43f957357e9eba230beb917cce1bcb5761b24eb2:0, script=0 0ffc7719879896d3d20a81f3fe25b21b7669b275442dd209fe379f74665d7707 at current height=796991: 43d7f9ec080970d9ae3d4470b80866c523550c95ebb75bf81c24f5c4dba6d5f1[0] spending 6310df9b5ea48533c67965db43f957357e9eba230beb917cce1bcb5761b24eb2:0 at height=796029
Jul 03 10:38:56 lnd[589808]: panic: runtime error: index out of range [1] with length 0
Jul 03 10:38:56 lnd[589808]: goroutine 1599 [running]:
Jul 03 10:38:56 lnd[589808]: github.com/lightningnetwork/lnd/contractcourt.isPreimageSpend(...)
Jul 03 10:38:56 lnd[589808]: github.com/lightningnetwork/lnd/contractcourt/htlc_timeout_resolver.go:255
Jul 03 10:38:56 lnd[589808]: github.com/lightningnetwork/lnd/contractcourt.(*htlcTimeoutResolver).Resolve(0xc001b4b400)
Jul 03 10:38:56 lnd[589808]: github.com/lightningnetwork/lnd/contractcourt/htlc_timeout_resolver.go:281 +0x2a9
Jul 03 10:38:56 lnd[589808]: github.com/lightningnetwork/lnd/contractcourt.(*ChannelArbitrator).resolveContract(0xc001c28840, {0x2258ac0?, 0xc001b4b400})
Jul 03 10:38:56 lnd[589808]: github.com/lightningnetwork/lnd/contractcourt/channel_arbitrator.go:2371 +0x2a9
Jul 03 10:38:56 lnd[589808]: created by github.com/lightningnetwork/lnd/contractcourt.(*ChannelArbitrator).launchResolvers
Jul 03 10:38:56 lnd[589808]: github.com/lightningnetwork/lnd/contractcourt/channel_arbitrator.go:1376 +0xd2
Jul 03 10:38:56 systemd[1]: lnd.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jul 03 10:38:56 systemd[1]: lnd.service: Failed with result 'exit-code'.
Jul 03 10:39:56 systemd[1]: lnd.service: Scheduled restart job, restart counter is at 7.
Jul 03 10:39:56 systemd[1]: Stopped Lightning Network Daemon.