Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bitcoin Core 0.15.0 crashes repeatedly while rebuilding local blockchain copy, no error reported #11356

Closed
drkskwlkr opened this issue Sep 17, 2017 · 10 comments
Closed

Bitcoin Core 0.15.0 crashes repeatedly while rebuilding local blockchain copy, no error reported #11356

drkskwlkr opened this issue Sep 17, 2017 · 10 comments
Labels
GUI

Comments

@drkskwlkr
Copy link

What follows is a report of unexplained crashes of the new Bitcoin Core 0.15. The available information may not be enough to identify the cause but perhaps other users can recognize identical behavior and use this ticket to acknowledge.

In case the kind dev team wants to investigate the issue further, I will cooperate fully. Thanks for your time to whoever reads this.

Describe the issue

I run a Bitcoin Core node 0.14.2 node under Ubuntu 17.04. I decided to upgrade to the new 0.15 version as soon as it became possible to do so via apt upgrade.

It is important to note that the node was fully synced and running fine before the update. It is an old piece of hardware with a brand new HDD added, and a fresh copy of the Ubuntu OS installed & configured just for the purpose of running Bitcoin Core.

The first time I launched bitcoin-qt 0.15, it said a DB upgrade is necessary. Shortly afterwards it reported that the database was corrupt and the local copy of the blockchain needed to be rebuilt (I quote from memory, sadly I did not write down the exact message).

The program then proceeded to process the blocks on disk. Since I am running the node on underpowered hardware (full specs below) this is a slow process: 60+ hours and counting; approx. 75% done.

During this update/resync process, the bitcoin software has crashed at least 4 times so far without any apparent error message.

debug.log contents (attached below) look normal: long sets of UpdateTip: new best= messages for all processed blocks interspersed with service messages like Pre-allocating up to position xxxx in xxxx.dat and handshakes with other nodes.

Can you reliably reproduce the issue?

Sadly, no. Crashes may happen after more than 24 hours of work or as little as 12 hours after restarting the process.

The NUC is dedicated to this task only; it operates on its own and I only keep a remote shell open that runs a tail -f debug.log that I look at a couple of times a day.

bitcoin.conf contents:

onlynet=ipv4
server=1
rpcbind=127.0.0.1
rcpbind=10.9.9.46 // This is my inhouse VPN
rpcport=8332
rpcuser=[redacted]
rpcpassword=[redacted]

Besides the above configuration, I have increased the size of the database cache from the default value to 1024MB, using the bitcoin-qt settings panel. Memory usage is stable at 2GB out of 3.7GB usable, and looking for out of memory crashes using grep "Killed process" /var/log/syslog yields no results.

What version of bitcoin-core are you using?

I am using the package available from the ppa:bitcoin/bitcoin repository
Version string is v0.15.0.0-g3751912e8e

Machine specs:

  • OS: Ubuntu 17.04 (4.10.0-33-generic Ubuntu SMP Fri Aug 11 10:55:28 UTC 2017 x86_64 GNU/Linux)
  • CPU: Intel Celeron N2820
  • RAM: 4GB (3.7GB available)
  • SWAP: 8GB
  • Disk size: 1.0TB (850 GB available)
  • Disk Type: HDD

Any extra information that might be useful in the debugging process.

The contents of debug.log are provided as follows:

  • The startup part following a crash is attached as plaintext file.
    debug.log.txt
  • The full log (10.8MB uncompressed) is attached as zip file.
    debug.log.zip

db.log is empty. I know of no other logs related to this software, but will post any info upon request.
@achow101
Copy link
Member

Is it crashing immediately after the splashscreen goes away or later after the main window has appeared?

Can you please provide the contents of this file: ~/.config/Bitcoin/Bitcoin-Qt.conf?

This may be related to #11171

@maflcko
Copy link
Member

maflcko commented Sep 17, 2017

@achow101 I think they are not using the gui

@achow101
Copy link
Member

@achow101 I think they are not using the gui

I think they are. I see lines in their debug.log for the GUI, e.g. GUI: Platform customization: "other"

@maflcko maflcko added the GUI label Sep 17, 2017
@drkskwlkr
Copy link
Author

Is it crashing immediately after the splashscreen goes away or later after the main window has appeared?

@achow101 The software crashes much, much later than startup (many hours after restart). Behavior is not matching what is described in #11171

Contents of ~/.config/Bitcoin/Bitcoin-Qt.conf as requested:

[General]
strDataDir=/home/bitcoin/.bitcoin
fReset=false
nSettingsVersion=150000
fRestartRequired=false
fHideTrayIcon=false
fMinimizeToTray=true
fMinimizeOnClose=true
nDisplayUnit=0
strThirdPartyTxUrls=
fCoinControlFeatures=true
nDatabaseCache=1024
nThreadsScriptVerif=0
bSpendZeroConfChange=true
fUseUPnP=false
fListen=true
fUseProxy=false
addrProxy=127.0.0.1:9050
fUseSeparateProxyTor=false
addrSeparateProxyTor=127.0.0.1:9050
language=en
fFeeSectionMinimized=true
nFeeRadio=0
nCustomFeeRadio=0
nTransactionFee=0
fPayOnlyMinFee=false
nWindowPos=@Point(42 26)
nWindowSize=@Size(850 550)
nRPCConsoleWindowPos=@Point(128 376)
nRPCConsoleWindowSize=@Size(740 430)
nConfTarget=6
nSmartFeeSliderPosition=0

@achow101
Copy link
Member

@drkskwlkr If you run Bitcoin Core from the terminal, what is the message that it gets when it crashes? Can you also have it create a core dump file and debug that?

@drkskwlkr
Copy link
Author

If you run Bitcoin Core from the terminal, what is the message that it gets when it crashes?

I have never run Bitcoin Core as daemon so far, I just did that for the first time. It did not crash at startup (and I wouldn't expect it to). It just went on its business verifying blocks. I will leave it running like that and see if it would crash again at some point like the GUI version did.

As for creating core dump files, I haven't done that before (I know Linux just enough to run my webserver). But I checked /var/crash just now and there is a 100k crash file associated with bitcoin-qt from the last crash (see attached). Does it provide any meaningful information?

_usr_bin_bitcoin-qt.1000.crash.txt

@TheBlueMatt
Copy link
Contributor

Sadly the crash file provided isn't very useful (just indicates SEGFAULT). If possible, it would be of great help for you to install gdb (via apt), as well as the debug symbols from the launchpad build at https://launchpad.net/~bitcoin/+archive/ubuntu/bitcoin/+files/bitcoin-qt-dbgsym_0.15.0-zesty9_amd64.ddeb (installed as you would any other .deb package).

Then in termainal run gdb /usr/bin/bitcoin-qt, when it finishes loading, type "run" and hit enter. Eventually, when it crashes, type "thread apply all bt", hit enter, and paste the output of what it displays there. This should tell us exactly where the issue was.

@drkskwlkr
Copy link
Author

@TheBlueMatt: Hi, thank you for the detailed directions and sorry for not responding earlier.

Before I was able to do as you suggest, my node managed to fully resync the blockchain and is churning along nicely. However it did manage to crash one more time after reaching 100%.

I am now running bitcoin-qt via gdb as instructed and will update the thread if/when another crash occurs.

If nothing happens for a couple of days, and if I am able to figure out how to force my node to rebuild the blocks from the last 6-12 months, I will do that in order to provoke the same conditions under which the initial crashes were observed.

@gmaxwell
Copy link
Contributor

@drkskwlkr Any followup?

@drkskwlkr
Copy link
Author

@gmaxwell Hi Gregory - sorry, no. The node has been working reliably 24/7 since my report. As nobody else seems to have this issue, I think it'd be best to close the ticket.

@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
GUI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@TheBlueMatt @gmaxwell @achow101 @maflcko @drkskwlkr