Monday 25 June 2018

101 confirmations

Questions:
After mining, i do listtransactions command, i can see the address has amount of 50 coins in each transaction. Why getbalance command returns 0 ?

<pre>
$ src/bitcoin-cli -datadir=../datadir listtransactions
[
  {
    "account": "",
    "address": "15454L2G44NuZoZrE2HdBwMmCchQVcGvKm",
    "category": "immature",
    "amount": 50.00000000,
    "label": "",
    "vout": 0,
    "confirmations": 2,
    "generated": true,
    "blockhash": "00000000df3ae70ae6c5f3eb24b0dca0c37ef55b76fad5396f1386aaab2b0027",
    "blockindex": 0,
    "blocktime": 1527831787,
    "txid": "24db59cbb12a8ffd3f1421931f2a6a2293b1b6437021af88119da95937c8f737",
    "walletconflicts": [
    ],
    "time": 1527831787,
    "timereceived": 1527831828,
    "bip125-replaceable": "no"
  }, 
  {
    "account": "",
    "address": "15454L2G44NuZoZrE2HdBwMmCchQVcGvKm",
    "category": "immature",
    "amount": 50.00000000,
    "label": "",
    "vout": 0,
    "confirmations": 1,
    "generated": true,
    "blockhash": "00000000eb85f984adc2905671aaa8663d505c0ee71fb5a0d47996f76a12f336",
    "blockindex": 0,
    "blocktime": 1527831949,
    "txid": "d47efd3725fb3be3072131ea6612b2e6581a876f11760e844e91ecd8b414f22e",
    "walletconflicts": [
    ],
    "time": 1527831949,
    "timereceived": 1527831973,
    "bip125-replaceable": "no"
  }
]
$ src/bitcoin-cli -datadir=../datadir getbalance ""
0.00000000
</pre>

Generated coins cannot be spent until the generation transaction has gone through 101 confirmations. Transactions that try to spend generated coins before this will be rejected. The 101 confirmations is the maturity time.

The reason for this is that sometimes the block chain forks, blocks that were valid become invalid, and the mining reward in those blocks is lost. That is an unavoidable part of how Bitcoin works. If there was no maturation time, then whenever a fork happened, everyone who received coins that were generated on an unlucky fork (possibly through many intermediaries) would have their coins disappear, even without any sort of double-spend or other attack. On long forks, people could find coins disappearing from their wallets, even though there is no one actually attacking them and they had no reason to be suspicious of the money they were receiving. 

For example, without a maturation time, a miner might deposit 50 BTC into an EWallet, and if the user withdraws money from a completely unrelated account on the same EWallet, the withdrawn money might just disappear if there is a fork and he/she is unlucky enough to withdraw coins that have been "tainted" by the miner's now-invalid coins. 

Due to the way this sort of taint tends to "infect" transactions, far more than 50 BTC per block would be affected. Each invalidated block could cause transactions collectively worth hundreds of bitcoins to be reversed. The maturation time makes it impossible for anyone to lose coins by accident like this as long as a fork doesn't last longer than 100 blocks. 

If a fork does last longer than 100 blocks, then the damage caused by invalidated transactions would likely be a huge disaster.  This is unlike to happen, as something would have to be seriously wrong with Bitcoin or the Internet for a fork to last this long.



Friday 22 June 2018

Run Bitcoind in Docker container


Use Docker container to run Bitcoin core

0) Install Docker
 Docker can be installed on Windows, Linux, MacOS

1) Create new directory for running bitcoin core in Docker
 mkdir bitcoin-core-docker
 cd bitcoin-core-docker

2) Create new dir  to store bitcoin.conf
 mkdir node; cd node

3) Create bitcoin.conf and add the contents
server=1
regtest=1
port=12000
rpcport=12001
rpcallowip=0.0.0.0/0
rpcuser=username
rpcpassword=password
daemon=1
txindex=1

4) Build Docker container 
  cd .. ( to bitcoin-core-docker)
  create Dockerfile

5) Add contents to Docker
# Dockerfile must start with a FROM instruction
# FROM instruction specifies the Base Image from which you are building
# FROM <image>[:<tag>]
FROM ubuntu:16.04

ENV BTCVERSION=0.15.1

ENV BTCPREFIX=/bitcoin-prefix

RUN apt-get update && apt-get install -y git build-essential wget pkg-config curl libtool autotools-dev automake libssl-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-program-options-dev libboost-test-dev libboost-thread-dev

WORKDIR /

RUN mkdir -p /berkeleydb

#download berkeley db
RUN git clone -b 0.15 --single-branch  https://github.com/bitcoin/bitcoin.git

WORKDIR /berkeleydb

RUN wget http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz && tar -xvf db-4.8.30.NC.tar.gz && rm db-4.8.30.NC.tar.gz && mkdir -p db-4.8.30.NC/build_unix/build

ENV BDB_PREFIX=/berkeleydb/db-4.8.30.NC/build_unix/build
WORKDIR /berkeleydb/db-4.8.30.NC/build_unix

RUN ../dist/configure --disable-shared --enable-cxx --with-pic --prefix=$BDB_PREFIX

RUN make install

RUN apt-get update && apt-get install -y libminiupnpc-dev libzmq3-dev  libprotobuf-dev protobuf-compiler libqrencode-dev

WORKDIR /bitcoin

RUN git checkout v${BTCVERSION} && mkdir -p /bitcoin/bitcoin-${BTCVERSION}

WORKDIR /bitcoin

RUN ./autogen.sh

RUN ./configure CPPFLAGS="-I${BDB_PREFIX}/include/ -O2" LDFLAGS="-L${BDB_PREFIX}/lib/ -static-libstdc++" --prefix=${BTCPREFIX}

RUN make

RUN make install DESTDIR=/bitcoin/bitcoin-${BTCVERSION}

RUN mv /bitcoin/bitcoin-${BTCVERSION}${BTCPREFIX} /bitcoin-${BTCVERSION} && strip /bitcoin-${BTCVERSION}/bin/* && rm -rf /bitcoin-${BTCVERSION}/lib/pkgconfig && find /bitcoin-${BTCVERSION} -name "lib*.la" -delete && find /bitcoin-${BTCVERSION} -name "lib*.a" -delete

WORKDIR /

RUN tar cvf bitcoin-${BTCVERSION}.tar bitcoin-${BTCVERSION}

# copy bitcoin,conf
ADD . /bitcoin-${BTCVERSION}

# expose rpc port for the node to allow access from outside container
EXPOSE 12001

WORKDIR /bitcoin-${BTCVERSION}

6)build the docker image
   docker build -t bitcoin-docker .
  • The -t flag sets a name for the image.
  • The . tells Docker to looker for Dockerfile in the current directory

 6.1)list the built images:
  docker images

7) run the image in container
docker run -it -p 5000:12001 bitcoin-docker
  • -it is required for interactive processes (like a bash shell)
  • -p maps Ubuntu port 5000 to the container’s exposed port 12001, which is where Bitcoin rpc will be listening 

if everything works, docker present a bash shell. Both the node directory and the bitcoin.conf file were copied to the container by the ADD instruction in the Dockerfile, so they will be present in the current working directory. 

In the bash shell, run
7.1) bitcoind -datadir=node -daemon
7.2) bitcoin-cli -datadir=node getinfo

8) Connect to bitcoind from outside Docker (open second terminal window)
curl --user username:password --data '{"method": "getinfo"}' http://127.0.0.1:5000

9) In first terminal window, run this to stop bitcoind
 bitcoin-cli -datadir=node stop
9.1) In second terminal window, exit the docker container
 exit

Use Docker to run Bitcoin core as service
TBC…

PS: we can push the image to docker hub and share with others

# tag the image

docker tag  bitcoin <username>/bitcoin:custom
# push to docker hub

docker push <username>/bitcoin:custom
# run the image, if image is not available locally, docker pull from repo

docker run -it -p 5000:12001 <username>/bitcoin:custom

Wednesday 6 June 2018

Segregated Witness (Segwit)

Segwit
Introduction
Segregated Witness (Segwit) [1], proposed in BIP 141 [5], was activated on August 24, 2017. The contributions of Segwit [2]:
1) solve transaction malleability [3]
2) mitigate block size limitation problem
Problem
  1. Transaction malleability:
When transaction is signed, the signature (script_sig) does not cover all the data in a transaction. Specifically, the script_sig is part of the transaction, the signature will not be able to sign script_sig. So the signature does not cover script_sig. The script_sig is added, after the transaction is created and signed.


The script_sig is the tempering point. If script_sig changes, TXID will change. The script_sig can be changed by anyone has access to the corresponding private keys.
2) Block size limitation problem
Originally, Bitcoin does not have limit on block size. This allowed attackers to create large size block data. So a 1MB block size was introduced. The 1MB was a tradeoff, between network propagation times, node capability, and number of transactions that can fit into one block, etc [4].
Proposal
Segwit defines a new structure called witness. Signature and redeem script are moved into this structure, which is not included in the 1MB block size limit.
1)Transaction structure

The conventional transaction structure is used in TXID calculation, and script_sig is empty. Even if script_sig is tempered with, TXID does not change.
2) Lock/Unlock script
For a conventional P2PKH:
scriptPubKey (lock script)
OP_DUP OP_HASH160 <pubkey hash> OP_EQUALVERIFY OP_CHECKSIG
scriptSig (unlock script)
<sig> <pubkey>
For Segwit P2WPKH:
scriptPubKey (lock script)
0 <pubkey hash>
(unlock script)
scriptSig
Witness
Empty
<sig> <pubkey>
In scriptPubKey, there are no opcodes, only 2 data (version and hash) is pushed. When the lock script of this pattern is set, it is evaluated as a conventional P2PKH script.  The signature and public key are obtained from witness instead of scriptSig.
3) Witness extension method
In the extension, Segwit introduces OP_CLTV (OP_NPO2) and OP_CSV (OP_NOP3)
The witness structure
<witness version>
<witness program>
For Segwit, witness version is 0, the witness program is P2WPKH if hash length is 20 bytes and P2WSH if it is 32 bytes.
4) Address format
Segwit uses Bech32 address format. It is based on BCH code instead of previously used Base58 encoding, so that error correction is possible [6]. There is no distinction between uppercase and lowercase letters. QR code is also compact

5) Increase of block size
The increase of block size from Segwit depends on the types of transaction.
  • Before Segwit
block data ≦ 1,000,000 MB
  • After Segwit
block weight = base size × 3 + total size
base size: Size of transaction data not including witness
total size: Size of transaction data including witness
block weight ≦ 4,000,000 MB
  • blocks are non-Segwit transactions, block size is 1MB, same as before
  • all transactions in the block are transactions of P2WPKH with 1 input, 2 output, block size is about 1.6 MB.
  • block has one output and all other transactions are P2WPKH input, it is huge Tx, the block size is about 2.1 MB.
  • block consists of transactions of P2WSH with huge witness (all 15-of-15 multisig etc), the block size is about 3.7 MB.
6) Changes in signature data
The convention message digest items are based on the conventional transaction structure. The message digest items are:
version, txin count, txins, txout count, txouts, locktime, sighash type
For Segwit, the message digest items are:
version
hashPrevouts
Hash of all input outpoint
hashSequence
Hash of all input sequence (TxIns)
outpoint
Previous output (32byte TXID + 4byte index) in TxIns
script code
value
amount of coins held by TxIns
sequence
Sequence of TxIns
hash output
Hash of all outputs (TxOuts)
locktime
sighash type
Segwit changes the calculation of transaction hash for signatures, so that each byte of a transaction is hashed twice, at most [7]. The sighash calculation cost is reduced.
7) Witness commitment in Coinbase transaction
For a conventional transaction, the merkle root calculation is shown as below. The merkle root is calculated using original Tx format. 

Segwit adds the witness commitment. Merkle tree is constructed based on transaction data including signature data of witness. That merkle root is stored in one of coinbase transaction output to make commitment including the witness data.


Effects and Challenges
Segwit changes the consensus, P2P message, address format of Bitcoin protocol. It is amazing Segwit could be realised in soft fork.
Segwit introduces witness extension method. It cancels transaction malleability and increases block size. The actual block size increase depends on the transaction type.
References
  1. https://en.bitcoin.it/wiki/Segregated_Witness
  2. https://en.wikipedia.org/wiki/SegWit
  3. https://en.bitcoin.it/wiki/Transaction_malleability
  4. https://en.bitcoin.it/wiki/Block_size_limit_controversy
  5. https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki
  6. https://en.wikipedia.org/wiki/BCH_code
  7. https://bitcoincore.org/en/2016/01/26/segwit-benefits/#linear-scaling-of-sighash-operations