EMBEDDED: June 2018

Monday, 25 June 2018

101 confirmations

Questions:

After mining, i do listtransactions command, i can see the address has amount of 50 coins in each transaction. Why getbalance command returns 0 ?

<pre>

$ src/bitcoin-cli -datadir=../datadir listtransactions

[

{

"account": "",

"address": "15454L2G44NuZoZrE2HdBwMmCchQVcGvKm",

"category": "immature",

"amount": 50.00000000,

"label": "",

"vout": 0,

"confirmations": 2,

"generated": true,

"blockhash": "00000000df3ae70ae6c5f3eb24b0dca0c37ef55b76fad5396f1386aaab2b0027",

"blockindex": 0,

"blocktime": 1527831787,

"txid": "24db59cbb12a8ffd3f1421931f2a6a2293b1b6437021af88119da95937c8f737",

"walletconflicts": [

"time": 1527831787,

"timereceived": 1527831828,

"bip125-replaceable": "no"

{

"account": "",

"address": "15454L2G44NuZoZrE2HdBwMmCchQVcGvKm",

"category": "immature",

"amount": 50.00000000,

"label": "",

"vout": 0,

"confirmations": 1,

"generated": true,

"blockhash": "00000000eb85f984adc2905671aaa8663d505c0ee71fb5a0d47996f76a12f336",

"blockindex": 0,

"blocktime": 1527831949,

"txid": "d47efd3725fb3be3072131ea6612b2e6581a876f11760e844e91ecd8b414f22e",

"walletconflicts": [

"time": 1527831949,

"timereceived": 1527831973,

"bip125-replaceable": "no"

}

]

$ src/bitcoin-cli -datadir=../datadir getbalance ""

0.00000000

</pre>

Generated coins cannot be spent until the generation transaction has gone through 101 confirmations. Transactions that try to spend generated coins before this will be rejected. The 101 confirmations is the maturity time.

The reason for this is that sometimes the block chain forks, blocks that were valid become invalid, and the mining reward in those blocks is lost. That is an unavoidable part of how Bitcoin works. If there was no maturation time, then whenever a fork happened, everyone who received coins that were generated on an unlucky fork (possibly through many intermediaries) would have their coins disappear, even without any sort of double-spend or other attack. On long forks, people could find coins disappearing from their wallets, even though there is no one actually attacking them and they had no reason to be suspicious of the money they were receiving.

For example, without a maturation time, a miner might deposit 50 BTC into an EWallet, and if the user withdraws money from a completely unrelated account on the same EWallet, the withdrawn money might just disappear if there is a fork and he/she is unlucky enough to withdraw coins that have been "tainted" by the miner's now-invalid coins.

Due to the way this sort of taint tends to "infect" transactions, far more than 50 BTC per block would be affected. Each invalidated block could cause transactions collectively worth hundreds of bitcoins to be reversed. The maturation time makes it impossible for anyone to lose coins by accident like this as long as a fork doesn't last longer than 100 blocks.

If a fork does last longer than 100 blocks, then the damage caused by invalidated transactions would likely be a huge disaster. This is unlike to happen, as something would have to be seriously wrong with Bitcoin or the Internet for a fork to last this long.

Friday, 22 June 2018

Run Bitcoind in Docker container

Use Docker container to run Bitcoin core

0) Install Docker

Docker can be installed on Windows, Linux, MacOS

1) Create new directory for running bitcoin core in Docker

mkdir bitcoin-core-docker

cd bitcoin-core-docker

2) Create new dir to store bitcoin.conf

mkdir node; cd node

3) Create bitcoin.conf and add the contents

server=1

regtest=1

port=12000

rpcport=12001

rpcallowip=0.0.0.0/0

rpcuser=username

rpcpassword=password

daemon=1

txindex=1

4) Build Docker container

cd .. ( to bitcoin-core-docker)

create Dockerfile

5) Add contents to Docker

# Dockerfile must start with a FROM instruction

# FROM instruction specifies the Base Image from which you are building

# FROM <image>[:<tag>]

FROM ubuntu:16.04

ENV BTCVERSION=0.15.1

ENV BTCPREFIX=/bitcoin-prefix

RUN apt-get update && apt-get install -y git build-essential wget pkg-config curl libtool autotools-dev automake libssl-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-program-options-dev libboost-test-dev libboost-thread-dev

WORKDIR /

RUN mkdir -p /berkeleydb

#download berkeley db

RUN git clone -b 0.15 --single-branch https://github.com/bitcoin/bitcoin.git

WORKDIR /berkeleydb

RUN wget http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz && tar -xvf db-4.8.30.NC.tar.gz && rm db-4.8.30.NC.tar.gz && mkdir -p db-4.8.30.NC/build_unix/build

ENV BDB_PREFIX=/berkeleydb/db-4.8.30.NC/build_unix/build

WORKDIR /berkeleydb/db-4.8.30.NC/build_unix

RUN ../dist/configure --disable-shared --enable-cxx --with-pic --prefix=$BDB_PREFIX

RUN make install

RUN apt-get update && apt-get install -y libminiupnpc-dev libzmq3-dev libprotobuf-dev protobuf-compiler libqrencode-dev

WORKDIR /bitcoin

RUN git checkout v${BTCVERSION} && mkdir -p /bitcoin/bitcoin-${BTCVERSION}

WORKDIR /bitcoin

RUN ./autogen.sh

RUN ./configure CPPFLAGS="-I${BDB_PREFIX}/include/ -O2" LDFLAGS="-L${BDB_PREFIX}/lib/ -static-libstdc++" --prefix=${BTCPREFIX}

RUN make

RUN make install DESTDIR=/bitcoin/bitcoin-${BTCVERSION}

RUN mv /bitcoin/bitcoin-${BTCVERSION}${BTCPREFIX} /bitcoin-${BTCVERSION} && strip /bitcoin-${BTCVERSION}/bin/* && rm -rf /bitcoin-${BTCVERSION}/lib/pkgconfig && find /bitcoin-${BTCVERSION} -name "lib*.la" -delete && find /bitcoin-${BTCVERSION} -name "lib*.a" -delete

WORKDIR /

RUN tar cvf bitcoin-${BTCVERSION}.tar bitcoin-${BTCVERSION}

# copy bitcoin,conf

ADD . /bitcoin-${BTCVERSION}

# expose rpc port for the node to allow access from outside container

EXPOSE 12001

WORKDIR /bitcoin-${BTCVERSION}

6)build the docker image

docker build -t bitcoin-docker .

The -t flag sets a name for the image.
The . tells Docker to looker for Dockerfile in the current directory

6.1)list the built images:

docker images

7) run the image in container

docker run -it -p 5000:12001 bitcoin-docker

-it is required for interactive processes (like a bash shell)
-p maps Ubuntu port 5000 to the container’s exposed port 12001, which is where Bitcoin rpc will be listening

if everything works, docker present a bash shell. Both the node directory and the bitcoin.conf file were copied to the container by the ADD instruction in the Dockerfile, so they will be present in the current working directory.

In the bash shell, run

7.1) bitcoind -datadir=node -daemon

7.2) bitcoin-cli -datadir=node getinfo

8) Connect to bitcoind from outside Docker (open second terminal window)

curl --user username:password --data '{"method": "getinfo"}' http://127.0.0.1:5000

9) In first terminal window, run this to stop bitcoind

bitcoin-cli -datadir=node stop

9.1) In second terminal window, exit the docker container

exit

Use Docker to run Bitcoin core as service

TBC…

PS: we can push the image to docker hub and share with others

# tag the image

docker tag bitcoin <username>/bitcoin:custom

# push to docker hub

docker push <username>/bitcoin:custom

# run the image, if image is not available locally, docker pull from repo

docker run -it -p 5000:12001 <username>/bitcoin:custom

Wednesday, 6 June 2018

Segregated Witness (Segwit)

Segwit

Introduction

Segregated Witness (Segwit) [1], proposed in BIP 141 [5], was activated on August 24, 2017. The contributions of Segwit [2]:

1) solve transaction malleability [3]

2) mitigate block size limitation problem

Problem

Transaction malleability:

When transaction is signed, the signature (script_sig) does not cover all the data in a transaction. Specifically, the script_sig is part of the transaction, the signature will not be able to sign script_sig. So the signature does not cover script_sig. The script_sig is added, after the transaction is created and signed.

The script_sig is the tempering point. If script_sig changes, TXID will change. The script_sig can be changed by anyone has access to the corresponding private keys.

2) Block size limitation problem

Originally, Bitcoin does not have limit on block size. This allowed attackers to create large size block data. So a 1MB block size was introduced. The 1MB was a tradeoff, between network propagation times, node capability, and number of transactions that can fit into one block, etc [4].

Proposal

Segwit defines a new structure called witness. Signature and redeem script are moved into this structure, which is not included in the 1MB block size limit.

1)Transaction structure

The conventional transaction structure is used in TXID calculation, and script_sig is empty. Even if script_sig is tempered with, TXID does not change.

2) Lock/Unlock script

For a conventional P2PKH:

scriptPubKey (lock script)

OP_DUP OP_HASH160 <pubkey hash> OP_EQUALVERIFY OP_CHECKSIG

scriptSig (unlock script)

For Segwit P2WPKH:

scriptPubKey (lock script)

0 <pubkey hash>

(unlock script)

scriptSig	Witness
Empty	<sig> <pubkey>

In scriptPubKey, there are no opcodes, only 2 data (version and hash) is pushed. When the lock script of this pattern is set, it is evaluated as a conventional P2PKH script. The signature and public key are obtained from witness instead of scriptSig.

3) Witness extension method

In the extension, Segwit introduces OP_CLTV (OP_NPO2) and OP_CSV (OP_NOP3)

The witness structure

For Segwit, witness version is 0, the witness program is P2WPKH if hash length is 20 bytes and P2WSH if it is 32 bytes.

4) Address format

Segwit uses Bech32 address format. It is based on BCH code instead of previously used Base58 encoding, so that error correction is possible [6]. There is no distinction between uppercase and lowercase letters. QR code is also compact

5) Increase of block size

The increase of block size from Segwit depends on the types of transaction.

Before Segwit

block data ≦ 1,000,000 MB

After Segwit

block weight = base size × ３ + total size
base size： Size of transaction data not including witness

total size： Size of transaction data including witness

block weight ≦ 4,000,000 MB

blocks are non-Segwit transactions, block size is 1MB, same as before

all transactions in the block are transactions of P2WPKH with 1 input, 2 output, block size is about 1.6 MB.
block has one output and all other transactions are P2WPKH input, it is huge Tx, the block size is about 2.1 MB.
block consists of transactions of P2WSH with huge witness (all 15-of-15 multisig etc), the block size is about 3.7 MB.

6) Changes in signature data

The convention message digest items are based on the conventional transaction structure. The message digest items are:

version, txin count, txins, txout count, txouts, locktime, sighash type

For Segwit, the message digest items are:

version
hashPrevouts	Hash of all input outpoint
hashSequence	Hash of all input sequence (TxIns)
outpoint	Previous output (32byte TXID + 4byte index) in TxIns
script code
value	amount of coins held by TxIns
sequence	Sequence of TxIns
hash output	Hash of all outputs (TxOuts)
locktime
sighash type

Segwit changes the calculation of transaction hash for signatures, so that each byte of a transaction is hashed twice, at most [7]. The sighash calculation cost is reduced.

7) Witness commitment in Coinbase transaction

For a conventional transaction, the merkle root calculation is shown as below. The merkle root is calculated using original Tx format.

Segwit adds the witness commitment. Merkle tree is constructed based on transaction data including signature data of witness. That merkle root is stored in one of coinbase transaction output to make commitment including the witness data.

Effects and Challenges

Segwit changes the consensus, P2P message, address format of Bitcoin protocol. It is amazing Segwit could be realised in soft fork.

Segwit introduces witness extension method. It cancels transaction malleability and increases block size. The actual block size increase depends on the transaction type.

References